2024-03-24T00:21:47.440 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-24T00:21:47.440 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --owner mchangir --archive /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441 -- /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441/orig.config.yaml 2024-03-24T00:21:47.494 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-24T00:21:47.585 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441 branch: wip-mchangir-testing-main-20240318.032620 description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: mchangir@redhat.com first_in_suite: false job_id: '7616441' kernel: flavor: default kdb: true sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/reef ' name: mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-main-20240318.032620 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-main-20240318.032620 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 owner: mchangir priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 3550 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 sleep_before_teardown: 0 subset: 613/1024 suite: fs suite_branch: wip-mchangir-testing-main-20240318.032620 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 targets: smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKv0iFJvmiR8q1udVuYrwMqOSoSQVX/zb3R8/t+aBZmvy1eOCZvaK7DJSjp9kChJbNz3DqATqEdH22swFM9tUsU= smithi088.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsoD5ZKKVF75fb76X9tSs6uWMDeeNyemcJAdz71ILPjqQ3dkkblKXfdVUpJ2zT9jMCEkIqi8n9Xi4e9soCwnpo= tasks: - install: branch: reef exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:reef roleless: true - print: '**** done end installing reef cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_09:49:57 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-24T00:21:47.585 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa; will attempt to use it 2024-03-24T00:21:47.586 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks 2024-03-24T00:21:47.586 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-24T00:21:47.587 INFO:teuthology.task.internal:Checking packages... 2024-03-24T00:21:47.612 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3be9a5c9bc793e11e2800a8c0c696e8b46742033' 2024-03-24T00:21:47.612 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T00:21:47.612 INFO:teuthology.packaging:ref: None 2024-03-24T00:21:47.612 INFO:teuthology.packaging:tag: None 2024-03-24T00:21:47.612 INFO:teuthology.packaging:branch: wip-mchangir-testing-main-20240318.032620 2024-03-24T00:21:47.612 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:21:47.613 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-main-20240318.032620 2024-03-24T00:21:47.833 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2145.g3be9a5c9 2024-03-24T00:21:47.834 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-24T00:21:47.843 INFO:teuthology.task.internal:no buildpackages task found 2024-03-24T00:21:47.843 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-24T00:21:47.864 INFO:teuthology.task.internal:Saving configuration 2024-03-24T00:21:47.883 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-24T00:21:47.890 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T00:21:47.916 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 00:16:24.042579', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKv0iFJvmiR8q1udVuYrwMqOSoSQVX/zb3R8/t+aBZmvy1eOCZvaK7DJSjp9kChJbNz3DqATqEdH22swFM9tUsU='} 2024-03-24T00:21:47.937 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 00:16:24.044596', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsoD5ZKKVF75fb76X9tSs6uWMDeeNyemcJAdz71ILPjqQ3dkkblKXfdVUpJ2zT9jMCEkIqi8n9Xi4e9soCwnpo='} 2024-03-24T00:21:47.937 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-24T00:21:47.944 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-24T00:21:47.944 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-24T00:21:47.944 INFO:teuthology.run_tasks:Running task console_log... 2024-03-24T00:21:48.014 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f55789160d0>, signals=[15]) 2024-03-24T00:21:48.015 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-24T00:21:48.023 INFO:teuthology.task.internal:Opening connections... 2024-03-24T00:21:48.023 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:21:48.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T00:21:48.143 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2024-03-24T00:21:48.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T00:21:48.240 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-24T00:21:48.247 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-03-24T00:21:48.267 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-03-24T00:21:48.267 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-03-24T00:21:48.325 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-03-24T00:21:48.326 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-03-24T00:21:48.326 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-03-24T00:21:48.326 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-03-24T00:21:48.326 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-03-24T00:21:48.326 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-03-24T00:21:48.327 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-24T00:21:48.327 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-03-24T00:21:48.327 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-03-24T00:21:48.327 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-24T00:21:48.327 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-03-24T00:21:48.327 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-24T00:21:48.327 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-24T00:21:48.328 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-24T00:21:48.329 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-03-24T00:21:48.355 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2024-03-24T00:21:48.373 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2024-03-24T00:21:48.373 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-03-24T00:21:48.433 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2024-03-24T00:21:48.433 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9" 2024-03-24T00:21:48.433 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2024-03-24T00:21:48.433 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2024-03-24T00:21:48.434 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9" 2024-03-24T00:21:48.434 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9" 2024-03-24T00:21:48.434 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-24T00:21:48.434 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2024-03-24T00:21:48.434 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon" 2024-03-24T00:21:48.434 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-24T00:21:48.434 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2024-03-24T00:21:48.434 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-24T00:21:48.434 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-24T00:21:48.434 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-24T00:21:48.434 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-03-24T00:21:48.462 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-24T00:21:48.476 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-24T00:21:48.491 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-24T00:21:48.491 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-03-24T00:21:48.494 DEBUG:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2024-03-24T00:21:48.509 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-24T00:21:48.519 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-24T00:21:48.519 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-03-24T00:21:48.552 DEBUG:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2024-03-24T00:21:48.585 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-24T00:21:48.640 INFO:teuthology.run_tasks:Running task kernel... 2024-03-24T00:21:48.661 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'} 2024-03-24T00:21:48.661 INFO:teuthology.task.kernel:config {'host.a': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}, 'host.b': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}}, timeout 300 2024-03-24T00:21:48.661 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-24T00:21:48.662 DEBUG:teuthology.orchestra.run.smithi088:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-24T00:21:48.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T00:21:48.678 DEBUG:teuthology.task.kernel:sha1 for host.b is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-24T00:21:48.678 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-24T00:21:48.678 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-03-24T00:21:48.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T00:21:48.679 DEBUG:teuthology.task.kernel:sha1 for host.a is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-24T00:21:48.679 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-24T00:21:48.679 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-03-24T00:21:48.738 INFO:teuthology.orchestra.run.smithi088.stdout:5.14.0-361.el9.x86_64 2024-03-24T00:21:48.738 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-361.el9.x86_64 2024-03-24T00:21:48.738 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-24T00:21:48.738 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-24T00:21:48.739 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-24T00:21:48.740 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-24T00:21:48.740 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-24T00:21:48.740 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-24T00:21:48.890 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum upgrade -y linux-firmware 2024-03-24T00:21:48.961 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum upgrade -y linux-firmware 2024-03-24T00:21:50.898 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-24T00:21:51.055 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-24T00:21:52.290 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-9 - BaseOS 7.7 MB/s | 2.0 MB 00:00 2024-03-24T00:21:54.205 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 5.5 MB/s | 8.0 MB 00:01 2024-03-24T00:21:58.772 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 5.3 MB/s | 19 MB 00:03 2024-03-24T00:22:01.263 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS-9 - BaseOS 227 kB/s | 2.0 MB 00:09 2024-03-24T00:22:02.297 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-24T00:22:02.873 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.1 MB 00:00 2024-03-24T00:22:04.633 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-03-24T00:22:07.816 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 6.7 MB/s | 21 MB 00:03 2024-03-24T00:22:12.707 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-24T00:22:14.888 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:22:14.898 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:22:14.898 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-03-24T00:22:14.899 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:22:14.899 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrading: 2024-03-24T00:22:14.899 INFO:teuthology.orchestra.run.smithi088.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-24T00:22:14.899 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:22:14.899 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:22:14.899 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:22:14.900 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrade 1 Package 2024-03-24T00:22:14.900 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:22:14.900 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 375 M 2024-03-24T00:22:14.900 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:22:19.736 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-03-24T00:22:21.782 INFO:teuthology.orchestra.run.smithi088.stdout:linux-firmware-20240219-143.el9.noarch.rpm 54 MB/s | 375 MB 00:06 2024-03-24T00:22:21.783 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:22:21.783 INFO:teuthology.orchestra.run.smithi088.stdout:Total 54 MB/s | 375 MB 00:06 2024-03-24T00:22:21.784 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:22:21.878 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:22:21.878 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:22:23.821 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-03-24T00:22:24.258 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:22:24.258 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:22:25.655 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-03-24T00:22:27.008 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-24T00:22:30.613 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:22:30.856 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-24T00:22:30.948 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-24T00:22:32.073 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-24T00:22:32.312 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-24T00:22:32.312 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-24T00:22:32.709 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-24T00:22:32.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:22:32.710 INFO:teuthology.orchestra.run.smithi088.stdout:Upgraded: 2024-03-24T00:22:32.710 INFO:teuthology.orchestra.run.smithi088.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-24T00:22:32.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:22:32.710 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:22:33.011 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.b... 2024-03-24T00:22:33.011 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-24T00:22:33.154 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-24T00:22:33.155 DEBUG:teuthology.orchestra.run.smithi088:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-24T00:22:33.157 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi088.front.sepia.ceph.com to complete... 2024-03-24T00:22:34.322 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:22:34.335 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:22:34.335 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-03-24T00:22:34.335 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:22:34.335 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-03-24T00:22:34.335 INFO:teuthology.orchestra.run.smithi038.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-24T00:22:34.336 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:22:34.336 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:22:34.336 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:22:34.336 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 1 Package 2024-03-24T00:22:34.336 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:22:34.336 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 375 M 2024-03-24T00:22:34.336 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:22:41.132 INFO:teuthology.orchestra.run.smithi038.stdout:linux-firmware-20240219-143.el9.noarch.rpm 55 MB/s | 375 MB 00:06 2024-03-24T00:22:41.133 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:22:41.134 INFO:teuthology.orchestra.run.smithi038.stdout:Total 55 MB/s | 375 MB 00:06 2024-03-24T00:22:41.134 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:22:41.239 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:22:41.239 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:22:43.711 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:22:43.711 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:22:44.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24 00:22:44 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-24T00:22:44.567 INFO:teuthology.orchestra.run.smithi088.stderr:FINISHED --2024-03-24 00:22:44-- 2024-03-24T00:22:44.567 INFO:teuthology.orchestra.run.smithi088.stderr:Total wall clock time: 11s 2024-03-24T00:22:44.567 INFO:teuthology.orchestra.run.smithi088.stderr:Downloaded: 1 files, 321M in 11s (28.9 MB/s) 2024-03-24T00:22:44.569 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.b... 2024-03-24T00:22:44.570 DEBUG:teuthology.orchestra.run.smithi088:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-24T00:22:46.502 INFO:teuthology.orchestra.run.smithi088.stdout:Verifying... ######################################## 2024-03-24T00:22:46.551 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing... ######################################## 2024-03-24T00:22:46.631 INFO:teuthology.orchestra.run.smithi088.stdout:Updating / installing... 2024-03-24T00:22:50.166 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:22:50.442 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-24T00:22:50.536 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-24T00:22:51.885 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-24T00:22:51.885 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-24T00:22:52.311 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-24T00:22:52.311 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:22:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-03-24T00:22:52.311 INFO:teuthology.orchestra.run.smithi038.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-24T00:22:52.312 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:22:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:22:52.597 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.a... 2024-03-24T00:22:52.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-24T00:22:52.816 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-24T00:22:52.816 DEBUG:teuthology.orchestra.run.smithi038:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-24T00:22:52.819 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi038.front.sepia.ceph.com to complete... 2024-03-24T00:22:54.101 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-24T00:23:04.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24 00:23:04 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-24T00:23:04.438 INFO:teuthology.orchestra.run.smithi038.stderr:FINISHED --2024-03-24 00:23:04-- 2024-03-24T00:23:04.438 INFO:teuthology.orchestra.run.smithi038.stderr:Total wall clock time: 12s 2024-03-24T00:23:04.438 INFO:teuthology.orchestra.run.smithi038.stderr:Downloaded: 1 files, 321M in 11s (28.5 MB/s) 2024-03-24T00:23:04.441 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.a... 2024-03-24T00:23:04.441 DEBUG:teuthology.orchestra.run.smithi038:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-24T00:23:06.465 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying... ######################################## 2024-03-24T00:23:06.528 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing... ######################################## 2024-03-24T00:23:06.614 INFO:teuthology.orchestra.run.smithi038.stdout:Updating / installing... 2024-03-24T00:23:10.457 INFO:teuthology.orchestra.run.smithi088.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-24T00:23:14.438 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-24T00:23:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-24T00:23:32.933 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi088.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-24T00:23:32.933 DEBUG:teuthology.orchestra.run.smithi088:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-24T00:23:33.003 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-24T00:23:33.004 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-24T00:23:33.005 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-24T00:23:33.006 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-24T00:23:33.007 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-24T00:23:33.008 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-24T00:23:33.009 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-24T00:23:33.010 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-24T00:23:33.011 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-24T00:23:33.012 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-24T00:23:33.013 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-24T00:23:33.014 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-24T00:23:33.014 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-24T00:23:33.014 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-24T00:23:33.014 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-24T00:23:33.014 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-24T00:23:33.014 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-24T00:23:33.014 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-24T00:23:33.014 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-24T00:23:33.014 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-24T00:23:33.014 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-24T00:23:33.014 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-24T00:23:33.015 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-24T00:23:33.016 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-24T00:23:33.017 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-24T00:23:33.018 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-24T00:23:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-24T00:23:33.020 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-24T00:23:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-24T00:23:33.022 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-24T00:23:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-24T00:23:33.024 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-24T00:23:33.025 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-24T00:23:33.026 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-24T00:23:33.027 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-24T00:23:33.028 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-24T00:23:33.029 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-24T00:23:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-24T00:23:33.031 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-24T00:23:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-24T00:23:33.033 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-24T00:23:33.034 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-24T00:23:33.035 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-24T00:23:33.036 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-24T00:23:33.037 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-24T00:23:33.038 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-24T00:23:33.039 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-24T00:23:33.040 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-24T00:23:33.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-24T00:23:33.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-24T00:23:33.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-24T00:23:33.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-24T00:23:33.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-24T00:23:33.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-24T00:23:33.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-24T00:23:33.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-24T00:23:33.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-24T00:23:33.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-24T00:23:33.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-24T00:23:33.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-24T00:23:33.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-24T00:23:33.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-24T00:23:33.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-24T00:23:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-24T00:23:33.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-24T00:23:33.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-24T00:23:33.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-24T00:23:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-24T00:23:33.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-24T00:23:33.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-24T00:23:33.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-24T00:23:33.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-24T00:23:33.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-24T00:23:33.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-24T00:23:33.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-24T00:23:33.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-24T00:23:33.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-24T00:23:33.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-24T00:23:33.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-24T00:23:33.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-24T00:23:33.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-24T00:23:33.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-24T00:23:33.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-24T00:23:33.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-24T00:23:33.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-24T00:23:33.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-24T00:23:33.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-24T00:23:33.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-24T00:23:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-24T00:23:33.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-24T00:23:33.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-24T00:23:33.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-24T00:23:33.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-24T00:23:33.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-24T00:23:33.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-24T00:23:33.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-24T00:23:33.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-24T00:23:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-24T00:23:33.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-24T00:23:33.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-24T00:23:33.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-24T00:23:33.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-24T00:23:33.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-24T00:23:33.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-24T00:23:33.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-24T00:23:33.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-24T00:23:33.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-24T00:23:33.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-24T00:23:33.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-24T00:23:33.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-24T00:23:33.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-24T00:23:33.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-24T00:23:33.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-24T00:23:33.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-24T00:23:33.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-24T00:23:33.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-24T00:23:33.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-24T00:23:33.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-24T00:23:33.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-24T00:23:33.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-24T00:23:33.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-24T00:23:33.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-24T00:23:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-24T00:23:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-24T00:23:33.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-24T00:23:33.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-24T00:23:33.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-24T00:23:33.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-24T00:23:33.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-24T00:23:33.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-24T00:23:33.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-24T00:23:33.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-24T00:23:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-24T00:23:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-24T00:23:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-24T00:23:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-24T00:23:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-24T00:23:33.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-24T00:23:33.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-24T00:23:33.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-24T00:23:33.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-24T00:23:33.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-24T00:23:33.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-24T00:23:33.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-24T00:23:33.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-24T00:23:33.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-24T00:23:33.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-24T00:23:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-24T00:23:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-24T00:23:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-24T00:23:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-24T00:23:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-24T00:23:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-24T00:23:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-24T00:23:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-24T00:23:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-24T00:23:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-24T00:23:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-24T00:23:33.144 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:33.144 DEBUG:teuthology.orchestra.run.smithi088:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-24T00:23:33.167 INFO:teuthology.orchestra.run.smithi088.stdout:postinstall scriptlet (using /bin/sh): 2024-03-24T00:23:33.167 INFO:teuthology.orchestra.run.smithi088.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-24T00:23:33.167 INFO:teuthology.orchestra.run.smithi088.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-24T00:23:33.167 INFO:teuthology.orchestra.run.smithi088.stdout:fi 2024-03-24T00:23:33.167 INFO:teuthology.orchestra.run.smithi088.stdout:for file in vmlinuz System.map config; do 2024-03-24T00:23:33.167 INFO:teuthology.orchestra.run.smithi088.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-24T00:23:33.167 INFO:teuthology.orchestra.run.smithi088.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-24T00:23:33.167 INFO:teuthology.orchestra.run.smithi088.stdout: fi 2024-03-24T00:23:33.167 INFO:teuthology.orchestra.run.smithi088.stdout:done 2024-03-24T00:23:33.167 INFO:teuthology.orchestra.run.smithi088.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-24T00:23:33.167 INFO:teuthology.orchestra.run.smithi088.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-24T00:23:33.168 INFO:teuthology.orchestra.run.smithi088.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-24T00:23:33.168 INFO:teuthology.orchestra.run.smithi088.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-24T00:23:33.168 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:33.168 INFO:teuthology.orchestra.run.smithi088.stdout:fi 2024-03-24T00:23:33.168 INFO:teuthology.orchestra.run.smithi088.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-24T00:23:33.168 INFO:teuthology.orchestra.run.smithi088.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-24T00:23:33.168 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:33.168 INFO:teuthology.orchestra.run.smithi088.stdout:fi 2024-03-24T00:23:33.169 DEBUG:teuthology.orchestra.run.smithi088:> sudo rpm -qi grub2-tools 2024-03-24T00:23:33.243 INFO:teuthology.orchestra.run.smithi088.stdout:Name : grub2-tools 2024-03-24T00:23:33.243 INFO:teuthology.orchestra.run.smithi088.stdout:Epoch : 1 2024-03-24T00:23:33.243 INFO:teuthology.orchestra.run.smithi088.stdout:Version : 2.06 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:Release : 68.el9 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:Architecture: x86_64 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:Group : Unspecified 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:Size : 8264254 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:License : GPLv3+ 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:Packager : builder@centos.org 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:Vendor : CentOS 2024-03-24T00:23:33.244 INFO:teuthology.orchestra.run.smithi088.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-24T00:23:33.245 INFO:teuthology.orchestra.run.smithi088.stdout:Summary : Support tools for GRUB. 2024-03-24T00:23:33.245 INFO:teuthology.orchestra.run.smithi088.stdout:Description : 2024-03-24T00:23:33.245 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:23:33.245 INFO:teuthology.orchestra.run.smithi088.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-24T00:23:33.245 INFO:teuthology.orchestra.run.smithi088.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-24T00:23:33.245 INFO:teuthology.orchestra.run.smithi088.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-24T00:23:33.245 INFO:teuthology.orchestra.run.smithi088.stdout:hardware devices. 2024-03-24T00:23:33.245 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:23:33.245 INFO:teuthology.orchestra.run.smithi088.stdout:This subpackage provides tools for support of all platforms. 2024-03-24T00:23:33.246 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-24T00:23:33.246 INFO:teuthology.task.kernel:Updating grub on smithi088 to boot 6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:33.247 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-24T00:23:33.932 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2024-03-24T00:23:35.278 INFO:teuthology.orchestra.run.smithi088.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-24T00:23:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:done 2024-03-24T00:23:35.327 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-03-24T00:23:35.345 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.AgLu2sPRMX 2024-03-24T00:23:35.346 DEBUG:teuthology.orchestra.run.smithi088:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AgLu2sPRMX 2024-03-24T00:23:35.426 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.AgLu2sPRMX 2024-03-24T00:23:35.572 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.AgLu2sPRMX is 6KB 2024-03-24T00:23:35.621 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.AgLu2sPRMX 2024-03-24T00:23:35.635 DEBUG:teuthology.orchestra.run.smithi088:> sudo /bin/ls /boot/loader/entries || true 2024-03-24T00:23:35.698 INFO:teuthology.orchestra.run.smithi088.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-24T00:23:35.699 INFO:teuthology.orchestra.run.smithi088.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-24T00:23:35.699 INFO:teuthology.orchestra.run.smithi088.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-24T00:23:35.699 INFO:teuthology.orchestra.run.smithi088.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-24T00:23:35.700 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:35.851 DEBUG:teuthology.orchestra.run.smithi088:> sudo shutdown -r now 2024-03-24T00:23:53.977 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi038.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-24T00:23:53.978 DEBUG:teuthology.orchestra.run.smithi038:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-24T00:23:54.053 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:54.053 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-24T00:23:54.053 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-24T00:23:54.053 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-24T00:23:54.053 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-24T00:23:54.053 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-24T00:23:54.053 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-24T00:23:54.053 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-24T00:23:54.053 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-24T00:23:54.053 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-24T00:23:54.054 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-24T00:23:54.055 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-24T00:23:54.056 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-24T00:23:54.057 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-24T00:23:54.058 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-24T00:23:54.059 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-24T00:23:54.060 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-24T00:23:54.061 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-24T00:23:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-24T00:23:54.063 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-24T00:23:54.064 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-24T00:23:54.065 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-24T00:23:54.066 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-24T00:23:54.067 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-24T00:23:54.068 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-24T00:23:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-24T00:23:54.070 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-24T00:23:54.071 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-24T00:23:54.072 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-24T00:23:54.073 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-24T00:23:54.074 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-24T00:23:54.075 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-24T00:23:54.076 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-24T00:23:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-24T00:23:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-24T00:23:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-24T00:23:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-24T00:23:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-24T00:23:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-24T00:23:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-24T00:23:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-24T00:23:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-24T00:23:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-24T00:23:54.078 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-24T00:23:54.079 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-24T00:23:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-24T00:23:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-24T00:23:54.082 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-24T00:23:54.083 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-24T00:23:54.084 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-24T00:23:54.085 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-24T00:23:54.086 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-24T00:23:54.087 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-24T00:23:54.088 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-24T00:23:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-24T00:23:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-24T00:23:54.091 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-24T00:23:54.092 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-24T00:23:54.093 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-24T00:23:54.094 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-24T00:23:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-24T00:23:54.096 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-24T00:23:54.097 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-24T00:23:54.098 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-24T00:23:54.099 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-24T00:23:54.100 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-24T00:23:54.101 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-24T00:23:54.102 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-24T00:23:54.103 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-24T00:23:54.104 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-24T00:23:54.105 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-24T00:23:54.105 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-24T00:23:54.105 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-24T00:23:54.105 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-24T00:23:54.105 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-24T00:23:54.105 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-24T00:23:54.105 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-24T00:23:54.105 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-24T00:23:54.106 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-24T00:23:54.107 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-24T00:23:54.108 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-24T00:23:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-24T00:23:54.110 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-24T00:23:54.111 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-24T00:23:54.112 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-24T00:23:54.113 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-24T00:23:54.114 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-24T00:23:54.115 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-24T00:23:54.116 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-24T00:23:54.117 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-24T00:23:54.118 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-24T00:23:54.119 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-24T00:23:54.120 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-24T00:23:54.121 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-24T00:23:54.122 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-24T00:23:54.123 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-24T00:23:54.124 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-24T00:23:54.125 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-24T00:23:54.126 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-24T00:23:54.127 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-24T00:23:54.127 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-24T00:23:54.127 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-24T00:23:54.127 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-24T00:23:54.127 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-24T00:23:54.127 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-24T00:23:54.127 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-24T00:23:54.127 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-24T00:23:54.127 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-24T00:23:54.127 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-24T00:23:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-24T00:23:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-24T00:23:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-24T00:23:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-24T00:23:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-24T00:23:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-24T00:23:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-24T00:23:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-24T00:23:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-24T00:23:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-24T00:23:54.128 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-24T00:23:54.129 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-24T00:23:54.130 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-24T00:23:54.131 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-24T00:23:54.132 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-24T00:23:54.133 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-24T00:23:54.134 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-24T00:23:54.135 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-24T00:23:54.136 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-24T00:23:54.137 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-24T00:23:54.138 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-24T00:23:54.139 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-24T00:23:54.140 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-24T00:23:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-24T00:23:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-24T00:23:54.143 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-24T00:23:54.144 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-24T00:23:54.145 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-24T00:23:54.146 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-24T00:23:54.147 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-24T00:23:54.148 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-24T00:23:54.149 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-24T00:23:54.150 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-24T00:23:54.151 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-24T00:23:54.152 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-24T00:23:54.153 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-24T00:23:54.154 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-24T00:23:54.155 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-24T00:23:54.156 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-24T00:23:54.157 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-24T00:23:54.158 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-24T00:23:54.159 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-24T00:23:54.160 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-24T00:23:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-24T00:23:54.162 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-24T00:23:54.163 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-24T00:23:54.164 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-24T00:23:54.165 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-24T00:23:54.166 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-24T00:23:54.167 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-24T00:23:54.168 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-24T00:23:54.169 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-24T00:23:54.170 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-24T00:23:54.171 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-24T00:23:54.172 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-24T00:23:54.173 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-24T00:23:54.174 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-24T00:23:54.175 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-24T00:23:54.176 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-24T00:23:54.177 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-24T00:23:54.178 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-24T00:23:54.179 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-24T00:23:54.180 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-24T00:23:54.181 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-24T00:23:54.182 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-24T00:23:54.183 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-24T00:23:54.184 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-24T00:23:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-24T00:23:54.186 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-24T00:23:54.187 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-24T00:23:54.187 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-24T00:23:54.187 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-24T00:23:54.187 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-24T00:23:54.187 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-24T00:23:54.187 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-24T00:23:54.187 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-24T00:23:54.187 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-24T00:23:54.187 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-24T00:23:54.187 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-24T00:23:54.187 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-24T00:23:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-24T00:23:54.189 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-24T00:23:54.190 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-24T00:23:54.190 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-24T00:23:54.190 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-24T00:23:54.190 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-24T00:23:54.190 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-24T00:23:54.190 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-24T00:23:54.190 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-24T00:23:54.190 INFO:teuthology.orchestra.run.smithi038.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-24T00:23:54.193 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:54.194 DEBUG:teuthology.orchestra.run.smithi038:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout:postinstall scriptlet (using /bin/sh): 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout:fi 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout:for file in vmlinuz System.map config; do 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout: fi 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout:done 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-24T00:23:54.218 INFO:teuthology.orchestra.run.smithi038.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-24T00:23:54.219 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:54.219 INFO:teuthology.orchestra.run.smithi038.stdout:fi 2024-03-24T00:23:54.219 INFO:teuthology.orchestra.run.smithi038.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-24T00:23:54.219 INFO:teuthology.orchestra.run.smithi038.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-24T00:23:54.219 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:54.219 INFO:teuthology.orchestra.run.smithi038.stdout:fi 2024-03-24T00:23:54.220 DEBUG:teuthology.orchestra.run.smithi038:> sudo rpm -qi grub2-tools 2024-03-24T00:23:54.301 INFO:teuthology.orchestra.run.smithi038.stdout:Name : grub2-tools 2024-03-24T00:23:54.301 INFO:teuthology.orchestra.run.smithi038.stdout:Epoch : 1 2024-03-24T00:23:54.301 INFO:teuthology.orchestra.run.smithi038.stdout:Version : 2.06 2024-03-24T00:23:54.301 INFO:teuthology.orchestra.run.smithi038.stdout:Release : 68.el9 2024-03-24T00:23:54.301 INFO:teuthology.orchestra.run.smithi038.stdout:Architecture: x86_64 2024-03-24T00:23:54.301 INFO:teuthology.orchestra.run.smithi038.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:Group : Unspecified 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:Size : 8264254 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:License : GPLv3+ 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:Packager : builder@centos.org 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:Vendor : CentOS 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:Summary : Support tools for GRUB. 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:Description : 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:23:54.302 INFO:teuthology.orchestra.run.smithi038.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-24T00:23:54.303 INFO:teuthology.orchestra.run.smithi038.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-24T00:23:54.303 INFO:teuthology.orchestra.run.smithi038.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-24T00:23:54.303 INFO:teuthology.orchestra.run.smithi038.stdout:hardware devices. 2024-03-24T00:23:54.303 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:23:54.303 INFO:teuthology.orchestra.run.smithi038.stdout:This subpackage provides tools for support of all platforms. 2024-03-24T00:23:54.304 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-24T00:23:54.304 INFO:teuthology.task.kernel:Updating grub on smithi038 to boot 6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:54.304 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-24T00:23:54.969 INFO:teuthology.orchestra.run.smithi038.stderr:Generating grub configuration file ... 2024-03-24T00:23:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-24T00:23:56.396 INFO:teuthology.orchestra.run.smithi038.stderr:done 2024-03-24T00:23:56.409 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2024-03-24T00:23:56.424 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.lWRlRaSzpV 2024-03-24T00:23:56.424 DEBUG:teuthology.orchestra.run.smithi038:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lWRlRaSzpV 2024-03-24T00:23:56.495 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.lWRlRaSzpV 2024-03-24T00:23:56.634 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.lWRlRaSzpV is 6KB 2024-03-24T00:23:56.682 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.lWRlRaSzpV 2024-03-24T00:23:56.696 DEBUG:teuthology.orchestra.run.smithi038:> sudo /bin/ls /boot/loader/entries || true 2024-03-24T00:23:56.765 INFO:teuthology.orchestra.run.smithi038.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-24T00:23:56.765 INFO:teuthology.orchestra.run.smithi038.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-24T00:23:56.765 INFO:teuthology.orchestra.run.smithi038.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-24T00:23:56.765 INFO:teuthology.orchestra.run.smithi038.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-24T00:23:56.766 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-24T00:23:56.948 DEBUG:teuthology.orchestra.run.smithi038:> sudo shutdown -r now 2024-03-24T00:24:05.862 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-24T00:24:05.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-03-24T00:24:05.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T00:24:26.970 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-24T00:24:26.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-03-24T00:24:26.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T00:25:05.906 DEBUG:teuthology.orchestra.remote:timed out 2024-03-24T00:25:14.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-03-24T00:25:14.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T00:25:26.977 DEBUG:teuthology.orchestra.remote:timed out 2024-03-24T00:25:33.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-03-24T00:25:35.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-03-24T00:25:35.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T00:25:39.454 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-03-24T00:25:40.187 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-03-24T00:25:40.187 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "6.8.0-g5ec3e6e88b2e"... 2024-03-24T00:25:40.187 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-03-24T00:25:40.227 INFO:teuthology.orchestra.run.smithi038.stdout:6.8.0-g5ec3e6e88b2e 2024-03-24T00:25:40.227 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-24T00:25:40.228 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-24T00:25:40.228 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-24T00:25:41.229 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-24T00:25:41.229 DEBUG:teuthology.orchestra.run.smithi038:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-24T00:25:41.370 INFO:teuthology.orchestra.run.smithi038.stdout:ttyS1 2024-03-24T00:25:41.425 DEBUG:teuthology.parallel:result is None 2024-03-24T00:25:45.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-03-24T00:25:45.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T00:25:45.926 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-03-24T00:25:46.518 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-03-24T00:25:46.518 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "6.8.0-g5ec3e6e88b2e"... 2024-03-24T00:25:46.518 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-03-24T00:25:46.555 INFO:teuthology.orchestra.run.smithi088.stdout:6.8.0-g5ec3e6e88b2e 2024-03-24T00:25:46.555 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-24T00:25:46.556 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-24T00:25:46.556 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-24T00:25:47.556 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-24T00:25:47.556 DEBUG:teuthology.orchestra.run.smithi088:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-24T00:25:47.658 INFO:teuthology.orchestra.run.smithi088.stdout:ttyS1 2024-03-24T00:25:47.716 DEBUG:teuthology.parallel:result is None 2024-03-24T00:25:47.716 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-24T00:25:47.723 INFO:teuthology.task.internal:Creating test directory... 2024-03-24T00:25:47.723 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-24T00:25:47.729 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-24T00:25:47.778 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-24T00:25:47.785 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-24T00:25:47.794 INFO:teuthology.task.internal:Creating archive directory... 2024-03-24T00:25:47.795 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-24T00:25:47.799 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-24T00:25:47.860 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-24T00:25:47.866 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-24T00:25:47.866 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-24T00:25:47.873 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-24T00:25:47.951 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T00:25:47.964 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T00:25:47.987 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T00:25:47.999 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T00:25:48.003 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-24T00:25:48.012 INFO:teuthology.task.internal:Configuring sudo... 2024-03-24T00:25:48.013 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-24T00:25:48.016 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-24T00:25:48.094 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-24T00:25:48.103 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-24T00:25:48.103 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-24T00:25:48.107 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-24T00:25:48.154 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T00:25:48.249 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T00:25:48.343 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T00:25:48.407 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T00:25:48.498 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:25:48.498 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-24T00:25:48.578 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T00:25:48.656 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T00:25:48.763 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T00:25:48.862 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T00:25:48.947 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:25:48.947 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-24T00:25:49.016 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-03-24T00:25:49.022 DEBUG:teuthology.orchestra.run.smithi088:> sudo service rsyslog restart 2024-03-24T00:25:49.121 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T00:25:49.178 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T00:25:49.535 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-24T00:25:49.543 INFO:teuthology.task.internal:Starting timer... 2024-03-24T00:25:49.543 INFO:teuthology.run_tasks:Running task pcp... 2024-03-24T00:25:49.553 INFO:teuthology.run_tasks:Running task selinux... 2024-03-24T00:25:49.563 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-24T00:25:49.563 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-03-24T00:25:49.676 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-03-24T00:25:49.682 DEBUG:teuthology.orchestra.run.smithi088:> sudo service auditd rotate 2024-03-24T00:25:49.900 INFO:teuthology.orchestra.run.smithi088.stdout:Rotating logs: 2024-03-24T00:25:49.906 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-24T00:25:49.906 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-03-24T00:25:49.959 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-03-24T00:25:49.959 DEBUG:teuthology.orchestra.run.smithi088:> /usr/sbin/getenforce 2024-03-24T00:25:50.041 INFO:teuthology.orchestra.run.smithi088.stdout:Permissive 2024-03-24T00:25:50.041 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi038.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi088.front.sepia.ceph.com': 'permissive'} 2024-03-24T00:25:50.041 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T00:25:50.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T00:25:50.126 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T00:25:50.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T00:25:50.275 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-24T00:25:50.275 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-03-24T00:25:50.372 DEBUG:teuthology.orchestra.run.smithi088:> sudo /usr/sbin/setenforce permissive 2024-03-24T00:25:50.499 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-24T00:25:50.509 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-24T00:25:50.515 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-24T00:25:50.665 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-24T00:25:50.679 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-24T00:25:50.680 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi038.front.sepia.ceph.com,smithi088.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-24T00:31:57.781 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi088.front.sepia.ceph.com')] 2024-03-24T00:31:57.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-03-24T00:31:57.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T00:31:57.938 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-03-24T00:31:58.098 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-03-24T00:31:58.099 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-03-24T00:31:58.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T00:31:58.242 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-03-24T00:31:58.397 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-03-24T00:31:58.397 INFO:teuthology.run_tasks:Running task clock... 2024-03-24T00:31:58.410 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-24T00:31:58.410 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-24T00:31:58.411 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T00:31:58.418 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-24T00:31:58.418 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T00:31:58.513 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-24T00:31:58.523 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-24T00:31:58.564 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-24T00:31:58.581 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-24T00:31:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: ntpd: command not found 2024-03-24T00:31:58.688 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-03-24T00:31:58.691 INFO:teuthology.orchestra.run.smithi088.stdout:506 Cannot talk to daemon 2024-03-24T00:31:58.736 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-03-24T00:31:58.741 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-24T00:31:58.790 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-24T00:31:58.792 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-24T00:31:58.844 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-24T00:31:58.921 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-03-24T00:31:58.929 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T00:31:58.929 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-03-24T00:31:58.929 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T00:31:58.929 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T00:31:58.929 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T00:31:58.929 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T00:31:58.985 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-03-24T00:31:58.995 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T00:31:58.995 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-03-24T00:31:58.995 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T00:31:58.995 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T00:31:58.995 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T00:31:58.996 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T00:31:58.997 INFO:teuthology.run_tasks:Running task install... 2024-03-24T00:31:59.006 DEBUG:teuthology.task.install:project ceph 2024-03-24T00:31:59.006 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}} 2024-03-24T00:31:59.006 DEBUG:teuthology.task.install:config {'branch': 'reef', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-24T00:31:59.007 INFO:teuthology.task.install:Using flavor: default 2024-03-24T00:31:59.015 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-24T00:31:59.015 INFO:teuthology.task.install:extra packages: [] 2024-03-24T00:31:59.015 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-24T00:31:59.016 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T00:31:59.016 INFO:teuthology.packaging:ref: None 2024-03-24T00:31:59.016 INFO:teuthology.packaging:tag: None 2024-03-24T00:31:59.016 INFO:teuthology.packaging:branch: reef 2024-03-24T00:31:59.016 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:31:59.016 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-24T00:31:59.018 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-24T00:31:59.018 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T00:31:59.019 INFO:teuthology.packaging:ref: None 2024-03-24T00:31:59.019 INFO:teuthology.packaging:tag: None 2024-03-24T00:31:59.019 INFO:teuthology.packaging:branch: reef 2024-03-24T00:31:59.019 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:31:59.019 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-24T00:31:59.255 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/738c2fd67dbc0b17e2e99568a03974712d998229/centos/9/flavors/default/ 2024-03-24T00:31:59.255 INFO:teuthology.task.install.rpm:Package version is 18.2.2-952.g738c2fd6 2024-03-24T00:31:59.359 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/738c2fd67dbc0b17e2e99568a03974712d998229/centos/9/flavors/default/ 2024-03-24T00:31:59.359 INFO:teuthology.task.install.rpm:Package version is 18.2.2-952.g738c2fd6 2024-03-24T00:31:59.369 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/738c2fd67dbc0b17e2e99568a03974712d998229/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/738c2fd67dbc0b17e2e99568a03974712d998229/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/738c2fd67dbc0b17e2e99568a03974712d998229/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-24T00:31:59.369 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:31:59.369 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-24T00:31:59.470 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-24T00:31:59.470 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T00:31:59.470 INFO:teuthology.packaging:ref: None 2024-03-24T00:31:59.470 INFO:teuthology.packaging:tag: None 2024-03-24T00:31:59.470 INFO:teuthology.packaging:branch: reef 2024-03-24T00:31:59.470 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:31:59.470 DEBUG:teuthology.orchestra.run.smithi088:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-24T00:31:59.531 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/738c2fd67dbc0b17e2e99568a03974712d998229/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/738c2fd67dbc0b17e2e99568a03974712d998229/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/738c2fd67dbc0b17e2e99568a03974712d998229/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-24T00:31:59.531 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:31:59.531 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-24T00:31:59.609 DEBUG:teuthology.orchestra.run.smithi088:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-24T00:31:59.638 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-24T00:31:59.638 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T00:31:59.639 INFO:teuthology.packaging:ref: None 2024-03-24T00:31:59.639 INFO:teuthology.packaging:tag: None 2024-03-24T00:31:59.639 INFO:teuthology.packaging:branch: reef 2024-03-24T00:31:59.639 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:31:59.639 DEBUG:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-24T00:31:59.734 DEBUG:teuthology.orchestra.run.smithi088:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-24T00:31:59.786 DEBUG:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-24T00:31:59.822 INFO:teuthology.orchestra.run.smithi088.stdout:check_obsoletes = 1 2024-03-24T00:31:59.826 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum clean all 2024-03-24T00:31:59.930 DEBUG:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-24T00:32:00.028 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2024-03-24T00:32:00.032 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum clean all 2024-03-24T00:32:00.176 INFO:teuthology.orchestra.run.smithi088.stdout:74 files removed 2024-03-24T00:32:00.217 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-radosgw 2024-03-24T00:32:00.409 INFO:teuthology.orchestra.run.smithi038.stdout:74 files removed 2024-03-24T00:32:00.449 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw 2024-03-24T00:32:01.205 INFO:teuthology.orchestra.run.smithi088.stdout:ceph packages for x86_64 159 kB/s | 77 kB 00:00 2024-03-24T00:32:01.463 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 158 kB/s | 77 kB 00:00 2024-03-24T00:32:01.763 INFO:teuthology.orchestra.run.smithi088.stdout:ceph noarch packages 21 kB/s | 9.9 kB 00:00 2024-03-24T00:32:01.963 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 24 kB/s | 9.9 kB 00:00 2024-03-24T00:32:02.340 INFO:teuthology.orchestra.run.smithi088.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-03-24T00:32:02.464 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-03-24T00:32:03.309 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-9 - AppStream 7.5 MB/s | 6.7 MB 00:00 2024-03-24T00:32:03.384 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS-9 - AppStream 7.9 MB/s | 6.7 MB 00:00 2024-03-24T00:32:05.076 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-24T00:32:05.118 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-24T00:32:06.464 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.0 MB 00:00 2024-03-24T00:32:07.422 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-03-24T00:32:09.030 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-03-24T00:32:09.373 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-03-24T00:32:13.715 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-03-24T00:32:13.933 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.1 MB 00:00 2024-03-24T00:32:15.374 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-03-24T00:32:15.708 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-03-24T00:32:16.623 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-03-24T00:32:16.985 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-03-24T00:32:22.501 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-03-24T00:32:23.101 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-24T00:32:25.101 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:32:25.120 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:32:25.120 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-03-24T00:32:25.120 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:32:25.120 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:32:25.120 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 7.7 M 2024-03-24T00:32:25.120 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrading: 2024-03-24T00:32:25.121 INFO:teuthology.orchestra.run.smithi088.stdout: librados2 x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 3.3 M 2024-03-24T00:32:25.121 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1 x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 3.0 M 2024-03-24T00:32:25.121 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-03-24T00:32:25.121 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-24T00:32:25.121 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 5.1 M 2024-03-24T00:32:25.121 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 18 M 2024-03-24T00:32:25.121 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 25 k 2024-03-24T00:32:25.121 INFO:teuthology.orchestra.run.smithi088.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-24T00:32:25.121 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-24T00:32:25.121 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-24T00:32:25.122 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-24T00:32:25.122 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2 x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 702 k 2024-03-24T00:32:25.122 INFO:teuthology.orchestra.run.smithi088.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-24T00:32:25.122 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-24T00:32:25.122 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-24T00:32:25.122 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1 x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 474 k 2024-03-24T00:32:25.122 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-24T00:32:25.122 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2 x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 4.5 M 2024-03-24T00:32:25.122 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-24T00:32:25.122 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-24T00:32:25.122 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-24T00:32:25.123 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 45 k 2024-03-24T00:32:25.123 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 127 k 2024-03-24T00:32:25.123 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 161 k 2024-03-24T00:32:25.123 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-24T00:32:25.123 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 321 k 2024-03-24T00:32:25.123 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 298 k 2024-03-24T00:32:25.123 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 99 k 2024-03-24T00:32:25.123 INFO:teuthology.orchestra.run.smithi088.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-24T00:32:25.123 INFO:teuthology.orchestra.run.smithi088.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-24T00:32:25.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:32:25.123 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:32:25.124 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:32:25.124 INFO:teuthology.orchestra.run.smithi088.stdout:Install 28 Packages 2024-03-24T00:32:25.124 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrade 2 Packages 2024-03-24T00:32:25.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:32:25.124 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 53 M 2024-03-24T00:32:25.124 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:32:25.785 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:32:25.798 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:32:25.798 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-03-24T00:32:25.798 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:32:25.798 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 7.7 M 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 3.3 M 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 3.0 M 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 5.1 M 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 18 M 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 25 k 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-24T00:32:25.799 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 702 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 474 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 4.5 M 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 45 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 127 k 2024-03-24T00:32:25.800 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 161 k 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 321 k 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 298 k 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 99 k 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout:Install 28 Packages 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 2 Packages 2024-03-24T00:32:25.801 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:32:25.802 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 53 M 2024-03-24T00:32:25.802 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:32:27.010 INFO:teuthology.orchestra.run.smithi088.stdout:(1/30): ceph-base-18.2.2-952.g738c2fd6.el9.x86_ 3.0 MB/s | 5.1 MB 00:01 2024-03-24T00:32:27.060 INFO:teuthology.orchestra.run.smithi088.stdout:(2/30): ceph-selinux-18.2.2-952.g738c2fd6.el9.x 515 kB/s | 25 kB 00:00 2024-03-24T00:32:27.219 INFO:teuthology.orchestra.run.smithi088.stdout:(3/30): libcephfs2-18.2.2-952.g738c2fd6.el9.x86 4.4 MB/s | 702 kB 00:00 2024-03-24T00:32:27.377 INFO:teuthology.orchestra.run.smithi088.stdout:(4/30): ceph-radosgw-18.2.2-952.g738c2fd6.el9.x 3.8 MB/s | 7.7 MB 00:02 2024-03-24T00:32:27.453 INFO:teuthology.orchestra.run.smithi088.stdout:(5/30): libradosstriper1-18.2.2-952.g738c2fd6.e 2.0 MB/s | 474 kB 00:00 2024-03-24T00:32:27.511 INFO:teuthology.orchestra.run.smithi088.stdout:(6/30): python3-ceph-argparse-18.2.2-952.g738c2 783 kB/s | 45 kB 00:00 2024-03-24T00:32:27.611 INFO:teuthology.orchestra.run.smithi088.stdout:(7/30): python3-ceph-common-18.2.2-952.g738c2fd 1.2 MB/s | 127 kB 00:00 2024-03-24T00:32:27.712 INFO:teuthology.orchestra.run.smithi088.stdout:(8/30): python3-cephfs-18.2.2-952.g738c2fd6.el9 1.6 MB/s | 161 kB 00:00 2024-03-24T00:32:27.812 INFO:teuthology.orchestra.run.smithi088.stdout:(9/30): python3-rados-18.2.2-952.g738c2fd6.el9. 3.2 MB/s | 321 kB 00:00 2024-03-24T00:32:27.963 INFO:teuthology.orchestra.run.smithi088.stdout:(10/30): librgw2-18.2.2-952.g738c2fd6.el9.x86_6 7.6 MB/s | 4.5 MB 00:00 2024-03-24T00:32:27.996 INFO:teuthology.orchestra.run.smithi088.stdout:(11/30): python3-rbd-18.2.2-952.g738c2fd6.el9.x 1.6 MB/s | 298 kB 00:00 2024-03-24T00:32:28.096 INFO:teuthology.orchestra.run.smithi088.stdout:(12/30): python3-rgw-18.2.2-952.g738c2fd6.el9.x 753 kB/s | 99 kB 00:00 2024-03-24T00:32:28.155 INFO:teuthology.orchestra.run.smithi088.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 671 kB/s | 106 kB 00:00 2024-03-24T00:32:28.171 INFO:teuthology.orchestra.run.smithi088.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-24T00:32:28.197 INFO:teuthology.orchestra.run.smithi088.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-03-24T00:32:28.239 INFO:teuthology.orchestra.run.smithi088.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 661 kB 00:00 2024-03-24T00:32:28.272 INFO:teuthology.orchestra.run.smithi088.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.0 MB/s | 282 kB 00:00 2024-03-24T00:32:28.323 INFO:teuthology.orchestra.run.smithi088.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 512 kB/s | 41 kB 00:00 2024-03-24T00:32:28.428 INFO:teuthology.orchestra.run.smithi038.stdout:(1/30): ceph-radosgw-18.2.2-952.g738c2fd6.el9.x 3.2 MB/s | 7.7 MB 00:02 2024-03-24T00:32:28.478 INFO:teuthology.orchestra.run.smithi038.stdout:(2/30): ceph-selinux-18.2.2-952.g738c2fd6.el9.x 517 kB/s | 25 kB 00:00 2024-03-24T00:32:28.565 INFO:teuthology.orchestra.run.smithi088.stdout:(19/30): ceph-common-18.2.2-952.g738c2fd6.el9.x 5.7 MB/s | 18 MB 00:03 2024-03-24T00:32:28.578 INFO:teuthology.orchestra.run.smithi038.stdout:(3/30): libcephfs2-18.2.2-952.g738c2fd6.el9.x86 6.9 MB/s | 702 kB 00:00 2024-03-24T00:32:28.695 INFO:teuthology.orchestra.run.smithi038.stdout:(4/30): libradosstriper1-18.2.2-952.g738c2fd6.e 4.0 MB/s | 474 kB 00:00 2024-03-24T00:32:28.776 INFO:teuthology.orchestra.run.smithi088.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 221 kB/s | 111 kB 00:00 2024-03-24T00:32:29.064 INFO:teuthology.orchestra.run.smithi088.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 608 kB/s | 309 kB 00:00 2024-03-24T00:32:29.065 INFO:teuthology.orchestra.run.smithi088.stdout:(22/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 317 kB/s | 26 kB 00:00 2024-03-24T00:32:29.065 INFO:teuthology.orchestra.run.smithi088.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 841 kB/s | 49 kB 00:00 2024-03-24T00:32:29.065 INFO:teuthology.orchestra.run.smithi088.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-24T00:32:29.065 INFO:teuthology.orchestra.run.smithi088.stdout:(25/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.2 MB/s | 838 kB 00:00 2024-03-24T00:32:29.065 INFO:teuthology.orchestra.run.smithi088.stdout:(26/30): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-03-24T00:32:29.150 INFO:teuthology.orchestra.run.smithi088.stdout:(27/30): libarrow-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 4.4 MB 00:00 2024-03-24T00:32:29.188 INFO:teuthology.orchestra.run.smithi038.stdout:(5/30): ceph-common-18.2.2-952.g738c2fd6.el9.x8 5.8 MB/s | 18 MB 00:03 2024-03-24T00:32:29.237 INFO:teuthology.orchestra.run.smithi038.stdout:(6/30): python3-ceph-argparse-18.2.2-952.g738c2 923 kB/s | 45 kB 00:00 2024-03-24T00:32:29.242 INFO:teuthology.orchestra.run.smithi088.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-03-24T00:32:29.346 INFO:teuthology.orchestra.run.smithi038.stdout:(7/30): librgw2-18.2.2-952.g738c2fd6.el9.x86_64 6.9 MB/s | 4.5 MB 00:00 2024-03-24T00:32:29.379 INFO:teuthology.orchestra.run.smithi038.stdout:(8/30): python3-ceph-common-18.2.2-952.g738c2fd 900 kB/s | 127 kB 00:00 2024-03-24T00:32:29.413 INFO:teuthology.orchestra.run.smithi038.stdout:(9/30): python3-cephfs-18.2.2-952.g738c2fd6.el9 2.4 MB/s | 161 kB 00:00 2024-03-24T00:32:29.450 INFO:teuthology.orchestra.run.smithi088.stdout:(29/30): librbd1-18.2.2-952.g738c2fd6.el9.x86_6 10 MB/s | 3.0 MB 00:00 2024-03-24T00:32:29.522 INFO:teuthology.orchestra.run.smithi038.stdout:(10/30): ceph-base-18.2.2-952.g738c2fd6.el9.x86 1.5 MB/s | 5.1 MB 00:03 2024-03-24T00:32:29.564 INFO:teuthology.orchestra.run.smithi038.stdout:(11/30): python3-rados-18.2.2-952.g738c2fd6.el9 1.7 MB/s | 321 kB 00:00 2024-03-24T00:32:29.597 INFO:teuthology.orchestra.run.smithi038.stdout:(12/30): python3-rgw-18.2.2-952.g738c2fd6.el9.x 1.3 MB/s | 99 kB 00:00 2024-03-24T00:32:29.698 INFO:teuthology.orchestra.run.smithi038.stdout:(13/30): python3-rbd-18.2.2-952.g738c2fd6.el9.x 1.0 MB/s | 298 kB 00:00 2024-03-24T00:32:29.815 INFO:teuthology.orchestra.run.smithi038.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 424 kB/s | 106 kB 00:00 2024-03-24T00:32:29.831 INFO:teuthology.orchestra.run.smithi038.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 619 kB/s | 44 kB 00:00 2024-03-24T00:32:29.890 INFO:teuthology.orchestra.run.smithi038.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 719 kB/s | 159 kB 00:00 2024-03-24T00:32:29.901 INFO:teuthology.orchestra.run.smithi088.stdout:(30/30): librados2-18.2.2-952.g738c2fd6.el9.x86 3.9 MB/s | 3.3 MB 00:00 2024-03-24T00:32:29.908 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:32:29.908 INFO:teuthology.orchestra.run.smithi088.stdout:Total 11 MB/s | 53 MB 00:04 2024-03-24T00:32:29.909 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:32:29.966 INFO:teuthology.orchestra.run.smithi038.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 661 kB 00:00 2024-03-24T00:32:29.985 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:32:29.985 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:32:29.999 INFO:teuthology.orchestra.run.smithi038.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 282 kB 00:00 2024-03-24T00:32:30.024 INFO:teuthology.orchestra.run.smithi038.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 313 kB/s | 41 kB 00:00 2024-03-24T00:32:30.458 INFO:teuthology.orchestra.run.smithi038.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 227 kB/s | 111 kB 00:00 2024-03-24T00:32:30.510 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:32:30.511 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:32:30.550 INFO:teuthology.orchestra.run.smithi038.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 589 kB/s | 309 kB 00:00 2024-03-24T00:32:30.733 INFO:teuthology.orchestra.run.smithi038.stdout:(22/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 95 kB/s | 26 kB 00:00 2024-03-24T00:32:30.892 INFO:teuthology.orchestra.run.smithi038.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-03-24T00:32:31.084 INFO:teuthology.orchestra.run.smithi038.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 193 kB/s | 67 kB 00:00 2024-03-24T00:32:31.259 INFO:teuthology.orchestra.run.smithi038.stdout:(25/30): libarrow-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-03-24T00:32:31.501 INFO:teuthology.orchestra.run.smithi038.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.3 MB/s | 838 kB 00:00 2024-03-24T00:32:31.593 INFO:teuthology.orchestra.run.smithi038.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 377 kB/s | 192 kB 00:00 2024-03-24T00:32:31.902 INFO:teuthology.orchestra.run.smithi038.stdout:(28/30): librbd1-18.2.2-952.g738c2fd6.el9.x86_6 9.8 MB/s | 3.0 MB 00:00 2024-03-24T00:32:32.068 INFO:teuthology.orchestra.run.smithi038.stdout:(29/30): thrift-0.15.0-2.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-03-24T00:32:32.094 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:32:32.190 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-24T00:32:32.225 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-24T00:32:32.252 INFO:teuthology.orchestra.run.smithi038.stdout:(30/30): librados2-18.2.2-952.g738c2fd6.el9.x86 4.4 MB/s | 3.3 MB 00:00 2024-03-24T00:32:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:32:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:Total 8.2 MB/s | 53 MB 00:06 2024-03-24T00:32:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:32:32.321 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-24T00:32:32.344 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:32:32.344 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:32:32.744 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-24T00:32:32.788 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librados2-2:18.2.2-952.g738c2fd6.el9.x86_64 5/32 2024-03-24T00:32:32.862 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:32:32.863 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:32:32.921 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:18.2.2-952.g738c2fd6.el9.x86_64 5/32 2024-03-24T00:32:33.031 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rados-2:18.2.2-952.g738c2fd6.el9.x86_64 6/32 2024-03-24T00:32:33.065 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs2-2:18.2.2-952.g738c2fd6.el9.x86_64 7/32 2024-03-24T00:32:33.148 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephfs2-2:18.2.2-952.g738c2fd6.el9.x86_64 7/32 2024-03-24T00:32:33.190 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-24T00:32:33.263 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-argparse-2:18.2.2-952.g738c2fd6.el9.x 9/32 2024-03-24T00:32:33.375 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cephfs-2:18.2.2-952.g738c2fd6.el9.x86_64 10/32 2024-03-24T00:32:33.410 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libradosstriper1-2:18.2.2-952.g738c2fd6.el9.x86_64 11/32 2024-03-24T00:32:33.509 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libradosstriper1-2:18.2.2-952.g738c2fd6.el9.x86_64 11/32 2024-03-24T00:32:33.686 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-24T00:32:33.752 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-24T00:32:33.809 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-24T00:32:33.843 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-24T00:32:33.883 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-24T00:32:33.932 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-24T00:32:33.989 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-24T00:32:34.386 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:32:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-24T00:32:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librbd1-2:18.2.2-952.g738c2fd6.el9.x86_64 20/32 2024-03-24T00:32:34.523 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-24T00:32:34.561 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-24T00:32:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:18.2.2-952.g738c2fd6.el9.x86_64 20/32 2024-03-24T00:32:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rbd-2:18.2.2-952.g738c2fd6.el9.x86_64 21/32 2024-03-24T00:32:34.665 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-24T00:32:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-24T00:32:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-24T00:32:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:18.2.2-952.g738c2fd6.el9.x86_64 5/32 2024-03-24T00:32:35.199 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-24T00:32:35.234 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librgw2-2:18.2.2-952.g738c2fd6.el9.x86_64 24/32 2024-03-24T00:32:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:18.2.2-952.g738c2fd6.el9.x86_64 5/32 2024-03-24T00:32:35.318 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librgw2-2:18.2.2-952.g738c2fd6.el9.x86_64 24/32 2024-03-24T00:32:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:18.2.2-952.g738c2fd6.el9.x86_64 6/32 2024-03-24T00:32:35.416 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rgw-2:18.2.2-952.g738c2fd6.el9.x86_64 25/32 2024-03-24T00:32:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:18.2.2-952.g738c2fd6.el9.x86_64 7/32 2024-03-24T00:32:35.462 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-common-2:18.2.2-952.g738c2fd6.el9.x86 26/32 2024-03-24T00:32:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:18.2.2-952.g738c2fd6.el9.x86_64 7/32 2024-03-24T00:32:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-24T00:32:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:18.2.2-952.g738c2fd6.el9.x 9/32 2024-03-24T00:32:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:18.2.2-952.g738c2fd6.el9.x86_64 10/32 2024-03-24T00:32:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:18.2.2-952.g738c2fd6.el9.x86_64 11/32 2024-03-24T00:32:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:18.2.2-952.g738c2fd6.el9.x86_64 11/32 2024-03-24T00:32:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-24T00:32:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-24T00:32:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-24T00:32:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-24T00:32:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-24T00:32:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-24T00:32:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-24T00:32:36.762 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-24T00:32:36.791 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:18.2.2-952.g738c2fd6.el9.x86_64 20/32 2024-03-24T00:32:36.888 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:18.2.2-952.g738c2fd6.el9.x86_64 20/32 2024-03-24T00:32:36.922 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:18.2.2-952.g738c2fd6.el9.x86_64 21/32 2024-03-24T00:32:36.983 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-24T00:32:37.477 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-24T00:32:37.502 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:18.2.2-952.g738c2fd6.el9.x86_64 24/32 2024-03-24T00:32:37.593 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:18.2.2-952.g738c2fd6.el9.x86_64 24/32 2024-03-24T00:32:37.697 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:18.2.2-952.g738c2fd6.el9.x86_64 25/32 2024-03-24T00:32:37.738 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:18.2.2-952.g738c2fd6.el9.x86 26/32 2024-03-24T00:32:37.953 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 27/32 2024-03-24T00:32:38.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 27/32 2024-03-24T00:32:38.624 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 27/32 2024-03-24T00:32:38.695 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-base-2:18.2.2-952.g738c2fd6.el9.x86_64 28/32 2024-03-24T00:32:39.105 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-base-2:18.2.2-952.g738c2fd6.el9.x86_64 28/32 2024-03-24T00:32:39.105 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-24T00:32:39.105 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-24T00:32:39.105 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:32:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-selinux-2:18.2.2-952.g738c2fd6.el9.x86_64 29/32 2024-03-24T00:32:40.192 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 27/32 2024-03-24T00:32:40.235 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 27/32 2024-03-24T00:32:40.878 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 27/32 2024-03-24T00:32:40.922 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:18.2.2-952.g738c2fd6.el9.x86_64 28/32 2024-03-24T00:32:41.086 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:18.2.2-952.g738c2fd6.el9.x86_64 28/32 2024-03-24T00:32:41.086 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-24T00:32:41.086 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-24T00:32:41.086 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:32:41.122 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:18.2.2-952.g738c2fd6.el9.x86_64 29/32 2024-03-24T00:32:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-selinux-2:18.2.2-952.g738c2fd6.el9.x86_64 29/32 2024-03-24T00:32:49.826 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /sys 2024-03-24T00:32:49.826 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /proc 2024-03-24T00:32:49.826 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /mnt 2024-03-24T00:32:49.826 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /var/tmp 2024-03-24T00:32:49.826 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /home 2024-03-24T00:32:49.826 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /root 2024-03-24T00:32:49.826 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /tmp 2024-03-24T00:32:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:32:50.697 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-radosgw-2:18.2.2-952.g738c2fd6.el9.x86_64 30/32 2024-03-24T00:32:50.782 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-952.g738c2fd6.el9.x86_64 30/32 2024-03-24T00:32:50.782 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:32:50.782 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-24T00:32:50.782 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T00:32:50.782 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T00:32:50.782 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:32:50.787 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-24T00:32:50.909 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-24T00:32:50.909 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-24T00:32:52.422 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-base-2:18.2.2-952.g738c2fd6.el9.x86_64 1/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 2/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-radosgw-2:18.2.2-952.g738c2fd6.el9.x86_64 3/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-selinux-2:18.2.2-952.g738c2fd6.el9.x86_64 4/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs2-2:18.2.2-952.g738c2fd6.el9.x86_64 5/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libradosstriper1-2:18.2.2-952.g738c2fd6.el9.x86_64 6/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librgw2-2:18.2.2-952.g738c2fd6.el9.x86_64 7/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-argparse-2:18.2.2-952.g738c2fd6.el9.x 8/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-common-2:18.2.2-952.g738c2fd6.el9.x86 9/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cephfs-2:18.2.2-952.g738c2fd6.el9.x86_64 10/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rados-2:18.2.2-952.g738c2fd6.el9.x86_64 11/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rbd-2:18.2.2-952.g738c2fd6.el9.x86_64 12/32 2024-03-24T00:32:52.423 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rgw-2:18.2.2-952.g738c2fd6.el9.x86_64 13/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-24T00:32:52.424 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-24T00:32:52.425 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-24T00:32:52.425 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:18.2.2-952.g738c2fd6.el9.x86_64 29/32 2024-03-24T00:32:52.425 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-24T00:32:52.425 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:18.2.2-952.g738c2fd6.el9.x86_64 31/32 2024-03-24T00:32:52.546 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:18.2.2-952.g738c2fd6.el9.x86_64 29/32 2024-03-24T00:32:52.546 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2024-03-24T00:32:52.546 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2024-03-24T00:32:52.547 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2024-03-24T00:32:52.547 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2024-03-24T00:32:52.547 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2024-03-24T00:32:52.547 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /root 2024-03-24T00:32:52.547 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2024-03-24T00:32:52.547 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:32:53.341 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-24T00:32:53.341 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout:Upgraded: 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout: librados2-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-24T00:32:53.342 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.344 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.344 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-24T00:32:53.344 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.344 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.344 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:53.344 INFO:teuthology.orchestra.run.smithi088.stdout: re2-1:20211101-3.el9.x86_64 2024-03-24T00:32:53.344 INFO:teuthology.orchestra.run.smithi088.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-24T00:32:53.344 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:32:53.344 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:32:53.491 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:18.2.2-952.g738c2fd6.el9.x86_64 30/32 2024-03-24T00:32:53.573 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-952.g738c2fd6.el9.x86_64 30/32 2024-03-24T00:32:53.573 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:32:53.573 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-24T00:32:53.573 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T00:32:53.573 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T00:32:53.573 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:32:53.579 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-24T00:32:53.682 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-24T00:32:53.682 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-24T00:32:53.763 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-test 2024-03-24T00:32:54.598 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:32 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:32:54.761 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:32:54.762 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:32:54.762 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-03-24T00:32:54.762 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:32:54.762 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:32:54.762 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 40 M 2024-03-24T00:32:54.762 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-03-24T00:32:54.762 INFO:teuthology.orchestra.run.smithi088.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-24T00:32:54.762 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 166 k 2024-03-24T00:32:54.763 INFO:teuthology.orchestra.run.smithi088.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-24T00:32:54.763 INFO:teuthology.orchestra.run.smithi088.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-24T00:32:54.763 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-24T00:32:54.763 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:32:54.763 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:32:54.763 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:32:54.763 INFO:teuthology.orchestra.run.smithi088.stdout:Install 6 Packages 2024-03-24T00:32:54.763 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:32:54.764 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 41 M 2024-03-24T00:32:54.764 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 165 M 2024-03-24T00:32:54.764 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:32:55.155 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-24T00:32:55.155 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:18.2.2-952.g738c2fd6.el9.x86_64 1/32 2024-03-24T00:32:55.155 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 2/32 2024-03-24T00:32:55.155 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:18.2.2-952.g738c2fd6.el9.x86_64 3/32 2024-03-24T00:32:55.155 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:18.2.2-952.g738c2fd6.el9.x86_64 4/32 2024-03-24T00:32:55.155 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:18.2.2-952.g738c2fd6.el9.x86_64 5/32 2024-03-24T00:32:55.155 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:18.2.2-952.g738c2fd6.el9.x86_64 6/32 2024-03-24T00:32:55.155 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:18.2.2-952.g738c2fd6.el9.x86_64 7/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:18.2.2-952.g738c2fd6.el9.x 8/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:18.2.2-952.g738c2fd6.el9.x86 9/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:18.2.2-952.g738c2fd6.el9.x86_64 10/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:18.2.2-952.g738c2fd6.el9.x86_64 11/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:18.2.2-952.g738c2fd6.el9.x86_64 12/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:18.2.2-952.g738c2fd6.el9.x86_64 13/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-24T00:32:55.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-24T00:32:55.157 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-24T00:32:55.157 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-24T00:32:55.157 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-24T00:32:55.157 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-24T00:32:55.157 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-24T00:32:55.157 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-24T00:32:55.158 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-24T00:32:55.158 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:18.2.2-952.g738c2fd6.el9.x86_64 29/32 2024-03-24T00:32:55.158 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-24T00:32:55.158 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:18.2.2-952.g738c2fd6.el9.x86_64 31/32 2024-03-24T00:32:55.317 INFO:teuthology.orchestra.run.smithi088.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 915 kB/s | 217 kB 00:00 2024-03-24T00:32:55.377 INFO:teuthology.orchestra.run.smithi088.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-24T00:32:55.418 INFO:teuthology.orchestra.run.smithi088.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-24T00:32:55.560 INFO:teuthology.orchestra.run.smithi088.stdout:(4/6): libcephsqlite-18.2.2-952.g738c2fd6.el9.x 346 kB/s | 166 kB 00:00 2024-03-24T00:32:55.935 INFO:teuthology.orchestra.run.smithi088.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 363 kB/s | 187 kB 00:00 2024-03-24T00:32:55.979 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-24T00:32:55.980 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.981 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.982 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-24T00:32:55.982 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.982 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.982 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:32:55.982 INFO:teuthology.orchestra.run.smithi038.stdout: re2-1:20211101-3.el9.x86_64 2024-03-24T00:32:55.982 INFO:teuthology.orchestra.run.smithi038.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-24T00:32:55.982 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:32:55.982 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:32:56.420 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-test 2024-03-24T00:32:57.289 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:34 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:32:57.461 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:32:57.462 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:32:57.462 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-03-24T00:32:57.462 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:32:57.462 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:32:57.462 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 40 M 2024-03-24T00:32:57.462 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-03-24T00:32:57.463 INFO:teuthology.orchestra.run.smithi038.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-24T00:32:57.463 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 166 k 2024-03-24T00:32:57.463 INFO:teuthology.orchestra.run.smithi038.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-24T00:32:57.463 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-24T00:32:57.463 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-24T00:32:57.463 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:32:57.463 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:32:57.463 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:32:57.463 INFO:teuthology.orchestra.run.smithi038.stdout:Install 6 Packages 2024-03-24T00:32:57.463 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:32:57.464 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 41 M 2024-03-24T00:32:57.464 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 165 M 2024-03-24T00:32:57.464 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:32:57.709 INFO:teuthology.orchestra.run.smithi038.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.1 MB/s | 217 kB 00:00 2024-03-24T00:32:57.760 INFO:teuthology.orchestra.run.smithi038.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-24T00:32:57.810 INFO:teuthology.orchestra.run.smithi038.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-24T00:32:57.835 INFO:teuthology.orchestra.run.smithi038.stdout:(4/6): libcephsqlite-18.2.2-952.g738c2fd6.el9.x 520 kB/s | 166 kB 00:00 2024-03-24T00:32:57.943 INFO:teuthology.orchestra.run.smithi088.stdout:(6/6): ceph-test-18.2.2-952.g738c2fd6.el9.x86_6 14 MB/s | 40 MB 00:02 2024-03-24T00:32:57.949 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:32:57.950 INFO:teuthology.orchestra.run.smithi088.stdout:Total 13 MB/s | 41 MB 00:03 2024-03-24T00:32:57.990 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:32:58.044 INFO:teuthology.orchestra.run.smithi038.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 805 kB/s | 187 kB 00:00 2024-03-24T00:32:58.044 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:32:58.044 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:32:58.445 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:32:58.446 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:32:59.084 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:32:59.156 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-24T00:32:59.203 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-24T00:32:59.259 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T00:32:59.328 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-24T00:32:59.355 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephsqlite-2:18.2.2-952.g738c2fd6.el9.x86_64 5/6 2024-03-24T00:33:00.352 INFO:teuthology.orchestra.run.smithi038.stdout:(6/6): ceph-test-18.2.2-952.g738c2fd6.el9.x86_6 14 MB/s | 40 MB 00:02 2024-03-24T00:33:00.358 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:00.359 INFO:teuthology.orchestra.run.smithi038.stdout:Total 14 MB/s | 41 MB 00:02 2024-03-24T00:33:00.401 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:33:00.460 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:33:00.460 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:33:00.869 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:33:00.869 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:33:01.487 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:33:01.568 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-24T00:33:01.651 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-24T00:33:01.706 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T00:33:01.781 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-24T00:33:01.805 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephsqlite-2:18.2.2-952.g738c2fd6.el9.x86_64 5/6 2024-03-24T00:33:03.452 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephsqlite-2:18.2.2-952.g738c2fd6.el9.x86_64 5/6 2024-03-24T00:33:03.570 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-test-2:18.2.2-952.g738c2fd6.el9.x86_64 6/6 2024-03-24T00:33:05.633 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-test-2:18.2.2-952.g738c2fd6.el9.x86_64 6/6 2024-03-24T00:33:05.633 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-test-2:18.2.2-952.g738c2fd6.el9.x86_64 1/6 2024-03-24T00:33:05.634 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephsqlite-2:18.2.2-952.g738c2fd6.el9.x86_64 2/6 2024-03-24T00:33:05.634 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T00:33:05.634 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-24T00:33:05.634 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-24T00:33:05.997 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-24T00:33:05.997 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:05.997 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:33:05.998 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:05.998 INFO:teuthology.orchestra.run.smithi088.stdout: jq-1.6-16.el9.x86_64 2024-03-24T00:33:05.998 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:05.998 INFO:teuthology.orchestra.run.smithi088.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-24T00:33:05.998 INFO:teuthology.orchestra.run.smithi088.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-24T00:33:05.998 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-24T00:33:05.998 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:05.998 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:33:06.063 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephsqlite-2:18.2.2-952.g738c2fd6.el9.x86_64 5/6 2024-03-24T00:33:06.178 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:18.2.2-952.g738c2fd6.el9.x86_64 6/6 2024-03-24T00:33:06.255 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph 2024-03-24T00:33:07.103 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:45 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:33:07.272 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================== 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================== 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout: ceph x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 6.5 k 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 2.1 M 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 1.4 M 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 245 k 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 4.7 M 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 16 M 2024-03-24T00:33:07.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-24T00:33:07.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:07.277 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:33:07.278 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================== 2024-03-24T00:33:07.278 INFO:teuthology.orchestra.run.smithi088.stdout:Install 33 Packages 2024-03-24T00:33:07.278 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:07.278 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 29 M 2024-03-24T00:33:07.278 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 104 M 2024-03-24T00:33:07.278 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:33:07.861 INFO:teuthology.orchestra.run.smithi088.stdout:(1/33): ceph-18.2.2-952.g738c2fd6.el9.x86_64.rp 24 kB/s | 6.5 kB 00:00 2024-03-24T00:33:08.069 INFO:teuthology.orchestra.run.smithi088.stdout:(2/33): ceph-mds-18.2.2-952.g738c2fd6.el9.x86_6 4.4 MB/s | 2.1 MB 00:00 2024-03-24T00:33:08.128 INFO:teuthology.orchestra.run.smithi088.stdout:(3/33): ceph-mgr-18.2.2-952.g738c2fd6.el9.x86_6 2.7 MB/s | 1.4 MB 00:00 2024-03-24T00:33:08.155 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-test-2:18.2.2-952.g738c2fd6.el9.x86_64 6/6 2024-03-24T00:33:08.155 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:18.2.2-952.g738c2fd6.el9.x86_64 1/6 2024-03-24T00:33:08.155 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephsqlite-2:18.2.2-952.g738c2fd6.el9.x86_64 2/6 2024-03-24T00:33:08.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T00:33:08.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-24T00:33:08.156 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-24T00:33:08.228 INFO:teuthology.orchestra.run.smithi088.stdout:(4/33): ceph-mgr-modules-core-18.2.2-952.g738c2 2.4 MB/s | 245 kB 00:00 2024-03-24T00:33:08.378 INFO:teuthology.orchestra.run.smithi088.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 968 kB/s | 144 kB 00:00 2024-03-24T00:33:08.412 INFO:teuthology.orchestra.run.smithi088.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 967 kB/s | 32 kB 00:00 2024-03-24T00:33:08.446 INFO:teuthology.orchestra.run.smithi088.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-24T00:33:08.504 INFO:teuthology.orchestra.run.smithi088.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 670 kB/s | 39 kB 00:00 2024-03-24T00:33:08.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-24T00:33:08.545 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:08.545 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:33:08.545 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:08.545 INFO:teuthology.orchestra.run.smithi038.stdout: jq-1.6-16.el9.x86_64 2024-03-24T00:33:08.545 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:08.545 INFO:teuthology.orchestra.run.smithi038.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-24T00:33:08.545 INFO:teuthology.orchestra.run.smithi038.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-24T00:33:08.545 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-24T00:33:08.545 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:08.546 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:33:08.554 INFO:teuthology.orchestra.run.smithi088.stdout:(9/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.8 MB/s | 241 kB 00:00 2024-03-24T00:33:08.630 INFO:teuthology.orchestra.run.smithi088.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-03-24T00:33:08.672 INFO:teuthology.orchestra.run.smithi088.stdout:(11/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 103 kB 00:00 2024-03-24T00:33:08.714 INFO:teuthology.orchestra.run.smithi088.stdout:(12/33): python3-pycparser-2.20-6.el9.noarch.rp 3.0 MB/s | 124 kB 00:00 2024-03-24T00:33:08.756 INFO:teuthology.orchestra.run.smithi088.stdout:(13/33): python3-urllib3-1.26.5-5.el9.noarch.rp 4.5 MB/s | 187 kB 00:00 2024-03-24T00:33:08.890 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph 2024-03-24T00:33:09.015 INFO:teuthology.orchestra.run.smithi088.stdout:(14/33): ceph-osd-18.2.2-952.g738c2fd6.el9.x86_ 17 MB/s | 16 MB 00:00 2024-03-24T00:33:09.132 INFO:teuthology.orchestra.run.smithi088.stdout:(15/33): ceph-mon-18.2.2-952.g738c2fd6.el9.x86_ 3.7 MB/s | 4.7 MB 00:01 2024-03-24T00:33:09.173 INFO:teuthology.orchestra.run.smithi088.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 300 kB/s | 125 kB 00:00 2024-03-24T00:33:09.365 INFO:teuthology.orchestra.run.smithi088.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 125 kB/s | 43 kB 00:00 2024-03-24T00:33:09.466 INFO:teuthology.orchestra.run.smithi088.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 108 kB/s | 11 kB 00:00 2024-03-24T00:33:09.566 INFO:teuthology.orchestra.run.smithi088.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 178 kB/s | 18 kB 00:00 2024-03-24T00:33:09.591 INFO:teuthology.orchestra.run.smithi088.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 375 kB/s | 172 kB 00:00 2024-03-24T00:33:09.658 INFO:teuthology.orchestra.run.smithi088.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 254 kB/s | 23 kB 00:00 2024-03-24T00:33:09.683 INFO:teuthology.orchestra.run.smithi088.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 213 kB/s | 19 kB 00:00 2024-03-24T00:33:09.717 INFO:teuthology.orchestra.run.smithi088.stdout:(23/33): python3-cherrypy-18.6.1-2.el9.noarch.r 661 kB/s | 358 kB 00:00 2024-03-24T00:33:09.750 INFO:teuthology.orchestra.run.smithi088.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 214 kB/s | 20 kB 00:00 2024-03-24T00:33:09.770 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:46 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:33:09.776 INFO:teuthology.orchestra.run.smithi088.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 506 kB/s | 46 kB 00:00 2024-03-24T00:33:09.818 INFO:teuthology.orchestra.run.smithi088.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 789 kB/s | 79 kB 00:00 2024-03-24T00:33:09.893 INFO:teuthology.orchestra.run.smithi088.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 141 kB/s | 16 kB 00:00 2024-03-24T00:33:09.918 INFO:teuthology.orchestra.run.smithi088.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 902 kB/s | 90 kB 00:00 2024-03-24T00:33:09.937 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:33:09.939 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================== 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================== 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 6.5 k 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 2.1 M 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 1.4 M 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 245 k 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 4.7 M 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 16 M 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-24T00:33:09.940 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-24T00:33:09.941 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:09.942 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:33:09.943 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================== 2024-03-24T00:33:09.943 INFO:teuthology.orchestra.run.smithi038.stdout:Install 33 Packages 2024-03-24T00:33:09.943 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:09.943 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 29 M 2024-03-24T00:33:09.943 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 104 M 2024-03-24T00:33:09.943 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:33:09.993 INFO:teuthology.orchestra.run.smithi088.stdout:(29/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-03-24T00:33:10.019 INFO:teuthology.orchestra.run.smithi088.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 287 kB/s | 36 kB 00:00 2024-03-24T00:33:10.052 INFO:teuthology.orchestra.run.smithi088.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-03-24T00:33:10.111 INFO:teuthology.orchestra.run.smithi088.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 219 kB/s | 20 kB 00:00 2024-03-24T00:33:10.161 INFO:teuthology.orchestra.run.smithi088.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2024-03-24T00:33:10.168 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:10.168 INFO:teuthology.orchestra.run.smithi088.stdout:Total 10 MB/s | 29 MB 00:02 2024-03-24T00:33:10.208 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:33:10.292 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:33:10.292 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:33:10.720 INFO:teuthology.orchestra.run.smithi038.stdout:(1/33): ceph-18.2.2-952.g738c2fd6.el9.x86_64.rp 30 kB/s | 6.5 kB 00:00 2024-03-24T00:33:10.730 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:33:10.731 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:33:10.946 INFO:teuthology.orchestra.run.smithi038.stdout:(2/33): ceph-mds-18.2.2-952.g738c2fd6.el9.x86_6 4.8 MB/s | 2.1 MB 00:00 2024-03-24T00:33:11.313 INFO:teuthology.orchestra.run.smithi038.stdout:(3/33): ceph-mon-18.2.2-952.g738c2fd6.el9.x86_6 7.9 MB/s | 4.7 MB 00:00 2024-03-24T00:33:11.388 INFO:teuthology.orchestra.run.smithi038.stdout:(4/33): ceph-mgr-modules-core-18.2.2-952.g738c2 3.3 MB/s | 245 kB 00:00 2024-03-24T00:33:11.538 INFO:teuthology.orchestra.run.smithi038.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 971 kB/s | 144 kB 00:00 2024-03-24T00:33:11.605 INFO:teuthology.orchestra.run.smithi038.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 478 kB/s | 32 kB 00:00 2024-03-24T00:33:11.655 INFO:teuthology.orchestra.run.smithi038.stdout:(7/33): ceph-mgr-18.2.2-952.g738c2fd6.el9.x86_6 1.3 MB/s | 1.4 MB 00:01 2024-03-24T00:33:11.681 INFO:teuthology.orchestra.run.smithi038.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 635 kB/s | 47 kB 00:00 2024-03-24T00:33:11.706 INFO:teuthology.orchestra.run.smithi038.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 790 kB/s | 39 kB 00:00 2024-03-24T00:33:11.740 INFO:teuthology.orchestra.run.smithi038.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-24T00:33:11.807 INFO:teuthology.orchestra.run.smithi038.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2024-03-24T00:33:11.832 INFO:teuthology.orchestra.run.smithi038.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-24T00:33:11.874 INFO:teuthology.orchestra.run.smithi038.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 3.0 MB/s | 124 kB 00:00 2024-03-24T00:33:11.908 INFO:teuthology.orchestra.run.smithi038.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.5 MB/s | 187 kB 00:00 2024-03-24T00:33:11.983 INFO:teuthology.orchestra.run.smithi038.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 1.7 MB/s | 125 kB 00:00 2024-03-24T00:33:12.190 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:33:12.258 INFO:teuthology.orchestra.run.smithi038.stdout:(16/33): ceph-osd-18.2.2-952.g738c2fd6.el9.x86_ 13 MB/s | 16 MB 00:01 2024-03-24T00:33:12.264 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-24T00:33:12.313 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-24T00:33:12.357 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-24T00:33:12.408 INFO:teuthology.orchestra.run.smithi038.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 87 kB/s | 43 kB 00:00 2024-03-24T00:33:12.506 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-24T00:33:12.542 INFO:teuthology.orchestra.run.smithi038.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-03-24T00:33:12.559 INFO:teuthology.orchestra.run.smithi038.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 72 kB/s | 11 kB 00:00 2024-03-24T00:33:12.592 INFO:teuthology.orchestra.run.smithi038.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 317 kB/s | 172 kB 00:00 2024-03-24T00:33:12.599 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-24T00:33:12.609 INFO:teuthology.orchestra.run.smithi038.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 269 kB/s | 18 kB 00:00 2024-03-24T00:33:12.626 INFO:teuthology.orchestra.run.smithi038.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 349 kB/s | 23 kB 00:00 2024-03-24T00:33:12.654 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-24T00:33:12.668 INFO:teuthology.orchestra.run.smithi038.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 261 kB/s | 19 kB 00:00 2024-03-24T00:33:12.685 INFO:teuthology.orchestra.run.smithi038.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 264 kB/s | 20 kB 00:00 2024-03-24T00:33:12.702 INFO:teuthology.orchestra.run.smithi038.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 618 kB/s | 46 kB 00:00 2024-03-24T00:33:12.744 INFO:teuthology.orchestra.run.smithi038.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-03-24T00:33:12.747 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-24T00:33:12.777 INFO:teuthology.orchestra.run.smithi038.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-03-24T00:33:12.795 INFO:teuthology.orchestra.run.smithi038.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 179 kB/s | 16 kB 00:00 2024-03-24T00:33:12.808 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-24T00:33:12.820 INFO:teuthology.orchestra.run.smithi038.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-03-24T00:33:12.836 INFO:teuthology.orchestra.run.smithi038.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 618 kB/s | 36 kB 00:00 2024-03-24T00:33:12.850 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-24T00:33:12.887 INFO:teuthology.orchestra.run.smithi038.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-03-24T00:33:12.915 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-24T00:33:12.920 INFO:teuthology.orchestra.run.smithi038.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-03-24T00:33:12.945 INFO:teuthology.orchestra.run.smithi038.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 185 kB/s | 20 kB 00:00 2024-03-24T00:33:12.956 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:12.957 INFO:teuthology.orchestra.run.smithi038.stdout:Total 9.6 MB/s | 29 MB 00:03 2024-03-24T00:33:12.987 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-24T00:33:12.998 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:33:13.092 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:33:13.092 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:33:13.270 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-24T00:33:13.330 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-24T00:33:13.366 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-24T00:33:13.410 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-24T00:33:13.448 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-24T00:33:13.572 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:33:13.573 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:33:13.580 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-24T00:33:13.624 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-24T00:33:13.659 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-24T00:33:13.709 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-24T00:33:13.764 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-24T00:33:13.804 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-24T00:33:13.945 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-24T00:33:14.008 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-24T00:33:14.120 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-24T00:33:14.309 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-24T00:33:14.469 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-24T00:33:14.658 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-952.g738c2fd6.el9.n 28/33 2024-03-24T00:33:14.684 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-2:18.2.2-952.g738c2fd6.el9.x86_64 29/33 2024-03-24T00:33:14.754 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-2:18.2.2-952.g738c2fd6.el9.x86_64 29/33 2024-03-24T00:33:14.754 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:33:14.754 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-24T00:33:14.754 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T00:33:14.754 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T00:33:14.755 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:14.943 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:33:15.008 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-24T00:33:15.055 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-24T00:33:15.107 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-24T00:33:15.254 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-24T00:33:15.355 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-24T00:33:15.415 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-24T00:33:15.511 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-24T00:33:15.577 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-24T00:33:15.618 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-24T00:33:15.686 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-24T00:33:15.758 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-24T00:33:16.008 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-24T00:33:16.092 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-24T00:33:16.135 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-24T00:33:16.181 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-24T00:33:16.218 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-24T00:33:16.293 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-osd-2:18.2.2-952.g738c2fd6.el9.x86_64 30/33 2024-03-24T00:33:16.334 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-24T00:33:16.366 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-osd-2:18.2.2-952.g738c2fd6.el9.x86_64 30/33 2024-03-24T00:33:16.367 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:33:16.367 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-24T00:33:16.367 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T00:33:16.367 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T00:33:16.367 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:16.368 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-24T00:33:16.403 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-24T00:33:16.451 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-24T00:33:16.507 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-24T00:33:16.549 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-24T00:33:16.676 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-24T00:33:16.727 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-24T00:33:16.848 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-24T00:33:16.852 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mon-2:18.2.2-952.g738c2fd6.el9.x86_64 31/33 2024-03-24T00:33:16.921 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mon-2:18.2.2-952.g738c2fd6.el9.x86_64 31/33 2024-03-24T00:33:16.921 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:33:16.921 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-24T00:33:16.921 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T00:33:16.921 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T00:33:16.922 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:17.041 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-24T00:33:17.152 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mds-2:18.2.2-952.g738c2fd6.el9.x86_64 32/33 2024-03-24T00:33:17.198 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-24T00:33:17.220 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mds-2:18.2.2-952.g738c2fd6.el9.x86_64 32/33 2024-03-24T00:33:17.220 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:33:17.220 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-24T00:33:17.220 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T00:33:17.221 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T00:33:17.221 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:17.284 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-2:18.2.2-952.g738c2fd6.el9.x86_64 33/33 2024-03-24T00:33:17.378 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-952.g738c2fd6.el9.n 28/33 2024-03-24T00:33:17.395 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-2:18.2.2-952.g738c2fd6.el9.x86_64 33/33 2024-03-24T00:33:17.395 INFO:teuthology.orchestra.run.smithi088.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-24T00:33:17.395 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:17.410 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:18.2.2-952.g738c2fd6.el9.x86_64 29/33 2024-03-24T00:33:17.487 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:18.2.2-952.g738c2fd6.el9.x86_64 29/33 2024-03-24T00:33:17.487 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:33:17.487 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-24T00:33:17.487 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T00:33:17.487 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T00:33:17.488 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:18.2.2-952.g738c2fd6.el9.x86_64 30/33 2024-03-24T00:33:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:18.2.2-952.g738c2fd6.el9.x86_64 30/33 2024-03-24T00:33:19.191 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:33:19.191 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-24T00:33:19.191 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T00:33:19.191 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T00:33:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:19.238 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-2:18.2.2-952.g738c2fd6.el9.x86_64 1/33 2024-03-24T00:33:19.238 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mds-2:18.2.2-952.g738c2fd6.el9.x86_64 2/33 2024-03-24T00:33:19.238 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-2:18.2.2-952.g738c2fd6.el9.x86_64 3/33 2024-03-24T00:33:19.238 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mon-2:18.2.2-952.g738c2fd6.el9.x86_64 4/33 2024-03-24T00:33:19.238 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-osd-2:18.2.2-952.g738c2fd6.el9.x86_64 5/33 2024-03-24T00:33:19.239 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-952.g738c2fd6.el9.n 6/33 2024-03-24T00:33:19.239 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-24T00:33:19.239 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-24T00:33:19.239 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-24T00:33:19.239 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-24T00:33:19.239 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-24T00:33:19.239 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-24T00:33:19.240 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-24T00:33:19.241 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-24T00:33:19.241 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-24T00:33:19.241 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-24T00:33:19.241 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-24T00:33:19.241 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-24T00:33:19.241 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-24T00:33:19.241 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-24T00:33:19.241 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-24T00:33:19.704 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:18.2.2-952.g738c2fd6.el9.x86_64 31/33 2024-03-24T00:33:19.774 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:18.2.2-952.g738c2fd6.el9.x86_64 31/33 2024-03-24T00:33:19.774 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:33:19.774 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-24T00:33:19.774 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T00:33:19.774 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T00:33:19.774 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:20.004 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:18.2.2-952.g738c2fd6.el9.x86_64 32/33 2024-03-24T00:33:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:18.2.2-952.g738c2fd6.el9.x86_64 32/33 2024-03-24T00:33:20.072 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:33:20.073 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-24T00:33:20.073 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T00:33:20.073 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T00:33:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:20.164 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:18.2.2-952.g738c2fd6.el9.x86_64 33/33 2024-03-24T00:33:20.289 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-2:18.2.2-952.g738c2fd6.el9.x86_64 33/33 2024-03-24T00:33:20.289 INFO:teuthology.orchestra.run.smithi038.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-24T00:33:20.289 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:20.386 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-24T00:33:20.386 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:20.386 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:33:20.387 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:20.387 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:20.387 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:20.387 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:20.387 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:20.387 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:20.387 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-24T00:33:20.387 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-24T00:33:20.387 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-24T00:33:20.387 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-24T00:33:20.387 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply-3.11-14.el9.noarch 2024-03-24T00:33:20.388 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:20.389 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:33:20.774 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-base 2024-03-24T00:33:21.595 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:59 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:33:21.697 INFO:teuthology.orchestra.run.smithi088.stdout:Package ceph-base-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:33:21.751 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:33:21.752 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-03-24T00:33:21.752 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:33:21.818 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install cephadm 2024-03-24T00:33:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:18.2.2-952.g738c2fd6.el9.x86_64 1/33 2024-03-24T00:33:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:18.2.2-952.g738c2fd6.el9.x86_64 2/33 2024-03-24T00:33:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:18.2.2-952.g738c2fd6.el9.x86_64 3/33 2024-03-24T00:33:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:18.2.2-952.g738c2fd6.el9.x86_64 4/33 2024-03-24T00:33:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:18.2.2-952.g738c2fd6.el9.x86_64 5/33 2024-03-24T00:33:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-952.g738c2fd6.el9.n 6/33 2024-03-24T00:33:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-24T00:33:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-24T00:33:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-24T00:33:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-24T00:33:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-24T00:33:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-24T00:33:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-24T00:33:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-24T00:33:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-24T00:33:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-24T00:33:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-24T00:33:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-24T00:33:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-24T00:33:22.638 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:00 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:33:22.787 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:33:22.787 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:22.788 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-03-24T00:33:22.788 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:22.788 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:33:22.788 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 224 k 2024-03-24T00:33:22.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:22.788 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:33:22.788 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:22.788 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-03-24T00:33:22.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:22.789 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 224 k 2024-03-24T00:33:22.789 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 215 k 2024-03-24T00:33:22.789 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:33:23.051 INFO:teuthology.orchestra.run.smithi088.stdout:cephadm-18.2.2-952.g738c2fd6.el9.noarch.rpm 857 kB/s | 224 kB 00:00 2024-03-24T00:33:23.052 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:23.053 INFO:teuthology.orchestra.run.smithi088.stdout:Total 847 kB/s | 224 kB 00:00 2024-03-24T00:33:23.053 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:33:23.070 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:33:23.070 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:33:23.079 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:33:23.080 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:33:23.203 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:33:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-24T00:33:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-24T00:33:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply-3.11-14.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-24T00:33:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-24T00:33:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-24T00:33:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-24T00:33:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-24T00:33:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-24T00:33:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-24T00:33:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-24T00:33:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:23.234 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:33:23.624 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-base 2024-03-24T00:33:23.824 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 1/1 2024-03-24T00:33:23.950 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 1/1 2024-03-24T00:33:24.434 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 1/1 2024-03-24T00:33:24.490 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:01 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:33:24.597 INFO:teuthology.orchestra.run.smithi038.stdout:Package ceph-base-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:33:24.658 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:33:24.658 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-03-24T00:33:24.658 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:33:24.726 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install cephadm 2024-03-24T00:33:24.759 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 1/1 2024-03-24T00:33:24.759 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:24.759 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:33:24.759 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:24.759 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:24.759 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:33:25.036 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-immutable-object-cache 2024-03-24T00:33:25.547 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:02 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:33:25.710 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:33:25.711 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:33:25.711 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-03-24T00:33:25.711 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:33:25.712 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:33:25.712 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 224 k 2024-03-24T00:33:25.712 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:25.712 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:33:25.712 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:33:25.712 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-03-24T00:33:25.712 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:25.712 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 224 k 2024-03-24T00:33:25.712 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 215 k 2024-03-24T00:33:25.712 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:33:25.897 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:03 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:33:25.944 INFO:teuthology.orchestra.run.smithi038.stdout:cephadm-18.2.2-952.g738c2fd6.el9.noarch.rpm 968 kB/s | 224 kB 00:00 2024-03-24T00:33:25.946 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:25.947 INFO:teuthology.orchestra.run.smithi038.stdout:Total 954 kB/s | 224 kB 00:00 2024-03-24T00:33:25.947 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:33:25.965 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:33:25.965 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:33:25.974 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:33:25.975 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:33:26.053 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:33:26.053 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:26.053 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repo Size 2024-03-24T00:33:26.054 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:26.054 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:33:26.054 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 143 k 2024-03-24T00:33:26.054 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:26.054 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:33:26.054 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:26.054 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-03-24T00:33:26.054 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:26.055 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 143 k 2024-03-24T00:33:26.055 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 395 k 2024-03-24T00:33:26.055 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:33:26.100 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:33:26.431 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-immutable-object-cache-18.2.2-952.g738c2fd 380 kB/s | 143 kB 00:00 2024-03-24T00:33:26.432 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:Total 377 kB/s | 143 kB 00:00 2024-03-24T00:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:33:26.459 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:33:26.459 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:33:26.554 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:33:26.555 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:33:26.758 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:33:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 1/1 2024-03-24T00:33:26.840 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-952.g738c2fd6.e 1/1 2024-03-24T00:33:26.909 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 1/1 2024-03-24T00:33:26.912 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-952.g738c2fd6.e 1/1 2024-03-24T00:33:26.912 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:33:26.912 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-24T00:33:26.912 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:27.449 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 1/1 2024-03-24T00:33:27.795 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 1/1 2024-03-24T00:33:27.795 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:27.795 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:33:27.795 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:27.795 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:27.795 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:33:28.032 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-immutable-object-cache 2024-03-24T00:33:28.176 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-952.g738c2fd6.e 1/1 2024-03-24T00:33:28.176 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:28.176 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:33:28.176 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:28.176 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:28.176 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:33:28.486 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr 2024-03-24T00:33:28.917 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:05 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:33:29.080 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:33:29.080 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repo Size 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 143 k 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 143 k 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 395 k 2024-03-24T00:33:29.081 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:33:29.362 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:07 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:33:29.449 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-immutable-object-cache-18.2.2-952.g738c2fd 388 kB/s | 143 kB 00:00 2024-03-24T00:33:29.451 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:29.452 INFO:teuthology.orchestra.run.smithi038.stdout:Total 385 kB/s | 143 kB 00:00 2024-03-24T00:33:29.452 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:33:29.461 INFO:teuthology.orchestra.run.smithi088.stdout:Package ceph-mgr-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:33:29.478 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:33:29.478 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:33:29.515 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:33:29.516 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-03-24T00:33:29.516 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:33:29.579 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-dashboard 2024-03-24T00:33:29.580 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:33:29.580 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:33:29.771 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:33:29.833 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-952.g738c2fd6.e 1/1 2024-03-24T00:33:29.913 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-952.g738c2fd6.e 1/1 2024-03-24T00:33:29.913 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:33:29.913 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-24T00:33:29.913 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:30.407 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:08 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:33:30.569 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:33:30.570 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:30.570 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-03-24T00:33:30.570 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:30.570 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:33:30.571 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 1.8 M 2024-03-24T00:33:30.571 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-03-24T00:33:30.571 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 23 k 2024-03-24T00:33:30.571 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 15 k 2024-03-24T00:33:30.571 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-24T00:33:30.571 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-24T00:33:30.571 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:30.571 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:33:30.571 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:30.571 INFO:teuthology.orchestra.run.smithi088.stdout:Install 5 Packages 2024-03-24T00:33:30.571 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:30.572 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 2.0 M 2024-03-24T00:33:30.572 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 8.6 M 2024-03-24T00:33:30.572 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:33:30.908 INFO:teuthology.orchestra.run.smithi088.stdout:(1/5): ceph-prometheus-alerts-18.2.2-952.g738c2 54 kB/s | 15 kB 00:00 2024-03-24T00:33:30.933 INFO:teuthology.orchestra.run.smithi088.stdout:(2/5): ceph-grafana-dashboards-18.2.2-952.g738c 77 kB/s | 23 kB 00:00 2024-03-24T00:33:31.142 INFO:teuthology.orchestra.run.smithi088.stdout:(3/5): ceph-mgr-dashboard-18.2.2-952.g738c2fd6. 3.5 MB/s | 1.8 MB 00:00 2024-03-24T00:33:31.217 INFO:teuthology.orchestra.run.smithi088.stdout:(4/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 100 kB/s | 31 kB 00:00 2024-03-24T00:33:31.263 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-952.g738c2fd6.e 1/1 2024-03-24T00:33:31.264 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:31.264 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:33:31.264 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:33:31.264 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:31.264 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:33:31.376 INFO:teuthology.orchestra.run.smithi088.stdout:(5/5): python3-routes-2.5.1-5.el9.noarch.rpm 428 kB/s | 188 kB 00:00 2024-03-24T00:33:31.384 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:31.385 INFO:teuthology.orchestra.run.smithi088.stdout:Total 2.5 MB/s | 2.0 MB 00:00 2024-03-24T00:33:31.385 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:33:31.408 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:33:31.408 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:33:31.528 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr 2024-03-24T00:33:31.567 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:33:31.569 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:33:31.908 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:33:32.034 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-03-24T00:33:32.074 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-03-24T00:33:32.119 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-952.g738c2fd6.el9.no 3/5 2024-03-24T00:33:32.389 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:09 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:33:32.434 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-952.g738c2fd6.el9.n 4/5 2024-03-24T00:33:32.470 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-952.g738c2fd6.el9.noarch 5/5 2024-03-24T00:33:32.494 INFO:teuthology.orchestra.run.smithi038.stdout:Package ceph-mgr-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:33:32.552 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:33:32.553 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-03-24T00:33:32.553 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:33:32.619 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-dashboard 2024-03-24T00:33:33.095 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-952.g738c2fd6.el9.noarch 5/5 2024-03-24T00:33:33.095 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-952.g738c2fd6.el9.n 1/5 2024-03-24T00:33:33.095 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-952.g738c2fd6.el9.noarch 2/5 2024-03-24T00:33:33.095 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-952.g738c2fd6.el9.no 3/5 2024-03-24T00:33:33.095 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-03-24T00:33:33.433 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-03-24T00:33:33.433 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:33.433 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:33:33.434 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:33.434 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:33.434 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:33.434 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-24T00:33:33.434 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-24T00:33:33.434 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:33.434 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:33:33.469 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:10 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:33:33.626 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:33:33.627 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:33:33.627 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-03-24T00:33:33.627 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:33:33.627 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:33:33.627 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 1.8 M 2024-03-24T00:33:33.628 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-03-24T00:33:33.628 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 23 k 2024-03-24T00:33:33.628 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 15 k 2024-03-24T00:33:33.628 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-24T00:33:33.628 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-24T00:33:33.628 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:33.628 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:33:33.628 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:33:33.628 INFO:teuthology.orchestra.run.smithi038.stdout:Install 5 Packages 2024-03-24T00:33:33.628 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:33.629 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 2.0 M 2024-03-24T00:33:33.629 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 8.6 M 2024-03-24T00:33:33.629 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:33:33.714 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-24T00:33:34.003 INFO:teuthology.orchestra.run.smithi038.stdout:(1/5): ceph-prometheus-alerts-18.2.2-952.g738c2 53 kB/s | 15 kB 00:00 2024-03-24T00:33:34.028 INFO:teuthology.orchestra.run.smithi038.stdout:(2/5): ceph-grafana-dashboards-18.2.2-952.g738c 76 kB/s | 23 kB 00:00 2024-03-24T00:33:34.245 INFO:teuthology.orchestra.run.smithi038.stdout:(3/5): ceph-mgr-dashboard-18.2.2-952.g738c2fd6. 3.4 MB/s | 1.8 MB 00:00 2024-03-24T00:33:34.353 INFO:teuthology.orchestra.run.smithi038.stdout:(4/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 88 kB/s | 31 kB 00:00 2024-03-24T00:33:34.553 INFO:teuthology.orchestra.run.smithi038.stdout:(5/5): python3-routes-2.5.1-5.el9.noarch.rpm 360 kB/s | 188 kB 00:00 2024-03-24T00:33:34.562 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:34.562 INFO:teuthology.orchestra.run.smithi038.stdout:Total 2.2 MB/s | 2.0 MB 00:00 2024-03-24T00:33:34.563 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:33:34.571 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:12 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:33:34.587 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:33:34.587 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:33:34.722 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:33:34.723 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:33:34.725 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:33:34.726 INFO:teuthology.orchestra.run.smithi088.stdout:=========================================================================================== 2024-03-24T00:33:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-03-24T00:33:34.726 INFO:teuthology.orchestra.run.smithi088.stdout:=========================================================================================== 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 7.4 M 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-24T00:33:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-24T00:33:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-24T00:33:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:34.728 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:33:34.728 INFO:teuthology.orchestra.run.smithi088.stdout:=========================================================================================== 2024-03-24T00:33:34.728 INFO:teuthology.orchestra.run.smithi088.stdout:Install 12 Packages 2024-03-24T00:33:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:34.728 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 37 M 2024-03-24T00:33:34.729 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 209 M 2024-03-24T00:33:34.729 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:33:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:33:35.140 INFO:teuthology.orchestra.run.smithi088.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 160 kB/s | 32 kB 00:00 2024-03-24T00:33:35.182 INFO:teuthology.orchestra.run.smithi088.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 450 kB/s | 17 kB 00:00 2024-03-24T00:33:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-03-24T00:33:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-03-24T00:33:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-952.g738c2fd6.el9.no 3/5 2024-03-24T00:33:35.281 INFO:teuthology.orchestra.run.smithi088.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.8 MB/s | 3.0 MB 00:00 2024-03-24T00:33:35.307 INFO:teuthology.orchestra.run.smithi088.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 313 kB/s | 34 kB 00:00 2024-03-24T00:33:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 25 MB/s | 4.7 MB 00:00 2024-03-24T00:33:35.591 INFO:teuthology.orchestra.run.smithi088.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-03-24T00:33:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-952.g738c2fd6.el9.n 4/5 2024-03-24T00:33:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-952.g738c2fd6.el9.noarch 5/5 2024-03-24T00:33:35.625 INFO:teuthology.orchestra.run.smithi088.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.5 MB/s | 217 kB 00:00 2024-03-24T00:33:35.933 INFO:teuthology.orchestra.run.smithi088.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-03-24T00:33:36.092 INFO:teuthology.orchestra.run.smithi088.stdout:(9/12): ceph-mgr-diskprediction-local-18.2.2-95 6.4 MB/s | 7.4 MB 00:01 2024-03-24T00:33:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-952.g738c2fd6.el9.noarch 5/5 2024-03-24T00:33:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-952.g738c2fd6.el9.n 1/5 2024-03-24T00:33:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-952.g738c2fd6.el9.noarch 2/5 2024-03-24T00:33:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-952.g738c2fd6.el9.no 3/5 2024-03-24T00:33:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-03-24T00:33:36.509 INFO:teuthology.orchestra.run.smithi088.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 924 kB/s | 803 kB 00:00 2024-03-24T00:33:36.534 INFO:teuthology.orchestra.run.smithi088.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 326 kB/s | 195 kB 00:00 2024-03-24T00:33:36.575 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-03-24T00:33:36.575 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:36.575 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:33:36.575 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:36.575 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:36.575 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:36.575 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-24T00:33:36.575 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-24T00:33:36.575 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:36.575 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:33:36.676 INFO:teuthology.orchestra.run.smithi088.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 421 kB/s | 245 kB 00:00 2024-03-24T00:33:36.681 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:36.681 INFO:teuthology.orchestra.run.smithi088.stdout:Total 19 MB/s | 37 MB 00:01 2024-03-24T00:33:36.796 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:33:36.861 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:33:36.861 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:33:36.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-24T00:33:37.393 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:33:37.393 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:33:37.736 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:14 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:33:37.896 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:33:37.897 INFO:teuthology.orchestra.run.smithi038.stdout:=========================================================================================== 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout:=========================================================================================== 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 7.4 M 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-24T00:33:37.898 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-24T00:33:37.899 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-24T00:33:37.899 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-24T00:33:37.899 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-24T00:33:37.899 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-24T00:33:37.899 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:37.899 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:33:37.899 INFO:teuthology.orchestra.run.smithi038.stdout:=========================================================================================== 2024-03-24T00:33:37.899 INFO:teuthology.orchestra.run.smithi038.stdout:Install 12 Packages 2024-03-24T00:33:37.899 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:37.900 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 37 M 2024-03-24T00:33:37.900 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 209 M 2024-03-24T00:33:37.900 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:33:38.239 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:33:38.324 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-24T00:33:38.358 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-24T00:33:38.460 INFO:teuthology.orchestra.run.smithi038.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 141 kB/s | 32 kB 00:00 2024-03-24T00:33:38.502 INFO:teuthology.orchestra.run.smithi038.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 464 kB/s | 17 kB 00:00 2024-03-24T00:33:38.505 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-24T00:33:38.543 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-24T00:33:38.594 INFO:teuthology.orchestra.run.smithi038.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.4 MB/s | 3.0 MB 00:00 2024-03-24T00:33:38.619 INFO:teuthology.orchestra.run.smithi038.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 339 kB/s | 34 kB 00:00 2024-03-24T00:33:38.793 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T00:33:38.819 INFO:teuthology.orchestra.run.smithi038.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2024-03-24T00:33:38.833 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T00:33:38.936 INFO:teuthology.orchestra.run.smithi038.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 4.8 MB 00:00 2024-03-24T00:33:38.957 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-24T00:33:38.970 INFO:teuthology.orchestra.run.smithi038.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.5 MB/s | 217 kB 00:00 2024-03-24T00:33:39.237 INFO:teuthology.orchestra.run.smithi038.stdout:(8/12): ceph-mgr-diskprediction-local-18.2.2-95 7.4 MB/s | 7.4 MB 00:01 2024-03-24T00:33:39.437 INFO:teuthology.orchestra.run.smithi038.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-03-24T00:33:39.562 INFO:teuthology.orchestra.run.smithi038.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-03-24T00:33:39.587 INFO:teuthology.orchestra.run.smithi038.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 563 kB/s | 195 kB 00:00 2024-03-24T00:33:39.654 INFO:teuthology.orchestra.run.smithi038.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-03-24T00:33:39.660 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:39.661 INFO:teuthology.orchestra.run.smithi038.stdout:Total 21 MB/s | 37 MB 00:01 2024-03-24T00:33:39.784 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:33:39.799 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-24T00:33:39.852 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:33:39.852 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:33:39.927 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-24T00:33:40.400 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:33:40.401 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:33:41.273 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:33:41.350 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-24T00:33:41.393 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-24T00:33:41.458 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-24T00:33:41.544 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-24T00:33:41.613 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-24T00:33:41.878 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T00:33:41.911 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T00:33:42.031 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-24T00:33:42.607 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-24T00:33:42.639 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-952.g738c2f 12/12 2024-03-24T00:33:42.903 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-24T00:33:43.034 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-24T00:33:44.656 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-24T00:33:45.147 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-952.g738c2f 12/12 2024-03-24T00:33:45.147 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-952.g738c2f 1/12 2024-03-24T00:33:45.148 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-24T00:33:45.148 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-24T00:33:45.148 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-24T00:33:45.148 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T00:33:45.148 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T00:33:45.148 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-24T00:33:45.148 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-24T00:33:45.148 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-24T00:33:45.148 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-24T00:33:45.149 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-24T00:33:45.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-24T00:33:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-24T00:33:45.623 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-24T00:33:45.623 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-24T00:33:45.623 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:45.623 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:33:45.860 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-24T00:33:45.893 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-952.g738c2f 12/12 2024-03-24T00:33:45.995 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-rook 2024-03-24T00:33:46.840 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:24 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:33:46.998 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:33:46.999 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-03-24T00:33:46.999 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-03-24T00:33:46.999 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-03-24T00:33:46.999 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:33:46.999 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 49 k 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-24T00:33:47.000 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-24T00:33:47.001 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:47.001 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:33:47.001 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-03-24T00:33:47.001 INFO:teuthology.orchestra.run.smithi088.stdout:Install 13 Packages 2024-03-24T00:33:47.001 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:47.001 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 2.0 M 2024-03-24T00:33:47.001 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 26 M 2024-03-24T00:33:47.001 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:33:47.289 INFO:teuthology.orchestra.run.smithi088.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 123 kB/s | 24 kB 00:00 2024-03-24T00:33:47.314 INFO:teuthology.orchestra.run.smithi088.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 81 kB/s | 18 kB 00:00 2024-03-24T00:33:47.365 INFO:teuthology.orchestra.run.smithi088.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-24T00:33:47.390 INFO:teuthology.orchestra.run.smithi088.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-24T00:33:47.424 INFO:teuthology.orchestra.run.smithi088.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 211 kB 00:00 2024-03-24T00:33:47.449 INFO:teuthology.orchestra.run.smithi088.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 763 kB/s | 43 kB 00:00 2024-03-24T00:33:47.474 INFO:teuthology.orchestra.run.smithi088.stdout:(7/13): ceph-mgr-rook-18.2.2-952.g738c2fd6.el9. 128 kB/s | 49 kB 00:00 2024-03-24T00:33:47.974 INFO:teuthology.orchestra.run.smithi088.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 29 kB/s | 15 kB 00:00 2024-03-24T00:33:47.999 INFO:teuthology.orchestra.run.smithi088.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 56 kB/s | 32 kB 00:00 2024-03-24T00:33:48.206 INFO:teuthology.orchestra.run.smithi088.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 277 kB/s | 203 kB 00:00 2024-03-24T00:33:48.258 INFO:teuthology.orchestra.run.smithi088.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 229 kB/s | 59 kB 00:00 2024-03-24T00:33:48.358 INFO:teuthology.orchestra.run.smithi088.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 590 kB/s | 90 kB 00:00 2024-03-24T00:33:48.543 INFO:teuthology.orchestra.run.smithi088.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.8 MB/s | 1.0 MB 00:00 2024-03-24T00:33:48.546 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:48.547 INFO:teuthology.orchestra.run.smithi088.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2024-03-24T00:33:48.548 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:33:48.589 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:33:48.589 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:33:48.689 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-952.g738c2f 12/12 2024-03-24T00:33:48.689 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-952.g738c2f 1/12 2024-03-24T00:33:48.689 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-24T00:33:48.689 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-24T00:33:48.690 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-24T00:33:48.690 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T00:33:48.690 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T00:33:48.690 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-24T00:33:48.690 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-24T00:33:48.690 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-24T00:33:48.690 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-24T00:33:48.690 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-24T00:33:48.742 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:33:48.742 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-24T00:33:49.124 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-24T00:33:49.125 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-24T00:33:49.125 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-24T00:33:49.125 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-24T00:33:49.125 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:49.125 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:33:49.420 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:33:49.505 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-rook 2024-03-24T00:33:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-24T00:33:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-24T00:33:49.801 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-24T00:33:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-24T00:33:49.885 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-24T00:33:50.010 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-24T00:33:50.158 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-24T00:33:50.220 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-24T00:33:50.378 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:27 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:33:50.545 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 49 k 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-24T00:33:50.547 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-24T00:33:50.548 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-24T00:33:50.548 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-24T00:33:50.548 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-24T00:33:50.548 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-24T00:33:50.548 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-24T00:33:50.548 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-24T00:33:50.548 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:50.548 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:33:50.548 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-03-24T00:33:50.548 INFO:teuthology.orchestra.run.smithi038.stdout:Install 13 Packages 2024-03-24T00:33:50.548 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:50.549 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 2.0 M 2024-03-24T00:33:50.549 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 26 M 2024-03-24T00:33:50.549 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:33:51.142 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-24T00:33:51.161 INFO:teuthology.orchestra.run.smithi038.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 105 kB/s | 24 kB 00:00 2024-03-24T00:33:51.178 INFO:teuthology.orchestra.run.smithi038.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 72 kB/s | 18 kB 00:00 2024-03-24T00:33:51.194 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-24T00:33:51.228 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-24T00:33:51.236 INFO:teuthology.orchestra.run.smithi038.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-24T00:33:51.261 INFO:teuthology.orchestra.run.smithi038.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-03-24T00:33:51.283 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-24T00:33:51.295 INFO:teuthology.orchestra.run.smithi038.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 211 kB 00:00 2024-03-24T00:33:51.313 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-rook-2:18.2.2-952.g738c2fd6.el9.noarch 13/13 2024-03-24T00:33:51.321 INFO:teuthology.orchestra.run.smithi038.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 756 kB/s | 43 kB 00:00 2024-03-24T00:33:51.346 INFO:teuthology.orchestra.run.smithi038.stdout:(7/13): ceph-mgr-rook-18.2.2-952.g738c2fd6.el9. 118 kB/s | 49 kB 00:00 2024-03-24T00:33:51.388 INFO:teuthology.orchestra.run.smithi038.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 227 kB/s | 15 kB 00:00 2024-03-24T00:33:51.413 INFO:teuthology.orchestra.run.smithi038.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 278 kB/s | 32 kB 00:00 2024-03-24T00:33:51.471 INFO:teuthology.orchestra.run.smithi038.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 1.6 MB/s | 203 kB 00:00 2024-03-24T00:33:51.497 INFO:teuthology.orchestra.run.smithi038.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 717 kB/s | 59 kB 00:00 2024-03-24T00:33:51.547 INFO:teuthology.orchestra.run.smithi038.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-03-24T00:33:51.589 INFO:teuthology.orchestra.run.smithi038.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.1 MB/s | 1.0 MB 00:00 2024-03-24T00:33:51.594 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:51.594 INFO:teuthology.orchestra.run.smithi038.stdout:Total 2.0 MB/s | 2.0 MB 00:01 2024-03-24T00:33:51.595 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:33:51.639 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:33:51.639 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:33:51.798 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:33:51.798 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:33:52.176 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-952.g738c2fd6.el9.noarch 13/13 2024-03-24T00:33:52.176 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-rook-2:18.2.2-952.g738c2fd6.el9.noarch 1/13 2024-03-24T00:33:52.176 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-24T00:33:52.176 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-24T00:33:52.176 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-24T00:33:52.176 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-24T00:33:52.176 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-24T00:33:52.177 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-24T00:33:52.177 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-24T00:33:52.177 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-24T00:33:52.177 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-24T00:33:52.177 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-24T00:33:52.177 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-24T00:33:52.493 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:33:52.656 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-24T00:33:52.657 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:52.657 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:33:52.657 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:52.657 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-24T00:33:52.657 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-24T00:33:52.657 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-24T00:33:52.657 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-24T00:33:52.657 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-24T00:33:52.657 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-24T00:33:52.658 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-24T00:33:52.658 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-24T00:33:52.658 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-24T00:33:52.658 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-24T00:33:52.658 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-24T00:33:52.658 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-24T00:33:52.659 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:52.659 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:33:52.719 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-24T00:33:52.791 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-24T00:33:52.858 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-24T00:33:52.895 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-24T00:33:52.977 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-24T00:33:52.984 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-cephadm 2024-03-24T00:33:53.111 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-24T00:33:53.256 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-24T00:33:53.317 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-24T00:33:53.813 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:31 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:33:53.974 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:33:53.975 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-03-24T00:33:53.975 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-03-24T00:33:53.975 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-03-24T00:33:53.975 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:33:53.975 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 138 k 2024-03-24T00:33:53.975 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-03-24T00:33:53.975 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-24T00:33:53.975 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-24T00:33:53.976 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-24T00:33:53.976 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-24T00:33:53.976 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-24T00:33:53.976 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:53.976 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:33:53.976 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-03-24T00:33:53.976 INFO:teuthology.orchestra.run.smithi088.stdout:Install 6 Packages 2024-03-24T00:33:53.976 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:53.977 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 6.8 M 2024-03-24T00:33:53.977 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 33 M 2024-03-24T00:33:53.977 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:33:54.308 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-24T00:33:54.367 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-24T00:33:54.400 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-24T00:33:54.452 INFO:teuthology.orchestra.run.smithi088.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-03-24T00:33:54.456 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-24T00:33:54.485 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:18.2.2-952.g738c2fd6.el9.noarch 13/13 2024-03-24T00:33:54.627 INFO:teuthology.orchestra.run.smithi088.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 5.8 MB 00:00 2024-03-24T00:33:54.727 INFO:teuthology.orchestra.run.smithi088.stdout:(3/6): ceph-mgr-cephadm-18.2.2-952.g738c2fd6.el 291 kB/s | 138 kB 00:00 2024-03-24T00:33:54.778 INFO:teuthology.orchestra.run.smithi088.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 385 kB/s | 58 kB 00:00 2024-03-24T00:33:54.895 INFO:teuthology.orchestra.run.smithi088.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.2 MB/s | 548 kB 00:00 2024-03-24T00:33:54.920 INFO:teuthology.orchestra.run.smithi088.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 268 kB/s | 51 kB 00:00 2024-03-24T00:33:54.922 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:54.923 INFO:teuthology.orchestra.run.smithi088.stdout:Total 7.2 MB/s | 6.8 MB 00:00 2024-03-24T00:33:54.963 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:33:54.991 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:33:54.991 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:33:55.114 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:33:55.115 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:33:55.439 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-952.g738c2fd6.el9.noarch 13/13 2024-03-24T00:33:55.440 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:18.2.2-952.g738c2fd6.el9.noarch 1/13 2024-03-24T00:33:55.440 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-24T00:33:55.440 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-24T00:33:55.441 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-24T00:33:55.441 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-24T00:33:55.441 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-24T00:33:55.441 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-24T00:33:55.441 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-24T00:33:55.441 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-24T00:33:55.441 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-24T00:33:55.441 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-24T00:33:55.441 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-24T00:33:55.507 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:33:55.715 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-24T00:33:55.786 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-24T00:33:55.998 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-24T00:33:55.998 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:55.998 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-24T00:33:55.999 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:56.000 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:33:56.306 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-cephadm 2024-03-24T00:33:56.488 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-24T00:33:56.596 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-24T00:33:56.684 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-24T00:33:56.716 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 6/6 2024-03-24T00:33:57.175 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:34 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:33:57.344 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:33:57.345 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-03-24T00:33:57.345 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-03-24T00:33:57.345 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-03-24T00:33:57.345 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:33:57.345 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch 2:18.2.2-952.g738c2fd6.el9 ceph-noarch 138 k 2024-03-24T00:33:57.345 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-03-24T00:33:57.345 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-24T00:33:57.345 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-24T00:33:57.345 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-24T00:33:57.345 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-24T00:33:57.346 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-24T00:33:57.346 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:57.346 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:33:57.346 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-03-24T00:33:57.346 INFO:teuthology.orchestra.run.smithi038.stdout:Install 6 Packages 2024-03-24T00:33:57.346 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:33:57.346 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 6.8 M 2024-03-24T00:33:57.347 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 33 M 2024-03-24T00:33:57.347 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:33:57.604 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 6/6 2024-03-24T00:33:57.604 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 1/6 2024-03-24T00:33:57.604 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-24T00:33:57.604 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-24T00:33:57.604 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-24T00:33:57.604 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-24T00:33:57.925 INFO:teuthology.orchestra.run.smithi038.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 900 kB/s | 248 kB 00:00 2024-03-24T00:33:57.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-24T00:33:57.958 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:57.958 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:33:57.958 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:33:57.958 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-24T00:33:57.958 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-24T00:33:57.958 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-24T00:33:57.959 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-24T00:33:57.959 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-24T00:33:57.959 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:57.959 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:33:58.017 INFO:teuthology.orchestra.run.smithi038.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-03-24T00:33:58.042 INFO:teuthology.orchestra.run.smithi038.stdout:(3/6): ceph-mgr-cephadm-18.2.2-952.g738c2fd6.el 351 kB/s | 138 kB 00:00 2024-03-24T00:33:58.159 INFO:teuthology.orchestra.run.smithi038.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 440 kB/s | 51 kB 00:00 2024-03-24T00:33:58.176 INFO:teuthology.orchestra.run.smithi038.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 366 kB/s | 58 kB 00:00 2024-03-24T00:33:58.243 INFO:teuthology.orchestra.run.smithi038.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.7 MB/s | 548 kB 00:00 2024-03-24T00:33:58.247 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:58.248 INFO:teuthology.orchestra.run.smithi038.stdout:Total 7.5 MB/s | 6.8 MB 00:00 2024-03-24T00:33:58.276 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-fuse 2024-03-24T00:33:58.290 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:33:58.326 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:33:58.326 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:33:58.469 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:33:58.470 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:33:58.866 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:33:59.082 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:37 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:33:59.094 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-24T00:33:59.151 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-24T00:33:59.234 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:33:59.235 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:59.235 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-03-24T00:33:59.235 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:59.235 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:33:59.235 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 844 k 2024-03-24T00:33:59.235 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-03-24T00:33:59.235 INFO:teuthology.orchestra.run.smithi088.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-24T00:33:59.235 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:59.235 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:33:59.235 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:33:59.235 INFO:teuthology.orchestra.run.smithi088.stdout:Install 2 Packages 2024-03-24T00:33:59.236 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:33:59.236 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 922 k 2024-03-24T00:33:59.236 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 2.7 M 2024-03-24T00:33:59.236 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:33:59.374 INFO:teuthology.orchestra.run.smithi088.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 577 kB/s | 79 kB 00:00 2024-03-24T00:33:59.599 INFO:teuthology.orchestra.run.smithi088.stdout:(2/2): ceph-fuse-18.2.2-952.g738c2fd6.el9.x86_6 2.3 MB/s | 844 kB 00:00 2024-03-24T00:33:59.602 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:33:59.603 INFO:teuthology.orchestra.run.smithi088.stdout:Total 2.5 MB/s | 922 kB 00:00 2024-03-24T00:33:59.603 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:33:59.671 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:33:59.672 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:33:59.765 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:33:59.766 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:33:59.901 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-24T00:33:59.998 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-24T00:34:00.027 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:34:00.087 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-24T00:34:00.123 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 6/6 2024-03-24T00:34:00.216 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-24T00:34:00.299 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 2/2 2024-03-24T00:34:00.980 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 6/6 2024-03-24T00:34:00.980 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 1/6 2024-03-24T00:34:00.980 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-24T00:34:00.980 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-24T00:34:00.980 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-24T00:34:00.980 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-24T00:34:01.243 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 2/2 2024-03-24T00:34:01.243 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 1/2 2024-03-24T00:34:01.328 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-24T00:34:01.328 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:01.328 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:34:01.328 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:18.2.2-952.g738c2fd6.el9.noarch 2024-03-24T00:34:01.328 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-24T00:34:01.328 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-24T00:34:01.329 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-24T00:34:01.329 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-24T00:34:01.329 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-24T00:34:01.329 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:01.329 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:01.619 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-fuse 2024-03-24T00:34:01.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-24T00:34:01.626 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:01.626 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:34:01.626 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-24T00:34:01.626 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:01.627 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:01.912 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install librados-devel 2024-03-24T00:34:02.484 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:39 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:02.643 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:02.644 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:02.644 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-03-24T00:34:02.644 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:02.645 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:34:02.645 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 844 k 2024-03-24T00:34:02.645 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-03-24T00:34:02.645 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-24T00:34:02.645 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:02.645 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:34:02.645 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:02.645 INFO:teuthology.orchestra.run.smithi038.stdout:Install 2 Packages 2024-03-24T00:34:02.645 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:02.646 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 922 k 2024-03-24T00:34:02.646 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 2.7 M 2024-03-24T00:34:02.646 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:34:02.785 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:40 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:02.805 INFO:teuthology.orchestra.run.smithi038.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 495 kB/s | 79 kB 00:00 2024-03-24T00:34:02.943 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:02.943 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:02.944 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repo Size 2024-03-24T00:34:02.944 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:02.944 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:34:02.944 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 127 k 2024-03-24T00:34:02.944 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:02.944 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:34:02.944 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:02.944 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-03-24T00:34:02.944 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:02.944 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 127 k 2024-03-24T00:34:02.944 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 456 k 2024-03-24T00:34:02.945 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:34:03.138 INFO:teuthology.orchestra.run.smithi038.stdout:(2/2): ceph-fuse-18.2.2-952.g738c2fd6.el9.x86_6 1.7 MB/s | 844 kB 00:00 2024-03-24T00:34:03.142 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:34:03.143 INFO:teuthology.orchestra.run.smithi038.stdout:Total 1.8 MB/s | 922 kB 00:00 2024-03-24T00:34:03.143 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:34:03.185 INFO:teuthology.orchestra.run.smithi088.stdout:librados-devel-18.2.2-952.g738c2fd6.el9.x86_64. 527 kB/s | 127 kB 00:00 2024-03-24T00:34:03.187 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:34:03.187 INFO:teuthology.orchestra.run.smithi088.stdout:Total 520 kB/s | 127 kB 00:00 2024-03-24T00:34:03.188 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:34:03.206 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:34:03.206 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:34:03.214 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:34:03.214 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:34:03.307 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:34:03.308 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:34:03.313 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:34:03.314 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:34:03.503 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:34:03.572 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:34:03.633 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librados-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:03.756 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-24T00:34:03.844 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 2/2 2024-03-24T00:34:04.163 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:04.496 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:04.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:04.496 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:34:04.497 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:34:04.497 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:04.497 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:04.824 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 2/2 2024-03-24T00:34:04.824 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 1/2 2024-03-24T00:34:04.857 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install libcephfs2 2024-03-24T00:34:05.178 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-24T00:34:05.178 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:05.178 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:34:05.178 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-24T00:34:05.178 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:05.178 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:05.519 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install librados-devel 2024-03-24T00:34:05.700 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:43 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:05.803 INFO:teuthology.orchestra.run.smithi088.stdout:Package libcephfs2-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:05.857 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:05.858 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-03-24T00:34:05.859 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:05.933 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install libcephfs-devel 2024-03-24T00:34:06.381 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:43 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:06.546 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:06.547 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:06.547 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repo Size 2024-03-24T00:34:06.547 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:06.547 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:34:06.547 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 127 k 2024-03-24T00:34:06.548 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:06.548 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:34:06.548 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:06.548 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-03-24T00:34:06.548 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:06.548 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 127 k 2024-03-24T00:34:06.548 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 456 k 2024-03-24T00:34:06.548 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:34:06.738 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:44 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:06.877 INFO:teuthology.orchestra.run.smithi038.stdout:librados-devel-18.2.2-952.g738c2fd6.el9.x86_64. 386 kB/s | 127 kB 00:00 2024-03-24T00:34:06.878 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:34:06.879 INFO:teuthology.orchestra.run.smithi038.stdout:Total 382 kB/s | 127 kB 00:00 2024-03-24T00:34:06.879 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:34:06.895 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:06.895 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:06.895 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repo Size 2024-03-24T00:34:06.895 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:06.896 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:34:06.896 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 31 k 2024-03-24T00:34:06.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:06.896 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:34:06.896 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:06.896 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-03-24T00:34:06.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:06.897 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 31 k 2024-03-24T00:34:06.897 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 139 k 2024-03-24T00:34:06.897 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:34:06.898 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:34:06.898 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:34:07.006 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:34:07.006 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:34:07.147 INFO:teuthology.orchestra.run.smithi088.stdout:libcephfs-devel-18.2.2-952.g738c2fd6.el9.x86_64 124 kB/s | 31 kB 00:00 2024-03-24T00:34:07.148 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:34:07.149 INFO:teuthology.orchestra.run.smithi088.stdout:Total 123 kB/s | 31 kB 00:00 2024-03-24T00:34:07.149 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:34:07.161 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:34:07.161 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:34:07.200 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:34:07.212 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:34:07.213 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:34:07.317 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:07.353 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:34:07.519 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:07.866 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:07.987 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:08.182 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:08.182 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:08.182 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:34:08.183 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:34:08.183 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:08.183 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:08.320 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:08.320 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:08.320 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:34:08.320 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:34:08.320 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:08.320 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:08.525 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install libcephfs2 2024-03-24T00:34:08.530 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install librados2 2024-03-24T00:34:09.351 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:47 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:09.389 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:46 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:09.446 INFO:teuthology.orchestra.run.smithi088.stdout:Package librados2-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:09.495 INFO:teuthology.orchestra.run.smithi038.stdout:Package libcephfs2-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:09.501 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:09.502 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-03-24T00:34:09.502 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:09.553 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:09.554 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-03-24T00:34:09.554 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:09.574 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install librbd1 2024-03-24T00:34:09.623 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install libcephfs-devel 2024-03-24T00:34:10.402 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:48 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:10.448 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:47 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:10.497 INFO:teuthology.orchestra.run.smithi088.stdout:Package librbd1-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:10.555 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:10.557 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-03-24T00:34:10.557 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:10.614 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repo Size 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 31 k 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 31 k 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 139 k 2024-03-24T00:34:10.615 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:34:10.625 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-rados 2024-03-24T00:34:10.863 INFO:teuthology.orchestra.run.smithi038.stdout:libcephfs-devel-18.2.2-952.g738c2fd6.el9.x86_64 126 kB/s | 31 kB 00:00 2024-03-24T00:34:10.865 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:34:10.866 INFO:teuthology.orchestra.run.smithi038.stdout:Total 124 kB/s | 31 kB 00:00 2024-03-24T00:34:10.866 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:34:10.878 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:34:10.878 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:34:10.937 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:34:10.938 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:34:11.090 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:34:11.255 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:11.414 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:49 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:11.517 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-rados-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:11.574 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:11.575 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-03-24T00:34:11.575 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:11.636 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-rgw 2024-03-24T00:34:11.711 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:12.003 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:12.003 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:12.003 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:34:12.003 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:34:12.003 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:12.003 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:12.213 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install librados2 2024-03-24T00:34:12.449 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:50 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:12.550 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-rgw-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:12.609 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:12.610 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-03-24T00:34:12.610 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:12.677 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-cephfs 2024-03-24T00:34:13.077 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:50 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:13.177 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:13.235 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:13.237 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-03-24T00:34:13.237 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:13.306 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install librbd1 2024-03-24T00:34:13.497 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:51 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:13.606 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-cephfs-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:13.664 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:13.665 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-03-24T00:34:13.665 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:13.738 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-rbd 2024-03-24T00:34:14.171 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:51 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:14.271 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:14.331 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:14.332 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-03-24T00:34:14.332 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:14.400 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-rados 2024-03-24T00:34:14.564 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:52 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:14.667 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-rbd-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:14.723 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:14.725 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-03-24T00:34:14.725 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:14.794 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install rbd-fuse 2024-03-24T00:34:15.226 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:52 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:15.332 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-rados-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:15.393 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:15.394 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-03-24T00:34:15.394 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:15.467 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-rgw 2024-03-24T00:34:15.587 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:53 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:15.738 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:15.739 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:15.739 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2024-03-24T00:34:15.739 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:15.739 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:34:15.739 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 87 k 2024-03-24T00:34:15.739 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:15.739 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:34:15.739 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:15.740 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-03-24T00:34:15.740 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:15.740 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 87 k 2024-03-24T00:34:15.740 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 227 k 2024-03-24T00:34:15.740 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:34:16.057 INFO:teuthology.orchestra.run.smithi088.stdout:rbd-fuse-18.2.2-952.g738c2fd6.el9.x86_64.rpm 273 kB/s | 87 kB 00:00 2024-03-24T00:34:16.059 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:34:16.060 INFO:teuthology.orchestra.run.smithi088.stdout:Total 271 kB/s | 87 kB 00:00 2024-03-24T00:34:16.060 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:34:16.084 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:34:16.084 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:34:16.184 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:34:16.184 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:34:16.291 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:53 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:16.365 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:34:16.397 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-rgw-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:16.458 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:16.459 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-03-24T00:34:16.459 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:16.496 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:16.522 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-cephfs 2024-03-24T00:34:17.048 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:17.364 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:54 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:17.383 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:17.383 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:17.383 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:34:17.383 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:34:17.383 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:17.383 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:17.473 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-cephfs-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:17.533 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:17.534 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-03-24T00:34:17.534 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:17.597 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-rbd 2024-03-24T00:34:17.661 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install rbd-mirror 2024-03-24T00:34:18.419 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:55 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:18.525 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-rbd-2:18.2.2-952.g738c2fd6.el9.x86_64 is already installed. 2024-03-24T00:34:18.529 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:56 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:18.583 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:18.584 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-03-24T00:34:18.585 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:18.645 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install rbd-fuse 2024-03-24T00:34:18.688 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:18.688 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:18.689 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repo Size 2024-03-24T00:34:18.689 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:18.689 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:34:18.689 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 3.0 M 2024-03-24T00:34:18.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:18.689 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:34:18.689 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:18.689 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-03-24T00:34:18.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:18.690 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 3.0 M 2024-03-24T00:34:18.690 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 12 M 2024-03-24T00:34:18.690 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:34:19.102 INFO:teuthology.orchestra.run.smithi088.stdout:rbd-mirror-18.2.2-952.g738c2fd6.el9.x86_64.rpm 7.3 MB/s | 3.0 MB 00:00 2024-03-24T00:34:19.104 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:34:19.105 INFO:teuthology.orchestra.run.smithi088.stdout:Total 7.2 MB/s | 3.0 MB 00:00 2024-03-24T00:34:19.105 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:34:19.135 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:34:19.135 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:34:19.246 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:34:19.246 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:34:19.498 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:56 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:19.657 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:19.658 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:19.658 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-03-24T00:34:19.658 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:19.658 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:34:19.658 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 87 k 2024-03-24T00:34:19.659 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:19.659 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:34:19.659 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:19.659 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-03-24T00:34:19.659 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:19.659 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 87 k 2024-03-24T00:34:19.659 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 227 k 2024-03-24T00:34:19.659 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:34:19.767 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:34:19.828 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-mirror-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:19.927 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-mirror-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:19.927 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:34:19.928 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-24T00:34:19.928 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T00:34:19.928 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T00:34:19.928 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:19.984 INFO:teuthology.orchestra.run.smithi038.stdout:rbd-fuse-18.2.2-952.g738c2fd6.el9.x86_64.rpm 267 kB/s | 87 kB 00:00 2024-03-24T00:34:19.985 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:34:19.986 INFO:teuthology.orchestra.run.smithi038.stdout:Total 264 kB/s | 87 kB 00:00 2024-03-24T00:34:19.986 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:34:20.010 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:34:20.011 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:34:20.087 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:34:20.088 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:34:20.266 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:34:20.398 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:20.903 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:21.246 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:21.247 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:21.247 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:34:21.247 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:34:21.247 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:21.247 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-mirror-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:21.318 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:34:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:34:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:21.318 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:21.558 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install rbd-mirror 2024-03-24T00:34:21.639 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install rbd-nbd 2024-03-24T00:34:22.454 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:59 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:22.480 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:02:00 ago on Sun 24 Mar 2024 12:32:22 AM UTC. 2024-03-24T00:34:22.619 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:22.620 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:22.620 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repo Size 2024-03-24T00:34:22.620 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:22.620 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:34:22.620 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 3.0 M 2024-03-24T00:34:22.621 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:22.621 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:34:22.621 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:22.621 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-03-24T00:34:22.621 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:22.621 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 3.0 M 2024-03-24T00:34:22.621 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 12 M 2024-03-24T00:34:22.621 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:34:22.631 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-03-24T00:34:22.632 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:22.632 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2024-03-24T00:34:22.632 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:22.632 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-03-24T00:34:22.633 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 172 k 2024-03-24T00:34:22.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:22.633 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-03-24T00:34:22.633 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-03-24T00:34:22.633 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-03-24T00:34:22.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:22.633 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 172 k 2024-03-24T00:34:22.633 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 494 k 2024-03-24T00:34:22.634 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-03-24T00:34:22.889 INFO:teuthology.orchestra.run.smithi088.stdout:rbd-nbd-18.2.2-952.g738c2fd6.el9.x86_64.rpm 675 kB/s | 172 kB 00:00 2024-03-24T00:34:22.890 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:34:22.891 INFO:teuthology.orchestra.run.smithi088.stdout:Total 667 kB/s | 172 kB 00:00 2024-03-24T00:34:22.891 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-03-24T00:34:22.917 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-03-24T00:34:22.918 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-03-24T00:34:23.016 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-03-24T00:34:23.017 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-03-24T00:34:23.186 INFO:teuthology.orchestra.run.smithi038.stdout:rbd-mirror-18.2.2-952.g738c2fd6.el9.x86_64.rpm 5.3 MB/s | 3.0 MB 00:00 2024-03-24T00:34:23.188 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:34:23.189 INFO:teuthology.orchestra.run.smithi038.stdout:Total 5.3 MB/s | 3.0 MB 00:00 2024-03-24T00:34:23.189 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:34:23.222 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:34:23.222 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:34:23.225 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-03-24T00:34:23.335 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:34:23.336 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:34:23.339 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-nbd-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:23.854 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:34:23.912 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:23.920 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-nbd-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:23.991 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:23.991 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T00:34:23.991 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-24T00:34:23.991 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T00:34:23.992 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T00:34:23.992 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:24.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-nbd-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:24.228 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:24.228 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-03-24T00:34:24.228 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:34:24.228 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:34:24.228 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-03-24T00:34:24.486 DEBUG:teuthology.parallel:result is None 2024-03-24T00:34:25.467 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:25.467 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:25.467 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:34:25.467 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:34:25.468 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:25.468 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:25.773 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install rbd-nbd 2024-03-24T00:34:26.646 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:03 ago on Sun 24 Mar 2024 12:32:23 AM UTC. 2024-03-24T00:34:26.807 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-03-24T00:34:26.807 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:26.808 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-03-24T00:34:26.808 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:26.808 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-03-24T00:34:26.808 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:18.2.2-952.g738c2fd6.el9 ceph 172 k 2024-03-24T00:34:26.808 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:26.808 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-03-24T00:34:26.808 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-03-24T00:34:26.808 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-03-24T00:34:26.808 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:26.809 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 172 k 2024-03-24T00:34:26.809 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 494 k 2024-03-24T00:34:26.809 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-03-24T00:34:27.164 INFO:teuthology.orchestra.run.smithi038.stdout:rbd-nbd-18.2.2-952.g738c2fd6.el9.x86_64.rpm 486 kB/s | 172 kB 00:00 2024-03-24T00:34:27.166 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:34:27.167 INFO:teuthology.orchestra.run.smithi038.stdout:Total 481 kB/s | 172 kB 00:00 2024-03-24T00:34:27.167 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-03-24T00:34:27.194 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-03-24T00:34:27.194 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-03-24T00:34:27.298 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-03-24T00:34:27.299 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-03-24T00:34:27.516 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-03-24T00:34:27.641 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:28.192 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-nbd-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:28.547 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:18.2.2-952.g738c2fd6.el9.x86_64 1/1 2024-03-24T00:34:28.547 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:28.547 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-03-24T00:34:28.548 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:18.2.2-952.g738c2fd6.el9.x86_64 2024-03-24T00:34:28.548 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:34:28.548 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-03-24T00:34:28.844 DEBUG:teuthology.parallel:result is None 2024-03-24T00:34:28.844 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T00:34:28.844 INFO:teuthology.packaging:ref: None 2024-03-24T00:34:28.845 INFO:teuthology.packaging:tag: None 2024-03-24T00:34:28.845 INFO:teuthology.packaging:branch: reef 2024-03-24T00:34:28.845 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:34:28.845 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-24T00:34:29.108 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-24T00:34:29.182 INFO:teuthology.orchestra.run.smithi038.stdout:18.2.2-952.g738c2fd6.el9 2024-03-24T00:34:29.186 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-952.g738c2fd6.el9 2024-03-24T00:34:29.186 INFO:teuthology.task.install:The correct ceph version 18.2.2-952.g738c2fd6 is installed. 2024-03-24T00:34:29.188 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T00:34:29.188 INFO:teuthology.packaging:ref: None 2024-03-24T00:34:29.188 INFO:teuthology.packaging:tag: None 2024-03-24T00:34:29.188 INFO:teuthology.packaging:branch: reef 2024-03-24T00:34:29.188 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:34:29.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-24T00:34:29.443 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-24T00:34:29.513 INFO:teuthology.orchestra.run.smithi088.stdout:18.2.2-952.g738c2fd6.el9 2024-03-24T00:34:29.513 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-952.g738c2fd6.el9 2024-03-24T00:34:29.513 INFO:teuthology.task.install:The correct ceph version 18.2.2-952.g738c2fd6 is installed. 2024-03-24T00:34:29.515 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-24T00:34:29.515 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:34:29.516 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-24T00:34:29.620 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:34:29.620 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-24T00:34:29.721 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-24T00:34:29.721 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:34:29.721 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-03-24T00:34:29.824 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-24T00:34:29.920 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:34:29.921 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/daemon-helper 2024-03-24T00:34:30.015 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-24T00:34:30.153 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-24T00:34:30.153 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:34:30.153 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-24T00:34:30.259 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-24T00:34:30.403 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:34:30.403 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-24T00:34:30.500 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-24T00:34:30.639 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-24T00:34:30.639 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:34:30.639 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-03-24T00:34:30.743 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-24T00:34:30.847 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:34:30.847 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/stdin-killer 2024-03-24T00:34:30.945 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-24T00:34:31.040 INFO:teuthology.run_tasks:Running task print... 2024-03-24T00:34:31.050 INFO:teuthology.task.print:**** done install task... 2024-03-24T00:34:31.050 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-24T00:34:31.152 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:reef', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-24T00:34:31.152 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:reef 2024-03-24T00:34:31.153 INFO:tasks.cephadm:Cluster fsid is 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:34:31.153 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-24T00:34:31.153 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-24T00:34:31.153 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi038': '172.21.15.38', 'mon.smithi088': '172.21.15.88'} 2024-03-24T00:34:31.153 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-24T00:34:31.153 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname $(hostname -s) 2024-03-24T00:34:31.263 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname $(hostname -s) 2024-03-24T00:34:31.366 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-03-24T00:34:31.366 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:34:31.583 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77877/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2145-g3be9a5c9', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2145.g3be9a5c9'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-22 05:51:42.267384', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-main-20240318.032620', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-24T00:34:31.843 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 738c2fd67dbc0b17e2e99568a03974712d998229 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-03-24T00:34:31.844 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/738c2fd67dbc0b17e2e99568a03974712d998229/centos/9/x86_64/flavors/default/cephadm 2024-03-24T00:34:31.845 DEBUG:teuthology.orchestra.run.smithi038:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/738c2fd67dbc0b17e2e99568a03974712d998229/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-24T00:34:32.344 INFO:teuthology.orchestra.run.smithi038.stdout:-rw-r--r--. 1 ubuntu ubuntu 215105 Mar 24 00:34 /home/ubuntu/cephtest/cephadm 2024-03-24T00:34:32.344 DEBUG:teuthology.orchestra.run.smithi088:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/738c2fd67dbc0b17e2e99568a03974712d998229/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-24T00:34:32.841 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r--. 1 ubuntu ubuntu 215105 Mar 24 00:34 /home/ubuntu/cephtest/cephadm 2024-03-24T00:34:32.842 DEBUG:teuthology.orchestra.run.smithi038:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-24T00:34:32.905 DEBUG:teuthology.orchestra.run.smithi088:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-24T00:34:33.617 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:reef on all hosts... 2024-03-24T00:34:33.617 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-03-24T00:34:33.625 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-03-24T00:34:34.757 INFO:teuthology.orchestra.run.smithi038.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-03-24T00:34:34.770 INFO:teuthology.orchestra.run.smithi088.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-03-24T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)", 2024-03-24T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "image_id": "68aff9aff3b2fe3c16548acc72564779ed9f377b279f3076ce38885b01df0d46", 2024-03-24T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "repo_digests": [ 2024-03-24T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d" 2024-03-24T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-03-24T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:35:24.984 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-03-24T00:35:24.984 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)", 2024-03-24T00:35:24.984 INFO:teuthology.orchestra.run.smithi088.stdout: "image_id": "68aff9aff3b2fe3c16548acc72564779ed9f377b279f3076ce38885b01df0d46", 2024-03-24T00:35:24.984 INFO:teuthology.orchestra.run.smithi088.stdout: "repo_digests": [ 2024-03-24T00:35:24.984 INFO:teuthology.orchestra.run.smithi088.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d" 2024-03-24T00:35:24.984 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-03-24T00:35:24.984 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-03-24T00:35:25.020 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph 2024-03-24T00:35:25.129 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /etc/ceph 2024-03-24T00:35:25.231 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /etc/ceph 2024-03-24T00:35:25.332 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 777 /etc/ceph 2024-03-24T00:35:25.430 INFO:tasks.cephadm:Writing seed config... 2024-03-24T00:35:25.431 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-24T00:35:25.431 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-24T00:35:25.431 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-24T00:35:25.431 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-24T00:35:25.431 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-24T00:35:25.432 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-24T00:35:25.433 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-24T00:35:25.434 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-24T00:35:25.434 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-24T00:35:25.434 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-24T00:35:25.434 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-24T00:35:25.434 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-24T00:35:25.434 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-24T00:35:25.435 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:35:25.435 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-24T00:35:25.495 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4796a0d6-e976-11ee-95cd-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-24T00:35:25.495 DEBUG:teuthology.orchestra.run.smithi038:mon.smithi038> sudo journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi038.service 2024-03-24T00:35:25.543 INFO:tasks.cephadm:Bootstrapping... 2024-03-24T00:35:25.543 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef -v bootstrap --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.38 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-24T00:35:25.791 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-03-24T00:35:25.791 INFO:teuthology.orchestra.run.smithi038.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:reef', '-v', 'bootstrap', '--fsid', '4796a0d6-e976-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.38', '--skip-admin-label'] 2024-03-24T00:35:25.848 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T00:35:25.848 INFO:teuthology.orchestra.run.smithi038.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-24T00:35:25.848 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying podman|docker is present... 2024-03-24T00:35:25.899 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T00:35:25.899 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying lvm2 is present... 2024-03-24T00:35:25.899 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying time synchronization is in place... 2024-03-24T00:35:25.915 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-24T00:35:25.915 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-24T00:35:25.930 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-24T00:35:25.931 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-03-24T00:35:25.950 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-03-24T00:35:25.972 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-03-24T00:35:25.972 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-03-24T00:35:25.973 INFO:teuthology.orchestra.run.smithi038.stdout:Repeating the final host check... 2024-03-24T00:35:26.037 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T00:35:26.037 INFO:teuthology.orchestra.run.smithi038.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-24T00:35:26.037 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl is present 2024-03-24T00:35:26.037 INFO:teuthology.orchestra.run.smithi038.stdout:lvcreate is present 2024-03-24T00:35:26.054 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-24T00:35:26.054 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-24T00:35:26.070 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-24T00:35:26.070 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-03-24T00:35:26.091 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-03-24T00:35:26.110 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-03-24T00:35:26.110 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-03-24T00:35:26.111 INFO:teuthology.orchestra.run.smithi038.stdout:Host looks OK 2024-03-24T00:35:26.111 INFO:teuthology.orchestra.run.smithi038.stdout:Cluster fsid: 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:35:26.112 INFO:teuthology.orchestra.run.smithi038.stdout:Acquiring lock 140320096571744 on /run/cephadm/4796a0d6-e976-11ee-95cd-87774f69a715.lock 2024-03-24T00:35:26.112 INFO:teuthology.orchestra.run.smithi038.stdout:Lock 140320096571744 acquired on /run/cephadm/4796a0d6-e976-11ee-95cd-87774f69a715.lock 2024-03-24T00:35:26.113 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 3300 ... 2024-03-24T00:35:26.114 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 6789 ... 2024-03-24T00:35:26.115 INFO:teuthology.orchestra.run.smithi038.stdout:Base mon IP(s) is [172.21.15.38:3300, 172.21.15.38:6789], mon addrv is [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-03-24T00:35:26.129 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.38 metric 100 2024-03-24T00:35:26.130 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.38 metric 100 2024-03-24T00:35:26.142 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-24T00:35:26.143 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-24T00:35:26.143 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-24T00:35:26.154 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-24T00:35:26.154 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-24T00:35:26.154 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-24T00:35:26.155 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-24T00:35:26.155 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-03-24T00:35:26.155 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-24T00:35:26.157 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-03-24T00:35:26.157 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-03-24T00:35:26.158 INFO:teuthology.orchestra.run.smithi038.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-24T00:35:26.160 INFO:teuthology.orchestra.run.smithi038.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-24T00:35:26.162 INFO:teuthology.orchestra.run.smithi038.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-03-24T00:35:26.827 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 68aff9aff3b2fe3c16548acc72564779ed9f377b279f3076ce38885b01df0d46 2024-03-24T00:35:26.827 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:reef... 2024-03-24T00:35:26.827 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Getting image source signatures 2024-03-24T00:35:26.827 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-24T00:35:26.827 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:8566210fcee75c6a55e6eeb2f06888d926f27e0de261f0bce516c1cbd24b1704 2024-03-24T00:35:26.828 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying config sha256:68aff9aff3b2fe3c16548acc72564779ed9f377b279f3076ce38885b01df0d46 2024-03-24T00:35:26.828 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-24T00:35:28.038 INFO:teuthology.orchestra.run.smithi038.stdout:ceph: stdout ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable) 2024-03-24T00:35:28.039 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph version: ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable) 2024-03-24T00:35:28.039 INFO:teuthology.orchestra.run.smithi038.stdout:Extracting ceph user uid/gid from container image... 2024-03-24T00:35:29.171 INFO:teuthology.orchestra.run.smithi038.stdout:stat: stdout 167 167 2024-03-24T00:35:29.171 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial keys... 2024-03-24T00:35:30.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQBRdf9l696PJRAAStD/t38qtEl3NmSAtlVI2g== 2024-03-24T00:35:31.373 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQBSdf9lcnnWLRAAeYFrZte7tSuhmKj9U3Fl8A== 2024-03-24T00:35:32.472 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQBTdf9ltW/HMhAAUjOWEtBW8lfYzMxH91KOSA== 2024-03-24T00:35:32.474 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial monmap... 2024-03-24T00:35:33.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-24T00:35:33.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-24T00:35:33.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:35:33.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-24T00:35:33.561 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool for smithi038 [v2:172.21.15.38:3300,v1:172.21.15.38:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-24T00:35:33.561 INFO:teuthology.orchestra.run.smithi038.stdout:setting min_mon_release = pacific 2024-03-24T00:35:33.561 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: set fsid to 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:35:33.562 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-24T00:35:33.562 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:35:33.562 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mon... 2024-03-24T00:35:34.665 INFO:teuthology.orchestra.run.smithi038.stdout:create mon.smithi038 on 2024-03-24T00:35:34.994 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-24T00:35:35.235 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-24T00:35:35.523 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4796a0d6-e976-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-4796a0d6-e976-11ee-95cd-87774f69a715.target. 2024-03-24T00:35:35.524 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4796a0d6-e976-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-4796a0d6-e976-11ee-95cd-87774f69a715.target. 2024-03-24T00:35:35.899 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi038 2024-03-24T00:35:35.899 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi038.service: Unit ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi038.service not loaded. 2024-03-24T00:35:36.224 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4796a0d6-e976-11ee-95cd-87774f69a715.target.wants/ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi038.service → /etc/systemd/system/ceph-4796a0d6-e976-11ee-95cd-87774f69a715@.service. 2024-03-24T00:35:36.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:36 smithi038 podman[28887]: 2024-03-24 00:35:36.632046038 +0000 UTC m=+0.148622181 container create 7da50bf243817ee381c2957e95370ad0ddb8ebddcc3ee422d4078884e4acc5b0 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=reef-738c2fd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-24T00:35:36.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:36 smithi038 podman[28887]: 2024-03-24 00:35:36.878126757 +0000 UTC m=+0.394702906 container init 7da50bf243817ee381c2957e95370ad0ddb8ebddcc3ee422d4078884e4acc5b0 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, RELEASE=reef-738c2fd, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-24T00:35:37.016 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T00:35:37.016 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-03-24T00:35:37.033 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T00:35:37.033 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-03-24T00:35:37.033 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-03-24T00:35:37.033 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-03-24T00:35:37.034 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon to start... 2024-03-24T00:35:37.034 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon... 2024-03-24T00:35:37.183 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:36 smithi038 podman[28887]: 2024-03-24 00:35:36.888322973 +0000 UTC m=+0.404899123 container start 7da50bf243817ee381c2957e95370ad0ddb8ebddcc3ee422d4078884e4acc5b0 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=reef-738c2fd) 2024-03-24T00:35:37.184 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:36 smithi038 ceph-mon[28905]: mkfs 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:35:37.184 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:36 smithi038 ceph-mon[28905]: mon.smithi038 is new leader, mons smithi038 in quorum (ranks 0) 2024-03-24T00:35:37.184 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:36 smithi038 bash[28887]: 7da50bf243817ee381c2957e95370ad0ddb8ebddcc3ee422d4078884e4acc5b0 2024-03-24T00:35:37.184 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:36 smithi038 systemd[1]: Started Ceph mon.smithi038 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:35:38.084 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:37 smithi038 ceph-mon[28905]: mon.smithi038 is new leader, mons smithi038 in quorum (ranks 0) 2024-03-24T00:35:38.084 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:37 smithi038 ceph-mon[28905]: monmap e1: 1 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T00:35:38.084 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:37 smithi038 ceph-mon[28905]: fsmap 2024-03-24T00:35:38.084 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:37 smithi038 ceph-mon[28905]: osdmap e1: 0 total, 0 up, 0 in 2024-03-24T00:35:38.084 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:37 smithi038 ceph-mon[28905]: mgrmap e1: no daemons active 2024-03-24T00:35:38.084 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:37 smithi038 ceph-mon[28905]: from='client.? 172.21.15.38:0/2508906409' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-24T00:35:38.360 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout cluster: 2024-03-24T00:35:38.360 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout id: 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:35:38.361 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-24T00:35:38.361 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:35:38.361 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout services: 2024-03-24T00:35:38.361 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi038 (age 0.764832s) 2024-03-24T00:35:38.361 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-24T00:35:38.361 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-24T00:35:38.361 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:35:38.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout data: 2024-03-24T00:35:38.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-24T00:35:38.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-24T00:35:38.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-24T00:35:38.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pgs: 2024-03-24T00:35:38.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:35:38.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.638+0000 7ff5f3600700 1 Processor -- start 2024-03-24T00:35:38.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.638+0000 7ff5f3600700 1 -- start start 2024-03-24T00:35:38.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.639+0000 7ff5f3600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1091a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:38.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.639+0000 7ff5f3600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec074580 con 0x7ff5ec106db0 2024-03-24T00:35:38.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.640+0000 7ff5f0c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:38.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.640+0000 7ff5f0c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1091a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35254/0 (socket says 172.21.15.38:35254) 2024-03-24T00:35:38.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.640+0000 7ff5f0c00700 1 -- 172.21.15.38:0/2441120155 learned_addr learned my addr 172.21.15.38:0/2441120155 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.640+0000 7ff5f0c00700 1 -- 172.21.15.38:0/2441120155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec1096e0 con 0x7ff5ec106db0 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.641+0000 7ff5f0c00700 1 --2- 172.21.15.38:0/2441120155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1091a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009e80 tx=0x7ff5e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fada7c5e50d02244 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.641+0000 7ff5eb400700 1 -- 172.21.15.38:0/2441120155 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5e000e070 con 0x7ff5ec106db0 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.641+0000 7ff5eb400700 1 -- 172.21.15.38:0/2441120155 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e0003a20 con 0x7ff5ec106db0 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.641+0000 7ff5eb400700 1 -- 172.21.15.38:0/2441120155 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5e0003bd0 con 0x7ff5ec106db0 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.642+0000 7ff5f3600700 1 -- 172.21.15.38:0/2441120155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 msgr2=0x7ff5ec1091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.642+0000 7ff5f3600700 1 --2- 172.21.15.38:0/2441120155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1091a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009e80 tx=0x7ff5e000b330 comp rx=0 tx=0).stop 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.642+0000 7ff5f3600700 1 -- 172.21.15.38:0/2441120155 shutdown_connections 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.642+0000 7ff5f3600700 1 --2- 172.21.15.38:0/2441120155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1091a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.642+0000 7ff5f3600700 1 -- 172.21.15.38:0/2441120155 >> 172.21.15.38:0/2441120155 conn(0x7ff5ec100800 msgr2=0x7ff5ec102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.643+0000 7ff5f3600700 1 -- 172.21.15.38:0/2441120155 shutdown_connections 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.643+0000 7ff5f3600700 1 -- 172.21.15.38:0/2441120155 wait complete. 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.644+0000 7ff5f3600700 1 Processor -- start 2024-03-24T00:35:38.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.644+0000 7ff5f3600700 1 -- start start 2024-03-24T00:35:38.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.644+0000 7ff5f3600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1065e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:38.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.644+0000 7ff5f3600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec106b20 con 0x7ff5ec106db0 2024-03-24T00:35:38.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.645+0000 7ff5f0c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:38.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.645+0000 7ff5f0c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1065e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35260/0 (socket says 172.21.15.38:35260) 2024-03-24T00:35:38.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.645+0000 7ff5f0c00700 1 -- 172.21.15.38:0/2508906409 learned_addr learned my addr 172.21.15.38:0/2508906409 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:38.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.645+0000 7ff5f0c00700 1 -- 172.21.15.38:0/2508906409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0009b30 con 0x7ff5ec106db0 2024-03-24T00:35:38.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.646+0000 7ff5f0c00700 1 --2- 172.21.15.38:0/2508906409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1065e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0739c0 tx=0x7ff5e0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:38.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.646+0000 7ff5e9600700 1 -- 172.21.15.38:0/2508906409 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5e0004240 con 0x7ff5ec106db0 2024-03-24T00:35:38.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.646+0000 7ff5e9600700 1 -- 172.21.15.38:0/2508906409 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e00043e0 con 0x7ff5ec106db0 2024-03-24T00:35:38.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.646+0000 7ff5f3600700 1 -- 172.21.15.38:0/2508906409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ec104d60 con 0x7ff5ec106db0 2024-03-24T00:35:38.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.646+0000 7ff5e9600700 1 -- 172.21.15.38:0/2508906409 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5e00046e0 con 0x7ff5ec106db0 2024-03-24T00:35:38.370 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.646+0000 7ff5f3600700 1 -- 172.21.15.38:0/2508906409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ec105200 con 0x7ff5ec106db0 2024-03-24T00:35:38.370 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.647+0000 7ff5e9600700 1 -- 172.21.15.38:0/2508906409 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff5e0019050 con 0x7ff5ec106db0 2024-03-24T00:35:38.370 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.647+0000 7ff5e9600700 1 -- 172.21.15.38:0/2508906409 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff5e0025790 con 0x7ff5ec106db0 2024-03-24T00:35:38.370 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.647+0000 7ff5f3600700 1 -- 172.21.15.38:0/2508906409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ec105c10 con 0x7ff5ec106db0 2024-03-24T00:35:38.370 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.651+0000 7ff5e9600700 1 -- 172.21.15.38:0/2508906409 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7ff5e002a060 con 0x7ff5ec106db0 2024-03-24T00:35:38.370 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.730+0000 7ff5f3600700 1 -- 172.21.15.38:0/2508906409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff5ec193020 con 0x7ff5ec106db0 2024-03-24T00:35:38.370 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.730+0000 7ff5e9600700 1 -- 172.21.15.38:0/2508906409 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7ff5e0020030 con 0x7ff5ec106db0 2024-03-24T00:35:38.370 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.732+0000 7ff5f3600700 1 -- 172.21.15.38:0/2508906409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 msgr2=0x7ff5ec1065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:38.370 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.732+0000 7ff5f3600700 1 --2- 172.21.15.38:0/2508906409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1065e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0739c0 tx=0x7ff5e0004ad0 comp rx=0 tx=0).stop 2024-03-24T00:35:38.370 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.732+0000 7ff5f3600700 1 -- 172.21.15.38:0/2508906409 shutdown_connections 2024-03-24T00:35:38.371 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.732+0000 7ff5f3600700 1 --2- 172.21.15.38:0/2508906409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5ec106db0 0x7ff5ec1065e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:38.371 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.732+0000 7ff5f3600700 1 -- 172.21.15.38:0/2508906409 >> 172.21.15.38:0/2508906409 conn(0x7ff5ec100800 msgr2=0x7ff5ec100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:38.371 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.733+0000 7ff5f3600700 1 -- 172.21.15.38:0/2508906409 shutdown_connections 2024-03-24T00:35:38.371 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:37.733+0000 7ff5f3600700 1 -- 172.21.15.38:0/2508906409 wait complete. 2024-03-24T00:35:38.371 INFO:teuthology.orchestra.run.smithi038.stdout:mon is available 2024-03-24T00:35:38.371 INFO:teuthology.orchestra.run.smithi038.stdout:Assimilating anything we can from ceph.conf... 2024-03-24T00:35:39.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:39 smithi038 ceph-mon[28905]: from='client.? 172.21.15.38:0/3740242115' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-24T00:35:39.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:39 smithi038 ceph-mon[28905]: from='client.? 172.21.15.38:0/3740242115' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-24T00:35:39.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:35:39.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-03-24T00:35:39.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:35:39.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-03-24T00:35:39.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-24T00:35:39.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-24T00:35:39.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-24T00:35:39.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-24T00:35:39.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:35:39.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-24T00:35:39.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-24T00:35:39.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.961+0000 7f0800200700 1 Processor -- start 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.961+0000 7f0800200700 1 -- start start 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.962+0000 7f0800200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f8109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.962+0000 7f0800200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f80744a0 con 0x7f07f8106e90 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.962+0000 7f07fd800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f8109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.962+0000 7f07fd800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f8109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35262/0 (socket says 172.21.15.38:35262) 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.962+0000 7f07fd800700 1 -- 172.21.15.38:0/2174752869 learned_addr learned my addr 172.21.15.38:0/2174752869 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.963+0000 7f07fd800700 1 -- 172.21.15.38:0/2174752869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f80745e0 con 0x7f07f8106e90 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.963+0000 7f07fd800700 1 --2- 172.21.15.38:0/2174752869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f8109280 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0098a0 tx=0x7f07ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de03e64e2bfe326a server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.963+0000 7f07f7e00700 1 -- 172.21.15.38:0/2174752869 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07ec004180 con 0x7f07f8106e90 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.963+0000 7f07f7e00700 1 -- 172.21.15.38:0/2174752869 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ec0042e0 con 0x7f07f8106e90 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.963+0000 7f07f7e00700 1 -- 172.21.15.38:0/2174752869 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07ec0036a0 con 0x7f07f8106e90 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.964+0000 7f0800200700 1 -- 172.21.15.38:0/2174752869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 msgr2=0x7f07f8109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.964+0000 7f0800200700 1 --2- 172.21.15.38:0/2174752869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f8109280 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0098a0 tx=0x7f07ec009bb0 comp rx=0 tx=0).stop 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.964+0000 7f0800200700 1 -- 172.21.15.38:0/2174752869 shutdown_connections 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.964+0000 7f0800200700 1 --2- 172.21.15.38:0/2174752869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f8109280 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.964+0000 7f0800200700 1 -- 172.21.15.38:0/2174752869 >> 172.21.15.38:0/2174752869 conn(0x7f07f8100800 msgr2=0x7f07f8102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.964+0000 7f0800200700 1 -- 172.21.15.38:0/2174752869 shutdown_connections 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.964+0000 7f0800200700 1 -- 172.21.15.38:0/2174752869 wait complete. 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.965+0000 7f0800200700 1 Processor -- start 2024-03-24T00:35:39.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.966+0000 7f0800200700 1 -- start start 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.966+0000 7f0800200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f81066c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.966+0000 7f0800200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8106c00 con 0x7f07f8106e90 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.966+0000 7f07fd800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f81066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.966+0000 7f07fd800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f81066c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35278/0 (socket says 172.21.15.38:35278) 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.966+0000 7f07fd800700 1 -- 172.21.15.38:0/3740242115 learned_addr learned my addr 172.21.15.38:0/3740242115 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.967+0000 7f07fd800700 1 -- 172.21.15.38:0/3740242115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec009550 con 0x7f07f8106e90 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.967+0000 7f07fd800700 1 --2- 172.21.15.38:0/3740242115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f81066c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f07f80738e0 tx=0x7f07ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.967+0000 7f07f6000700 1 -- 172.21.15.38:0/3740242115 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07ec003ce0 con 0x7f07f8106e90 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.967+0000 7f07f6000700 1 -- 172.21.15.38:0/3740242115 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ec020070 con 0x7f07f8106e90 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.967+0000 7f0800200700 1 -- 172.21.15.38:0/3740242115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f8104e40 con 0x7f07f8106e90 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.967+0000 7f07f6000700 1 -- 172.21.15.38:0/3740242115 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07ec011630 con 0x7f07f8106e90 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.967+0000 7f0800200700 1 -- 172.21.15.38:0/3740242115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f81052e0 con 0x7f07f8106e90 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.968+0000 7f07f6000700 1 -- 172.21.15.38:0/3740242115 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f07ec022030 con 0x7f07f8106e90 2024-03-24T00:35:39.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.968+0000 7f07f6000700 1 -- 172.21.15.38:0/3740242115 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f07ec025d90 con 0x7f07f8106e90 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.969+0000 7f0800200700 1 -- 172.21.15.38:0/3740242115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f80623e0 con 0x7f07f8106e90 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:38.971+0000 7f07f6000700 1 -- 172.21.15.38:0/3740242115 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f07ec025480 con 0x7f07f8106e90 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:39.051+0000 7f0800200700 1 -- 172.21.15.38:0/3740242115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f07f8193620 con 0x7f07f8106e90 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:39.057+0000 7f07f6000700 1 -- 172.21.15.38:0/3740242115 <== mon.0 v2:172.21.15.38:3300/0 7 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f07ec01b450 con 0x7f07f8106e90 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:39.057+0000 7f07f6000700 1 -- 172.21.15.38:0/3740242115 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f07ec01c450 con 0x7f07f8106e90 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:39.059+0000 7f0800200700 1 -- 172.21.15.38:0/3740242115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 msgr2=0x7f07f81066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:39.059+0000 7f0800200700 1 --2- 172.21.15.38:0/3740242115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f81066c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f07f80738e0 tx=0x7f07ec004750 comp rx=0 tx=0).stop 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:39.060+0000 7f0800200700 1 -- 172.21.15.38:0/3740242115 shutdown_connections 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:39.060+0000 7f0800200700 1 --2- 172.21.15.38:0/3740242115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07f8106e90 0x7f07f81066c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:39.060+0000 7f0800200700 1 -- 172.21.15.38:0/3740242115 >> 172.21.15.38:0/3740242115 conn(0x7f07f8100800 msgr2=0x7f07f8100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:39.060+0000 7f0800200700 1 -- 172.21.15.38:0/3740242115 shutdown_connections 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:39.060+0000 7f0800200700 1 -- 172.21.15.38:0/3740242115 wait complete. 2024-03-24T00:35:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:Generating new minimal ceph.conf... 2024-03-24T00:35:40.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:40 smithi038 ceph-mon[28905]: from='client.? 172.21.15.38:0/345195982' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:35:41.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.308+0000 7fcf16000700 1 Processor -- start 2024-03-24T00:35:41.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.308+0000 7fcf16000700 1 -- start start 2024-03-24T00:35:41.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.309+0000 7fcf16000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf101067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:41.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.309+0000 7fcf16000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10106da0 con 0x7fcf101063b0 2024-03-24T00:35:41.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.309+0000 7fcf0f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf101067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:41.041 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.310+0000 7fcf0f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf101067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35288/0 (socket says 172.21.15.38:35288) 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.310+0000 7fcf0f400700 1 -- 172.21.15.38:0/4166162696 learned_addr learned my addr 172.21.15.38:0/4166162696 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.310+0000 7fcf0f400700 1 -- 172.21.15.38:0/4166162696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf101075b0 con 0x7fcf101063b0 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.310+0000 7fcf0f400700 1 --2- 172.21.15.38:0/4166162696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf101067d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcf000098a0 tx=0x7fcf00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f041b84b1457553 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.310+0000 7fcf0ea00700 1 -- 172.21.15.38:0/4166162696 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf00004180 con 0x7fcf101063b0 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.311+0000 7fcf0ea00700 1 -- 172.21.15.38:0/4166162696 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcf000042e0 con 0x7fcf101063b0 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.311+0000 7fcf0ea00700 1 -- 172.21.15.38:0/4166162696 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf00011a70 con 0x7fcf101063b0 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.311+0000 7fcf16000700 1 -- 172.21.15.38:0/4166162696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 msgr2=0x7fcf101067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.311+0000 7fcf16000700 1 --2- 172.21.15.38:0/4166162696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf101067d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcf000098a0 tx=0x7fcf00009bb0 comp rx=0 tx=0).stop 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.311+0000 7fcf16000700 1 -- 172.21.15.38:0/4166162696 shutdown_connections 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.311+0000 7fcf16000700 1 --2- 172.21.15.38:0/4166162696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf101067d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.311+0000 7fcf16000700 1 -- 172.21.15.38:0/4166162696 >> 172.21.15.38:0/4166162696 conn(0x7fcf101019b0 msgr2=0x7fcf10103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.311+0000 7fcf16000700 1 -- 172.21.15.38:0/4166162696 shutdown_connections 2024-03-24T00:35:41.042 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.311+0000 7fcf16000700 1 -- 172.21.15.38:0/4166162696 wait complete. 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.312+0000 7fcf16000700 1 Processor -- start 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.313+0000 7fcf16000700 1 -- start start 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.313+0000 7fcf16000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf1007a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.313+0000 7fcf16000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1007aec0 con 0x7fcf101063b0 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.313+0000 7fcf0f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf1007a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.313+0000 7fcf0f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf1007a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35292/0 (socket says 172.21.15.38:35292) 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.313+0000 7fcf0f400700 1 -- 172.21.15.38:0/345195982 learned_addr learned my addr 172.21.15.38:0/345195982 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.314+0000 7fcf0f400700 1 -- 172.21.15.38:0/345195982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00009550 con 0x7fcf101063b0 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.314+0000 7fcf0f400700 1 --2- 172.21.15.38:0/345195982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf1007a980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf00006af0 tx=0x7fcf000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.314+0000 7fcf0cc00700 1 -- 172.21.15.38:0/345195982 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf00003a00 con 0x7fcf101063b0 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.314+0000 7fcf0cc00700 1 -- 172.21.15.38:0/345195982 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcf00003b60 con 0x7fcf101063b0 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.314+0000 7fcf0cc00700 1 -- 172.21.15.38:0/345195982 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf00023750 con 0x7fcf101063b0 2024-03-24T00:35:41.043 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.314+0000 7fcf16000700 1 -- 172.21.15.38:0/345195982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf10079100 con 0x7fcf101063b0 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.315+0000 7fcf16000700 1 -- 172.21.15.38:0/345195982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf100795a0 con 0x7fcf101063b0 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.315+0000 7fcf0cc00700 1 -- 172.21.15.38:0/345195982 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcf00023c90 con 0x7fcf101063b0 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.315+0000 7fcf0cc00700 1 -- 172.21.15.38:0/345195982 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcf00019710 con 0x7fcf101063b0 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.316+0000 7fcf16000700 1 -- 172.21.15.38:0/345195982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf100623e0 con 0x7fcf101063b0 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.318+0000 7fcf0cc00700 1 -- 172.21.15.38:0/345195982 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fcf0002ce40 con 0x7fcf101063b0 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.397+0000 7fcf16000700 1 -- 172.21.15.38:0/345195982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fcf10079f20 con 0x7fcf101063b0 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.397+0000 7fcf0cc00700 1 -- 172.21.15.38:0/345195982 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fcf00005a20 con 0x7fcf101063b0 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.399+0000 7fcf16000700 1 -- 172.21.15.38:0/345195982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 msgr2=0x7fcf1007a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.399+0000 7fcf16000700 1 --2- 172.21.15.38:0/345195982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf1007a980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf00006af0 tx=0x7fcf000047c0 comp rx=0 tx=0).stop 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.399+0000 7fcf16000700 1 -- 172.21.15.38:0/345195982 shutdown_connections 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.399+0000 7fcf16000700 1 --2- 172.21.15.38:0/345195982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf101063b0 0x7fcf1007a980 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.399+0000 7fcf16000700 1 -- 172.21.15.38:0/345195982 >> 172.21.15.38:0/345195982 conn(0x7fcf101019b0 msgr2=0x7fcf10102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:41.044 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.399+0000 7fcf16000700 1 -- 172.21.15.38:0/345195982 shutdown_connections 2024-03-24T00:35:41.045 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:40.400+0000 7fcf16000700 1 -- 172.21.15.38:0/345195982 wait complete. 2024-03-24T00:35:41.045 INFO:teuthology.orchestra.run.smithi038.stdout:Restarting the monitor... 2024-03-24T00:35:41.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:41 smithi038 systemd[1]: Stopping Ceph mon.smithi038 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:35:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:41 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[28901]: 2024-03-24T00:35:41.538+0000 7fee76e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi038 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T00:35:41.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:41 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[28901]: 2024-03-24T00:35:41.538+0000 7fee76e00700 -1 mon.smithi038@0(leader) e1 *** Got Signal Terminated *** 2024-03-24T00:35:41.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:41 smithi038 podman[29146]: 2024-03-24 00:35:41.663652039 +0000 UTC m=+0.248481426 container stop 7da50bf243817ee381c2957e95370ad0ddb8ebddcc3ee422d4078884e4acc5b0 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=reef-738c2fd, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-24T00:35:41.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:41 smithi038 podman[29146]: 2024-03-24 00:35:41.691628431 +0000 UTC m=+0.276457796 container died 7da50bf243817ee381c2957e95370ad0ddb8ebddcc3ee422d4078884e4acc5b0 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-738c2fd, org.label-schema.build-date=20240319, ceph=True) 2024-03-24T00:35:42.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:42 smithi038 podman[29146]: 2024-03-24 00:35:42.073011806 +0000 UTC m=+0.657841214 container cleanup 7da50bf243817ee381c2957e95370ad0ddb8ebddcc3ee422d4078884e4acc5b0 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-738c2fd, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-24T00:35:42.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:42 smithi038 bash[29146]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038 2024-03-24T00:35:42.661 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:42 smithi038 podman[29160]: 2024-03-24 00:35:42.365370563 +0000 UTC m=+0.682139344 container remove 7da50bf243817ee381c2957e95370ad0ddb8ebddcc3ee422d4078884e4acc5b0 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=reef-738c2fd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-24T00:35:43.079 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:42 smithi038 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi038.service: Deactivated successfully. 2024-03-24T00:35:43.080 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:42 smithi038 systemd[1]: Stopped Ceph mon.smithi038 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:35:43.080 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:42 smithi038 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi038.service: Consumed 1.947s CPU time. 2024-03-24T00:35:43.080 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:42 smithi038 systemd[1]: Starting Ceph mon.smithi038 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:35:43.331 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 podman[29253]: 2024-03-24 00:35:42.98724541 +0000 UTC m=+0.035340353 image pull 68aff9aff3b2fe3c16548acc72564779ed9f377b279f3076ce38885b01df0d46 quay.ceph.io/ceph-ci/ceph:reef 2024-03-24T00:35:43.331 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 podman[29253]: 2024-03-24 00:35:43.091377889 +0000 UTC m=+0.139472869 container create da6fd287d5c70413a7a6c4a6a5b88ada5e5bfbdd2979586ca60ba69901ecfb71 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-738c2fd, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T00:35:43.331 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 podman[29253]: 2024-03-24 00:35:43.319994604 +0000 UTC m=+0.368089654 container init da6fd287d5c70413a7a6c4a6a5b88ada5e5bfbdd2979586ca60ba69901ecfb71 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, RELEASE=reef-738c2fd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-24T00:35:43.331 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 podman[29253]: 2024-03-24 00:35:43.330800922 +0000 UTC m=+0.378895909 container start da6fd287d5c70413a7a6c4a6a5b88ada5e5bfbdd2979586ca60ba69901ecfb71 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=reef-738c2fd) 2024-03-24T00:35:43.412 INFO:teuthology.orchestra.run.smithi038.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable), process ceph-mon, pid 2 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: pidfile_write: ignore empty --pid-file 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: load: jerasure load: lrc 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: RocksDB version: 7.9.2 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Git sha 0 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Compile date 2024-03-23 03:05:26 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: DB SUMMARY 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: DB Session ID: TZQSOPGM0PZXSUWQWN2J 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: CURRENT file: CURRENT 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T00:35:43.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi038/store.db dir, Total Num: 1, files: 000008.sst 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi038/store.db: 000009.log size: 97586 ; 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.error_if_exists: 0 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.create_if_missing: 0 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.paranoid_checks: 1 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.env: 0x55ad0d874720 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.info_log: 0x55ad10dcb3c0 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.statistics: (nil) 2024-03-24T00:35:43.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.use_fsync: 0 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_log_file_size: 0 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.allow_fallocate: 1 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.use_direct_reads: 0 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.db_log_dir: 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.wal_dir: 2024-03-24T00:35:43.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T00:35:43.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T00:35:43.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T00:35:43.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T00:35:43.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T00:35:43.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.write_buffer_manager: 0x55ad10044140 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.unordered_write: 0 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T00:35:43.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.row_cache: None 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.wal_filter: None 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.two_write_queues: 0 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.wal_compression: 0 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.atomic_flush: 0 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T00:35:43.616 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.log_readahead_size: 0 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_background_jobs: 2 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_background_compactions: -1 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_subcompactions: 1 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T00:35:43.617 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_open_files: -1 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_background_flushes: -1 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Compression algorithms supported: 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T00:35:43.618 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: kZSTD supported: 0 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: kXpressCompression supported: 0 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: kZlibCompression supported: 1 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: kSnappyCompression supported: 1 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: kLZ4Compression supported: 1 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: kBZip2Compression supported: 0 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi038/store.db/MANIFEST-000010 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.merge_operator: 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_filter: None 2024-03-24T00:35:43.619 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ad10dcb4e0) 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: cache_index_and_filter_blocks: 1 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: pin_top_level_index_and_filter: 1 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: index_type: 0 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: data_block_index_type: 0 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: index_shortening: 1 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: checksum: 4 2024-03-24T00:35:43.620 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: no_block_cache: 0 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_cache: 0x55ad100e51f0 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_cache_name: BinnedLRUCache 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_cache_options: 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: capacity : 536870912 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: num_shard_bits : 4 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: strict_capacity_limit : 0 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: high_pri_pool_ratio: 0.000 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_cache_compressed: (nil) 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: persistent_cache: (nil) 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_size: 4096 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_size_deviation: 10 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_restart_interval: 16 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: index_block_restart_interval: 1 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: metadata_block_size: 4096 2024-03-24T00:35:43.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: partition_filters: 0 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: use_delta_encoding: 1 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: filter_policy: bloomfilter 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: whole_key_filtering: 1 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: verify_compression: 0 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: read_amp_bytes_per_bit: 0 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: format_version: 5 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: enable_index_compression: 1 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_align: 0 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: max_auto_readahead_size: 262144 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: prepopulate_block_cache: 0 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: initial_auto_readahead_size: 8192 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T00:35:43.622 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compression: NoCompression 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.num_levels: 7 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T00:35:43.623 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T00:35:43.624 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T00:35:43.624 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T00:35:43.624 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T00:35:43.624 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T00:35:43.625 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T00:35:43.626 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.table_properties_collectors: 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.inplace_update_support: 0 2024-03-24T00:35:43.627 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.bloom_locality: 0 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.max_successive_merges: 0 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.ttl: 2592000 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.enable_blob_files: false 2024-03-24T00:35:43.628 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.min_blob_size: 0 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi038/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f3a5330f-515b-487e-976f-10635dfa202b 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711240543389113, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-24T00:35:43.629 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711240543390992, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92936, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 310, "table_properties": {"data_size": 90953, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83668, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 269, "num_filter_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711240543, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f3a5330f-515b-487e-976f-10635dfa202b", "db_session_id": "TZQSOPGM0PZXSUWQWN2J", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711240543391148, "job": 1, "event": "recovery_finished"} 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi038/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ad10182000 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: DB pointer 0x55ad10170000 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: ** DB Stats ** 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T00:35:43.630 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: ** Compaction Stats [default] ** 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: L0 2/0 92.63 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Sum 2/0 92.63 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: ** Compaction Stats [default] ** 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 76.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T00:35:43.631 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Cumulative compaction: 0.00 GB write, 13.02 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Interval compaction: 0.00 GB write, 13.02 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Block cache BinnedLRUCache@0x55ad100e51f0#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-24T00:35:43.632 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: starting mon.smithi038 rank 0 at public addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] at bind addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi038 fsid 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: mon.smithi038@-1(???) e1 preinit fsid 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: mon.smithi038@-1(???).mds e1 new map 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: mon.smithi038@-1(???).mds e1 print_map 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: e1 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: legacy client fscid: -1 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: No filesystems configured 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: mon.smithi038@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: mon.smithi038@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: mon.smithi038@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T00:35:43.633 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: mon.smithi038@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T00:35:43.634 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: mon.smithi038@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-24T00:35:43.634 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 bash[29253]: da6fd287d5c70413a7a6c4a6a5b88ada5e5bfbdd2979586ca60ba69901ecfb71 2024-03-24T00:35:43.634 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 systemd[1]: Started Ceph mon.smithi038 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:35:43.634 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: mon.smithi038 is new leader, mons smithi038 in quorum (ranks 0) 2024-03-24T00:35:43.634 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: monmap e1: 1 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T00:35:43.634 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: fsmap 2024-03-24T00:35:43.634 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: osdmap e1: 0 total, 0 up, 0 in 2024-03-24T00:35:43.634 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:43 smithi038 ceph-mon[29271]: mgrmap e1: no daemons active 2024-03-24T00:35:44.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.008+0000 7f0cf5800700 1 Processor -- start 2024-03-24T00:35:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.008+0000 7f0cf5800700 1 -- start start 2024-03-24T00:35:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.009+0000 7f0cf5800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf010e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.009+0000 7f0cf5800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf010ea00 con 0x7f0cf010dfd0 2024-03-24T00:35:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.009+0000 7f0ceea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf010e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.010+0000 7f0ceea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf010e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49644/0 (socket says 172.21.15.38:49644) 2024-03-24T00:35:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.010+0000 7f0ceea00700 1 -- 172.21.15.38:0/2332926217 learned_addr learned my addr 172.21.15.38:0/2332926217 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.010+0000 7f0ceea00700 1 -- 172.21.15.38:0/2332926217 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf010cc40 con 0x7f0cf010dfd0 2024-03-24T00:35:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.011+0000 7f0ceea00700 1 --2- 172.21.15.38:0/2332926217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf010e3f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00098a0 tx=0x7f0ce0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e676b1025ae6ef1c server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.011+0000 7f0ced600700 1 -- 172.21.15.38:0/2332926217 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ce0004180 con 0x7f0cf010dfd0 2024-03-24T00:35:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.011+0000 7f0ced600700 1 -- 172.21.15.38:0/2332926217 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0ce00042e0 con 0x7f0cf010dfd0 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.011+0000 7f0ced600700 1 -- 172.21.15.38:0/2332926217 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ce0011a70 con 0x7f0cf010dfd0 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.012+0000 7f0cf5800700 1 -- 172.21.15.38:0/2332926217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 msgr2=0x7f0cf010e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.012+0000 7f0cf5800700 1 --2- 172.21.15.38:0/2332926217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf010e3f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00098a0 tx=0x7f0ce0009bb0 comp rx=0 tx=0).stop 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.012+0000 7f0cf5800700 1 -- 172.21.15.38:0/2332926217 shutdown_connections 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.012+0000 7f0cf5800700 1 --2- 172.21.15.38:0/2332926217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf010e3f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.012+0000 7f0cf5800700 1 -- 172.21.15.38:0/2332926217 >> 172.21.15.38:0/2332926217 conn(0x7f0cf007b940 msgr2=0x7f0cf007dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.012+0000 7f0cf5800700 1 -- 172.21.15.38:0/2332926217 shutdown_connections 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.012+0000 7f0cf5800700 1 -- 172.21.15.38:0/2332926217 wait complete. 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.013+0000 7f0cf5800700 1 Processor -- start 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.013+0000 7f0cf5800700 1 -- start start 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.013+0000 7f0cf5800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.013+0000 7f0cf5800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf01a0be0 con 0x7f0cf010dfd0 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.014+0000 7f0ceea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:44.727 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.014+0000 7f0ceea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf01a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49656/0 (socket says 172.21.15.38:49656) 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.014+0000 7f0ceea00700 1 -- 172.21.15.38:0/638815794 learned_addr learned my addr 172.21.15.38:0/638815794 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.014+0000 7f0ceea00700 1 -- 172.21.15.38:0/638815794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce0009550 con 0x7f0cf010dfd0 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.014+0000 7f0ceea00700 1 --2- 172.21.15.38:0/638815794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf01a06a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0011710 tx=0x7f0ce00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.015+0000 7f0ce7400700 1 -- 172.21.15.38:0/638815794 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ce0003b10 con 0x7f0cf010dfd0 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.015+0000 7f0ce7400700 1 -- 172.21.15.38:0/638815794 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0ce001b8c0 con 0x7f0cf010dfd0 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.015+0000 7f0cf5800700 1 -- 172.21.15.38:0/638815794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf01a0de0 con 0x7f0cf010dfd0 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.015+0000 7f0ce7400700 1 -- 172.21.15.38:0/638815794 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ce00197e0 con 0x7f0cf010dfd0 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.015+0000 7f0cf5800700 1 -- 172.21.15.38:0/638815794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf01a1280 con 0x7f0cf010dfd0 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.015+0000 7f0ce7400700 1 -- 172.21.15.38:0/638815794 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0ce0019cd0 con 0x7f0cf010dfd0 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.016+0000 7f0ce7400700 1 -- 172.21.15.38:0/638815794 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0ce001a710 con 0x7f0cf010dfd0 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.016+0000 7f0cf5800700 1 -- 172.21.15.38:0/638815794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf00623e0 con 0x7f0cf010dfd0 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.020+0000 7f0ce7400700 1 -- 172.21.15.38:0/638815794 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f0ce0029060 con 0x7f0cf010dfd0 2024-03-24T00:35:44.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.095+0000 7f0cf5800700 1 -- 172.21.15.38:0/638815794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f0cf0110e80 con 0x7f0cf010dfd0 2024-03-24T00:35:44.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.101+0000 7f0ce7400700 1 -- 172.21.15.38:0/638815794 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0ce001f070 con 0x7f0cf010dfd0 2024-03-24T00:35:44.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.102+0000 7f0cf5800700 1 -- 172.21.15.38:0/638815794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 msgr2=0x7f0cf01a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:44.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.102+0000 7f0cf5800700 1 --2- 172.21.15.38:0/638815794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf01a06a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0011710 tx=0x7f0ce00047c0 comp rx=0 tx=0).stop 2024-03-24T00:35:44.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.102+0000 7f0cf5800700 1 -- 172.21.15.38:0/638815794 shutdown_connections 2024-03-24T00:35:44.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.102+0000 7f0cf5800700 1 --2- 172.21.15.38:0/638815794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cf010dfd0 0x7f0cf01a06a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:44.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.102+0000 7f0cf5800700 1 -- 172.21.15.38:0/638815794 >> 172.21.15.38:0/638815794 conn(0x7f0cf007b940 msgr2=0x7f0cf007c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:44.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.102+0000 7f0cf5800700 1 -- 172.21.15.38:0/638815794 shutdown_connections 2024-03-24T00:35:44.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:44.103+0000 7f0cf5800700 1 -- 172.21.15.38:0/638815794 wait complete. 2024-03-24T00:35:44.729 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-24T00:35:44.749 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-24T00:35:44.749 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mgr... 2024-03-24T00:35:44.750 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-24T00:35:44.751 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-24T00:35:44.751 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:8443 ... 2024-03-24T00:35:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mgr.smithi038.huhioz 2024-03-24T00:35:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mgr.smithi038.huhioz.service: Unit ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mgr.smithi038.huhioz.service not loaded. 2024-03-24T00:35:45.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:45 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/638815794' entity='client.admin' 2024-03-24T00:35:45.475 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4796a0d6-e976-11ee-95cd-87774f69a715.target.wants/ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mgr.smithi038.huhioz.service → /etc/systemd/system/ceph-4796a0d6-e976-11ee-95cd-87774f69a715@.service. 2024-03-24T00:35:53.812 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T00:35:53.813 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-03-24T00:35:53.837 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T00:35:53.837 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-03-24T00:35:53.837 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-03-24T00:35:53.837 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-03-24T00:35:53.864 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T00:35:53.864 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-03-24T00:35:53.885 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T00:35:53.885 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-03-24T00:35:53.885 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-03-24T00:35:53.885 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-03-24T00:35:53.886 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr to start... 2024-03-24T00:35:53.886 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr... 2024-03-24T00:35:54.993 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:54 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1493025945' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T00:35:55.274 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:35:55.275 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-03-24T00:35:55.275 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "4796a0d6-e976-11ee-95cd-87774f69a715", 2024-03-24T00:35:55.275 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T00:35:55.275 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T00:35:55.277 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T00:35:55.277 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T00:35:55.277 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "smithi038" 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:55.278 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T00:35:55.279 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:35:55.279 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T00:35:55.279 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T00:35:55.279 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T00:35:55.287 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T00:35:55.288 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T00:35:36.969618+0000", 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.564+0000 7f6044c00700 1 Processor -- start 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.564+0000 7f6044c00700 1 -- start start 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.564+0000 7f6044c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60400791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:55.289 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.564+0000 7f6044c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6040079770 con 0x7f604007ad40 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.565+0000 7f603de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60400791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.565+0000 7f603de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60400791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49576/0 (socket says 172.21.15.38:49576) 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.565+0000 7f603de00700 1 -- 172.21.15.38:0/206262805 learned_addr learned my addr 172.21.15.38:0/206262805 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.565+0000 7f603de00700 1 -- 172.21.15.38:0/206262805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400798b0 con 0x7f604007ad40 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.565+0000 7f603de00700 1 --2- 172.21.15.38:0/206262805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60400791a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f60300098a0 tx=0x7f6030009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec8e003f0b390db server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.566+0000 7f603ca00700 1 -- 172.21.15.38:0/206262805 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6030004180 con 0x7f604007ad40 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.566+0000 7f603ca00700 1 -- 172.21.15.38:0/206262805 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f60300042e0 con 0x7f604007ad40 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.566+0000 7f603ca00700 1 -- 172.21.15.38:0/206262805 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6030011a70 con 0x7f604007ad40 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.567+0000 7f6044c00700 1 -- 172.21.15.38:0/206262805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 msgr2=0x7f60400791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.567+0000 7f6044c00700 1 --2- 172.21.15.38:0/206262805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60400791a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f60300098a0 tx=0x7f6030009bb0 comp rx=0 tx=0).stop 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.567+0000 7f6044c00700 1 -- 172.21.15.38:0/206262805 shutdown_connections 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.567+0000 7f6044c00700 1 --2- 172.21.15.38:0/206262805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60400791a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:55.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.567+0000 7f6044c00700 1 -- 172.21.15.38:0/206262805 >> 172.21.15.38:0/206262805 conn(0x7f60401019b0 msgr2=0x7f6040103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.567+0000 7f6044c00700 1 -- 172.21.15.38:0/206262805 shutdown_connections 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.567+0000 7f6044c00700 1 -- 172.21.15.38:0/206262805 wait complete. 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.568+0000 7f6044c00700 1 Processor -- start 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.569+0000 7f6044c00700 1 -- start start 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.569+0000 7f6044c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60401a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.569+0000 7f6044c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60401a0cb0 con 0x7f604007ad40 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.569+0000 7f603de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60401a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.569+0000 7f603de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60401a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49584/0 (socket says 172.21.15.38:49584) 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.569+0000 7f603de00700 1 -- 172.21.15.38:0/1493025945 learned_addr learned my addr 172.21.15.38:0/1493025945 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.570+0000 7f603de00700 1 -- 172.21.15.38:0/1493025945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6030009550 con 0x7f604007ad40 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.570+0000 7f603de00700 1 --2- 172.21.15.38:0/1493025945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60401a0770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6030006af0 tx=0x7f60300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.570+0000 7f6036a00700 1 -- 172.21.15.38:0/1493025945 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6030003a00 con 0x7f604007ad40 2024-03-24T00:35:55.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.570+0000 7f6036a00700 1 -- 172.21.15.38:0/1493025945 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6030003b60 con 0x7f604007ad40 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.570+0000 7f6044c00700 1 -- 172.21.15.38:0/1493025945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60401a0eb0 con 0x7f604007ad40 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.571+0000 7f6036a00700 1 -- 172.21.15.38:0/1493025945 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6030023750 con 0x7f604007ad40 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.571+0000 7f6044c00700 1 -- 172.21.15.38:0/1493025945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60401a1350 con 0x7f604007ad40 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.571+0000 7f6036a00700 1 -- 172.21.15.38:0/1493025945 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6030023c40 con 0x7f604007ad40 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.571+0000 7f6036a00700 1 -- 172.21.15.38:0/1493025945 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6030019710 con 0x7f604007ad40 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.572+0000 7f6044c00700 1 -- 172.21.15.38:0/1493025945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60401082c0 con 0x7f604007ad40 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.577+0000 7f6036a00700 1 -- 172.21.15.38:0/1493025945 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f603001bbe0 con 0x7f604007ad40 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.656+0000 7f6044c00700 1 -- 172.21.15.38:0/1493025945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f60401084b0 con 0x7f604007ad40 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.657+0000 7f6036a00700 1 -- 172.21.15.38:0/1493025945 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f60300198d0 con 0x7f604007ad40 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.658+0000 7f6044c00700 1 -- 172.21.15.38:0/1493025945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 msgr2=0x7f60401a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.658+0000 7f6044c00700 1 --2- 172.21.15.38:0/1493025945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60401a0770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6030006af0 tx=0x7f60300047c0 comp rx=0 tx=0).stop 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.658+0000 7f6044c00700 1 -- 172.21.15.38:0/1493025945 shutdown_connections 2024-03-24T00:35:55.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.658+0000 7f6044c00700 1 --2- 172.21.15.38:0/1493025945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f604007ad40 0x7f60401a0770 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:55.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.658+0000 7f6044c00700 1 -- 172.21.15.38:0/1493025945 >> 172.21.15.38:0/1493025945 conn(0x7f60401019b0 msgr2=0x7f6040103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:55.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.659+0000 7f6044c00700 1 -- 172.21.15.38:0/1493025945 shutdown_connections 2024-03-24T00:35:55.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:54.659+0000 7f6044c00700 1 -- 172.21.15.38:0/1493025945 wait complete. 2024-03-24T00:35:55.293 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (1/15)... 2024-03-24T00:35:58.486 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:58 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/832077469' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T00:35:58.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:35:58.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-03-24T00:35:58.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "4796a0d6-e976-11ee-95cd-87774f69a715", 2024-03-24T00:35:58.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T00:35:58.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T00:35:58.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T00:35:58.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "smithi038" 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T00:35:58.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:58.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T00:35:58.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:35:58.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T00:35:58.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T00:35:58.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T00:35:58.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T00:35:58.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T00:35:58.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T00:35:58.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:58.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T00:35:58.851 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T00:35:58.851 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T00:35:36.969618+0000", 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.037+0000 7f00cde00700 1 Processor -- start 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.037+0000 7f00cde00700 1 -- start start 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.037+0000 7f00cde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c8109060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.037+0000 7f00cde00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c80744e0 con 0x7f00c8106c70 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.038+0000 7f00cca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c8109060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.038+0000 7f00cca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c8109060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49586/0 (socket says 172.21.15.38:49586) 2024-03-24T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.038+0000 7f00cca00700 1 -- 172.21.15.38:0/41222367 learned_addr learned my addr 172.21.15.38:0/41222367 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:58.858 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.038+0000 7f00cca00700 1 -- 172.21.15.38:0/41222367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c8074620 con 0x7f00c8106c70 2024-03-24T00:35:58.858 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.039+0000 7f00cca00700 1 --2- 172.21.15.38:0/41222367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c8109060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0098a0 tx=0x7f00bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df509aa64c735d2f server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:58.858 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.039+0000 7f00c7400700 1 -- 172.21.15.38:0/41222367 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00bc004180 con 0x7f00c8106c70 2024-03-24T00:35:58.858 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.039+0000 7f00c7400700 1 -- 172.21.15.38:0/41222367 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f00bc0042e0 con 0x7f00c8106c70 2024-03-24T00:35:58.858 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.039+0000 7f00c7400700 1 -- 172.21.15.38:0/41222367 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00bc011a70 con 0x7f00c8106c70 2024-03-24T00:35:58.858 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.040+0000 7f00cde00700 1 -- 172.21.15.38:0/41222367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 msgr2=0x7f00c8109060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:58.861 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.040+0000 7f00cde00700 1 --2- 172.21.15.38:0/41222367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c8109060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0098a0 tx=0x7f00bc009bb0 comp rx=0 tx=0).stop 2024-03-24T00:35:58.861 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.040+0000 7f00cde00700 1 -- 172.21.15.38:0/41222367 shutdown_connections 2024-03-24T00:35:58.861 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.040+0000 7f00cde00700 1 --2- 172.21.15.38:0/41222367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c8109060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:58.861 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.040+0000 7f00cde00700 1 -- 172.21.15.38:0/41222367 >> 172.21.15.38:0/41222367 conn(0x7f00c8100620 msgr2=0x7f00c8102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.040+0000 7f00cde00700 1 -- 172.21.15.38:0/41222367 shutdown_connections 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.040+0000 7f00cde00700 1 -- 172.21.15.38:0/41222367 wait complete. 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.041+0000 7f00cde00700 1 Processor -- start 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.041+0000 7f00cde00700 1 -- start start 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.041+0000 7f00cde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c819c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.041+0000 7f00cde00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c819c630 con 0x7f00c8106c70 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.042+0000 7f00cca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c819c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.042+0000 7f00cca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c819c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49598/0 (socket says 172.21.15.38:49598) 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.042+0000 7f00cca00700 1 -- 172.21.15.38:0/832077469 learned_addr learned my addr 172.21.15.38:0/832077469 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.042+0000 7f00cca00700 1 -- 172.21.15.38:0/832077469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc009550 con 0x7f00c8106c70 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.042+0000 7f00cca00700 1 --2- 172.21.15.38:0/832077469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c819c0f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00bc011710 tx=0x7f00bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.043+0000 7f00c5600700 1 -- 172.21.15.38:0/832077469 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00bc003b10 con 0x7f00c8106c70 2024-03-24T00:35:58.862 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.043+0000 7f00c5600700 1 -- 172.21.15.38:0/832077469 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f00bc01b8c0 con 0x7f00c8106c70 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.043+0000 7f00cde00700 1 -- 172.21.15.38:0/832077469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c819c830 con 0x7f00c8106c70 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.043+0000 7f00c5600700 1 -- 172.21.15.38:0/832077469 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00bc0197e0 con 0x7f00c8106c70 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.043+0000 7f00cde00700 1 -- 172.21.15.38:0/832077469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c819ccd0 con 0x7f00c8106c70 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.043+0000 7f00c5600700 1 -- 172.21.15.38:0/832077469 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f00bc019cd0 con 0x7f00c8106c70 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.043+0000 7f00c5600700 1 -- 172.21.15.38:0/832077469 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f00bc01a710 con 0x7f00c8106c70 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.044+0000 7f00cde00700 1 -- 172.21.15.38:0/832077469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c810a280 con 0x7f00c8106c70 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.047+0000 7f00c5600700 1 -- 172.21.15.38:0/832077469 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f00bc029060 con 0x7f00c8106c70 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.125+0000 7f00cde00700 1 -- 172.21.15.38:0/832077469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f00c810a470 con 0x7f00c8106c70 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.126+0000 7f00c5600700 1 -- 172.21.15.38:0/832077469 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f00bc01f070 con 0x7f00c8106c70 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.127+0000 7f00cde00700 1 -- 172.21.15.38:0/832077469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 msgr2=0x7f00c819c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.127+0000 7f00cde00700 1 --2- 172.21.15.38:0/832077469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c819c0f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00bc011710 tx=0x7f00bc0047c0 comp rx=0 tx=0).stop 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.127+0000 7f00cde00700 1 -- 172.21.15.38:0/832077469 shutdown_connections 2024-03-24T00:35:58.863 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.127+0000 7f00cde00700 1 --2- 172.21.15.38:0/832077469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00c8106c70 0x7f00c819c0f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:35:58.864 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.127+0000 7f00cde00700 1 -- 172.21.15.38:0/832077469 >> 172.21.15.38:0/832077469 conn(0x7f00c8100620 msgr2=0x7f00c81011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:35:58.864 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.127+0000 7f00cde00700 1 -- 172.21.15.38:0/832077469 shutdown_connections 2024-03-24T00:35:58.864 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:35:58.127+0000 7f00cde00700 1 -- 172.21.15.38:0/832077469 wait complete. 2024-03-24T00:35:58.864 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (2/15)... 2024-03-24T00:36:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: Activating manager daemon smithi038.huhioz 2024-03-24T00:36:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: mgrmap e2: smithi038.huhioz(active, starting, since 0.013574s) 2024-03-24T00:36:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: from='mgr.14100 172.21.15.38:0/2908039652' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T00:36:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: from='mgr.14100 172.21.15.38:0/2908039652' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T00:36:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: from='mgr.14100 172.21.15.38:0/2908039652' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T00:36:00.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: from='mgr.14100 172.21.15.38:0/2908039652' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:36:00.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: from='mgr.14100 172.21.15.38:0/2908039652' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi038.huhioz", "id": "smithi038.huhioz"}]: dispatch 2024-03-24T00:36:00.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: Manager daemon smithi038.huhioz is now available 2024-03-24T00:36:00.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: from='mgr.14100 172.21.15.38:0/2908039652' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:36:00.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: from='mgr.14100 172.21.15.38:0/2908039652' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:00.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: from='mgr.14100 172.21.15.38:0/2908039652' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/trash_purge_schedule"}]: dispatch 2024-03-24T00:36:00.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:35:59 smithi038 ceph-mon[29271]: from='mgr.14100 172.21.15.38:0/2908039652' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:01.138 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:00 smithi038 ceph-mon[29271]: from='mgr.14100 172.21.15.38:0/2908039652' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:01.138 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:00 smithi038 ceph-mon[29271]: mgrmap e3: smithi038.huhioz(active, since 1.02691s) 2024-03-24T00:36:02.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:01 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2455760905' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T00:36:02.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:36:02.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-03-24T00:36:02.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "4796a0d6-e976-11ee-95cd-87774f69a715", 2024-03-24T00:36:02.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T00:36:02.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T00:36:02.473 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "smithi038" 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:36:02.474 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-24T00:36:02.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T00:36:02.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:36:02.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T00:36:02.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:36:02.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T00:36:02.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T00:36:02.475 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T00:36:02.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T00:36:02.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:36:02.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T00:36:02.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T00:35:36.969618+0000", 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.477+0000 7fe4aa400700 1 Processor -- start 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.478+0000 7fe4aa400700 1 -- start start 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.478+0000 7fe4aa400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a410d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:02.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.478+0000 7fe4aa400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a410d940 con 0x7fe4a410cf30 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.478+0000 7fe4a9000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a410d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.479+0000 7fe4a9000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a410d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49676/0 (socket says 172.21.15.38:49676) 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.479+0000 7fe4a9000700 1 -- 172.21.15.38:0/604614514 learned_addr learned my addr 172.21.15.38:0/604614514 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.479+0000 7fe4a9000700 1 -- 172.21.15.38:0/604614514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a410da80 con 0x7fe4a410cf30 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.479+0000 7fe4a9000700 1 --2- 172.21.15.38:0/604614514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a410d350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4980098a0 tx=0x7fe498009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfa164c43b5926 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.480+0000 7fe4a3400700 1 -- 172.21.15.38:0/604614514 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe498004180 con 0x7fe4a410cf30 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.480+0000 7fe4a3400700 1 -- 172.21.15.38:0/604614514 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe4980042e0 con 0x7fe4a410cf30 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.480+0000 7fe4a3400700 1 -- 172.21.15.38:0/604614514 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe498011a70 con 0x7fe4a410cf30 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.480+0000 7fe4aa400700 1 -- 172.21.15.38:0/604614514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 msgr2=0x7fe4a410d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.480+0000 7fe4aa400700 1 --2- 172.21.15.38:0/604614514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a410d350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4980098a0 tx=0x7fe498009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.481+0000 7fe4aa400700 1 -- 172.21.15.38:0/604614514 shutdown_connections 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.481+0000 7fe4aa400700 1 --2- 172.21.15.38:0/604614514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a410d350 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.481+0000 7fe4aa400700 1 -- 172.21.15.38:0/604614514 >> 172.21.15.38:0/604614514 conn(0x7fe4a407b7c0 msgr2=0x7fe4a407dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:02.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.481+0000 7fe4aa400700 1 -- 172.21.15.38:0/604614514 shutdown_connections 2024-03-24T00:36:02.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.481+0000 7fe4aa400700 1 -- 172.21.15.38:0/604614514 wait complete. 2024-03-24T00:36:02.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.482+0000 7fe4aa400700 1 Processor -- start 2024-03-24T00:36:02.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.482+0000 7fe4aa400700 1 -- start start 2024-03-24T00:36:02.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.483+0000 7fe4aa400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a41a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:02.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.483+0000 7fe4aa400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a41a0a70 con 0x7fe4a410cf30 2024-03-24T00:36:02.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.483+0000 7fe4a9000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a41a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:02.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.483+0000 7fe4a9000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a41a0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49690/0 (socket says 172.21.15.38:49690) 2024-03-24T00:36:02.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.483+0000 7fe4a9000700 1 -- 172.21.15.38:0/2455760905 learned_addr learned my addr 172.21.15.38:0/2455760905 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:02.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.483+0000 7fe4a9000700 1 -- 172.21.15.38:0/2455760905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe498009550 con 0x7fe4a410cf30 2024-03-24T00:36:02.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.484+0000 7fe4a9000700 1 --2- 172.21.15.38:0/2455760905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a41a0530 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe498011710 tx=0x7fe4980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:02.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.484+0000 7fe4a1600700 1 -- 172.21.15.38:0/2455760905 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe498003b10 con 0x7fe4a410cf30 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.484+0000 7fe4a1600700 1 -- 172.21.15.38:0/2455760905 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe49801b8c0 con 0x7fe4a410cf30 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.484+0000 7fe4aa400700 1 -- 172.21.15.38:0/2455760905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a41a0c70 con 0x7fe4a410cf30 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.484+0000 7fe4a1600700 1 -- 172.21.15.38:0/2455760905 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe498019770 con 0x7fe4a410cf30 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.484+0000 7fe4aa400700 1 -- 172.21.15.38:0/2455760905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a41a1110 con 0x7fe4a410cf30 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.485+0000 7fe4aa400700 1 -- 172.21.15.38:0/2455760905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a4110990 con 0x7fe4a410cf30 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.486+0000 7fe4a1600700 1 -- 172.21.15.38:0/2455760905 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 3) v1 ==== 45431+0+0 (secure 0 0 0) 0x7fe49801b440 con 0x7fe4a410cf30 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.487+0000 7fe4a1600700 1 --2- 172.21.15.38:0/2455760905 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fe48c038650 0x7fe48c03ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.487+0000 7fe4a1600700 1 -- 172.21.15.38:0/2455760905 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe49804c6d0 con 0x7fe4a410cf30 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.487+0000 7fe4a3e00700 1 --2- 172.21.15.38:0/2455760905 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fe48c038650 0x7fe48c03ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.488+0000 7fe4a3e00700 1 --2- 172.21.15.38:0/2455760905 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fe48c038650 0x7fe48c03ab10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe490006fd0 tx=0x7fe490006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.492+0000 7fe4a1600700 1 -- 172.21.15.38:0/2455760905 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe49802cd70 con 0x7fe4a410cf30 2024-03-24T00:36:02.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.800+0000 7fe4aa400700 1 -- 172.21.15.38:0/2455760905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe4a41a3e20 con 0x7fe4a410cf30 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.802+0000 7fe4a1600700 1 -- 172.21.15.38:0/2455760905 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fe49801f030 con 0x7fe4a410cf30 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.804+0000 7fe4aa400700 1 -- 172.21.15.38:0/2455760905 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fe48c038650 msgr2=0x7fe48c03ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.804+0000 7fe4aa400700 1 --2- 172.21.15.38:0/2455760905 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fe48c038650 0x7fe48c03ab10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe490006fd0 tx=0x7fe490006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.804+0000 7fe4aa400700 1 -- 172.21.15.38:0/2455760905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 msgr2=0x7fe4a41a0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.804+0000 7fe4aa400700 1 --2- 172.21.15.38:0/2455760905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a41a0530 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe498011710 tx=0x7fe4980047c0 comp rx=0 tx=0).stop 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.805+0000 7fe4aa400700 1 -- 172.21.15.38:0/2455760905 shutdown_connections 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.805+0000 7fe4aa400700 1 --2- 172.21.15.38:0/2455760905 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fe48c038650 0x7fe48c03ab10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.805+0000 7fe4aa400700 1 --2- 172.21.15.38:0/2455760905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4a410cf30 0x7fe4a41a0530 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.805+0000 7fe4aa400700 1 -- 172.21.15.38:0/2455760905 >> 172.21.15.38:0/2455760905 conn(0x7fe4a407b7c0 msgr2=0x7fe4a407c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.805+0000 7fe4aa400700 1 -- 172.21.15.38:0/2455760905 shutdown_connections 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:01.805+0000 7fe4aa400700 1 -- 172.21.15.38:0/2455760905 wait complete. 2024-03-24T00:36:02.491 INFO:teuthology.orchestra.run.smithi038.stdout:mgr is available 2024-03-24T00:36:03.075 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:02 smithi038 ceph-mon[29271]: mgrmap e4: smithi038.huhioz(active, since 2s) 2024-03-24T00:36:03.994 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:03 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/4051967092' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-24T00:36:04.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:36:04.014 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-03-24T00:36:04.014 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:36:04.014 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-24T00:36:04.015 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-24T00:36:04.015 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-24T00:36:04.015 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-24T00:36:04.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:36:04.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-24T00:36:04.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-24T00:36:04.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-03-24T00:36:04.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-03-24T00:36:04.016 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-24T00:36:04.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-24T00:36:04.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-24T00:36:04.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-24T00:36:04.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-24T00:36:04.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.075+0000 7f076aa00700 1 Processor -- start 2024-03-24T00:36:04.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.075+0000 7f076aa00700 1 -- start start 2024-03-24T00:36:04.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.075+0000 7f076aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f0764106760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:04.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.075+0000 7f076aa00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0764106d30 con 0x7f0764106340 2024-03-24T00:36:04.018 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.076+0000 7f0763e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f0764106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:04.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.076+0000 7f0763e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f0764106760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46712/0 (socket says 172.21.15.38:46712) 2024-03-24T00:36:04.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.076+0000 7f0763e00700 1 -- 172.21.15.38:0/3846525003 learned_addr learned my addr 172.21.15.38:0/3846525003 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:04.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.076+0000 7f0763e00700 1 -- 172.21.15.38:0/3846525003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0764107540 con 0x7f0764106340 2024-03-24T00:36:04.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.077+0000 7f0763e00700 1 --2- 172.21.15.38:0/3846525003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f0764106760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f07540098a0 tx=0x7f0754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e6c7f3cf0a6d3b server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:04.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.077+0000 7f0762a00700 1 -- 172.21.15.38:0/3846525003 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0754004180 con 0x7f0764106340 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.077+0000 7f0762a00700 1 -- 172.21.15.38:0/3846525003 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f07540042e0 con 0x7f0764106340 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.077+0000 7f0762a00700 1 -- 172.21.15.38:0/3846525003 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0754011a70 con 0x7f0764106340 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.078+0000 7f076aa00700 1 -- 172.21.15.38:0/3846525003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 msgr2=0x7f0764106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.078+0000 7f076aa00700 1 --2- 172.21.15.38:0/3846525003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f0764106760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f07540098a0 tx=0x7f0754009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.078+0000 7f076aa00700 1 -- 172.21.15.38:0/3846525003 shutdown_connections 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.078+0000 7f076aa00700 1 --2- 172.21.15.38:0/3846525003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f0764106760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.078+0000 7f076aa00700 1 -- 172.21.15.38:0/3846525003 >> 172.21.15.38:0/3846525003 conn(0x7f07641018e0 msgr2=0x7f0764103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.078+0000 7f076aa00700 1 -- 172.21.15.38:0/3846525003 shutdown_connections 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.078+0000 7f076aa00700 1 -- 172.21.15.38:0/3846525003 wait complete. 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.079+0000 7f076aa00700 1 Processor -- start 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.079+0000 7f076aa00700 1 -- start start 2024-03-24T00:36:04.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.079+0000 7f076aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f076419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:04.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.079+0000 7f076aa00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076419c7f0 con 0x7f0764106340 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.080+0000 7f0763e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f076419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.080+0000 7f0763e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f076419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46718/0 (socket says 172.21.15.38:46718) 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.080+0000 7f0763e00700 1 -- 172.21.15.38:0/4051967092 learned_addr learned my addr 172.21.15.38:0/4051967092 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.080+0000 7f0763e00700 1 -- 172.21.15.38:0/4051967092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0754009550 con 0x7f0764106340 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.080+0000 7f0763e00700 1 --2- 172.21.15.38:0/4051967092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f076419c2b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0754011710 tx=0x7f07540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.080+0000 7f0760c00700 1 -- 172.21.15.38:0/4051967092 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0754003b10 con 0x7f0764106340 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.081+0000 7f0760c00700 1 -- 172.21.15.38:0/4051967092 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f075401b8c0 con 0x7f0764106340 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.081+0000 7f076aa00700 1 -- 172.21.15.38:0/4051967092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076419c9f0 con 0x7f0764106340 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.081+0000 7f0760c00700 1 -- 172.21.15.38:0/4051967092 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0754019770 con 0x7f0764106340 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.081+0000 7f076aa00700 1 -- 172.21.15.38:0/4051967092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076419ce90 con 0x7f0764106340 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.082+0000 7f0760c00700 1 -- 172.21.15.38:0/4051967092 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 45537+0+0 (secure 0 0 0) 0x7f075401ba30 con 0x7f0764106340 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.082+0000 7f076aa00700 1 -- 172.21.15.38:0/4051967092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076410a770 con 0x7f0764106340 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.082+0000 7f0760c00700 1 --2- 172.21.15.38:0/4051967092 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f07480386d0 0x7f074803ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.082+0000 7f0760c00700 1 -- 172.21.15.38:0/4051967092 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f075404c870 con 0x7f0764106340 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.083+0000 7f0763400700 1 --2- 172.21.15.38:0/4051967092 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f07480386d0 0x7f074803ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.083+0000 7f0763400700 1 --2- 172.21.15.38:0/4051967092 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f07480386d0 0x7f074803ab90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0750006fd0 tx=0x7f0750006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.088+0000 7f0760c00700 1 -- 172.21.15.38:0/4051967092 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f075402cce0 con 0x7f0764106340 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.308+0000 7f076aa00700 1 -- 172.21.15.38:0/4051967092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f076419fba0 con 0x7f0764106340 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.309+0000 7f0760c00700 1 -- 172.21.15.38:0/4051967092 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f075401f030 con 0x7f0764106340 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.313+0000 7f076aa00700 1 -- 172.21.15.38:0/4051967092 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f07480386d0 msgr2=0x7f074803ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.313+0000 7f076aa00700 1 --2- 172.21.15.38:0/4051967092 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f07480386d0 0x7f074803ab90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0750006fd0 tx=0x7f0750006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.313+0000 7f076aa00700 1 -- 172.21.15.38:0/4051967092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 msgr2=0x7f076419c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.313+0000 7f076aa00700 1 --2- 172.21.15.38:0/4051967092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f076419c2b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0754011710 tx=0x7f07540047c0 comp rx=0 tx=0).stop 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.313+0000 7f076aa00700 1 -- 172.21.15.38:0/4051967092 shutdown_connections 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.313+0000 7f076aa00700 1 --2- 172.21.15.38:0/4051967092 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f07480386d0 0x7f074803ab90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.313+0000 7f076aa00700 1 --2- 172.21.15.38:0/4051967092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0764106340 0x7f076419c2b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:04.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.313+0000 7f076aa00700 1 -- 172.21.15.38:0/4051967092 >> 172.21.15.38:0/4051967092 conn(0x7f07641018e0 msgr2=0x7f07641023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:04.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.313+0000 7f076aa00700 1 -- 172.21.15.38:0/4051967092 shutdown_connections 2024-03-24T00:36:04.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:03.313+0000 7f076aa00700 1 -- 172.21.15.38:0/4051967092 wait complete. 2024-03-24T00:36:04.027 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling cephadm module... 2024-03-24T00:36:05.278 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:04 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2507818072' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-24T00:36:05.578 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.579+0000 7fb94f000700 1 Processor -- start 2024-03-24T00:36:05.579 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.580+0000 7fb94f000700 1 -- start start 2024-03-24T00:36:05.579 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.580+0000 7fb94f000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb9480791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:05.579 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.580+0000 7fb94f000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb948079770 con 0x7fb94807ad40 2024-03-24T00:36:05.579 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.580+0000 7fb94dc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb9480791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:05.579 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.581+0000 7fb94dc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb9480791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46726/0 (socket says 172.21.15.38:46726) 2024-03-24T00:36:05.579 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.581+0000 7fb94dc00700 1 -- 172.21.15.38:0/2652945667 learned_addr learned my addr 172.21.15.38:0/2652945667 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:05.579 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.581+0000 7fb94dc00700 1 -- 172.21.15.38:0/2652945667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9480798b0 con 0x7fb94807ad40 2024-03-24T00:36:05.579 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.581+0000 7fb94dc00700 1 --2- 172.21.15.38:0/2652945667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb9480791a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0098a0 tx=0x7fb93c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3922bde54db0b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:05.579 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.582+0000 7fb947e00700 1 -- 172.21.15.38:0/2652945667 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb93c004180 con 0x7fb94807ad40 2024-03-24T00:36:05.579 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.582+0000 7fb947e00700 1 -- 172.21.15.38:0/2652945667 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb93c0042e0 con 0x7fb94807ad40 2024-03-24T00:36:05.579 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.582+0000 7fb947e00700 1 -- 172.21.15.38:0/2652945667 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb93c011a70 con 0x7fb94807ad40 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.582+0000 7fb94f000700 1 -- 172.21.15.38:0/2652945667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 msgr2=0x7fb9480791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.582+0000 7fb94f000700 1 --2- 172.21.15.38:0/2652945667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb9480791a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0098a0 tx=0x7fb93c009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.583+0000 7fb94f000700 1 -- 172.21.15.38:0/2652945667 shutdown_connections 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.583+0000 7fb94f000700 1 --2- 172.21.15.38:0/2652945667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb9480791a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.583+0000 7fb94f000700 1 -- 172.21.15.38:0/2652945667 >> 172.21.15.38:0/2652945667 conn(0x7fb948101770 msgr2=0x7fb948103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.583+0000 7fb94f000700 1 -- 172.21.15.38:0/2652945667 shutdown_connections 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.583+0000 7fb94f000700 1 -- 172.21.15.38:0/2652945667 wait complete. 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.584+0000 7fb94f000700 1 Processor -- start 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.584+0000 7fb94f000700 1 -- start start 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.584+0000 7fb94f000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb94819c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.584+0000 7fb94f000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94819c630 con 0x7fb94807ad40 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.584+0000 7fb94dc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb94819c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:05.580 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.585+0000 7fb94dc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb94819c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46732/0 (socket says 172.21.15.38:46732) 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.585+0000 7fb94dc00700 1 -- 172.21.15.38:0/2507818072 learned_addr learned my addr 172.21.15.38:0/2507818072 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.585+0000 7fb94dc00700 1 -- 172.21.15.38:0/2507818072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c009550 con 0x7fb94807ad40 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.585+0000 7fb94dc00700 1 --2- 172.21.15.38:0/2507818072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb94819c0f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb93c011710 tx=0x7fb93c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.585+0000 7fb946000700 1 -- 172.21.15.38:0/2507818072 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb93c003b10 con 0x7fb94807ad40 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.585+0000 7fb946000700 1 -- 172.21.15.38:0/2507818072 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb93c01b8c0 con 0x7fb94807ad40 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.585+0000 7fb946000700 1 -- 172.21.15.38:0/2507818072 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb93c019770 con 0x7fb94807ad40 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.586+0000 7fb94f000700 1 -- 172.21.15.38:0/2507818072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94819c830 con 0x7fb94807ad40 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.586+0000 7fb94f000700 1 -- 172.21.15.38:0/2507818072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94819ccd0 con 0x7fb94807ad40 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.587+0000 7fb946000700 1 -- 172.21.15.38:0/2507818072 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 45537+0+0 (secure 0 0 0) 0x7fb93c01ba30 con 0x7fb94807ad40 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.587+0000 7fb94f000700 1 -- 172.21.15.38:0/2507818072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94810c4d0 con 0x7fb94807ad40 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.587+0000 7fb946000700 1 --2- 172.21.15.38:0/2507818072 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb930038720 0x7fb93003abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.587+0000 7fb946000700 1 -- 172.21.15.38:0/2507818072 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb93c04c860 con 0x7fb94807ad40 2024-03-24T00:36:05.581 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.588+0000 7fb94d200700 1 --2- 172.21.15.38:0/2507818072 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb930038720 0x7fb93003abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.588+0000 7fb94d200700 1 --2- 172.21.15.38:0/2507818072 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb930038720 0x7fb93003abe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb934006fd0 tx=0x7fb934006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.593+0000 7fb946000700 1 -- 172.21.15.38:0/2507818072 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb93c02c9a0 con 0x7fb94807ad40 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.876+0000 7fb94f000700 1 -- 172.21.15.38:0/2507818072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fb94810c6f0 con 0x7fb94807ad40 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.913+0000 7fb946000700 1 -- 172.21.15.38:0/2507818072 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 5) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fb93c0199a0 con 0x7fb94807ad40 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.916+0000 7fb946000700 1 -- 172.21.15.38:0/2507818072 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fb93c04c280 con 0x7fb94807ad40 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.918+0000 7fb94f000700 1 -- 172.21.15.38:0/2507818072 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb930038720 msgr2=0x7fb93003abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.918+0000 7fb94f000700 1 --2- 172.21.15.38:0/2507818072 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb930038720 0x7fb93003abe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb934006fd0 tx=0x7fb934006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.919+0000 7fb94f000700 1 -- 172.21.15.38:0/2507818072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 msgr2=0x7fb94819c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.919+0000 7fb94f000700 1 --2- 172.21.15.38:0/2507818072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb94819c0f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb93c011710 tx=0x7fb93c0047c0 comp rx=0 tx=0).stop 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.919+0000 7fb94f000700 1 -- 172.21.15.38:0/2507818072 shutdown_connections 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.919+0000 7fb94f000700 1 --2- 172.21.15.38:0/2507818072 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb930038720 0x7fb93003abe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.919+0000 7fb94f000700 1 --2- 172.21.15.38:0/2507818072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb94807ad40 0x7fb94819c0f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:05.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.919+0000 7fb94f000700 1 -- 172.21.15.38:0/2507818072 >> 172.21.15.38:0/2507818072 conn(0x7fb948101770 msgr2=0x7fb9481022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:05.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.919+0000 7fb94f000700 1 -- 172.21.15.38:0/2507818072 shutdown_connections 2024-03-24T00:36:05.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:04.919+0000 7fb94f000700 1 -- 172.21.15.38:0/2507818072 wait complete. 2024-03-24T00:36:06.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:05 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2507818072' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-24T00:36:06.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:05 smithi038 ceph-mon[29271]: mgrmap e5: smithi038.huhioz(active, since 5s) 2024-03-24T00:36:07.082 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:06 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/576812478' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-24T00:36:07.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-03-24T00:36:07.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-24T00:36:07.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T00:36:07.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "smithi038.huhioz", 2024-03-24T00:36:07.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-24T00:36:07.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-03-24T00:36:07.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.150+0000 7fb7aee00700 1 Processor -- start 2024-03-24T00:36:07.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.150+0000 7fb7aee00700 1 -- start start 2024-03-24T00:36:07.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.150+0000 7fb7aee00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a8108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:07.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.150+0000 7fb7aee00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a8108f60 con 0x7fb7a8108570 2024-03-24T00:36:07.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.151+0000 7fb7a7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a8108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:07.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.151+0000 7fb7a7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a8108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46750/0 (socket says 172.21.15.38:46750) 2024-03-24T00:36:07.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.151+0000 7fb7a7e00700 1 -- 172.21.15.38:0/4210329105 learned_addr learned my addr 172.21.15.38:0/4210329105 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:07.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.151+0000 7fb7a7e00700 1 -- 172.21.15.38:0/4210329105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a8109770 con 0x7fb7a8108570 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.151+0000 7fb7a7e00700 1 --2- 172.21.15.38:0/4210329105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a8108990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098a0 tx=0x7fb798009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d234c9e7c9ffe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.152+0000 7fb7a6a00700 1 -- 172.21.15.38:0/4210329105 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb798004180 con 0x7fb7a8108570 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.152+0000 7fb7a6a00700 1 -- 172.21.15.38:0/4210329105 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb7980042e0 con 0x7fb7a8108570 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.152+0000 7fb7a6a00700 1 -- 172.21.15.38:0/4210329105 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb798011a70 con 0x7fb7a8108570 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.152+0000 7fb7aee00700 1 -- 172.21.15.38:0/4210329105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 msgr2=0x7fb7a8108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.152+0000 7fb7aee00700 1 --2- 172.21.15.38:0/4210329105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a8108990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098a0 tx=0x7fb798009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.152+0000 7fb7aee00700 1 -- 172.21.15.38:0/4210329105 shutdown_connections 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.152+0000 7fb7aee00700 1 --2- 172.21.15.38:0/4210329105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a8108990 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.152+0000 7fb7aee00700 1 -- 172.21.15.38:0/4210329105 >> 172.21.15.38:0/4210329105 conn(0x7fb7a8103b50 msgr2=0x7fb7a8105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.152+0000 7fb7aee00700 1 -- 172.21.15.38:0/4210329105 shutdown_connections 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.153+0000 7fb7aee00700 1 -- 172.21.15.38:0/4210329105 wait complete. 2024-03-24T00:36:07.206 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.153+0000 7fb7aee00700 1 Processor -- start 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.154+0000 7fb7aee00700 1 -- start start 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.154+0000 7fb7aee00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.154+0000 7fb7aee00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a819c860 con 0x7fb7a8108570 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.154+0000 7fb7a7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.154+0000 7fb7a7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a819c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46756/0 (socket says 172.21.15.38:46756) 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.154+0000 7fb7a7e00700 1 -- 172.21.15.38:0/576812478 learned_addr learned my addr 172.21.15.38:0/576812478 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.155+0000 7fb7a7e00700 1 -- 172.21.15.38:0/576812478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798009550 con 0x7fb7a8108570 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.155+0000 7fb7a7e00700 1 --2- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a819c320 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb798006af0 tx=0x7fb7980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.155+0000 7fb7a4c00700 1 -- 172.21.15.38:0/576812478 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb798003a00 con 0x7fb7a8108570 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.155+0000 7fb7a4c00700 1 -- 172.21.15.38:0/576812478 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb798003b60 con 0x7fb7a8108570 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.155+0000 7fb7aee00700 1 -- 172.21.15.38:0/576812478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a819ca60 con 0x7fb7a8108570 2024-03-24T00:36:07.207 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.155+0000 7fb7a4c00700 1 -- 172.21.15.38:0/576812478 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb798023750 con 0x7fb7a8108570 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.155+0000 7fb7aee00700 1 -- 172.21.15.38:0/576812478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a819cf00 con 0x7fb7a8108570 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.157+0000 7fb7aee00700 1 -- 172.21.15.38:0/576812478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a810c9a0 con 0x7fb7a8108570 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.157+0000 7fb7a4c00700 1 -- 172.21.15.38:0/576812478 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fb798023940 con 0x7fb7a8108570 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.157+0000 7fb7a4c00700 1 --2- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb78c038770 0x7fb78c03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.158+0000 7fb7a4c00700 1 -- 172.21.15.38:0/576812478 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb79804c960 con 0x7fb7a8108570 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.158+0000 7fb7a7400700 1 -- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb78c038770 msgr2=0x7fb78c03ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3284714719 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.158+0000 7fb7a7400700 1 --2- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb78c038770 0x7fb78c03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.163+0000 7fb7a4c00700 1 -- 172.21.15.38:0/576812478 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb79802b760 con 0x7fb7a8108570 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.359+0000 7fb7a7400700 1 -- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb78c038770 msgr2=0x7fb78c03ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3284714719 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.359+0000 7fb7a7400700 1 --2- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb78c038770 0x7fb78c03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.443+0000 7fb7aee00700 1 -- 172.21.15.38:0/576812478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb7a819fc10 con 0x7fb7a8108570 2024-03-24T00:36:07.208 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.443+0000 7fb7a4c00700 1 -- 172.21.15.38:0/576812478 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fb79802b350 con 0x7fb7a8108570 2024-03-24T00:36:07.209 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.447+0000 7fb7aee00700 1 -- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb78c038770 msgr2=0x7fb78c03ac30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T00:36:07.209 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.447+0000 7fb7aee00700 1 --2- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb78c038770 0x7fb78c03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:07.209 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.447+0000 7fb7aee00700 1 -- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 msgr2=0x7fb7a819c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:07.209 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.447+0000 7fb7aee00700 1 --2- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a819c320 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb798006af0 tx=0x7fb7980047c0 comp rx=0 tx=0).stop 2024-03-24T00:36:07.209 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.447+0000 7fb7aee00700 1 -- 172.21.15.38:0/576812478 shutdown_connections 2024-03-24T00:36:07.209 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.447+0000 7fb7aee00700 1 --2- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7fb78c038770 0x7fb78c03ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:07.209 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.447+0000 7fb7aee00700 1 --2- 172.21.15.38:0/576812478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7a8108570 0x7fb7a819c320 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:07.209 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.447+0000 7fb7aee00700 1 -- 172.21.15.38:0/576812478 >> 172.21.15.38:0/576812478 conn(0x7fb7a8103b50 msgr2=0x7fb7a8104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:07.209 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.448+0000 7fb7aee00700 1 -- 172.21.15.38:0/576812478 shutdown_connections 2024-03-24T00:36:07.209 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:06.448+0000 7fb7aee00700 1 -- 172.21.15.38:0/576812478 wait complete. 2024-03-24T00:36:07.210 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-03-24T00:36:07.210 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 5... 2024-03-24T00:36:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: Active manager daemon smithi038.huhioz restarted 2024-03-24T00:36:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: Activating manager daemon smithi038.huhioz 2024-03-24T00:36:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: osdmap e2: 0 total, 0 up, 0 in 2024-03-24T00:36:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: mgrmap e6: smithi038.huhioz(active, starting, since 0.0136452s) 2024-03-24T00:36:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:36:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi038.huhioz", "id": "smithi038.huhioz"}]: dispatch 2024-03-24T00:36:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T00:36:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T00:36:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T00:36:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: Manager daemon smithi038.huhioz is now available 2024-03-24T00:36:14.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: Found migration_current of "None". Setting to last migration. 2024-03-24T00:36:14.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:14.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:14.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:36:14.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:36:14.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:36:14.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:36:14.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:14 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/trash_purge_schedule"}]: dispatch 2024-03-24T00:36:15.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:15 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:15.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:15 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:15.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:15 smithi038 ceph-mon[29271]: mgrmap e7: smithi038.huhioz(active, since 1.02349s) 2024-03-24T00:36:15.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:15 smithi038 ceph-mon[29271]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-24T00:36:15.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:15 smithi038 ceph-mon[29271]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-24T00:36:15.728 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-03-24T00:36:15.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-24T00:36:15.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-24T00:36:15.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-03-24T00:36:15.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.770+0000 7f4f27600700 1 Processor -- start 2024-03-24T00:36:15.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.771+0000 7f4f27600700 1 -- start start 2024-03-24T00:36:15.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.771+0000 7f4f27600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f20105040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:15.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.771+0000 7f4f27600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f20074500 con 0x7f4f20104c20 2024-03-24T00:36:15.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.772+0000 7f4f24c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f20105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:15.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.772+0000 7f4f24c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f20105040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46764/0 (socket says 172.21.15.38:46764) 2024-03-24T00:36:15.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.772+0000 7f4f24c00700 1 -- 172.21.15.38:0/3550178121 learned_addr learned my addr 172.21.15.38:0/3550178121 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:15.729 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.772+0000 7f4f24c00700 1 -- 172.21.15.38:0/3550178121 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f20105580 con 0x7f4f20104c20 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.772+0000 7f4f24c00700 1 --2- 172.21.15.38:0/3550178121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f20105040 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f140098a0 tx=0x7f4f14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165a42d8264c3304 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.773+0000 7f4f1f400700 1 -- 172.21.15.38:0/3550178121 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f14004180 con 0x7f4f20104c20 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.773+0000 7f4f1f400700 1 -- 172.21.15.38:0/3550178121 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f4f140042e0 con 0x7f4f20104c20 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.773+0000 7f4f1f400700 1 -- 172.21.15.38:0/3550178121 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f14011a70 con 0x7f4f20104c20 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.773+0000 7f4f27600700 1 -- 172.21.15.38:0/3550178121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 msgr2=0x7f4f20105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.773+0000 7f4f27600700 1 --2- 172.21.15.38:0/3550178121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f20105040 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f140098a0 tx=0x7f4f14009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.773+0000 7f4f27600700 1 -- 172.21.15.38:0/3550178121 shutdown_connections 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.774+0000 7f4f27600700 1 --2- 172.21.15.38:0/3550178121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f20105040 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.774+0000 7f4f27600700 1 -- 172.21.15.38:0/3550178121 >> 172.21.15.38:0/3550178121 conn(0x7f4f20100800 msgr2=0x7f4f20102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.774+0000 7f4f27600700 1 -- 172.21.15.38:0/3550178121 shutdown_connections 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.774+0000 7f4f27600700 1 -- 172.21.15.38:0/3550178121 wait complete. 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.775+0000 7f4f27600700 1 Processor -- start 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.775+0000 7f4f27600700 1 -- start start 2024-03-24T00:36:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.775+0000 7f4f27600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f2019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.775+0000 7f4f27600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2019c7d0 con 0x7f4f20104c20 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.775+0000 7f4f24c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f2019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.775+0000 7f4f24c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f2019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46768/0 (socket says 172.21.15.38:46768) 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.775+0000 7f4f24c00700 1 -- 172.21.15.38:0/1982385283 learned_addr learned my addr 172.21.15.38:0/1982385283 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.776+0000 7f4f24c00700 1 -- 172.21.15.38:0/1982385283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f14009550 con 0x7f4f20104c20 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.776+0000 7f4f24c00700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f2019c290 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f14011710 tx=0x7f4f140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.776+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f14003b10 con 0x7f4f20104c20 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.776+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f4f1401b8c0 con 0x7f4f20104c20 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.776+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f140197e0 con 0x7f4f20104c20 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.777+0000 7f4f27600700 1 -- 172.21.15.38:0/1982385283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2019c9d0 con 0x7f4f20104c20 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.777+0000 7f4f27600700 1 -- 172.21.15.38:0/1982385283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2019ce70 con 0x7f4f20104c20 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.778+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f4f1401ba30 con 0x7f4f20104c20 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.779+0000 7f4f1d600700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 0x7f4f0803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.779+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 --> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f0803b340 con 0x7f4f08038770 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.779+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4f1404c930 con 0x7f4f20104c20 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.779+0000 7f4f1fe00700 1 -- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 msgr2=0x7f4f0803ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3284714719 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.779+0000 7f4f1fe00700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 0x7f4f0803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.980+0000 7f4f1fe00700 1 -- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 msgr2=0x7f4f0803ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3284714719 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:07.980+0000 7f4f1fe00700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 0x7f4f0803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:08.381+0000 7f4f1fe00700 1 -- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 msgr2=0x7f4f0803ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3284714719 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:08.381+0000 7f4f1fe00700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 0x7f4f0803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:09.182+0000 7f4f1fe00700 1 -- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 msgr2=0x7f4f0803ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3284714719 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:09.182+0000 7f4f1fe00700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 0x7f4f0803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:10.784+0000 7f4f1fe00700 1 -- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 msgr2=0x7f4f0803ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3284714719 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:10.784+0000 7f4f1fe00700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 0x7f4f0803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:13.969+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 6) v1 ==== 45315+0+0 (secure 0 0 0) 0x7f4f14019940 con 0x7f4f20104c20 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:13.969+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 msgr2=0x7f4f0803ac30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T00:36:15.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:13.969+0000 7f4f1d600700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 0x7f4f0803ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.975+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 7) v1 ==== 45442+0+0 (secure 0 0 0) 0x7f4f1404e350 con 0x7f4f20104c20 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.976+0000 7f4f1d600700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4f0803bfb0 0x7f4f0803e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.976+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f0803b340 con 0x7f4f0803bfb0 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.978+0000 7f4f1fe00700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4f0803bfb0 0x7f4f0803e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.981+0000 7f4f1fe00700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4f0803bfb0 0x7f4f0803e3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c003a10 tx=0x7f4f0c0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.982+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f4f0803b340 con 0x7f4f0803bfb0 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.990+0000 7f4f27600700 1 -- 172.21.15.38:0/1982385283 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4f201063a0 con 0x7f4f0803bfb0 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.991+0000 7f4f1d600700 1 -- 172.21.15.38:0/1982385283 <== mgr.14118 v2:172.21.15.38:6800/3111371494 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4f201063a0 con 0x7f4f0803bfb0 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.991+0000 7f4f27600700 1 -- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4f0803bfb0 msgr2=0x7f4f0803e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.991+0000 7f4f27600700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4f0803bfb0 0x7f4f0803e3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c003a10 tx=0x7f4f0c0092b0 comp rx=0 tx=0).stop 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.991+0000 7f4f27600700 1 -- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 msgr2=0x7f4f2019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.991+0000 7f4f27600700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f2019c290 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f14011710 tx=0x7f4f140047c0 comp rx=0 tx=0).stop 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.992+0000 7f4f27600700 1 -- 172.21.15.38:0/1982385283 shutdown_connections 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.992+0000 7f4f27600700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4f0803bfb0 0x7f4f0803e3a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.992+0000 7f4f27600700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f20104c20 0x7f4f2019c290 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:15.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.992+0000 7f4f27600700 1 --2- 172.21.15.38:0/1982385283 >> [v2:172.21.15.38:6800/3284714719,v1:172.21.15.38:6801/3284714719] conn(0x7f4f08038770 0x7f4f0803ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:15.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.992+0000 7f4f27600700 1 -- 172.21.15.38:0/1982385283 >> 172.21.15.38:0/1982385283 conn(0x7f4f20100800 msgr2=0x7f4f201930e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:15.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.992+0000 7f4f27600700 1 -- 172.21.15.38:0/1982385283 shutdown_connections 2024-03-24T00:36:15.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:14.992+0000 7f4f27600700 1 -- 172.21.15.38:0/1982385283 wait complete. 2024-03-24T00:36:15.734 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 5 is available 2024-03-24T00:36:15.734 INFO:teuthology.orchestra.run.smithi038.stdout:Setting orchestrator backend to cephadm... 2024-03-24T00:36:16.581 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:16 smithi038 ceph-mon[29271]: [24/Mar/2024:00:36:15] ENGINE Bus STARTING 2024-03-24T00:36:16.582 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:16 smithi038 ceph-mon[29271]: [24/Mar/2024:00:36:15] ENGINE Serving on http://172.21.15.38:8765 2024-03-24T00:36:16.582 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:16 smithi038 ceph-mon[29271]: [24/Mar/2024:00:36:15] ENGINE Serving on https://172.21.15.38:7150 2024-03-24T00:36:16.582 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:16 smithi038 ceph-mon[29271]: [24/Mar/2024:00:36:15] ENGINE Bus STARTED 2024-03-24T00:36:16.582 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:16 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:36:17.215 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.325+0000 7f94ff600700 1 Processor -- start 2024-03-24T00:36:17.215 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.325+0000 7f94ff600700 1 -- start start 2024-03-24T00:36:17.215 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.326+0000 7f94ff600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f8106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:17.216 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.326+0000 7f94ff600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f8106d40 con 0x7f94f8106350 2024-03-24T00:36:17.216 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.326+0000 7f94fcc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f8106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:17.216 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.327+0000 7f94fcc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f8106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53448/0 (socket says 172.21.15.38:53448) 2024-03-24T00:36:17.216 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.327+0000 7f94fcc00700 1 -- 172.21.15.38:0/3256663492 learned_addr learned my addr 172.21.15.38:0/3256663492 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:17.216 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.327+0000 7f94fcc00700 1 -- 172.21.15.38:0/3256663492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f8107550 con 0x7f94f8106350 2024-03-24T00:36:17.216 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.327+0000 7f94fcc00700 1 --2- 172.21.15.38:0/3256663492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f8106770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0098a0 tx=0x7f94ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea90b7b9dbdd0c82 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:17.216 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.328+0000 7f94f7400700 1 -- 172.21.15.38:0/3256663492 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f94ec004180 con 0x7f94f8106350 2024-03-24T00:36:17.216 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.328+0000 7f94f7400700 1 -- 172.21.15.38:0/3256663492 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f94ec0042e0 con 0x7f94f8106350 2024-03-24T00:36:17.221 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.328+0000 7f94f7400700 1 -- 172.21.15.38:0/3256663492 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f94ec011a70 con 0x7f94f8106350 2024-03-24T00:36:17.221 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.329+0000 7f94ff600700 1 -- 172.21.15.38:0/3256663492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 msgr2=0x7f94f8106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:17.221 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.329+0000 7f94ff600700 1 --2- 172.21.15.38:0/3256663492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f8106770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0098a0 tx=0x7f94ec009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:17.221 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.329+0000 7f94ff600700 1 -- 172.21.15.38:0/3256663492 shutdown_connections 2024-03-24T00:36:17.221 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.329+0000 7f94ff600700 1 --2- 172.21.15.38:0/3256663492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f8106770 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:17.221 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.329+0000 7f94ff600700 1 -- 172.21.15.38:0/3256663492 >> 172.21.15.38:0/3256663492 conn(0x7f94f8101930 msgr2=0x7f94f8103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:17.221 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.329+0000 7f94ff600700 1 -- 172.21.15.38:0/3256663492 shutdown_connections 2024-03-24T00:36:17.221 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.329+0000 7f94ff600700 1 -- 172.21.15.38:0/3256663492 wait complete. 2024-03-24T00:36:17.221 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.330+0000 7f94ff600700 1 Processor -- start 2024-03-24T00:36:17.221 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.330+0000 7f94ff600700 1 -- start start 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.331+0000 7f94ff600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.331+0000 7f94ff600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f819c830 con 0x7f94f8106350 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.331+0000 7f94fcc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.331+0000 7f94fcc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f819c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53456/0 (socket says 172.21.15.38:53456) 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.331+0000 7f94fcc00700 1 -- 172.21.15.38:0/4003392189 learned_addr learned my addr 172.21.15.38:0/4003392189 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.331+0000 7f94fcc00700 1 -- 172.21.15.38:0/4003392189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec009550 con 0x7f94f8106350 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.332+0000 7f94fcc00700 1 --2- 172.21.15.38:0/4003392189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f819c2f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f94ec011710 tx=0x7f94ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.332+0000 7f94f5600700 1 -- 172.21.15.38:0/4003392189 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f94ec003b10 con 0x7f94f8106350 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.332+0000 7f94f5600700 1 -- 172.21.15.38:0/4003392189 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f94ec01b8c0 con 0x7f94f8106350 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.332+0000 7f94ff600700 1 -- 172.21.15.38:0/4003392189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f819ca30 con 0x7f94f8106350 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.332+0000 7f94f5600700 1 -- 172.21.15.38:0/4003392189 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f94ec019770 con 0x7f94f8106350 2024-03-24T00:36:17.222 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.332+0000 7f94ff600700 1 -- 172.21.15.38:0/4003392189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f819ced0 con 0x7f94f8106350 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.333+0000 7f94f5600700 1 -- 172.21.15.38:0/4003392189 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 7) v1 ==== 45442+0+0 (secure 0 0 0) 0x7f94ec01ba30 con 0x7f94f8106350 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.333+0000 7f94ff600700 1 -- 172.21.15.38:0/4003392189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f810a780 con 0x7f94f8106350 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.334+0000 7f94f5600700 1 --2- 172.21.15.38:0/4003392189 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f94e0038650 0x7f94e003ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.334+0000 7f94f5600700 1 -- 172.21.15.38:0/4003392189 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f94ec04c780 con 0x7f94f8106350 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.334+0000 7f94f7e00700 1 --2- 172.21.15.38:0/4003392189 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f94e0038650 0x7f94e003ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.335+0000 7f94f7e00700 1 --2- 172.21.15.38:0/4003392189 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f94e0038650 0x7f94e003ab10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f94e4006fd0 tx=0x7f94e4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.340+0000 7f94f5600700 1 -- 172.21.15.38:0/4003392189 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f94ec02cce0 con 0x7f94f8106350 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.511+0000 7f94f5600700 1 -- 172.21.15.38:0/4003392189 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f94ec01b440 con 0x7f94f8106350 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.581+0000 7f94ff600700 1 -- 172.21.15.38:0/4003392189 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f94f8102b50 con 0x7f94e0038650 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.602+0000 7f94f5600700 1 -- 172.21.15.38:0/4003392189 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f94f8102b50 con 0x7f94e0038650 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.605+0000 7f94ff600700 1 -- 172.21.15.38:0/4003392189 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f94e0038650 msgr2=0x7f94e003ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.605+0000 7f94ff600700 1 --2- 172.21.15.38:0/4003392189 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f94e0038650 0x7f94e003ab10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f94e4006fd0 tx=0x7f94e4006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.605+0000 7f94ff600700 1 -- 172.21.15.38:0/4003392189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 msgr2=0x7f94f819c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.605+0000 7f94ff600700 1 --2- 172.21.15.38:0/4003392189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f819c2f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f94ec011710 tx=0x7f94ec0047c0 comp rx=0 tx=0).stop 2024-03-24T00:36:17.223 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.605+0000 7f94ff600700 1 -- 172.21.15.38:0/4003392189 shutdown_connections 2024-03-24T00:36:17.224 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.605+0000 7f94ff600700 1 --2- 172.21.15.38:0/4003392189 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f94e0038650 0x7f94e003ab10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:17.224 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.605+0000 7f94ff600700 1 --2- 172.21.15.38:0/4003392189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f8106350 0x7f94f819c2f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:17.224 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.605+0000 7f94ff600700 1 -- 172.21.15.38:0/4003392189 >> 172.21.15.38:0/4003392189 conn(0x7f94f8101930 msgr2=0x7f94f8102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:17.224 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.605+0000 7f94ff600700 1 -- 172.21.15.38:0/4003392189 shutdown_connections 2024-03-24T00:36:17.224 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:16.606+0000 7f94ff600700 1 -- 172.21.15.38:0/4003392189 wait complete. 2024-03-24T00:36:17.765 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:17 smithi038 ceph-mon[29271]: mgrmap e8: smithi038.huhioz(active, since 2s) 2024-03-24T00:36:17.765 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:17 smithi038 ceph-mon[29271]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:17.765 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:17 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:17.765 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:17 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:36:18.753 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-24T00:36:18.753 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.839+0000 7f5acfa00700 1 Processor -- start 2024-03-24T00:36:18.753 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.839+0000 7f5acfa00700 1 -- start start 2024-03-24T00:36:18.753 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.840+0000 7f5acfa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac8079120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:18.753 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.840+0000 7f5acfa00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac80796f0 con 0x7f5ac807acc0 2024-03-24T00:36:18.756 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.840+0000 7f5acd000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac8079120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:18.756 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.840+0000 7f5acd000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac8079120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53464/0 (socket says 172.21.15.38:53464) 2024-03-24T00:36:18.756 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.840+0000 7f5acd000700 1 -- 172.21.15.38:0/1883008371 learned_addr learned my addr 172.21.15.38:0/1883008371 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:18.756 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.841+0000 7f5acd000700 1 -- 172.21.15.38:0/1883008371 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac8079830 con 0x7f5ac807acc0 2024-03-24T00:36:18.756 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.841+0000 7f5acd000700 1 --2- 172.21.15.38:0/1883008371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac8079120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0098a0 tx=0x7f5abc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d94d1f03478006 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:18.756 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.841+0000 7f5ac7400700 1 -- 172.21.15.38:0/1883008371 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5abc004180 con 0x7f5ac807acc0 2024-03-24T00:36:18.756 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.841+0000 7f5ac7400700 1 -- 172.21.15.38:0/1883008371 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5abc0042e0 con 0x7f5ac807acc0 2024-03-24T00:36:18.756 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.841+0000 7f5ac7400700 1 -- 172.21.15.38:0/1883008371 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5abc011a70 con 0x7f5ac807acc0 2024-03-24T00:36:18.756 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.842+0000 7f5acfa00700 1 -- 172.21.15.38:0/1883008371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 msgr2=0x7f5ac8079120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:18.756 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.842+0000 7f5acfa00700 1 --2- 172.21.15.38:0/1883008371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac8079120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0098a0 tx=0x7f5abc009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.842+0000 7f5acfa00700 1 -- 172.21.15.38:0/1883008371 shutdown_connections 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.842+0000 7f5acfa00700 1 --2- 172.21.15.38:0/1883008371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac8079120 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.842+0000 7f5acfa00700 1 -- 172.21.15.38:0/1883008371 >> 172.21.15.38:0/1883008371 conn(0x7f5ac81018f0 msgr2=0x7f5ac8103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.842+0000 7f5acfa00700 1 -- 172.21.15.38:0/1883008371 shutdown_connections 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.842+0000 7f5acfa00700 1 -- 172.21.15.38:0/1883008371 wait complete. 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.843+0000 7f5acfa00700 1 Processor -- start 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.843+0000 7f5acfa00700 1 -- start start 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.844+0000 7f5acfa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.844+0000 7f5acfa00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac819c7c0 con 0x7f5ac807acc0 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.844+0000 7f5acd000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.844+0000 7f5acd000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac819c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53478/0 (socket says 172.21.15.38:53478) 2024-03-24T00:36:18.757 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.844+0000 7f5acd000700 1 -- 172.21.15.38:0/760615136 learned_addr learned my addr 172.21.15.38:0/760615136 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:18.760 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.844+0000 7f5acd000700 1 -- 172.21.15.38:0/760615136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc009550 con 0x7f5ac807acc0 2024-03-24T00:36:18.760 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.845+0000 7f5acd000700 1 --2- 172.21.15.38:0/760615136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac819c280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5abc011710 tx=0x7f5abc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:18.760 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.845+0000 7f5ac5600700 1 -- 172.21.15.38:0/760615136 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5abc003b10 con 0x7f5ac807acc0 2024-03-24T00:36:18.760 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.845+0000 7f5ac5600700 1 -- 172.21.15.38:0/760615136 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5abc01b8c0 con 0x7f5ac807acc0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.845+0000 7f5ac5600700 1 -- 172.21.15.38:0/760615136 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5abc0197e0 con 0x7f5ac807acc0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.845+0000 7f5acfa00700 1 -- 172.21.15.38:0/760615136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac819c9c0 con 0x7f5ac807acc0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.845+0000 7f5acfa00700 1 -- 172.21.15.38:0/760615136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac819ce60 con 0x7f5ac807acc0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.846+0000 7f5ac5600700 1 -- 172.21.15.38:0/760615136 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f5abc01ba30 con 0x7f5ac807acc0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.846+0000 7f5acfa00700 1 -- 172.21.15.38:0/760615136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac810c690 con 0x7f5ac807acc0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.847+0000 7f5ac5600700 1 --2- 172.21.15.38:0/760615136 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f5ab00387c0 0x7f5ab003ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.847+0000 7f5ac5600700 1 -- 172.21.15.38:0/760615136 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5abc04c9b0 con 0x7f5ac807acc0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.847+0000 7f5ac7e00700 1 --2- 172.21.15.38:0/760615136 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f5ab00387c0 0x7f5ab003ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.848+0000 7f5ac7e00700 1 --2- 172.21.15.38:0/760615136 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f5ab00387c0 0x7f5ab003ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4006fd0 tx=0x7f5ab4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:17.852+0000 7f5ac5600700 1 -- 172.21.15.38:0/760615136 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5abc01bce0 con 0x7f5ac807acc0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.076+0000 7f5acfa00700 1 -- 172.21.15.38:0/760615136 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ac8102ba0 con 0x7f5ab00387c0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.078+0000 7f5ac5600700 1 -- 172.21.15.38:0/760615136 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f5ac8102ba0 con 0x7f5ab00387c0 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.080+0000 7f5acfa00700 1 -- 172.21.15.38:0/760615136 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f5ab00387c0 msgr2=0x7f5ab003ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:18.761 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.080+0000 7f5acfa00700 1 --2- 172.21.15.38:0/760615136 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f5ab00387c0 0x7f5ab003ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4006fd0 tx=0x7f5ab4006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:18.762 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.080+0000 7f5acfa00700 1 -- 172.21.15.38:0/760615136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 msgr2=0x7f5ac819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:18.762 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.080+0000 7f5acfa00700 1 --2- 172.21.15.38:0/760615136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac819c280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5abc011710 tx=0x7f5abc0047c0 comp rx=0 tx=0).stop 2024-03-24T00:36:18.762 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.081+0000 7f5acfa00700 1 -- 172.21.15.38:0/760615136 shutdown_connections 2024-03-24T00:36:18.762 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.081+0000 7f5acfa00700 1 --2- 172.21.15.38:0/760615136 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f5ab00387c0 0x7f5ab003ac80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:18.762 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.081+0000 7f5acfa00700 1 --2- 172.21.15.38:0/760615136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ac807acc0 0x7f5ac819c280 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:18.762 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.081+0000 7f5acfa00700 1 -- 172.21.15.38:0/760615136 >> 172.21.15.38:0/760615136 conn(0x7f5ac81018f0 msgr2=0x7f5ac8102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:18.762 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.081+0000 7f5acfa00700 1 -- 172.21.15.38:0/760615136 shutdown_connections 2024-03-24T00:36:18.762 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:18.081+0000 7f5acfa00700 1 -- 172.21.15.38:0/760615136 wait complete. 2024-03-24T00:36:18.762 INFO:teuthology.orchestra.run.smithi038.stdout:Generating ssh key... 2024-03-24T00:36:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:19 smithi038 ceph-mon[29271]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:20.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.373+0000 7f1d09600700 1 Processor -- start 2024-03-24T00:36:20.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.373+0000 7f1d09600700 1 -- start start 2024-03-24T00:36:20.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.374+0000 7f1d09600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d04108790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:20.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.374+0000 7f1d09600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04108d60 con 0x7f1d04108370 2024-03-24T00:36:20.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.374+0000 7f1d03e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d04108790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:20.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.374+0000 7f1d03e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d04108790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53490/0 (socket says 172.21.15.38:53490) 2024-03-24T00:36:20.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.374+0000 7f1d03e00700 1 -- 172.21.15.38:0/2605480011 learned_addr learned my addr 172.21.15.38:0/2605480011 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:20.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.375+0000 7f1d03e00700 1 -- 172.21.15.38:0/2605480011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04109570 con 0x7f1d04108370 2024-03-24T00:36:20.486 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.375+0000 7f1d03e00700 1 --2- 172.21.15.38:0/2605480011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d04108790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40098a0 tx=0x7f1cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf684ac3b26fdb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:20.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.375+0000 7f1d02a00700 1 -- 172.21.15.38:0/2605480011 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cf4004180 con 0x7f1d04108370 2024-03-24T00:36:20.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.375+0000 7f1d02a00700 1 -- 172.21.15.38:0/2605480011 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1cf40042e0 con 0x7f1d04108370 2024-03-24T00:36:20.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.375+0000 7f1d02a00700 1 -- 172.21.15.38:0/2605480011 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cf4011a70 con 0x7f1d04108370 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.376+0000 7f1d09600700 1 -- 172.21.15.38:0/2605480011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 msgr2=0x7f1d04108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.376+0000 7f1d09600700 1 --2- 172.21.15.38:0/2605480011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d04108790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40098a0 tx=0x7f1cf4009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.376+0000 7f1d09600700 1 -- 172.21.15.38:0/2605480011 shutdown_connections 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.376+0000 7f1d09600700 1 --2- 172.21.15.38:0/2605480011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d04108790 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.376+0000 7f1d09600700 1 -- 172.21.15.38:0/2605480011 >> 172.21.15.38:0/2605480011 conn(0x7f1d0407b790 msgr2=0x7f1d04106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.376+0000 7f1d09600700 1 -- 172.21.15.38:0/2605480011 shutdown_connections 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.376+0000 7f1d09600700 1 -- 172.21.15.38:0/2605480011 wait complete. 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.377+0000 7f1d09600700 1 Processor -- start 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.378+0000 7f1d09600700 1 -- start start 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.378+0000 7f1d09600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d0419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.378+0000 7f1d09600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0419c660 con 0x7f1d04108370 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.378+0000 7f1d03e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d0419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:20.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.378+0000 7f1d03e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d0419c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53500/0 (socket says 172.21.15.38:53500) 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.378+0000 7f1d03e00700 1 -- 172.21.15.38:0/4069774567 learned_addr learned my addr 172.21.15.38:0/4069774567 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.379+0000 7f1d03e00700 1 -- 172.21.15.38:0/4069774567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf4009550 con 0x7f1d04108370 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.379+0000 7f1d03e00700 1 --2- 172.21.15.38:0/4069774567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d0419c120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4011710 tx=0x7f1cf40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.379+0000 7f1d00c00700 1 -- 172.21.15.38:0/4069774567 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cf4003b10 con 0x7f1d04108370 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.379+0000 7f1d00c00700 1 -- 172.21.15.38:0/4069774567 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1cf401b8c0 con 0x7f1d04108370 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.379+0000 7f1d09600700 1 -- 172.21.15.38:0/4069774567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0419c860 con 0x7f1d04108370 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.379+0000 7f1d00c00700 1 -- 172.21.15.38:0/4069774567 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cf4019770 con 0x7f1d04108370 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.379+0000 7f1d09600700 1 -- 172.21.15.38:0/4069774567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0419cd00 con 0x7f1d04108370 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.380+0000 7f1d09600700 1 -- 172.21.15.38:0/4069774567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0410c7a0 con 0x7f1d04108370 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.381+0000 7f1d00c00700 1 -- 172.21.15.38:0/4069774567 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f1cf401ba30 con 0x7f1d04108370 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.381+0000 7f1d00c00700 1 --2- 172.21.15.38:0/4069774567 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f1ce8038770 0x7f1ce803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.381+0000 7f1d00c00700 1 -- 172.21.15.38:0/4069774567 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1cf404c980 con 0x7f1d04108370 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.382+0000 7f1d03400700 1 --2- 172.21.15.38:0/4069774567 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f1ce8038770 0x7f1ce803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.382+0000 7f1d03400700 1 --2- 172.21.15.38:0/4069774567 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f1ce8038770 0x7f1ce803ac30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0006fd0 tx=0x7f1cf0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.387+0000 7f1d00c00700 1 -- 172.21.15.38:0/4069774567 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1cf402c9a0 con 0x7f1d04108370 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.611+0000 7f1d09600700 1 -- 172.21.15.38:0/4069774567 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d04106320 con 0x7f1ce8038770 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.832+0000 7f1d00c00700 1 -- 172.21.15.38:0/4069774567 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1d04106320 con 0x7f1ce8038770 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.835+0000 7f1d09600700 1 -- 172.21.15.38:0/4069774567 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f1ce8038770 msgr2=0x7f1ce803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.835+0000 7f1d09600700 1 --2- 172.21.15.38:0/4069774567 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f1ce8038770 0x7f1ce803ac30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0006fd0 tx=0x7f1cf0006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.835+0000 7f1d09600700 1 -- 172.21.15.38:0/4069774567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 msgr2=0x7f1d0419c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.835+0000 7f1d09600700 1 --2- 172.21.15.38:0/4069774567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d0419c120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4011710 tx=0x7f1cf40047c0 comp rx=0 tx=0).stop 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.836+0000 7f1d09600700 1 -- 172.21.15.38:0/4069774567 shutdown_connections 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.836+0000 7f1d09600700 1 --2- 172.21.15.38:0/4069774567 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f1ce8038770 0x7f1ce803ac30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.836+0000 7f1d09600700 1 --2- 172.21.15.38:0/4069774567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04108370 0x7f1d0419c120 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:20.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.836+0000 7f1d09600700 1 -- 172.21.15.38:0/4069774567 >> 172.21.15.38:0/4069774567 conn(0x7f1d0407b790 msgr2=0x7f1d04105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:20.494 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.836+0000 7f1d09600700 1 -- 172.21.15.38:0/4069774567 shutdown_connections 2024-03-24T00:36:20.494 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:19.836+0000 7f1d09600700 1 -- 172.21.15.38:0/4069774567 wait complete. 2024-03-24T00:36:21.061 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:20 smithi038 ceph-mon[29271]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:21.061 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:20 smithi038 ceph-mon[29271]: Generating ssh key... 2024-03-24T00:36:21.061 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:20 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:21.061 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:20 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:22.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDciVCtyVmHKLYB381xqHbea5IAKdRR7D/OpflBIUFOars7paBDT6WQ5pXjw7D6V2ZVvMXhAwKaXZLdNc1k34KD54L+t3jMmnnXSTeVyj8a2X1Oev2O8mmThUJpCqQ/qLEGDxqsbSAAeHo7wL5AXwYbphVLEkJdcU59mFiHBo1iI2DA+h5cA+X7r2X7Z1gb7jykX+YI121tXzt0czSECpVyLSWTQ+lKAq01UtsyRs/e41rR5Xbfm5yEfzVZSnZ6sMyOxwmNqB/kbaN8vJjvzhakRdoxJqzCxU0iRUj8QcRkkUXf++bPRT2hlqc0H3TbY3f95bNUA7cNHfKypbxd2oCRjcbEcj8f653hc9yk/BQZ/wD/hdlBp9TgBxRP4CmI2L6DSGITtMMM5LZbxuxDRdtuhxG+Wz6ZWnhkPL6fcnh4JG4dHcttphfBgGyPWYOs8hLuxrsma8F6pExK/R0ooIb0LOA6FhyAXMGXYsJUW2AGfosH7vtMJaQ4ZfpcPQ7fib0= ceph-4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:36:22.004 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.134+0000 7fa59b600700 1 Processor -- start 2024-03-24T00:36:22.004 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.135+0000 7fa59b600700 1 -- start start 2024-03-24T00:36:22.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.135+0000 7fa59b600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa5941067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:22.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.135+0000 7fa59b600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594106da0 con 0x7fa5941063b0 2024-03-24T00:36:22.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.135+0000 7fa598c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa5941067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:22.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.136+0000 7fa598c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa5941067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53508/0 (socket says 172.21.15.38:53508) 2024-03-24T00:36:22.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.136+0000 7fa598c00700 1 -- 172.21.15.38:0/1133726294 learned_addr learned my addr 172.21.15.38:0/1133726294 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:22.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.136+0000 7fa598c00700 1 -- 172.21.15.38:0/1133726294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5941075b0 con 0x7fa5941063b0 2024-03-24T00:36:22.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.136+0000 7fa598c00700 1 --2- 172.21.15.38:0/1133726294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa5941067d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa5840098a0 tx=0x7fa584009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594a028aaeb33f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:22.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.137+0000 7fa593e00700 1 -- 172.21.15.38:0/1133726294 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa584004180 con 0x7fa5941063b0 2024-03-24T00:36:22.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.137+0000 7fa593e00700 1 -- 172.21.15.38:0/1133726294 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa5840042e0 con 0x7fa5941063b0 2024-03-24T00:36:22.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.137+0000 7fa593e00700 1 -- 172.21.15.38:0/1133726294 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa584011a70 con 0x7fa5941063b0 2024-03-24T00:36:22.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.137+0000 7fa59b600700 1 -- 172.21.15.38:0/1133726294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 msgr2=0x7fa5941067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:22.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.137+0000 7fa59b600700 1 --2- 172.21.15.38:0/1133726294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa5941067d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa5840098a0 tx=0x7fa584009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:22.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.138+0000 7fa59b600700 1 -- 172.21.15.38:0/1133726294 shutdown_connections 2024-03-24T00:36:22.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.138+0000 7fa59b600700 1 --2- 172.21.15.38:0/1133726294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa5941067d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:22.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.138+0000 7fa59b600700 1 -- 172.21.15.38:0/1133726294 >> 172.21.15.38:0/1133726294 conn(0x7fa5941019b0 msgr2=0x7fa594103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:22.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.138+0000 7fa59b600700 1 -- 172.21.15.38:0/1133726294 shutdown_connections 2024-03-24T00:36:22.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.138+0000 7fa59b600700 1 -- 172.21.15.38:0/1133726294 wait complete. 2024-03-24T00:36:22.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.139+0000 7fa59b600700 1 Processor -- start 2024-03-24T00:36:22.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.139+0000 7fa59b600700 1 -- start start 2024-03-24T00:36:22.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.140+0000 7fa59b600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa59407a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:22.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.140+0000 7fa59b600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59407aec0 con 0x7fa5941063b0 2024-03-24T00:36:22.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.140+0000 7fa598c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa59407a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:22.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.140+0000 7fa598c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa59407a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53522/0 (socket says 172.21.15.38:53522) 2024-03-24T00:36:22.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.140+0000 7fa598c00700 1 -- 172.21.15.38:0/317055181 learned_addr learned my addr 172.21.15.38:0/317055181 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:22.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.140+0000 7fa598c00700 1 -- 172.21.15.38:0/317055181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009550 con 0x7fa5941063b0 2024-03-24T00:36:22.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.141+0000 7fa598c00700 1 --2- 172.21.15.38:0/317055181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa59407a980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa584006af0 tx=0x7fa5840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:22.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.141+0000 7fa592000700 1 -- 172.21.15.38:0/317055181 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa584003a00 con 0x7fa5941063b0 2024-03-24T00:36:22.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.141+0000 7fa592000700 1 -- 172.21.15.38:0/317055181 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa584003b60 con 0x7fa5941063b0 2024-03-24T00:36:22.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.141+0000 7fa592000700 1 -- 172.21.15.38:0/317055181 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa584023750 con 0x7fa5941063b0 2024-03-24T00:36:22.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.141+0000 7fa59b600700 1 -- 172.21.15.38:0/317055181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa594079100 con 0x7fa5941063b0 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.141+0000 7fa59b600700 1 -- 172.21.15.38:0/317055181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5940795a0 con 0x7fa5941063b0 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.142+0000 7fa592000700 1 -- 172.21.15.38:0/317055181 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fa584023940 con 0x7fa5941063b0 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.143+0000 7fa59b600700 1 -- 172.21.15.38:0/317055181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5940623e0 con 0x7fa5941063b0 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.143+0000 7fa592000700 1 --2- 172.21.15.38:0/317055181 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fa574038770 0x7fa57403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.143+0000 7fa592000700 1 -- 172.21.15.38:0/317055181 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa58404c9a0 con 0x7fa5941063b0 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.143+0000 7fa58be00700 1 --2- 172.21.15.38:0/317055181 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fa574038770 0x7fa57403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.144+0000 7fa58be00700 1 --2- 172.21.15.38:0/317055181 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fa574038770 0x7fa57403ac30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa57c006fd0 tx=0x7fa57c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.148+0000 7fa592000700 1 -- 172.21.15.38:0/317055181 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa58402b760 con 0x7fa5941063b0 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.373+0000 7fa59b600700 1 -- 172.21.15.38:0/317055181 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa594079ff0 con 0x7fa574038770 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.375+0000 7fa592000700 1 -- 172.21.15.38:0/317055181 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fa57c009e20 con 0x7fa574038770 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.378+0000 7fa59b600700 1 -- 172.21.15.38:0/317055181 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fa574038770 msgr2=0x7fa57403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.378+0000 7fa59b600700 1 --2- 172.21.15.38:0/317055181 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fa574038770 0x7fa57403ac30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa57c006fd0 tx=0x7fa57c006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:22.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.378+0000 7fa59b600700 1 -- 172.21.15.38:0/317055181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 msgr2=0x7fa59407a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:22.014 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.378+0000 7fa59b600700 1 --2- 172.21.15.38:0/317055181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa59407a980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa584006af0 tx=0x7fa5840047c0 comp rx=0 tx=0).stop 2024-03-24T00:36:22.014 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.378+0000 7fa59b600700 1 -- 172.21.15.38:0/317055181 shutdown_connections 2024-03-24T00:36:22.014 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.378+0000 7fa59b600700 1 --2- 172.21.15.38:0/317055181 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fa574038770 0x7fa57403ac30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:22.014 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.378+0000 7fa59b600700 1 --2- 172.21.15.38:0/317055181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5941063b0 0x7fa59407a980 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:22.014 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.378+0000 7fa59b600700 1 -- 172.21.15.38:0/317055181 >> 172.21.15.38:0/317055181 conn(0x7fa5941019b0 msgr2=0x7fa594102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:22.014 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.378+0000 7fa59b600700 1 -- 172.21.15.38:0/317055181 shutdown_connections 2024-03-24T00:36:22.014 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:21.379+0000 7fa59b600700 1 -- 172.21.15.38:0/317055181 wait complete. 2024-03-24T00:36:22.015 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-24T00:36:22.015 INFO:teuthology.orchestra.run.smithi038.stdout:Adding key to root@localhost authorized_keys... 2024-03-24T00:36:22.015 INFO:teuthology.orchestra.run.smithi038.stdout:Adding host smithi038... 2024-03-24T00:36:22.297 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:22 smithi038 ceph-mon[29271]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:23.130 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:23 smithi038 ceph-mon[29271]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:25.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:25 smithi038 ceph-mon[29271]: Deploying cephadm binary to smithi038 2024-03-24T00:36:28.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Added host 'smithi038' with addr '172.21.15.38' 2024-03-24T00:36:28.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.596+0000 7fe00fc00700 1 Processor -- start 2024-03-24T00:36:28.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.596+0000 7fe00fc00700 1 -- start start 2024-03-24T00:36:28.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.597+0000 7fe00fc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe008105030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:28.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.597+0000 7fe00fc00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008074520 con 0x7fe008104c10 2024-03-24T00:36:28.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.597+0000 7fe00d200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe008105030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:28.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.597+0000 7fe00d200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe008105030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53536/0 (socket says 172.21.15.38:53536) 2024-03-24T00:36:28.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.597+0000 7fe00d200700 1 -- 172.21.15.38:0/1893035992 learned_addr learned my addr 172.21.15.38:0/1893035992 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:28.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.598+0000 7fe00d200700 1 -- 172.21.15.38:0/1893035992 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008105570 con 0x7fe008104c10 2024-03-24T00:36:28.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.598+0000 7fe00d200700 1 --2- 172.21.15.38:0/1893035992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe008105030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0098a0 tx=0x7fdffc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c749ed1e6fdcabb server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:28.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.598+0000 7fe007400700 1 -- 172.21.15.38:0/1893035992 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdffc004180 con 0x7fe008104c10 2024-03-24T00:36:28.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.598+0000 7fe007400700 1 -- 172.21.15.38:0/1893035992 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdffc0042e0 con 0x7fe008104c10 2024-03-24T00:36:28.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.598+0000 7fe007400700 1 -- 172.21.15.38:0/1893035992 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdffc011a70 con 0x7fe008104c10 2024-03-24T00:36:28.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.599+0000 7fe00fc00700 1 -- 172.21.15.38:0/1893035992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 msgr2=0x7fe008105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:28.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.599+0000 7fe00fc00700 1 --2- 172.21.15.38:0/1893035992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe008105030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0098a0 tx=0x7fdffc009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:28.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.599+0000 7fe00fc00700 1 -- 172.21.15.38:0/1893035992 shutdown_connections 2024-03-24T00:36:28.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.599+0000 7fe00fc00700 1 --2- 172.21.15.38:0/1893035992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe008105030 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:28.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.599+0000 7fe00fc00700 1 -- 172.21.15.38:0/1893035992 >> 172.21.15.38:0/1893035992 conn(0x7fe008100810 msgr2=0x7fe008102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:28.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.600+0000 7fe00fc00700 1 -- 172.21.15.38:0/1893035992 shutdown_connections 2024-03-24T00:36:28.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.600+0000 7fe00fc00700 1 -- 172.21.15.38:0/1893035992 wait complete. 2024-03-24T00:36:28.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.601+0000 7fe00fc00700 1 Processor -- start 2024-03-24T00:36:28.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.601+0000 7fe00fc00700 1 -- start start 2024-03-24T00:36:28.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.601+0000 7fe00fc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe00819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:28.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.601+0000 7fe00fc00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00819c7d0 con 0x7fe008104c10 2024-03-24T00:36:28.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.601+0000 7fe00d200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe00819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:28.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.601+0000 7fe00d200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe00819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53546/0 (socket says 172.21.15.38:53546) 2024-03-24T00:36:28.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.601+0000 7fe00d200700 1 -- 172.21.15.38:0/689135985 learned_addr learned my addr 172.21.15.38:0/689135985 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:28.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.602+0000 7fe00d200700 1 -- 172.21.15.38:0/689135985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc009550 con 0x7fe008104c10 2024-03-24T00:36:28.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.602+0000 7fe00d200700 1 --2- 172.21.15.38:0/689135985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe00819c290 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdffc011710 tx=0x7fdffc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:28.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.602+0000 7fe005600700 1 -- 172.21.15.38:0/689135985 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdffc003b10 con 0x7fe008104c10 2024-03-24T00:36:28.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.602+0000 7fe005600700 1 -- 172.21.15.38:0/689135985 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdffc01b8c0 con 0x7fe008104c10 2024-03-24T00:36:28.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.602+0000 7fe005600700 1 -- 172.21.15.38:0/689135985 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdffc0197e0 con 0x7fe008104c10 2024-03-24T00:36:28.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.602+0000 7fe00fc00700 1 -- 172.21.15.38:0/689135985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00819c9d0 con 0x7fe008104c10 2024-03-24T00:36:28.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.603+0000 7fe00fc00700 1 -- 172.21.15.38:0/689135985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00819ce70 con 0x7fe008104c10 2024-03-24T00:36:28.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.604+0000 7fe00fc00700 1 -- 172.21.15.38:0/689135985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe008195fe0 con 0x7fe008104c10 2024-03-24T00:36:28.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.604+0000 7fe005600700 1 -- 172.21.15.38:0/689135985 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fdffc01ba30 con 0x7fe008104c10 2024-03-24T00:36:28.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.604+0000 7fe005600700 1 --2- 172.21.15.38:0/689135985 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fdff0038770 0x7fdff003ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:28.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.604+0000 7fe005600700 1 -- 172.21.15.38:0/689135985 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdffc04c980 con 0x7fe008104c10 2024-03-24T00:36:28.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.605+0000 7fe007e00700 1 --2- 172.21.15.38:0/689135985 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fdff0038770 0x7fdff003ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.605+0000 7fe007e00700 1 --2- 172.21.15.38:0/689135985 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fdff0038770 0x7fdff003ac30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdff4006fd0 tx=0x7fdff4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.609+0000 7fe005600700 1 -- 172.21.15.38:0/689135985 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdffc02cd20 con 0x7fe008104c10 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:22.835+0000 7fe00fc00700 1 -- 172.21.15.38:0/689135985 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0080611f0 con 0x7fdff0038770 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:27.733+0000 7fe005600700 1 -- 172.21.15.38:0/689135985 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fe0080611f0 con 0x7fdff0038770 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:27.737+0000 7fe00fc00700 1 -- 172.21.15.38:0/689135985 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fdff0038770 msgr2=0x7fdff003ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:27.737+0000 7fe00fc00700 1 --2- 172.21.15.38:0/689135985 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fdff0038770 0x7fdff003ac30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdff4006fd0 tx=0x7fdff4006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:27.737+0000 7fe00fc00700 1 -- 172.21.15.38:0/689135985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 msgr2=0x7fe00819c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:27.737+0000 7fe00fc00700 1 --2- 172.21.15.38:0/689135985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe00819c290 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdffc011710 tx=0x7fdffc0047c0 comp rx=0 tx=0).stop 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:27.737+0000 7fe00fc00700 1 -- 172.21.15.38:0/689135985 shutdown_connections 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:27.737+0000 7fe00fc00700 1 --2- 172.21.15.38:0/689135985 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fdff0038770 0x7fdff003ac30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:27.737+0000 7fe00fc00700 1 --2- 172.21.15.38:0/689135985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe008104c10 0x7fe00819c290 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:27.737+0000 7fe00fc00700 1 -- 172.21.15.38:0/689135985 >> 172.21.15.38:0/689135985 conn(0x7fe008100810 msgr2=0x7fe008193120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:27.737+0000 7fe00fc00700 1 -- 172.21.15.38:0/689135985 shutdown_connections 2024-03-24T00:36:28.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:27.737+0000 7fe00fc00700 1 -- 172.21.15.38:0/689135985 wait complete. 2024-03-24T00:36:28.499 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying mon service with default placement... 2024-03-24T00:36:29.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:28 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:29.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:28 smithi038 ceph-mon[29271]: Added host smithi038 2024-03-24T00:36:29.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:28 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:36:30.401 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:30 smithi038 ceph-mon[29271]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:30.401 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:30 smithi038 ceph-mon[29271]: Saving service mon spec with placement count:5 2024-03-24T00:36:30.401 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:30 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:30.670 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:30 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:30.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-24T00:36:30.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.131+0000 7fe8dca00700 1 Processor -- start 2024-03-24T00:36:30.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.131+0000 7fe8dca00700 1 -- start start 2024-03-24T00:36:30.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.132+0000 7fe8dca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d8108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:30.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.132+0000 7fe8dca00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8108f50 con 0x7fe8d8108560 2024-03-24T00:36:30.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.132+0000 7fe8d5e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d8108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:30.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.132+0000 7fe8d5e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d8108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43156/0 (socket says 172.21.15.38:43156) 2024-03-24T00:36:30.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.132+0000 7fe8d5e00700 1 -- 172.21.15.38:0/542072634 learned_addr learned my addr 172.21.15.38:0/542072634 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:30.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.133+0000 7fe8d5e00700 1 -- 172.21.15.38:0/542072634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d8109760 con 0x7fe8d8108560 2024-03-24T00:36:30.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.133+0000 7fe8d5e00700 1 --2- 172.21.15.38:0/542072634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d8108980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80098a0 tx=0x7fe8c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ea6c7609471ab0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:30.687 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.133+0000 7fe8d4a00700 1 -- 172.21.15.38:0/542072634 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8c8004180 con 0x7fe8d8108560 2024-03-24T00:36:30.687 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.133+0000 7fe8d4a00700 1 -- 172.21.15.38:0/542072634 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe8c80042e0 con 0x7fe8d8108560 2024-03-24T00:36:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.134+0000 7fe8d4a00700 1 -- 172.21.15.38:0/542072634 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8c8011a70 con 0x7fe8d8108560 2024-03-24T00:36:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.134+0000 7fe8dca00700 1 -- 172.21.15.38:0/542072634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 msgr2=0x7fe8d8108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.134+0000 7fe8dca00700 1 --2- 172.21.15.38:0/542072634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d8108980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80098a0 tx=0x7fe8c8009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.134+0000 7fe8dca00700 1 -- 172.21.15.38:0/542072634 shutdown_connections 2024-03-24T00:36:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.134+0000 7fe8dca00700 1 --2- 172.21.15.38:0/542072634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d8108980 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.134+0000 7fe8dca00700 1 -- 172.21.15.38:0/542072634 >> 172.21.15.38:0/542072634 conn(0x7fe8d807b920 msgr2=0x7fe8d8106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.135+0000 7fe8dca00700 1 -- 172.21.15.38:0/542072634 shutdown_connections 2024-03-24T00:36:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.135+0000 7fe8dca00700 1 -- 172.21.15.38:0/542072634 wait complete. 2024-03-24T00:36:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.136+0000 7fe8dca00700 1 Processor -- start 2024-03-24T00:36:30.688 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.136+0000 7fe8dca00700 1 -- start start 2024-03-24T00:36:30.689 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.136+0000 7fe8dca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:30.689 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.136+0000 7fe8dca00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d819c830 con 0x7fe8d8108560 2024-03-24T00:36:30.689 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.136+0000 7fe8d5e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:30.689 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.137+0000 7fe8d5e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d819c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43158/0 (socket says 172.21.15.38:43158) 2024-03-24T00:36:30.691 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.137+0000 7fe8d5e00700 1 -- 172.21.15.38:0/1506265122 learned_addr learned my addr 172.21.15.38:0/1506265122 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:30.691 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.137+0000 7fe8d5e00700 1 -- 172.21.15.38:0/1506265122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c8009550 con 0x7fe8d8108560 2024-03-24T00:36:30.691 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.137+0000 7fe8d5e00700 1 --2- 172.21.15.38:0/1506265122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d819c2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8011710 tx=0x7fe8c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:30.691 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.137+0000 7fe8cea00700 1 -- 172.21.15.38:0/1506265122 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8c8003b10 con 0x7fe8d8108560 2024-03-24T00:36:30.691 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.138+0000 7fe8dca00700 1 -- 172.21.15.38:0/1506265122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d819ca30 con 0x7fe8d8108560 2024-03-24T00:36:30.691 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.138+0000 7fe8cea00700 1 -- 172.21.15.38:0/1506265122 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe8c801b8c0 con 0x7fe8d8108560 2024-03-24T00:36:30.691 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.138+0000 7fe8cea00700 1 -- 172.21.15.38:0/1506265122 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8c8019770 con 0x7fe8d8108560 2024-03-24T00:36:30.691 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.138+0000 7fe8dca00700 1 -- 172.21.15.38:0/1506265122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d819ced0 con 0x7fe8d8108560 2024-03-24T00:36:30.691 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.139+0000 7fe8dca00700 1 -- 172.21.15.38:0/1506265122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d810c990 con 0x7fe8d8108560 2024-03-24T00:36:30.692 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.139+0000 7fe8cea00700 1 -- 172.21.15.38:0/1506265122 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fe8c801b440 con 0x7fe8d8108560 2024-03-24T00:36:30.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.140+0000 7fe8cea00700 1 --2- 172.21.15.38:0/1506265122 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fe8bc038720 0x7fe8bc03abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:30.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.140+0000 7fe8cea00700 1 -- 172.21.15.38:0/1506265122 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe8c804c9c0 con 0x7fe8d8108560 2024-03-24T00:36:30.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.140+0000 7fe8d5400700 1 --2- 172.21.15.38:0/1506265122 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fe8bc038720 0x7fe8bc03abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:30.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.141+0000 7fe8d5400700 1 --2- 172.21.15.38:0/1506265122 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fe8bc038720 0x7fe8bc03abe0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4006fd0 tx=0x7fe8c4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:30.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.149+0000 7fe8cea00700 1 -- 172.21.15.38:0/1506265122 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe8c8018530 con 0x7fe8d8108560 2024-03-24T00:36:30.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.390+0000 7fe8dca00700 1 -- 172.21.15.38:0/1506265122 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8d81062c0 con 0x7fe8bc038720 2024-03-24T00:36:30.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.400+0000 7fe8cea00700 1 -- 172.21.15.38:0/1506265122 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe8d81062c0 con 0x7fe8bc038720 2024-03-24T00:36:30.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.402+0000 7fe8dca00700 1 -- 172.21.15.38:0/1506265122 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fe8bc038720 msgr2=0x7fe8bc03abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:30.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.402+0000 7fe8dca00700 1 --2- 172.21.15.38:0/1506265122 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fe8bc038720 0x7fe8bc03abe0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4006fd0 tx=0x7fe8c4006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:30.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.402+0000 7fe8dca00700 1 -- 172.21.15.38:0/1506265122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 msgr2=0x7fe8d819c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:30.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.402+0000 7fe8dca00700 1 --2- 172.21.15.38:0/1506265122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d819c2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8011710 tx=0x7fe8c80047c0 comp rx=0 tx=0).stop 2024-03-24T00:36:30.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.403+0000 7fe8dca00700 1 -- 172.21.15.38:0/1506265122 shutdown_connections 2024-03-24T00:36:30.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.403+0000 7fe8dca00700 1 --2- 172.21.15.38:0/1506265122 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fe8bc038720 0x7fe8bc03abe0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:30.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.403+0000 7fe8dca00700 1 --2- 172.21.15.38:0/1506265122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8d8108560 0x7fe8d819c2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:30.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.403+0000 7fe8dca00700 1 -- 172.21.15.38:0/1506265122 >> 172.21.15.38:0/1506265122 conn(0x7fe8d807b920 msgr2=0x7fe8d8105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:30.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.403+0000 7fe8dca00700 1 -- 172.21.15.38:0/1506265122 shutdown_connections 2024-03-24T00:36:30.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:29.403+0000 7fe8dca00700 1 -- 172.21.15.38:0/1506265122 wait complete. 2024-03-24T00:36:30.695 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying mgr service with default placement... 2024-03-24T00:36:32.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-24T00:36:32.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.394+0000 7fd877e00700 1 Processor -- start 2024-03-24T00:36:32.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.394+0000 7fd877e00700 1 -- start start 2024-03-24T00:36:32.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.394+0000 7fd877e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd878106590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:32.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.394+0000 7fd877e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd878106b60 con 0x7fd878106170 2024-03-24T00:36:32.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.395+0000 7fd876a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd878106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:32.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.395+0000 7fd876a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd878106590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43172/0 (socket says 172.21.15.38:43172) 2024-03-24T00:36:32.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.395+0000 7fd876a00700 1 -- 172.21.15.38:0/2578431798 learned_addr learned my addr 172.21.15.38:0/2578431798 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:32.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.395+0000 7fd876a00700 1 -- 172.21.15.38:0/2578431798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd878107370 con 0x7fd878106170 2024-03-24T00:36:32.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.396+0000 7fd876a00700 1 --2- 172.21.15.38:0/2578431798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd878106590 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd8680098a0 tx=0x7fd868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6685df7ca41100e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:32.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.396+0000 7fd875600700 1 -- 172.21.15.38:0/2578431798 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd868004180 con 0x7fd878106170 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.396+0000 7fd875600700 1 -- 172.21.15.38:0/2578431798 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8680042e0 con 0x7fd878106170 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.396+0000 7fd875600700 1 -- 172.21.15.38:0/2578431798 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd868011b10 con 0x7fd878106170 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.397+0000 7fd877e00700 1 -- 172.21.15.38:0/2578431798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 msgr2=0x7fd878106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.397+0000 7fd877e00700 1 --2- 172.21.15.38:0/2578431798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd878106590 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd8680098a0 tx=0x7fd868009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.397+0000 7fd877e00700 1 -- 172.21.15.38:0/2578431798 shutdown_connections 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.397+0000 7fd877e00700 1 --2- 172.21.15.38:0/2578431798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd878106590 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.397+0000 7fd877e00700 1 -- 172.21.15.38:0/2578431798 >> 172.21.15.38:0/2578431798 conn(0x7fd8781016d0 msgr2=0x7fd878103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.397+0000 7fd877e00700 1 -- 172.21.15.38:0/2578431798 shutdown_connections 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.397+0000 7fd877e00700 1 -- 172.21.15.38:0/2578431798 wait complete. 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.398+0000 7fd877e00700 1 Processor -- start 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.398+0000 7fd877e00700 1 -- start start 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.398+0000 7fd877e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd87819c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.398+0000 7fd877e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87819c680 con 0x7fd878106170 2024-03-24T00:36:32.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.399+0000 7fd876a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd87819c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:32.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.399+0000 7fd876a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd87819c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43178/0 (socket says 172.21.15.38:43178) 2024-03-24T00:36:32.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.399+0000 7fd876a00700 1 -- 172.21.15.38:0/1948270869 learned_addr learned my addr 172.21.15.38:0/1948270869 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:32.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.399+0000 7fd876a00700 1 -- 172.21.15.38:0/1948270869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868009550 con 0x7fd878106170 2024-03-24T00:36:32.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.399+0000 7fd876a00700 1 --2- 172.21.15.38:0/1948270869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd87819c140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd8680117b0 tx=0x7fd868011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:32.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.400+0000 7fd87ca00700 1 -- 172.21.15.38:0/1948270869 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8680038b0 con 0x7fd878106170 2024-03-24T00:36:32.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.400+0000 7fd87ca00700 1 -- 172.21.15.38:0/1948270869 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd868003a10 con 0x7fd878106170 2024-03-24T00:36:32.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.400+0000 7fd87ca00700 1 -- 172.21.15.38:0/1948270869 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8680246e0 con 0x7fd878106170 2024-03-24T00:36:32.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.400+0000 7fd877e00700 1 -- 172.21.15.38:0/1948270869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87819c880 con 0x7fd878106170 2024-03-24T00:36:32.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.400+0000 7fd877e00700 1 -- 172.21.15.38:0/1948270869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87819cd20 con 0x7fd878106170 2024-03-24T00:36:32.301 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.401+0000 7fd877e00700 1 -- 172.21.15.38:0/1948270869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87810a5a0 con 0x7fd878106170 2024-03-24T00:36:32.301 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.401+0000 7fd87ca00700 1 -- 172.21.15.38:0/1948270869 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fd86801ab80 con 0x7fd878106170 2024-03-24T00:36:32.301 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.401+0000 7fd87ca00700 1 --2- 172.21.15.38:0/1948270869 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd85c038770 0x7fd85c03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.401+0000 7fd87ca00700 1 -- 172.21.15.38:0/1948270869 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd86804c930 con 0x7fd878106170 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.402+0000 7fd876000700 1 --2- 172.21.15.38:0/1948270869 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd85c038770 0x7fd85c03ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.402+0000 7fd876000700 1 --2- 172.21.15.38:0/1948270869 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd85c038770 0x7fd85c03ac30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd864006fd0 tx=0x7fd864006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.407+0000 7fd87ca00700 1 -- 172.21.15.38:0/1948270869 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd86801a430 con 0x7fd878106170 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.639+0000 7fd877e00700 1 -- 172.21.15.38:0/1948270869 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fd878102940 con 0x7fd85c038770 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.648+0000 7fd87ca00700 1 -- 172.21.15.38:0/1948270869 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd878102940 con 0x7fd85c038770 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.651+0000 7fd877e00700 1 -- 172.21.15.38:0/1948270869 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd85c038770 msgr2=0x7fd85c03ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.651+0000 7fd877e00700 1 --2- 172.21.15.38:0/1948270869 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd85c038770 0x7fd85c03ac30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd864006fd0 tx=0x7fd864006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.651+0000 7fd877e00700 1 -- 172.21.15.38:0/1948270869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 msgr2=0x7fd87819c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.651+0000 7fd877e00700 1 --2- 172.21.15.38:0/1948270869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd87819c140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd8680117b0 tx=0x7fd868011890 comp rx=0 tx=0).stop 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.651+0000 7fd877e00700 1 -- 172.21.15.38:0/1948270869 shutdown_connections 2024-03-24T00:36:32.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.651+0000 7fd877e00700 1 --2- 172.21.15.38:0/1948270869 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd85c038770 0x7fd85c03ac30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:32.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.651+0000 7fd877e00700 1 --2- 172.21.15.38:0/1948270869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd878106170 0x7fd87819c140 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:32.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.651+0000 7fd877e00700 1 -- 172.21.15.38:0/1948270869 >> 172.21.15.38:0/1948270869 conn(0x7fd8781016d0 msgr2=0x7fd878102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:32.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.652+0000 7fd877e00700 1 -- 172.21.15.38:0/1948270869 shutdown_connections 2024-03-24T00:36:32.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:31.652+0000 7fd877e00700 1 -- 172.21.15.38:0/1948270869 wait complete. 2024-03-24T00:36:32.303 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying crash service with default placement... 2024-03-24T00:36:32.590 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:32 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:32.591 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:32 smithi038 ceph-mon[29271]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:32.591 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:32 smithi038 ceph-mon[29271]: Saving service mgr spec with placement count:2 2024-03-24T00:36:32.591 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:32 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:34.344 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:34 smithi038 ceph-mon[29271]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:34.345 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:34 smithi038 ceph-mon[29271]: Saving service crash spec with placement * 2024-03-24T00:36:34.345 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:34 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:34.345 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:34 smithi038 ceph-mon[29271]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:36:34.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-24T00:36:34.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.893+0000 7f38e5200700 1 Processor -- start 2024-03-24T00:36:34.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.893+0000 7f38e5200700 1 -- start start 2024-03-24T00:36:34.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.894+0000 7f38e5200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e0106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:34.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.894+0000 7f38e5200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0106d40 con 0x7f38e0106350 2024-03-24T00:36:34.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.894+0000 7f38dde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e0106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:34.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.894+0000 7f38dde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e0106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43182/0 (socket says 172.21.15.38:43182) 2024-03-24T00:36:34.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.894+0000 7f38dde00700 1 -- 172.21.15.38:0/2520032121 learned_addr learned my addr 172.21.15.38:0/2520032121 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:34.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.895+0000 7f38dde00700 1 -- 172.21.15.38:0/2520032121 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e0107550 con 0x7f38e0106350 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.895+0000 7f38dde00700 1 --2- 172.21.15.38:0/2520032121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e0106770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f38d00098a0 tx=0x7f38d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a781b969cd69b74 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.895+0000 7f38dca00700 1 -- 172.21.15.38:0/2520032121 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38d0004180 con 0x7f38e0106350 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.895+0000 7f38dca00700 1 -- 172.21.15.38:0/2520032121 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38d00042e0 con 0x7f38e0106350 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.895+0000 7f38dca00700 1 -- 172.21.15.38:0/2520032121 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38d0011b10 con 0x7f38e0106350 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.896+0000 7f38e5200700 1 -- 172.21.15.38:0/2520032121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 msgr2=0x7f38e0106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.896+0000 7f38e5200700 1 --2- 172.21.15.38:0/2520032121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e0106770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f38d00098a0 tx=0x7f38d0009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.897+0000 7f38e5200700 1 -- 172.21.15.38:0/2520032121 shutdown_connections 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.897+0000 7f38e5200700 1 --2- 172.21.15.38:0/2520032121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e0106770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.897+0000 7f38e5200700 1 -- 172.21.15.38:0/2520032121 >> 172.21.15.38:0/2520032121 conn(0x7f38e01018f0 msgr2=0x7f38e0103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.897+0000 7f38e5200700 1 -- 172.21.15.38:0/2520032121 shutdown_connections 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.897+0000 7f38e5200700 1 -- 172.21.15.38:0/2520032121 wait complete. 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.898+0000 7f38e5200700 1 Processor -- start 2024-03-24T00:36:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.898+0000 7f38e5200700 1 -- start start 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.899+0000 7f38e5200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.899+0000 7f38e5200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e019c8c0 con 0x7f38e0106350 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.899+0000 7f38dde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.899+0000 7f38dde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e019c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43196/0 (socket says 172.21.15.38:43196) 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.899+0000 7f38dde00700 1 -- 172.21.15.38:0/898462958 learned_addr learned my addr 172.21.15.38:0/898462958 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.899+0000 7f38dde00700 1 -- 172.21.15.38:0/898462958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0009550 con 0x7f38e0106350 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.900+0000 7f38dde00700 1 --2- 172.21.15.38:0/898462958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e019c380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f38d00117b0 tx=0x7f38d0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.900+0000 7f38d6a00700 1 -- 172.21.15.38:0/898462958 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38d00038b0 con 0x7f38e0106350 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.900+0000 7f38e5200700 1 -- 172.21.15.38:0/898462958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e019cac0 con 0x7f38e0106350 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.900+0000 7f38e5200700 1 -- 172.21.15.38:0/898462958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e019cf60 con 0x7f38e0106350 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.900+0000 7f38d6a00700 1 -- 172.21.15.38:0/898462958 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38d0003a10 con 0x7f38e0106350 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.900+0000 7f38d6a00700 1 -- 172.21.15.38:0/898462958 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38d00246e0 con 0x7f38e0106350 2024-03-24T00:36:34.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.902+0000 7f38e5200700 1 -- 172.21.15.38:0/898462958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c00052f0 con 0x7f38e0106350 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.902+0000 7f38d6a00700 1 -- 172.21.15.38:0/898462958 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f38d001a430 con 0x7f38e0106350 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.902+0000 7f38d6a00700 1 --2- 172.21.15.38:0/898462958 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f38c40387c0 0x7f38c403ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.902+0000 7f38d6a00700 1 -- 172.21.15.38:0/898462958 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f38d004c8a0 con 0x7f38e0106350 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.903+0000 7f38dd400700 1 --2- 172.21.15.38:0/898462958 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f38c40387c0 0x7f38c403ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.904+0000 7f38dd400700 1 --2- 172.21.15.38:0/898462958 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f38c40387c0 0x7f38c403ac80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f38cc006fd0 tx=0x7f38cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:32.908+0000 7f38d6a00700 1 -- 172.21.15.38:0/898462958 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f38d0018b40 con 0x7f38e0106350 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.152+0000 7f38e5200700 1 -- 172.21.15.38:0/898462958 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f38c0000bc0 con 0x7f38c40387c0 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.164+0000 7f38d6a00700 1 -- 172.21.15.38:0/898462958 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f38c0000bc0 con 0x7f38c40387c0 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.167+0000 7f38e5200700 1 -- 172.21.15.38:0/898462958 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f38c40387c0 msgr2=0x7f38c403ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.167+0000 7f38e5200700 1 --2- 172.21.15.38:0/898462958 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f38c40387c0 0x7f38c403ac80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f38cc006fd0 tx=0x7f38cc006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.167+0000 7f38e5200700 1 -- 172.21.15.38:0/898462958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 msgr2=0x7f38e019c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.167+0000 7f38e5200700 1 --2- 172.21.15.38:0/898462958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e019c380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f38d00117b0 tx=0x7f38d0011890 comp rx=0 tx=0).stop 2024-03-24T00:36:34.367 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.167+0000 7f38e5200700 1 -- 172.21.15.38:0/898462958 shutdown_connections 2024-03-24T00:36:34.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.167+0000 7f38e5200700 1 --2- 172.21.15.38:0/898462958 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f38c40387c0 0x7f38c403ac80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:34.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.167+0000 7f38e5200700 1 --2- 172.21.15.38:0/898462958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38e0106350 0x7f38e019c380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:34.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.167+0000 7f38e5200700 1 -- 172.21.15.38:0/898462958 >> 172.21.15.38:0/898462958 conn(0x7f38e01018f0 msgr2=0x7f38e0102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:34.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.168+0000 7f38e5200700 1 -- 172.21.15.38:0/898462958 shutdown_connections 2024-03-24T00:36:34.368 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:33.168+0000 7f38e5200700 1 -- 172.21.15.38:0/898462958 wait complete. 2024-03-24T00:36:34.368 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying ceph-exporter service with default placement... 2024-03-24T00:36:35.560 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:35 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:35.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-03-24T00:36:35.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.931+0000 7f4ea9a00700 1 Processor -- start 2024-03-24T00:36:35.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.931+0000 7f4ea9a00700 1 -- start start 2024-03-24T00:36:35.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.931+0000 7f4ea9a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea4079140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:35.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.931+0000 7f4ea9a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4079710 con 0x7f4ea407ace0 2024-03-24T00:36:35.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.932+0000 7f4ea2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea4079140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:35.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.932+0000 7f4ea2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea4079140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35172/0 (socket says 172.21.15.38:35172) 2024-03-24T00:36:35.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.932+0000 7f4ea2a00700 1 -- 172.21.15.38:0/561268039 learned_addr learned my addr 172.21.15.38:0/561268039 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:35.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.933+0000 7f4ea2a00700 1 -- 172.21.15.38:0/561268039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea4079850 con 0x7f4ea407ace0 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.933+0000 7f4ea2a00700 1 --2- 172.21.15.38:0/561268039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea4079140 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009e80 tx=0x7f4e9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46536aad03dc29a server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.933+0000 7f4ea2000700 1 -- 172.21.15.38:0/561268039 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e9400e070 con 0x7f4ea407ace0 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.933+0000 7f4ea2000700 1 -- 172.21.15.38:0/561268039 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e9400b8c0 con 0x7f4ea407ace0 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.933+0000 7f4ea2000700 1 -- 172.21.15.38:0/561268039 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e940044d0 con 0x7f4ea407ace0 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.934+0000 7f4ea9a00700 1 -- 172.21.15.38:0/561268039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 msgr2=0x7f4ea4079140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.934+0000 7f4ea9a00700 1 --2- 172.21.15.38:0/561268039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea4079140 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009e80 tx=0x7f4e9400b330 comp rx=0 tx=0).stop 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.934+0000 7f4ea9a00700 1 -- 172.21.15.38:0/561268039 shutdown_connections 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.934+0000 7f4ea9a00700 1 --2- 172.21.15.38:0/561268039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea4079140 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.934+0000 7f4ea9a00700 1 -- 172.21.15.38:0/561268039 >> 172.21.15.38:0/561268039 conn(0x7f4ea41018e0 msgr2=0x7f4ea4103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.934+0000 7f4ea9a00700 1 -- 172.21.15.38:0/561268039 shutdown_connections 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.934+0000 7f4ea9a00700 1 -- 172.21.15.38:0/561268039 wait complete. 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.935+0000 7f4ea9a00700 1 Processor -- start 2024-03-24T00:36:35.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.936+0000 7f4ea9a00700 1 -- start start 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.936+0000 7f4ea9a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.936+0000 7f4ea9a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea419c7b0 con 0x7f4ea407ace0 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.936+0000 7f4ea2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.936+0000 7f4ea2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea419c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35174/0 (socket says 172.21.15.38:35174) 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.936+0000 7f4ea2a00700 1 -- 172.21.15.38:0/2533071885 learned_addr learned my addr 172.21.15.38:0/2533071885 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.937+0000 7f4ea2a00700 1 -- 172.21.15.38:0/2533071885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e94009b30 con 0x7f4ea407ace0 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.937+0000 7f4ea2a00700 1 --2- 172.21.15.38:0/2533071885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea419c270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e940072a0 tx=0x7f4e9400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.937+0000 7f4e9be00700 1 -- 172.21.15.38:0/2533071885 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e9400e040 con 0x7f4ea407ace0 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.937+0000 7f4e9be00700 1 -- 172.21.15.38:0/2533071885 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e94003dc0 con 0x7f4ea407ace0 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.937+0000 7f4e9be00700 1 -- 172.21.15.38:0/2533071885 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e94012750 con 0x7f4ea407ace0 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.937+0000 7f4ea9a00700 1 -- 172.21.15.38:0/2533071885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea419c9b0 con 0x7f4ea407ace0 2024-03-24T00:36:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.937+0000 7f4ea9a00700 1 -- 172.21.15.38:0/2533071885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea419ceb0 con 0x7f4ea407ace0 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.938+0000 7f4ea9a00700 1 -- 172.21.15.38:0/2533071885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea410c720 con 0x7f4ea407ace0 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.939+0000 7f4e9be00700 1 -- 172.21.15.38:0/2533071885 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f4e94019040 con 0x7f4ea407ace0 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.939+0000 7f4e9be00700 1 --2- 172.21.15.38:0/2533071885 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4e88038770 0x7f4e8803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.939+0000 7f4e9be00700 1 -- 172.21.15.38:0/2533071885 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4e9404c180 con 0x7f4ea407ace0 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.940+0000 7f4e9a000700 1 --2- 172.21.15.38:0/2533071885 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4e88038770 0x7f4e8803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.940+0000 7f4e9a000700 1 --2- 172.21.15.38:0/2533071885 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4e88038770 0x7f4e8803ac30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c006fd0 tx=0x7f4e8c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:34.945+0000 7f4e9be00700 1 -- 172.21.15.38:0/2533071885 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4e9401a750 con 0x7f4ea407ace0 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.204+0000 7f4ea9a00700 1 -- 172.21.15.38:0/2533071885 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ea419fbc0 con 0x7f4e88038770 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.215+0000 7f4e9be00700 1 -- 172.21.15.38:0/2533071885 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f4ea419fbc0 con 0x7f4e88038770 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.218+0000 7f4ea9a00700 1 -- 172.21.15.38:0/2533071885 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4e88038770 msgr2=0x7f4e8803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.218+0000 7f4ea9a00700 1 --2- 172.21.15.38:0/2533071885 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4e88038770 0x7f4e8803ac30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c006fd0 tx=0x7f4e8c006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.218+0000 7f4ea9a00700 1 -- 172.21.15.38:0/2533071885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 msgr2=0x7f4ea419c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:35.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.218+0000 7f4ea9a00700 1 --2- 172.21.15.38:0/2533071885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea419c270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e940072a0 tx=0x7f4e9400be00 comp rx=0 tx=0).stop 2024-03-24T00:36:35.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.218+0000 7f4ea9a00700 1 -- 172.21.15.38:0/2533071885 shutdown_connections 2024-03-24T00:36:35.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.218+0000 7f4ea9a00700 1 --2- 172.21.15.38:0/2533071885 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f4e88038770 0x7f4e8803ac30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:35.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.218+0000 7f4ea9a00700 1 --2- 172.21.15.38:0/2533071885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea407ace0 0x7f4ea419c270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:35.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.218+0000 7f4ea9a00700 1 -- 172.21.15.38:0/2533071885 >> 172.21.15.38:0/2533071885 conn(0x7f4ea41018e0 msgr2=0x7f4ea4102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:35.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.219+0000 7f4ea9a00700 1 -- 172.21.15.38:0/2533071885 shutdown_connections 2024-03-24T00:36:35.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:35.219+0000 7f4ea9a00700 1 -- 172.21.15.38:0/2533071885 wait complete. 2024-03-24T00:36:35.870 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying prometheus service with default placement... 2024-03-24T00:36:36.457 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:36 smithi038 ceph-mon[29271]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:36.457 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:36 smithi038 ceph-mon[29271]: Saving service ceph-exporter spec with placement * 2024-03-24T00:36:36.457 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:36 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:36.457 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:36 smithi038 ceph-mon[29271]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:36:37.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:37 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:37.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:37 smithi038 ceph-mon[29271]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:37.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:37 smithi038 ceph-mon[29271]: Saving service prometheus spec with placement count:1 2024-03-24T00:36:37.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:37 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:37.428 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.455+0000 7f08f6400700 1 Processor -- start 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.455+0000 7f08f6400700 1 -- start start 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.456+0000 7f08f6400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f01067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.456+0000 7f08f6400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f0106da0 con 0x7f08f01063b0 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.456+0000 7f08ef400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f01067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.456+0000 7f08ef400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f01067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35176/0 (socket says 172.21.15.38:35176) 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.456+0000 7f08ef400700 1 -- 172.21.15.38:0/1465408981 learned_addr learned my addr 172.21.15.38:0/1465408981 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.457+0000 7f08ef400700 1 -- 172.21.15.38:0/1465408981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f01075b0 con 0x7f08f01063b0 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.457+0000 7f08ef400700 1 --2- 172.21.15.38:0/1465408981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f01067d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08e00098a0 tx=0x7f08e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139d4bb6a9369c06 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.458+0000 7f08eea00700 1 -- 172.21.15.38:0/1465408981 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08e0004180 con 0x7f08f01063b0 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.458+0000 7f08eea00700 1 -- 172.21.15.38:0/1465408981 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08e00042e0 con 0x7f08f01063b0 2024-03-24T00:36:37.429 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.458+0000 7f08eea00700 1 -- 172.21.15.38:0/1465408981 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08e0011b10 con 0x7f08f01063b0 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.458+0000 7f08f6400700 1 -- 172.21.15.38:0/1465408981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 msgr2=0x7f08f01067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.458+0000 7f08f6400700 1 --2- 172.21.15.38:0/1465408981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f01067d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08e00098a0 tx=0x7f08e0009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.459+0000 7f08f6400700 1 -- 172.21.15.38:0/1465408981 shutdown_connections 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.459+0000 7f08f6400700 1 --2- 172.21.15.38:0/1465408981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f01067d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.459+0000 7f08f6400700 1 -- 172.21.15.38:0/1465408981 >> 172.21.15.38:0/1465408981 conn(0x7f08f01019b0 msgr2=0x7f08f0103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.459+0000 7f08f6400700 1 -- 172.21.15.38:0/1465408981 shutdown_connections 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.459+0000 7f08f6400700 1 -- 172.21.15.38:0/1465408981 wait complete. 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.460+0000 7f08f6400700 1 Processor -- start 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.460+0000 7f08f6400700 1 -- start start 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.460+0000 7f08f6400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f007a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.460+0000 7f08f6400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f007aec0 con 0x7f08f01063b0 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.461+0000 7f08ef400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f007a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:37.430 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.461+0000 7f08ef400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f007a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35186/0 (socket says 172.21.15.38:35186) 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.461+0000 7f08ef400700 1 -- 172.21.15.38:0/983491071 learned_addr learned my addr 172.21.15.38:0/983491071 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.461+0000 7f08ef400700 1 -- 172.21.15.38:0/983491071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e0009550 con 0x7f08f01063b0 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.462+0000 7f08ef400700 1 --2- 172.21.15.38:0/983491071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f007a980 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08e0000c00 tx=0x7f08e0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.462+0000 7f08ecc00700 1 -- 172.21.15.38:0/983491071 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08e00038b0 con 0x7f08f01063b0 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.462+0000 7f08ecc00700 1 -- 172.21.15.38:0/983491071 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08e0003a10 con 0x7f08f01063b0 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.462+0000 7f08f6400700 1 -- 172.21.15.38:0/983491071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f0079100 con 0x7f08f01063b0 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.462+0000 7f08ecc00700 1 -- 172.21.15.38:0/983491071 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08e0024780 con 0x7f08f01063b0 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.462+0000 7f08f6400700 1 -- 172.21.15.38:0/983491071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f00795a0 con 0x7f08f01063b0 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.463+0000 7f08f6400700 1 -- 172.21.15.38:0/983491071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f00623e0 con 0x7f08f01063b0 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.464+0000 7f08ecc00700 1 -- 172.21.15.38:0/983491071 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f08e001ac10 con 0x7f08f01063b0 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.464+0000 7f08ecc00700 1 --2- 172.21.15.38:0/983491071 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f08d4038770 0x7f08d403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.464+0000 7f08ecc00700 1 -- 172.21.15.38:0/983491071 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f08e004c560 con 0x7f08f01063b0 2024-03-24T00:36:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.465+0000 7f08e7e00700 1 --2- 172.21.15.38:0/983491071 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f08d4038770 0x7f08d403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.466+0000 7f08e7e00700 1 --2- 172.21.15.38:0/983491071 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f08d4038770 0x7f08d403ac30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08d8006fd0 tx=0x7f08d8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.470+0000 7f08ecc00700 1 -- 172.21.15.38:0/983491071 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f08e001a430 con 0x7f08f01063b0 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.707+0000 7f08f6400700 1 -- 172.21.15.38:0/983491071 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f08f010a7b0 con 0x7f08d4038770 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.717+0000 7f08ecc00700 1 -- 172.21.15.38:0/983491071 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f08f010a7b0 con 0x7f08d4038770 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.720+0000 7f08f6400700 1 -- 172.21.15.38:0/983491071 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f08d4038770 msgr2=0x7f08d403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.720+0000 7f08f6400700 1 --2- 172.21.15.38:0/983491071 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f08d4038770 0x7f08d403ac30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08d8006fd0 tx=0x7f08d8006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.720+0000 7f08f6400700 1 -- 172.21.15.38:0/983491071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 msgr2=0x7f08f007a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.720+0000 7f08f6400700 1 --2- 172.21.15.38:0/983491071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f007a980 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08e0000c00 tx=0x7f08e0011890 comp rx=0 tx=0).stop 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.720+0000 7f08f6400700 1 -- 172.21.15.38:0/983491071 shutdown_connections 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.720+0000 7f08f6400700 1 --2- 172.21.15.38:0/983491071 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f08d4038770 0x7f08d403ac30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.720+0000 7f08f6400700 1 --2- 172.21.15.38:0/983491071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f01063b0 0x7f08f007a980 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.720+0000 7f08f6400700 1 -- 172.21.15.38:0/983491071 >> 172.21.15.38:0/983491071 conn(0x7f08f01019b0 msgr2=0x7f08f0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.721+0000 7f08f6400700 1 -- 172.21.15.38:0/983491071 shutdown_connections 2024-03-24T00:36:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:36.721+0000 7f08f6400700 1 -- 172.21.15.38:0/983491071 wait complete. 2024-03-24T00:36:37.433 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying grafana service with default placement... 2024-03-24T00:36:38.334 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:38 smithi038 ceph-mon[29271]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:36:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-24T00:36:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.074+0000 7f689f200700 1 Processor -- start 2024-03-24T00:36:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.074+0000 7f689f200700 1 -- start start 2024-03-24T00:36:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.075+0000 7f689f200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f68981065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.075+0000 7f689f200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6898106b90 con 0x7f68981061a0 2024-03-24T00:36:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.075+0000 7f689de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f68981065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.075+0000 7f689de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f68981065c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35194/0 (socket says 172.21.15.38:35194) 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.075+0000 7f689de00700 1 -- 172.21.15.38:0/2690510323 learned_addr learned my addr 172.21.15.38:0/2690510323 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.076+0000 7f689de00700 1 -- 172.21.15.38:0/2690510323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68981073a0 con 0x7f68981061a0 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.076+0000 7f689de00700 1 --2- 172.21.15.38:0/2690510323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f68981065c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f688c0098a0 tx=0x7f688c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9887e9be23e87492 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.077+0000 7f689ca00700 1 -- 172.21.15.38:0/2690510323 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f688c004180 con 0x7f68981061a0 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.077+0000 7f689ca00700 1 -- 172.21.15.38:0/2690510323 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f688c0042e0 con 0x7f68981061a0 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.077+0000 7f689ca00700 1 -- 172.21.15.38:0/2690510323 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f688c011b10 con 0x7f68981061a0 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.077+0000 7f689f200700 1 -- 172.21.15.38:0/2690510323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 msgr2=0x7f68981065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.077+0000 7f689f200700 1 --2- 172.21.15.38:0/2690510323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f68981065c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f688c0098a0 tx=0x7f688c009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.078+0000 7f689f200700 1 -- 172.21.15.38:0/2690510323 shutdown_connections 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.078+0000 7f689f200700 1 --2- 172.21.15.38:0/2690510323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f68981065c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.078+0000 7f689f200700 1 -- 172.21.15.38:0/2690510323 >> 172.21.15.38:0/2690510323 conn(0x7f6898101720 msgr2=0x7f6898103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.078+0000 7f689f200700 1 -- 172.21.15.38:0/2690510323 shutdown_connections 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.078+0000 7f689f200700 1 -- 172.21.15.38:0/2690510323 wait complete. 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.079+0000 7f689f200700 1 Processor -- start 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.079+0000 7f689f200700 1 -- start start 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.079+0000 7f689f200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f689807ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.079+0000 7f689f200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689807bfd0 con 0x7f68981061a0 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.080+0000 7f689de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f689807ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.080+0000 7f689de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f689807ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35210/0 (socket says 172.21.15.38:35210) 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.080+0000 7f689de00700 1 -- 172.21.15.38:0/885222935 learned_addr learned my addr 172.21.15.38:0/885222935 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.080+0000 7f689de00700 1 -- 172.21.15.38:0/885222935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c009550 con 0x7f68981061a0 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.080+0000 7f689de00700 1 --2- 172.21.15.38:0/885222935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f689807ba90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f688c0117b0 tx=0x7f688c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.081+0000 7f6892a00700 1 -- 172.21.15.38:0/885222935 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f688c0038b0 con 0x7f68981061a0 2024-03-24T00:36:39.230 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.081+0000 7f6892a00700 1 -- 172.21.15.38:0/885222935 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f688c003a10 con 0x7f68981061a0 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.081+0000 7f6892a00700 1 -- 172.21.15.38:0/885222935 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f688c0246e0 con 0x7f68981061a0 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.081+0000 7f689f200700 1 -- 172.21.15.38:0/885222935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689807a210 con 0x7f68981061a0 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.081+0000 7f689f200700 1 -- 172.21.15.38:0/885222935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689807a6b0 con 0x7f68981061a0 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.082+0000 7f689f200700 1 -- 172.21.15.38:0/885222935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68980623e0 con 0x7f68981061a0 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.082+0000 7f6892a00700 1 -- 172.21.15.38:0/885222935 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f688c01ab80 con 0x7f68981061a0 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.083+0000 7f6892a00700 1 --2- 172.21.15.38:0/885222935 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f6880038770 0x7f688003ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.083+0000 7f6892a00700 1 -- 172.21.15.38:0/885222935 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f688c04c9f0 con 0x7f68981061a0 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.083+0000 7f689d400700 1 --2- 172.21.15.38:0/885222935 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f6880038770 0x7f688003ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.084+0000 7f689d400700 1 --2- 172.21.15.38:0/885222935 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f6880038770 0x7f688003ac30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6888006fd0 tx=0x7f6888006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.088+0000 7f6892a00700 1 -- 172.21.15.38:0/885222935 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f688c010380 con 0x7f68981061a0 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.333+0000 7f689f200700 1 -- 172.21.15.38:0/885222935 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f689807b090 con 0x7f6880038770 2024-03-24T00:36:39.231 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.340+0000 7f6892a00700 1 -- 172.21.15.38:0/885222935 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f689807b090 con 0x7f6880038770 2024-03-24T00:36:39.232 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.343+0000 7f689f200700 1 -- 172.21.15.38:0/885222935 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f6880038770 msgr2=0x7f688003ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:39.232 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.343+0000 7f689f200700 1 --2- 172.21.15.38:0/885222935 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f6880038770 0x7f688003ac30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6888006fd0 tx=0x7f6888006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:39.232 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.343+0000 7f689f200700 1 -- 172.21.15.38:0/885222935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 msgr2=0x7f689807ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:39.232 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.343+0000 7f689f200700 1 --2- 172.21.15.38:0/885222935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f689807ba90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f688c0117b0 tx=0x7f688c011890 comp rx=0 tx=0).stop 2024-03-24T00:36:39.232 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.344+0000 7f689f200700 1 -- 172.21.15.38:0/885222935 shutdown_connections 2024-03-24T00:36:39.232 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.344+0000 7f689f200700 1 --2- 172.21.15.38:0/885222935 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f6880038770 0x7f688003ac30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:39.232 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.344+0000 7f689f200700 1 --2- 172.21.15.38:0/885222935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68981061a0 0x7f689807ba90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:39.232 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.344+0000 7f689f200700 1 -- 172.21.15.38:0/885222935 >> 172.21.15.38:0/885222935 conn(0x7f6898101720 msgr2=0x7f68981022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:39.232 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.344+0000 7f689f200700 1 -- 172.21.15.38:0/885222935 shutdown_connections 2024-03-24T00:36:39.232 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:38.344+0000 7f689f200700 1 -- 172.21.15.38:0/885222935 wait complete. 2024-03-24T00:36:39.232 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying node-exporter service with default placement... 2024-03-24T00:36:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:39 smithi038 ceph-mon[29271]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:39 smithi038 ceph-mon[29271]: Saving service grafana spec with placement count:1 2024-03-24T00:36:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:39 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:40.584 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:40 smithi038 ceph-mon[29271]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:36:40.584 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:40 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:40.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-24T00:36:40.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.025+0000 7fd534600700 1 Processor -- start 2024-03-24T00:36:40.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.026+0000 7fd534600700 1 -- start start 2024-03-24T00:36:40.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.026+0000 7fd534600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c10e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:40.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.026+0000 7fd534600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c10ea00 con 0x7fd52c10dfd0 2024-03-24T00:36:40.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.027+0000 7fd531c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c10e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:40.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.027+0000 7fd531c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c10e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35212/0 (socket says 172.21.15.38:35212) 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.027+0000 7fd531c00700 1 -- 172.21.15.38:0/1163838267 learned_addr learned my addr 172.21.15.38:0/1163838267 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.027+0000 7fd531c00700 1 -- 172.21.15.38:0/1163838267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c10cc40 con 0x7fd52c10dfd0 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.028+0000 7fd531c00700 1 --2- 172.21.15.38:0/1163838267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c10e3f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd5200098a0 tx=0x7fd520009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f5d960a94e82ad server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.028+0000 7fd52be00700 1 -- 172.21.15.38:0/1163838267 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd520004180 con 0x7fd52c10dfd0 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.028+0000 7fd52be00700 1 -- 172.21.15.38:0/1163838267 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5200042e0 con 0x7fd52c10dfd0 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.028+0000 7fd52be00700 1 -- 172.21.15.38:0/1163838267 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd520011b10 con 0x7fd52c10dfd0 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.029+0000 7fd534600700 1 -- 172.21.15.38:0/1163838267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 msgr2=0x7fd52c10e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.029+0000 7fd534600700 1 --2- 172.21.15.38:0/1163838267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c10e3f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd5200098a0 tx=0x7fd520009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.029+0000 7fd534600700 1 -- 172.21.15.38:0/1163838267 shutdown_connections 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.029+0000 7fd534600700 1 --2- 172.21.15.38:0/1163838267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c10e3f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.029+0000 7fd534600700 1 -- 172.21.15.38:0/1163838267 >> 172.21.15.38:0/1163838267 conn(0x7fd52c07b920 msgr2=0x7fd52c07dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.029+0000 7fd534600700 1 -- 172.21.15.38:0/1163838267 shutdown_connections 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.029+0000 7fd534600700 1 -- 172.21.15.38:0/1163838267 wait complete. 2024-03-24T00:36:40.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.030+0000 7fd534600700 1 Processor -- start 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.030+0000 7fd534600700 1 -- start start 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.030+0000 7fd534600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.030+0000 7fd534600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c1a0be0 con 0x7fd52c10dfd0 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.031+0000 7fd531c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.031+0000 7fd531c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35220/0 (socket says 172.21.15.38:35220) 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.031+0000 7fd531c00700 1 -- 172.21.15.38:0/671406533 learned_addr learned my addr 172.21.15.38:0/671406533 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.031+0000 7fd531c00700 1 -- 172.21.15.38:0/671406533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520009550 con 0x7fd52c10dfd0 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.031+0000 7fd531c00700 1 --2- 172.21.15.38:0/671406533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c1a06a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd5200117b0 tx=0x7fd520011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.032+0000 7fd52a000700 1 -- 172.21.15.38:0/671406533 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5200038b0 con 0x7fd52c10dfd0 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.032+0000 7fd52a000700 1 -- 172.21.15.38:0/671406533 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd520003a10 con 0x7fd52c10dfd0 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.032+0000 7fd534600700 1 -- 172.21.15.38:0/671406533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c1a0de0 con 0x7fd52c10dfd0 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.032+0000 7fd52a000700 1 -- 172.21.15.38:0/671406533 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd520024780 con 0x7fd52c10dfd0 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.032+0000 7fd534600700 1 -- 172.21.15.38:0/671406533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c1a1280 con 0x7fd52c10dfd0 2024-03-24T00:36:40.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.033+0000 7fd52a000700 1 -- 172.21.15.38:0/671406533 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fd52001ac10 con 0x7fd52c10dfd0 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.033+0000 7fd534600700 1 -- 172.21.15.38:0/671406533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c110e80 con 0x7fd52c10dfd0 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.033+0000 7fd52a000700 1 --2- 172.21.15.38:0/671406533 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd514038770 0x7fd51403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.034+0000 7fd52a000700 1 -- 172.21.15.38:0/671406533 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd52004c860 con 0x7fd52c10dfd0 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.034+0000 7fd531200700 1 --2- 172.21.15.38:0/671406533 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd514038770 0x7fd51403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.034+0000 7fd531200700 1 --2- 172.21.15.38:0/671406533 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd514038770 0x7fd51403ac30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd518006fd0 tx=0x7fd518006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.039+0000 7fd52a000700 1 -- 172.21.15.38:0/671406533 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd5200248e0 con 0x7fd52c10dfd0 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.277+0000 7fd534600700 1 -- 172.21.15.38:0/671406533 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fd52c07cb70 con 0x7fd514038770 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.289+0000 7fd52a000700 1 -- 172.21.15.38:0/671406533 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fd52c07cb70 con 0x7fd514038770 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.292+0000 7fd534600700 1 -- 172.21.15.38:0/671406533 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd514038770 msgr2=0x7fd51403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.292+0000 7fd534600700 1 --2- 172.21.15.38:0/671406533 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd514038770 0x7fd51403ac30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd518006fd0 tx=0x7fd518006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.292+0000 7fd534600700 1 -- 172.21.15.38:0/671406533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 msgr2=0x7fd52c1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.292+0000 7fd534600700 1 --2- 172.21.15.38:0/671406533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c1a06a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd5200117b0 tx=0x7fd520011890 comp rx=0 tx=0).stop 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.293+0000 7fd534600700 1 -- 172.21.15.38:0/671406533 shutdown_connections 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.293+0000 7fd534600700 1 --2- 172.21.15.38:0/671406533 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fd514038770 0x7fd51403ac30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:40.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.293+0000 7fd534600700 1 --2- 172.21.15.38:0/671406533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c10dfd0 0x7fd52c1a06a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:40.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.293+0000 7fd534600700 1 -- 172.21.15.38:0/671406533 >> 172.21.15.38:0/671406533 conn(0x7fd52c07b920 msgr2=0x7fd52c07c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:40.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.293+0000 7fd534600700 1 -- 172.21.15.38:0/671406533 shutdown_connections 2024-03-24T00:36:40.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:40.293+0000 7fd534600700 1 -- 172.21.15.38:0/671406533 wait complete. 2024-03-24T00:36:40.952 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying alertmanager service with default placement... 2024-03-24T00:36:41.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:41 smithi038 ceph-mon[29271]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:41.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:41 smithi038 ceph-mon[29271]: Saving service node-exporter spec with placement * 2024-03-24T00:36:42.427 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-24T00:36:42.427 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.540+0000 7f886d000700 1 Processor -- start 2024-03-24T00:36:42.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.540+0000 7f886d000700 1 -- start start 2024-03-24T00:36:42.432 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.541+0000 7f886d000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f886807a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.541+0000 7f886d000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886807a880 con 0x7f886807be50 2024-03-24T00:36:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.541+0000 7f8865e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f886807a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.541+0000 7f8865e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f886807a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35222/0 (socket says 172.21.15.38:35222) 2024-03-24T00:36:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.541+0000 7f8865e00700 1 -- 172.21.15.38:0/1727085410 learned_addr learned my addr 172.21.15.38:0/1727085410 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.542+0000 7f8865e00700 1 -- 172.21.15.38:0/1727085410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886807a9c0 con 0x7f886807be50 2024-03-24T00:36:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.542+0000 7f8865e00700 1 --2- 172.21.15.38:0/1727085410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f886807a2b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88580098a0 tx=0x7f8858009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480e3b4f1c77e258 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.542+0000 7f8864a00700 1 -- 172.21.15.38:0/1727085410 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8858004180 con 0x7f886807be50 2024-03-24T00:36:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.542+0000 7f8864a00700 1 -- 172.21.15.38:0/1727085410 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88580042e0 con 0x7f886807be50 2024-03-24T00:36:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.542+0000 7f8864a00700 1 -- 172.21.15.38:0/1727085410 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8858011b10 con 0x7f886807be50 2024-03-24T00:36:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.543+0000 7f886d000700 1 -- 172.21.15.38:0/1727085410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 msgr2=0x7f886807a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:42.433 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.543+0000 7f886d000700 1 --2- 172.21.15.38:0/1727085410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f886807a2b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88580098a0 tx=0x7f8858009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.543+0000 7f886d000700 1 -- 172.21.15.38:0/1727085410 shutdown_connections 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.543+0000 7f886d000700 1 --2- 172.21.15.38:0/1727085410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f886807a2b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.543+0000 7f886d000700 1 -- 172.21.15.38:0/1727085410 >> 172.21.15.38:0/1727085410 conn(0x7f88681018e0 msgr2=0x7f8868103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.543+0000 7f886d000700 1 -- 172.21.15.38:0/1727085410 shutdown_connections 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.543+0000 7f886d000700 1 -- 172.21.15.38:0/1727085410 wait complete. 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.544+0000 7f886d000700 1 Processor -- start 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.545+0000 7f886d000700 1 -- start start 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.545+0000 7f886d000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f88681a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.545+0000 7f886d000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88681a0c40 con 0x7f886807be50 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.545+0000 7f8865e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f88681a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.545+0000 7f8865e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f88681a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35236/0 (socket says 172.21.15.38:35236) 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.545+0000 7f8865e00700 1 -- 172.21.15.38:0/317310321 learned_addr learned my addr 172.21.15.38:0/317310321 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:42.434 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.546+0000 7f8865e00700 1 -- 172.21.15.38:0/317310321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8858009550 con 0x7f886807be50 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.546+0000 7f8865e00700 1 --2- 172.21.15.38:0/317310321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f88681a0700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f88580117b0 tx=0x7f8858011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.546+0000 7f885ea00700 1 -- 172.21.15.38:0/317310321 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88580038b0 con 0x7f886807be50 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.546+0000 7f885ea00700 1 -- 172.21.15.38:0/317310321 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8858003a10 con 0x7f886807be50 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.546+0000 7f885ea00700 1 -- 172.21.15.38:0/317310321 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88580246e0 con 0x7f886807be50 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.546+0000 7f886d000700 1 -- 172.21.15.38:0/317310321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88681a0e40 con 0x7f886807be50 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.546+0000 7f886d000700 1 -- 172.21.15.38:0/317310321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88681a12e0 con 0x7f886807be50 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.548+0000 7f886d000700 1 -- 172.21.15.38:0/317310321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8868108300 con 0x7f886807be50 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.548+0000 7f885ea00700 1 -- 172.21.15.38:0/317310321 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f885801ab80 con 0x7f886807be50 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.548+0000 7f885ea00700 1 --2- 172.21.15.38:0/317310321 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f884c038770 0x7f884c03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.548+0000 7f885ea00700 1 -- 172.21.15.38:0/317310321 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f885804c9a0 con 0x7f886807be50 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.549+0000 7f8865400700 1 --2- 172.21.15.38:0/317310321 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f884c038770 0x7f884c03ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.549+0000 7f8865400700 1 --2- 172.21.15.38:0/317310321 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f884c038770 0x7f884c03ac30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8854006fd0 tx=0x7f8854006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:42.435 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.553+0000 7f885ea00700 1 -- 172.21.15.38:0/317310321 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8858010380 con 0x7f886807be50 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.788+0000 7f886d000700 1 -- 172.21.15.38:0/317310321 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f88680611f0 con 0x7f884c038770 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.799+0000 7f885ea00700 1 -- 172.21.15.38:0/317310321 <== mgr.14118 v2:172.21.15.38:6800/3111371494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f88680611f0 con 0x7f884c038770 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.803+0000 7f886d000700 1 -- 172.21.15.38:0/317310321 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f884c038770 msgr2=0x7f884c03ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.803+0000 7f886d000700 1 --2- 172.21.15.38:0/317310321 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f884c038770 0x7f884c03ac30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8854006fd0 tx=0x7f8854006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.803+0000 7f886d000700 1 -- 172.21.15.38:0/317310321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 msgr2=0x7f88681a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.803+0000 7f886d000700 1 --2- 172.21.15.38:0/317310321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f88681a0700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f88580117b0 tx=0x7f8858011890 comp rx=0 tx=0).stop 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.803+0000 7f886d000700 1 -- 172.21.15.38:0/317310321 shutdown_connections 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.803+0000 7f886d000700 1 --2- 172.21.15.38:0/317310321 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f884c038770 0x7f884c03ac30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.803+0000 7f886d000700 1 --2- 172.21.15.38:0/317310321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f886807be50 0x7f88681a0700 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.803+0000 7f886d000700 1 -- 172.21.15.38:0/317310321 >> 172.21.15.38:0/317310321 conn(0x7f88681018e0 msgr2=0x7f8868103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.803+0000 7f886d000700 1 -- 172.21.15.38:0/317310321 shutdown_connections 2024-03-24T00:36:42.436 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:41.803+0000 7f886d000700 1 -- 172.21.15.38:0/317310321 wait complete. 2024-03-24T00:36:42.985 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:42 smithi038 ceph-mon[29271]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:36:42.985 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:42 smithi038 ceph-mon[29271]: Saving service alertmanager spec with placement count:1 2024-03-24T00:36:42.985 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:42 smithi038 ceph-mon[29271]: from='mgr.14118 172.21.15.38:0/1665158082' entity='mgr.smithi038.huhioz' 2024-03-24T00:36:42.986 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:42 smithi038 ceph-mon[29271]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:36:43.864 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.984+0000 7faa93e00700 1 Processor -- start 2024-03-24T00:36:43.864 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.984+0000 7faa93e00700 1 -- start start 2024-03-24T00:36:43.864 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.984+0000 7faa93e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa941087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:43.864 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.984+0000 7faa93e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa94108d80 con 0x7faa94108390 2024-03-24T00:36:43.864 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.985+0000 7faa92a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa941087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:43.864 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.985+0000 7faa92a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa941087b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35242/0 (socket says 172.21.15.38:35242) 2024-03-24T00:36:43.864 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.985+0000 7faa92a00700 1 -- 172.21.15.38:0/1970389372 learned_addr learned my addr 172.21.15.38:0/1970389372 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:43.864 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.985+0000 7faa92a00700 1 -- 172.21.15.38:0/1970389372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa94109590 con 0x7faa94108390 2024-03-24T00:36:43.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.986+0000 7faa92a00700 1 --2- 172.21.15.38:0/1970389372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa941087b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa840098a0 tx=0x7faa84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de658ed4c175bd2f server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:43.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.986+0000 7faa91600700 1 -- 172.21.15.38:0/1970389372 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa84004180 con 0x7faa94108390 2024-03-24T00:36:43.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.986+0000 7faa91600700 1 -- 172.21.15.38:0/1970389372 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa840042e0 con 0x7faa94108390 2024-03-24T00:36:43.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.986+0000 7faa91600700 1 -- 172.21.15.38:0/1970389372 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa84011b10 con 0x7faa94108390 2024-03-24T00:36:43.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.987+0000 7faa93e00700 1 -- 172.21.15.38:0/1970389372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 msgr2=0x7faa941087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:43.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.987+0000 7faa93e00700 1 --2- 172.21.15.38:0/1970389372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa941087b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa840098a0 tx=0x7faa84009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:43.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.987+0000 7faa93e00700 1 -- 172.21.15.38:0/1970389372 shutdown_connections 2024-03-24T00:36:43.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.987+0000 7faa93e00700 1 --2- 172.21.15.38:0/1970389372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa941087b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:43.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.987+0000 7faa93e00700 1 -- 172.21.15.38:0/1970389372 >> 172.21.15.38:0/1970389372 conn(0x7faa9407b720 msgr2=0x7faa94106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:43.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.987+0000 7faa93e00700 1 -- 172.21.15.38:0/1970389372 shutdown_connections 2024-03-24T00:36:43.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.987+0000 7faa93e00700 1 -- 172.21.15.38:0/1970389372 wait complete. 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.989+0000 7faa93e00700 1 Processor -- start 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.989+0000 7faa93e00700 1 -- start start 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.989+0000 7faa93e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa9419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.989+0000 7faa93e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9419c660 con 0x7faa94108390 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.990+0000 7faa92a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa9419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.990+0000 7faa92a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa9419c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38514/0 (socket says 172.21.15.38:38514) 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.990+0000 7faa92a00700 1 -- 172.21.15.38:0/27730928 learned_addr learned my addr 172.21.15.38:0/27730928 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.990+0000 7faa92a00700 1 -- 172.21.15.38:0/27730928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa84009550 con 0x7faa94108390 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.990+0000 7faa92a00700 1 --2- 172.21.15.38:0/27730928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa9419c120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa840117b0 tx=0x7faa84011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.991+0000 7faa98a00700 1 -- 172.21.15.38:0/27730928 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa840038b0 con 0x7faa94108390 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.991+0000 7faa98a00700 1 -- 172.21.15.38:0/27730928 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa84003a10 con 0x7faa94108390 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.991+0000 7faa93e00700 1 -- 172.21.15.38:0/27730928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9419c860 con 0x7faa94108390 2024-03-24T00:36:43.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.991+0000 7faa98a00700 1 -- 172.21.15.38:0/27730928 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa840246e0 con 0x7faa94108390 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.991+0000 7faa93e00700 1 -- 172.21.15.38:0/27730928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9419cd00 con 0x7faa94108390 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.992+0000 7faa98a00700 1 -- 172.21.15.38:0/27730928 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7faa8401a430 con 0x7faa94108390 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.992+0000 7faa93e00700 1 -- 172.21.15.38:0/27730928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9410c7c0 con 0x7faa94108390 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.993+0000 7faa98a00700 1 --2- 172.21.15.38:0/27730928 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7faa78038770 0x7faa7803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.993+0000 7faa98a00700 1 -- 172.21.15.38:0/27730928 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faa8404c8c0 con 0x7faa94108390 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.993+0000 7faa92000700 1 --2- 172.21.15.38:0/27730928 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7faa78038770 0x7faa7803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.994+0000 7faa92000700 1 --2- 172.21.15.38:0/27730928 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7faa78038770 0x7faa7803ac30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faa80006fd0 tx=0x7faa80006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:42.998+0000 7faa98a00700 1 -- 172.21.15.38:0/27730928 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faa84018b40 con 0x7faa94108390 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.222+0000 7faa93e00700 1 -- 172.21.15.38:0/27730928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7faa9419fd80 con 0x7faa94108390 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.227+0000 7faa98a00700 1 -- 172.21.15.38:0/27730928 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7faa84018b40 con 0x7faa94108390 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.230+0000 7faa93e00700 1 -- 172.21.15.38:0/27730928 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7faa78038770 msgr2=0x7faa7803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.230+0000 7faa93e00700 1 --2- 172.21.15.38:0/27730928 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7faa78038770 0x7faa7803ac30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faa80006fd0 tx=0x7faa80006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.230+0000 7faa93e00700 1 -- 172.21.15.38:0/27730928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 msgr2=0x7faa9419c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:43.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.230+0000 7faa93e00700 1 --2- 172.21.15.38:0/27730928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa9419c120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa840117b0 tx=0x7faa84011890 comp rx=0 tx=0).stop 2024-03-24T00:36:43.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.231+0000 7faa93e00700 1 -- 172.21.15.38:0/27730928 shutdown_connections 2024-03-24T00:36:43.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.231+0000 7faa93e00700 1 --2- 172.21.15.38:0/27730928 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7faa78038770 0x7faa7803ac30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:43.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.231+0000 7faa93e00700 1 --2- 172.21.15.38:0/27730928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94108390 0x7faa9419c120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:43.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.231+0000 7faa93e00700 1 -- 172.21.15.38:0/27730928 >> 172.21.15.38:0/27730928 conn(0x7faa9407b720 msgr2=0x7faa94105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:43.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.231+0000 7faa93e00700 1 -- 172.21.15.38:0/27730928 shutdown_connections 2024-03-24T00:36:43.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:43.232+0000 7faa93e00700 1 -- 172.21.15.38:0/27730928 wait complete. 2024-03-24T00:36:44.295 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:44 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/27730928' entity='client.admin' 2024-03-24T00:36:44.295 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:44 smithi038 ceph-mon[29271]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:36:45.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.465+0000 7f27d0800700 1 Processor -- start 2024-03-24T00:36:45.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.465+0000 7f27d0800700 1 -- start start 2024-03-24T00:36:45.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.465+0000 7f27d0800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c8106760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:45.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.466+0000 7f27d0800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8106d30 con 0x7f27c8106340 2024-03-24T00:36:45.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.466+0000 7f27cde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c8106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:45.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.466+0000 7f27cde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c8106760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38524/0 (socket says 172.21.15.38:38524) 2024-03-24T00:36:45.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.466+0000 7f27cde00700 1 -- 172.21.15.38:0/2283054782 learned_addr learned my addr 172.21.15.38:0/2283054782 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:45.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.467+0000 7f27cde00700 1 -- 172.21.15.38:0/2283054782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c8107540 con 0x7f27c8106340 2024-03-24T00:36:45.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.467+0000 7f27cde00700 1 --2- 172.21.15.38:0/2283054782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c8106760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0098a0 tx=0x7f27bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d5a41a3abbb594 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:45.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.467+0000 7f27cca00700 1 -- 172.21.15.38:0/2283054782 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27bc004180 con 0x7f27c8106340 2024-03-24T00:36:45.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.467+0000 7f27cca00700 1 -- 172.21.15.38:0/2283054782 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27bc0042e0 con 0x7f27c8106340 2024-03-24T00:36:45.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.467+0000 7f27cca00700 1 -- 172.21.15.38:0/2283054782 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27bc011b10 con 0x7f27c8106340 2024-03-24T00:36:45.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.468+0000 7f27d0800700 1 -- 172.21.15.38:0/2283054782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 msgr2=0x7f27c8106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:45.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.468+0000 7f27d0800700 1 --2- 172.21.15.38:0/2283054782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c8106760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0098a0 tx=0x7f27bc009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:45.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.468+0000 7f27d0800700 1 -- 172.21.15.38:0/2283054782 shutdown_connections 2024-03-24T00:36:45.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.468+0000 7f27d0800700 1 --2- 172.21.15.38:0/2283054782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c8106760 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:45.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.468+0000 7f27d0800700 1 -- 172.21.15.38:0/2283054782 >> 172.21.15.38:0/2283054782 conn(0x7f27c8101920 msgr2=0x7f27c8103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:45.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.468+0000 7f27d0800700 1 -- 172.21.15.38:0/2283054782 shutdown_connections 2024-03-24T00:36:45.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.468+0000 7f27d0800700 1 -- 172.21.15.38:0/2283054782 wait complete. 2024-03-24T00:36:45.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.469+0000 7f27d0800700 1 Processor -- start 2024-03-24T00:36:45.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.469+0000 7f27d0800700 1 -- start start 2024-03-24T00:36:45.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.470+0000 7f27d0800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c819c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:45.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.470+0000 7f27d0800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819c840 con 0x7f27c8106340 2024-03-24T00:36:45.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.470+0000 7f27cde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c819c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:45.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.470+0000 7f27cde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c819c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38530/0 (socket says 172.21.15.38:38530) 2024-03-24T00:36:45.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.470+0000 7f27cde00700 1 -- 172.21.15.38:0/1210207628 learned_addr learned my addr 172.21.15.38:0/1210207628 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:45.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.470+0000 7f27cde00700 1 -- 172.21.15.38:0/1210207628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc009550 con 0x7f27c8106340 2024-03-24T00:36:45.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.471+0000 7f27cde00700 1 --2- 172.21.15.38:0/1210207628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c819c300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0117b0 tx=0x7f27bc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:45.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.471+0000 7f27c2a00700 1 -- 172.21.15.38:0/1210207628 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27bc0038b0 con 0x7f27c8106340 2024-03-24T00:36:45.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.471+0000 7f27c2a00700 1 -- 172.21.15.38:0/1210207628 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27bc003a10 con 0x7f27c8106340 2024-03-24T00:36:45.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.471+0000 7f27c2a00700 1 -- 172.21.15.38:0/1210207628 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27bc0246e0 con 0x7f27c8106340 2024-03-24T00:36:45.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.471+0000 7f27d0800700 1 -- 172.21.15.38:0/1210207628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c819ca40 con 0x7f27c8106340 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.471+0000 7f27d0800700 1 -- 172.21.15.38:0/1210207628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c819cee0 con 0x7f27c8106340 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.472+0000 7f27d0800700 1 -- 172.21.15.38:0/1210207628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c810a770 con 0x7f27c8106340 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.472+0000 7f27c2a00700 1 -- 172.21.15.38:0/1210207628 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f27bc01ab80 con 0x7f27c8106340 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.473+0000 7f27c2a00700 1 --2- 172.21.15.38:0/1210207628 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f27b0038770 0x7f27b003ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.473+0000 7f27c2a00700 1 -- 172.21.15.38:0/1210207628 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f27bc04c9a0 con 0x7f27c8106340 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.473+0000 7f27cd400700 1 --2- 172.21.15.38:0/1210207628 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f27b0038770 0x7f27b003ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.474+0000 7f27cd400700 1 --2- 172.21.15.38:0/1210207628 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f27b0038770 0x7f27b003ac30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27b8006fd0 tx=0x7f27b8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.479+0000 7f27c2a00700 1 -- 172.21.15.38:0/1210207628 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f27bc010380 con 0x7f27c8106340 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.704+0000 7f27d0800700 1 -- 172.21.15.38:0/1210207628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f27c819fbf0 con 0x7f27c8106340 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.714+0000 7f27c2a00700 1 -- 172.21.15.38:0/1210207628 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f27bc018b40 con 0x7f27c8106340 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.717+0000 7f27d0800700 1 -- 172.21.15.38:0/1210207628 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f27b0038770 msgr2=0x7f27b003ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:45.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.717+0000 7f27d0800700 1 --2- 172.21.15.38:0/1210207628 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f27b0038770 0x7f27b003ac30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27b8006fd0 tx=0x7f27b8006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:45.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.718+0000 7f27d0800700 1 -- 172.21.15.38:0/1210207628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 msgr2=0x7f27c819c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:45.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.718+0000 7f27d0800700 1 --2- 172.21.15.38:0/1210207628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c819c300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0117b0 tx=0x7f27bc011890 comp rx=0 tx=0).stop 2024-03-24T00:36:45.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.718+0000 7f27d0800700 1 -- 172.21.15.38:0/1210207628 shutdown_connections 2024-03-24T00:36:45.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.718+0000 7f27d0800700 1 --2- 172.21.15.38:0/1210207628 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f27b0038770 0x7f27b003ac30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:45.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.718+0000 7f27d0800700 1 --2- 172.21.15.38:0/1210207628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27c8106340 0x7f27c819c300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:45.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.718+0000 7f27d0800700 1 -- 172.21.15.38:0/1210207628 >> 172.21.15.38:0/1210207628 conn(0x7f27c8101920 msgr2=0x7f27c81024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:45.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.718+0000 7f27d0800700 1 -- 172.21.15.38:0/1210207628 shutdown_connections 2024-03-24T00:36:45.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:44.718+0000 7f27d0800700 1 -- 172.21.15.38:0/1210207628 wait complete. 2024-03-24T00:36:45.339 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling the dashboard module... 2024-03-24T00:36:45.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:45 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1210207628' entity='client.admin' 2024-03-24T00:36:47.107 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:46 smithi038 ceph-mon[29271]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:36:47.107 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:46 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1171477987' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-24T00:36:47.449 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.026+0000 7fbce4a00700 1 Processor -- start 2024-03-24T00:36:47.449 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.027+0000 7fbce4a00700 1 -- start start 2024-03-24T00:36:47.449 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.027+0000 7fbce4a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce010e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.027+0000 7fbce4a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce010ea00 con 0x7fbce010dfd0 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.027+0000 7fbcdde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce010e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.028+0000 7fbcdde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce010e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38542/0 (socket says 172.21.15.38:38542) 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.028+0000 7fbcdde00700 1 -- 172.21.15.38:0/2017363260 learned_addr learned my addr 172.21.15.38:0/2017363260 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.028+0000 7fbcdde00700 1 -- 172.21.15.38:0/2017363260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce010cc40 con 0x7fbce010dfd0 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.028+0000 7fbcdde00700 1 --2- 172.21.15.38:0/2017363260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce010e3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00098a0 tx=0x7fbcd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb69270f7b98d37 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.029+0000 7fbcdd400700 1 -- 172.21.15.38:0/2017363260 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbcd0004180 con 0x7fbce010dfd0 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.029+0000 7fbcdd400700 1 -- 172.21.15.38:0/2017363260 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcd00042e0 con 0x7fbce010dfd0 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.029+0000 7fbcdd400700 1 -- 172.21.15.38:0/2017363260 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbcd0011b10 con 0x7fbce010dfd0 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.029+0000 7fbce4a00700 1 -- 172.21.15.38:0/2017363260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 msgr2=0x7fbce010e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.029+0000 7fbce4a00700 1 --2- 172.21.15.38:0/2017363260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce010e3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00098a0 tx=0x7fbcd0009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:47.450 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.030+0000 7fbce4a00700 1 -- 172.21.15.38:0/2017363260 shutdown_connections 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.030+0000 7fbce4a00700 1 --2- 172.21.15.38:0/2017363260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce010e3f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.030+0000 7fbce4a00700 1 -- 172.21.15.38:0/2017363260 >> 172.21.15.38:0/2017363260 conn(0x7fbce007b940 msgr2=0x7fbce007dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.030+0000 7fbce4a00700 1 -- 172.21.15.38:0/2017363260 shutdown_connections 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.030+0000 7fbce4a00700 1 -- 172.21.15.38:0/2017363260 wait complete. 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.031+0000 7fbce4a00700 1 Processor -- start 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.031+0000 7fbce4a00700 1 -- start start 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.031+0000 7fbce4a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.031+0000 7fbce4a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce01a0bf0 con 0x7fbce010dfd0 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.032+0000 7fbcdde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.032+0000 7fbcdde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce01a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38544/0 (socket says 172.21.15.38:38544) 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.032+0000 7fbcdde00700 1 -- 172.21.15.38:0/1171477987 learned_addr learned my addr 172.21.15.38:0/1171477987 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.032+0000 7fbcdde00700 1 -- 172.21.15.38:0/1171477987 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd0009550 con 0x7fbce010dfd0 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.033+0000 7fbcdde00700 1 --2- 172.21.15.38:0/1171477987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce01a06b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00117b0 tx=0x7fbcd0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:47.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.033+0000 7fbcd6a00700 1 -- 172.21.15.38:0/1171477987 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbcd00038b0 con 0x7fbce010dfd0 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.033+0000 7fbce4a00700 1 -- 172.21.15.38:0/1171477987 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce01a0df0 con 0x7fbce010dfd0 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.033+0000 7fbcd6a00700 1 -- 172.21.15.38:0/1171477987 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcd0003a10 con 0x7fbce010dfd0 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.033+0000 7fbcd6a00700 1 -- 172.21.15.38:0/1171477987 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbcd0024780 con 0x7fbce010dfd0 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.033+0000 7fbce4a00700 1 -- 172.21.15.38:0/1171477987 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce01a1290 con 0x7fbce010dfd0 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.034+0000 7fbcd6a00700 1 -- 172.21.15.38:0/1171477987 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fbcd001ac10 con 0x7fbce010dfd0 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.034+0000 7fbce4a00700 1 -- 172.21.15.38:0/1171477987 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce0110e80 con 0x7fbce010dfd0 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.035+0000 7fbcd6a00700 1 --2- 172.21.15.38:0/1171477987 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fbcc4038770 0x7fbcc403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.035+0000 7fbcd6a00700 1 -- 172.21.15.38:0/1171477987 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbcd004c810 con 0x7fbce010dfd0 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.035+0000 7fbcd7e00700 1 --2- 172.21.15.38:0/1171477987 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fbcc4038770 0x7fbcc403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.036+0000 7fbcd7e00700 1 --2- 172.21.15.38:0/1171477987 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fbcc4038770 0x7fbcc403ac30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8006fd0 tx=0x7fbcc8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.040+0000 7fbcd6a00700 1 -- 172.21.15.38:0/1171477987 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbcd002cb40 con 0x7fbce010dfd0 2024-03-24T00:36:47.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.323+0000 7fbce4a00700 1 -- 172.21.15.38:0/1171477987 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fbce01a3f40 con 0x7fbce010dfd0 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.725+0000 7fbcd6a00700 1 -- 172.21.15.38:0/1171477987 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 9) v1 ==== 45561+0+0 (secure 0 0 0) 0x7fbcd002c350 con 0x7fbce010dfd0 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.728+0000 7fbcd6a00700 1 -- 172.21.15.38:0/1171477987 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fbcd004c160 con 0x7fbce010dfd0 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.731+0000 7fbce4a00700 1 -- 172.21.15.38:0/1171477987 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fbcc4038770 msgr2=0x7fbcc403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.731+0000 7fbce4a00700 1 --2- 172.21.15.38:0/1171477987 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fbcc4038770 0x7fbcc403ac30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8006fd0 tx=0x7fbcc8006e40 comp rx=0 tx=0).stop 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.731+0000 7fbce4a00700 1 -- 172.21.15.38:0/1171477987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 msgr2=0x7fbce01a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.731+0000 7fbce4a00700 1 --2- 172.21.15.38:0/1171477987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce01a06b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00117b0 tx=0x7fbcd0011890 comp rx=0 tx=0).stop 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.731+0000 7fbce4a00700 1 -- 172.21.15.38:0/1171477987 shutdown_connections 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.731+0000 7fbce4a00700 1 --2- 172.21.15.38:0/1171477987 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7fbcc4038770 0x7fbcc403ac30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.731+0000 7fbce4a00700 1 --2- 172.21.15.38:0/1171477987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbce010dfd0 0x7fbce01a06b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.732+0000 7fbce4a00700 1 -- 172.21.15.38:0/1171477987 >> 172.21.15.38:0/1171477987 conn(0x7fbce007b940 msgr2=0x7fbce007c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.732+0000 7fbce4a00700 1 -- 172.21.15.38:0/1171477987 shutdown_connections 2024-03-24T00:36:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:46.732+0000 7fbce4a00700 1 -- 172.21.15.38:0/1171477987 wait complete. 2024-03-24T00:36:48.016 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:47 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1171477987' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-24T00:36:48.017 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:47 smithi038 ceph-mon[29271]: mgrmap e9: smithi038.huhioz(active, since 32s) 2024-03-24T00:36:48.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:48 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/816661357' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-24T00:36:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-03-24T00:36:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-24T00:36:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T00:36:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "smithi038.huhioz", 2024-03-24T00:36:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-24T00:36:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-03-24T00:36:49.019 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.076+0000 7f3bc9800700 1 Processor -- start 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.077+0000 7f3bc9800700 1 -- start start 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.077+0000 7f3bc9800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc4079140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.077+0000 7f3bc9800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4079710 con 0x7f3bc407ace0 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.077+0000 7f3bc2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc4079140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.077+0000 7f3bc2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc4079140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38562/0 (socket says 172.21.15.38:38562) 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.077+0000 7f3bc2a00700 1 -- 172.21.15.38:0/1482492149 learned_addr learned my addr 172.21.15.38:0/1482492149 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.078+0000 7f3bc2a00700 1 -- 172.21.15.38:0/1482492149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc4079850 con 0x7f3bc407ace0 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.078+0000 7f3bc2a00700 1 --2- 172.21.15.38:0/1482492149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc4079140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098a0 tx=0x7f3bb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20751b17ceb77fe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.078+0000 7f3bc2000700 1 -- 172.21.15.38:0/1482492149 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bb4004180 con 0x7f3bc407ace0 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.078+0000 7f3bc2000700 1 -- 172.21.15.38:0/1482492149 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bb40042e0 con 0x7f3bc407ace0 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.078+0000 7f3bc2000700 1 -- 172.21.15.38:0/1482492149 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bb4011b10 con 0x7f3bc407ace0 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.079+0000 7f3bc9800700 1 -- 172.21.15.38:0/1482492149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 msgr2=0x7f3bc4079140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:49.020 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.079+0000 7f3bc9800700 1 --2- 172.21.15.38:0/1482492149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc4079140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098a0 tx=0x7f3bb4009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.079+0000 7f3bc9800700 1 -- 172.21.15.38:0/1482492149 shutdown_connections 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.079+0000 7f3bc9800700 1 --2- 172.21.15.38:0/1482492149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc4079140 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.079+0000 7f3bc9800700 1 -- 172.21.15.38:0/1482492149 >> 172.21.15.38:0/1482492149 conn(0x7f3bc41018e0 msgr2=0x7f3bc4103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.079+0000 7f3bc9800700 1 -- 172.21.15.38:0/1482492149 shutdown_connections 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.079+0000 7f3bc9800700 1 -- 172.21.15.38:0/1482492149 wait complete. 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.080+0000 7f3bc9800700 1 Processor -- start 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.080+0000 7f3bc9800700 1 -- start start 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.081+0000 7f3bc9800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.081+0000 7f3bc9800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419c7f0 con 0x7f3bc407ace0 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.081+0000 7f3bc2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.081+0000 7f3bc2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38576/0 (socket says 172.21.15.38:38576) 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.081+0000 7f3bc2a00700 1 -- 172.21.15.38:0/816661357 learned_addr learned my addr 172.21.15.38:0/816661357 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.081+0000 7f3bc2a00700 1 -- 172.21.15.38:0/816661357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb4009550 con 0x7f3bc407ace0 2024-03-24T00:36:49.021 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.082+0000 7f3bc2a00700 1 --2- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc419c2b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40117b0 tx=0x7f3bb4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.082+0000 7f3bbb400700 1 -- 172.21.15.38:0/816661357 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bb40038b0 con 0x7f3bc407ace0 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.082+0000 7f3bbb400700 1 -- 172.21.15.38:0/816661357 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bb4003a10 con 0x7f3bc407ace0 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.082+0000 7f3bc9800700 1 -- 172.21.15.38:0/816661357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc419c9f0 con 0x7f3bc407ace0 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.082+0000 7f3bbb400700 1 -- 172.21.15.38:0/816661357 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bb4024780 con 0x7f3bc407ace0 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.082+0000 7f3bc9800700 1 -- 172.21.15.38:0/816661357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc419ce90 con 0x7f3bc407ace0 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.083+0000 7f3bc9800700 1 -- 172.21.15.38:0/816661357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc410c720 con 0x7f3bc407ace0 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.084+0000 7f3bbb400700 1 -- 172.21.15.38:0/816661357 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 9) v1 ==== 45561+0+0 (secure 0 0 0) 0x7f3bb401ac10 con 0x7f3bc407ace0 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.084+0000 7f3bbb400700 1 --2- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f3ba8038810 0x7f3ba803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.084+0000 7f3bbb400700 1 -- 172.21.15.38:0/816661357 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3bb404c850 con 0x7f3bc407ace0 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.084+0000 7f3bbbe00700 1 -- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f3ba8038810 msgr2=0x7f3ba803acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3111371494 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.084+0000 7f3bbbe00700 1 --2- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f3ba8038810 0x7f3ba803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.089+0000 7f3bbb400700 1 -- 172.21.15.38:0/816661357 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3bb402cb00 con 0x7f3bc407ace0 2024-03-24T00:36:49.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.285+0000 7f3bbbe00700 1 -- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f3ba8038810 msgr2=0x7f3ba803acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3111371494 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.285+0000 7f3bbbe00700 1 --2- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f3ba8038810 0x7f3ba803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.367+0000 7f3bc9800700 1 -- 172.21.15.38:0/816661357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3bc419fba0 con 0x7f3bc407ace0 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.368+0000 7f3bbb400700 1 -- 172.21.15.38:0/816661357 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f3bb402cb00 con 0x7f3bc407ace0 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.371+0000 7f3bc9800700 1 -- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f3ba8038810 msgr2=0x7f3ba803acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.371+0000 7f3bc9800700 1 --2- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f3ba8038810 0x7f3ba803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.371+0000 7f3bc9800700 1 -- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 msgr2=0x7f3bc419c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.371+0000 7f3bc9800700 1 --2- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc419c2b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40117b0 tx=0x7f3bb4011890 comp rx=0 tx=0).stop 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.371+0000 7f3bc9800700 1 -- 172.21.15.38:0/816661357 shutdown_connections 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.371+0000 7f3bc9800700 1 --2- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f3ba8038810 0x7f3ba803acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.371+0000 7f3bc9800700 1 --2- 172.21.15.38:0/816661357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bc407ace0 0x7f3bc419c2b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.371+0000 7f3bc9800700 1 -- 172.21.15.38:0/816661357 >> 172.21.15.38:0/816661357 conn(0x7f3bc41018e0 msgr2=0x7f3bc41024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.371+0000 7f3bc9800700 1 -- 172.21.15.38:0/816661357 shutdown_connections 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:48.371+0000 7f3bc9800700 1 -- 172.21.15.38:0/816661357 wait complete. 2024-03-24T00:36:49.023 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-03-24T00:36:49.024 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 9... 2024-03-24T00:36:56.032 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:55 smithi038 ceph-mon[29271]: Active manager daemon smithi038.huhioz restarted 2024-03-24T00:36:56.033 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:55 smithi038 ceph-mon[29271]: Activating manager daemon smithi038.huhioz 2024-03-24T00:36:56.033 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:55 smithi038 ceph-mon[29271]: osdmap e3: 0 total, 0 up, 0 in 2024-03-24T00:36:56.033 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:55 smithi038 ceph-mon[29271]: mgrmap e10: smithi038.huhioz(active, starting, since 0.0134407s) 2024-03-24T00:36:56.033 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:55 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:36:56.033 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:55 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi038.huhioz", "id": "smithi038.huhioz"}]: dispatch 2024-03-24T00:36:56.033 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:55 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T00:36:56.033 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:55 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T00:36:56.033 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:55 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T00:36:56.033 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:55 smithi038 ceph-mon[29271]: Manager daemon smithi038.huhioz is now available 2024-03-24T00:36:57.257 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:56 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:36:57.257 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:56 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:36:57.257 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:56 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:36:57.258 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:56 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/trash_purge_schedule"}]: dispatch 2024-03-24T00:36:57.258 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:56 smithi038 ceph-mon[29271]: [24/Mar/2024:00:36:56] ENGINE Bus STARTING 2024-03-24T00:36:57.258 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:56 smithi038 ceph-mon[29271]: [24/Mar/2024:00:36:56] ENGINE Serving on https://172.21.15.38:7150 2024-03-24T00:36:57.258 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:56 smithi038 ceph-mon[29271]: [24/Mar/2024:00:36:56] ENGINE Serving on http://172.21.15.38:8765 2024-03-24T00:36:57.258 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:56 smithi038 ceph-mon[29271]: [24/Mar/2024:00:36:56] ENGINE Bus STARTED 2024-03-24T00:36:57.258 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:56 smithi038 ceph-mon[29271]: mgrmap e11: smithi038.huhioz(active, since 1.01032s) 2024-03-24T00:36:58.282 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:58 smithi038 ceph-mon[29271]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-24T00:36:58.282 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:58 smithi038 ceph-mon[29271]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-24T00:36:58.290 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-03-24T00:36:58.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-24T00:36:58.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-24T00:36:58.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-03-24T00:36:58.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.562+0000 7f865aa00700 1 Processor -- start 2024-03-24T00:36:58.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.562+0000 7f865aa00700 1 -- start start 2024-03-24T00:36:58.294 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.563+0000 7f865aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f8654106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.563+0000 7f865aa00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8654106d40 con 0x7f8654106350 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.563+0000 7f8653e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f8654106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.563+0000 7f8653e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f8654106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38588/0 (socket says 172.21.15.38:38588) 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.563+0000 7f8653e00700 1 -- 172.21.15.38:0/544193080 learned_addr learned my addr 172.21.15.38:0/544193080 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.564+0000 7f8653e00700 1 -- 172.21.15.38:0/544193080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654107550 con 0x7f8654106350 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.564+0000 7f8653e00700 1 --2- 172.21.15.38:0/544193080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f8654106770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f86440098a0 tx=0x7f8644009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff41bf226dd8e07 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.564+0000 7f8652a00700 1 -- 172.21.15.38:0/544193080 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8644004180 con 0x7f8654106350 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.565+0000 7f8652a00700 1 -- 172.21.15.38:0/544193080 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86440042e0 con 0x7f8654106350 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.565+0000 7f8652a00700 1 -- 172.21.15.38:0/544193080 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8644011b10 con 0x7f8654106350 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.565+0000 7f865aa00700 1 -- 172.21.15.38:0/544193080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 msgr2=0x7f8654106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.565+0000 7f865aa00700 1 --2- 172.21.15.38:0/544193080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f8654106770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f86440098a0 tx=0x7f8644009bb0 comp rx=0 tx=0).stop 2024-03-24T00:36:58.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.565+0000 7f865aa00700 1 -- 172.21.15.38:0/544193080 shutdown_connections 2024-03-24T00:36:58.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.565+0000 7f865aa00700 1 --2- 172.21.15.38:0/544193080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f8654106770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:58.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.565+0000 7f865aa00700 1 -- 172.21.15.38:0/544193080 >> 172.21.15.38:0/544193080 conn(0x7f86541018f0 msgr2=0x7f8654103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:58.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.566+0000 7f865aa00700 1 -- 172.21.15.38:0/544193080 shutdown_connections 2024-03-24T00:36:58.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.566+0000 7f865aa00700 1 -- 172.21.15.38:0/544193080 wait complete. 2024-03-24T00:36:58.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.567+0000 7f865aa00700 1 Processor -- start 2024-03-24T00:36:58.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.567+0000 7f865aa00700 1 -- start start 2024-03-24T00:36:58.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.567+0000 7f865aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f865419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:58.301 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.567+0000 7f865aa00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865419a690 con 0x7f8654106350 2024-03-24T00:36:58.301 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.567+0000 7f8653e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f865419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:58.301 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.567+0000 7f8653e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f865419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38592/0 (socket says 172.21.15.38:38592) 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.567+0000 7f8653e00700 1 -- 172.21.15.38:0/4233190601 learned_addr learned my addr 172.21.15.38:0/4233190601 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.568+0000 7f8653e00700 1 -- 172.21.15.38:0/4233190601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8644009550 con 0x7f8654106350 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.568+0000 7f8653e00700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f865419a150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f86440117b0 tx=0x7f8644011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.568+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86440038b0 con 0x7f8654106350 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.568+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8644003a10 con 0x7f8654106350 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.568+0000 7f865aa00700 1 -- 172.21.15.38:0/4233190601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865419a890 con 0x7f8654106350 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.568+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86440246e0 con 0x7f8654106350 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.569+0000 7f865aa00700 1 -- 172.21.15.38:0/4233190601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865419ad30 con 0x7f8654106350 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.570+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 9) v1 ==== 45561+0+0 (secure 0 0 0) 0x7f864401a430 con 0x7f8654106350 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.570+0000 7f8650c00700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 0x7f863803ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.570+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 --> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f863803b390 con 0x7f86380387c0 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.570+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f864404c970 con 0x7f8654106350 2024-03-24T00:36:58.302 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.570+0000 7f8653400700 1 -- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 msgr2=0x7f863803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3111371494 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.570+0000 7f8653400700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 0x7f863803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.771+0000 7f8653400700 1 -- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 msgr2=0x7f863803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3111371494 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:49.771+0000 7f8653400700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 0x7f863803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:50.172+0000 7f8653400700 1 -- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 msgr2=0x7f863803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3111371494 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:50.172+0000 7f8653400700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 0x7f863803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:50.973+0000 7f8653400700 1 -- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 msgr2=0x7f863803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3111371494 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:50.973+0000 7f8653400700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 0x7f863803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:52.574+0000 7f8653400700 1 -- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 msgr2=0x7f863803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3111371494 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:52.575+0000 7f8653400700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 0x7f863803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:55.779+0000 7f8653400700 1 -- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 msgr2=0x7f863803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3111371494 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:55.779+0000 7f8653400700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 0x7f863803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:55.942+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 10) v1 ==== 45328+0+0 (secure 0 0 0) 0x7f864401ab80 con 0x7f8654106350 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:55.942+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 msgr2=0x7f863803ac80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T00:36:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:55.942+0000 7f8650c00700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 0x7f863803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.940+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 11) v1 ==== 45455+0+0 (secure 0 0 0) 0x7f864404e340 con 0x7f8654106350 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.940+0000 7f8650c00700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f863803bfc0 0x7f863803e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.941+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 --> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f863803b390 con 0x7f863803bfc0 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.941+0000 7f8653400700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f863803bfc0 0x7f863803e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.943+0000 7f8653400700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f863803bfc0 0x7f863803e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8640003a10 tx=0x7f8640009270 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.944+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 <== mgr.14162 v2:172.21.15.38:6800/1124244493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f863803b390 con 0x7f863803bfc0 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.952+0000 7f865aa00700 1 -- 172.21.15.38:0/4233190601 --> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f86540623e0 con 0x7f863803bfc0 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.952+0000 7f8650c00700 1 -- 172.21.15.38:0/4233190601 <== mgr.14162 v2:172.21.15.38:6800/1124244493 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f86540623e0 con 0x7f863803bfc0 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.953+0000 7f865aa00700 1 -- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f863803bfc0 msgr2=0x7f863803e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.953+0000 7f865aa00700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f863803bfc0 0x7f863803e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8640003a10 tx=0x7f8640009270 comp rx=0 tx=0).stop 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.953+0000 7f865aa00700 1 -- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 msgr2=0x7f865419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.953+0000 7f865aa00700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f865419a150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f86440117b0 tx=0x7f8644011890 comp rx=0 tx=0).stop 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.953+0000 7f865aa00700 1 -- 172.21.15.38:0/4233190601 shutdown_connections 2024-03-24T00:36:58.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.953+0000 7f865aa00700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f863803bfc0 0x7f863803e3b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:58.305 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.953+0000 7f865aa00700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8654106350 0x7f865419a150 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:58.305 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.953+0000 7f865aa00700 1 --2- 172.21.15.38:0/4233190601 >> [v2:172.21.15.38:6800/3111371494,v1:172.21.15.38:6801/3111371494] conn(0x7f86380387c0 0x7f863803ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:36:58.305 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.953+0000 7f865aa00700 1 -- 172.21.15.38:0/4233190601 >> 172.21.15.38:0/4233190601 conn(0x7f86541018f0 msgr2=0x7f86541024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:36:58.306 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.953+0000 7f865aa00700 1 -- 172.21.15.38:0/4233190601 shutdown_connections 2024-03-24T00:36:58.306 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:56.953+0000 7f865aa00700 1 -- 172.21.15.38:0/4233190601 wait complete. 2024-03-24T00:36:58.306 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 9 is available 2024-03-24T00:36:58.306 INFO:teuthology.orchestra.run.smithi038.stdout:Generating a dashboard self-signed certificate... 2024-03-24T00:36:59.345 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:59 smithi038 ceph-mon[29271]: mgrmap e12: smithi038.huhioz(active, since 2s) 2024-03-24T00:36:59.346 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:36:59 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:00.033 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-24T00:37:00.034 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.920+0000 7f7e5ae00700 1 Processor -- start 2024-03-24T00:37:00.034 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.921+0000 7f7e5ae00700 1 -- start start 2024-03-24T00:37:00.034 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.921+0000 7f7e5ae00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e54105060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:00.034 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.921+0000 7f7e5ae00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540744e0 con 0x7f7e54104c40 2024-03-24T00:37:00.034 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.922+0000 7f7e53e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e54105060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:00.034 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.922+0000 7f7e53e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e54105060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38066/0 (socket says 172.21.15.38:38066) 2024-03-24T00:37:00.034 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.922+0000 7f7e53e00700 1 -- 172.21.15.38:0/3915901708 learned_addr learned my addr 172.21.15.38:0/3915901708 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:00.035 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.922+0000 7f7e53e00700 1 -- 172.21.15.38:0/3915901708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e54074620 con 0x7f7e54104c40 2024-03-24T00:37:00.035 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.923+0000 7f7e53e00700 1 --2- 172.21.15.38:0/3915901708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e54105060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098a0 tx=0x7f7e44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f173bd8405731df6 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:00.035 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.923+0000 7f7e52a00700 1 -- 172.21.15.38:0/3915901708 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e44004180 con 0x7f7e54104c40 2024-03-24T00:37:00.035 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.923+0000 7f7e52a00700 1 -- 172.21.15.38:0/3915901708 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e440042e0 con 0x7f7e54104c40 2024-03-24T00:37:00.035 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.923+0000 7f7e52a00700 1 -- 172.21.15.38:0/3915901708 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e44011b10 con 0x7f7e54104c40 2024-03-24T00:37:00.035 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.924+0000 7f7e5ae00700 1 -- 172.21.15.38:0/3915901708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 msgr2=0x7f7e54105060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:00.035 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.924+0000 7f7e5ae00700 1 --2- 172.21.15.38:0/3915901708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e54105060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098a0 tx=0x7f7e44009bb0 comp rx=0 tx=0).stop 2024-03-24T00:37:00.036 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.924+0000 7f7e5ae00700 1 -- 172.21.15.38:0/3915901708 shutdown_connections 2024-03-24T00:37:00.036 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.924+0000 7f7e5ae00700 1 --2- 172.21.15.38:0/3915901708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e54105060 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:00.036 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.924+0000 7f7e5ae00700 1 -- 172.21.15.38:0/3915901708 >> 172.21.15.38:0/3915901708 conn(0x7f7e54100800 msgr2=0x7f7e54102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:00.036 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.924+0000 7f7e5ae00700 1 -- 172.21.15.38:0/3915901708 shutdown_connections 2024-03-24T00:37:00.036 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.924+0000 7f7e5ae00700 1 -- 172.21.15.38:0/3915901708 wait complete. 2024-03-24T00:37:00.036 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.925+0000 7f7e5ae00700 1 Processor -- start 2024-03-24T00:37:00.037 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.926+0000 7f7e5ae00700 1 -- start start 2024-03-24T00:37:00.037 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.926+0000 7f7e5ae00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e5419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:00.037 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.926+0000 7f7e5ae00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5419c840 con 0x7f7e54104c40 2024-03-24T00:37:00.037 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.926+0000 7f7e53e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e5419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:00.037 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.926+0000 7f7e53e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e5419c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38074/0 (socket says 172.21.15.38:38074) 2024-03-24T00:37:00.037 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.926+0000 7f7e53e00700 1 -- 172.21.15.38:0/1005416781 learned_addr learned my addr 172.21.15.38:0/1005416781 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:00.037 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.927+0000 7f7e53e00700 1 -- 172.21.15.38:0/1005416781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e44009550 con 0x7f7e54104c40 2024-03-24T00:37:00.037 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.927+0000 7f7e53e00700 1 --2- 172.21.15.38:0/1005416781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e5419c300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e440117b0 tx=0x7f7e44011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:00.037 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.927+0000 7f7e50c00700 1 -- 172.21.15.38:0/1005416781 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e440038b0 con 0x7f7e54104c40 2024-03-24T00:37:00.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.927+0000 7f7e50c00700 1 -- 172.21.15.38:0/1005416781 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e44003a10 con 0x7f7e54104c40 2024-03-24T00:37:00.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.928+0000 7f7e50c00700 1 -- 172.21.15.38:0/1005416781 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e44024780 con 0x7f7e54104c40 2024-03-24T00:37:00.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.928+0000 7f7e5ae00700 1 -- 172.21.15.38:0/1005416781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5419ca40 con 0x7f7e54104c40 2024-03-24T00:37:00.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.928+0000 7f7e5ae00700 1 -- 172.21.15.38:0/1005416781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5419cee0 con 0x7f7e54104c40 2024-03-24T00:37:00.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.929+0000 7f7e50c00700 1 -- 172.21.15.38:0/1005416781 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) v1 ==== 45561+0+0 (secure 0 0 0) 0x7f7e4401ac10 con 0x7f7e54104c40 2024-03-24T00:37:00.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.929+0000 7f7e5ae00700 1 -- 172.21.15.38:0/1005416781 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e54196010 con 0x7f7e54104c40 2024-03-24T00:37:00.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.929+0000 7f7e50c00700 1 --2- 172.21.15.38:0/1005416781 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f7e380387c0 0x7f7e3803ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:00.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.930+0000 7f7e50c00700 1 -- 172.21.15.38:0/1005416781 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7e4404c980 con 0x7f7e54104c40 2024-03-24T00:37:00.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.930+0000 7f7e53400700 1 --2- 172.21.15.38:0/1005416781 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f7e380387c0 0x7f7e3803ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:00.038 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.931+0000 7f7e53400700 1 --2- 172.21.15.38:0/1005416781 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f7e380387c0 0x7f7e3803ac80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e40006fd0 tx=0x7f7e40006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:58.936+0000 7f7e50c00700 1 -- 172.21.15.38:0/1005416781 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7e44018760 con 0x7f7e54104c40 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.168+0000 7f7e5ae00700 1 -- 172.21.15.38:0/1005416781 --> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e540611f0 con 0x7f7e380387c0 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.345+0000 7f7e50c00700 1 -- 172.21.15.38:0/1005416781 <== mgr.14162 v2:172.21.15.38:6800/1124244493 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f7e540611f0 con 0x7f7e380387c0 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.348+0000 7f7e5ae00700 1 -- 172.21.15.38:0/1005416781 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f7e380387c0 msgr2=0x7f7e3803ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.348+0000 7f7e5ae00700 1 --2- 172.21.15.38:0/1005416781 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f7e380387c0 0x7f7e3803ac80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e40006fd0 tx=0x7f7e40006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.348+0000 7f7e5ae00700 1 -- 172.21.15.38:0/1005416781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 msgr2=0x7f7e5419c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.348+0000 7f7e5ae00700 1 --2- 172.21.15.38:0/1005416781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e5419c300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e440117b0 tx=0x7f7e44011890 comp rx=0 tx=0).stop 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.348+0000 7f7e5ae00700 1 -- 172.21.15.38:0/1005416781 shutdown_connections 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.348+0000 7f7e5ae00700 1 --2- 172.21.15.38:0/1005416781 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f7e380387c0 0x7f7e3803ac80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.348+0000 7f7e5ae00700 1 --2- 172.21.15.38:0/1005416781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e54104c40 0x7f7e5419c300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.348+0000 7f7e5ae00700 1 -- 172.21.15.38:0/1005416781 >> 172.21.15.38:0/1005416781 conn(0x7f7e54100800 msgr2=0x7f7e54193150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.349+0000 7f7e5ae00700 1 -- 172.21.15.38:0/1005416781 shutdown_connections 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:36:59.349+0000 7f7e5ae00700 1 -- 172.21.15.38:0/1005416781 wait complete. 2024-03-24T00:37:00.039 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial admin user... 2024-03-24T00:37:00.575 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:00 smithi038 ceph-mon[29271]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:37:00.575 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:00 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:00.575 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:00 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:00.575 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:00 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:02.477 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:02 smithi038 ceph-mon[29271]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:37:02.477 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:02 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:02.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$oToIvpMGuGQOwuFwpyyz0OV8PuLi3FXzBGKggGu1i3sv5/eYOzEb.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711240621, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-24T00:37:02.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.653+0000 7f392be00700 1 Processor -- start 2024-03-24T00:37:02.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.654+0000 7f392be00700 1 -- start start 2024-03-24T00:37:02.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.654+0000 7f392be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f3924079180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:02.494 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.654+0000 7f392be00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924079750 con 0x7f392407ad20 2024-03-24T00:37:02.495 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.655+0000 7f3929400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f3924079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:02.495 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.655+0000 7f3929400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f3924079180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38076/0 (socket says 172.21.15.38:38076) 2024-03-24T00:37:02.495 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.655+0000 7f3929400700 1 -- 172.21.15.38:0/288378229 learned_addr learned my addr 172.21.15.38:0/288378229 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:02.495 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.655+0000 7f3929400700 1 -- 172.21.15.38:0/288378229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3924079890 con 0x7f392407ad20 2024-03-24T00:37:02.495 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.655+0000 7f3929400700 1 --2- 172.21.15.38:0/288378229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f3924079180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39180098a0 tx=0x7f3918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff9c8bf4ff2a39 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:02.495 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.656+0000 7f3923e00700 1 -- 172.21.15.38:0/288378229 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3918004180 con 0x7f392407ad20 2024-03-24T00:37:02.495 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.656+0000 7f3923e00700 1 -- 172.21.15.38:0/288378229 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39180042e0 con 0x7f392407ad20 2024-03-24T00:37:02.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.656+0000 7f3923e00700 1 -- 172.21.15.38:0/288378229 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3918011b10 con 0x7f392407ad20 2024-03-24T00:37:02.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.656+0000 7f392be00700 1 -- 172.21.15.38:0/288378229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 msgr2=0x7f3924079180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:02.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.656+0000 7f392be00700 1 --2- 172.21.15.38:0/288378229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f3924079180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39180098a0 tx=0x7f3918009bb0 comp rx=0 tx=0).stop 2024-03-24T00:37:02.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.657+0000 7f392be00700 1 -- 172.21.15.38:0/288378229 shutdown_connections 2024-03-24T00:37:02.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.657+0000 7f392be00700 1 --2- 172.21.15.38:0/288378229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f3924079180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:02.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.657+0000 7f392be00700 1 -- 172.21.15.38:0/288378229 >> 172.21.15.38:0/288378229 conn(0x7f39241018f0 msgr2=0x7f3924103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:02.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.657+0000 7f392be00700 1 -- 172.21.15.38:0/288378229 shutdown_connections 2024-03-24T00:37:02.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.657+0000 7f392be00700 1 -- 172.21.15.38:0/288378229 wait complete. 2024-03-24T00:37:02.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.658+0000 7f392be00700 1 Processor -- start 2024-03-24T00:37:02.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.658+0000 7f392be00700 1 -- start start 2024-03-24T00:37:02.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.659+0000 7f392be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f392419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:02.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.659+0000 7f392be00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392419c840 con 0x7f392407ad20 2024-03-24T00:37:02.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.659+0000 7f3929400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f392419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:02.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.659+0000 7f3929400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f392419c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38080/0 (socket says 172.21.15.38:38080) 2024-03-24T00:37:02.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.659+0000 7f3929400700 1 -- 172.21.15.38:0/3897057661 learned_addr learned my addr 172.21.15.38:0/3897057661 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:02.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.659+0000 7f3929400700 1 -- 172.21.15.38:0/3897057661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3918009550 con 0x7f392407ad20 2024-03-24T00:37:02.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.660+0000 7f3929400700 1 --2- 172.21.15.38:0/3897057661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f392419c300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39180117b0 tx=0x7f3918011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:02.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.660+0000 7f3922000700 1 -- 172.21.15.38:0/3897057661 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39180038b0 con 0x7f392407ad20 2024-03-24T00:37:02.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.660+0000 7f3922000700 1 -- 172.21.15.38:0/3897057661 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3918003a10 con 0x7f392407ad20 2024-03-24T00:37:02.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.660+0000 7f392be00700 1 -- 172.21.15.38:0/3897057661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392419ca40 con 0x7f392407ad20 2024-03-24T00:37:02.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.660+0000 7f3922000700 1 -- 172.21.15.38:0/3897057661 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39180246e0 con 0x7f392407ad20 2024-03-24T00:37:02.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.660+0000 7f392be00700 1 -- 172.21.15.38:0/3897057661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392419cee0 con 0x7f392407ad20 2024-03-24T00:37:02.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.662+0000 7f3922000700 1 -- 172.21.15.38:0/3897057661 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) v1 ==== 45561+0+0 (secure 0 0 0) 0x7f391801ab80 con 0x7f392407ad20 2024-03-24T00:37:02.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.662+0000 7f392be00700 1 -- 172.21.15.38:0/3897057661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392410c6f0 con 0x7f392407ad20 2024-03-24T00:37:02.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.662+0000 7f3922000700 1 --2- 172.21.15.38:0/3897057661 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f390c038810 0x7f390c03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:02.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.662+0000 7f3922000700 1 -- 172.21.15.38:0/3897057661 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f391804c980 con 0x7f392407ad20 2024-03-24T00:37:02.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.662+0000 7f3928a00700 1 --2- 172.21.15.38:0/3897057661 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f390c038810 0x7f390c03acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:02.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.663+0000 7f3928a00700 1 --2- 172.21.15.38:0/3897057661 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f390c038810 0x7f390c03acd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3910006fd0 tx=0x7f3910006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:02.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.672+0000 7f3922000700 1 -- 172.21.15.38:0/3897057661 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3918018760 con 0x7f392407ad20 2024-03-24T00:37:02.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:00.925+0000 7f392be00700 1 -- 172.21.15.38:0/3897057661 --> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f392410c8e0 con 0x7f390c038810 2024-03-24T00:37:02.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:01.197+0000 7f3922000700 1 -- 172.21.15.38:0/3897057661 <== mgr.14162 v2:172.21.15.38:6800/1124244493 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f392410c8e0 con 0x7f390c038810 2024-03-24T00:37:02.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:01.200+0000 7f392be00700 1 -- 172.21.15.38:0/3897057661 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f390c038810 msgr2=0x7f390c03acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:02.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:01.200+0000 7f392be00700 1 --2- 172.21.15.38:0/3897057661 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f390c038810 0x7f390c03acd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3910006fd0 tx=0x7f3910006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:02.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:01.200+0000 7f392be00700 1 -- 172.21.15.38:0/3897057661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 msgr2=0x7f392419c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:02.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:01.200+0000 7f392be00700 1 --2- 172.21.15.38:0/3897057661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f392419c300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39180117b0 tx=0x7f3918011890 comp rx=0 tx=0).stop 2024-03-24T00:37:02.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:01.200+0000 7f392be00700 1 -- 172.21.15.38:0/3897057661 shutdown_connections 2024-03-24T00:37:02.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:01.200+0000 7f392be00700 1 --2- 172.21.15.38:0/3897057661 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f390c038810 0x7f390c03acd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:02.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:01.200+0000 7f392be00700 1 --2- 172.21.15.38:0/3897057661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392407ad20 0x7f392419c300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:02.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:01.200+0000 7f392be00700 1 -- 172.21.15.38:0/3897057661 >> 172.21.15.38:0/3897057661 conn(0x7f39241018f0 msgr2=0x7f3924102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:02.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:01.200+0000 7f392be00700 1 -- 172.21.15.38:0/3897057661 shutdown_connections 2024-03-24T00:37:02.507 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:01.200+0000 7f392be00700 1 -- 172.21.15.38:0/3897057661 wait complete. 2024-03-24T00:37:02.507 INFO:teuthology.orchestra.run.smithi038.stdout:Fetching dashboard port number... 2024-03-24T00:37:03.390 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:03 smithi038 ceph-mon[29271]: mgrmap e13: smithi038.huhioz(active, since 6s) 2024-03-24T00:37:03.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 8443 2024-03-24T00:37:03.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.128+0000 7f2911a00700 1 Processor -- start 2024-03-24T00:37:03.999 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.128+0000 7f2911a00700 1 -- start start 2024-03-24T00:37:04.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.128+0000 7f2911a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c1066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:04.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.128+0000 7f2911a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c106cb0 con 0x7f290c1062c0 2024-03-24T00:37:04.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.129+0000 7f290aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c1066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:04.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.129+0000 7f290aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c1066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44146/0 (socket says 172.21.15.38:44146) 2024-03-24T00:37:04.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.129+0000 7f290aa00700 1 -- 172.21.15.38:0/657362819 learned_addr learned my addr 172.21.15.38:0/657362819 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:04.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.130+0000 7f290aa00700 1 -- 172.21.15.38:0/657362819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c1074c0 con 0x7f290c1062c0 2024-03-24T00:37:04.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.130+0000 7f290aa00700 1 --2- 172.21.15.38:0/657362819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c1066e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0098a0 tx=0x7f28fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f745e53bbe85c71f server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:04.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.130+0000 7f2909600700 1 -- 172.21.15.38:0/657362819 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28fc004180 con 0x7f290c1062c0 2024-03-24T00:37:04.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.130+0000 7f2909600700 1 -- 172.21.15.38:0/657362819 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28fc0042e0 con 0x7f290c1062c0 2024-03-24T00:37:04.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.130+0000 7f2909600700 1 -- 172.21.15.38:0/657362819 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28fc011b10 con 0x7f290c1062c0 2024-03-24T00:37:04.000 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.131+0000 7f2911a00700 1 -- 172.21.15.38:0/657362819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 msgr2=0x7f290c1066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.131+0000 7f2911a00700 1 --2- 172.21.15.38:0/657362819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c1066e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0098a0 tx=0x7f28fc009bb0 comp rx=0 tx=0).stop 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.131+0000 7f2911a00700 1 -- 172.21.15.38:0/657362819 shutdown_connections 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.131+0000 7f2911a00700 1 --2- 172.21.15.38:0/657362819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c1066e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.131+0000 7f2911a00700 1 -- 172.21.15.38:0/657362819 >> 172.21.15.38:0/657362819 conn(0x7f290c101840 msgr2=0x7f290c103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.131+0000 7f2911a00700 1 -- 172.21.15.38:0/657362819 shutdown_connections 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.132+0000 7f2911a00700 1 -- 172.21.15.38:0/657362819 wait complete. 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.133+0000 7f2911a00700 1 Processor -- start 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.133+0000 7f2911a00700 1 -- start start 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.133+0000 7f2911a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.133+0000 7f2911a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c19c820 con 0x7f290c1062c0 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.134+0000 7f290aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:04.001 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.134+0000 7f290aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c19c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44156/0 (socket says 172.21.15.38:44156) 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.134+0000 7f290aa00700 1 -- 172.21.15.38:0/297039226 learned_addr learned my addr 172.21.15.38:0/297039226 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.134+0000 7f290aa00700 1 -- 172.21.15.38:0/297039226 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc009550 con 0x7f290c1062c0 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.134+0000 7f290aa00700 1 --2- 172.21.15.38:0/297039226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c19c2e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0117b0 tx=0x7f28fc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.135+0000 7f2903400700 1 -- 172.21.15.38:0/297039226 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28fc0038b0 con 0x7f290c1062c0 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.135+0000 7f2903400700 1 -- 172.21.15.38:0/297039226 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28fc003a10 con 0x7f290c1062c0 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.135+0000 7f2903400700 1 -- 172.21.15.38:0/297039226 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28fc0246e0 con 0x7f290c1062c0 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.135+0000 7f2911a00700 1 -- 172.21.15.38:0/297039226 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290c19ca20 con 0x7f290c1062c0 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.135+0000 7f2911a00700 1 -- 172.21.15.38:0/297039226 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290c19cec0 con 0x7f290c1062c0 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.136+0000 7f2903400700 1 -- 172.21.15.38:0/297039226 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f28fc01ab80 con 0x7f290c1062c0 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.136+0000 7f2911a00700 1 -- 172.21.15.38:0/297039226 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290c10a6f0 con 0x7f290c1062c0 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.136+0000 7f2903400700 1 --2- 172.21.15.38:0/297039226 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f28f0038860 0x7f28f003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:04.002 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.136+0000 7f2903400700 1 -- 172.21.15.38:0/297039226 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f28fc04cac0 con 0x7f290c1062c0 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.137+0000 7f290a000700 1 --2- 172.21.15.38:0/297039226 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f28f0038860 0x7f28f003ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.138+0000 7f290a000700 1 --2- 172.21.15.38:0/297039226 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f28f0038860 0x7f28f003ad20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f28f8006fd0 tx=0x7f28f8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.142+0000 7f2903400700 1 -- 172.21.15.38:0/297039226 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f28fc01a430 con 0x7f290c1062c0 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.367+0000 7f2911a00700 1 -- 172.21.15.38:0/297039226 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f290c19fb70 con 0x7f290c1062c0 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.368+0000 7f2903400700 1 -- 172.21.15.38:0/297039226 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f28fc018b40 con 0x7f290c1062c0 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.371+0000 7f2911a00700 1 -- 172.21.15.38:0/297039226 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f28f0038860 msgr2=0x7f28f003ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.371+0000 7f2911a00700 1 --2- 172.21.15.38:0/297039226 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f28f0038860 0x7f28f003ad20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f28f8006fd0 tx=0x7f28f8006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.371+0000 7f2911a00700 1 -- 172.21.15.38:0/297039226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 msgr2=0x7f290c19c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.371+0000 7f2911a00700 1 --2- 172.21.15.38:0/297039226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c19c2e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0117b0 tx=0x7f28fc011890 comp rx=0 tx=0).stop 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.371+0000 7f2911a00700 1 -- 172.21.15.38:0/297039226 shutdown_connections 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.371+0000 7f2911a00700 1 --2- 172.21.15.38:0/297039226 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f28f0038860 0x7f28f003ad20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.371+0000 7f2911a00700 1 --2- 172.21.15.38:0/297039226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f290c1062c0 0x7f290c19c2e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.371+0000 7f2911a00700 1 -- 172.21.15.38:0/297039226 >> 172.21.15.38:0/297039226 conn(0x7f290c101840 msgr2=0x7f290c1023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:04.003 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.371+0000 7f2911a00700 1 -- 172.21.15.38:0/297039226 shutdown_connections 2024-03-24T00:37:04.004 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:03.372+0000 7f2911a00700 1 -- 172.21.15.38:0/297039226 wait complete. 2024-03-24T00:37:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T00:37:04.025 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-03-24T00:37:04.050 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T00:37:04.050 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-03-24T00:37:04.050 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-03-24T00:37:04.051 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-24T00:37:04.058 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph Dashboard is now available at: 2024-03-24T00:37:04.058 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:37:04.058 INFO:teuthology.orchestra.run.smithi038.stdout: URL: https://smithi038.front.sepia.ceph.com:8443/ 2024-03-24T00:37:04.058 INFO:teuthology.orchestra.run.smithi038.stdout: User: admin 2024-03-24T00:37:04.058 INFO:teuthology.orchestra.run.smithi038.stdout: Password: dbszno9mfb 2024-03-24T00:37:04.058 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:37:04.059 INFO:teuthology.orchestra.run.smithi038.stdout:Saving cluster configuration to /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config directory 2024-03-24T00:37:04.061 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling autotune for osd_memory_target 2024-03-24T00:37:04.358 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:04 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/297039226' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-24T00:37:05.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.661+0000 7fbbee200700 1 Processor -- start 2024-03-24T00:37:05.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.662+0000 7fbbee200700 1 -- start start 2024-03-24T00:37:05.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.662+0000 7fbbee200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe81091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:05.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.662+0000 7fbbee200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8079e20 con 0x7fbbe8106dd0 2024-03-24T00:37:05.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.662+0000 7fbbe7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe81091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:05.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.663+0000 7fbbe7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe81091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44172/0 (socket says 172.21.15.38:44172) 2024-03-24T00:37:05.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.663+0000 7fbbe7400700 1 -- 172.21.15.38:0/3206081266 learned_addr learned my addr 172.21.15.38:0/3206081266 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:05.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.663+0000 7fbbe7400700 1 -- 172.21.15.38:0/3206081266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe8109700 con 0x7fbbe8106dd0 2024-03-24T00:37:05.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.664+0000 7fbbe7400700 1 --2- 172.21.15.38:0/3206081266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe81091c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80098a0 tx=0x7fbbd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782cf965c0a10d00 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:05.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.664+0000 7fbbe6000700 1 -- 172.21.15.38:0/3206081266 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbd8004180 con 0x7fbbe8106dd0 2024-03-24T00:37:05.494 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.664+0000 7fbbe6000700 1 -- 172.21.15.38:0/3206081266 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbd80042e0 con 0x7fbbe8106dd0 2024-03-24T00:37:05.494 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.664+0000 7fbbe6000700 1 -- 172.21.15.38:0/3206081266 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbd8011b10 con 0x7fbbe8106dd0 2024-03-24T00:37:05.494 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.665+0000 7fbbee200700 1 -- 172.21.15.38:0/3206081266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 msgr2=0x7fbbe81091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.665+0000 7fbbee200700 1 --2- 172.21.15.38:0/3206081266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe81091c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80098a0 tx=0x7fbbd8009bb0 comp rx=0 tx=0).stop 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.665+0000 7fbbee200700 1 -- 172.21.15.38:0/3206081266 shutdown_connections 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.665+0000 7fbbee200700 1 --2- 172.21.15.38:0/3206081266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe81091c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.665+0000 7fbbee200700 1 -- 172.21.15.38:0/3206081266 >> 172.21.15.38:0/3206081266 conn(0x7fbbe8100820 msgr2=0x7fbbe8102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.665+0000 7fbbee200700 1 -- 172.21.15.38:0/3206081266 shutdown_connections 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.665+0000 7fbbee200700 1 -- 172.21.15.38:0/3206081266 wait complete. 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.666+0000 7fbbee200700 1 Processor -- start 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.666+0000 7fbbee200700 1 -- start start 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.667+0000 7fbbee200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.667+0000 7fbbee200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe819a650 con 0x7fbbe8106dd0 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.667+0000 7fbbe7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.667+0000 7fbbe7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44176/0 (socket says 172.21.15.38:44176) 2024-03-24T00:37:05.496 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.667+0000 7fbbe7400700 1 -- 172.21.15.38:0/957258229 learned_addr learned my addr 172.21.15.38:0/957258229 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:05.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.667+0000 7fbbe7400700 1 -- 172.21.15.38:0/957258229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8009550 con 0x7fbbe8106dd0 2024-03-24T00:37:05.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.668+0000 7fbbe7400700 1 --2- 172.21.15.38:0/957258229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe819a110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009fb0 tx=0x7fbbd8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:05.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.668+0000 7fbbdfe00700 1 -- 172.21.15.38:0/957258229 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbd80038b0 con 0x7fbbe8106dd0 2024-03-24T00:37:05.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.668+0000 7fbbdfe00700 1 -- 172.21.15.38:0/957258229 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbd8003a10 con 0x7fbbe8106dd0 2024-03-24T00:37:05.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.668+0000 7fbbdfe00700 1 -- 172.21.15.38:0/957258229 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbd8019750 con 0x7fbbe8106dd0 2024-03-24T00:37:05.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.668+0000 7fbbee200700 1 -- 172.21.15.38:0/957258229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe819a850 con 0x7fbbe8106dd0 2024-03-24T00:37:05.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.668+0000 7fbbee200700 1 -- 172.21.15.38:0/957258229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe819acf0 con 0x7fbbe8106dd0 2024-03-24T00:37:05.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.669+0000 7fbbee200700 1 -- 172.21.15.38:0/957258229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe810a4f0 con 0x7fbbe8106dd0 2024-03-24T00:37:05.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.670+0000 7fbbdfe00700 1 -- 172.21.15.38:0/957258229 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7fbbd801ac10 con 0x7fbbe8106dd0 2024-03-24T00:37:05.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.670+0000 7fbbdfe00700 1 --2- 172.21.15.38:0/957258229 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fbbcc038820 0x7fbbcc03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:05.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.670+0000 7fbbdfe00700 1 -- 172.21.15.38:0/957258229 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbbd804ca10 con 0x7fbbe8106dd0 2024-03-24T00:37:05.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.670+0000 7fbbe6a00700 1 --2- 172.21.15.38:0/957258229 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fbbcc038820 0x7fbbcc03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:05.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.671+0000 7fbbe6a00700 1 --2- 172.21.15.38:0/957258229 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fbbcc038820 0x7fbbcc03ace0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4006fd0 tx=0x7fbbd4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:05.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.676+0000 7fbbdfe00700 1 -- 172.21.15.38:0/957258229 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbbd80198b0 con 0x7fbbe8106dd0 2024-03-24T00:37:05.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.901+0000 7fbbee200700 1 -- 172.21.15.38:0/957258229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fbbe819d950 con 0x7fbbe8106dd0 2024-03-24T00:37:05.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.902+0000 7fbbdfe00700 1 -- 172.21.15.38:0/957258229 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fbbd8022310 con 0x7fbbe8106dd0 2024-03-24T00:37:05.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.905+0000 7fbbee200700 1 -- 172.21.15.38:0/957258229 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fbbcc038820 msgr2=0x7fbbcc03ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:05.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.905+0000 7fbbee200700 1 --2- 172.21.15.38:0/957258229 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fbbcc038820 0x7fbbcc03ace0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4006fd0 tx=0x7fbbd4006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:05.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.905+0000 7fbbee200700 1 -- 172.21.15.38:0/957258229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 msgr2=0x7fbbe819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:05.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.905+0000 7fbbee200700 1 --2- 172.21.15.38:0/957258229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe819a110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009fb0 tx=0x7fbbd8011890 comp rx=0 tx=0).stop 2024-03-24T00:37:05.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.906+0000 7fbbee200700 1 -- 172.21.15.38:0/957258229 shutdown_connections 2024-03-24T00:37:05.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.906+0000 7fbbee200700 1 --2- 172.21.15.38:0/957258229 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fbbcc038820 0x7fbbcc03ace0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:05.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.906+0000 7fbbee200700 1 --2- 172.21.15.38:0/957258229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbe8106dd0 0x7fbbe819a110 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:05.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.906+0000 7fbbee200700 1 -- 172.21.15.38:0/957258229 >> 172.21.15.38:0/957258229 conn(0x7fbbe8100820 msgr2=0x7fbbe8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:05.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.906+0000 7fbbee200700 1 -- 172.21.15.38:0/957258229 shutdown_connections 2024-03-24T00:37:05.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:04.906+0000 7fbbee200700 1 -- 172.21.15.38:0/957258229 wait complete. 2024-03-24T00:37:07.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.135+0000 7fee9e000700 1 Processor -- start 2024-03-24T00:37:07.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.135+0000 7fee9e000700 1 -- start start 2024-03-24T00:37:07.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.136+0000 7fee9e000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee981067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:07.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.136+0000 7fee9e000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee98106d70 con 0x7fee98106380 2024-03-24T00:37:07.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.136+0000 7fee97400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee981067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:07.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.136+0000 7fee97400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee981067a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44188/0 (socket says 172.21.15.38:44188) 2024-03-24T00:37:07.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.136+0000 7fee97400700 1 -- 172.21.15.38:0/1470654035 learned_addr learned my addr 172.21.15.38:0/1470654035 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:07.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.136+0000 7fee97400700 1 -- 172.21.15.38:0/1470654035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee98107580 con 0x7fee98106380 2024-03-24T00:37:07.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.137+0000 7fee97400700 1 --2- 172.21.15.38:0/1470654035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee981067a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee880098a0 tx=0x7fee88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417b272772a39009 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:07.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.137+0000 7fee96a00700 1 -- 172.21.15.38:0/1470654035 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee88004180 con 0x7fee98106380 2024-03-24T00:37:07.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.137+0000 7fee96a00700 1 -- 172.21.15.38:0/1470654035 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee880042e0 con 0x7fee98106380 2024-03-24T00:37:07.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.137+0000 7fee96a00700 1 -- 172.21.15.38:0/1470654035 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee88011b10 con 0x7fee98106380 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.138+0000 7fee9e000700 1 -- 172.21.15.38:0/1470654035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 msgr2=0x7fee981067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.138+0000 7fee9e000700 1 --2- 172.21.15.38:0/1470654035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee981067a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee880098a0 tx=0x7fee88009bb0 comp rx=0 tx=0).stop 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.138+0000 7fee9e000700 1 -- 172.21.15.38:0/1470654035 shutdown_connections 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.138+0000 7fee9e000700 1 --2- 172.21.15.38:0/1470654035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee981067a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.138+0000 7fee9e000700 1 -- 172.21.15.38:0/1470654035 >> 172.21.15.38:0/1470654035 conn(0x7fee98101980 msgr2=0x7fee98103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.138+0000 7fee9e000700 1 -- 172.21.15.38:0/1470654035 shutdown_connections 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.138+0000 7fee9e000700 1 -- 172.21.15.38:0/1470654035 wait complete. 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.139+0000 7fee9e000700 1 Processor -- start 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.139+0000 7fee9e000700 1 -- start start 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.140+0000 7fee9e000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee9819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.140+0000 7fee9e000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9819c890 con 0x7fee98106380 2024-03-24T00:37:07.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.140+0000 7fee97400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee9819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.140+0000 7fee97400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee9819c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44202/0 (socket says 172.21.15.38:44202) 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.140+0000 7fee97400700 1 -- 172.21.15.38:0/2428227381 learned_addr learned my addr 172.21.15.38:0/2428227381 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.140+0000 7fee97400700 1 -- 172.21.15.38:0/2428227381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee88009550 con 0x7fee98106380 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.140+0000 7fee97400700 1 --2- 172.21.15.38:0/2428227381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee9819c350 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee880117b0 tx=0x7fee88011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.141+0000 7fee94c00700 1 -- 172.21.15.38:0/2428227381 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee880038b0 con 0x7fee98106380 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.141+0000 7fee94c00700 1 -- 172.21.15.38:0/2428227381 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee88003a10 con 0x7fee98106380 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.141+0000 7fee9e000700 1 -- 172.21.15.38:0/2428227381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9819ca90 con 0x7fee98106380 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.141+0000 7fee94c00700 1 -- 172.21.15.38:0/2428227381 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee880246e0 con 0x7fee98106380 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.141+0000 7fee9e000700 1 -- 172.21.15.38:0/2428227381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9819cf30 con 0x7fee98106380 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.142+0000 7fee94c00700 1 -- 172.21.15.38:0/2428227381 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7fee8801ab80 con 0x7fee98106380 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.142+0000 7fee9e000700 1 -- 172.21.15.38:0/2428227381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9810a7b0 con 0x7fee98106380 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.143+0000 7fee94c00700 1 --2- 172.21.15.38:0/2428227381 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fee7c0388b0 0x7fee7c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.143+0000 7fee94c00700 1 -- 172.21.15.38:0/2428227381 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee8804cad0 con 0x7fee98106380 2024-03-24T00:37:07.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.143+0000 7fee8fe00700 1 --2- 172.21.15.38:0/2428227381 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fee7c0388b0 0x7fee7c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.143+0000 7fee8fe00700 1 --2- 172.21.15.38:0/2428227381 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fee7c0388b0 0x7fee7c03ad70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fee80006fd0 tx=0x7fee80006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.148+0000 7fee94c00700 1 -- 172.21.15.38:0/2428227381 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee8801a430 con 0x7fee98106380 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.488+0000 7fee9e000700 1 -- 172.21.15.38:0/2428227381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fee9819fbe0 con 0x7fee98106380 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.494+0000 7fee94c00700 1 -- 172.21.15.38:0/2428227381 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7fee88018b40 con 0x7fee98106380 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.497+0000 7fee9e000700 1 -- 172.21.15.38:0/2428227381 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fee7c0388b0 msgr2=0x7fee7c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.497+0000 7fee9e000700 1 --2- 172.21.15.38:0/2428227381 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fee7c0388b0 0x7fee7c03ad70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fee80006fd0 tx=0x7fee80006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.497+0000 7fee9e000700 1 -- 172.21.15.38:0/2428227381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 msgr2=0x7fee9819c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.497+0000 7fee9e000700 1 --2- 172.21.15.38:0/2428227381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee9819c350 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee880117b0 tx=0x7fee88011890 comp rx=0 tx=0).stop 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.497+0000 7fee9e000700 1 -- 172.21.15.38:0/2428227381 shutdown_connections 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.497+0000 7fee9e000700 1 --2- 172.21.15.38:0/2428227381 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fee7c0388b0 0x7fee7c03ad70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.497+0000 7fee9e000700 1 --2- 172.21.15.38:0/2428227381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee98106380 0x7fee9819c350 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.498+0000 7fee9e000700 1 -- 172.21.15.38:0/2428227381 >> 172.21.15.38:0/2428227381 conn(0x7fee98101980 msgr2=0x7fee98102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:07.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.498+0000 7fee9e000700 1 -- 172.21.15.38:0/2428227381 shutdown_connections 2024-03-24T00:37:07.142 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-03-24T00:37:06.498+0000 7fee9e000700 1 -- 172.21.15.38:0/2428227381 wait complete. 2024-03-24T00:37:07.142 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-24T00:37:07.142 INFO:teuthology.orchestra.run.smithi038.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-24T00:37:07.142 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:37:07.142 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-24T00:37:07.142 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:37:07.142 INFO:teuthology.orchestra.run.smithi038.stdout:Or, if you are only running a single cluster on this host: 2024-03-24T00:37:07.143 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:37:07.143 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-24T00:37:07.143 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:37:07.143 INFO:teuthology.orchestra.run.smithi038.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-24T00:37:07.143 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:37:07.143 INFO:teuthology.orchestra.run.smithi038.stdout: ceph telemetry on 2024-03-24T00:37:07.143 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:37:07.143 INFO:teuthology.orchestra.run.smithi038.stdout:For more information see: 2024-03-24T00:37:07.143 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:37:07.144 INFO:teuthology.orchestra.run.smithi038.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-24T00:37:07.144 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:37:07.144 INFO:teuthology.orchestra.run.smithi038.stdout:Bootstrap complete. 2024-03-24T00:37:07.236 INFO:tasks.cephadm:Fetching config... 2024-03-24T00:37:07.236 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:37:07.237 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-24T00:37:07.325 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-24T00:37:07.325 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:37:07.325 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-24T00:37:07.415 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-24T00:37:07.416 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:37:07.416 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/keyring of=/dev/stdout 2024-03-24T00:37:07.560 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-24T00:37:07.560 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:37:07.560 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-24T00:37:07.615 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-24T00:37:07.615 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDciVCtyVmHKLYB381xqHbea5IAKdRR7D/OpflBIUFOars7paBDT6WQ5pXjw7D6V2ZVvMXhAwKaXZLdNc1k34KD54L+t3jMmnnXSTeVyj8a2X1Oev2O8mmThUJpCqQ/qLEGDxqsbSAAeHo7wL5AXwYbphVLEkJdcU59mFiHBo1iI2DA+h5cA+X7r2X7Z1gb7jykX+YI121tXzt0czSECpVyLSWTQ+lKAq01UtsyRs/e41rR5Xbfm5yEfzVZSnZ6sMyOxwmNqB/kbaN8vJjvzhakRdoxJqzCxU0iRUj8QcRkkUXf++bPRT2hlqc0H3TbY3f95bNUA7cNHfKypbxd2oCRjcbEcj8f653hc9yk/BQZ/wD/hdlBp9TgBxRP4CmI2L6DSGITtMMM5LZbxuxDRdtuhxG+Wz6ZWnhkPL6fcnh4JG4dHcttphfBgGyPWYOs8hLuxrsma8F6pExK/R0ooIb0LOA6FhyAXMGXYsJUW2AGfosH7vtMJaQ4ZfpcPQ7fib0= ceph-4796a0d6-e976-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-24T00:37:07.695 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:07 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2428227381' entity='client.admin' 2024-03-24T00:37:07.736 INFO:teuthology.orchestra.run.smithi038.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDciVCtyVmHKLYB381xqHbea5IAKdRR7D/OpflBIUFOars7paBDT6WQ5pXjw7D6V2ZVvMXhAwKaXZLdNc1k34KD54L+t3jMmnnXSTeVyj8a2X1Oev2O8mmThUJpCqQ/qLEGDxqsbSAAeHo7wL5AXwYbphVLEkJdcU59mFiHBo1iI2DA+h5cA+X7r2X7Z1gb7jykX+YI121tXzt0czSECpVyLSWTQ+lKAq01UtsyRs/e41rR5Xbfm5yEfzVZSnZ6sMyOxwmNqB/kbaN8vJjvzhakRdoxJqzCxU0iRUj8QcRkkUXf++bPRT2hlqc0H3TbY3f95bNUA7cNHfKypbxd2oCRjcbEcj8f653hc9yk/BQZ/wD/hdlBp9TgBxRP4CmI2L6DSGITtMMM5LZbxuxDRdtuhxG+Wz6ZWnhkPL6fcnh4JG4dHcttphfBgGyPWYOs8hLuxrsma8F6pExK/R0ooIb0LOA6FhyAXMGXYsJUW2AGfosH7vtMJaQ4ZfpcPQ7fib0= ceph-4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:37:07.790 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDciVCtyVmHKLYB381xqHbea5IAKdRR7D/OpflBIUFOars7paBDT6WQ5pXjw7D6V2ZVvMXhAwKaXZLdNc1k34KD54L+t3jMmnnXSTeVyj8a2X1Oev2O8mmThUJpCqQ/qLEGDxqsbSAAeHo7wL5AXwYbphVLEkJdcU59mFiHBo1iI2DA+h5cA+X7r2X7Z1gb7jykX+YI121tXzt0czSECpVyLSWTQ+lKAq01UtsyRs/e41rR5Xbfm5yEfzVZSnZ6sMyOxwmNqB/kbaN8vJjvzhakRdoxJqzCxU0iRUj8QcRkkUXf++bPRT2hlqc0H3TbY3f95bNUA7cNHfKypbxd2oCRjcbEcj8f653hc9yk/BQZ/wD/hdlBp9TgBxRP4CmI2L6DSGITtMMM5LZbxuxDRdtuhxG+Wz6ZWnhkPL6fcnh4JG4dHcttphfBgGyPWYOs8hLuxrsma8F6pExK/R0ooIb0LOA6FhyAXMGXYsJUW2AGfosH7vtMJaQ4ZfpcPQ7fib0= ceph-4796a0d6-e976-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-24T00:37:07.924 INFO:teuthology.orchestra.run.smithi088.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDciVCtyVmHKLYB381xqHbea5IAKdRR7D/OpflBIUFOars7paBDT6WQ5pXjw7D6V2ZVvMXhAwKaXZLdNc1k34KD54L+t3jMmnnXSTeVyj8a2X1Oev2O8mmThUJpCqQ/qLEGDxqsbSAAeHo7wL5AXwYbphVLEkJdcU59mFiHBo1iI2DA+h5cA+X7r2X7Z1gb7jykX+YI121tXzt0czSECpVyLSWTQ+lKAq01UtsyRs/e41rR5Xbfm5yEfzVZSnZ6sMyOxwmNqB/kbaN8vJjvzhakRdoxJqzCxU0iRUj8QcRkkUXf++bPRT2hlqc0H3TbY3f95bNUA7cNHfKypbxd2oCRjcbEcj8f653hc9yk/BQZ/wD/hdlBp9TgBxRP4CmI2L6DSGITtMMM5LZbxuxDRdtuhxG+Wz6ZWnhkPL6fcnh4JG4dHcttphfBgGyPWYOs8hLuxrsma8F6pExK/R0ooIb0LOA6FhyAXMGXYsJUW2AGfosH7vtMJaQ4ZfpcPQ7fib0= ceph-4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:37:07.994 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-24T00:37:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:37:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.846+0000 7f8fd3e00700 1 -- 172.21.15.38:0/715168886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fcc1011d0 msgr2=0x7f8fcc1035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.846+0000 7f8fd3e00700 1 --2- 172.21.15.38:0/715168886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fcc1011d0 0x7f8fcc1035c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009910 tx=0x7f8fc0009c20 comp rx=0 tx=0).stop 2024-03-24T00:37:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.846+0000 7f8fd3e00700 1 -- 172.21.15.38:0/715168886 shutdown_connections 2024-03-24T00:37:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.846+0000 7f8fd3e00700 1 --2- 172.21.15.38:0/715168886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fcc1011d0 0x7f8fcc1035c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.846+0000 7f8fd3e00700 1 -- 172.21.15.38:0/715168886 >> 172.21.15.38:0/715168886 conn(0x7f8fcc0fabe0 msgr2=0x7f8fcc0fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.847+0000 7f8fd3e00700 1 -- 172.21.15.38:0/715168886 shutdown_connections 2024-03-24T00:37:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.847+0000 7f8fd3e00700 1 -- 172.21.15.38:0/715168886 wait complete. 2024-03-24T00:37:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.848+0000 7f8fd3e00700 1 Processor -- start 2024-03-24T00:37:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.848+0000 7f8fd3e00700 1 -- start start 2024-03-24T00:37:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.848+0000 7f8fd3e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fcc1011d0 0x7f8fcc197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.848+0000 7f8fd3e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1982e0 con 0x7f8fcc1011d0 2024-03-24T00:37:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.848+0000 7f8fd1400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fcc1011d0 0x7f8fcc197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.848+0000 7f8fd1400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fcc1011d0 0x7f8fcc197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44214/0 (socket says 172.21.15.38:44214) 2024-03-24T00:37:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.848+0000 7f8fd1400700 1 -- 172.21.15.38:0/1032413854 learned_addr learned my addr 172.21.15.38:0/1032413854 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:10.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.849+0000 7f8fd1400700 1 -- 172.21.15.38:0/1032413854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00095f0 con 0x7f8fcc1011d0 2024-03-24T00:37:10.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.849+0000 7f8fd1400700 1 --2- 172.21.15.38:0/1032413854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fcc1011d0 0x7f8fcc197da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009ed0 tx=0x7f8fc0004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:10.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.849+0000 7f8fca000700 1 -- 172.21.15.38:0/1032413854 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8fc001d070 con 0x7f8fcc1011d0 2024-03-24T00:37:10.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.849+0000 7f8fca000700 1 -- 172.21.15.38:0/1032413854 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fc000be20 con 0x7f8fcc1011d0 2024-03-24T00:37:10.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.849+0000 7f8fd3e00700 1 -- 172.21.15.38:0/1032413854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc1984e0 con 0x7f8fcc1011d0 2024-03-24T00:37:10.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.849+0000 7f8fca000700 1 -- 172.21.15.38:0/1032413854 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8fc000f460 con 0x7f8fcc1011d0 2024-03-24T00:37:10.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.850+0000 7f8fd3e00700 1 -- 172.21.15.38:0/1032413854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc198980 con 0x7f8fcc1011d0 2024-03-24T00:37:10.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.851+0000 7f8fd3e00700 1 -- 172.21.15.38:0/1032413854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc191970 con 0x7f8fcc1011d0 2024-03-24T00:37:10.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.851+0000 7f8fca000700 1 -- 172.21.15.38:0/1032413854 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f8fc0022800 con 0x7f8fcc1011d0 2024-03-24T00:37:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.851+0000 7f8fca000700 1 --2- 172.21.15.38:0/1032413854 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8fb4038820 0x7f8fb403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.851+0000 7f8fca000700 1 -- 172.21.15.38:0/1032413854 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8fc004cc70 con 0x7f8fcc1011d0 2024-03-24T00:37:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.852+0000 7f8fd0a00700 1 --2- 172.21.15.38:0/1032413854 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8fb4038820 0x7f8fb403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.852+0000 7f8fd0a00700 1 --2- 172.21.15.38:0/1032413854 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8fb4038820 0x7f8fb403ace0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8006fd0 tx=0x7f8fb8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:10.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:10.857+0000 7f8fca000700 1 -- 172.21.15.38:0/1032413854 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8fc002ba00 con 0x7f8fcc1011d0 2024-03-24T00:37:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.086+0000 7f8fd3e00700 1 -- 172.21.15.38:0/1032413854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f8fcc0623e0 con 0x7f8fcc1011d0 2024-03-24T00:37:11.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.092+0000 7f8fca000700 1 -- 172.21.15.38:0/1032413854 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8fcc0623e0 con 0x7f8fcc1011d0 2024-03-24T00:37:11.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.095+0000 7f8fd3e00700 1 -- 172.21.15.38:0/1032413854 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8fb4038820 msgr2=0x7f8fb403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:11.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.095+0000 7f8fd3e00700 1 --2- 172.21.15.38:0/1032413854 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8fb4038820 0x7f8fb403ace0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8006fd0 tx=0x7f8fb8006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:11.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.095+0000 7f8fd3e00700 1 -- 172.21.15.38:0/1032413854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fcc1011d0 msgr2=0x7f8fcc197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:11.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.095+0000 7f8fd3e00700 1 --2- 172.21.15.38:0/1032413854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fcc1011d0 0x7f8fcc197da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009ed0 tx=0x7f8fc0004690 comp rx=0 tx=0).stop 2024-03-24T00:37:11.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.095+0000 7f8fd3e00700 1 -- 172.21.15.38:0/1032413854 shutdown_connections 2024-03-24T00:37:11.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.095+0000 7f8fd3e00700 1 --2- 172.21.15.38:0/1032413854 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8fb4038820 0x7f8fb403ace0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:11.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.095+0000 7f8fd3e00700 1 --2- 172.21.15.38:0/1032413854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fcc1011d0 0x7f8fcc197da0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:11.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.095+0000 7f8fd3e00700 1 -- 172.21.15.38:0/1032413854 >> 172.21.15.38:0/1032413854 conn(0x7f8fcc0fabe0 msgr2=0x7f8fcc0fb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:11.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.095+0000 7f8fd3e00700 1 -- 172.21.15.38:0/1032413854 shutdown_connections 2024-03-24T00:37:11.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:11.096+0000 7f8fd3e00700 1 -- 172.21.15.38:0/1032413854 wait complete. 2024-03-24T00:37:11.817 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-24T00:37:11.817 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-24T00:37:12.190 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:37:12.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:12 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1032413854' entity='client.admin' 2024-03-24T00:37:14.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.549+0000 7f7671a00700 1 -- 172.21.15.38:0/4253880345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f766c075760 msgr2=0x7f766c075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:14.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.549+0000 7f7671a00700 1 --2- 172.21.15.38:0/4253880345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f766c075760 0x7f766c075b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f765c009910 tx=0x7f765c009c20 comp rx=0 tx=0).stop 2024-03-24T00:37:14.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.549+0000 7f7671a00700 1 -- 172.21.15.38:0/4253880345 shutdown_connections 2024-03-24T00:37:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.549+0000 7f7671a00700 1 --2- 172.21.15.38:0/4253880345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f766c075760 0x7f766c075b80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.549+0000 7f7671a00700 1 -- 172.21.15.38:0/4253880345 >> 172.21.15.38:0/4253880345 conn(0x7f766c0fdef0 msgr2=0x7f766c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.549+0000 7f7671a00700 1 -- 172.21.15.38:0/4253880345 shutdown_connections 2024-03-24T00:37:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.549+0000 7f7671a00700 1 -- 172.21.15.38:0/4253880345 wait complete. 2024-03-24T00:37:14.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.550+0000 7f7671a00700 1 Processor -- start 2024-03-24T00:37:14.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.550+0000 7f7671a00700 1 -- start start 2024-03-24T00:37:14.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.551+0000 7f7671a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f766c075760 0x7f766c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:14.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.551+0000 7f7671a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c19c720 con 0x7f766c075760 2024-03-24T00:37:14.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.551+0000 7f766aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f766c075760 0x7f766c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:14.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.551+0000 7f766aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f766c075760 0x7f766c19c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40804/0 (socket says 172.21.15.38:40804) 2024-03-24T00:37:14.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.551+0000 7f766aa00700 1 -- 172.21.15.38:0/3522781176 learned_addr learned my addr 172.21.15.38:0/3522781176 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:14.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.552+0000 7f766aa00700 1 -- 172.21.15.38:0/3522781176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c0095f0 con 0x7f766c075760 2024-03-24T00:37:14.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.552+0000 7f766aa00700 1 --2- 172.21.15.38:0/3522781176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f766c075760 0x7f766c19c1e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f765c006010 tx=0x7f765c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:14.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.552+0000 7f7663400700 1 -- 172.21.15.38:0/3522781176 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f765c01d070 con 0x7f766c075760 2024-03-24T00:37:14.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.552+0000 7f7663400700 1 -- 172.21.15.38:0/3522781176 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f765c00be20 con 0x7f766c075760 2024-03-24T00:37:14.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.552+0000 7f7663400700 1 -- 172.21.15.38:0/3522781176 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f765c00f460 con 0x7f766c075760 2024-03-24T00:37:14.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.552+0000 7f7671a00700 1 -- 172.21.15.38:0/3522781176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766c19c920 con 0x7f766c075760 2024-03-24T00:37:14.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.552+0000 7f7671a00700 1 -- 172.21.15.38:0/3522781176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c19cdc0 con 0x7f766c075760 2024-03-24T00:37:14.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.554+0000 7f7663400700 1 -- 172.21.15.38:0/3522781176 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f765c00f720 con 0x7f766c075760 2024-03-24T00:37:14.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.554+0000 7f7671a00700 1 -- 172.21.15.38:0/3522781176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766c195b70 con 0x7f766c075760 2024-03-24T00:37:14.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.554+0000 7f7663400700 1 --2- 172.21.15.38:0/3522781176 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f76500387d0 0x7f765003ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:14.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.554+0000 7f7663400700 1 -- 172.21.15.38:0/3522781176 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f765c04cba0 con 0x7f766c075760 2024-03-24T00:37:14.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.555+0000 7f766a000700 1 --2- 172.21.15.38:0/3522781176 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f76500387d0 0x7f765003ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:14.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.556+0000 7f766a000700 1 --2- 172.21.15.38:0/3522781176 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f76500387d0 0x7f765003ac90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7658006fd0 tx=0x7f7658006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:14.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.562+0000 7f7663400700 1 -- 172.21.15.38:0/3522781176 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f765c020dd0 con 0x7f766c075760 2024-03-24T00:37:14.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.798+0000 7f7671a00700 1 -- 172.21.15.38:0/3522781176 --> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f766c0611f0 con 0x7f76500387d0 2024-03-24T00:37:14.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.810+0000 7f7663400700 1 -- 172.21.15.38:0/3522781176 <== mgr.14162 v2:172.21.15.38:6800/1124244493 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f766c0611f0 con 0x7f76500387d0 2024-03-24T00:37:14.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.813+0000 7f7671a00700 1 -- 172.21.15.38:0/3522781176 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f76500387d0 msgr2=0x7f765003ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:14.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.813+0000 7f7671a00700 1 --2- 172.21.15.38:0/3522781176 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f76500387d0 0x7f765003ac90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7658006fd0 tx=0x7f7658006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:14.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.813+0000 7f7671a00700 1 -- 172.21.15.38:0/3522781176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f766c075760 msgr2=0x7f766c19c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:14.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.813+0000 7f7671a00700 1 --2- 172.21.15.38:0/3522781176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f766c075760 0x7f766c19c1e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f765c006010 tx=0x7f765c0050b0 comp rx=0 tx=0).stop 2024-03-24T00:37:14.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.813+0000 7f7671a00700 1 -- 172.21.15.38:0/3522781176 shutdown_connections 2024-03-24T00:37:14.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.813+0000 7f7671a00700 1 --2- 172.21.15.38:0/3522781176 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f76500387d0 0x7f765003ac90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:14.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.813+0000 7f7671a00700 1 --2- 172.21.15.38:0/3522781176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f766c075760 0x7f766c19c1e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:14.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.813+0000 7f7671a00700 1 -- 172.21.15.38:0/3522781176 >> 172.21.15.38:0/3522781176 conn(0x7f766c0fdef0 msgr2=0x7f766c0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:14.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.813+0000 7f7671a00700 1 -- 172.21.15.38:0/3522781176 shutdown_connections 2024-03-24T00:37:14.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:14.813+0000 7f7671a00700 1 -- 172.21.15.38:0/3522781176 wait complete. 2024-03-24T00:37:14.909 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:14 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:14.910 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:14 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:14.910 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:14 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:37:14.910 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:14 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:14.910 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:14 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:37:14.910 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:14 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-24T00:37:14.910 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:14 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:37:14.910 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:14 smithi038 ceph-mon[29271]: Deploying daemon ceph-exporter.smithi038 on smithi038 2024-03-24T00:37:15.970 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:15 smithi038 ceph-mon[29271]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:37:15.970 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:15 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:16.114 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi088 2024-03-24T00:37:16.114 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:37:16.115 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/etc/ceph/ceph.conf 2024-03-24T00:37:16.171 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:37:16.172 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:37:16.265 INFO:tasks.cephadm:Adding host smithi088 to orchestrator... 2024-03-24T00:37:16.265 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph orch host add smithi088 2024-03-24T00:37:16.671 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:37:17.948 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:17 smithi038 ceph-mon[29271]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:18.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.905+0000 7f5056200700 1 -- 172.21.15.38:0/2234436240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50501011d0 msgr2=0x7f50501035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:18.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.905+0000 7f5056200700 1 --2- 172.21.15.38:0/2234436240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50501011d0 0x7f50501035c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5040009910 tx=0x7f5040009c20 comp rx=0 tx=0).stop 2024-03-24T00:37:18.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.906+0000 7f5056200700 1 -- 172.21.15.38:0/2234436240 shutdown_connections 2024-03-24T00:37:18.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.906+0000 7f5056200700 1 --2- 172.21.15.38:0/2234436240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50501011d0 0x7f50501035c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:18.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.906+0000 7f5056200700 1 -- 172.21.15.38:0/2234436240 >> 172.21.15.38:0/2234436240 conn(0x7f50500fabe0 msgr2=0x7f50500fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:18.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.906+0000 7f5056200700 1 -- 172.21.15.38:0/2234436240 shutdown_connections 2024-03-24T00:37:18.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.906+0000 7f5056200700 1 -- 172.21.15.38:0/2234436240 wait complete. 2024-03-24T00:37:18.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.907+0000 7f5056200700 1 Processor -- start 2024-03-24T00:37:18.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.907+0000 7f5056200700 1 -- start start 2024-03-24T00:37:18.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.907+0000 7f5056200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50501011d0 0x7f5050197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:18.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.907+0000 7f5056200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50501982c0 con 0x7f50501011d0 2024-03-24T00:37:18.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.908+0000 7f504f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50501011d0 0x7f5050197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:18.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.908+0000 7f504f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50501011d0 0x7f5050197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40838/0 (socket says 172.21.15.38:40838) 2024-03-24T00:37:18.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.908+0000 7f504f400700 1 -- 172.21.15.38:0/3158914264 learned_addr learned my addr 172.21.15.38:0/3158914264 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:18.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.908+0000 7f504f400700 1 -- 172.21.15.38:0/3158914264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400095f0 con 0x7f50501011d0 2024-03-24T00:37:18.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.908+0000 7f504f400700 1 --2- 172.21.15.38:0/3158914264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50501011d0 0x7f5050197d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5040006010 tx=0x7f50400050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:18.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.909+0000 7f5047e00700 1 -- 172.21.15.38:0/3158914264 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f504001d070 con 0x7f50501011d0 2024-03-24T00:37:18.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.909+0000 7f5047e00700 1 -- 172.21.15.38:0/3158914264 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f504000be20 con 0x7f50501011d0 2024-03-24T00:37:18.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.909+0000 7f5047e00700 1 -- 172.21.15.38:0/3158914264 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f504000f460 con 0x7f50501011d0 2024-03-24T00:37:18.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.909+0000 7f5056200700 1 -- 172.21.15.38:0/3158914264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50501984c0 con 0x7f50501011d0 2024-03-24T00:37:18.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.909+0000 7f5056200700 1 -- 172.21.15.38:0/3158914264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5050198960 con 0x7f50501011d0 2024-03-24T00:37:18.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.910+0000 7f5047e00700 1 -- 172.21.15.38:0/3158914264 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f504000b960 con 0x7f50501011d0 2024-03-24T00:37:18.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.910+0000 7f5056200700 1 -- 172.21.15.38:0/3158914264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5050191af0 con 0x7f50501011d0 2024-03-24T00:37:18.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.911+0000 7f5047e00700 1 --2- 172.21.15.38:0/3158914264 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f5034038820 0x7f503403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:18.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.911+0000 7f5047e00700 1 -- 172.21.15.38:0/3158914264 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f504004c9b0 con 0x7f50501011d0 2024-03-24T00:37:18.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.911+0000 7f504ea00700 1 --2- 172.21.15.38:0/3158914264 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f5034038820 0x7f503403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:18.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.912+0000 7f504ea00700 1 --2- 172.21.15.38:0/3158914264 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f5034038820 0x7f503403ace0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f503c006fd0 tx=0x7f503c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:18.917+0000 7f5047e00700 1 -- 172.21.15.38:0/3158914264 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f504002a360 con 0x7f50501011d0 2024-03-24T00:37:19.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:19.152+0000 7f5056200700 1 -- 172.21.15.38:0/3158914264 --> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi088", "target": ["mon-mgr", ""]}) v1 -- 0x7f50500611f0 con 0x7f5034038820 2024-03-24T00:37:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:19 smithi038 ceph-mon[29271]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:19 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:19 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:19 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:19 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:19 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:37:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:19 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-24T00:37:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:19 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:37:21.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:20 smithi038 ceph-mon[29271]: Deploying daemon crash.smithi038 on smithi038 2024-03-24T00:37:21.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:20 smithi038 ceph-mon[29271]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi088", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:37:22.089 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:21 smithi038 ceph-mon[29271]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:22.089 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:21 smithi038 ceph-mon[29271]: Deploying cephadm binary to smithi088 2024-03-24T00:37:22.089 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:21 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:23.695 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:23 smithi038 ceph-mon[29271]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:23.695 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:23 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:23.695 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:23 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:23.695 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:23 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:23.695 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:23 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:23.696 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:23 smithi038 ceph-mon[29271]: Deploying daemon node-exporter.smithi038 on smithi038 2024-03-24T00:37:23.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:23.733+0000 7f5047e00700 1 -- 172.21.15.38:0/3158914264 <== mgr.14162 v2:172.21.15.38:6800/1124244493 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f50500611f0 con 0x7f5034038820 2024-03-24T00:37:23.736 INFO:teuthology.orchestra.run.smithi038.stdout:Added host 'smithi088' with addr '172.21.15.88' 2024-03-24T00:37:23.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:23.737+0000 7f5056200700 1 -- 172.21.15.38:0/3158914264 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f5034038820 msgr2=0x7f503403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:23.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:23.737+0000 7f5056200700 1 --2- 172.21.15.38:0/3158914264 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f5034038820 0x7f503403ace0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f503c006fd0 tx=0x7f503c006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:23.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:23.737+0000 7f5056200700 1 -- 172.21.15.38:0/3158914264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50501011d0 msgr2=0x7f5050197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:23.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:23.737+0000 7f5056200700 1 --2- 172.21.15.38:0/3158914264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50501011d0 0x7f5050197d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5040006010 tx=0x7f50400050b0 comp rx=0 tx=0).stop 2024-03-24T00:37:23.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:23.737+0000 7f5056200700 1 -- 172.21.15.38:0/3158914264 shutdown_connections 2024-03-24T00:37:23.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:23.737+0000 7f5056200700 1 --2- 172.21.15.38:0/3158914264 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f5034038820 0x7f503403ace0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:23.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:23.737+0000 7f5056200700 1 --2- 172.21.15.38:0/3158914264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50501011d0 0x7f5050197d80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:23.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:23.737+0000 7f5056200700 1 -- 172.21.15.38:0/3158914264 >> 172.21.15.38:0/3158914264 conn(0x7f50500fabe0 msgr2=0x7f50500fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:23.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:23.737+0000 7f5056200700 1 -- 172.21.15.38:0/3158914264 shutdown_connections 2024-03-24T00:37:23.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:23.737+0000 7f5056200700 1 -- 172.21.15.38:0/3158914264 wait complete. 2024-03-24T00:37:24.496 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-24T00:37:24.871 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:37:25.014 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:24 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:25.014 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:24 smithi038 ceph-mon[29271]: Added host smithi088 2024-03-24T00:37:26.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:25 smithi038 ceph-mon[29271]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:27.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:27 smithi038 ceph-mon[29271]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:27.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:27 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:28.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.001+0000 7f64c2800700 1 -- 172.21.15.38:0/1410538241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc107070 msgr2=0x7f64bc107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:28.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.001+0000 7f64c2800700 1 --2- 172.21.15.38:0/1410538241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc107070 0x7f64bc107490 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009920 tx=0x7f64ac009c30 comp rx=0 tx=0).stop 2024-03-24T00:37:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.001+0000 7f64c2800700 1 -- 172.21.15.38:0/1410538241 shutdown_connections 2024-03-24T00:37:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.001+0000 7f64c2800700 1 --2- 172.21.15.38:0/1410538241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc107070 0x7f64bc107490 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.001+0000 7f64c2800700 1 -- 172.21.15.38:0/1410538241 >> 172.21.15.38:0/1410538241 conn(0x7f64bc075e30 msgr2=0x7f64bc078290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.001+0000 7f64c2800700 1 -- 172.21.15.38:0/1410538241 shutdown_connections 2024-03-24T00:37:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.001+0000 7f64c2800700 1 -- 172.21.15.38:0/1410538241 wait complete. 2024-03-24T00:37:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.003+0000 7f64c2800700 1 Processor -- start 2024-03-24T00:37:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.003+0000 7f64c2800700 1 -- start start 2024-03-24T00:37:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.003+0000 7f64c2800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc107070 0x7f64bc19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.003+0000 7f64c2800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc19c790 con 0x7f64bc107070 2024-03-24T00:37:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.004+0000 7f64bb400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc107070 0x7f64bc19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.004+0000 7f64bb400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc107070 0x7f64bc19c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57920/0 (socket says 172.21.15.38:57920) 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.004+0000 7f64bb400700 1 -- 172.21.15.38:0/3704711230 learned_addr learned my addr 172.21.15.38:0/3704711230 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.004+0000 7f64bb400700 1 -- 172.21.15.38:0/3704711230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac009580 con 0x7f64bc107070 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.004+0000 7f64bb400700 1 --2- 172.21.15.38:0/3704711230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc107070 0x7f64bc19c250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f64ac004f60 tx=0x7f64ac005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.005+0000 7f64b8c00700 1 -- 172.21.15.38:0/3704711230 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64ac01d070 con 0x7f64bc107070 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.005+0000 7f64c2800700 1 -- 172.21.15.38:0/3704711230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64bc19c990 con 0x7f64bc107070 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.005+0000 7f64b8c00700 1 -- 172.21.15.38:0/3704711230 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64ac005da0 con 0x7f64bc107070 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.005+0000 7f64b8c00700 1 -- 172.21.15.38:0/3704711230 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64ac00f460 con 0x7f64bc107070 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.005+0000 7f64c2800700 1 -- 172.21.15.38:0/3704711230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64bc19ce30 con 0x7f64bc107070 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.006+0000 7f64b8c00700 1 -- 172.21.15.38:0/3704711230 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f64ac00b960 con 0x7f64bc107070 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.006+0000 7f64c2800700 1 -- 172.21.15.38:0/3704711230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64bc195b80 con 0x7f64bc107070 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.007+0000 7f64b8c00700 1 --2- 172.21.15.38:0/3704711230 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f649c0387c0 0x7f649c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.007+0000 7f64b8c00700 1 -- 172.21.15.38:0/3704711230 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f64ac04c7a0 con 0x7f64bc107070 2024-03-24T00:37:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.007+0000 7f64b2a00700 1 --2- 172.21.15.38:0/3704711230 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f649c0387c0 0x7f649c03ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.008+0000 7f64b2a00700 1 --2- 172.21.15.38:0/3704711230 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f649c0387c0 0x7f649c03ac80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f64a4006fd0 tx=0x7f64a4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.015+0000 7f64b8c00700 1 -- 172.21.15.38:0/3704711230 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f64ac02a360 con 0x7f64bc107070 2024-03-24T00:37:28.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.250+0000 7f64c2800700 1 -- 172.21.15.38:0/3704711230 --> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f64bc0611f0 con 0x7f649c0387c0 2024-03-24T00:37:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.253+0000 7f64b8c00700 1 -- 172.21.15.38:0/3704711230 <== mgr.14162 v2:172.21.15.38:6800/1124244493 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f64bc0611f0 con 0x7f649c0387c0 2024-03-24T00:37:28.254 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:37:28.254 INFO:teuthology.orchestra.run.smithi038.stdout:[{"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.88", "hostname": "smithi088", "labels": [], "status": ""}] 2024-03-24T00:37:28.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.256+0000 7f64c2800700 1 -- 172.21.15.38:0/3704711230 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f649c0387c0 msgr2=0x7f649c03ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:28.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.256+0000 7f64c2800700 1 --2- 172.21.15.38:0/3704711230 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f649c0387c0 0x7f649c03ac80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f64a4006fd0 tx=0x7f64a4006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:28.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.256+0000 7f64c2800700 1 -- 172.21.15.38:0/3704711230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc107070 msgr2=0x7f64bc19c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:28.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.256+0000 7f64c2800700 1 --2- 172.21.15.38:0/3704711230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc107070 0x7f64bc19c250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f64ac004f60 tx=0x7f64ac005040 comp rx=0 tx=0).stop 2024-03-24T00:37:28.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.256+0000 7f64c2800700 1 -- 172.21.15.38:0/3704711230 shutdown_connections 2024-03-24T00:37:28.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.256+0000 7f64c2800700 1 --2- 172.21.15.38:0/3704711230 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f649c0387c0 0x7f649c03ac80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:28.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.256+0000 7f64c2800700 1 --2- 172.21.15.38:0/3704711230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc107070 0x7f64bc19c250 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:28.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.256+0000 7f64c2800700 1 -- 172.21.15.38:0/3704711230 >> 172.21.15.38:0/3704711230 conn(0x7f64bc075e30 msgr2=0x7f64bc076990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:28.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.256+0000 7f64c2800700 1 -- 172.21.15.38:0/3704711230 shutdown_connections 2024-03-24T00:37:28.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:28.256+0000 7f64c2800700 1 -- 172.21.15.38:0/3704711230 wait complete. 2024-03-24T00:37:29.236 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-24T00:37:29.236 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-24T00:37:29.640 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:37:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:29 smithi038 ceph-mon[29271]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:29 smithi038 ceph-mon[29271]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T00:37:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:29 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:29 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:29 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:29 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:31.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:30 smithi038 ceph-mon[29271]: Deploying daemon alertmanager.smithi038 on smithi038 2024-03-24T00:37:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:31 smithi038 ceph-mon[29271]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:31 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:32.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.205+0000 7f0ce4600700 1 -- 172.21.15.38:0/3429317717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc107070 msgr2=0x7f0cdc107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:32.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.205+0000 7f0ce4600700 1 --2- 172.21.15.38:0/3429317717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc107070 0x7f0cdc107490 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009920 tx=0x7f0cd0009c30 comp rx=0 tx=0).stop 2024-03-24T00:37:32.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.205+0000 7f0ce4600700 1 -- 172.21.15.38:0/3429317717 shutdown_connections 2024-03-24T00:37:32.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.205+0000 7f0ce4600700 1 --2- 172.21.15.38:0/3429317717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc107070 0x7f0cdc107490 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:32.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.205+0000 7f0ce4600700 1 -- 172.21.15.38:0/3429317717 >> 172.21.15.38:0/3429317717 conn(0x7f0cdc075e30 msgr2=0x7f0cdc078290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:32.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.206+0000 7f0ce4600700 1 -- 172.21.15.38:0/3429317717 shutdown_connections 2024-03-24T00:37:32.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.206+0000 7f0ce4600700 1 -- 172.21.15.38:0/3429317717 wait complete. 2024-03-24T00:37:32.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.207+0000 7f0ce4600700 1 Processor -- start 2024-03-24T00:37:32.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.207+0000 7f0ce4600700 1 -- start start 2024-03-24T00:37:32.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.207+0000 7f0ce4600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc107070 0x7f0cdc19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:32.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.207+0000 7f0ce4600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc19c790 con 0x7f0cdc107070 2024-03-24T00:37:32.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.207+0000 7f0ce1c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc107070 0x7f0cdc19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:32.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.207+0000 7f0ce1c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc107070 0x7f0cdc19c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57944/0 (socket says 172.21.15.38:57944) 2024-03-24T00:37:32.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.207+0000 7f0ce1c00700 1 -- 172.21.15.38:0/801517397 learned_addr learned my addr 172.21.15.38:0/801517397 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:37:32.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.208+0000 7f0ce1c00700 1 -- 172.21.15.38:0/801517397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd0009580 con 0x7f0cdc107070 2024-03-24T00:37:32.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.208+0000 7f0ce1c00700 1 --2- 172.21.15.38:0/801517397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc107070 0x7f0cdc19c250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0004eb0 tx=0x7f0cd0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:32.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.208+0000 7f0cda000700 1 -- 172.21.15.38:0/801517397 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cd001d070 con 0x7f0cdc107070 2024-03-24T00:37:32.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.208+0000 7f0cda000700 1 -- 172.21.15.38:0/801517397 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cd000bc10 con 0x7f0cdc107070 2024-03-24T00:37:32.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.208+0000 7f0cda000700 1 -- 172.21.15.38:0/801517397 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cd00039d0 con 0x7f0cdc107070 2024-03-24T00:37:32.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.208+0000 7f0ce4600700 1 -- 172.21.15.38:0/801517397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc19c990 con 0x7f0cdc107070 2024-03-24T00:37:32.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.209+0000 7f0ce4600700 1 -- 172.21.15.38:0/801517397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc19ce30 con 0x7f0cdc107070 2024-03-24T00:37:32.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.210+0000 7f0ce4600700 1 -- 172.21.15.38:0/801517397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cdc195b80 con 0x7f0cdc107070 2024-03-24T00:37:32.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.210+0000 7f0cda000700 1 -- 172.21.15.38:0/801517397 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f0cd000bd80 con 0x7f0cdc107070 2024-03-24T00:37:32.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.210+0000 7f0cda000700 1 --2- 172.21.15.38:0/801517397 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f0cc4038860 0x7f0cc403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:32.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.211+0000 7f0cda000700 1 -- 172.21.15.38:0/801517397 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0cd004c350 con 0x7f0cdc107070 2024-03-24T00:37:32.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.211+0000 7f0ce1200700 1 --2- 172.21.15.38:0/801517397 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f0cc4038860 0x7f0cc403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:32.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.212+0000 7f0ce1200700 1 --2- 172.21.15.38:0/801517397 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f0cc4038860 0x7f0cc403ad20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8006fd0 tx=0x7f0cc8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.216+0000 7f0cda000700 1 -- 172.21.15.38:0/801517397 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0cd002abf0 con 0x7f0cdc107070 2024-03-24T00:37:32.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.453+0000 7f0ce4600700 1 -- 172.21.15.38:0/801517397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0cdc0623e0 con 0x7f0cdc107070 2024-03-24T00:37:32.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.872+0000 7f0cda000700 1 -- 172.21.15.38:0/801517397 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0cdc0623e0 con 0x7f0cdc107070 2024-03-24T00:37:32.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.875+0000 7f0ce4600700 1 -- 172.21.15.38:0/801517397 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f0cc4038860 msgr2=0x7f0cc403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:32.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.875+0000 7f0ce4600700 1 --2- 172.21.15.38:0/801517397 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f0cc4038860 0x7f0cc403ad20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8006fd0 tx=0x7f0cc8006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:32.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.875+0000 7f0ce4600700 1 -- 172.21.15.38:0/801517397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc107070 msgr2=0x7f0cdc19c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:32.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.875+0000 7f0ce4600700 1 --2- 172.21.15.38:0/801517397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc107070 0x7f0cdc19c250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0004eb0 tx=0x7f0cd0004f90 comp rx=0 tx=0).stop 2024-03-24T00:37:32.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.876+0000 7f0ce4600700 1 -- 172.21.15.38:0/801517397 shutdown_connections 2024-03-24T00:37:32.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.876+0000 7f0ce4600700 1 --2- 172.21.15.38:0/801517397 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f0cc4038860 0x7f0cc403ad20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:32.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.876+0000 7f0ce4600700 1 --2- 172.21.15.38:0/801517397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cdc107070 0x7f0cdc19c250 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:32.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.876+0000 7f0ce4600700 1 -- 172.21.15.38:0/801517397 >> 172.21.15.38:0/801517397 conn(0x7f0cdc075e30 msgr2=0x7f0cdc076990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:32.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.876+0000 7f0ce4600700 1 -- 172.21.15.38:0/801517397 shutdown_connections 2024-03-24T00:37:32.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:37:32.876+0000 7f0ce4600700 1 -- 172.21.15.38:0/801517397 wait complete. 2024-03-24T00:37:32.879 INFO:teuthology.orchestra.run.smithi038.stderr:adjusted tunables profile to default 2024-03-24T00:37:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:32 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/801517397' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-24T00:37:34.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:33 smithi038 ceph-mon[29271]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:34.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:33 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/801517397' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-24T00:37:34.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:33 smithi038 ceph-mon[29271]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T00:37:34.583 INFO:tasks.cephadm:Adding mon.smithi038 on smithi038 2024-03-24T00:37:34.583 INFO:tasks.cephadm:Adding mon.smithi088 on smithi088 2024-03-24T00:37:34.583 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph orch apply mon '2;smithi038:172.21.15.38=smithi038;smithi088:172.21.15.88=smithi088' 2024-03-24T00:37:34.957 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:35.052 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:36.131 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:35 smithi038 ceph-mon[29271]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:37.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.097+0000 7fda1b200700 1 -- 172.21.15.88:0/136377933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda14100580 msgr2=0x7fda141009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:37.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.097+0000 7fda1b200700 1 --2- 172.21.15.88:0/136377933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda14100580 0x7fda141009a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fda08009910 tx=0x7fda08009c20 comp rx=0 tx=0).stop 2024-03-24T00:37:37.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.097+0000 7fda1b200700 1 -- 172.21.15.88:0/136377933 shutdown_connections 2024-03-24T00:37:37.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.097+0000 7fda1b200700 1 --2- 172.21.15.88:0/136377933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda14100580 0x7fda141009a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:37.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.097+0000 7fda1b200700 1 -- 172.21.15.88:0/136377933 >> 172.21.15.88:0/136377933 conn(0x7fda140fbb20 msgr2=0x7fda140fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:37.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.097+0000 7fda1b200700 1 -- 172.21.15.88:0/136377933 shutdown_connections 2024-03-24T00:37:37.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.097+0000 7fda1b200700 1 -- 172.21.15.88:0/136377933 wait complete. 2024-03-24T00:37:37.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.098+0000 7fda1b200700 1 Processor -- start 2024-03-24T00:37:37.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.099+0000 7fda1b200700 1 -- start start 2024-03-24T00:37:37.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.099+0000 7fda1b200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda14100580 0x7fda14197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:37.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.099+0000 7fda1b200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda14198150 con 0x7fda14100580 2024-03-24T00:37:37.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.099+0000 7fda19e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda14100580 0x7fda14197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:37.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.099+0000 7fda19e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda14100580 0x7fda14197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:58354/0 (socket says 172.21.15.88:58354) 2024-03-24T00:37:37.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.099+0000 7fda19e00700 1 -- 172.21.15.88:0/3062919741 learned_addr learned my addr 172.21.15.88:0/3062919741 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:37:37.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.100+0000 7fda19e00700 1 -- 172.21.15.88:0/3062919741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda080095f0 con 0x7fda14100580 2024-03-24T00:37:37.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.100+0000 7fda19e00700 1 --2- 172.21.15.88:0/3062919741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda14100580 0x7fda14197c10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fda08006010 tx=0x7fda080050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:37.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.100+0000 7fda0ea00700 1 -- 172.21.15.88:0/3062919741 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda0801d070 con 0x7fda14100580 2024-03-24T00:37:37.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.100+0000 7fda0ea00700 1 -- 172.21.15.88:0/3062919741 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda0800be20 con 0x7fda14100580 2024-03-24T00:37:37.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.100+0000 7fda0ea00700 1 -- 172.21.15.88:0/3062919741 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda08022de0 con 0x7fda14100580 2024-03-24T00:37:37.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.100+0000 7fda1b200700 1 -- 172.21.15.88:0/3062919741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda14198350 con 0x7fda14100580 2024-03-24T00:37:37.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.101+0000 7fda1b200700 1 -- 172.21.15.88:0/3062919741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda141987f0 con 0x7fda14100580 2024-03-24T00:37:37.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.102+0000 7fda1b200700 1 -- 172.21.15.88:0/3062919741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda141917b0 con 0x7fda14100580 2024-03-24T00:37:37.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.102+0000 7fda0ea00700 1 -- 172.21.15.88:0/3062919741 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7fda080226c0 con 0x7fda14100580 2024-03-24T00:37:37.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.102+0000 7fda0ea00700 1 --2- 172.21.15.88:0/3062919741 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd9fc038820 0x7fd9fc03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:37.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.102+0000 7fda0ea00700 1 -- 172.21.15.88:0/3062919741 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fda0804c960 con 0x7fda14100580 2024-03-24T00:37:37.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.103+0000 7fda19400700 1 --2- 172.21.15.88:0/3062919741 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd9fc038820 0x7fd9fc03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:37.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.103+0000 7fda19400700 1 --2- 172.21.15.88:0/3062919741 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd9fc038820 0x7fd9fc03ace0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fda04006fd0 tx=0x7fda04006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.109+0000 7fda0ea00700 1 -- 172.21.15.88:0/3062919741 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fda0800f460 con 0x7fda14100580 2024-03-24T00:37:37.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.339+0000 7fda1b200700 1 -- 172.21.15.88:0/3062919741 --> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi038:172.21.15.38=smithi038;smithi088:172.21.15.88=smithi088", "target": ["mon-mgr", ""]}) v1 -- 0x7fda140611f0 con 0x7fd9fc038820 2024-03-24T00:37:37.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.353+0000 7fda0ea00700 1 -- 172.21.15.88:0/3062919741 <== mgr.14162 v2:172.21.15.38:6800/1124244493 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fda140611f0 con 0x7fd9fc038820 2024-03-24T00:37:37.354 INFO:teuthology.orchestra.run.smithi088.stdout:Scheduled mon update... 2024-03-24T00:37:37.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.356+0000 7fda1b200700 1 -- 172.21.15.88:0/3062919741 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd9fc038820 msgr2=0x7fd9fc03ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:37.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.356+0000 7fda1b200700 1 --2- 172.21.15.88:0/3062919741 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd9fc038820 0x7fd9fc03ace0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fda04006fd0 tx=0x7fda04006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:37.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.356+0000 7fda1b200700 1 -- 172.21.15.88:0/3062919741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda14100580 msgr2=0x7fda14197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:37.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.356+0000 7fda1b200700 1 --2- 172.21.15.88:0/3062919741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda14100580 0x7fda14197c10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fda08006010 tx=0x7fda080050b0 comp rx=0 tx=0).stop 2024-03-24T00:37:37.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.356+0000 7fda1b200700 1 -- 172.21.15.88:0/3062919741 shutdown_connections 2024-03-24T00:37:37.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.356+0000 7fda1b200700 1 --2- 172.21.15.88:0/3062919741 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd9fc038820 0x7fd9fc03ace0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:37.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.356+0000 7fda1b200700 1 --2- 172.21.15.88:0/3062919741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda14100580 0x7fda14197c10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:37.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.356+0000 7fda1b200700 1 -- 172.21.15.88:0/3062919741 >> 172.21.15.88:0/3062919741 conn(0x7fda140fbb20 msgr2=0x7fda140fc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:37.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.356+0000 7fda1b200700 1 -- 172.21.15.88:0/3062919741 shutdown_connections 2024-03-24T00:37:37.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:37.356+0000 7fda1b200700 1 -- 172.21.15.88:0/3062919741 wait complete. 2024-03-24T00:37:38.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:37 smithi038 ceph-mon[29271]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:38.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:37 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:38.183 DEBUG:teuthology.orchestra.run.smithi088:mon.smithi088> sudo journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi088.service 2024-03-24T00:37:38.191 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:37:38.191 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:37:38.561 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:38.665 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:39.002 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:38 smithi038 ceph-mon[29271]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi038:172.21.15.38=smithi038;smithi088:172.21.15.88=smithi088", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:37:39.002 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:38 smithi038 ceph-mon[29271]: Saving service mon spec with placement smithi038:172.21.15.38=smithi038;smithi088:172.21.15.88=smithi088;count:2 2024-03-24T00:37:40.292 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:39 smithi038 ceph-mon[29271]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:40.292 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:39 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:40.293 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:39 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:40.293 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:39 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:40.293 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:39 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:40.293 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:39 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:40.293 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:39 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:40.293 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:39 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:40.293 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:39 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:40.293 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:39 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T00:37:40.293 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:39 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.660+0000 7ff964c00700 1 -- 172.21.15.88:0/3878356161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9600fe820 msgr2=0x7ff9600fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.660+0000 7ff964c00700 1 --2- 172.21.15.88:0/3878356161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9600fe820 0x7ff9600fec40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff950009910 tx=0x7ff950009c20 comp rx=0 tx=0).stop 2024-03-24T00:37:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.660+0000 7ff964c00700 1 -- 172.21.15.88:0/3878356161 shutdown_connections 2024-03-24T00:37:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.660+0000 7ff964c00700 1 --2- 172.21.15.88:0/3878356161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9600fe820 0x7ff9600fec40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.660+0000 7ff964c00700 1 -- 172.21.15.88:0/3878356161 >> 172.21.15.88:0/3878356161 conn(0x7ff9600fa1a0 msgr2=0x7ff9600fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.660+0000 7ff964c00700 1 -- 172.21.15.88:0/3878356161 shutdown_connections 2024-03-24T00:37:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.660+0000 7ff964c00700 1 -- 172.21.15.88:0/3878356161 wait complete. 2024-03-24T00:37:40.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.661+0000 7ff964c00700 1 Processor -- start 2024-03-24T00:37:40.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.661+0000 7ff964c00700 1 -- start start 2024-03-24T00:37:40.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.662+0000 7ff964c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9600fe820 0x7ff960101030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:40.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.662+0000 7ff964c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff960101570 con 0x7ff9600fe820 2024-03-24T00:37:40.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.662+0000 7ff95f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9600fe820 0x7ff960101030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:40.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.662+0000 7ff95f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9600fe820 0x7ff960101030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:58360/0 (socket says 172.21.15.88:58360) 2024-03-24T00:37:40.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.662+0000 7ff95f400700 1 -- 172.21.15.88:0/3429694333 learned_addr learned my addr 172.21.15.88:0/3429694333 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:37:40.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.662+0000 7ff95f400700 1 -- 172.21.15.88:0/3429694333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500095f0 con 0x7ff9600fe820 2024-03-24T00:37:40.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.663+0000 7ff95f400700 1 --2- 172.21.15.88:0/3429694333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9600fe820 0x7ff960101030 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff950006010 tx=0x7ff9500050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:40.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.663+0000 7ff957e00700 1 -- 172.21.15.88:0/3429694333 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff95001d070 con 0x7ff9600fe820 2024-03-24T00:37:40.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.663+0000 7ff957e00700 1 -- 172.21.15.88:0/3429694333 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff95000be20 con 0x7ff9600fe820 2024-03-24T00:37:40.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.663+0000 7ff964c00700 1 -- 172.21.15.88:0/3429694333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9600ff7b0 con 0x7ff9600fe820 2024-03-24T00:37:40.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.663+0000 7ff957e00700 1 -- 172.21.15.88:0/3429694333 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff950022de0 con 0x7ff9600fe820 2024-03-24T00:37:40.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.663+0000 7ff964c00700 1 -- 172.21.15.88:0/3429694333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9600ffc50 con 0x7ff9600fe820 2024-03-24T00:37:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.664+0000 7ff957e00700 1 -- 172.21.15.88:0/3429694333 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7ff9500226c0 con 0x7ff9600fe820 2024-03-24T00:37:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.664+0000 7ff964c00700 1 -- 172.21.15.88:0/3429694333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff960100660 con 0x7ff9600fe820 2024-03-24T00:37:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.665+0000 7ff957e00700 1 --2- 172.21.15.88:0/3429694333 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7ff9440387b0 0x7ff94403ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.665+0000 7ff957e00700 1 -- 172.21.15.88:0/3429694333 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff95004c960 con 0x7ff9600fe820 2024-03-24T00:37:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.665+0000 7ff95ea00700 1 --2- 172.21.15.88:0/3429694333 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7ff9440387b0 0x7ff94403ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:40.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.666+0000 7ff95ea00700 1 --2- 172.21.15.88:0/3429694333 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7ff9440387b0 0x7ff94403ac70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff94c006fd0 tx=0x7ff94c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:40.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.675+0000 7ff957e00700 1 -- 172.21.15.88:0/3429694333 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff95002ba00 con 0x7ff9600fe820 2024-03-24T00:37:40.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.982+0000 7ff964c00700 1 -- 172.21.15.88:0/3429694333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff9600ff350 con 0x7ff9600fe820 2024-03-24T00:37:40.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.983+0000 7ff957e00700 1 -- 172.21.15.88:0/3429694333 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff9600ff350 con 0x7ff9600fe820 2024-03-24T00:37:40.984 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:37:40.984 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:37:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.986+0000 7ff964c00700 1 -- 172.21.15.88:0/3429694333 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7ff9440387b0 msgr2=0x7ff94403ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.986+0000 7ff964c00700 1 --2- 172.21.15.88:0/3429694333 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7ff9440387b0 0x7ff94403ac70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff94c006fd0 tx=0x7ff94c006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.986+0000 7ff964c00700 1 -- 172.21.15.88:0/3429694333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9600fe820 msgr2=0x7ff960101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.986+0000 7ff964c00700 1 --2- 172.21.15.88:0/3429694333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9600fe820 0x7ff960101030 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff950006010 tx=0x7ff9500050b0 comp rx=0 tx=0).stop 2024-03-24T00:37:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.986+0000 7ff964c00700 1 -- 172.21.15.88:0/3429694333 shutdown_connections 2024-03-24T00:37:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.986+0000 7ff964c00700 1 --2- 172.21.15.88:0/3429694333 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7ff9440387b0 0x7ff94403ac70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.986+0000 7ff964c00700 1 --2- 172.21.15.88:0/3429694333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9600fe820 0x7ff960101030 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.986+0000 7ff964c00700 1 -- 172.21.15.88:0/3429694333 >> 172.21.15.88:0/3429694333 conn(0x7ff9600fa1a0 msgr2=0x7ff9600fac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.986+0000 7ff964c00700 1 -- 172.21.15.88:0/3429694333 shutdown_connections 2024-03-24T00:37:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:40.986+0000 7ff964c00700 1 -- 172.21.15.88:0/3429694333 wait complete. 2024-03-24T00:37:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:37:41.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:40 smithi038 ceph-mon[29271]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-24T00:37:41.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:40 smithi038 ceph-mon[29271]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T00:37:41.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:40 smithi038 ceph-mon[29271]: Deploying daemon grafana.smithi038 on smithi038 2024-03-24T00:37:42.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:41 smithi038 ceph-mon[29271]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:42.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:41 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/3429694333' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:37:42.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:41 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:37:42.800 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:37:42.800 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:37:43.169 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:43.265 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:43.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:42 smithi038 ceph-mon[29271]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:45.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.143+0000 7fb5e0400700 1 -- 172.21.15.88:0/2562094716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d8102950 msgr2=0x7fb5d8102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:45.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.143+0000 7fb5e0400700 1 --2- 172.21.15.88:0/2562094716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d8102950 0x7fb5d8102d70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009920 tx=0x7fb5cc009c30 comp rx=0 tx=0).stop 2024-03-24T00:37:45.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.144+0000 7fb5e0400700 1 -- 172.21.15.88:0/2562094716 shutdown_connections 2024-03-24T00:37:45.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.144+0000 7fb5e0400700 1 --2- 172.21.15.88:0/2562094716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d8102950 0x7fb5d8102d70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:45.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.144+0000 7fb5e0400700 1 -- 172.21.15.88:0/2562094716 >> 172.21.15.88:0/2562094716 conn(0x7fb5d80fded0 msgr2=0x7fb5d8100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:45.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.144+0000 7fb5e0400700 1 -- 172.21.15.88:0/2562094716 shutdown_connections 2024-03-24T00:37:45.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.144+0000 7fb5e0400700 1 -- 172.21.15.88:0/2562094716 wait complete. 2024-03-24T00:37:45.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.145+0000 7fb5e0400700 1 Processor -- start 2024-03-24T00:37:45.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.145+0000 7fb5e0400700 1 -- start start 2024-03-24T00:37:45.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.146+0000 7fb5e0400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d8102950 0x7fb5d8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:45.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.146+0000 7fb5e0400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d81983a0 con 0x7fb5d8102950 2024-03-24T00:37:45.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.146+0000 7fb5dda00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d8102950 0x7fb5d8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:45.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.146+0000 7fb5dda00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d8102950 0x7fb5d8197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:53712/0 (socket says 172.21.15.88:53712) 2024-03-24T00:37:45.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.146+0000 7fb5dda00700 1 -- 172.21.15.88:0/2909734826 learned_addr learned my addr 172.21.15.88:0/2909734826 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:37:45.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.147+0000 7fb5dda00700 1 -- 172.21.15.88:0/2909734826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc009580 con 0x7fb5d8102950 2024-03-24T00:37:45.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.147+0000 7fb5dda00700 1 --2- 172.21.15.88:0/2909734826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d8102950 0x7fb5d8197e60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc004eb0 tx=0x7fb5cc004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:45.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.148+0000 7fb5d6000700 1 -- 172.21.15.88:0/2909734826 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5cc01d070 con 0x7fb5d8102950 2024-03-24T00:37:45.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.148+0000 7fb5e0400700 1 -- 172.21.15.88:0/2909734826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d81985a0 con 0x7fb5d8102950 2024-03-24T00:37:45.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.148+0000 7fb5e0400700 1 -- 172.21.15.88:0/2909734826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d8198a40 con 0x7fb5d8102950 2024-03-24T00:37:45.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.148+0000 7fb5d6000700 1 -- 172.21.15.88:0/2909734826 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5cc00bc10 con 0x7fb5d8102950 2024-03-24T00:37:45.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.148+0000 7fb5d6000700 1 -- 172.21.15.88:0/2909734826 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5cc003950 con 0x7fb5d8102950 2024-03-24T00:37:45.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.149+0000 7fb5e0400700 1 -- 172.21.15.88:0/2909734826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d8191780 con 0x7fb5d8102950 2024-03-24T00:37:45.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.150+0000 7fb5d6000700 1 -- 172.21.15.88:0/2909734826 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7fb5cc00bd80 con 0x7fb5d8102950 2024-03-24T00:37:45.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.150+0000 7fb5d6000700 1 --2- 172.21.15.88:0/2909734826 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb5c0038860 0x7fb5c003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:45.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.150+0000 7fb5d6000700 1 -- 172.21.15.88:0/2909734826 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb5cc04c4f0 con 0x7fb5d8102950 2024-03-24T00:37:45.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.151+0000 7fb5dd000700 1 --2- 172.21.15.88:0/2909734826 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb5c0038860 0x7fb5c003ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:45.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.153+0000 7fb5dd000700 1 --2- 172.21.15.88:0/2909734826 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb5c0038860 0x7fb5c003ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4006fd0 tx=0x7fb5c4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:45.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.166+0000 7fb5d6000700 1 -- 172.21.15.88:0/2909734826 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb5cc003b10 con 0x7fb5d8102950 2024-03-24T00:37:45.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.467+0000 7fb5e0400700 1 -- 172.21.15.88:0/2909734826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb5d80623e0 con 0x7fb5d8102950 2024-03-24T00:37:45.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.469+0000 7fb5d6000700 1 -- 172.21.15.88:0/2909734826 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fb5cc026020 con 0x7fb5d8102950 2024-03-24T00:37:45.470 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:37:45.470 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:37:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.472+0000 7fb5e0400700 1 -- 172.21.15.88:0/2909734826 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb5c0038860 msgr2=0x7fb5c003ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.472+0000 7fb5e0400700 1 --2- 172.21.15.88:0/2909734826 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb5c0038860 0x7fb5c003ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4006fd0 tx=0x7fb5c4006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.472+0000 7fb5e0400700 1 -- 172.21.15.88:0/2909734826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d8102950 msgr2=0x7fb5d8197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.472+0000 7fb5e0400700 1 --2- 172.21.15.88:0/2909734826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d8102950 0x7fb5d8197e60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc004eb0 tx=0x7fb5cc004f90 comp rx=0 tx=0).stop 2024-03-24T00:37:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.472+0000 7fb5e0400700 1 -- 172.21.15.88:0/2909734826 shutdown_connections 2024-03-24T00:37:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.472+0000 7fb5e0400700 1 --2- 172.21.15.88:0/2909734826 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb5c0038860 0x7fb5c003ad20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.472+0000 7fb5e0400700 1 --2- 172.21.15.88:0/2909734826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d8102950 0x7fb5d8197e60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.472+0000 7fb5e0400700 1 -- 172.21.15.88:0/2909734826 >> 172.21.15.88:0/2909734826 conn(0x7fb5d80fded0 msgr2=0x7fb5d80fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.472+0000 7fb5e0400700 1 -- 172.21.15.88:0/2909734826 shutdown_connections 2024-03-24T00:37:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:45.472+0000 7fb5e0400700 1 -- 172.21.15.88:0/2909734826 wait complete. 2024-03-24T00:37:45.474 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:37:45.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:45 smithi038 ceph-mon[29271]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:46.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:46 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/2909734826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:37:47.186 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:37:47.186 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:37:47.560 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:47.654 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:47.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:47 smithi038 ceph-mon[29271]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:49.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.601+0000 7f3793200700 1 -- 172.21.15.88:0/3205602701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37941027f0 msgr2=0x7f3794102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:49.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.601+0000 7f3793200700 1 --2- 172.21.15.88:0/3205602701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37941027f0 0x7f3794102c10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3784009910 tx=0x7f3784009c20 comp rx=0 tx=0).stop 2024-03-24T00:37:49.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.601+0000 7f3793200700 1 -- 172.21.15.88:0/3205602701 shutdown_connections 2024-03-24T00:37:49.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.601+0000 7f3793200700 1 --2- 172.21.15.88:0/3205602701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37941027f0 0x7f3794102c10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:49.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.601+0000 7f3793200700 1 -- 172.21.15.88:0/3205602701 >> 172.21.15.88:0/3205602701 conn(0x7f37940fdd90 msgr2=0x7f37941001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:49.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.602+0000 7f3793200700 1 -- 172.21.15.88:0/3205602701 shutdown_connections 2024-03-24T00:37:49.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.602+0000 7f3793200700 1 -- 172.21.15.88:0/3205602701 wait complete. 2024-03-24T00:37:49.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.603+0000 7f3793200700 1 Processor -- start 2024-03-24T00:37:49.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.603+0000 7f3793200700 1 -- start start 2024-03-24T00:37:49.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.604+0000 7f3793200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37941027f0 0x7f3794197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:49.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.604+0000 7f3793200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37941981d0 con 0x7f37941027f0 2024-03-24T00:37:49.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.604+0000 7f3791e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37941027f0 0x7f3794197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:49.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.604+0000 7f3791e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37941027f0 0x7f3794197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:53724/0 (socket says 172.21.15.88:53724) 2024-03-24T00:37:49.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.604+0000 7f3791e00700 1 -- 172.21.15.88:0/1350098508 learned_addr learned my addr 172.21.15.88:0/1350098508 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:37:49.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.605+0000 7f3791e00700 1 -- 172.21.15.88:0/1350098508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37840095f0 con 0x7f37941027f0 2024-03-24T00:37:49.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.606+0000 7f3791e00700 1 --2- 172.21.15.88:0/1350098508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37941027f0 0x7f3794197c90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3784006010 tx=0x7f37840050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:49.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.606+0000 7f378aa00700 1 -- 172.21.15.88:0/1350098508 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f378401d070 con 0x7f37941027f0 2024-03-24T00:37:49.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.606+0000 7f378aa00700 1 -- 172.21.15.88:0/1350098508 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f378400be20 con 0x7f37941027f0 2024-03-24T00:37:49.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.606+0000 7f378aa00700 1 -- 172.21.15.88:0/1350098508 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3784022de0 con 0x7f37941027f0 2024-03-24T00:37:49.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.606+0000 7f3793200700 1 -- 172.21.15.88:0/1350098508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37941983d0 con 0x7f37941027f0 2024-03-24T00:37:49.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.606+0000 7f3793200700 1 -- 172.21.15.88:0/1350098508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37941987f0 con 0x7f37941027f0 2024-03-24T00:37:49.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.607+0000 7f3793200700 1 -- 172.21.15.88:0/1350098508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3794191620 con 0x7f37941027f0 2024-03-24T00:37:49.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.608+0000 7f378aa00700 1 -- 172.21.15.88:0/1350098508 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f37840226c0 con 0x7f37941027f0 2024-03-24T00:37:49.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.608+0000 7f378aa00700 1 --2- 172.21.15.88:0/1350098508 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f3778038820 0x7f377803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:49.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.608+0000 7f378aa00700 1 -- 172.21.15.88:0/1350098508 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f378404cac0 con 0x7f37941027f0 2024-03-24T00:37:49.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.609+0000 7f3791400700 1 --2- 172.21.15.88:0/1350098508 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f3778038820 0x7f377803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:49.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.610+0000 7f3791400700 1 --2- 172.21.15.88:0/1350098508 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f3778038820 0x7f377803ace0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3780006fd0 tx=0x7f3780006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:49.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.626+0000 7f378aa00700 1 -- 172.21.15.88:0/1350098508 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f378402ba00 con 0x7f37941027f0 2024-03-24T00:37:49.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:49 smithi038 ceph-mon[29271]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.931+0000 7f3793200700 1 -- 172.21.15.88:0/1350098508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f379402d140 con 0x7f37941027f0 2024-03-24T00:37:49.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.932+0000 7f378aa00700 1 -- 172.21.15.88:0/1350098508 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f3784027030 con 0x7f37941027f0 2024-03-24T00:37:49.933 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:37:49.933 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:37:49.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.935+0000 7f3793200700 1 -- 172.21.15.88:0/1350098508 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f3778038820 msgr2=0x7f377803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:49.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.935+0000 7f3793200700 1 --2- 172.21.15.88:0/1350098508 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f3778038820 0x7f377803ace0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3780006fd0 tx=0x7f3780006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:49.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.935+0000 7f3793200700 1 -- 172.21.15.88:0/1350098508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37941027f0 msgr2=0x7f3794197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:49.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.935+0000 7f3793200700 1 --2- 172.21.15.88:0/1350098508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37941027f0 0x7f3794197c90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3784006010 tx=0x7f37840050b0 comp rx=0 tx=0).stop 2024-03-24T00:37:49.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.935+0000 7f3793200700 1 -- 172.21.15.88:0/1350098508 shutdown_connections 2024-03-24T00:37:49.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.935+0000 7f3793200700 1 --2- 172.21.15.88:0/1350098508 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f3778038820 0x7f377803ace0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:49.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.935+0000 7f3793200700 1 --2- 172.21.15.88:0/1350098508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37941027f0 0x7f3794197c90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:49.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.935+0000 7f3793200700 1 -- 172.21.15.88:0/1350098508 >> 172.21.15.88:0/1350098508 conn(0x7f37940fdd90 msgr2=0x7f37940fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:49.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.935+0000 7f3793200700 1 -- 172.21.15.88:0/1350098508 shutdown_connections 2024-03-24T00:37:49.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:49.935+0000 7f3793200700 1 -- 172.21.15.88:0/1350098508 wait complete. 2024-03-24T00:37:49.937 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:37:50.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:50 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/1350098508' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:37:51.648 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:37:51.649 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:37:51.761 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:51 smithi038 ceph-mon[29271]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:52.026 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:52.120 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:53.794 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:53 smithi038 ceph-mon[29271]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:54.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.064+0000 7f91d2c00700 1 -- 172.21.15.88:0/3008970714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91cc107070 msgr2=0x7f91cc107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:54.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.064+0000 7f91d2c00700 1 --2- 172.21.15.88:0/3008970714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91cc107070 0x7f91cc107490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009910 tx=0x7f91bc009c20 comp rx=0 tx=0).stop 2024-03-24T00:37:54.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.064+0000 7f91d2c00700 1 -- 172.21.15.88:0/3008970714 shutdown_connections 2024-03-24T00:37:54.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.064+0000 7f91d2c00700 1 --2- 172.21.15.88:0/3008970714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91cc107070 0x7f91cc107490 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:54.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.064+0000 7f91d2c00700 1 -- 172.21.15.88:0/3008970714 >> 172.21.15.88:0/3008970714 conn(0x7f91cc075e30 msgr2=0x7f91cc078290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:54.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.064+0000 7f91d2c00700 1 -- 172.21.15.88:0/3008970714 shutdown_connections 2024-03-24T00:37:54.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.064+0000 7f91d2c00700 1 -- 172.21.15.88:0/3008970714 wait complete. 2024-03-24T00:37:54.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.065+0000 7f91d2c00700 1 Processor -- start 2024-03-24T00:37:54.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.066+0000 7f91d2c00700 1 -- start start 2024-03-24T00:37:54.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.066+0000 7f91d2c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91cc107070 0x7f91cc19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:54.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.066+0000 7f91d2c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc19c700 con 0x7f91cc107070 2024-03-24T00:37:54.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.067+0000 7f91cbe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91cc107070 0x7f91cc19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:54.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.067+0000 7f91cbe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91cc107070 0x7f91cc19c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:59448/0 (socket says 172.21.15.88:59448) 2024-03-24T00:37:54.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.067+0000 7f91cbe00700 1 -- 172.21.15.88:0/2777042129 learned_addr learned my addr 172.21.15.88:0/2777042129 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:37:54.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.067+0000 7f91cbe00700 1 -- 172.21.15.88:0/2777042129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc0095f0 con 0x7f91cc107070 2024-03-24T00:37:54.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.068+0000 7f91cbe00700 1 --2- 172.21.15.88:0/2777042129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91cc107070 0x7f91cc19c1c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f91bc006010 tx=0x7f91bc0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:54.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.068+0000 7f91c8c00700 1 -- 172.21.15.88:0/2777042129 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91bc01d070 con 0x7f91cc107070 2024-03-24T00:37:54.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.068+0000 7f91c8c00700 1 -- 172.21.15.88:0/2777042129 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91bc00be20 con 0x7f91cc107070 2024-03-24T00:37:54.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.068+0000 7f91d2c00700 1 -- 172.21.15.88:0/2777042129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc19c900 con 0x7f91cc107070 2024-03-24T00:37:54.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.068+0000 7f91c8c00700 1 -- 172.21.15.88:0/2777042129 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91bc022de0 con 0x7f91cc107070 2024-03-24T00:37:54.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.069+0000 7f91d2c00700 1 -- 172.21.15.88:0/2777042129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc19cda0 con 0x7f91cc107070 2024-03-24T00:37:54.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.070+0000 7f91d2c00700 1 -- 172.21.15.88:0/2777042129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc195b80 con 0x7f91cc107070 2024-03-24T00:37:54.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.070+0000 7f91c8c00700 1 -- 172.21.15.88:0/2777042129 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f91bc0226c0 con 0x7f91cc107070 2024-03-24T00:37:54.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.070+0000 7f91c8c00700 1 --2- 172.21.15.88:0/2777042129 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f91b0038820 0x7f91b003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:54.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.070+0000 7f91c8c00700 1 -- 172.21.15.88:0/2777042129 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f91bc04c9b0 con 0x7f91cc107070 2024-03-24T00:37:54.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.071+0000 7f91cb400700 1 --2- 172.21.15.88:0/2777042129 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f91b0038820 0x7f91b003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:54.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.072+0000 7f91cb400700 1 --2- 172.21.15.88:0/2777042129 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f91b0038820 0x7f91b003ace0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f91b8006fd0 tx=0x7f91b8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:54.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.087+0000 7f91c8c00700 1 -- 172.21.15.88:0/2777042129 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f91bc02b590 con 0x7f91cc107070 2024-03-24T00:37:54.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.390+0000 7f91d2c00700 1 -- 172.21.15.88:0/2777042129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f91cc0623e0 con 0x7f91cc107070 2024-03-24T00:37:54.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.391+0000 7f91c8c00700 1 -- 172.21.15.88:0/2777042129 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f91bc027030 con 0x7f91cc107070 2024-03-24T00:37:54.392 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:37:54.392 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:37:54.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.394+0000 7f91d2c00700 1 -- 172.21.15.88:0/2777042129 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f91b0038820 msgr2=0x7f91b003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:54.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.394+0000 7f91d2c00700 1 --2- 172.21.15.88:0/2777042129 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f91b0038820 0x7f91b003ace0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f91b8006fd0 tx=0x7f91b8006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:54.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.394+0000 7f91d2c00700 1 -- 172.21.15.88:0/2777042129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91cc107070 msgr2=0x7f91cc19c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:54.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.394+0000 7f91d2c00700 1 --2- 172.21.15.88:0/2777042129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91cc107070 0x7f91cc19c1c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f91bc006010 tx=0x7f91bc0050b0 comp rx=0 tx=0).stop 2024-03-24T00:37:54.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.394+0000 7f91d2c00700 1 -- 172.21.15.88:0/2777042129 shutdown_connections 2024-03-24T00:37:54.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.394+0000 7f91d2c00700 1 --2- 172.21.15.88:0/2777042129 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f91b0038820 0x7f91b003ace0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:54.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.394+0000 7f91d2c00700 1 --2- 172.21.15.88:0/2777042129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91cc107070 0x7f91cc19c1c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:54.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.394+0000 7f91d2c00700 1 -- 172.21.15.88:0/2777042129 >> 172.21.15.88:0/2777042129 conn(0x7f91cc075e30 msgr2=0x7f91cc0768a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:54.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.394+0000 7f91d2c00700 1 -- 172.21.15.88:0/2777042129 shutdown_connections 2024-03-24T00:37:54.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:54.394+0000 7f91d2c00700 1 -- 172.21.15.88:0/2777042129 wait complete. 2024-03-24T00:37:54.395 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:37:54.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:54 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/2777042129' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:37:55.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:55 smithi038 ceph-mon[29271]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:56.158 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:37:56.158 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:37:56.521 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:56.616 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:37:57.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:57 smithi038 ceph-mon[29271]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:58.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.583+0000 7fd7ac400700 1 -- 172.21.15.88:0/1346601960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4102960 msgr2=0x7fd7a4102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:58.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.583+0000 7fd7ac400700 1 --2- 172.21.15.88:0/1346601960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4102960 0x7fd7a4102d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd798009910 tx=0x7fd798009c20 comp rx=0 tx=0).stop 2024-03-24T00:37:58.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.583+0000 7fd7ac400700 1 -- 172.21.15.88:0/1346601960 shutdown_connections 2024-03-24T00:37:58.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.583+0000 7fd7ac400700 1 --2- 172.21.15.88:0/1346601960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4102960 0x7fd7a4102d80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:58.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.583+0000 7fd7ac400700 1 -- 172.21.15.88:0/1346601960 >> 172.21.15.88:0/1346601960 conn(0x7fd7a40fdf20 msgr2=0x7fd7a4100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:58.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.583+0000 7fd7ac400700 1 -- 172.21.15.88:0/1346601960 shutdown_connections 2024-03-24T00:37:58.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.583+0000 7fd7ac400700 1 -- 172.21.15.88:0/1346601960 wait complete. 2024-03-24T00:37:58.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.585+0000 7fd7ac400700 1 Processor -- start 2024-03-24T00:37:58.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.585+0000 7fd7ac400700 1 -- start start 2024-03-24T00:37:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.585+0000 7fd7ac400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4102960 0x7fd7a4197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.585+0000 7fd7ac400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a41982e0 con 0x7fd7a4102960 2024-03-24T00:37:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.586+0000 7fd7a9a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4102960 0x7fd7a4197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.586+0000 7fd7a9a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4102960 0x7fd7a4197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:59464/0 (socket says 172.21.15.88:59464) 2024-03-24T00:37:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.586+0000 7fd7a9a00700 1 -- 172.21.15.88:0/1426511280 learned_addr learned my addr 172.21.15.88:0/1426511280 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:37:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.586+0000 7fd7a9a00700 1 -- 172.21.15.88:0/1426511280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7980095f0 con 0x7fd7a4102960 2024-03-24T00:37:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.586+0000 7fd7a9a00700 1 --2- 172.21.15.88:0/1426511280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4102960 0x7fd7a4197da0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd798006010 tx=0x7fd798009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:58.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.587+0000 7fd7a2000700 1 -- 172.21.15.88:0/1426511280 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd79801d070 con 0x7fd7a4102960 2024-03-24T00:37:58.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.587+0000 7fd7ac400700 1 -- 172.21.15.88:0/1426511280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a41984e0 con 0x7fd7a4102960 2024-03-24T00:37:58.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.587+0000 7fd7ac400700 1 -- 172.21.15.88:0/1426511280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a4198980 con 0x7fd7a4102960 2024-03-24T00:37:58.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.587+0000 7fd7a2000700 1 -- 172.21.15.88:0/1426511280 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd798005dc0 con 0x7fd7a4102960 2024-03-24T00:37:58.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.587+0000 7fd7a2000700 1 -- 172.21.15.88:0/1426511280 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd798022e70 con 0x7fd7a4102960 2024-03-24T00:37:58.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.588+0000 7fd7a2000700 1 -- 172.21.15.88:0/1426511280 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7fd798022730 con 0x7fd7a4102960 2024-03-24T00:37:58.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.588+0000 7fd7ac400700 1 -- 172.21.15.88:0/1426511280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a4191790 con 0x7fd7a4102960 2024-03-24T00:37:58.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.589+0000 7fd7a2000700 1 --2- 172.21.15.88:0/1426511280 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd78c038770 0x7fd78c03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:37:58.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.589+0000 7fd7a2000700 1 -- 172.21.15.88:0/1426511280 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd79804c8a0 con 0x7fd7a4102960 2024-03-24T00:37:58.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.589+0000 7fd7a9000700 1 --2- 172.21.15.88:0/1426511280 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd78c038770 0x7fd78c03ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:37:58.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.590+0000 7fd7a9000700 1 --2- 172.21.15.88:0/1426511280 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd78c038770 0x7fd78c03ac30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd790006fd0 tx=0x7fd790006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:37:58.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.604+0000 7fd7a2000700 1 -- 172.21.15.88:0/1426511280 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd79800b960 con 0x7fd7a4102960 2024-03-24T00:37:58.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.907+0000 7fd7ac400700 1 -- 172.21.15.88:0/1426511280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd7a40623e0 con 0x7fd7a4102960 2024-03-24T00:37:58.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.908+0000 7fd7a2000700 1 -- 172.21.15.88:0/1426511280 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fd798027030 con 0x7fd7a4102960 2024-03-24T00:37:58.909 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:37:58.910 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:37:58.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.911+0000 7fd7ac400700 1 -- 172.21.15.88:0/1426511280 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd78c038770 msgr2=0x7fd78c03ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:58.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.911+0000 7fd7ac400700 1 --2- 172.21.15.88:0/1426511280 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd78c038770 0x7fd78c03ac30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd790006fd0 tx=0x7fd790006e40 comp rx=0 tx=0).stop 2024-03-24T00:37:58.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.911+0000 7fd7ac400700 1 -- 172.21.15.88:0/1426511280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4102960 msgr2=0x7fd7a4197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:37:58.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.911+0000 7fd7ac400700 1 --2- 172.21.15.88:0/1426511280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4102960 0x7fd7a4197da0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd798006010 tx=0x7fd798009f80 comp rx=0 tx=0).stop 2024-03-24T00:37:58.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.911+0000 7fd7ac400700 1 -- 172.21.15.88:0/1426511280 shutdown_connections 2024-03-24T00:37:58.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.911+0000 7fd7ac400700 1 --2- 172.21.15.88:0/1426511280 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fd78c038770 0x7fd78c03ac30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:58.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.911+0000 7fd7ac400700 1 --2- 172.21.15.88:0/1426511280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a4102960 0x7fd7a4197da0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:37:58.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.911+0000 7fd7ac400700 1 -- 172.21.15.88:0/1426511280 >> 172.21.15.88:0/1426511280 conn(0x7fd7a40fdf20 msgr2=0x7fd7a40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:37:58.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.911+0000 7fd7ac400700 1 -- 172.21.15.88:0/1426511280 shutdown_connections 2024-03-24T00:37:58.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:37:58.912+0000 7fd7ac400700 1 -- 172.21.15.88:0/1426511280 wait complete. 2024-03-24T00:37:58.913 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:37:59.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:59 smithi038 ceph-mon[29271]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:37:59.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:37:59 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/1426511280' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:00.641 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:00.641 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:01.005 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:01.101 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:00 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:00 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:00 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:00 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:00 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:00 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:00 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:00 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:02.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:01 smithi038 ceph-mon[29271]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:02.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:01 smithi038 ceph-mon[29271]: Deploying daemon prometheus.smithi038 on smithi038 2024-03-24T00:38:02.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:01 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:03.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.110+0000 7f8e15e00700 1 -- 172.21.15.88:0/1503683098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e10101270 msgr2=0x7f8e10103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:03.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.110+0000 7f8e15e00700 1 --2- 172.21.15.88:0/1503683098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e10101270 0x7f8e10103660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009910 tx=0x7f8e00009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:03.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.110+0000 7f8e15e00700 1 -- 172.21.15.88:0/1503683098 shutdown_connections 2024-03-24T00:38:03.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.110+0000 7f8e15e00700 1 --2- 172.21.15.88:0/1503683098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e10101270 0x7f8e10103660 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:03.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.110+0000 7f8e15e00700 1 -- 172.21.15.88:0/1503683098 >> 172.21.15.88:0/1503683098 conn(0x7f8e100fabe0 msgr2=0x7f8e100fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:03.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.110+0000 7f8e15e00700 1 -- 172.21.15.88:0/1503683098 shutdown_connections 2024-03-24T00:38:03.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.110+0000 7f8e15e00700 1 -- 172.21.15.88:0/1503683098 wait complete. 2024-03-24T00:38:03.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.111+0000 7f8e15e00700 1 Processor -- start 2024-03-24T00:38:03.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.111+0000 7f8e15e00700 1 -- start start 2024-03-24T00:38:03.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.112+0000 7f8e15e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e10101270 0x7f8e10195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:03.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.112+0000 7f8e15e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e101960c0 con 0x7f8e10101270 2024-03-24T00:38:03.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.112+0000 7f8e0ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e10101270 0x7f8e10195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:03.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.112+0000 7f8e0ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e10101270 0x7f8e10195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:59476/0 (socket says 172.21.15.88:59476) 2024-03-24T00:38:03.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.112+0000 7f8e0ea00700 1 -- 172.21.15.88:0/2598205669 learned_addr learned my addr 172.21.15.88:0/2598205669 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:03.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.113+0000 7f8e0ea00700 1 -- 172.21.15.88:0/2598205669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e000095f0 con 0x7f8e10101270 2024-03-24T00:38:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.113+0000 7f8e0ea00700 1 --2- 172.21.15.88:0/2598205669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e10101270 0x7f8e10195b80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ed0 tx=0x7f8e00004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.114+0000 7f8e14a00700 1 -- 172.21.15.88:0/2598205669 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e0001d070 con 0x7f8e10101270 2024-03-24T00:38:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.114+0000 7f8e14a00700 1 -- 172.21.15.88:0/2598205669 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e0000be20 con 0x7f8e10101270 2024-03-24T00:38:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.114+0000 7f8e15e00700 1 -- 172.21.15.88:0/2598205669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e101962c0 con 0x7f8e10101270 2024-03-24T00:38:03.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.114+0000 7f8e14a00700 1 -- 172.21.15.88:0/2598205669 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e0000f460 con 0x7f8e10101270 2024-03-24T00:38:03.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.114+0000 7f8e15e00700 1 -- 172.21.15.88:0/2598205669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e10196760 con 0x7f8e10101270 2024-03-24T00:38:03.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.115+0000 7f8e15e00700 1 -- 172.21.15.88:0/2598205669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1018f570 con 0x7f8e10101270 2024-03-24T00:38:03.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.116+0000 7f8e14a00700 1 -- 172.21.15.88:0/2598205669 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f8e00022800 con 0x7f8e10101270 2024-03-24T00:38:03.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.116+0000 7f8e14a00700 1 --2- 172.21.15.88:0/2598205669 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8df4038820 0x7f8df403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:03.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.116+0000 7f8e14a00700 1 -- 172.21.15.88:0/2598205669 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e0004cbb0 con 0x7f8e10101270 2024-03-24T00:38:03.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.116+0000 7f8e06000700 1 --2- 172.21.15.88:0/2598205669 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8df4038820 0x7f8df403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:03.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.117+0000 7f8e06000700 1 --2- 172.21.15.88:0/2598205669 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8df4038820 0x7f8df403ace0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8df8006fd0 tx=0x7f8df8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.133+0000 7f8e14a00700 1 -- 172.21.15.88:0/2598205669 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8e0002ba00 con 0x7f8e10101270 2024-03-24T00:38:03.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:02 smithi038 ceph-mon[29271]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:03.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.440+0000 7f8e15e00700 1 -- 172.21.15.88:0/2598205669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8e100623e0 con 0x7f8e10101270 2024-03-24T00:38:03.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.441+0000 7f8e14a00700 1 -- 172.21.15.88:0/2598205669 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8e0002b3b0 con 0x7f8e10101270 2024-03-24T00:38:03.443 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:03.443 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:03.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.445+0000 7f8e15e00700 1 -- 172.21.15.88:0/2598205669 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8df4038820 msgr2=0x7f8df403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:03.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.445+0000 7f8e15e00700 1 --2- 172.21.15.88:0/2598205669 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8df4038820 0x7f8df403ace0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8df8006fd0 tx=0x7f8df8006e40 comp rx=0 tx=0).stop 2024-03-24T00:38:03.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.445+0000 7f8e15e00700 1 -- 172.21.15.88:0/2598205669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e10101270 msgr2=0x7f8e10195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:03.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.445+0000 7f8e15e00700 1 --2- 172.21.15.88:0/2598205669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e10101270 0x7f8e10195b80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ed0 tx=0x7f8e00004690 comp rx=0 tx=0).stop 2024-03-24T00:38:03.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.445+0000 7f8e15e00700 1 -- 172.21.15.88:0/2598205669 shutdown_connections 2024-03-24T00:38:03.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.445+0000 7f8e15e00700 1 --2- 172.21.15.88:0/2598205669 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f8df4038820 0x7f8df403ace0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:03.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.445+0000 7f8e15e00700 1 --2- 172.21.15.88:0/2598205669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e10101270 0x7f8e10195b80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:03.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.445+0000 7f8e15e00700 1 -- 172.21.15.88:0/2598205669 >> 172.21.15.88:0/2598205669 conn(0x7f8e100fabe0 msgr2=0x7f8e100fb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:03.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.445+0000 7f8e15e00700 1 -- 172.21.15.88:0/2598205669 shutdown_connections 2024-03-24T00:38:03.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:03.445+0000 7f8e15e00700 1 -- 172.21.15.88:0/2598205669 wait complete. 2024-03-24T00:38:03.447 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:04.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:03 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/2598205669' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:05.168 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:05.168 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:05.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:04 smithi038 ceph-mon[29271]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:05.550 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:05.645 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:07.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:07 smithi038 ceph-mon[29271]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:07.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.649+0000 7fb291c00700 1 -- 172.21.15.88:0/1464325909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb28c107070 msgr2=0x7fb28c107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:07.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.649+0000 7fb291c00700 1 --2- 172.21.15.88:0/1464325909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb28c107070 0x7fb28c107490 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009910 tx=0x7fb27c009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:07.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.649+0000 7fb291c00700 1 -- 172.21.15.88:0/1464325909 shutdown_connections 2024-03-24T00:38:07.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.649+0000 7fb291c00700 1 --2- 172.21.15.88:0/1464325909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb28c107070 0x7fb28c107490 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:07.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.649+0000 7fb291c00700 1 -- 172.21.15.88:0/1464325909 >> 172.21.15.88:0/1464325909 conn(0x7fb28c075e30 msgr2=0x7fb28c078290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:07.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.649+0000 7fb291c00700 1 -- 172.21.15.88:0/1464325909 shutdown_connections 2024-03-24T00:38:07.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.649+0000 7fb291c00700 1 -- 172.21.15.88:0/1464325909 wait complete. 2024-03-24T00:38:07.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.650+0000 7fb291c00700 1 Processor -- start 2024-03-24T00:38:07.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.651+0000 7fb291c00700 1 -- start start 2024-03-24T00:38:07.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.651+0000 7fb291c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb28c107070 0x7fb28c19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:07.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.651+0000 7fb291c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c19c700 con 0x7fb28c107070 2024-03-24T00:38:07.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.652+0000 7fb28aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb28c107070 0x7fb28c19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:07.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.652+0000 7fb28aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb28c107070 0x7fb28c19c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:52454/0 (socket says 172.21.15.88:52454) 2024-03-24T00:38:07.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.652+0000 7fb28aa00700 1 -- 172.21.15.88:0/1626223357 learned_addr learned my addr 172.21.15.88:0/1626223357 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:07.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.652+0000 7fb28aa00700 1 -- 172.21.15.88:0/1626223357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c0095f0 con 0x7fb28c107070 2024-03-24T00:38:07.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.653+0000 7fb28aa00700 1 --2- 172.21.15.88:0/1626223357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb28c107070 0x7fb28c19c1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb27c006010 tx=0x7fb27c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:07.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.653+0000 7fb283400700 1 -- 172.21.15.88:0/1626223357 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb27c01d070 con 0x7fb28c107070 2024-03-24T00:38:07.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.653+0000 7fb283400700 1 -- 172.21.15.88:0/1626223357 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb27c00be20 con 0x7fb28c107070 2024-03-24T00:38:07.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.653+0000 7fb283400700 1 -- 172.21.15.88:0/1626223357 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb27c022de0 con 0x7fb28c107070 2024-03-24T00:38:07.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.653+0000 7fb291c00700 1 -- 172.21.15.88:0/1626223357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28c19c900 con 0x7fb28c107070 2024-03-24T00:38:07.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.654+0000 7fb291c00700 1 -- 172.21.15.88:0/1626223357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28c19cda0 con 0x7fb28c107070 2024-03-24T00:38:07.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.655+0000 7fb291c00700 1 -- 172.21.15.88:0/1626223357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28c195b80 con 0x7fb28c107070 2024-03-24T00:38:07.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.655+0000 7fb283400700 1 -- 172.21.15.88:0/1626223357 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7fb27c0226c0 con 0x7fb28c107070 2024-03-24T00:38:07.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.656+0000 7fb283400700 1 --2- 172.21.15.88:0/1626223357 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb270038820 0x7fb27003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:07.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.656+0000 7fb283400700 1 -- 172.21.15.88:0/1626223357 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb27c04c930 con 0x7fb28c107070 2024-03-24T00:38:07.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.656+0000 7fb28a000700 1 --2- 172.21.15.88:0/1626223357 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb270038820 0x7fb27003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:07.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.658+0000 7fb28a000700 1 --2- 172.21.15.88:0/1626223357 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb270038820 0x7fb27003ace0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb278006fd0 tx=0x7fb278006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:07.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.672+0000 7fb283400700 1 -- 172.21.15.88:0/1626223357 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb27c00b960 con 0x7fb28c107070 2024-03-24T00:38:07.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.974+0000 7fb291c00700 1 -- 172.21.15.88:0/1626223357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb28c0623e0 con 0x7fb28c107070 2024-03-24T00:38:07.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.975+0000 7fb283400700 1 -- 172.21.15.88:0/1626223357 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fb27c027030 con 0x7fb28c107070 2024-03-24T00:38:07.976 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:07.976 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.978+0000 7fb291c00700 1 -- 172.21.15.88:0/1626223357 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb270038820 msgr2=0x7fb27003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.978+0000 7fb291c00700 1 --2- 172.21.15.88:0/1626223357 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb270038820 0x7fb27003ace0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb278006fd0 tx=0x7fb278006e40 comp rx=0 tx=0).stop 2024-03-24T00:38:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.978+0000 7fb291c00700 1 -- 172.21.15.88:0/1626223357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb28c107070 msgr2=0x7fb28c19c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.978+0000 7fb291c00700 1 --2- 172.21.15.88:0/1626223357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb28c107070 0x7fb28c19c1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb27c006010 tx=0x7fb27c0050b0 comp rx=0 tx=0).stop 2024-03-24T00:38:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.978+0000 7fb291c00700 1 -- 172.21.15.88:0/1626223357 shutdown_connections 2024-03-24T00:38:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.978+0000 7fb291c00700 1 --2- 172.21.15.88:0/1626223357 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fb270038820 0x7fb27003ace0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.978+0000 7fb291c00700 1 --2- 172.21.15.88:0/1626223357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb28c107070 0x7fb28c19c1c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.978+0000 7fb291c00700 1 -- 172.21.15.88:0/1626223357 >> 172.21.15.88:0/1626223357 conn(0x7fb28c075e30 msgr2=0x7fb28c0768a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.978+0000 7fb291c00700 1 -- 172.21.15.88:0/1626223357 shutdown_connections 2024-03-24T00:38:07.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:07.978+0000 7fb291c00700 1 -- 172.21.15.88:0/1626223357 wait complete. 2024-03-24T00:38:07.980 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:08.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:08 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/1626223357' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:09.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:09 smithi038 ceph-mon[29271]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:09.697 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:09.697 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:10.071 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:10.167 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:11.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:11 smithi038 ceph-mon[29271]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:12.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.304+0000 7f2c6ba00700 1 -- 172.21.15.88:0/3707130141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c64102950 msgr2=0x7f2c64102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:12.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.304+0000 7f2c6ba00700 1 --2- 172.21.15.88:0/3707130141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c64102950 0x7f2c64102d70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009910 tx=0x7f2c58009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:12.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.304+0000 7f2c6ba00700 1 -- 172.21.15.88:0/3707130141 shutdown_connections 2024-03-24T00:38:12.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.304+0000 7f2c6ba00700 1 --2- 172.21.15.88:0/3707130141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c64102950 0x7f2c64102d70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:12.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.304+0000 7f2c6ba00700 1 -- 172.21.15.88:0/3707130141 >> 172.21.15.88:0/3707130141 conn(0x7f2c640fdef0 msgr2=0x7f2c64100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:12.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.305+0000 7f2c6ba00700 1 -- 172.21.15.88:0/3707130141 shutdown_connections 2024-03-24T00:38:12.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.305+0000 7f2c6ba00700 1 -- 172.21.15.88:0/3707130141 wait complete. 2024-03-24T00:38:12.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.306+0000 7f2c6ba00700 1 Processor -- start 2024-03-24T00:38:12.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.306+0000 7f2c6ba00700 1 -- start start 2024-03-24T00:38:12.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.306+0000 7f2c6ba00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c64102950 0x7f2c64197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:12.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.306+0000 7f2c6ba00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64198330 con 0x7f2c64102950 2024-03-24T00:38:12.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.307+0000 7f2c69000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c64102950 0x7f2c64197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:12.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.307+0000 7f2c69000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c64102950 0x7f2c64197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:52468/0 (socket says 172.21.15.88:52468) 2024-03-24T00:38:12.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.307+0000 7f2c69000700 1 -- 172.21.15.88:0/2738351157 learned_addr learned my addr 172.21.15.88:0/2738351157 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:12.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.308+0000 7f2c69000700 1 -- 172.21.15.88:0/2738351157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c580095f0 con 0x7f2c64102950 2024-03-24T00:38:12.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.308+0000 7f2c69000700 1 --2- 172.21.15.88:0/2738351157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c64102950 0x7f2c64197df0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c58006010 tx=0x7f2c580050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:12.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.308+0000 7f2c61600700 1 -- 172.21.15.88:0/2738351157 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c5801d070 con 0x7f2c64102950 2024-03-24T00:38:12.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.309+0000 7f2c61600700 1 -- 172.21.15.88:0/2738351157 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c5800be20 con 0x7f2c64102950 2024-03-24T00:38:12.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.309+0000 7f2c6ba00700 1 -- 172.21.15.88:0/2738351157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c64198530 con 0x7f2c64102950 2024-03-24T00:38:12.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.309+0000 7f2c61600700 1 -- 172.21.15.88:0/2738351157 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c58022de0 con 0x7f2c64102950 2024-03-24T00:38:12.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.309+0000 7f2c6ba00700 1 -- 172.21.15.88:0/2738351157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c641989d0 con 0x7f2c64102950 2024-03-24T00:38:12.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.310+0000 7f2c6ba00700 1 -- 172.21.15.88:0/2738351157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c64191780 con 0x7f2c64102950 2024-03-24T00:38:12.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.310+0000 7f2c61600700 1 -- 172.21.15.88:0/2738351157 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f2c580226c0 con 0x7f2c64102950 2024-03-24T00:38:12.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.311+0000 7f2c61600700 1 --2- 172.21.15.88:0/2738351157 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f2c4c0387b0 0x7f2c4c03ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:12.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.311+0000 7f2c61600700 1 -- 172.21.15.88:0/2738351157 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2c5804ca30 con 0x7f2c64102950 2024-03-24T00:38:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.311+0000 7f2c63e00700 1 --2- 172.21.15.88:0/2738351157 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f2c4c0387b0 0x7f2c4c03ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:12.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.313+0000 7f2c63e00700 1 --2- 172.21.15.88:0/2738351157 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f2c4c0387b0 0x7f2c4c03ac70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2c50006fd0 tx=0x7f2c50006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:12.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.327+0000 7f2c61600700 1 -- 172.21.15.88:0/2738351157 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2c5802ba00 con 0x7f2c64102950 2024-03-24T00:38:12.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.628+0000 7f2c6ba00700 1 -- 172.21.15.88:0/2738351157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2c6402ccb0 con 0x7f2c64102950 2024-03-24T00:38:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.629+0000 7f2c61600700 1 -- 172.21.15.88:0/2738351157 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f2c58027030 con 0x7f2c64102950 2024-03-24T00:38:12.629 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:12.629 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:12.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.632+0000 7f2c6ba00700 1 -- 172.21.15.88:0/2738351157 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f2c4c0387b0 msgr2=0x7f2c4c03ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:12.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.632+0000 7f2c6ba00700 1 --2- 172.21.15.88:0/2738351157 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f2c4c0387b0 0x7f2c4c03ac70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2c50006fd0 tx=0x7f2c50006e40 comp rx=0 tx=0).stop 2024-03-24T00:38:12.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.632+0000 7f2c6ba00700 1 -- 172.21.15.88:0/2738351157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c64102950 msgr2=0x7f2c64197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:12.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.632+0000 7f2c6ba00700 1 --2- 172.21.15.88:0/2738351157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c64102950 0x7f2c64197df0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c58006010 tx=0x7f2c580050b0 comp rx=0 tx=0).stop 2024-03-24T00:38:12.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.633+0000 7f2c6ba00700 1 -- 172.21.15.88:0/2738351157 shutdown_connections 2024-03-24T00:38:12.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.633+0000 7f2c6ba00700 1 --2- 172.21.15.88:0/2738351157 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f2c4c0387b0 0x7f2c4c03ac70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:12.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.633+0000 7f2c6ba00700 1 --2- 172.21.15.88:0/2738351157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c64102950 0x7f2c64197df0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:12.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.633+0000 7f2c6ba00700 1 -- 172.21.15.88:0/2738351157 >> 172.21.15.88:0/2738351157 conn(0x7f2c640fdef0 msgr2=0x7f2c640fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:12.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.633+0000 7f2c6ba00700 1 -- 172.21.15.88:0/2738351157 shutdown_connections 2024-03-24T00:38:12.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:12.633+0000 7f2c6ba00700 1 -- 172.21.15.88:0/2738351157 wait complete. 2024-03-24T00:38:12.633 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:13.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:13 smithi038 ceph-mon[29271]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:13.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:13 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/2738351157' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:14.359 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:14.359 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:14.734 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:14.835 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:15.760 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:15 smithi038 ceph-mon[29271]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.744+0000 7f43be400700 1 -- 172.21.15.88:0/1720673886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102950 msgr2=0x7f43b8102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.744+0000 7f43be400700 1 --2- 172.21.15.88:0/1720673886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102950 0x7f43b8102d70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009910 tx=0x7f43a8009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.744+0000 7f43be400700 1 -- 172.21.15.88:0/1720673886 shutdown_connections 2024-03-24T00:38:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.744+0000 7f43be400700 1 --2- 172.21.15.88:0/1720673886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102950 0x7f43b8102d70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.744+0000 7f43be400700 1 -- 172.21.15.88:0/1720673886 >> 172.21.15.88:0/1720673886 conn(0x7f43b80fdef0 msgr2=0x7f43b8100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.745+0000 7f43be400700 1 -- 172.21.15.88:0/1720673886 shutdown_connections 2024-03-24T00:38:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.745+0000 7f43be400700 1 -- 172.21.15.88:0/1720673886 wait complete. 2024-03-24T00:38:16.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.746+0000 7f43be400700 1 Processor -- start 2024-03-24T00:38:16.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.747+0000 7f43be400700 1 -- start start 2024-03-24T00:38:16.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.747+0000 7f43be400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102950 0x7f43b8197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:16.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.747+0000 7f43be400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8198330 con 0x7f43b8102950 2024-03-24T00:38:16.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.747+0000 7f43b7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102950 0x7f43b8197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:16.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.748+0000 7f43b7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102950 0x7f43b8197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:38968/0 (socket says 172.21.15.88:38968) 2024-03-24T00:38:16.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.748+0000 7f43b7400700 1 -- 172.21.15.88:0/1004070275 learned_addr learned my addr 172.21.15.88:0/1004070275 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:16.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.748+0000 7f43b7400700 1 -- 172.21.15.88:0/1004070275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a80095f0 con 0x7f43b8102950 2024-03-24T00:38:16.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.748+0000 7f43b7400700 1 --2- 172.21.15.88:0/1004070275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102950 0x7f43b8197df0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43a8006010 tx=0x7f43a80050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:16.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.749+0000 7f43afe00700 1 -- 172.21.15.88:0/1004070275 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43a801d070 con 0x7f43b8102950 2024-03-24T00:38:16.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.749+0000 7f43be400700 1 -- 172.21.15.88:0/1004070275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b8198530 con 0x7f43b8102950 2024-03-24T00:38:16.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.749+0000 7f43afe00700 1 -- 172.21.15.88:0/1004070275 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43a800be20 con 0x7f43b8102950 2024-03-24T00:38:16.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.749+0000 7f43be400700 1 -- 172.21.15.88:0/1004070275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b81989d0 con 0x7f43b8102950 2024-03-24T00:38:16.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.749+0000 7f43afe00700 1 -- 172.21.15.88:0/1004070275 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43a8022df0 con 0x7f43b8102950 2024-03-24T00:38:16.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.750+0000 7f43be400700 1 -- 172.21.15.88:0/1004070275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b8191780 con 0x7f43b8102950 2024-03-24T00:38:16.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.751+0000 7f43afe00700 1 -- 172.21.15.88:0/1004070275 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f43a80226c0 con 0x7f43b8102950 2024-03-24T00:38:16.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.751+0000 7f43afe00700 1 --2- 172.21.15.88:0/1004070275 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f439c0387d0 0x7f439c03ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:16.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.751+0000 7f43afe00700 1 -- 172.21.15.88:0/1004070275 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f43a804c960 con 0x7f43b8102950 2024-03-24T00:38:16.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.752+0000 7f43b6a00700 1 --2- 172.21.15.88:0/1004070275 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f439c0387d0 0x7f439c03ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:16.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.753+0000 7f43b6a00700 1 --2- 172.21.15.88:0/1004070275 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f439c0387d0 0x7f439c03ac90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f43a4006fd0 tx=0x7f43a4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:16.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:16.768+0000 7f43afe00700 1 -- 172.21.15.88:0/1004070275 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f43a800b960 con 0x7f43b8102950 2024-03-24T00:38:17.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.067+0000 7f43be400700 1 -- 172.21.15.88:0/1004070275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f43b80623e0 con 0x7f43b8102950 2024-03-24T00:38:17.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.068+0000 7f43afe00700 1 -- 172.21.15.88:0/1004070275 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f43a8027030 con 0x7f43b8102950 2024-03-24T00:38:17.069 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:17.069 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:17.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.071+0000 7f43be400700 1 -- 172.21.15.88:0/1004070275 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f439c0387d0 msgr2=0x7f439c03ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:17.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.071+0000 7f43be400700 1 --2- 172.21.15.88:0/1004070275 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f439c0387d0 0x7f439c03ac90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f43a4006fd0 tx=0x7f43a4006e40 comp rx=0 tx=0).stop 2024-03-24T00:38:17.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.071+0000 7f43be400700 1 -- 172.21.15.88:0/1004070275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102950 msgr2=0x7f43b8197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:17.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.071+0000 7f43be400700 1 --2- 172.21.15.88:0/1004070275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102950 0x7f43b8197df0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43a8006010 tx=0x7f43a80050b0 comp rx=0 tx=0).stop 2024-03-24T00:38:17.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.071+0000 7f43be400700 1 -- 172.21.15.88:0/1004070275 shutdown_connections 2024-03-24T00:38:17.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.071+0000 7f43be400700 1 --2- 172.21.15.88:0/1004070275 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f439c0387d0 0x7f439c03ac90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:17.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.071+0000 7f43be400700 1 --2- 172.21.15.88:0/1004070275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43b8102950 0x7f43b8197df0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:17.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.071+0000 7f43be400700 1 -- 172.21.15.88:0/1004070275 >> 172.21.15.88:0/1004070275 conn(0x7f43b80fdef0 msgr2=0x7f43b80fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:17.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.071+0000 7f43be400700 1 -- 172.21.15.88:0/1004070275 shutdown_connections 2024-03-24T00:38:17.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:17.071+0000 7f43be400700 1 -- 172.21.15.88:0/1004070275 wait complete. 2024-03-24T00:38:17.073 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:17.685 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:17 smithi038 ceph-mon[29271]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:17.686 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:17 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/1004070275' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:18.845 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:18.846 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:19.225 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:19.322 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:19.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:19 smithi038 ceph-mon[29271]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:19.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:19 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:19.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:19 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:19.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:19 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:19.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:19 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T00:38:20.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:20 smithi038 ceph-mon[29271]: from='mgr.14162 172.21.15.38:0/1285695466' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T00:38:20.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:20 smithi038 ceph-mon[29271]: mgrmap e14: smithi038.huhioz(active, since 83s) 2024-03-24T00:38:21.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.298+0000 7f6281600700 1 -- 172.21.15.88:0/1455940720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c100ff0 msgr2=0x7f627c1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:21.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.298+0000 7f6281600700 1 --2- 172.21.15.88:0/1455940720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c100ff0 0x7f627c1033e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f626c009910 tx=0x7f626c009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:21.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.298+0000 7f6281600700 1 -- 172.21.15.88:0/1455940720 shutdown_connections 2024-03-24T00:38:21.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.298+0000 7f6281600700 1 --2- 172.21.15.88:0/1455940720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c100ff0 0x7f627c1033e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:21.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.298+0000 7f6281600700 1 -- 172.21.15.88:0/1455940720 >> 172.21.15.88:0/1455940720 conn(0x7f627c0faa00 msgr2=0x7f627c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:21.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.299+0000 7f6281600700 1 -- 172.21.15.88:0/1455940720 shutdown_connections 2024-03-24T00:38:21.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.299+0000 7f6281600700 1 -- 172.21.15.88:0/1455940720 wait complete. 2024-03-24T00:38:21.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.300+0000 7f6281600700 1 Processor -- start 2024-03-24T00:38:21.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.300+0000 7f6281600700 1 -- start start 2024-03-24T00:38:21.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.300+0000 7f6281600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c100ff0 0x7f627c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:21.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.300+0000 7f6281600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c1980e0 con 0x7f627c100ff0 2024-03-24T00:38:21.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.301+0000 7f627be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c100ff0 0x7f627c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:21.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.301+0000 7f627be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c100ff0 0x7f627c197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:38984/0 (socket says 172.21.15.88:38984) 2024-03-24T00:38:21.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.301+0000 7f627be00700 1 -- 172.21.15.88:0/3804360205 learned_addr learned my addr 172.21.15.88:0/3804360205 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:21.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.301+0000 7f627be00700 1 -- 172.21.15.88:0/3804360205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c0095f0 con 0x7f627c100ff0 2024-03-24T00:38:21.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.302+0000 7f627be00700 1 --2- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c100ff0 0x7f627c197ba0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f626c006010 tx=0x7f626c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:21.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.302+0000 7f6278c00700 1 -- 172.21.15.88:0/3804360205 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f626c01d070 con 0x7f627c100ff0 2024-03-24T00:38:21.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.302+0000 7f6278c00700 1 -- 172.21.15.88:0/3804360205 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f626c00be20 con 0x7f627c100ff0 2024-03-24T00:38:21.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.302+0000 7f6281600700 1 -- 172.21.15.88:0/3804360205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c1982e0 con 0x7f627c100ff0 2024-03-24T00:38:21.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.302+0000 7f6278c00700 1 -- 172.21.15.88:0/3804360205 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f626c00f460 con 0x7f627c100ff0 2024-03-24T00:38:21.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.302+0000 7f6281600700 1 -- 172.21.15.88:0/3804360205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c198780 con 0x7f627c100ff0 2024-03-24T00:38:21.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.303+0000 7f6281600700 1 -- 172.21.15.88:0/3804360205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c191790 con 0x7f627c100ff0 2024-03-24T00:38:21.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.304+0000 7f6278c00700 1 -- 172.21.15.88:0/3804360205 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 14) v1 ==== 45618+0+0 (secure 0 0 0) 0x7f626c022470 con 0x7f627c100ff0 2024-03-24T00:38:21.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.304+0000 7f6278c00700 1 --2- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f6260038820 0x7f626003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:21.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.304+0000 7f6278c00700 1 -- 172.21.15.88:0/3804360205 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f626c04cd50 con 0x7f627c100ff0 2024-03-24T00:38:21.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.304+0000 7f627b400700 1 -- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f6260038820 msgr2=0x7f626003ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1124244493 2024-03-24T00:38:21.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.304+0000 7f627b400700 1 --2- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f6260038820 0x7f626003ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T00:38:21.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.320+0000 7f6278c00700 1 -- 172.21.15.88:0/3804360205 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f626c005c00 con 0x7f627c100ff0 2024-03-24T00:38:21.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.505+0000 7f627b400700 1 -- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f6260038820 msgr2=0x7f626003ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1124244493 2024-03-24T00:38:21.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.505+0000 7f627b400700 1 --2- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f6260038820 0x7f626003ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T00:38:21.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.614+0000 7f6281600700 1 -- 172.21.15.88:0/3804360205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f627c0623e0 con 0x7f627c100ff0 2024-03-24T00:38:21.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.615+0000 7f6278c00700 1 -- 172.21.15.88:0/3804360205 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f626c02bba0 con 0x7f627c100ff0 2024-03-24T00:38:21.616 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:21.616 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:21.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.618+0000 7f6281600700 1 -- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f6260038820 msgr2=0x7f626003ace0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T00:38:21.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.618+0000 7f6281600700 1 --2- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f6260038820 0x7f626003ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:21.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.618+0000 7f6281600700 1 -- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c100ff0 msgr2=0x7f627c197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:21.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.618+0000 7f6281600700 1 --2- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c100ff0 0x7f627c197ba0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f626c006010 tx=0x7f626c0050b0 comp rx=0 tx=0).stop 2024-03-24T00:38:21.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.618+0000 7f6281600700 1 -- 172.21.15.88:0/3804360205 shutdown_connections 2024-03-24T00:38:21.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.618+0000 7f6281600700 1 --2- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7f6260038820 0x7f626003ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:21.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.618+0000 7f6281600700 1 --2- 172.21.15.88:0/3804360205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f627c100ff0 0x7f627c197ba0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:21.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.618+0000 7f6281600700 1 -- 172.21.15.88:0/3804360205 >> 172.21.15.88:0/3804360205 conn(0x7f627c0faa00 msgr2=0x7f627c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:21.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.618+0000 7f6281600700 1 -- 172.21.15.88:0/3804360205 shutdown_connections 2024-03-24T00:38:21.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:21.618+0000 7f6281600700 1 -- 172.21.15.88:0/3804360205 wait complete. 2024-03-24T00:38:21.620 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:21.807 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:21 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/3804360205' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:23.275 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:23.276 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:23.656 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:23.752 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.653+0000 7fce93800700 1 -- 172.21.15.88:0/2837739707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8c100730 msgr2=0x7fce8c100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.653+0000 7fce93800700 1 --2- 172.21.15.88:0/2837739707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8c100730 0x7fce8c100b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fce80009910 tx=0x7fce80009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.654+0000 7fce93800700 1 -- 172.21.15.88:0/2837739707 shutdown_connections 2024-03-24T00:38:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.654+0000 7fce93800700 1 --2- 172.21.15.88:0/2837739707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8c100730 0x7fce8c100b50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.654+0000 7fce93800700 1 -- 172.21.15.88:0/2837739707 >> 172.21.15.88:0/2837739707 conn(0x7fce8c0fbcd0 msgr2=0x7fce8c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.654+0000 7fce93800700 1 -- 172.21.15.88:0/2837739707 shutdown_connections 2024-03-24T00:38:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.654+0000 7fce93800700 1 -- 172.21.15.88:0/2837739707 wait complete. 2024-03-24T00:38:25.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.655+0000 7fce93800700 1 Processor -- start 2024-03-24T00:38:25.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.655+0000 7fce93800700 1 -- start start 2024-03-24T00:38:25.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.656+0000 7fce93800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8c100730 0x7fce8c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:25.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.656+0000 7fce93800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c198330 con 0x7fce8c100730 2024-03-24T00:38:25.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.656+0000 7fce90e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8c100730 0x7fce8c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:25.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.656+0000 7fce90e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8c100730 0x7fce8c197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:43138/0 (socket says 172.21.15.88:43138) 2024-03-24T00:38:25.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.656+0000 7fce90e00700 1 -- 172.21.15.88:0/1172762989 learned_addr learned my addr 172.21.15.88:0/1172762989 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:25.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.657+0000 7fce90e00700 1 -- 172.21.15.88:0/1172762989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce800095f0 con 0x7fce8c100730 2024-03-24T00:38:25.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.657+0000 7fce90e00700 1 --2- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8c100730 0x7fce8c197df0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce80006010 tx=0x7fce800050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:25.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.657+0000 7fce89600700 1 -- 172.21.15.88:0/1172762989 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce8001d070 con 0x7fce8c100730 2024-03-24T00:38:25.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.657+0000 7fce89600700 1 -- 172.21.15.88:0/1172762989 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce8000be20 con 0x7fce8c100730 2024-03-24T00:38:25.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.657+0000 7fce89600700 1 -- 172.21.15.88:0/1172762989 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce80022de0 con 0x7fce8c100730 2024-03-24T00:38:25.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.657+0000 7fce93800700 1 -- 172.21.15.88:0/1172762989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c198530 con 0x7fce8c100730 2024-03-24T00:38:25.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.658+0000 7fce93800700 1 -- 172.21.15.88:0/1172762989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c1989d0 con 0x7fce8c100730 2024-03-24T00:38:25.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.659+0000 7fce89600700 1 -- 172.21.15.88:0/1172762989 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 14) v1 ==== 45618+0+0 (secure 0 0 0) 0x7fce800226c0 con 0x7fce8c100730 2024-03-24T00:38:25.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.659+0000 7fce93800700 1 -- 172.21.15.88:0/1172762989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8c191960 con 0x7fce8c100730 2024-03-24T00:38:25.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.659+0000 7fce89600700 1 --2- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fce74038870 0x7fce7403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:25.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.659+0000 7fce89600700 1 -- 172.21.15.88:0/1172762989 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fce8004c970 con 0x7fce8c100730 2024-03-24T00:38:25.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.659+0000 7fce8be00700 1 -- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fce74038870 msgr2=0x7fce7403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1124244493 2024-03-24T00:38:25.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.659+0000 7fce8be00700 1 --2- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fce74038870 0x7fce7403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T00:38:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.666+0000 7fce89600700 1 -- 172.21.15.88:0/1172762989 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fce8000b960 con 0x7fce8c100730 2024-03-24T00:38:25.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.860+0000 7fce8be00700 1 -- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fce74038870 msgr2=0x7fce7403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1124244493 2024-03-24T00:38:25.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.860+0000 7fce8be00700 1 --2- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fce74038870 0x7fce7403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T00:38:25.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.973+0000 7fce93800700 1 -- 172.21.15.88:0/1172762989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fce8c0623e0 con 0x7fce8c100730 2024-03-24T00:38:25.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.975+0000 7fce89600700 1 -- 172.21.15.88:0/1172762989 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fce80027030 con 0x7fce8c100730 2024-03-24T00:38:25.975 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:25.976 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:25.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.977+0000 7fce93800700 1 -- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fce74038870 msgr2=0x7fce7403ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T00:38:25.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.977+0000 7fce93800700 1 --2- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fce74038870 0x7fce7403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:25.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.977+0000 7fce93800700 1 -- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8c100730 msgr2=0x7fce8c197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:25.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.977+0000 7fce93800700 1 --2- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8c100730 0x7fce8c197df0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce80006010 tx=0x7fce800050b0 comp rx=0 tx=0).stop 2024-03-24T00:38:25.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.978+0000 7fce93800700 1 -- 172.21.15.88:0/1172762989 shutdown_connections 2024-03-24T00:38:25.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.978+0000 7fce93800700 1 --2- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:6800/1124244493,v1:172.21.15.38:6801/1124244493] conn(0x7fce74038870 0x7fce7403ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:25.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.978+0000 7fce93800700 1 --2- 172.21.15.88:0/1172762989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8c100730 0x7fce8c197df0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:25.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.978+0000 7fce93800700 1 -- 172.21.15.88:0/1172762989 >> 172.21.15.88:0/1172762989 conn(0x7fce8c0fbcd0 msgr2=0x7fce8c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:25.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.978+0000 7fce93800700 1 -- 172.21.15.88:0/1172762989 shutdown_connections 2024-03-24T00:38:25.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:25.978+0000 7fce93800700 1 -- 172.21.15.88:0/1172762989 wait complete. 2024-03-24T00:38:25.980 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:26.346 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:26 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/1172762989' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:27.659 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:27.659 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:28.030 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:28.136 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:29.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: Active manager daemon smithi038.huhioz restarted 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: Activating manager daemon smithi038.huhioz 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: osdmap e5: 0 total, 0 up, 0 in 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: mgrmap e15: smithi038.huhioz(active, starting, since 0.00954321s) 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi038.huhioz", "id": "smithi038.huhioz"}]: dispatch 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: Manager daemon smithi038.huhioz is now available 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:38:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:38:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:38:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/trash_purge_schedule"}]: dispatch 2024-03-24T00:38:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.032+0000 7f6e2c000700 1 -- 172.21.15.88:0/325421753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e24100720 msgr2=0x7f6e24100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.032+0000 7f6e2c000700 1 --2- 172.21.15.88:0/325421753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e24100720 0x7f6e24100b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009910 tx=0x7f6e18009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.032+0000 7f6e2c000700 1 -- 172.21.15.88:0/325421753 shutdown_connections 2024-03-24T00:38:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.032+0000 7f6e2c000700 1 --2- 172.21.15.88:0/325421753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e24100720 0x7f6e24100b40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.032+0000 7f6e2c000700 1 -- 172.21.15.88:0/325421753 >> 172.21.15.88:0/325421753 conn(0x7f6e240fbcc0 msgr2=0x7f6e240fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.032+0000 7f6e2c000700 1 -- 172.21.15.88:0/325421753 shutdown_connections 2024-03-24T00:38:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.032+0000 7f6e2c000700 1 -- 172.21.15.88:0/325421753 wait complete. 2024-03-24T00:38:30.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.033+0000 7f6e2c000700 1 Processor -- start 2024-03-24T00:38:30.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.033+0000 7f6e2c000700 1 -- start start 2024-03-24T00:38:30.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.034+0000 7f6e2c000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e24100720 0x7f6e24197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:30.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.034+0000 7f6e2c000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e24198320 con 0x7f6e24100720 2024-03-24T00:38:30.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.034+0000 7f6e29600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e24100720 0x7f6e24197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:30.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.034+0000 7f6e29600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e24100720 0x7f6e24197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:43158/0 (socket says 172.21.15.88:43158) 2024-03-24T00:38:30.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.034+0000 7f6e29600700 1 -- 172.21.15.88:0/1651913573 learned_addr learned my addr 172.21.15.88:0/1651913573 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.035+0000 7f6e29600700 1 -- 172.21.15.88:0/1651913573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e180095f0 con 0x7f6e24100720 2024-03-24T00:38:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.035+0000 7f6e29600700 1 --2- 172.21.15.88:0/1651913573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e24100720 0x7f6e24197de0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6e18006010 tx=0x7f6e180050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:30.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.036+0000 7f6e22000700 1 -- 172.21.15.88:0/1651913573 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e1801d070 con 0x7f6e24100720 2024-03-24T00:38:30.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.036+0000 7f6e2c000700 1 -- 172.21.15.88:0/1651913573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e24198520 con 0x7f6e24100720 2024-03-24T00:38:30.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.036+0000 7f6e2c000700 1 -- 172.21.15.88:0/1651913573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e241989c0 con 0x7f6e24100720 2024-03-24T00:38:30.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.036+0000 7f6e22000700 1 -- 172.21.15.88:0/1651913573 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e1800be20 con 0x7f6e24100720 2024-03-24T00:38:30.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.036+0000 7f6e22000700 1 -- 172.21.15.88:0/1651913573 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e18022dc0 con 0x7f6e24100720 2024-03-24T00:38:30.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.037+0000 7f6e2c000700 1 -- 172.21.15.88:0/1651913573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e04005320 con 0x7f6e24100720 2024-03-24T00:38:30.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.038+0000 7f6e22000700 1 -- 172.21.15.88:0/1651913573 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 45469+0+0 (secure 0 0 0) 0x7f6e1800b960 con 0x7f6e24100720 2024-03-24T00:38:30.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.038+0000 7f6e22000700 1 --2- 172.21.15.88:0/1651913573 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6e0c0386a0 0x7f6e0c03ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:30.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.038+0000 7f6e22000700 1 -- 172.21.15.88:0/1651913573 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f6e1804ca60 con 0x7f6e24100720 2024-03-24T00:38:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.039+0000 7f6e28c00700 1 --2- 172.21.15.88:0/1651913573 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6e0c0386a0 0x7f6e0c03ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.040+0000 7f6e28c00700 1 --2- 172.21.15.88:0/1651913573 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6e0c0386a0 0x7f6e0c03ab60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e10006fd0 tx=0x7f6e10006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:30.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.048+0000 7f6e22000700 1 -- 172.21.15.88:0/1651913573 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6e1802b590 con 0x7f6e24100720 2024-03-24T00:38:30.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.352+0000 7f6e2c000700 1 -- 172.21.15.88:0/1651913573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6e04005190 con 0x7f6e24100720 2024-03-24T00:38:30.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.353+0000 7f6e22000700 1 -- 172.21.15.88:0/1651913573 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6e18027020 con 0x7f6e24100720 2024-03-24T00:38:30.354 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:30.354 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:30.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.356+0000 7f6e2c000700 1 -- 172.21.15.88:0/1651913573 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6e0c0386a0 msgr2=0x7f6e0c03ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:30.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.356+0000 7f6e2c000700 1 --2- 172.21.15.88:0/1651913573 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6e0c0386a0 0x7f6e0c03ab60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e10006fd0 tx=0x7f6e10006e40 comp rx=0 tx=0).stop 2024-03-24T00:38:30.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.356+0000 7f6e2c000700 1 -- 172.21.15.88:0/1651913573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e24100720 msgr2=0x7f6e24197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:30.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.356+0000 7f6e2c000700 1 --2- 172.21.15.88:0/1651913573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e24100720 0x7f6e24197de0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6e18006010 tx=0x7f6e180050b0 comp rx=0 tx=0).stop 2024-03-24T00:38:30.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.356+0000 7f6e2c000700 1 -- 172.21.15.88:0/1651913573 shutdown_connections 2024-03-24T00:38:30.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.356+0000 7f6e2c000700 1 --2- 172.21.15.88:0/1651913573 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6e0c0386a0 0x7f6e0c03ab60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:30.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.356+0000 7f6e2c000700 1 --2- 172.21.15.88:0/1651913573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e24100720 0x7f6e24197de0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:30.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.356+0000 7f6e2c000700 1 -- 172.21.15.88:0/1651913573 >> 172.21.15.88:0/1651913573 conn(0x7f6e240fbcc0 msgr2=0x7f6e240fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:30.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.356+0000 7f6e2c000700 1 -- 172.21.15.88:0/1651913573 shutdown_connections 2024-03-24T00:38:30.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:30.356+0000 7f6e2c000700 1 -- 172.21.15.88:0/1651913573 wait complete. 2024-03-24T00:38:30.358 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:30.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:30 smithi038 ceph-mon[29271]: [24/Mar/2024:00:38:29] ENGINE Bus STARTING 2024-03-24T00:38:30.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:30 smithi038 ceph-mon[29271]: [24/Mar/2024:00:38:29] ENGINE Serving on http://172.21.15.38:8765 2024-03-24T00:38:30.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:30 smithi038 ceph-mon[29271]: [24/Mar/2024:00:38:29] ENGINE Serving on https://172.21.15.38:7150 2024-03-24T00:38:30.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:30 smithi038 ceph-mon[29271]: [24/Mar/2024:00:38:29] ENGINE Bus STARTED 2024-03-24T00:38:30.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:30 smithi038 ceph-mon[29271]: mgrmap e16: smithi038.huhioz(active, since 1.01412s) 2024-03-24T00:38:30.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:30 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:30.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:30 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/1651913573' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:32.074 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:32.074 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:32.191 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:31 smithi038 ceph-mon[29271]: mgrmap e17: smithi038.huhioz(active, since 2s) 2024-03-24T00:38:32.192 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:31 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:32.435 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:32.548 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:34.098 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:33 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:34.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.759+0000 7f7c7e200700 1 -- 172.21.15.88:0/3649511812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c78074de0 msgr2=0x7f7c78073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:34.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.759+0000 7f7c7e200700 1 --2- 172.21.15.88:0/3649511812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c78074de0 0x7f7c78073240 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009910 tx=0x7f7c6c009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:34.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.759+0000 7f7c7e200700 1 -- 172.21.15.88:0/3649511812 shutdown_connections 2024-03-24T00:38:34.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.759+0000 7f7c7e200700 1 --2- 172.21.15.88:0/3649511812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c78074de0 0x7f7c78073240 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:34.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.759+0000 7f7c7e200700 1 -- 172.21.15.88:0/3649511812 >> 172.21.15.88:0/3649511812 conn(0x7f7c780fbb20 msgr2=0x7f7c780fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:34.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.759+0000 7f7c7e200700 1 -- 172.21.15.88:0/3649511812 shutdown_connections 2024-03-24T00:38:34.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.759+0000 7f7c7e200700 1 -- 172.21.15.88:0/3649511812 wait complete. 2024-03-24T00:38:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.760+0000 7f7c7e200700 1 Processor -- start 2024-03-24T00:38:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.760+0000 7f7c7e200700 1 -- start start 2024-03-24T00:38:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.761+0000 7f7c7e200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c78074de0 0x7f7c78197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.761+0000 7f7c7e200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c781981d0 con 0x7f7c78074de0 2024-03-24T00:38:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.761+0000 7f7c7ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c78074de0 0x7f7c78197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.761+0000 7f7c7ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c78074de0 0x7f7c78197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:44604/0 (socket says 172.21.15.88:44604) 2024-03-24T00:38:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.761+0000 7f7c7ce00700 1 -- 172.21.15.88:0/486962747 learned_addr learned my addr 172.21.15.88:0/486962747 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:34.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.762+0000 7f7c7ce00700 1 -- 172.21.15.88:0/486962747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c0095f0 con 0x7f7c78074de0 2024-03-24T00:38:34.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.762+0000 7f7c7ce00700 1 --2- 172.21.15.88:0/486962747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c78074de0 0x7f7c78197c90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c006010 tx=0x7f7c6c005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:34.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.763+0000 7f7c75600700 1 -- 172.21.15.88:0/486962747 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c6c01d070 con 0x7f7c78074de0 2024-03-24T00:38:34.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.763+0000 7f7c7e200700 1 -- 172.21.15.88:0/486962747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c781983d0 con 0x7f7c78074de0 2024-03-24T00:38:34.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.763+0000 7f7c75600700 1 -- 172.21.15.88:0/486962747 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c6c00be20 con 0x7f7c78074de0 2024-03-24T00:38:34.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.763+0000 7f7c75600700 1 -- 172.21.15.88:0/486962747 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c6c022e40 con 0x7f7c78074de0 2024-03-24T00:38:34.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.763+0000 7f7c7e200700 1 -- 172.21.15.88:0/486962747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c78198870 con 0x7f7c78074de0 2024-03-24T00:38:34.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.764+0000 7f7c7e200700 1 -- 172.21.15.88:0/486962747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c781915c0 con 0x7f7c78074de0 2024-03-24T00:38:34.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.764+0000 7f7c75600700 1 -- 172.21.15.88:0/486962747 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7f7c6c022700 con 0x7f7c78074de0 2024-03-24T00:38:34.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.765+0000 7f7c75600700 1 --2- 172.21.15.88:0/486962747 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7c600388a0 0x7f7c6003ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:34.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.765+0000 7f7c75600700 1 -- 172.21.15.88:0/486962747 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7c6c04cbf0 con 0x7f7c78074de0 2024-03-24T00:38:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.765+0000 7f7c77e00700 1 --2- 172.21.15.88:0/486962747 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7c600388a0 0x7f7c6003ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:34.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.767+0000 7f7c77e00700 1 --2- 172.21.15.88:0/486962747 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7c600388a0 0x7f7c6003ad60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7c64006fd0 tx=0x7f7c64006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:34.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:34.783+0000 7f7c75600700 1 -- 172.21.15.88:0/486962747 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7c6c02ba00 con 0x7f7c78074de0 2024-03-24T00:38:35.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.089+0000 7f7c7e200700 1 -- 172.21.15.88:0/486962747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7c7802ccb0 con 0x7f7c78074de0 2024-03-24T00:38:35.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.090+0000 7f7c75600700 1 -- 172.21.15.88:0/486962747 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f7c6c027030 con 0x7f7c78074de0 2024-03-24T00:38:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:35.091 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:35.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.093+0000 7f7c7e200700 1 -- 172.21.15.88:0/486962747 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7c600388a0 msgr2=0x7f7c6003ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:35.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.093+0000 7f7c7e200700 1 --2- 172.21.15.88:0/486962747 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7c600388a0 0x7f7c6003ad60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7c64006fd0 tx=0x7f7c64006e40 comp rx=0 tx=0).stop 2024-03-24T00:38:35.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.093+0000 7f7c7e200700 1 -- 172.21.15.88:0/486962747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c78074de0 msgr2=0x7f7c78197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:35.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.093+0000 7f7c7e200700 1 --2- 172.21.15.88:0/486962747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c78074de0 0x7f7c78197c90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c006010 tx=0x7f7c6c005ee0 comp rx=0 tx=0).stop 2024-03-24T00:38:35.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.093+0000 7f7c7e200700 1 -- 172.21.15.88:0/486962747 shutdown_connections 2024-03-24T00:38:35.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.093+0000 7f7c7e200700 1 --2- 172.21.15.88:0/486962747 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7c600388a0 0x7f7c6003ad60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:35.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.093+0000 7f7c7e200700 1 --2- 172.21.15.88:0/486962747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c78074de0 0x7f7c78197c90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:35.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.093+0000 7f7c7e200700 1 -- 172.21.15.88:0/486962747 >> 172.21.15.88:0/486962747 conn(0x7f7c780fbb20 msgr2=0x7f7c780fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:35.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.093+0000 7f7c7e200700 1 -- 172.21.15.88:0/486962747 shutdown_connections 2024-03-24T00:38:35.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:35.093+0000 7f7c7e200700 1 -- 172.21.15.88:0/486962747 wait complete. 2024-03-24T00:38:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:35.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:35 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:35.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:35 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:35.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:35 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/486962747' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:37.120 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:37.120 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:37.477 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:37.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:37.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:37.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:38:37.610 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T00:38:39.856 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:39.856 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:39.856 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:39.857 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:39.857 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi088", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:38:39.857 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:38:39.857 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:38:39.857 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:39 smithi038 ceph-mon[29271]: Updating smithi038:/etc/ceph/ceph.conf 2024-03-24T00:38:39.857 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:39 smithi038 ceph-mon[29271]: Updating smithi088:/etc/ceph/ceph.conf 2024-03-24T00:38:40.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.095+0000 7f3640e00700 1 -- 172.21.15.88:0/1946775512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c074da0 msgr2=0x7f363c073200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:40.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.095+0000 7f3640e00700 1 --2- 172.21.15.88:0/1946775512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c074da0 0x7f363c073200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f362c009910 tx=0x7f362c009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:40.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.095+0000 7f3640e00700 1 -- 172.21.15.88:0/1946775512 shutdown_connections 2024-03-24T00:38:40.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.095+0000 7f3640e00700 1 --2- 172.21.15.88:0/1946775512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c074da0 0x7f363c073200 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:40.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.095+0000 7f3640e00700 1 -- 172.21.15.88:0/1946775512 >> 172.21.15.88:0/1946775512 conn(0x7f363c0fbcd0 msgr2=0x7f363c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:40.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.095+0000 7f3640e00700 1 -- 172.21.15.88:0/1946775512 shutdown_connections 2024-03-24T00:38:40.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.095+0000 7f3640e00700 1 -- 172.21.15.88:0/1946775512 wait complete. 2024-03-24T00:38:40.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.096+0000 7f3640e00700 1 Processor -- start 2024-03-24T00:38:40.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.096+0000 7f3640e00700 1 -- start start 2024-03-24T00:38:40.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.097+0000 7f3640e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c074da0 0x7f363c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:40.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.097+0000 7f3640e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c198350 con 0x7f363c074da0 2024-03-24T00:38:40.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.097+0000 7f3639e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c074da0 0x7f363c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:40.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.097+0000 7f3639e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c074da0 0x7f363c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:44610/0 (socket says 172.21.15.88:44610) 2024-03-24T00:38:40.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.097+0000 7f3639e00700 1 -- 172.21.15.88:0/1448482515 learned_addr learned my addr 172.21.15.88:0/1448482515 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:40.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.098+0000 7f3639e00700 1 -- 172.21.15.88:0/1448482515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0095f0 con 0x7f363c074da0 2024-03-24T00:38:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.098+0000 7f3639e00700 1 --2- 172.21.15.88:0/1448482515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c074da0 0x7f363c197e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f362c006010 tx=0x7f362c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:40.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.098+0000 7f3632a00700 1 -- 172.21.15.88:0/1448482515 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f362c01d070 con 0x7f363c074da0 2024-03-24T00:38:40.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.098+0000 7f3632a00700 1 -- 172.21.15.88:0/1448482515 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f362c00be20 con 0x7f363c074da0 2024-03-24T00:38:40.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.098+0000 7f3640e00700 1 -- 172.21.15.88:0/1448482515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c198550 con 0x7f363c074da0 2024-03-24T00:38:40.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.099+0000 7f3632a00700 1 -- 172.21.15.88:0/1448482515 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f362c022de0 con 0x7f363c074da0 2024-03-24T00:38:40.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.099+0000 7f3640e00700 1 -- 172.21.15.88:0/1448482515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c1989f0 con 0x7f363c074da0 2024-03-24T00:38:40.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.100+0000 7f3640e00700 1 -- 172.21.15.88:0/1448482515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c191770 con 0x7f363c074da0 2024-03-24T00:38:40.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.100+0000 7f3632a00700 1 -- 172.21.15.88:0/1448482515 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7f362c00b960 con 0x7f363c074da0 2024-03-24T00:38:40.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.100+0000 7f3632a00700 1 --2- 172.21.15.88:0/1448482515 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3620038960 0x7f362003ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:40.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.101+0000 7f3632a00700 1 -- 172.21.15.88:0/1448482515 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f362c04ccd0 con 0x7f363c074da0 2024-03-24T00:38:40.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.101+0000 7f3639400700 1 --2- 172.21.15.88:0/1448482515 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3620038960 0x7f362003ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:40.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.102+0000 7f3639400700 1 --2- 172.21.15.88:0/1448482515 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3620038960 0x7f362003ae20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3628006fd0 tx=0x7f3628006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:40.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.110+0000 7f3632a00700 1 -- 172.21.15.88:0/1448482515 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f362c02bd40 con 0x7f363c074da0 2024-03-24T00:38:40.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.434+0000 7f3640e00700 1 -- 172.21.15.88:0/1448482515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f363c0623e0 con 0x7f363c074da0 2024-03-24T00:38:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.435+0000 7f3632a00700 1 -- 172.21.15.88:0/1448482515 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f362c027030 con 0x7f363c074da0 2024-03-24T00:38:40.436 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:40.436 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:40.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.438+0000 7f3640e00700 1 -- 172.21.15.88:0/1448482515 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3620038960 msgr2=0x7f362003ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:40.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.438+0000 7f3640e00700 1 --2- 172.21.15.88:0/1448482515 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3620038960 0x7f362003ae20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3628006fd0 tx=0x7f3628006e40 comp rx=0 tx=0).stop 2024-03-24T00:38:40.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.438+0000 7f3640e00700 1 -- 172.21.15.88:0/1448482515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c074da0 msgr2=0x7f363c197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:40.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.438+0000 7f3640e00700 1 --2- 172.21.15.88:0/1448482515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c074da0 0x7f363c197e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f362c006010 tx=0x7f362c0050b0 comp rx=0 tx=0).stop 2024-03-24T00:38:40.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.438+0000 7f3640e00700 1 -- 172.21.15.88:0/1448482515 shutdown_connections 2024-03-24T00:38:40.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.438+0000 7f3640e00700 1 --2- 172.21.15.88:0/1448482515 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3620038960 0x7f362003ae20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:40.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.438+0000 7f3640e00700 1 --2- 172.21.15.88:0/1448482515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c074da0 0x7f363c197e10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:40.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.438+0000 7f3640e00700 1 -- 172.21.15.88:0/1448482515 >> 172.21.15.88:0/1448482515 conn(0x7f363c0fbcd0 msgr2=0x7f363c0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:40.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.438+0000 7f3640e00700 1 -- 172.21.15.88:0/1448482515 shutdown_connections 2024-03-24T00:38:40.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:40.438+0000 7f3640e00700 1 -- 172.21.15.88:0/1448482515 wait complete. 2024-03-24T00:38:40.440 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:40 smithi038 ceph-mon[29271]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:38:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:40 smithi038 ceph-mon[29271]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:38:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:40 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/1448482515' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:41 smithi038 ceph-mon[29271]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:38:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:41 smithi038 ceph-mon[29271]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:38:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:41 smithi038 ceph-mon[29271]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:38:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:41 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:41 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:42.216 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:42.217 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:42.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:42 smithi038 ceph-mon[29271]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:38:42.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:42 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:42.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:42 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:42.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:42 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:42.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:42 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:38:42.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:42 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-24T00:38:42.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:42 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:38:43.527 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:38:43.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:43 smithi038 ceph-mon[29271]: Deploying daemon ceph-exporter.smithi088 on smithi088 2024-03-24T00:38:44.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:38:46.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.255+0000 7f95e0c00700 1 -- 172.21.15.88:0/3590701500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95dc1088d0 msgr2=0x7f95dc108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.255+0000 7f95e0c00700 1 --2- 172.21.15.88:0/3590701500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95dc1088d0 0x7f95dc108cb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009910 tx=0x7f95cc009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.255+0000 7f95e0c00700 1 -- 172.21.15.88:0/3590701500 shutdown_connections 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.255+0000 7f95e0c00700 1 --2- 172.21.15.88:0/3590701500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95dc1088d0 0x7f95dc108cb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.255+0000 7f95e0c00700 1 -- 172.21.15.88:0/3590701500 >> 172.21.15.88:0/3590701500 conn(0x7f95dc0fe400 msgr2=0x7f95dc100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.255+0000 7f95e0c00700 1 -- 172.21.15.88:0/3590701500 shutdown_connections 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.255+0000 7f95e0c00700 1 -- 172.21.15.88:0/3590701500 wait complete. 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.256+0000 7f95e0c00700 1 Processor -- start 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.256+0000 7f95e0c00700 1 -- start start 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.256+0000 7f95e0c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95dc1088d0 0x7f95dc1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.256+0000 7f95e0c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc198710 con 0x7f95dc1088d0 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.257+0000 7f95d9e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95dc1088d0 0x7f95dc1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:46.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.257+0000 7f95d9e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95dc1088d0 0x7f95dc1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:40780/0 (socket says 172.21.15.88:40780) 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.257+0000 7f95d9e00700 1 -- 172.21.15.88:0/3177436723 learned_addr learned my addr 172.21.15.88:0/3177436723 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.258+0000 7f95d9e00700 1 -- 172.21.15.88:0/3177436723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0095f0 con 0x7f95dc1088d0 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.258+0000 7f95d9e00700 1 --2- 172.21.15.88:0/3177436723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95dc1088d0 0x7f95dc1981d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0098e0 tx=0x7f95cc005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.259+0000 7f95d2a00700 1 -- 172.21.15.88:0/3177436723 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95cc01d070 con 0x7f95dc1088d0 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.259+0000 7f95e0c00700 1 -- 172.21.15.88:0/3177436723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc198970 con 0x7f95dc1088d0 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.259+0000 7f95e0c00700 1 -- 172.21.15.88:0/3177436723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc198df0 con 0x7f95dc1088d0 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.259+0000 7f95d2a00700 1 -- 172.21.15.88:0/3177436723 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95cc0056b0 con 0x7f95dc1088d0 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.259+0000 7f95d2a00700 1 -- 172.21.15.88:0/3177436723 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95cc00fd90 con 0x7f95dc1088d0 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.260+0000 7f95e0c00700 1 -- 172.21.15.88:0/3177436723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95bc0052f0 con 0x7f95dc1088d0 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.260+0000 7f95d2a00700 1 -- 172.21.15.88:0/3177436723 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7f95cc00f5d0 con 0x7f95dc1088d0 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.261+0000 7f95d2a00700 1 --2- 172.21.15.88:0/3177436723 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f95c00388b0 0x7f95c003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.261+0000 7f95d2a00700 1 -- 172.21.15.88:0/3177436723 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f95cc04c960 con 0x7f95dc1088d0 2024-03-24T00:38:46.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.261+0000 7f95d9400700 1 --2- 172.21.15.88:0/3177436723 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f95c00388b0 0x7f95c003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:46.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.263+0000 7f95d9400700 1 --2- 172.21.15.88:0/3177436723 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f95c00388b0 0x7f95c003ad70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95c8006fd0 tx=0x7f95c8006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:46.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.277+0000 7f95d2a00700 1 -- 172.21.15.88:0/3177436723 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f95cc00e5c0 con 0x7f95dc1088d0 2024-03-24T00:38:46.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.599+0000 7f95e0c00700 1 -- 172.21.15.88:0/3177436723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f95bc005160 con 0x7f95dc1088d0 2024-03-24T00:38:46.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.600+0000 7f95d2a00700 1 -- 172.21.15.88:0/3177436723 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f95cc027070 con 0x7f95dc1088d0 2024-03-24T00:38:46.601 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:46.601 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:46.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.602+0000 7f95e0c00700 1 -- 172.21.15.88:0/3177436723 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f95c00388b0 msgr2=0x7f95c003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:46.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.602+0000 7f95e0c00700 1 --2- 172.21.15.88:0/3177436723 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f95c00388b0 0x7f95c003ad70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95c8006fd0 tx=0x7f95c8006e40 comp rx=0 tx=0).stop 2024-03-24T00:38:46.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.602+0000 7f95e0c00700 1 -- 172.21.15.88:0/3177436723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95dc1088d0 msgr2=0x7f95dc1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:46.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.602+0000 7f95e0c00700 1 --2- 172.21.15.88:0/3177436723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95dc1088d0 0x7f95dc1981d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0098e0 tx=0x7f95cc005520 comp rx=0 tx=0).stop 2024-03-24T00:38:46.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.602+0000 7f95e0c00700 1 -- 172.21.15.88:0/3177436723 shutdown_connections 2024-03-24T00:38:46.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.602+0000 7f95e0c00700 1 --2- 172.21.15.88:0/3177436723 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f95c00388b0 0x7f95c003ad70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:46.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.602+0000 7f95e0c00700 1 --2- 172.21.15.88:0/3177436723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95dc1088d0 0x7f95dc1981d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:46.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.602+0000 7f95e0c00700 1 -- 172.21.15.88:0/3177436723 >> 172.21.15.88:0/3177436723 conn(0x7f95dc0fe400 msgr2=0x7f95dc10a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:46.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.603+0000 7f95e0c00700 1 -- 172.21.15.88:0/3177436723 shutdown_connections 2024-03-24T00:38:46.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:46.603+0000 7f95e0c00700 1 -- 172.21.15.88:0/3177436723 wait complete. 2024-03-24T00:38:46.604 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:47.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:47.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:47.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:47.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:47.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:38:47.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-24T00:38:47.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:38:47.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:47 smithi038 ceph-mon[29271]: Deploying daemon crash.smithi088 on smithi088 2024-03-24T00:38:47.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:47 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/3177436723' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:48.367 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:48.368 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:48.793 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:38:50.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:49 smithi038 ceph-mon[29271]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:50.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:49 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:51.012 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:50 smithi038 ceph-mon[29271]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:51.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.038+0000 7fab07200700 1 -- 172.21.15.88:0/3940650366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab001066d0 msgr2=0x7fab00106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.038+0000 7fab07200700 1 --2- 172.21.15.88:0/3940650366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab001066d0 0x7fab00106ab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009910 tx=0x7faaf0009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.039+0000 7fab07200700 1 -- 172.21.15.88:0/3940650366 shutdown_connections 2024-03-24T00:38:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.039+0000 7fab07200700 1 --2- 172.21.15.88:0/3940650366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab001066d0 0x7fab00106ab0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.039+0000 7fab07200700 1 -- 172.21.15.88:0/3940650366 >> 172.21.15.88:0/3940650366 conn(0x7fab00075110 msgr2=0x7fab00075520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.039+0000 7fab07200700 1 -- 172.21.15.88:0/3940650366 shutdown_connections 2024-03-24T00:38:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.039+0000 7fab07200700 1 -- 172.21.15.88:0/3940650366 wait complete. 2024-03-24T00:38:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.040+0000 7fab07200700 1 Processor -- start 2024-03-24T00:38:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.041+0000 7fab07200700 1 -- start start 2024-03-24T00:38:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.041+0000 7fab07200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab001066d0 0x7fab000ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.041+0000 7fab07200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab001015a0 con 0x7fab001066d0 2024-03-24T00:38:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.042+0000 7faaffe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab001066d0 0x7fab000ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.042+0000 7faaffe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab001066d0 0x7fab000ff6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:40804/0 (socket says 172.21.15.88:40804) 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.042+0000 7faaffe00700 1 -- 172.21.15.88:0/3227177922 learned_addr learned my addr 172.21.15.88:0/3227177922 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.042+0000 7faaffe00700 1 -- 172.21.15.88:0/3227177922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf00095f0 con 0x7fab001066d0 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.043+0000 7faaffe00700 1 --2- 172.21.15.88:0/3227177922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab001066d0 0x7fab000ff6e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faaf0005490 tx=0x7faaf0005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.043+0000 7faafcc00700 1 -- 172.21.15.88:0/3227177922 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faaf001d070 con 0x7fab001066d0 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.043+0000 7faafcc00700 1 -- 172.21.15.88:0/3227177922 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faaf0005700 con 0x7fab001066d0 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.043+0000 7faafcc00700 1 -- 172.21.15.88:0/3227177922 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faaf00038e0 con 0x7fab001066d0 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.043+0000 7fab07200700 1 -- 172.21.15.88:0/3227177922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab000ffc20 con 0x7fab001066d0 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.044+0000 7fab07200700 1 -- 172.21.15.88:0/3227177922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab001000a0 con 0x7fab001066d0 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.044+0000 7fab07200700 1 -- 172.21.15.88:0/3227177922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab00108ff0 con 0x7fab001066d0 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.046+0000 7faafcc00700 1 -- 172.21.15.88:0/3227177922 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7faaf0003ba0 con 0x7fab001066d0 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.046+0000 7faafcc00700 1 --2- 172.21.15.88:0/3227177922 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faae40388c0 0x7faae403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:51.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.046+0000 7faafcc00700 1 -- 172.21.15.88:0/3227177922 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7faaf004c5d0 con 0x7fab001066d0 2024-03-24T00:38:51.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.047+0000 7faaff400700 1 --2- 172.21.15.88:0/3227177922 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faae40388c0 0x7faae403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:51.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.048+0000 7faaff400700 1 --2- 172.21.15.88:0/3227177922 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faae40388c0 0x7faae403ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faaec006fd0 tx=0x7faaec006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:51.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.062+0000 7faafcc00700 1 -- 172.21.15.88:0/3227177922 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faaf002a980 con 0x7fab001066d0 2024-03-24T00:38:51.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.367+0000 7fab07200700 1 -- 172.21.15.88:0/3227177922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fab00100cc0 con 0x7fab001066d0 2024-03-24T00:38:51.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.368+0000 7faafcc00700 1 -- 172.21.15.88:0/3227177922 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7faaf0026070 con 0x7fab001066d0 2024-03-24T00:38:51.368 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:51.369 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:51.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.371+0000 7fab07200700 1 -- 172.21.15.88:0/3227177922 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faae40388c0 msgr2=0x7faae403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:51.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.371+0000 7fab07200700 1 --2- 172.21.15.88:0/3227177922 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faae40388c0 0x7faae403ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faaec006fd0 tx=0x7faaec006e40 comp rx=0 tx=0).stop 2024-03-24T00:38:51.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.371+0000 7fab07200700 1 -- 172.21.15.88:0/3227177922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab001066d0 msgr2=0x7fab000ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:51.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.371+0000 7fab07200700 1 --2- 172.21.15.88:0/3227177922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab001066d0 0x7fab000ff6e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faaf0005490 tx=0x7faaf0005570 comp rx=0 tx=0).stop 2024-03-24T00:38:51.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.371+0000 7fab07200700 1 -- 172.21.15.88:0/3227177922 shutdown_connections 2024-03-24T00:38:51.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.371+0000 7fab07200700 1 --2- 172.21.15.88:0/3227177922 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faae40388c0 0x7faae403ad80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:51.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.371+0000 7fab07200700 1 --2- 172.21.15.88:0/3227177922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab001066d0 0x7fab000ff6e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:51.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.371+0000 7fab07200700 1 -- 172.21.15.88:0/3227177922 >> 172.21.15.88:0/3227177922 conn(0x7fab00075110 msgr2=0x7fab000fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:51.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.371+0000 7fab07200700 1 -- 172.21.15.88:0/3227177922 shutdown_connections 2024-03-24T00:38:51.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:51.372+0000 7fab07200700 1 -- 172.21.15.88:0/3227177922 wait complete. 2024-03-24T00:38:51.373 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:52.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:52.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:52.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:52.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:52 smithi038 ceph-mon[29271]: Deploying daemon node-exporter.smithi088 on smithi088 2024-03-24T00:38:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:52 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/3227177922' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:53.112 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:53.113 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:53.474 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:38:53.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:53 smithi038 ceph-mon[29271]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:55.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:54 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:55.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:54 smithi038 ceph-mon[29271]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:57.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.161+0000 7f0126e00700 1 -- 172.21.15.88:0/489346031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0120073150 msgr2=0x7f0120073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:57.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.161+0000 7f0126e00700 1 --2- 172.21.15.88:0/489346031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0120073150 0x7f0120073530 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0110009910 tx=0x7f0110009c20 comp rx=0 tx=0).stop 2024-03-24T00:38:57.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.161+0000 7f0126e00700 1 -- 172.21.15.88:0/489346031 shutdown_connections 2024-03-24T00:38:57.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.161+0000 7f0126e00700 1 --2- 172.21.15.88:0/489346031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0120073150 0x7f0120073530 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:57.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.161+0000 7f0126e00700 1 -- 172.21.15.88:0/489346031 >> 172.21.15.88:0/489346031 conn(0x7f01200fc1d0 msgr2=0x7f01200fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:57.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.162+0000 7f0126e00700 1 -- 172.21.15.88:0/489346031 shutdown_connections 2024-03-24T00:38:57.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.162+0000 7f0126e00700 1 -- 172.21.15.88:0/489346031 wait complete. 2024-03-24T00:38:57.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.163+0000 7f0126e00700 1 Processor -- start 2024-03-24T00:38:57.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.163+0000 7f0126e00700 1 -- start start 2024-03-24T00:38:57.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.163+0000 7f0126e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0120073150 0x7f012019c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:57.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.163+0000 7f0126e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012019cb40 con 0x7f0120073150 2024-03-24T00:38:57.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.164+0000 7f011fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0120073150 0x7f012019c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:57.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.164+0000 7f011fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0120073150 0x7f012019c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:45944/0 (socket says 172.21.15.88:45944) 2024-03-24T00:38:57.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.164+0000 7f011fe00700 1 -- 172.21.15.88:0/1279355940 learned_addr learned my addr 172.21.15.88:0/1279355940 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:38:57.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.165+0000 7f011fe00700 1 -- 172.21.15.88:0/1279355940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01100095f0 con 0x7f0120073150 2024-03-24T00:38:57.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.165+0000 7f011fe00700 1 --2- 172.21.15.88:0/1279355940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0120073150 0x7f012019c600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f01100098e0 tx=0x7f0110005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:57.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.166+0000 7f011cc00700 1 -- 172.21.15.88:0/1279355940 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f011001d070 con 0x7f0120073150 2024-03-24T00:38:57.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.166+0000 7f0126e00700 1 -- 172.21.15.88:0/1279355940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012019cda0 con 0x7f0120073150 2024-03-24T00:38:57.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.166+0000 7f011cc00700 1 -- 172.21.15.88:0/1279355940 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01100056b0 con 0x7f0120073150 2024-03-24T00:38:57.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.166+0000 7f0126e00700 1 -- 172.21.15.88:0/1279355940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012019d220 con 0x7f0120073150 2024-03-24T00:38:57.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.166+0000 7f011cc00700 1 -- 172.21.15.88:0/1279355940 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0110022d20 con 0x7f0120073150 2024-03-24T00:38:57.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.167+0000 7f0126e00700 1 -- 172.21.15.88:0/1279355940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0120074890 con 0x7f0120073150 2024-03-24T00:38:57.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.167+0000 7f011cc00700 1 -- 172.21.15.88:0/1279355940 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7f0110022470 con 0x7f0120073150 2024-03-24T00:38:57.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.168+0000 7f011cc00700 1 --2- 172.21.15.88:0/1279355940 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f01040388b0 0x7f010403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:38:57.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.168+0000 7f011cc00700 1 -- 172.21.15.88:0/1279355940 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f011004c860 con 0x7f0120073150 2024-03-24T00:38:57.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.168+0000 7f011f400700 1 --2- 172.21.15.88:0/1279355940 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f01040388b0 0x7f010403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:38:57.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.170+0000 7f011f400700 1 --2- 172.21.15.88:0/1279355940 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f01040388b0 0x7f010403ad70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f010c006fd0 tx=0x7f010c006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:38:57.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.185+0000 7f011cc00700 1 -- 172.21.15.88:0/1279355940 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f011000f460 con 0x7f0120073150 2024-03-24T00:38:57.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.508+0000 7f0126e00700 1 -- 172.21.15.88:0/1279355940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f01201a0ea0 con 0x7f0120073150 2024-03-24T00:38:57.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.511+0000 7f011cc00700 1 -- 172.21.15.88:0/1279355940 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f0110027030 con 0x7f0120073150 2024-03-24T00:38:57.511 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:38:57.511 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:38:57.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.513+0000 7f0126e00700 1 -- 172.21.15.88:0/1279355940 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f01040388b0 msgr2=0x7f010403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:57.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.513+0000 7f0126e00700 1 --2- 172.21.15.88:0/1279355940 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f01040388b0 0x7f010403ad70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f010c006fd0 tx=0x7f010c006e40 comp rx=0 tx=0).stop 2024-03-24T00:38:57.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.513+0000 7f0126e00700 1 -- 172.21.15.88:0/1279355940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0120073150 msgr2=0x7f012019c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:38:57.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.513+0000 7f0126e00700 1 --2- 172.21.15.88:0/1279355940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0120073150 0x7f012019c600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f01100098e0 tx=0x7f0110005520 comp rx=0 tx=0).stop 2024-03-24T00:38:57.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.513+0000 7f0126e00700 1 -- 172.21.15.88:0/1279355940 shutdown_connections 2024-03-24T00:38:57.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.513+0000 7f0126e00700 1 --2- 172.21.15.88:0/1279355940 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f01040388b0 0x7f010403ad70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:57.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.513+0000 7f0126e00700 1 --2- 172.21.15.88:0/1279355940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0120073150 0x7f012019c600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:38:57.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.513+0000 7f0126e00700 1 -- 172.21.15.88:0/1279355940 >> 172.21.15.88:0/1279355940 conn(0x7f01200fc1d0 msgr2=0x7f01200fc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:38:57.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.513+0000 7f0126e00700 1 -- 172.21.15.88:0/1279355940 shutdown_connections 2024-03-24T00:38:57.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:38:57.513+0000 7f0126e00700 1 -- 172.21.15.88:0/1279355940 wait complete. 2024-03-24T00:38:57.515 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:38:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:57 smithi038 ceph-mon[29271]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:38:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:58.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:58.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:57 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/1279355940' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:38:58.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:58.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:38:58.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:38:58.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T00:38:58.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:38:58.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:38:59.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:58 smithi038 ceph-mon[29271]: Deploying daemon mgr.smithi088.njctea on smithi088 2024-03-24T00:38:59.216 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:38:59.217 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:38:59.676 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:39:00.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:59 smithi038 ceph-mon[29271]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:39:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:38:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:02.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:01 smithi038 ceph-mon[29271]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:02.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.166+0000 7f9187800700 1 -- 172.21.15.88:0/3573430466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91801066d0 msgr2=0x7f9180106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:02.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.166+0000 7f9187800700 1 --2- 172.21.15.88:0/3573430466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91801066d0 0x7f9180106ab0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9174009910 tx=0x7f9174009c20 comp rx=0 tx=0).stop 2024-03-24T00:39:02.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.166+0000 7f9187800700 1 -- 172.21.15.88:0/3573430466 shutdown_connections 2024-03-24T00:39:02.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.166+0000 7f9187800700 1 --2- 172.21.15.88:0/3573430466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91801066d0 0x7f9180106ab0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:02.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.166+0000 7f9187800700 1 -- 172.21.15.88:0/3573430466 >> 172.21.15.88:0/3573430466 conn(0x7f9180075110 msgr2=0x7f9180075520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:39:02.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.166+0000 7f9187800700 1 -- 172.21.15.88:0/3573430466 shutdown_connections 2024-03-24T00:39:02.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.166+0000 7f9187800700 1 -- 172.21.15.88:0/3573430466 wait complete. 2024-03-24T00:39:02.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.168+0000 7f9187800700 1 Processor -- start 2024-03-24T00:39:02.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.168+0000 7f9187800700 1 -- start start 2024-03-24T00:39:02.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.168+0000 7f9187800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91801066d0 0x7f9180195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:39:02.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.168+0000 7f9187800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9180196530 con 0x7f91801066d0 2024-03-24T00:39:02.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.169+0000 7f9184e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91801066d0 0x7f9180195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:39:02.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.169+0000 7f9184e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91801066d0 0x7f9180195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:45994/0 (socket says 172.21.15.88:45994) 2024-03-24T00:39:02.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.169+0000 7f9184e00700 1 -- 172.21.15.88:0/578527579 learned_addr learned my addr 172.21.15.88:0/578527579 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:39:02.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.170+0000 7f9184e00700 1 -- 172.21.15.88:0/578527579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91740095f0 con 0x7f91801066d0 2024-03-24T00:39:02.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.170+0000 7f9184e00700 1 --2- 172.21.15.88:0/578527579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91801066d0 0x7f9180195ff0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f91740098e0 tx=0x7f9174005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:39:02.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.171+0000 7f917d600700 1 -- 172.21.15.88:0/578527579 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f917401d070 con 0x7f91801066d0 2024-03-24T00:39:02.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.171+0000 7f9187800700 1 -- 172.21.15.88:0/578527579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9180196790 con 0x7f91801066d0 2024-03-24T00:39:02.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.171+0000 7f917d600700 1 -- 172.21.15.88:0/578527579 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91740056b0 con 0x7f91801066d0 2024-03-24T00:39:02.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.171+0000 7f917d600700 1 -- 172.21.15.88:0/578527579 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9174022d80 con 0x7f91801066d0 2024-03-24T00:39:02.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.171+0000 7f9187800700 1 -- 172.21.15.88:0/578527579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9180196c10 con 0x7f91801066d0 2024-03-24T00:39:02.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.172+0000 7f9187800700 1 -- 172.21.15.88:0/578527579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9180101e10 con 0x7f91801066d0 2024-03-24T00:39:02.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.172+0000 7f917d600700 1 -- 172.21.15.88:0/578527579 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7f917400b960 con 0x7f91801066d0 2024-03-24T00:39:02.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.173+0000 7f917d600700 1 --2- 172.21.15.88:0/578527579 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9168038900 0x7f916803adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:39:02.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.173+0000 7f917d600700 1 -- 172.21.15.88:0/578527579 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f917404c9c0 con 0x7f91801066d0 2024-03-24T00:39:02.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.173+0000 7f917fe00700 1 --2- 172.21.15.88:0/578527579 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9168038900 0x7f916803adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:39:02.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.174+0000 7f917fe00700 1 --2- 172.21.15.88:0/578527579 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9168038900 0x7f916803adc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f916c006fd0 tx=0x7f916c006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:39:02.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.188+0000 7f917d600700 1 -- 172.21.15.88:0/578527579 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f917402b590 con 0x7f91801066d0 2024-03-24T00:39:02.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.499+0000 7f9187800700 1 -- 172.21.15.88:0/578527579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f918019a940 con 0x7f91801066d0 2024-03-24T00:39:02.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.500+0000 7f917d600700 1 -- 172.21.15.88:0/578527579 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f9174027070 con 0x7f91801066d0 2024-03-24T00:39:02.500 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:39:02.500 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":1,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:35:32.910962Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T00:39:02.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.503+0000 7f9187800700 1 -- 172.21.15.88:0/578527579 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9168038900 msgr2=0x7f916803adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:02.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.503+0000 7f9187800700 1 --2- 172.21.15.88:0/578527579 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9168038900 0x7f916803adc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f916c006fd0 tx=0x7f916c006e40 comp rx=0 tx=0).stop 2024-03-24T00:39:02.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.503+0000 7f9187800700 1 -- 172.21.15.88:0/578527579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91801066d0 msgr2=0x7f9180195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:02.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.503+0000 7f9187800700 1 --2- 172.21.15.88:0/578527579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91801066d0 0x7f9180195ff0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f91740098e0 tx=0x7f9174005520 comp rx=0 tx=0).stop 2024-03-24T00:39:02.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.503+0000 7f9187800700 1 -- 172.21.15.88:0/578527579 shutdown_connections 2024-03-24T00:39:02.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.503+0000 7f9187800700 1 --2- 172.21.15.88:0/578527579 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9168038900 0x7f916803adc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:02.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.503+0000 7f9187800700 1 --2- 172.21.15.88:0/578527579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91801066d0 0x7f9180195ff0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:02.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.503+0000 7f9187800700 1 -- 172.21.15.88:0/578527579 >> 172.21.15.88:0/578527579 conn(0x7f9180075110 msgr2=0x7f91800fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:39:02.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.503+0000 7f9187800700 1 -- 172.21.15.88:0/578527579 shutdown_connections 2024-03-24T00:39:02.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:02.503+0000 7f9187800700 1 -- 172.21.15.88:0/578527579 wait complete. 2024-03-24T00:39:02.504 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 1 2024-03-24T00:39:03.755 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:03.755 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:03.755 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:03 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/578527579' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:39:03.755 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:03.755 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:03.755 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:39:03.755 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:03.755 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:03 smithi038 ceph-mon[29271]: Deploying daemon mon.smithi088 on smithi088 2024-03-24T00:39:03.756 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:03 smithi038 ceph-mon[29271]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:04.296 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T00:39:04.296 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T00:39:05.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:04 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:05.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:04 smithi038 ceph-mon[29271]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:06.086 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi088/config 2024-03-24T00:39:07.797 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:07 smithi088 podman[36563]: 2024-03-24 00:39:07.622344914 +0000 UTC m=+0.449105267 container create 7850aeed066fd515bc359d883e32a04103504f246ddb31f64d8d99cc40846533 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-738c2fd, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-24T00:39:08.059 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 podman[36563]: 2024-03-24 00:39:08.009525184 +0000 UTC m=+0.836285496 container init 7850aeed066fd515bc359d883e32a04103504f246ddb31f64d8d99cc40846533 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-738c2fd, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-24T00:39:08.059 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 podman[36563]: 2024-03-24 00:39:08.020122996 +0000 UTC m=+0.846883310 container start 7850aeed066fd515bc359d883e32a04103504f246ddb31f64d8d99cc40846533 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=reef-738c2fd) 2024-03-24T00:39:08.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:07 smithi038 ceph-mon[29271]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:08.340 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T00:39:08.341 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable), process ceph-mon, pid 2 2024-03-24T00:39:08.341 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: pidfile_write: ignore empty --pid-file 2024-03-24T00:39:08.341 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: load: jerasure load: lrc 2024-03-24T00:39:08.341 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: RocksDB version: 7.9.2 2024-03-24T00:39:08.341 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Git sha 0 2024-03-24T00:39:08.341 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Compile date 2024-03-23 03:05:26 2024-03-24T00:39:08.341 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: DB SUMMARY 2024-03-24T00:39:08.341 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: DB Session ID: 33X39524W8BI7FC3QDX0 2024-03-24T00:39:08.341 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: CURRENT file: CURRENT 2024-03-24T00:39:08.341 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi088/store.db dir, Total Num: 0, files: 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi088/store.db: 000004.log size: 511 ; 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.error_if_exists: 0 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.create_if_missing: 0 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.paranoid_checks: 1 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.env: 0x55854b047720 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.info_log: 0x55854e5893c0 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T00:39:08.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.statistics: (nil) 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.use_fsync: 0 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_log_file_size: 0 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.allow_fallocate: 1 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.use_direct_reads: 0 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.db_log_dir: 2024-03-24T00:39:08.343 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.wal_dir: 2024-03-24T00:39:08.344 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T00:39:08.344 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T00:39:08.344 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T00:39:08.344 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T00:39:08.344 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T00:39:08.344 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.write_buffer_manager: 0x55854d802140 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.unordered_write: 0 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T00:39:08.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T00:39:08.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T00:39:08.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T00:39:08.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.row_cache: None 2024-03-24T00:39:08.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.wal_filter: None 2024-03-24T00:39:08.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T00:39:08.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T00:39:08.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.two_write_queues: 0 2024-03-24T00:39:08.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T00:39:08.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.wal_compression: 0 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.atomic_flush: 0 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.log_readahead_size: 0 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T00:39:08.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_background_jobs: 2 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_background_compactions: -1 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_subcompactions: 1 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_open_files: -1 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T00:39:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_background_flushes: -1 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Compression algorithms supported: 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: kZSTD supported: 0 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: kXpressCompression supported: 0 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: kZlibCompression supported: 1 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: kSnappyCompression supported: 1 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: kLZ4Compression supported: 1 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: kBZip2Compression supported: 0 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T00:39:08.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T00:39:08.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi088/store.db/MANIFEST-000005 2024-03-24T00:39:08.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T00:39:08.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T00:39:08.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.merge_operator: 2024-03-24T00:39:08.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_filter: None 2024-03-24T00:39:08.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T00:39:08.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T00:39:08.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T00:39:08.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T00:39:08.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55854e5894e0) 2024-03-24T00:39:08.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: cache_index_and_filter_blocks: 1 2024-03-24T00:39:08.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T00:39:08.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T00:39:08.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: pin_top_level_index_and_filter: 1 2024-03-24T00:39:08.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_type: 0 2024-03-24T00:39:08.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: data_block_index_type: 0 2024-03-24T00:39:08.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_shortening: 1 2024-03-24T00:39:08.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T00:39:08.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: checksum: 4 2024-03-24T00:39:08.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: no_block_cache: 0 2024-03-24T00:39:08.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache: 0x55854d8a3350 2024-03-24T00:39:08.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_name: BinnedLRUCache 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_options: 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: capacity : 536870912 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: num_shard_bits : 4 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: strict_capacity_limit : 0 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: high_pri_pool_ratio: 0.000 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_compressed: (nil) 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: persistent_cache: (nil) 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_size: 4096 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_size_deviation: 10 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_restart_interval: 16 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_block_restart_interval: 1 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: metadata_block_size: 4096 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: partition_filters: 0 2024-03-24T00:39:08.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: use_delta_encoding: 1 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: filter_policy: bloomfilter 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: whole_key_filtering: 1 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: verify_compression: 0 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: read_amp_bytes_per_bit: 0 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: format_version: 5 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: enable_index_compression: 1 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_align: 0 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: max_auto_readahead_size: 262144 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: prepopulate_block_cache: 0 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: initial_auto_readahead_size: 8192 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T00:39:08.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compression: NoCompression 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.num_levels: 7 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T00:39:08.355 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T00:39:08.356 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T00:39:08.357 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T00:39:08.358 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.table_properties_collectors: 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.inplace_update_support: 0 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.bloom_locality: 0 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.max_successive_merges: 0 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T00:39:08.359 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.ttl: 2592000 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.enable_blob_files: false 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.min_blob_size: 0 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T00:39:08.360 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi088/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c7c1ceb2-c7ae-4362-880a-60bb5eb2f13c 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711240748074356, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711240748076017, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711240748, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c7c1ceb2-c7ae-4362-880a-60bb5eb2f13c", "db_session_id": "33X39524W8BI7FC3QDX0", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711240748076312, "job": 1, "event": "recovery_finished"} 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi088/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55854d940000 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: DB pointer 0x55854d92e000 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T00:39:08.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** DB Stats ** 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** Compaction Stats [default] ** 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-24T00:39:08.362 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** Compaction Stats [default] ** 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative compaction: 0.00 GB write, 0.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T00:39:08.363 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval compaction: 0.00 GB write, 0.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Block cache BinnedLRUCache@0x55854d8a3350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088 does not exist in monmap, will attempt to join an existing cluster 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: using public_addr v2:172.21.15.88:0/0 -> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: starting mon.smithi088 rank -1 at public addrs [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] at bind addrs [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi088 fsid 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(???) e0 preinit fsid 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).mds e1 new map 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).mds e1 print_map 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: e1 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:39:08.364 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: legacy client fscid: -1 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: No filesystems configured 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T00:39:08.365 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Active manager daemon smithi038.huhioz restarted 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Activating manager daemon smithi038.huhioz 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: osdmap e5: 0 total, 0 up, 0 in 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mgrmap e15: smithi038.huhioz(active, starting, since 0.00954321s) 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi038.huhioz", "id": "smithi038.huhioz"}]: dispatch 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Manager daemon smithi038.huhioz is now available 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:39:08.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/trash_purge_schedule"}]: dispatch 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: [24/Mar/2024:00:38:29] ENGINE Bus STARTING 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: [24/Mar/2024:00:38:29] ENGINE Serving on http://172.21.15.38:8765 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: [24/Mar/2024:00:38:29] ENGINE Serving on https://172.21.15.38:7150 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: [24/Mar/2024:00:38:29] ENGINE Bus STARTED 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mgrmap e16: smithi038.huhioz(active, since 1.01412s) 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/1651913573' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mgrmap e17: smithi038.huhioz(active, since 2s) 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.367 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/486962747' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi088", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:39:08.368 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:08.369 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:39:08.369 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Updating smithi038:/etc/ceph/ceph.conf 2024-03-24T00:39:08.369 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Updating smithi088:/etc/ceph/ceph.conf 2024-03-24T00:39:08.369 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:39:08.369 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:39:08.369 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/1448482515' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:39:08.369 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Deploying daemon ceph-exporter.smithi088 on smithi088 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:39:08.370 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Deploying daemon crash.smithi088 on smithi088 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/3177436723' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.371 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Deploying daemon node-exporter.smithi088 on smithi088 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/3227177922' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/1279355940' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:39:08.372 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Deploying daemon mgr.smithi088.njctea on smithi088 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/578527579' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:08.373 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: Deploying daemon mon.smithi088 on smithi088 2024-03-24T00:39:08.374 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:08.374 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:08.374 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:08.374 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:08.374 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 ceph-mon[36610]: mon.smithi088@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-03-24T00:39:08.374 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 bash[36563]: 7850aeed066fd515bc359d883e32a04103504f246ddb31f64d8d99cc40846533 2024-03-24T00:39:08.374 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:08 smithi088 systemd[1]: Started Ceph mon.smithi088 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:39:13.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:39:13.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: mon.smithi038 calling monitor election 2024-03-24T00:39:13.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:13.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: mon.smithi088 calling monitor election 2024-03-24T00:39:13.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:13.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/231420256' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/crt"}]: dispatch 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: mon.smithi038 is new leader, mons smithi038,smithi088 in quorum (ranks 0,1) 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: monmap e2: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: fsmap 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: osdmap e5: 0 total, 0 up, 0 in 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: mgrmap e17: smithi038.huhioz(active, since 44s) 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: Standby manager daemon smithi088.njctea started 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/231420256' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: overall HEALTH_OK 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/231420256' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/key"}]: dispatch 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/231420256' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:39:13.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:13.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:13.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:39:13.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: mon.smithi038 calling monitor election 2024-03-24T00:39:13.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:13.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: mon.smithi088 calling monitor election 2024-03-24T00:39:13.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:13.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/231420256' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/crt"}]: dispatch 2024-03-24T00:39:13.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: mon.smithi038 is new leader, mons smithi038,smithi088 in quorum (ranks 0,1) 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: monmap e2: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: fsmap 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: osdmap e5: 0 total, 0 up, 0 in 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: mgrmap e17: smithi038.huhioz(active, since 44s) 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: Standby manager daemon smithi088.njctea started 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/231420256' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: overall HEALTH_OK 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/231420256' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/key"}]: dispatch 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/231420256' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:13.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:14 smithi038 ceph-mon[29271]: mgrmap e18: smithi038.huhioz(active, since 44s), standbys: smithi088.njctea 2024-03-24T00:39:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi088.njctea", "id": "smithi088.njctea"}]: dispatch 2024-03-24T00:39:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:39:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:39:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:14.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.499+0000 7f2c73e00700 1 -- 172.21.15.88:0/1951517184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540062a0 msgr2=0x7f2c54008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:14.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.499+0000 7f2c73e00700 1 --2- 172.21.15.88:0/1951517184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540062a0 0x7f2c54008710 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c1018b0 tx=0x7f2c6000fe90 comp rx=0 tx=0).stop 2024-03-24T00:39:14.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.499+0000 7f2c73e00700 1 -- 172.21.15.88:0/1951517184 shutdown_connections 2024-03-24T00:39:14.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.499+0000 7f2c73e00700 1 --2- 172.21.15.88:0/1951517184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540062a0 0x7f2c54008710 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:14.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.499+0000 7f2c73e00700 1 -- 172.21.15.88:0/1951517184 >> 172.21.15.88:0/1951517184 conn(0x7f2c6c0fc300 msgr2=0x7f2c6c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:39:14.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.499+0000 7f2c73e00700 1 -- 172.21.15.88:0/1951517184 shutdown_connections 2024-03-24T00:39:14.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.499+0000 7f2c73e00700 1 -- 172.21.15.88:0/1951517184 wait complete. 2024-03-24T00:39:14.502 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:14 smithi088 ceph-mon[36610]: mgrmap e18: smithi038.huhioz(active, since 44s), standbys: smithi088.njctea 2024-03-24T00:39:14.502 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi088.njctea", "id": "smithi088.njctea"}]: dispatch 2024-03-24T00:39:14.502 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:14.502 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:14.502 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:14.502 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:39:14.502 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:39:14.502 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:14.502 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:39:14.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.501+0000 7f2c73e00700 1 Processor -- start 2024-03-24T00:39:14.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.501+0000 7f2c73e00700 1 -- start start 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.502+0000 7f2c73e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c540062a0 0x7f2c6c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.502+0000 7f2c73e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c198a30 0x7f2c6c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.502+0000 7f2c73e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c199050 con 0x7f2c6c198a30 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.502+0000 7f2c73e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c1991c0 con 0x7f2c540062a0 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.503+0000 7f2c70a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c198a30 0x7f2c6c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.503+0000 7f2c70a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c198a30 0x7f2c6c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.88:36734/0 (socket says 172.21.15.88:36734) 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.503+0000 7f2c70a00700 1 -- 172.21.15.88:0/159457756 learned_addr learned my addr 172.21.15.88:0/159457756 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.503+0000 7f2c71400700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c540062a0 0x7f2c6c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.503+0000 7f2c71400700 1 -- 172.21.15.88:0/159457756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c540062a0 msgr2=0x7f2c6c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.503+0000 7f2c71400700 1 -- 172.21.15.88:0/159457756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c540062a0 msgr2=0x7f2c6c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.503+0000 7f2c71400700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c540062a0 0x7f2c6c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-24T00:39:14.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.503+0000 7f2c71400700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c540062a0 0x7f2c6c1984f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T00:39:14.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.503+0000 7f2c70a00700 1 -- 172.21.15.88:0/159457756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c540062a0 msgr2=0x7f2c6c1984f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T00:39:14.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.503+0000 7f2c70a00700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c540062a0 0x7f2c6c1984f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:14.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.503+0000 7f2c70a00700 1 -- 172.21.15.88:0/159457756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6000f8e0 con 0x7f2c6c198a30 2024-03-24T00:39:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.504+0000 7f2c70a00700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c198a30 0x7f2c6c19cde0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800b700 tx=0x7f2c5800bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:39:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.504+0000 7f2c6a000700 1 -- 172.21.15.88:0/159457756 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c58011800 con 0x7f2c6c198a30 2024-03-24T00:39:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.504+0000 7f2c73e00700 1 -- 172.21.15.88:0/159457756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c19d3e0 con 0x7f2c6c198a30 2024-03-24T00:39:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.505+0000 7f2c73e00700 1 -- 172.21.15.88:0/159457756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c19d930 con 0x7f2c6c198a30 2024-03-24T00:39:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.505+0000 7f2c6a000700 1 -- 172.21.15.88:0/159457756 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c58011e20 con 0x7f2c6c198a30 2024-03-24T00:39:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.505+0000 7f2c6a000700 1 -- 172.21.15.88:0/159457756 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c580193b0 con 0x7f2c6c198a30 2024-03-24T00:39:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.506+0000 7f2c73e00700 1 -- 172.21.15.88:0/159457756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c105aa0 con 0x7f2c6c198a30 2024-03-24T00:39:14.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.508+0000 7f2c6a000700 1 -- 172.21.15.88:0/159457756 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 90786+0+0 (secure 0 0 0) 0x7f2c5801a800 con 0x7f2c6c198a30 2024-03-24T00:39:14.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.508+0000 7f2c6a000700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c4c06cdb0 0x7f2c4c06f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:39:14.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.508+0000 7f2c6a000700 1 -- 172.21.15.88:0/159457756 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f2c5808b120 con 0x7f2c6c198a30 2024-03-24T00:39:14.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.509+0000 7f2c71400700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c4c06cdb0 0x7f2c4c06f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:39:14.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.510+0000 7f2c71400700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c4c06cdb0 0x7f2c4c06f270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000f550 tx=0x7f2c6000f4a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:39:14.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.524+0000 7f2c6a000700 1 -- 172.21.15.88:0/159457756 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2c580565d0 con 0x7f2c6c198a30 2024-03-24T00:39:14.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.835+0000 7f2c73e00700 1 -- 172.21.15.88:0/159457756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2c6c19dc00 con 0x7f2c6c198a30 2024-03-24T00:39:14.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.836+0000 7f2c6a000700 1 -- 172.21.15.88:0/159457756 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1018 (secure 0 0 0) 0x7f2c5808a8e0 con 0x7f2c6c198a30 2024-03-24T00:39:14.837 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-03-24T00:39:14.837 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":2,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","modified":"2024-03-24T00:39:08.135721Z","created":"2024-03-24T00:35:32.910962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-24T00:39:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.839+0000 7f2c73e00700 1 -- 172.21.15.88:0/159457756 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c4c06cdb0 msgr2=0x7f2c4c06f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.839+0000 7f2c73e00700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c4c06cdb0 0x7f2c4c06f270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000f550 tx=0x7f2c6000f4a0 comp rx=0 tx=0).stop 2024-03-24T00:39:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.839+0000 7f2c73e00700 1 -- 172.21.15.88:0/159457756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c198a30 msgr2=0x7f2c6c19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.839+0000 7f2c73e00700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c198a30 0x7f2c6c19cde0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800b700 tx=0x7f2c5800bac0 comp rx=0 tx=0).stop 2024-03-24T00:39:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.839+0000 7f2c73e00700 1 -- 172.21.15.88:0/159457756 shutdown_connections 2024-03-24T00:39:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.839+0000 7f2c73e00700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c4c06cdb0 0x7f2c4c06f270 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.839+0000 7f2c73e00700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c540062a0 0x7f2c6c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.839+0000 7f2c73e00700 1 --2- 172.21.15.88:0/159457756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c198a30 0x7f2c6c19cde0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.839+0000 7f2c73e00700 1 -- 172.21.15.88:0/159457756 >> 172.21.15.88:0/159457756 conn(0x7f2c6c0fc300 msgr2=0x7f2c6c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:39:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.839+0000 7f2c73e00700 1 -- 172.21.15.88:0/159457756 shutdown_connections 2024-03-24T00:39:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:39:14.839+0000 7f2c73e00700 1 -- 172.21.15.88:0/159457756 wait complete. 2024-03-24T00:39:14.841 INFO:teuthology.orchestra.run.smithi088.stderr:dumped monmap epoch 2 2024-03-24T00:39:15.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:15 smithi088 ceph-mon[36610]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:15.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:15 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/159457756' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:39:15.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:15 smithi038 ceph-mon[29271]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:15.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:15 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/159457756' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T00:39:15.944 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-24T00:39:15.944 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-24T00:39:16.389 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:39:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:17 smithi038 ceph-mon[29271]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:17 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:17 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:17 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:17 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:39:17.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:17 smithi088 ceph-mon[36610]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:17.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:17 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:17.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:17 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:17.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:17 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:17.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:17 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:39:18.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.411+0000 7fb37e000700 1 -- 172.21.15.38:0/1172867480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3780ff470 msgr2=0x7fb3780ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:18.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.411+0000 7fb37e000700 1 --2- 172.21.15.38:0/1172867480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3780ff470 0x7fb3780ff850 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb368009910 tx=0x7fb368009c20 comp rx=0 tx=0).stop 2024-03-24T00:39:18.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.412+0000 7fb37e000700 1 -- 172.21.15.38:0/1172867480 shutdown_connections 2024-03-24T00:39:18.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.412+0000 7fb37e000700 1 --2- 172.21.15.38:0/1172867480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3780ff470 0x7fb3780ff850 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:18.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.412+0000 7fb37e000700 1 -- 172.21.15.38:0/1172867480 >> 172.21.15.38:0/1172867480 conn(0x7fb3780fb0b0 msgr2=0x7fb3780fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:39:18.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.412+0000 7fb37e000700 1 -- 172.21.15.38:0/1172867480 shutdown_connections 2024-03-24T00:39:18.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.412+0000 7fb37e000700 1 -- 172.21.15.38:0/1172867480 wait complete. 2024-03-24T00:39:18.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.413+0000 7fb37e000700 1 Processor -- start 2024-03-24T00:39:18.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.413+0000 7fb37e000700 1 -- start start 2024-03-24T00:39:18.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.414+0000 7fb37e000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3780ff470 0x7fb378198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:39:18.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.414+0000 7fb37e000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb378198890 0x7fb37819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:39:18.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.414+0000 7fb37e000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb378198eb0 con 0x7fb3780ff470 2024-03-24T00:39:18.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.414+0000 7fb37e000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb378199020 con 0x7fb378198890 2024-03-24T00:39:18.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.414+0000 7fb377400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3780ff470 0x7fb378198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:39:18.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.414+0000 7fb377400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3780ff470 0x7fb378198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54790/0 (socket says 172.21.15.38:54790) 2024-03-24T00:39:18.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.414+0000 7fb377400700 1 -- 172.21.15.38:0/1253797077 learned_addr learned my addr 172.21.15.38:0/1253797077 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:39:18.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.414+0000 7fb376a00700 1 --2- 172.21.15.38:0/1253797077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb378198890 0x7fb37819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:39:18.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.414+0000 7fb377400700 1 -- 172.21.15.38:0/1253797077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb378198890 msgr2=0x7fb37819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:18.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.414+0000 7fb377400700 1 --2- 172.21.15.38:0/1253797077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb378198890 0x7fb37819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:18.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.414+0000 7fb377400700 1 -- 172.21.15.38:0/1253797077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3680095f0 con 0x7fb3780ff470 2024-03-24T00:39:18.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.415+0000 7fb377400700 1 --2- 172.21.15.38:0/1253797077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3780ff470 0x7fb378198350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb368006010 tx=0x7fb36800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:39:18.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.415+0000 7fb36fe00700 1 -- 172.21.15.38:0/1253797077 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb36801d070 con 0x7fb3780ff470 2024-03-24T00:39:18.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.415+0000 7fb36fe00700 1 -- 172.21.15.38:0/1253797077 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb368003d50 con 0x7fb3780ff470 2024-03-24T00:39:18.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.415+0000 7fb36fe00700 1 -- 172.21.15.38:0/1253797077 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb368017440 con 0x7fb3780ff470 2024-03-24T00:39:18.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.415+0000 7fb37e000700 1 -- 172.21.15.38:0/1253797077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37819d1e0 con 0x7fb3780ff470 2024-03-24T00:39:18.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.415+0000 7fb37e000700 1 -- 172.21.15.38:0/1253797077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37819d6e0 con 0x7fb3780ff470 2024-03-24T00:39:18.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.417+0000 7fb37e000700 1 -- 172.21.15.38:0/1253797077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb378069320 con 0x7fb3780ff470 2024-03-24T00:39:18.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.418+0000 7fb36fe00700 1 -- 172.21.15.38:0/1253797077 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 90786+0+0 (secure 0 0 0) 0x7fb36800f460 con 0x7fb3780ff470 2024-03-24T00:39:18.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.419+0000 7fb36fe00700 1 --2- 172.21.15.38:0/1253797077 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb35c06cdb0 0x7fb35c06f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:39:18.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.419+0000 7fb36fe00700 1 -- 172.21.15.38:0/1253797077 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb36808c3e0 con 0x7fb3780ff470 2024-03-24T00:39:18.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.419+0000 7fb376a00700 1 --2- 172.21.15.38:0/1253797077 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb35c06cdb0 0x7fb35c06f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:39:18.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.420+0000 7fb376a00700 1 --2- 172.21.15.38:0/1253797077 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb35c06cdb0 0x7fb35c06f270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb364009850 tx=0x7fb364008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:39:18.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.424+0000 7fb36fe00700 1 -- 172.21.15.38:0/1253797077 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb368057890 con 0x7fb3780ff470 2024-03-24T00:39:18.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.660+0000 7fb37e000700 1 -- 172.21.15.38:0/1253797077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb37819d9c0 con 0x7fb3780ff470 2024-03-24T00:39:18.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.661+0000 7fb36fe00700 1 -- 172.21.15.38:0/1253797077 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7fb36808bba0 con 0x7fb3780ff470 2024-03-24T00:39:18.662 INFO:teuthology.orchestra.run.smithi038.stdout:# minimal ceph.conf for 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:39:18.662 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2024-03-24T00:39:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:39:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: mon_host = [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-03-24T00:39:18.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.663+0000 7fb37e000700 1 -- 172.21.15.38:0/1253797077 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb35c06cdb0 msgr2=0x7fb35c06f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:18.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.663+0000 7fb37e000700 1 --2- 172.21.15.38:0/1253797077 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb35c06cdb0 0x7fb35c06f270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb364009850 tx=0x7fb364008040 comp rx=0 tx=0).stop 2024-03-24T00:39:18.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.663+0000 7fb37e000700 1 -- 172.21.15.38:0/1253797077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3780ff470 msgr2=0x7fb378198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:18.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.663+0000 7fb37e000700 1 --2- 172.21.15.38:0/1253797077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3780ff470 0x7fb378198350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb368006010 tx=0x7fb36800bda0 comp rx=0 tx=0).stop 2024-03-24T00:39:18.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.663+0000 7fb37e000700 1 -- 172.21.15.38:0/1253797077 shutdown_connections 2024-03-24T00:39:18.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.663+0000 7fb37e000700 1 --2- 172.21.15.38:0/1253797077 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb35c06cdb0 0x7fb35c06f270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:18.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.663+0000 7fb37e000700 1 --2- 172.21.15.38:0/1253797077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3780ff470 0x7fb378198350 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:18.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.663+0000 7fb37e000700 1 --2- 172.21.15.38:0/1253797077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb378198890 0x7fb37819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:18.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.663+0000 7fb37e000700 1 -- 172.21.15.38:0/1253797077 >> 172.21.15.38:0/1253797077 conn(0x7fb3780fb0b0 msgr2=0x7fb378107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:39:18.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.664+0000 7fb37e000700 1 -- 172.21.15.38:0/1253797077 shutdown_connections 2024-03-24T00:39:18.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:18.664+0000 7fb37e000700 1 -- 172.21.15.38:0/1253797077 wait complete. 2024-03-24T00:39:18.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:18 smithi088 ceph-mon[36610]: Updating smithi038:/etc/ceph/ceph.conf 2024-03-24T00:39:18.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:18 smithi088 ceph-mon[36610]: Updating smithi088:/etc/ceph/ceph.conf 2024-03-24T00:39:18.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:18 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:18.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:18 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:18.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:18 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:18.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:18 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:18.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:18 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:19.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:18 smithi038 ceph-mon[29271]: Updating smithi038:/etc/ceph/ceph.conf 2024-03-24T00:39:19.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:18 smithi038 ceph-mon[29271]: Updating smithi088:/etc/ceph/ceph.conf 2024-03-24T00:39:19.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:18 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:19.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:18 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:19.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:18 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:19.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:18 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:19.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:18 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:19.388 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-24T00:39:19.388 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:39:19.388 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.conf 2024-03-24T00:39:19.460 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:39:19.460 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:39:19.583 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:39:19.583 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.conf 2024-03-24T00:39:19.666 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:19 smithi088 ceph-mon[36610]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:39:19.667 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:19 smithi088 ceph-mon[36610]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:39:19.667 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:19 smithi088 ceph-mon[36610]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:19.667 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:19 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1253797077' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:19.667 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:19 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:39:19.667 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:19 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:39:19.667 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:19 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:19.680 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:39:19.680 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:39:19.826 INFO:tasks.cephadm:Deploying OSDs... 2024-03-24T00:39:19.826 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:39:19.826 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-03-24T00:39:19.885 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-24T00:39:19.886 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-03-24T00:39:19.928 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-24T00:39:19.928 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T00:39:19.928 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-24T00:39:19.928 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T00:39:19.928 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T00:39:19.928 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-03-24 00:38:17.552868708 +0000 2024-03-24T00:39:19.928 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-03-24 00:31:25.382415372 +0000 2024-03-24T00:39:19.928 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-03-24 00:31:25.382415372 +0000 2024-03-24T00:39:19.928 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-03-24 00:31:25.382415372 +0000 2024-03-24T00:39:19.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-24T00:39:19.965 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:19 smithi038 ceph-mon[29271]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:39:19.965 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:19 smithi038 ceph-mon[29271]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:39:19.966 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:19 smithi038 ceph-mon[29271]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:19.966 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:19 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1253797077' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:19.966 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:19 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:39:19.966 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:19 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:39:19.966 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:19 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-03-24T00:39:20.055 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-03-24T00:39:20.055 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000426443 s, 1.2 MB/s 2024-03-24T00:39:20.059 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-24T00:39:20.119 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-03-24T00:39:20.172 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-24T00:39:20.172 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T00:39:20.172 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-24T00:39:20.172 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T00:39:20.172 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T00:39:20.172 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-03-24 00:38:17.553868689 +0000 2024-03-24T00:39:20.172 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-03-24 00:31:25.950404607 +0000 2024-03-24T00:39:20.173 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-03-24 00:31:25.950404607 +0000 2024-03-24T00:39:20.173 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-03-24 00:31:25.950404607 +0000 2024-03-24T00:39:20.173 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-24T00:39:20.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-03-24T00:39:20.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-03-24T00:39:20.266 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000441128 s, 1.2 MB/s 2024-03-24T00:39:20.269 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-24T00:39:20.318 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-03-24T00:39:20.365 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-24T00:39:20.365 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T00:39:20.365 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-24T00:39:20.365 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T00:39:20.365 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T00:39:20.366 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-03-24 00:38:17.554868671 +0000 2024-03-24T00:39:20.366 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-03-24 00:31:26.556393122 +0000 2024-03-24T00:39:20.366 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-03-24 00:31:26.556393122 +0000 2024-03-24T00:39:20.366 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-03-24 00:31:26.556393122 +0000 2024-03-24T00:39:20.366 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-24T00:39:20.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-03-24T00:39:20.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-03-24T00:39:20.457 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000541266 s, 946 kB/s 2024-03-24T00:39:20.461 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-24T00:39:20.518 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-03-24T00:39:20.571 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-24T00:39:20.571 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T00:39:20.571 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-24T00:39:20.571 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T00:39:20.571 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T00:39:20.571 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-03-24 00:38:17.555868652 +0000 2024-03-24T00:39:20.572 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-03-24 00:31:27.142382016 +0000 2024-03-24T00:39:20.572 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-03-24 00:31:27.142382016 +0000 2024-03-24T00:39:20.572 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-03-24 00:31:27.142382016 +0000 2024-03-24T00:39:20.572 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-24T00:39:20.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-03-24T00:39:20.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-03-24T00:39:20.659 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000501669 s, 1.0 MB/s 2024-03-24T00:39:20.664 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-24T00:39:20.717 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:39:20.717 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/scratch_devs of=/dev/stdout 2024-03-24T00:39:20.772 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-24T00:39:20.773 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_1 2024-03-24T00:39:20.825 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-24T00:39:20.825 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T00:39:20.825 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-24T00:39:20.825 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T00:39:20.825 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T00:39:20.825 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-03-24 00:39:06.712981594 +0000 2024-03-24T00:39:20.825 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-03-24 00:31:19.448040474 +0000 2024-03-24T00:39:20.825 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-03-24 00:31:19.448040474 +0000 2024-03-24T00:39:20.825 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-03-24 00:31:19.448040474 +0000 2024-03-24T00:39:20.826 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-24T00:39:20.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:20 smithi038 ceph-mon[29271]: Reconfiguring mon.smithi038 (unknown last config time)... 2024-03-24T00:39:20.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:20 smithi038 ceph-mon[29271]: Reconfiguring daemon mon.smithi038 on smithi038 2024-03-24T00:39:20.913 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-03-24T00:39:20.914 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-03-24T00:39:20.914 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000380978 s, 1.3 MB/s 2024-03-24T00:39:20.915 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:20 smithi088 ceph-mon[36610]: Reconfiguring mon.smithi038 (unknown last config time)... 2024-03-24T00:39:20.915 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:20 smithi088 ceph-mon[36610]: Reconfiguring daemon mon.smithi038 on smithi038 2024-03-24T00:39:20.917 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-24T00:39:20.970 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_2 2024-03-24T00:39:21.017 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-24T00:39:21.017 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T00:39:21.017 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-24T00:39:21.017 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T00:39:21.018 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T00:39:21.018 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-03-24 00:39:06.712981594 +0000 2024-03-24T00:39:21.018 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-03-24 00:31:20.014028907 +0000 2024-03-24T00:39:21.018 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-03-24 00:31:20.014028907 +0000 2024-03-24T00:39:21.018 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-03-24 00:31:20.014028907 +0000 2024-03-24T00:39:21.018 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-24T00:39:21.151 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-03-24T00:39:21.152 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-03-24T00:39:21.152 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000324584 s, 1.6 MB/s 2024-03-24T00:39:21.156 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-24T00:39:21.209 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_3 2024-03-24T00:39:21.256 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-24T00:39:21.256 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T00:39:21.256 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-24T00:39:21.256 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T00:39:21.256 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T00:39:21.256 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-03-24 00:39:06.713981574 +0000 2024-03-24T00:39:21.256 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-03-24 00:31:20.593017075 +0000 2024-03-24T00:39:21.256 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-03-24 00:31:20.593017075 +0000 2024-03-24T00:39:21.256 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-03-24 00:31:20.593017075 +0000 2024-03-24T00:39:21.257 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-24T00:39:21.342 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-03-24T00:39:21.343 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-03-24T00:39:21.343 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.00031049 s, 1.6 MB/s 2024-03-24T00:39:21.347 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-24T00:39:21.404 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_4 2024-03-24T00:39:21.452 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-24T00:39:21.452 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T00:39:21.452 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-24T00:39:21.452 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T00:39:21.452 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T00:39:21.452 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-03-24 00:39:06.714981554 +0000 2024-03-24T00:39:21.452 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-03-24 00:31:21.166005366 +0000 2024-03-24T00:39:21.452 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-03-24 00:31:21.166005366 +0000 2024-03-24T00:39:21.452 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-03-24 00:31:21.166005366 +0000 2024-03-24T00:39:21.453 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-24T00:39:21.536 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-03-24T00:39:21.536 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-03-24T00:39:21.536 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000364321 s, 1.4 MB/s 2024-03-24T00:39:21.540 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-24T00:39:21.594 INFO:tasks.cephadm:Deploying osd.0 on smithi038 with /dev/vg_nvme/lv_4... 2024-03-24T00:39:21.595 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-24T00:39:21.907 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:21 smithi038 ceph-mon[29271]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:21.907 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:21 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:21.907 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:21 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:21.907 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:21 smithi038 ceph-mon[29271]: Reconfiguring mgr.smithi038.huhioz (unknown last config time)... 2024-03-24T00:39:21.907 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:21 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.huhioz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:39:21.907 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:21 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:39:21.908 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:21 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:21.908 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:21 smithi038 ceph-mon[29271]: Reconfiguring daemon mgr.smithi038.huhioz on smithi038 2024-03-24T00:39:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:21 smithi088 ceph-mon[36610]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:21 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:21 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:21 smithi088 ceph-mon[36610]: Reconfiguring mgr.smithi038.huhioz (unknown last config time)... 2024-03-24T00:39:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:21 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.huhioz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:39:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:21 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:39:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:21 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:21 smithi088 ceph-mon[36610]: Reconfiguring daemon mgr.smithi038.huhioz on smithi038 2024-03-24T00:39:22.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:22 smithi038 ceph-mon[29271]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:22.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:22 smithi088 ceph-mon[36610]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:22.985 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:39:24.258 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:24 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:24.258 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:24 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:24.259 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:24 smithi038 ceph-mon[29271]: Reconfiguring ceph-exporter.smithi038 (monmap changed)... 2024-03-24T00:39:24.259 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:24 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:39:24.259 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:24 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:24.259 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:24 smithi038 ceph-mon[29271]: Reconfiguring daemon ceph-exporter.smithi038 on smithi038 2024-03-24T00:39:24.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:24 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:24.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:24 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:24 smithi088 ceph-mon[36610]: Reconfiguring ceph-exporter.smithi038 (monmap changed)... 2024-03-24T00:39:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:24 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:39:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:24 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:24 smithi088 ceph-mon[36610]: Reconfiguring daemon ceph-exporter.smithi038 on smithi038 2024-03-24T00:39:25.248 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:25 smithi038 ceph-mon[29271]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:25.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:25 smithi088 ceph-mon[36610]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:25.955 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-24T00:39:25.955 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-24T00:39:25.955 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-03-24T00:39:25.955 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-03-24T00:39:25.955 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0357802 s, 293 MB/s 2024-03-24T00:39:25.955 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-03-24T00:39:26.701 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi038:vg_nvme/lv_4 2024-03-24T00:39:27.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:27 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:27.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:27 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:27.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:27 smithi038 ceph-mon[29271]: Reconfiguring crash.smithi038 (monmap changed)... 2024-03-24T00:39:27.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:27 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:39:27.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:27 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:27.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:27 smithi038 ceph-mon[29271]: Reconfiguring daemon crash.smithi038 on smithi038 2024-03-24T00:39:27.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:27 smithi038 ceph-mon[29271]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:27.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:27 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:27.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:27 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:27.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:27 smithi088 ceph-mon[36610]: Reconfiguring crash.smithi038 (monmap changed)... 2024-03-24T00:39:27.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:27 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:39:27.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:27 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:27.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:27 smithi088 ceph-mon[36610]: Reconfiguring daemon crash.smithi038 on smithi038 2024-03-24T00:39:27.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:27 smithi088 ceph-mon[36610]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:28.144 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:39:29.435 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:29.436 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:29.436 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:29 smithi038 ceph-mon[29271]: Reconfiguring alertmanager.smithi038 (dependencies changed)... 2024-03-24T00:39:29.436 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:29 smithi038 ceph-mon[29271]: Reconfiguring daemon alertmanager.smithi038 on smithi038 2024-03-24T00:39:29.436 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:29 smithi038 ceph-mon[29271]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:29.436 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:39:29.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:29.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:29.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:29 smithi088 ceph-mon[36610]: Reconfiguring alertmanager.smithi038 (dependencies changed)... 2024-03-24T00:39:29.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:29 smithi088 ceph-mon[36610]: Reconfiguring daemon alertmanager.smithi038 on smithi038 2024-03-24T00:39:29.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:29 smithi088 ceph-mon[36610]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:29.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:39:30.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.554+0000 7f8176a00700 1 -- 172.21.15.38:0/1831755082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8170100b00 msgr2=0x7f8170100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:30.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.554+0000 7f8176a00700 1 --2- 172.21.15.38:0/1831755082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8170100b00 0x7f8170100f20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8160009910 tx=0x7f8160009c20 comp rx=0 tx=0).stop 2024-03-24T00:39:30.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.554+0000 7f8176a00700 1 -- 172.21.15.38:0/1831755082 shutdown_connections 2024-03-24T00:39:30.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.554+0000 7f8176a00700 1 --2- 172.21.15.38:0/1831755082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8170101d00 0x7f8170102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:30.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.554+0000 7f8176a00700 1 --2- 172.21.15.38:0/1831755082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8170100b00 0x7f8170100f20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:30.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.554+0000 7f8176a00700 1 -- 172.21.15.38:0/1831755082 >> 172.21.15.38:0/1831755082 conn(0x7f81700fc0a0 msgr2=0x7f81700fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:39:30.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.554+0000 7f8176a00700 1 -- 172.21.15.38:0/1831755082 shutdown_connections 2024-03-24T00:39:30.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.554+0000 7f8176a00700 1 -- 172.21.15.38:0/1831755082 wait complete. 2024-03-24T00:39:30.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.555+0000 7f8176a00700 1 Processor -- start 2024-03-24T00:39:30.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.555+0000 7f8176a00700 1 -- start start 2024-03-24T00:39:30.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.556+0000 7f8176a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8170100b00 0x7f8170198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:39:30.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.556+0000 7f8176a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8170101d00 0x7f8170198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:39:30.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.556+0000 7f8176a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170199380 con 0x7f8170101d00 2024-03-24T00:39:30.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.556+0000 7f8176a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81701994c0 con 0x7f8170100b00 2024-03-24T00:39:30.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.556+0000 7f816f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8170101d00 0x7f8170198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:39:30.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.556+0000 7f816f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8170101d00 0x7f8170198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34398/0 (socket says 172.21.15.38:34398) 2024-03-24T00:39:30.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.556+0000 7f816f400700 1 -- 172.21.15.38:0/1348109183 learned_addr learned my addr 172.21.15.38:0/1348109183 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:39:30.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.557+0000 7f816f400700 1 -- 172.21.15.38:0/1348109183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8170100b00 msgr2=0x7f8170198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:39:30.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.557+0000 7f816f400700 1 --2- 172.21.15.38:0/1348109183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8170100b00 0x7f8170198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:39:30.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.557+0000 7f816f400700 1 -- 172.21.15.38:0/1348109183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81600095f0 con 0x7f8170101d00 2024-03-24T00:39:30.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.557+0000 7f816f400700 1 --2- 172.21.15.38:0/1348109183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8170101d00 0x7f8170198d60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b4b0 tx=0x7f815c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:39:30.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.557+0000 7f816cc00700 1 -- 172.21.15.38:0/1348109183 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f815c011800 con 0x7f8170101d00 2024-03-24T00:39:30.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.557+0000 7f8176a00700 1 -- 172.21.15.38:0/1348109183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817019df20 con 0x7f8170101d00 2024-03-24T00:39:30.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.557+0000 7f816cc00700 1 -- 172.21.15.38:0/1348109183 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f815c011e20 con 0x7f8170101d00 2024-03-24T00:39:30.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.557+0000 7f816cc00700 1 -- 172.21.15.38:0/1348109183 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f815c00f2e0 con 0x7f8170101d00 2024-03-24T00:39:30.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.557+0000 7f8176a00700 1 -- 172.21.15.38:0/1348109183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817019e4c0 con 0x7f8170101d00 2024-03-24T00:39:30.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.559+0000 7f8176a00700 1 -- 172.21.15.38:0/1348109183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81701066d0 con 0x7f8170101d00 2024-03-24T00:39:30.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.559+0000 7f816cc00700 1 -- 172.21.15.38:0/1348109183 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 90786+0+0 (secure 0 0 0) 0x7f815c010380 con 0x7f8170101d00 2024-03-24T00:39:30.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.560+0000 7f816cc00700 1 --2- 172.21.15.38:0/1348109183 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f815406cc50 0x7f815406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:39:30.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.560+0000 7f816cc00700 1 -- 172.21.15.38:0/1348109183 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f815c08ad80 con 0x7f8170101d00 2024-03-24T00:39:30.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.561+0000 7f816fe00700 1 --2- 172.21.15.38:0/1348109183 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f815406cc50 0x7f815406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:39:30.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.561+0000 7f816fe00700 1 --2- 172.21.15.38:0/1348109183 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f815406cc50 0x7f815406f110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f816000b5c0 tx=0x7f8160005160 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:39:30.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.566+0000 7f816cc00700 1 -- 172.21.15.38:0/1348109183 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f815c059c20 con 0x7f8170101d00 2024-03-24T00:39:30.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:39:30.801+0000 7f8176a00700 1 -- 172.21.15.38:0/1348109183 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f81700611f0 con 0x7f815406cc50 2024-03-24T00:39:31.891 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:31 smithi038 ceph-mon[29271]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:31.891 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:31 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:39:31.891 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:31 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:39:31.892 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:31 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:31.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:31 smithi088 ceph-mon[36610]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:31.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:31 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:39:31.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:31 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:39:31.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:31 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:32.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:32 smithi038 ceph-mon[29271]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:39:32.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:32 smithi088 ceph-mon[36610]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:39:33.829 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:33 smithi038 ceph-mon[29271]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:33.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:33 smithi088 ceph-mon[36610]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:34.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:34 smithi088 ceph-mon[36610]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:34.983 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:34 smithi038 ceph-mon[29271]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:37.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:37.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:37.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:37 smithi038 ceph-mon[29271]: Reconfiguring grafana.smithi038 (dependencies changed)... 2024-03-24T00:39:37.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:37 smithi038 ceph-mon[29271]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:37.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:37 smithi038 ceph-mon[29271]: Reconfiguring daemon grafana.smithi038 on smithi038 2024-03-24T00:39:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:37 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:37 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:37 smithi088 ceph-mon[36610]: Reconfiguring grafana.smithi038 (dependencies changed)... 2024-03-24T00:39:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:37 smithi088 ceph-mon[36610]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:37 smithi088 ceph-mon[36610]: Reconfiguring daemon grafana.smithi038 on smithi038 2024-03-24T00:39:38.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:38 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3217294606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5df40af-b5d0-4f89-94b2-59ee2e710bc7"}]: dispatch 2024-03-24T00:39:38.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:38 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3217294606' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5df40af-b5d0-4f89-94b2-59ee2e710bc7"}]': finished 2024-03-24T00:39:38.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:38 smithi088 ceph-mon[36610]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T00:39:38.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:38 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:39:38.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:38 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3217294606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5df40af-b5d0-4f89-94b2-59ee2e710bc7"}]: dispatch 2024-03-24T00:39:38.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:38 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3217294606' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5df40af-b5d0-4f89-94b2-59ee2e710bc7"}]': finished 2024-03-24T00:39:38.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:38 smithi038 ceph-mon[29271]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T00:39:38.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:38 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:39:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:40 smithi038 ceph-mon[29271]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:40 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3197128677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:39:40.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:40 smithi088 ceph-mon[36610]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:40.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:40 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3197128677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:39:41.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:41 smithi088 ceph-mon[36610]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:41.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:41 smithi038 ceph-mon[29271]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:43.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:43 smithi088 ceph-mon[36610]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:43.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:43 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:39:44.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:43 smithi038 ceph-mon[29271]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:44.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:43 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:39:44.758 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:44 smithi038 ceph-mon[29271]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:45.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:44 smithi088 ceph-mon[36610]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:47.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:47 smithi088 ceph-mon[36610]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:48.008 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:47 smithi038 ceph-mon[29271]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:49.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:49 smithi038 ceph-mon[29271]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:49.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:49 smithi088 ceph-mon[36610]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:51.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:51 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:51.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:51 smithi088 ceph-mon[36610]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:51.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:51 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:51.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:51 smithi088 ceph-mon[36610]: Reconfiguring prometheus.smithi038 (dependencies changed)... 2024-03-24T00:39:51.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:51 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:51.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:51 smithi038 ceph-mon[29271]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:51.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:51 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:51.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:51 smithi038 ceph-mon[29271]: Reconfiguring prometheus.smithi038 (dependencies changed)... 2024-03-24T00:39:52.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:52 smithi038 ceph-mon[29271]: Reconfiguring daemon prometheus.smithi038 on smithi038 2024-03-24T00:39:52.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T00:39:52.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:52.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:52 smithi088 ceph-mon[36610]: Reconfiguring daemon prometheus.smithi038 on smithi038 2024-03-24T00:39:52.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T00:39:52.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:53.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:53 smithi038 ceph-mon[29271]: Deploying daemon osd.0 on smithi038 2024-03-24T00:39:53.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:53 smithi038 ceph-mon[29271]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:53.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:53 smithi088 ceph-mon[36610]: Deploying daemon osd.0 on smithi038 2024-03-24T00:39:53.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:53 smithi088 ceph-mon[36610]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:55.838 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:55 smithi038 ceph-mon[29271]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:55.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:55 smithi088 ceph-mon[36610]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:57.830 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:57 smithi038 ceph-mon[29271]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:57 smithi088 ceph-mon[36610]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:59.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:59.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:59.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:59 smithi038 ceph-mon[29271]: Reconfiguring ceph-exporter.smithi088 (monmap changed)... 2024-03-24T00:39:59.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:39:59.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:59.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:59 smithi038 ceph-mon[29271]: Reconfiguring daemon ceph-exporter.smithi088 on smithi088 2024-03-24T00:39:59.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:59 smithi038 ceph-mon[29271]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:59.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:39:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:39:59.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:59 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:59.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:59 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:39:59.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:59 smithi088 ceph-mon[36610]: Reconfiguring ceph-exporter.smithi088 (monmap changed)... 2024-03-24T00:39:59.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:59 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:39:59.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:59 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:39:59.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:59 smithi088 ceph-mon[36610]: Reconfiguring daemon ceph-exporter.smithi088 on smithi088 2024-03-24T00:39:59.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:59 smithi088 ceph-mon[36610]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:39:59.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:39:59 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:40:00.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:00 smithi088 ceph-mon[36610]: overall HEALTH_OK 2024-03-24T00:40:00.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:00 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:00.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:00 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:00.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:00 smithi038 ceph-mon[29271]: overall HEALTH_OK 2024-03-24T00:40:00.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:00 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:00.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:00 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:01.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:01 smithi088 ceph-mon[36610]: Reconfiguring crash.smithi088 (monmap changed)... 2024-03-24T00:40:01.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:40:01.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:01.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:01 smithi088 ceph-mon[36610]: Reconfiguring daemon crash.smithi088 on smithi088 2024-03-24T00:40:01.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:01 smithi088 ceph-mon[36610]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:01.719 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:01 smithi038 ceph-mon[29271]: Reconfiguring crash.smithi088 (monmap changed)... 2024-03-24T00:40:01.719 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:40:01.719 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:01.719 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:01 smithi038 ceph-mon[29271]: Reconfiguring daemon crash.smithi088 on smithi088 2024-03-24T00:40:01.719 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:01 smithi038 ceph-mon[29271]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:03.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:03 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:03.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:03 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:03.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:03 smithi088 ceph-mon[36610]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:03.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:03 smithi088 ceph-mon[36610]: Reconfiguring mgr.smithi088.njctea (monmap changed)... 2024-03-24T00:40:03.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:03 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:40:03.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:03 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:40:03.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:03 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:03.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:03 smithi088 ceph-mon[36610]: Reconfiguring daemon mgr.smithi088.njctea on smithi088 2024-03-24T00:40:03.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:03.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:03.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:03 smithi038 ceph-mon[29271]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:03.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:03 smithi038 ceph-mon[29271]: Reconfiguring mgr.smithi088.njctea (monmap changed)... 2024-03-24T00:40:03.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:40:03.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:40:03.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:03.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:03 smithi038 ceph-mon[29271]: Reconfiguring daemon mgr.smithi088.njctea on smithi088 2024-03-24T00:40:05.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:05.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:05.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:05 smithi088 ceph-mon[36610]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:05.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:05 smithi088 ceph-mon[36610]: Reconfiguring mon.smithi088 (monmap changed)... 2024-03-24T00:40:05.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:40:05.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:40:05.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:05.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:05 smithi088 ceph-mon[36610]: Reconfiguring daemon mon.smithi088 on smithi088 2024-03-24T00:40:05.763 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:05.763 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:05.763 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:05 smithi038 ceph-mon[29271]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:05.763 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:05 smithi038 ceph-mon[29271]: Reconfiguring mon.smithi088 (monmap changed)... 2024-03-24T00:40:05.764 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:40:05.764 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:40:05.764 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:05.764 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:05 smithi038 ceph-mon[29271]: Reconfiguring daemon mon.smithi088 on smithi088 2024-03-24T00:40:07.192 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.192 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.192 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:07.192 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.192 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.192 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T00:40:07.192 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T00:40:07.192 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi038.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T00:40:07.192 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.192 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T00:40:07.193 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi038.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T00:40:07.193 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.193 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:40:07.193 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi038.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T00:40:07.193 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.193 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:40:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T00:40:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T00:40:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi038.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T00:40:07.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T00:40:07.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi038.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T00:40:07.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:40:07.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi038.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T00:40:07.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:07.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:40:08.172 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:07 smithi038 ceph-mon[29271]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi038.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T00:40:08.172 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:07 smithi038 ceph-mon[29271]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T00:40:08.172 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:07 smithi038 ceph-mon[29271]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi038.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T00:40:08.172 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:07 smithi038 ceph-mon[29271]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:40:08.172 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:07 smithi038 ceph-mon[29271]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi038.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T00:40:08.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:07 smithi088 ceph-mon[36610]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi038.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T00:40:08.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:07 smithi088 ceph-mon[36610]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T00:40:08.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:07 smithi088 ceph-mon[36610]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi038.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T00:40:08.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:07 smithi088 ceph-mon[36610]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:40:08.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:07 smithi088 ceph-mon[36610]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi038.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T00:40:09.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:09 smithi038 ceph-mon[29271]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:09.857 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:09 smithi088 ceph-mon[36610]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:10.100 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 0 on host 'smithi038' 2024-03-24T00:40:10.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.099+0000 7f816cc00700 1 -- 172.21.15.38:0/1348109183 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f81700611f0 con 0x7f815406cc50 2024-03-24T00:40:10.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.102+0000 7f8176a00700 1 -- 172.21.15.38:0/1348109183 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f815406cc50 msgr2=0x7f815406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:40:10.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.102+0000 7f8176a00700 1 --2- 172.21.15.38:0/1348109183 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f815406cc50 0x7f815406f110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f816000b5c0 tx=0x7f8160005160 comp rx=0 tx=0).stop 2024-03-24T00:40:10.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.102+0000 7f8176a00700 1 -- 172.21.15.38:0/1348109183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8170101d00 msgr2=0x7f8170198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:40:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.102+0000 7f8176a00700 1 --2- 172.21.15.38:0/1348109183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8170101d00 0x7f8170198d60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b4b0 tx=0x7f815c00b870 comp rx=0 tx=0).stop 2024-03-24T00:40:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.102+0000 7f8176a00700 1 -- 172.21.15.38:0/1348109183 shutdown_connections 2024-03-24T00:40:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.102+0000 7f8176a00700 1 --2- 172.21.15.38:0/1348109183 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f815406cc50 0x7f815406f110 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:40:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.102+0000 7f8176a00700 1 --2- 172.21.15.38:0/1348109183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8170100b00 0x7f8170198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:40:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.102+0000 7f8176a00700 1 --2- 172.21.15.38:0/1348109183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8170101d00 0x7f8170198d60 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b4b0 tx=0x7f815c00b870 comp rx=0 tx=0).stop 2024-03-24T00:40:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.102+0000 7f8176a00700 1 -- 172.21.15.38:0/1348109183 >> 172.21.15.38:0/1348109183 conn(0x7f81700fc0a0 msgr2=0x7f8170073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:40:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.103+0000 7f8176a00700 1 -- 172.21.15.38:0/1348109183 shutdown_connections 2024-03-24T00:40:10.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:10.103+0000 7f8176a00700 1 -- 172.21.15.38:0/1348109183 wait complete. 2024-03-24T00:40:10.772 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:10 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:10.772 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:10 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:10.772 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:10 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:10.773 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:10 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:10.773 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:10 smithi088 ceph-mon[36610]: from='osd.0 [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T00:40:10.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:10 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:10.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:10 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:10.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:10 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:10.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:10 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:10.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:10 smithi038 ceph-mon[29271]: from='osd.0 [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T00:40:11.401 DEBUG:teuthology.orchestra.run.smithi038:osd.0> sudo journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.0.service 2024-03-24T00:40:11.405 INFO:tasks.cephadm:Deploying osd.1 on smithi038 with /dev/vg_nvme/lv_3... 2024-03-24T00:40:11.405 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-24T00:40:11.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:11 smithi038 ceph-mon[29271]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:11.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:11 smithi038 ceph-mon[29271]: from='osd.0 [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T00:40:11.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:11 smithi038 ceph-mon[29271]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T00:40:11.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:11 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:40:11.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:11 smithi038 ceph-mon[29271]: from='osd.0 [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:40:11.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:11 smithi088 ceph-mon[36610]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:11.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:11 smithi088 ceph-mon[36610]: from='osd.0 [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T00:40:11.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:11 smithi088 ceph-mon[36610]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T00:40:11.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:11 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:40:11.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:11 smithi088 ceph-mon[36610]: from='osd.0 [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:40:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:40:12.385 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:40:12 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0[47766]: 2024-03-24T00:40:12.122+0000 7fcb25600700 -1 osd.0 0 waiting for initial osdmap 2024-03-24T00:40:12.386 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:40:12 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0[47766]: 2024-03-24T00:40:12.132+0000 7fcb1e600700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T00:40:13.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:13 smithi038 ceph-mon[29271]: from='osd.0 [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-03-24T00:40:13.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:13 smithi038 ceph-mon[29271]: osdmap e8: 1 total, 0 up, 1 in 2024-03-24T00:40:13.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:40:13.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:40:13.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:13 smithi038 ceph-mon[29271]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:13.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:13 smithi088 ceph-mon[36610]: from='osd.0 [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-03-24T00:40:13.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:13 smithi088 ceph-mon[36610]: osdmap e8: 1 total, 0 up, 1 in 2024-03-24T00:40:13.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:40:13.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:40:13.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:13 smithi088 ceph-mon[36610]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T00:40:14.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:14 smithi038 ceph-mon[29271]: purged_snaps scrub starts 2024-03-24T00:40:14.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:14 smithi038 ceph-mon[29271]: purged_snaps scrub ok 2024-03-24T00:40:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:14 smithi038 ceph-mon[29271]: osd.0 [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666] boot 2024-03-24T00:40:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:14 smithi038 ceph-mon[29271]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T00:40:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:40:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:40:14.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:14 smithi088 ceph-mon[36610]: purged_snaps scrub starts 2024-03-24T00:40:14.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:14 smithi088 ceph-mon[36610]: purged_snaps scrub ok 2024-03-24T00:40:14.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:14 smithi088 ceph-mon[36610]: osd.0 [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666] boot 2024-03-24T00:40:14.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:14 smithi088 ceph-mon[36610]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T00:40:14.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:40:14.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:40:15.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:15 smithi038 ceph-mon[29271]: osdmap e10: 1 total, 1 up, 1 in 2024-03-24T00:40:15.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:15 smithi038 ceph-mon[29271]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:15.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:15 smithi088 ceph-mon[36610]: osdmap e10: 1 total, 1 up, 1 in 2024-03-24T00:40:15.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:15 smithi088 ceph-mon[36610]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:16.318 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-24T00:40:16.318 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-24T00:40:16.319 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-03-24T00:40:16.319 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-03-24T00:40:16.319 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0571033 s, 184 MB/s 2024-03-24T00:40:16.319 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-03-24T00:40:16.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:16 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:16.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:16 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:16 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:16 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:17.138 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi038:vg_nvme/lv_3 2024-03-24T00:40:17.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:17 smithi038 ceph-mon[29271]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:17.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:17 smithi088 ceph-mon[36610]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:18.663 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:40:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:19 smithi038 ceph-mon[29271]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:19.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:19 smithi088 ceph-mon[36610]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:21.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.369+0000 7faa63c00700 1 -- 172.21.15.38:0/2153036920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c074de0 msgr2=0x7faa5c073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:40:21.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.369+0000 7faa63c00700 1 --2- 172.21.15.38:0/2153036920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c074de0 0x7faa5c073240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faa50009910 tx=0x7faa50009c20 comp rx=0 tx=0).stop 2024-03-24T00:40:21.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.369+0000 7faa63c00700 1 -- 172.21.15.38:0/2153036920 shutdown_connections 2024-03-24T00:40:21.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.369+0000 7faa63c00700 1 --2- 172.21.15.38:0/2153036920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa5c073810 0x7faa5c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:40:21.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.369+0000 7faa63c00700 1 --2- 172.21.15.38:0/2153036920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c074de0 0x7faa5c073240 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:40:21.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.369+0000 7faa63c00700 1 -- 172.21.15.38:0/2153036920 >> 172.21.15.38:0/2153036920 conn(0x7faa5c0fc0a0 msgr2=0x7faa5c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:40:21.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.369+0000 7faa63c00700 1 -- 172.21.15.38:0/2153036920 shutdown_connections 2024-03-24T00:40:21.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.369+0000 7faa63c00700 1 -- 172.21.15.38:0/2153036920 wait complete. 2024-03-24T00:40:21.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.370+0000 7faa63c00700 1 Processor -- start 2024-03-24T00:40:21.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.370+0000 7faa63c00700 1 -- start start 2024-03-24T00:40:21.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.371+0000 7faa63c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c073810 0x7faa5c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:40:21.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.371+0000 7faa63c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa5c074de0 0x7faa5c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:40:21.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.371+0000 7faa63c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c19d610 con 0x7faa5c073810 2024-03-24T00:40:21.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.371+0000 7faa63c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c19d750 con 0x7faa5c074de0 2024-03-24T00:40:21.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.371+0000 7faa61200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c073810 0x7faa5c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:40:21.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.371+0000 7faa61200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c073810 0x7faa5c19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58366/0 (socket says 172.21.15.38:58366) 2024-03-24T00:40:21.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.371+0000 7faa61200700 1 -- 172.21.15.38:0/644936911 learned_addr learned my addr 172.21.15.38:0/644936911 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:40:21.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.372+0000 7faa5be00700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa5c074de0 0x7faa5c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:40:21.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.372+0000 7faa61200700 1 -- 172.21.15.38:0/644936911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa5c074de0 msgr2=0x7faa5c19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:40:21.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.372+0000 7faa61200700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa5c074de0 0x7faa5c19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:40:21.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.372+0000 7faa61200700 1 -- 172.21.15.38:0/644936911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa500095f0 con 0x7faa5c073810 2024-03-24T00:40:21.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.372+0000 7faa5be00700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa5c074de0 0x7faa5c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:40:21.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.372+0000 7faa61200700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c073810 0x7faa5c19cab0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa5000bd70 tx=0x7faa5000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:40:21.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.372+0000 7faa59600700 1 -- 172.21.15.38:0/644936911 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa5001d070 con 0x7faa5c073810 2024-03-24T00:40:21.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.372+0000 7faa59600700 1 -- 172.21.15.38:0/644936911 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa5000f460 con 0x7faa5c073810 2024-03-24T00:40:21.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.372+0000 7faa59600700 1 -- 172.21.15.38:0/644936911 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa50021600 con 0x7faa5c073810 2024-03-24T00:40:21.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.372+0000 7faa63c00700 1 -- 172.21.15.38:0/644936911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c1a2150 con 0x7faa5c073810 2024-03-24T00:40:21.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.373+0000 7faa63c00700 1 -- 172.21.15.38:0/644936911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c1a26c0 con 0x7faa5c073810 2024-03-24T00:40:21.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.374+0000 7faa63c00700 1 -- 172.21.15.38:0/644936911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5c196c90 con 0x7faa5c073810 2024-03-24T00:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.376+0000 7faa59600700 1 -- 172.21.15.38:0/644936911 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 90786+0+0 (secure 0 0 0) 0x7faa5000fa70 con 0x7faa5c073810 2024-03-24T00:40:21.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.377+0000 7faa59600700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faa4406cca0 0x7faa4406f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:40:21.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.377+0000 7faa59600700 1 -- 172.21.15.38:0/644936911 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7faa5008c980 con 0x7faa5c073810 2024-03-24T00:40:21.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.377+0000 7faa5be00700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faa4406cca0 0x7faa4406f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:40:21.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.378+0000 7faa5be00700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faa4406cca0 0x7faa4406f160 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faa48005fd0 tx=0x7faa48005f00 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:40:21.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.382+0000 7faa59600700 1 -- 172.21.15.38:0/644936911 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faa5005b550 con 0x7faa5c073810 2024-03-24T00:40:21.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:21.625+0000 7faa63c00700 1 -- 172.21.15.38:0/644936911 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7faa5c0611f0 con 0x7faa4406cca0 2024-03-24T00:40:21.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:21 smithi038 ceph-mon[29271]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:21.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:21 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:40:21.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:21 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:40:21.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:21 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:21 smithi088 ceph-mon[36610]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:21.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:21 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:40:21.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:21 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:40:21.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:21 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:22.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:22 smithi088 ceph-mon[36610]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:40:22.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:22 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:22.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:22 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:22.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:22 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:40:22.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:22 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:22.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:22 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:22.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:22 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:40:22.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:22 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:22.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:22 smithi038 ceph-mon[29271]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:40:22.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:22 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:22.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:22 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:22.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:22 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:40:22.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:22 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:22.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:22 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:22.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:22 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:40:22.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:22 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:23.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:23 smithi088 ceph-mon[36610]: Detected new or changed devices on smithi038 2024-03-24T00:40:23.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:23 smithi088 ceph-mon[36610]: Adjusting osd_memory_target on smithi038 to 11847M 2024-03-24T00:40:23.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:23 smithi088 ceph-mon[36610]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:23.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:23 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:40:24.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:23 smithi038 ceph-mon[29271]: Detected new or changed devices on smithi038 2024-03-24T00:40:24.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:23 smithi038 ceph-mon[29271]: Adjusting osd_memory_target on smithi038 to 11847M 2024-03-24T00:40:24.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:23 smithi038 ceph-mon[29271]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:24.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:23 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:40:24.954 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:24 smithi038 ceph-mon[29271]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:25.008 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:24 smithi088 ceph-mon[36610]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:26.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:26 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:26.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:26 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:26.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:26 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1454638539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac3c0960-dd6c-44f9-b4b0-234c10f838fc"}]: dispatch 2024-03-24T00:40:26.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:26 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1454638539' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac3c0960-dd6c-44f9-b4b0-234c10f838fc"}]': finished 2024-03-24T00:40:26.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:26 smithi038 ceph-mon[29271]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T00:40:26.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:26 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:26.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:26 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:26.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:26 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:26.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:26 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1454638539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac3c0960-dd6c-44f9-b4b0-234c10f838fc"}]: dispatch 2024-03-24T00:40:26.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:26 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1454638539' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac3c0960-dd6c-44f9-b4b0-234c10f838fc"}]': finished 2024-03-24T00:40:26.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:26 smithi088 ceph-mon[36610]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T00:40:26.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:26 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:27 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3115910513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:40:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:27 smithi038 ceph-mon[29271]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:28.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:27 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3115910513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:40:28.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:27 smithi088 ceph-mon[36610]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:29 smithi038 ceph-mon[29271]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:29.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:40:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:40:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:29 smithi088 ceph-mon[36610]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:40:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:40:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:31.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:31 smithi038 ceph-mon[29271]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:31.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:31 smithi088 ceph-mon[36610]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:33.089 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:32 smithi038 ceph-mon[29271]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:33.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:32 smithi088 ceph-mon[36610]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:35.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:35 smithi088 ceph-mon[36610]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:36.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:35 smithi038 ceph-mon[29271]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:36.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:36 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T00:40:36.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:36 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:37.099 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:36 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T00:40:37.099 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:36 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:40:38.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:37 smithi038 ceph-mon[29271]: Deploying daemon osd.1 on smithi038 2024-03-24T00:40:38.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:37 smithi038 ceph-mon[29271]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:38.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:37 smithi088 ceph-mon[36610]: Deploying daemon osd.1 on smithi038 2024-03-24T00:40:38.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:37 smithi088 ceph-mon[36610]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:39.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:39 smithi038 ceph-mon[29271]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:39.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:39 smithi088 ceph-mon[36610]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:41.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:41 smithi038 ceph-mon[29271]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:41.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:41 smithi088 ceph-mon[36610]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:43.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:43 smithi038 ceph-mon[29271]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:43.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:43 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:40:43.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:43 smithi088 ceph-mon[36610]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:43.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:43 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:40:45.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:44 smithi038 ceph-mon[29271]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:45.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:44 smithi088 ceph-mon[36610]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:46.039 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:45 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:46.039 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:45 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:40:46.039 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:45 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:46.219 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:45 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:46.219 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:45 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:40:46.219 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:45 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:47.277 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:46 smithi088 ceph-mon[36610]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:47.326 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:46 smithi038 ceph-mon[29271]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:49.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:48 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:49.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:48 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:49.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:48 smithi038 ceph-mon[29271]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:49.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:48 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:49.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:48 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:49.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:48 smithi088 ceph-mon[36610]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:49.833 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 1 on host 'smithi038' 2024-03-24T00:40:49.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.832+0000 7faa59600700 1 -- 172.21.15.38:0/644936911 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faa5c0611f0 con 0x7faa4406cca0 2024-03-24T00:40:49.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.834+0000 7faa63c00700 1 -- 172.21.15.38:0/644936911 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faa4406cca0 msgr2=0x7faa4406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:40:49.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.834+0000 7faa63c00700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faa4406cca0 0x7faa4406f160 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faa48005fd0 tx=0x7faa48005f00 comp rx=0 tx=0).stop 2024-03-24T00:40:49.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.835+0000 7faa63c00700 1 -- 172.21.15.38:0/644936911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c073810 msgr2=0x7faa5c19cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:40:49.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.835+0000 7faa63c00700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c073810 0x7faa5c19cab0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa5000bd70 tx=0x7faa5000be50 comp rx=0 tx=0).stop 2024-03-24T00:40:49.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.835+0000 7faa63c00700 1 -- 172.21.15.38:0/644936911 shutdown_connections 2024-03-24T00:40:49.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.835+0000 7faa63c00700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7faa4406cca0 0x7faa4406f160 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:40:49.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.835+0000 7faa63c00700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c073810 0x7faa5c19cab0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:40:49.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.835+0000 7faa63c00700 1 --2- 172.21.15.38:0/644936911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa5c074de0 0x7faa5c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:40:49.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.835+0000 7faa63c00700 1 -- 172.21.15.38:0/644936911 >> 172.21.15.38:0/644936911 conn(0x7faa5c0fc0a0 msgr2=0x7faa5c1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:40:49.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.835+0000 7faa63c00700 1 -- 172.21.15.38:0/644936911 shutdown_connections 2024-03-24T00:40:49.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:40:49.835+0000 7faa63c00700 1 -- 172.21.15.38:0/644936911 wait complete. 2024-03-24T00:40:49.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:49 smithi038 ceph-mon[29271]: from='osd.1 [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T00:40:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:49 smithi088 ceph-mon[36610]: from='osd.1 [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T00:40:50.559 DEBUG:teuthology.orchestra.run.smithi038:osd.1> sudo journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.1.service 2024-03-24T00:40:50.567 INFO:tasks.cephadm:Deploying osd.2 on smithi038 with /dev/vg_nvme/lv_2... 2024-03-24T00:40:50.567 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-24T00:40:51.096 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:50 smithi038 ceph-mon[29271]: from='osd.1 [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T00:40:51.096 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:50 smithi038 ceph-mon[29271]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T00:40:51.096 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:50 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:51.096 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:50 smithi038 ceph-mon[29271]: from='osd.1 [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:40:51.096 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:50 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:51.096 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:50 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:51.096 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:50 smithi038 ceph-mon[29271]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:51.097 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:40:50 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1[55392]: 2024-03-24T00:40:50.829+0000 7f3e3cc00700 -1 osd.1 0 waiting for initial osdmap 2024-03-24T00:40:51.097 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:40:50 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1[55392]: 2024-03-24T00:40:50.838+0000 7f3e35c00700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T00:40:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:50 smithi088 ceph-mon[36610]: from='osd.1 [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T00:40:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:50 smithi088 ceph-mon[36610]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T00:40:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:50 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:50 smithi088 ceph-mon[36610]: from='osd.1 [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:40:51.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:50 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:51.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:50 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:51.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:50 smithi088 ceph-mon[36610]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:51.447 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:40:51.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:51 smithi038 ceph-mon[29271]: from='osd.1 [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-03-24T00:40:51.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:51 smithi038 ceph-mon[29271]: osdmap e13: 2 total, 1 up, 2 in 2024-03-24T00:40:51.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:51 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:51.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:51 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:51 smithi088 ceph-mon[36610]: from='osd.1 [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-03-24T00:40:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:51 smithi088 ceph-mon[36610]: osdmap e13: 2 total, 1 up, 2 in 2024-03-24T00:40:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:51 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:51 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:53.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:52 smithi038 ceph-mon[29271]: purged_snaps scrub starts 2024-03-24T00:40:53.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:52 smithi038 ceph-mon[29271]: purged_snaps scrub ok 2024-03-24T00:40:53.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:53.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:52 smithi038 ceph-mon[29271]: osd.1 [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394] boot 2024-03-24T00:40:53.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:52 smithi038 ceph-mon[29271]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T00:40:53.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:53.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:52 smithi038 ceph-mon[29271]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:53.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:52 smithi088 ceph-mon[36610]: purged_snaps scrub starts 2024-03-24T00:40:53.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:52 smithi088 ceph-mon[36610]: purged_snaps scrub ok 2024-03-24T00:40:53.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:53.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:52 smithi088 ceph-mon[36610]: osd.1 [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394] boot 2024-03-24T00:40:53.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:52 smithi088 ceph-mon[36610]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T00:40:53.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:40:53.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:52 smithi088 ceph-mon[36610]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T00:40:53.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:53 smithi038 ceph-mon[29271]: osdmap e15: 2 total, 2 up, 2 in 2024-03-24T00:40:54.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:53 smithi088 ceph-mon[36610]: osdmap e15: 2 total, 2 up, 2 in 2024-03-24T00:40:55.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:54 smithi038 ceph-mon[29271]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:40:55.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:54 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:55.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:54 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:55.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:54 smithi088 ceph-mon[36610]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:40:55.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:54 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:55.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:54 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:40:55.360 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-24T00:40:55.360 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-24T00:40:55.361 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-03-24T00:40:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-03-24T00:40:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0324328 s, 323 MB/s 2024-03-24T00:40:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-03-24T00:40:56.384 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi038:vg_nvme/lv_2 2024-03-24T00:40:57.677 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:40:57.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:57 smithi038 ceph-mon[29271]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:40:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:57 smithi088 ceph-mon[36610]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:40:59.746 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:59 smithi038 ceph-mon[29271]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:40:59.747 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:40:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:40:59.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:59 smithi088 ceph-mon[36610]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:40:59.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:40:59 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:41:00.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.215+0000 7fbe26000700 1 -- 172.21.15.38:0/3829364787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe20103f00 msgr2=0x7fbe20104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:41:00.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.215+0000 7fbe26000700 1 --2- 172.21.15.38:0/3829364787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe20103f00 0x7fbe20104360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009910 tx=0x7fbe08009c20 comp rx=0 tx=0).stop 2024-03-24T00:41:00.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.215+0000 7fbe26000700 1 -- 172.21.15.38:0/3829364787 shutdown_connections 2024-03-24T00:41:00.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.215+0000 7fbe26000700 1 --2- 172.21.15.38:0/3829364787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe20103f00 0x7fbe20104360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:41:00.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.215+0000 7fbe26000700 1 --2- 172.21.15.38:0/3829364787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe20102d00 0x7fbe20103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:41:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.215+0000 7fbe26000700 1 -- 172.21.15.38:0/3829364787 >> 172.21.15.38:0/3829364787 conn(0x7fbe200fe2c0 msgr2=0x7fbe201006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:41:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.215+0000 7fbe26000700 1 -- 172.21.15.38:0/3829364787 shutdown_connections 2024-03-24T00:41:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.215+0000 7fbe26000700 1 -- 172.21.15.38:0/3829364787 wait complete. 2024-03-24T00:41:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.216+0000 7fbe26000700 1 Processor -- start 2024-03-24T00:41:00.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.216+0000 7fbe26000700 1 -- start start 2024-03-24T00:41:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe26000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe20102d00 0x7fbe201987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:41:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe26000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe20103f00 0x7fbe20198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:41:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe26000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe20199350 con 0x7fbe20103f00 2024-03-24T00:41:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe26000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe20199490 con 0x7fbe20102d00 2024-03-24T00:41:00.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe17e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe20103f00 0x7fbe20198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:41:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe17e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe20103f00 0x7fbe20198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56084/0 (socket says 172.21.15.38:56084) 2024-03-24T00:41:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe17e00700 1 -- 172.21.15.38:0/2338940453 learned_addr learned my addr 172.21.15.38:0/2338940453 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:41:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe1f400700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe20102d00 0x7fbe201987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:41:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe17e00700 1 -- 172.21.15.38:0/2338940453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe20102d00 msgr2=0x7fbe201987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:41:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe17e00700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe20102d00 0x7fbe201987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:41:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe17e00700 1 -- 172.21.15.38:0/2338940453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe080095f0 con 0x7fbe20103f00 2024-03-24T00:41:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.217+0000 7fbe1f400700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe20102d00 0x7fbe201987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:41:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.218+0000 7fbe17e00700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe20103f00 0x7fbe20198d30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbe080092e0 tx=0x7fbe0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:41:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.218+0000 7fbe1cc00700 1 -- 172.21.15.38:0/2338940453 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe0801d070 con 0x7fbe20103f00 2024-03-24T00:41:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.218+0000 7fbe1cc00700 1 -- 172.21.15.38:0/2338940453 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe0800f460 con 0x7fbe20103f00 2024-03-24T00:41:00.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.218+0000 7fbe1cc00700 1 -- 172.21.15.38:0/2338940453 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe08021640 con 0x7fbe20103f00 2024-03-24T00:41:00.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.218+0000 7fbe26000700 1 -- 172.21.15.38:0/2338940453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2019ddd0 con 0x7fbe20103f00 2024-03-24T00:41:00.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.218+0000 7fbe26000700 1 -- 172.21.15.38:0/2338940453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2019e340 con 0x7fbe20103f00 2024-03-24T00:41:00.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.219+0000 7fbe26000700 1 -- 172.21.15.38:0/2338940453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe20192920 con 0x7fbe20103f00 2024-03-24T00:41:00.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.221+0000 7fbe1cc00700 1 -- 172.21.15.38:0/2338940453 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 90786+0+0 (secure 0 0 0) 0x7fbe0800f5d0 con 0x7fbe20103f00 2024-03-24T00:41:00.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.222+0000 7fbe1cc00700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbe0406cc60 0x7fbe0406f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:41:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.222+0000 7fbe1cc00700 1 -- 172.21.15.38:0/2338940453 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7fbe0808cbe0 con 0x7fbe20103f00 2024-03-24T00:41:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.222+0000 7fbe1f400700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbe0406cc60 0x7fbe0406f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:41:00.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.223+0000 7fbe1f400700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbe0406cc60 0x7fbe0406f120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbe20103d60 tx=0x7fbe10008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:41:00.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.226+0000 7fbe1cc00700 1 -- 172.21.15.38:0/2338940453 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbe0805b830 con 0x7fbe20103f00 2024-03-24T00:41:00.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:00.460+0000 7fbe26000700 1 -- 172.21.15.38:0/2338940453 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe200611f0 con 0x7fbe0406cc60 2024-03-24T00:41:00.767 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:00 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:41:00.768 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:00 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:41:00.768 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:00 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:00.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:00 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:41:00.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:00 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:41:00.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:00 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:01.846 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:01 smithi038 ceph-mon[29271]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:41:01.846 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:01 smithi038 ceph-mon[29271]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:01.846 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:01.846 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:01.846 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:41:01.846 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:41:01.847 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:01.847 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:01.847 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:41:01.847 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:01.847 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:41:01.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:01 smithi088 ceph-mon[36610]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:41:01.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:01 smithi088 ceph-mon[36610]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:01.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:01.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:01.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:41:01.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:41:01.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:01.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:01.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:41:01.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:01.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:41:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:02 smithi088 ceph-mon[36610]: Detected new or changed devices on smithi038 2024-03-24T00:41:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:02 smithi088 ceph-mon[36610]: Adjusting osd_memory_target on smithi038 to 5923M 2024-03-24T00:41:02.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:02 smithi038 ceph-mon[29271]: Detected new or changed devices on smithi038 2024-03-24T00:41:02.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:02 smithi038 ceph-mon[29271]: Adjusting osd_memory_target on smithi038 to 5923M 2024-03-24T00:41:03.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:03 smithi038 ceph-mon[29271]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:03.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:03 smithi088 ceph-mon[36610]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:05.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:05.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:05.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:05 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1988512578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "921d9dd3-2471-4b13-b6bd-01e5b6c78648"}]: dispatch 2024-03-24T00:41:05.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:05 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1988512578' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "921d9dd3-2471-4b13-b6bd-01e5b6c78648"}]': finished 2024-03-24T00:41:05.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:05 smithi038 ceph-mon[29271]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T00:41:05.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:41:05.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:05 smithi038 ceph-mon[29271]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:05.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:05.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:05.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:05 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1988512578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "921d9dd3-2471-4b13-b6bd-01e5b6c78648"}]: dispatch 2024-03-24T00:41:05.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:05 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1988512578' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "921d9dd3-2471-4b13-b6bd-01e5b6c78648"}]': finished 2024-03-24T00:41:05.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:05 smithi088 ceph-mon[36610]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T00:41:05.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:41:05.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:05 smithi088 ceph-mon[36610]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:06.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:06 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2805124451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:41:06.621 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:06 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2805124451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:41:07.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:07 smithi038 ceph-mon[29271]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:07.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:07 smithi088 ceph-mon[36610]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:09.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:09.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:09.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:09.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:41:09.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:09.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:09 smithi088 ceph-mon[36610]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:09.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:09.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:09.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:09.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:41:09.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:09.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:09 smithi038 ceph-mon[29271]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:11.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:11 smithi088 ceph-mon[36610]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:11.944 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:11 smithi038 ceph-mon[29271]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:12.967 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:12 smithi038 ceph-mon[29271]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:13.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:12 smithi088 ceph-mon[36610]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:41:14.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:41:15.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:15 smithi038 ceph-mon[29271]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:15.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:15 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T00:41:15.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:15 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:15.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:15 smithi088 ceph-mon[36610]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:15.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:15 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T00:41:15.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:15 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:16.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:16 smithi038 ceph-mon[29271]: Deploying daemon osd.2 on smithi038 2024-03-24T00:41:16.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:16 smithi088 ceph-mon[36610]: Deploying daemon osd.2 on smithi038 2024-03-24T00:41:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:17 smithi038 ceph-mon[29271]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:17.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:17 smithi088 ceph-mon[36610]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:19.856 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:19 smithi038 ceph-mon[29271]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:19.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:19 smithi088 ceph-mon[36610]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:21 smithi088 ceph-mon[36610]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:22.019 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:21 smithi038 ceph-mon[29271]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:23.068 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:22 smithi038 ceph-mon[29271]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:23.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:22 smithi088 ceph-mon[36610]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:24.487 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:24 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:24.487 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:24 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:41:24.487 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:24 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:24 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:24 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:41:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:24 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:25.666 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:25 smithi038 ceph-mon[29271]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:25.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:25 smithi088 ceph-mon[36610]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:28.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:27 smithi038 ceph-mon[29271]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:28.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:27 smithi038 ceph-mon[29271]: from='osd.2 [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T00:41:28.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:27 smithi088 ceph-mon[36610]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:28.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:27 smithi088 ceph-mon[36610]: from='osd.2 [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T00:41:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.315+0000 7fbe1cc00700 1 -- 172.21.15.38:0/2338940453 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbe200611f0 con 0x7fbe0406cc60 2024-03-24T00:41:28.316 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 2 on host 'smithi038' 2024-03-24T00:41:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.318+0000 7fbe26000700 1 -- 172.21.15.38:0/2338940453 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbe0406cc60 msgr2=0x7fbe0406f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:41:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.318+0000 7fbe26000700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbe0406cc60 0x7fbe0406f120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbe20103d60 tx=0x7fbe10008040 comp rx=0 tx=0).stop 2024-03-24T00:41:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.318+0000 7fbe26000700 1 -- 172.21.15.38:0/2338940453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe20103f00 msgr2=0x7fbe20198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:41:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.318+0000 7fbe26000700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe20103f00 0x7fbe20198d30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbe080092e0 tx=0x7fbe0800be20 comp rx=0 tx=0).stop 2024-03-24T00:41:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.318+0000 7fbe26000700 1 -- 172.21.15.38:0/2338940453 shutdown_connections 2024-03-24T00:41:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.318+0000 7fbe26000700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbe0406cc60 0x7fbe0406f120 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:41:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.318+0000 7fbe26000700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe20102d00 0x7fbe201987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:41:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.318+0000 7fbe26000700 1 --2- 172.21.15.38:0/2338940453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe20103f00 0x7fbe20198d30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:41:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.318+0000 7fbe26000700 1 -- 172.21.15.38:0/2338940453 >> 172.21.15.38:0/2338940453 conn(0x7fbe200fe2c0 msgr2=0x7fbe201075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:41:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.318+0000 7fbe26000700 1 -- 172.21.15.38:0/2338940453 shutdown_connections 2024-03-24T00:41:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:41:28.318+0000 7fbe26000700 1 -- 172.21.15.38:0/2338940453 wait complete. 2024-03-24T00:41:28.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:28 smithi038 ceph-mon[29271]: from='osd.2 [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T00:41:28.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:28 smithi038 ceph-mon[29271]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T00:41:28.988 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:41:28.988 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:28 smithi038 ceph-mon[29271]: from='osd.2 [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:41:28.988 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:28.988 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:28.988 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:28 smithi038 ceph-mon[29271]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:28.988 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:41:29.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:28 smithi088 ceph-mon[36610]: from='osd.2 [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T00:41:29.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:28 smithi088 ceph-mon[36610]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T00:41:29.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:41:29.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:28 smithi088 ceph-mon[36610]: from='osd.2 [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:41:29.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:29.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:29.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:28 smithi088 ceph-mon[36610]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:29.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:41:29.806 DEBUG:teuthology.orchestra.run.smithi038:osd.2> sudo journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.2.service 2024-03-24T00:41:29.810 INFO:tasks.cephadm:Deploying osd.3 on smithi088 with /dev/vg_nvme/lv_4... 2024-03-24T00:41:29.810 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-24T00:41:30.054 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:29 smithi038 ceph-mon[29271]: from='osd.2 [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-03-24T00:41:30.054 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:29 smithi038 ceph-mon[29271]: osdmap e18: 3 total, 2 up, 3 in 2024-03-24T00:41:30.054 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:41:30.054 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:41:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:29 smithi088 ceph-mon[36610]: from='osd.2 [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-03-24T00:41:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:29 smithi088 ceph-mon[36610]: osdmap e18: 3 total, 2 up, 3 in 2024-03-24T00:41:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:41:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:41:30.214 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi088/config 2024-03-24T00:41:30.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:30 smithi088 ceph-mon[36610]: purged_snaps scrub starts 2024-03-24T00:41:30.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:30 smithi088 ceph-mon[36610]: purged_snaps scrub ok 2024-03-24T00:41:30.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:30 smithi088 ceph-mon[36610]: osd.2 [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158] boot 2024-03-24T00:41:30.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:30 smithi088 ceph-mon[36610]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T00:41:30.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:30 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:41:30.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:30 smithi088 ceph-mon[36610]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:30.994 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:30 smithi038 ceph-mon[29271]: purged_snaps scrub starts 2024-03-24T00:41:30.994 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:30 smithi038 ceph-mon[29271]: purged_snaps scrub ok 2024-03-24T00:41:30.994 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:30 smithi038 ceph-mon[29271]: osd.2 [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158] boot 2024-03-24T00:41:30.995 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:30 smithi038 ceph-mon[29271]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T00:41:30.995 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:30 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:41:30.995 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:30 smithi038 ceph-mon[29271]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T00:41:31.976 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:31 smithi088 ceph-mon[36610]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T00:41:32.016 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:31 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-24T00:41:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:31 smithi038 ceph-mon[29271]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T00:41:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:31 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-24T00:41:32.950 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-24T00:41:32.951 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-24T00:41:32.951 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-03-24T00:41:32.951 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-03-24T00:41:32.951 INFO:teuthology.orchestra.run.smithi088.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0326653 s, 321 MB/s 2024-03-24T00:41:32.951 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-03-24T00:41:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:32 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-24T00:41:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:32 smithi038 ceph-mon[29271]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T00:41:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:32 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T00:41:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:32 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:32 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:32 smithi038 ceph-mon[29271]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:33.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:32 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-24T00:41:33.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:32 smithi088 ceph-mon[36610]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T00:41:33.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:32 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T00:41:33.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:32 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:33.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:32 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:33.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:32 smithi088 ceph-mon[36610]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:33.764 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi088:vg_nvme/lv_4 2024-03-24T00:41:33.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:33 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T00:41:33.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:33 smithi088 ceph-mon[36610]: osdmap e22: 3 total, 3 up, 3 in 2024-03-24T00:41:33.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:33 smithi088 ceph-mon[36610]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T00:41:34.031 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:41:33 smithi038 sudo[64882]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T00:41:34.031 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:41:33 smithi038 sudo[64882]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T00:41:34.031 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:41:33 smithi038 sudo[64882]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T00:41:34.031 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:41:33 smithi038 sudo[64882]: pam_unix(sudo:session): session closed for user root 2024-03-24T00:41:34.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:33 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T00:41:34.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:33 smithi038 ceph-mon[29271]: osdmap e22: 3 total, 3 up, 3 in 2024-03-24T00:41:34.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:33 smithi038 ceph-mon[29271]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T00:41:34.139 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi088/config 2024-03-24T00:41:34.729 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:41:34 smithi038 sudo[64901]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T00:41:34.729 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:41:34 smithi038 sudo[64901]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T00:41:34.729 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:41:34 smithi038 sudo[64901]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T00:41:34.730 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:41:34 smithi038 sudo[64901]: pam_unix(sudo:session): session closed for user root 2024-03-24T00:41:35.010 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:34 smithi038 ceph-mon[29271]: osdmap e23: 3 total, 3 up, 3 in 2024-03-24T00:41:35.010 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:34 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:35.010 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:34 smithi038 ceph-mon[29271]: pgmap v104: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:35.010 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:41:34 smithi038 sudo[64908]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T00:41:35.010 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:41:34 smithi038 sudo[64908]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T00:41:35.010 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:41:34 smithi038 sudo[64908]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T00:41:35.011 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:41:35 smithi038 sudo[64908]: pam_unix(sudo:session): session closed for user root 2024-03-24T00:41:35.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:34 smithi088 ceph-mon[36610]: osdmap e23: 3 total, 3 up, 3 in 2024-03-24T00:41:35.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:34 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:35.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:34 smithi088 ceph-mon[36610]: pgmap v104: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:35.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:41:35 smithi038 sudo[64933]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T00:41:35.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:41:35 smithi038 sudo[64933]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T00:41:35.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:41:35 smithi038 sudo[64933]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T00:41:35.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:41:35 smithi038 sudo[64933]: pam_unix(sudo:session): session closed for user root 2024-03-24T00:41:35.970 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:35 smithi088 ceph-mon[36610]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T00:41:35.970 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:35 smithi088 ceph-mon[36610]: Cluster is now healthy 2024-03-24T00:41:35.970 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:35 smithi088 ceph-mon[36610]: mgrmap e19: smithi038.huhioz(active, since 3m), standbys: smithi088.njctea 2024-03-24T00:41:36.173 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:35 smithi038 ceph-mon[29271]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T00:41:36.173 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:35 smithi038 ceph-mon[29271]: Cluster is now healthy 2024-03-24T00:41:36.173 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:35 smithi038 ceph-mon[29271]: mgrmap e19: smithi038.huhioz(active, since 3m), standbys: smithi088.njctea 2024-03-24T00:41:36.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.236+0000 7fc710800700 1 -- 172.21.15.88:0/1523574699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc708101cd0 msgr2=0x7fc708102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:41:36.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.236+0000 7fc710800700 1 --2- 172.21.15.88:0/1523574699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc708101cd0 0x7fc708102130 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009910 tx=0x7fc6f8009c20 comp rx=0 tx=0).stop 2024-03-24T00:41:36.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.236+0000 7fc710800700 1 -- 172.21.15.88:0/1523574699 shutdown_connections 2024-03-24T00:41:36.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.236+0000 7fc710800700 1 --2- 172.21.15.88:0/1523574699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc708101cd0 0x7fc708102130 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:41:36.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.236+0000 7fc710800700 1 --2- 172.21.15.88:0/1523574699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc708100ad0 0x7fc708100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:41:36.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.236+0000 7fc710800700 1 -- 172.21.15.88:0/1523574699 >> 172.21.15.88:0/1523574699 conn(0x7fc7080fc090 msgr2=0x7fc7080fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:41:36.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.236+0000 7fc710800700 1 -- 172.21.15.88:0/1523574699 shutdown_connections 2024-03-24T00:41:36.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.237+0000 7fc710800700 1 -- 172.21.15.88:0/1523574699 wait complete. 2024-03-24T00:41:36.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.237+0000 7fc710800700 1 Processor -- start 2024-03-24T00:41:36.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.238+0000 7fc710800700 1 -- start start 2024-03-24T00:41:36.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.238+0000 7fc710800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc708100ad0 0x7fc708074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:41:36.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.238+0000 7fc710800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc708101cd0 0x7fc7080731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:41:36.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.238+0000 7fc710800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7080736e0 con 0x7fc708101cd0 2024-03-24T00:41:36.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.238+0000 7fc710800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc708073820 con 0x7fc708100ad0 2024-03-24T00:41:36.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.238+0000 7fc70de00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc708100ad0 0x7fc708074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:41:36.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.238+0000 7fc70de00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc708100ad0 0x7fc708074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35846/0 (socket says 172.21.15.88:35846) 2024-03-24T00:41:36.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.238+0000 7fc70de00700 1 -- 172.21.15.88:0/3849286785 learned_addr learned my addr 172.21.15.88:0/3849286785 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:41:36.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.238+0000 7fc70d400700 1 --2- 172.21.15.88:0/3849286785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc708101cd0 0x7fc7080731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:41:36.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.239+0000 7fc70de00700 1 -- 172.21.15.88:0/3849286785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc708101cd0 msgr2=0x7fc7080731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:41:36.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.239+0000 7fc70de00700 1 --2- 172.21.15.88:0/3849286785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc708101cd0 0x7fc7080731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:41:36.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.239+0000 7fc70de00700 1 -- 172.21.15.88:0/3849286785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f80095f0 con 0x7fc708100ad0 2024-03-24T00:41:36.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.239+0000 7fc70de00700 1 --2- 172.21.15.88:0/3849286785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc708100ad0 0x7fc708074b00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00d900 tx=0x7fc6fc00dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:41:36.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.239+0000 7fc702a00700 1 -- 172.21.15.88:0/3849286785 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6fc0041f0 con 0x7fc708100ad0 2024-03-24T00:41:36.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.239+0000 7fc702a00700 1 -- 172.21.15.88:0/3849286785 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6fc004350 con 0x7fc708100ad0 2024-03-24T00:41:36.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.239+0000 7fc710800700 1 -- 172.21.15.88:0/3849286785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc708073b00 con 0x7fc708100ad0 2024-03-24T00:41:36.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.239+0000 7fc702a00700 1 -- 172.21.15.88:0/3849286785 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6fc003cf0 con 0x7fc708100ad0 2024-03-24T00:41:36.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.239+0000 7fc710800700 1 -- 172.21.15.88:0/3849286785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70819e540 con 0x7fc708100ad0 2024-03-24T00:41:36.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.241+0000 7fc710800700 1 -- 172.21.15.88:0/3849286785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc708190650 con 0x7fc708100ad0 2024-03-24T00:41:36.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.242+0000 7fc702a00700 1 -- 172.21.15.88:0/3849286785 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc6fc009730 con 0x7fc708100ad0 2024-03-24T00:41:36.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.242+0000 7fc702a00700 1 --2- 172.21.15.88:0/3849286785 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc6f006ccf0 0x7fc6f006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:41:36.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.243+0000 7fc702a00700 1 -- 172.21.15.88:0/3849286785 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3232+0+0 (secure 0 0 0) 0x7fc6fc08b7c0 con 0x7fc708100ad0 2024-03-24T00:41:36.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.243+0000 7fc70d400700 1 --2- 172.21.15.88:0/3849286785 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc6f006ccf0 0x7fc6f006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:41:36.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.244+0000 7fc70d400700 1 --2- 172.21.15.88:0/3849286785 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc6f006ccf0 0x7fc6f006f1b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8006010 tx=0x7fc6f8005180 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:41:36.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.248+0000 7fc702a00700 1 -- 172.21.15.88:0/3849286785 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc6fc059ee0 con 0x7fc708100ad0 2024-03-24T00:41:36.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:41:36.474+0000 7fc710800700 1 -- 172.21.15.88:0/3849286785 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7080611f0 con 0x7fc6f006ccf0 2024-03-24T00:41:36.610 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:41:36 smithi038 sudo[64946]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T00:41:36.610 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:41:36 smithi038 sudo[64946]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T00:41:36.610 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:41:36 smithi038 sudo[64946]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T00:41:36.610 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:41:36 smithi038 sudo[64946]: pam_unix(sudo:session): session closed for user root 2024-03-24T00:41:37.109 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:41:36 smithi038 sudo[64973]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T00:41:37.109 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:41:36 smithi038 sudo[64973]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T00:41:37.109 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:41:36 smithi038 sudo[64973]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T00:41:37.109 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:41:36 smithi038 sudo[64973]: pam_unix(sudo:session): session closed for user root 2024-03-24T00:41:37.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 sudo[67236]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T00:41:37.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 sudo[67236]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T00:41:37.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 sudo[67236]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T00:41:37.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 sudo[67236]: pam_unix(sudo:session): session closed for user root 2024-03-24T00:41:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 sudo[67253]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T00:41:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 sudo[67253]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T00:41:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 sudo[67253]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T00:41:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 sudo[67253]: pam_unix(sudo:session): session closed for user root 2024-03-24T00:41:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 ceph-mon[29271]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:41:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:41:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:41:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:38.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 ceph-mon[29271]: pgmap v105: 1 pgs: 1 creating+activating; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:38.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:37 smithi038 ceph-mon[29271]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T00:41:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:38 smithi088 sudo[40385]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T00:41:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:38 smithi088 sudo[40385]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T00:41:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:38 smithi088 sudo[40385]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T00:41:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:38 smithi088 sudo[40385]: pam_unix(sudo:session): session closed for user root 2024-03-24T00:41:39.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:39 smithi038 ceph-mon[29271]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T00:41:39.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:41:39.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:41:39.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:41:39.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:41:39.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:39 smithi038 ceph-mon[29271]: Detected new or changed devices on smithi038 2024-03-24T00:41:39.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:39 smithi038 ceph-mon[29271]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:39.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 sudo[40414]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T00:41:39.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 sudo[40414]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 sudo[40414]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 sudo[40414]: pam_unix(sudo:session): session closed for user root 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: pgmap v105: 1 pgs: 1 creating+activating; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: Detected new or changed devices on smithi038 2024-03-24T00:41:39.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:39 smithi088 ceph-mon[36610]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T00:41:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T00:41:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: Adjusting osd_memory_target on smithi038 to 3949M 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/4153664838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e93813a-0036-4cea-8396-41b8b759c257"}]: dispatch 2024-03-24T00:41:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e93813a-0036-4cea-8396-41b8b759c257"}]: dispatch 2024-03-24T00:41:40.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e93813a-0036-4cea-8396-41b8b759c257"}]': finished 2024-03-24T00:41:40.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T00:41:40.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:41:40.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T00:41:40.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T00:41:40.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:40.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:40.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:41:40.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:41:40.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:41:40.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: Adjusting osd_memory_target on smithi038 to 3949M 2024-03-24T00:41:40.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:40.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:40.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:41:40.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:40.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:40.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:41:40.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/4153664838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e93813a-0036-4cea-8396-41b8b759c257"}]: dispatch 2024-03-24T00:41:40.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e93813a-0036-4cea-8396-41b8b759c257"}]: dispatch 2024-03-24T00:41:40.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e93813a-0036-4cea-8396-41b8b759c257"}]': finished 2024-03-24T00:41:40.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T00:41:40.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:41:41.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:41 smithi088 ceph-mon[36610]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:41.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:41 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/3242843055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:41:41.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:41 smithi038 ceph-mon[29271]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:41.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:41 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/3242843055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:41:43.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:43 smithi038 ceph-mon[29271]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:43.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:43 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:41:44.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:43 smithi088 ceph-mon[36610]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:44.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:43 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:41:44.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:44 smithi038 ceph-mon[29271]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-24T00:41:45.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:44 smithi088 ceph-mon[36610]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-24T00:41:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:46 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:46 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:46 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:46 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:41:46.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:46 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:46.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:46 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:46.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:46 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:46.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:46 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:46.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:46 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:41:46.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:46 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:47 smithi088 ceph-mon[36610]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-24T00:41:47.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:47 smithi038 ceph-mon[29271]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-24T00:41:49.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:49 smithi038 ceph-mon[29271]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-24T00:41:49.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:49 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T00:41:49.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:49 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:49.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:49 smithi088 ceph-mon[36610]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-24T00:41:49.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:49 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T00:41:49.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:49 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:41:50.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:50 smithi088 ceph-mon[36610]: Deploying daemon osd.3 on smithi088 2024-03-24T00:41:50.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:50 smithi038 ceph-mon[29271]: Deploying daemon osd.3 on smithi088 2024-03-24T00:41:51.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:51 smithi038 ceph-mon[29271]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 74 KiB/s wr, 5 op/s 2024-03-24T00:41:51.885 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:51 smithi088 ceph-mon[36610]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 74 KiB/s wr, 5 op/s 2024-03-24T00:41:53.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:53 smithi038 ceph-mon[29271]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-24T00:41:54.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:53 smithi088 ceph-mon[36610]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-24T00:41:55.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:54 smithi038 ceph-mon[29271]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-24T00:41:55.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:54 smithi088 ceph-mon[36610]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-24T00:41:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:57 smithi088 ceph-mon[36610]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:57 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:57 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:57 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:41:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:57 smithi038 ceph-mon[29271]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:41:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:41:59.707 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:59 smithi088 ceph-mon[36610]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:59.707 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:41:59 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:41:59.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:59 smithi038 ceph-mon[29271]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:41:59.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:41:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:42:01.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.926+0000 7fc702a00700 1 -- 172.21.15.88:0/3849286785 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc7080611f0 con 0x7fc6f006ccf0 2024-03-24T00:42:01.929 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 3 on host 'smithi088' 2024-03-24T00:42:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.930+0000 7fc710800700 1 -- 172.21.15.88:0/3849286785 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc6f006ccf0 msgr2=0x7fc6f006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:42:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.930+0000 7fc710800700 1 --2- 172.21.15.88:0/3849286785 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc6f006ccf0 0x7fc6f006f1b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8006010 tx=0x7fc6f8005180 comp rx=0 tx=0).stop 2024-03-24T00:42:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.930+0000 7fc710800700 1 -- 172.21.15.88:0/3849286785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc708100ad0 msgr2=0x7fc708074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:42:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.930+0000 7fc710800700 1 --2- 172.21.15.88:0/3849286785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc708100ad0 0x7fc708074b00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00d900 tx=0x7fc6fc00dc10 comp rx=0 tx=0).stop 2024-03-24T00:42:01.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:01 smithi088 ceph-mon[36610]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:42:01.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:01 smithi088 ceph-mon[36610]: from='osd.3 [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T00:42:01.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:01 smithi088 ceph-mon[36610]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T00:42:01.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.930+0000 7fc710800700 1 -- 172.21.15.88:0/3849286785 shutdown_connections 2024-03-24T00:42:01.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.930+0000 7fc710800700 1 --2- 172.21.15.88:0/3849286785 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc6f006ccf0 0x7fc6f006f1b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:01.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.930+0000 7fc710800700 1 --2- 172.21.15.88:0/3849286785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc708100ad0 0x7fc708074b00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:01.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.930+0000 7fc710800700 1 --2- 172.21.15.88:0/3849286785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc708101cd0 0x7fc7080731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:01.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.930+0000 7fc710800700 1 -- 172.21.15.88:0/3849286785 >> 172.21.15.88:0/3849286785 conn(0x7fc7080fc090 msgr2=0x7fc708105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:42:01.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.930+0000 7fc710800700 1 -- 172.21.15.88:0/3849286785 shutdown_connections 2024-03-24T00:42:01.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:01.931+0000 7fc710800700 1 -- 172.21.15.88:0/3849286785 wait complete. 2024-03-24T00:42:02.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:01 smithi038 ceph-mon[29271]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:42:02.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:01 smithi038 ceph-mon[29271]: from='osd.3 [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T00:42:02.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:01 smithi038 ceph-mon[29271]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T00:42:02.660 DEBUG:teuthology.orchestra.run.smithi088:osd.3> sudo journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.3.service 2024-03-24T00:42:02.666 INFO:tasks.cephadm:Deploying osd.4 on smithi088 with /dev/vg_nvme/lv_3... 2024-03-24T00:42:02.666 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-24T00:42:02.925 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:02 smithi088 ceph-mon[36610]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T00:42:02.926 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:02 smithi088 ceph-mon[36610]: osdmap e25: 4 total, 3 up, 4 in 2024-03-24T00:42:02.926 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:02 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:42:02.926 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:02 smithi088 ceph-mon[36610]: from='osd.3 [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:42:02.926 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:02 smithi088 ceph-mon[36610]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:42:02.926 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:02 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:02.926 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:02 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:02.926 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:02 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:02.926 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:02 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:02.926 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:02 smithi088 ceph-mon[36610]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:42:02.926 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:42:02 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[42559]: 2024-03-24T00:42:02.746+0000 7fa24ec00700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T00:42:03.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:02 smithi038 ceph-mon[29271]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T00:42:03.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:02 smithi038 ceph-mon[29271]: osdmap e25: 4 total, 3 up, 4 in 2024-03-24T00:42:03.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:02 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:42:03.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:02 smithi038 ceph-mon[29271]: from='osd.3 [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:42:03.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:02 smithi038 ceph-mon[29271]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:42:03.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:02 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:03.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:02 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:03.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:02 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:03.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:02 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:03.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:02 smithi038 ceph-mon[29271]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T00:42:03.619 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi088/config 2024-03-24T00:42:04.055 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:03 smithi088 ceph-mon[36610]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-03-24T00:42:04.055 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:03 smithi088 ceph-mon[36610]: osdmap e26: 4 total, 3 up, 4 in 2024-03-24T00:42:04.055 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:03 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:42:04.055 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:03 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:42:04.055 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:03 smithi088 ceph-mon[36610]: osd.3 [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048] boot 2024-03-24T00:42:04.055 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:03 smithi088 ceph-mon[36610]: osdmap e27: 4 total, 4 up, 4 in 2024-03-24T00:42:04.055 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:03 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:42:04.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:03 smithi038 ceph-mon[29271]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-03-24T00:42:04.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:03 smithi038 ceph-mon[29271]: osdmap e26: 4 total, 3 up, 4 in 2024-03-24T00:42:04.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:42:04.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:42:04.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:03 smithi038 ceph-mon[29271]: osd.3 [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048] boot 2024-03-24T00:42:04.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:03 smithi038 ceph-mon[29271]: osdmap e27: 4 total, 4 up, 4 in 2024-03-24T00:42:04.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:03 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:42:04.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:04 smithi038 ceph-mon[29271]: purged_snaps scrub starts 2024-03-24T00:42:04.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:04 smithi038 ceph-mon[29271]: purged_snaps scrub ok 2024-03-24T00:42:04.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:04 smithi038 ceph-mon[29271]: osdmap e28: 4 total, 4 up, 4 in 2024-03-24T00:42:04.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:04 smithi038 ceph-mon[29271]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:04.954 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:04 smithi088 ceph-mon[36610]: purged_snaps scrub starts 2024-03-24T00:42:04.954 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:04 smithi088 ceph-mon[36610]: purged_snaps scrub ok 2024-03-24T00:42:04.954 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:04 smithi088 ceph-mon[36610]: osdmap e28: 4 total, 4 up, 4 in 2024-03-24T00:42:04.954 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:04 smithi088 ceph-mon[36610]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:06.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:06 smithi088 ceph-mon[36610]: osdmap e29: 4 total, 4 up, 4 in 2024-03-24T00:42:06.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:06 smithi038 ceph-mon[29271]: osdmap e29: 4 total, 4 up, 4 in 2024-03-24T00:42:07.358 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-24T00:42:07.359 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-24T00:42:07.359 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-03-24T00:42:07.359 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-03-24T00:42:07.359 INFO:teuthology.orchestra.run.smithi088.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0355104 s, 295 MB/s 2024-03-24T00:42:07.359 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-03-24T00:42:07.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:07 smithi038 ceph-mon[29271]: osdmap e30: 4 total, 4 up, 4 in 2024-03-24T00:42:07.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:07 smithi038 ceph-mon[29271]: pgmap v127: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:07.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:07 smithi088 ceph-mon[36610]: osdmap e30: 4 total, 4 up, 4 in 2024-03-24T00:42:07.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:07 smithi088 ceph-mon[36610]: pgmap v127: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:08.485 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi088:vg_nvme/lv_3 2024-03-24T00:42:08.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: osdmap e31: 4 total, 4 up, 4 in 2024-03-24T00:42:08.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:08.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:08.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:42:08.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:08.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:08.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:42:08.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:08.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:42:08.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:08.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:42:08.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:08 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:08.875 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi088/config 2024-03-24T00:42:08.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: osdmap e31: 4 total, 4 up, 4 in 2024-03-24T00:42:08.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:08.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:08.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:42:08.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:08.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:08.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:42:08.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:08.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:42:08.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:08.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:42:08.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:08 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:09.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:09 smithi038 ceph-mon[29271]: Detected new or changed devices on smithi088 2024-03-24T00:42:09.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:09 smithi038 ceph-mon[29271]: Adjusting osd_memory_target on smithi088 to 14915M 2024-03-24T00:42:09.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:09 smithi038 ceph-mon[29271]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-03-24T00:42:09.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:09 smithi088 ceph-mon[36610]: Detected new or changed devices on smithi088 2024-03-24T00:42:09.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:09 smithi088 ceph-mon[36610]: Adjusting osd_memory_target on smithi088 to 14915M 2024-03-24T00:42:09.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:09 smithi088 ceph-mon[36610]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-03-24T00:42:10.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.747+0000 7f23d3c00700 1 -- 172.21.15.88:0/1594642284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23cc101d00 msgr2=0x7f23cc102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:42:10.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.747+0000 7f23d3c00700 1 --2- 172.21.15.88:0/1594642284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23cc101d00 0x7f23cc102160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009910 tx=0x7f23b8009c20 comp rx=0 tx=0).stop 2024-03-24T00:42:10.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.747+0000 7f23d3c00700 1 -- 172.21.15.88:0/1594642284 shutdown_connections 2024-03-24T00:42:10.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.747+0000 7f23d3c00700 1 --2- 172.21.15.88:0/1594642284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23cc101d00 0x7f23cc102160 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:10.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.747+0000 7f23d3c00700 1 --2- 172.21.15.88:0/1594642284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23cc100b00 0x7f23cc100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:10.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.747+0000 7f23d3c00700 1 -- 172.21.15.88:0/1594642284 >> 172.21.15.88:0/1594642284 conn(0x7f23cc0fc0a0 msgr2=0x7f23cc0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:42:10.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.747+0000 7f23d3c00700 1 -- 172.21.15.88:0/1594642284 shutdown_connections 2024-03-24T00:42:10.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.747+0000 7f23d3c00700 1 -- 172.21.15.88:0/1594642284 wait complete. 2024-03-24T00:42:10.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.748+0000 7f23d3c00700 1 Processor -- start 2024-03-24T00:42:10.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.749+0000 7f23d3c00700 1 -- start start 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.749+0000 7f23d3c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23cc100b00 0x7f23cc1964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.749+0000 7f23d3c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23cc101d00 0x7f23cc196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.749+0000 7f23d3c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc197020 con 0x7f23cc100b00 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.749+0000 7f23d3c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc197160 con 0x7f23cc101d00 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.749+0000 7f23cbe00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23cc101d00 0x7f23cc196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.749+0000 7f23cbe00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23cc101d00 0x7f23cc196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56158/0 (socket says 172.21.15.88:56158) 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.749+0000 7f23cbe00700 1 -- 172.21.15.88:0/2071837456 learned_addr learned my addr 172.21.15.88:0/2071837456 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.750+0000 7f23d1200700 1 --2- 172.21.15.88:0/2071837456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23cc100b00 0x7f23cc1964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.750+0000 7f23cbe00700 1 -- 172.21.15.88:0/2071837456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23cc100b00 msgr2=0x7f23cc1964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.750+0000 7f23cbe00700 1 --2- 172.21.15.88:0/2071837456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23cc100b00 0x7f23cc1964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.750+0000 7f23cbe00700 1 -- 172.21.15.88:0/2071837456 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80095f0 con 0x7f23cc101d00 2024-03-24T00:42:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.750+0000 7f23cbe00700 1 --2- 172.21.15.88:0/2071837456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23cc101d00 0x7f23cc196a00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f23b800bd70 tx=0x7f23b800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:42:10.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.750+0000 7f23c9600700 1 -- 172.21.15.88:0/2071837456 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23b801d070 con 0x7f23cc101d00 2024-03-24T00:42:10.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.750+0000 7f23c9600700 1 -- 172.21.15.88:0/2071837456 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23b8005dc0 con 0x7f23cc101d00 2024-03-24T00:42:10.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.750+0000 7f23d3c00700 1 -- 172.21.15.88:0/2071837456 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc19bb60 con 0x7f23cc101d00 2024-03-24T00:42:10.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.750+0000 7f23c9600700 1 -- 172.21.15.88:0/2071837456 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23b8021410 con 0x7f23cc101d00 2024-03-24T00:42:10.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.750+0000 7f23d3c00700 1 -- 172.21.15.88:0/2071837456 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc19c0d0 con 0x7f23cc101d00 2024-03-24T00:42:10.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.752+0000 7f23d3c00700 1 -- 172.21.15.88:0/2071837456 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc190680 con 0x7f23cc101d00 2024-03-24T00:42:10.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.753+0000 7f23c9600700 1 -- 172.21.15.88:0/2071837456 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f23b800f820 con 0x7f23cc101d00 2024-03-24T00:42:10.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.753+0000 7f23c9600700 1 --2- 172.21.15.88:0/2071837456 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f23b406ccf0 0x7f23b406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:42:10.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.754+0000 7f23c9600700 1 -- 172.21.15.88:0/2071837456 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f23b808cdf0 con 0x7f23cc101d00 2024-03-24T00:42:10.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.754+0000 7f23d1200700 1 --2- 172.21.15.88:0/2071837456 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f23b406ccf0 0x7f23b406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:42:10.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.755+0000 7f23d1200700 1 --2- 172.21.15.88:0/2071837456 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f23b406ccf0 0x7f23b406f1b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f23c0005950 tx=0x7f23c00058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:42:10.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.759+0000 7f23c9600700 1 -- 172.21.15.88:0/2071837456 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f23b805b2d0 con 0x7f23cc101d00 2024-03-24T00:42:10.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:10.994+0000 7f23d3c00700 1 -- 172.21.15.88:0/2071837456 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f23cc0611f0 con 0x7f23b406ccf0 2024-03-24T00:42:12.007 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:11 smithi088 ceph-mon[36610]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T00:42:12.007 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:11 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:42:12.007 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:11 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:42:12.008 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:11 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:12.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:11 smithi038 ceph-mon[29271]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T00:42:12.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:11 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:42:12.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:11 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:42:12.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:11 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:12.854 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:12 smithi088 ceph-mon[36610]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:42:12.854 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:12 smithi088 ceph-mon[36610]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-24T00:42:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:12 smithi038 ceph-mon[29271]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:42:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:12 smithi038 ceph-mon[29271]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-24T00:42:14.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:42:14.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:42:15.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:15 smithi038 ceph-mon[29271]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-24T00:42:15.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:15 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/475909807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b73f29f7-134e-4725-b341-e893b4bbff8c"}]: dispatch 2024-03-24T00:42:15.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:15 smithi038 ceph-mon[29271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b73f29f7-134e-4725-b341-e893b4bbff8c"}]: dispatch 2024-03-24T00:42:15.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:15 smithi038 ceph-mon[29271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b73f29f7-134e-4725-b341-e893b4bbff8c"}]': finished 2024-03-24T00:42:15.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:15 smithi038 ceph-mon[29271]: osdmap e32: 5 total, 4 up, 5 in 2024-03-24T00:42:15.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:15 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:42:15.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:15 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/4091162859' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:42:15.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:15 smithi088 ceph-mon[36610]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-24T00:42:15.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:15 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/475909807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b73f29f7-134e-4725-b341-e893b4bbff8c"}]: dispatch 2024-03-24T00:42:15.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:15 smithi088 ceph-mon[36610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b73f29f7-134e-4725-b341-e893b4bbff8c"}]: dispatch 2024-03-24T00:42:15.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:15 smithi088 ceph-mon[36610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b73f29f7-134e-4725-b341-e893b4bbff8c"}]': finished 2024-03-24T00:42:15.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:15 smithi088 ceph-mon[36610]: osdmap e32: 5 total, 4 up, 5 in 2024-03-24T00:42:15.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:15 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:42:15.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:15 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/4091162859' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:42:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:17 smithi038 ceph-mon[29271]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-24T00:42:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:17 smithi088 ceph-mon[36610]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-24T00:42:19.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:19 smithi088 ceph-mon[36610]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:19.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:19 smithi038 ceph-mon[29271]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:21 smithi088 ceph-mon[36610]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:22.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:21 smithi038 ceph-mon[29271]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:22.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:22 smithi038 ceph-mon[29271]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:22.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:22 smithi088 ceph-mon[36610]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:24.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:24 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T00:42:24.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:24 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:24.876 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:24 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T00:42:24.876 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:24 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:25.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:25 smithi038 ceph-mon[29271]: Deploying daemon osd.4 on smithi088 2024-03-24T00:42:25.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:25 smithi038 ceph-mon[29271]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:25.893 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:25 smithi088 ceph-mon[36610]: Deploying daemon osd.4 on smithi088 2024-03-24T00:42:25.893 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:25 smithi088 ceph-mon[36610]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:27.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:27 smithi088 ceph-mon[36610]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:27 smithi038 ceph-mon[29271]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:29.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:29 smithi088 ceph-mon[36610]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:29.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:42:29.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:29 smithi038 ceph-mon[29271]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:29.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:42:30.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:30 smithi088 ceph-mon[36610]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:31.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:30 smithi038 ceph-mon[29271]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:32.877 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:32 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:32.877 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:32 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:42:32.877 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:32 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:32.877 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:32 smithi088 ceph-mon[36610]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:32 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:32 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:42:33.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:32 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:33.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:32 smithi038 ceph-mon[29271]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:34.969 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:34 smithi088 ceph-mon[36610]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:35.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:34 smithi038 ceph-mon[29271]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:35.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:35 smithi088 ceph-mon[36610]: from='osd.4 [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T00:42:35.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:35 smithi088 ceph-mon[36610]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T00:42:36.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:35 smithi038 ceph-mon[29271]: from='osd.4 [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T00:42:36.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:35 smithi038 ceph-mon[29271]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T00:42:36.928 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:36 smithi088 ceph-mon[36610]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T00:42:36.928 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:36 smithi088 ceph-mon[36610]: osdmap e33: 5 total, 4 up, 5 in 2024-03-24T00:42:36.928 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:36 smithi088 ceph-mon[36610]: from='osd.4 [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:42:36.928 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:36 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:42:36.928 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:36 smithi088 ceph-mon[36610]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:42:36.928 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:36 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:36.929 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:36 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:36.929 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:36 smithi088 ceph-mon[36610]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:37.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:36 smithi038 ceph-mon[29271]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T00:42:37.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:36 smithi038 ceph-mon[29271]: osdmap e33: 5 total, 4 up, 5 in 2024-03-24T00:42:37.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:36 smithi038 ceph-mon[29271]: from='osd.4 [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:42:37.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:36 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:42:37.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:36 smithi038 ceph-mon[29271]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:42:37.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:36 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:37.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:36 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:37.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:36 smithi038 ceph-mon[29271]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-24T00:42:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.306+0000 7f23c9600700 1 -- 172.21.15.88:0/2071837456 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f23cc0611f0 con 0x7f23b406ccf0 2024-03-24T00:42:37.309 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 4 on host 'smithi088' 2024-03-24T00:42:37.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.310+0000 7f23d3c00700 1 -- 172.21.15.88:0/2071837456 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f23b406ccf0 msgr2=0x7f23b406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:42:37.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.310+0000 7f23d3c00700 1 --2- 172.21.15.88:0/2071837456 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f23b406ccf0 0x7f23b406f1b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f23c0005950 tx=0x7f23c00058e0 comp rx=0 tx=0).stop 2024-03-24T00:42:37.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.310+0000 7f23d3c00700 1 -- 172.21.15.88:0/2071837456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23cc101d00 msgr2=0x7f23cc196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:42:37.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.310+0000 7f23d3c00700 1 --2- 172.21.15.88:0/2071837456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23cc101d00 0x7f23cc196a00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f23b800bd70 tx=0x7f23b800be50 comp rx=0 tx=0).stop 2024-03-24T00:42:37.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.310+0000 7f23d3c00700 1 -- 172.21.15.88:0/2071837456 shutdown_connections 2024-03-24T00:42:37.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.310+0000 7f23d3c00700 1 --2- 172.21.15.88:0/2071837456 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f23b406ccf0 0x7f23b406f1b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:37.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.310+0000 7f23d3c00700 1 --2- 172.21.15.88:0/2071837456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23cc100b00 0x7f23cc1964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:37.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.310+0000 7f23d3c00700 1 --2- 172.21.15.88:0/2071837456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23cc101d00 0x7f23cc196a00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:37.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.310+0000 7f23d3c00700 1 -- 172.21.15.88:0/2071837456 >> 172.21.15.88:0/2071837456 conn(0x7f23cc0fc0a0 msgr2=0x7f23cc105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:42:37.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.310+0000 7f23d3c00700 1 -- 172.21.15.88:0/2071837456 shutdown_connections 2024-03-24T00:42:37.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:37.310+0000 7f23d3c00700 1 -- 172.21.15.88:0/2071837456 wait complete. 2024-03-24T00:42:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:37 smithi038 ceph-mon[29271]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-03-24T00:42:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:37 smithi038 ceph-mon[29271]: osdmap e34: 5 total, 4 up, 5 in 2024-03-24T00:42:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:42:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:42:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:38.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:38.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:37 smithi038 ceph-mon[29271]: osd.4 [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126] boot 2024-03-24T00:42:38.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:37 smithi038 ceph-mon[29271]: osdmap e35: 5 total, 5 up, 5 in 2024-03-24T00:42:38.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:37 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:42:38.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:37 smithi088 ceph-mon[36610]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-03-24T00:42:38.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:37 smithi088 ceph-mon[36610]: osdmap e34: 5 total, 4 up, 5 in 2024-03-24T00:42:38.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:37 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:42:38.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:37 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:42:38.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:37 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:38.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:37 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:38.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:37 smithi088 ceph-mon[36610]: osd.4 [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126] boot 2024-03-24T00:42:38.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:37 smithi088 ceph-mon[36610]: osdmap e35: 5 total, 5 up, 5 in 2024-03-24T00:42:38.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:37 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:42:38.222 DEBUG:teuthology.orchestra.run.smithi088:osd.4> sudo journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.4.service 2024-03-24T00:42:38.229 INFO:tasks.cephadm:Deploying osd.5 on smithi088 with /dev/vg_nvme/lv_2... 2024-03-24T00:42:38.229 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-24T00:42:39.801 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi088/config 2024-03-24T00:42:39.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:39 smithi038 ceph-mon[29271]: purged_snaps scrub starts 2024-03-24T00:42:39.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:39 smithi038 ceph-mon[29271]: purged_snaps scrub ok 2024-03-24T00:42:39.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:39 smithi038 ceph-mon[29271]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:39.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:39 smithi088 ceph-mon[36610]: purged_snaps scrub starts 2024-03-24T00:42:39.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:39 smithi088 ceph-mon[36610]: purged_snaps scrub ok 2024-03-24T00:42:39.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:39 smithi088 ceph-mon[36610]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:40.824 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:40 smithi088 ceph-mon[36610]: osdmap e36: 5 total, 5 up, 5 in 2024-03-24T00:42:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:40 smithi038 ceph-mon[29271]: osdmap e36: 5 total, 5 up, 5 in 2024-03-24T00:42:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:41 smithi038 ceph-mon[29271]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:41.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:41 smithi088 ceph-mon[36610]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:42.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:42 smithi088 ceph-mon[36610]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:43.108 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:42 smithi038 ceph-mon[29271]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:43.121 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-24T00:42:43.121 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-24T00:42:43.121 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-03-24T00:42:43.121 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-03-24T00:42:43.122 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0347604 s, 302 MB/s 2024-03-24T00:42:43.122 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-03-24T00:42:43.971 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi088:vg_nvme/lv_2 2024-03-24T00:42:44.367 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi088/config 2024-03-24T00:42:44.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: Detected new or changed devices on smithi088 2024-03-24T00:42:44.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:44.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:44.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:42:44.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:42:44.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: Adjusting osd_memory_target on smithi088 to 7457M 2024-03-24T00:42:44.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:44.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:44.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:42:44.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:44.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:42:44.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:42:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: Detected new or changed devices on smithi088 2024-03-24T00:42:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:42:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:42:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: Adjusting osd_memory_target on smithi088 to 7457M 2024-03-24T00:42:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:42:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:42:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:42:45.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:45 smithi088 ceph-mon[36610]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:45.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:45 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:45.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:45 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:42:45.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:45 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:45.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:45 smithi038 ceph-mon[29271]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:45.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:45 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:45.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:45 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:42:45.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:45 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:42:46.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.580+0000 7f9f37600700 1 -- 172.21.15.88:0/4136113641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f30107130 msgr2=0x7f9f30107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:42:46.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.580+0000 7f9f37600700 1 --2- 172.21.15.88:0/4136113641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f30107130 0x7f9f30107550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009910 tx=0x7f9f20009c20 comp rx=0 tx=0).stop 2024-03-24T00:42:46.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.580+0000 7f9f37600700 1 -- 172.21.15.88:0/4136113641 shutdown_connections 2024-03-24T00:42:46.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.580+0000 7f9f37600700 1 --2- 172.21.15.88:0/4136113641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30108300 0x7f9f30108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:46.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.580+0000 7f9f37600700 1 --2- 172.21.15.88:0/4136113641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f30107130 0x7f9f30107550 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:46.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.580+0000 7f9f37600700 1 -- 172.21.15.88:0/4136113641 >> 172.21.15.88:0/4136113641 conn(0x7f9f300762d0 msgr2=0x7f9f30078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:42:46.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.580+0000 7f9f37600700 1 -- 172.21.15.88:0/4136113641 shutdown_connections 2024-03-24T00:42:46.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.580+0000 7f9f37600700 1 -- 172.21.15.88:0/4136113641 wait complete. 2024-03-24T00:42:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.581+0000 7f9f37600700 1 Processor -- start 2024-03-24T00:42:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.582+0000 7f9f37600700 1 -- start start 2024-03-24T00:42:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.582+0000 7f9f37600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f30107130 0x7f9f3019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:42:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.582+0000 7f9f37600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30108300 0x7f9f3019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:42:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.582+0000 7f9f37600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3019d630 con 0x7f9f30108300 2024-03-24T00:42:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.582+0000 7f9f37600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3019d770 con 0x7f9f30107130 2024-03-24T00:42:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.582+0000 7f9f34c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f30107130 0x7f9f3019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:42:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.582+0000 7f9f34c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f30107130 0x7f9f3019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60728/0 (socket says 172.21.15.88:60728) 2024-03-24T00:42:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.582+0000 7f9f34c00700 1 -- 172.21.15.88:0/592304181 learned_addr learned my addr 172.21.15.88:0/592304181 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:42:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.583+0000 7f9f27e00700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30108300 0x7f9f3019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:42:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.583+0000 7f9f34c00700 1 -- 172.21.15.88:0/592304181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30108300 msgr2=0x7f9f3019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:42:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.583+0000 7f9f34c00700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30108300 0x7f9f3019d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:42:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.583+0000 7f9f34c00700 1 -- 172.21.15.88:0/592304181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f200095f0 con 0x7f9f30107130 2024-03-24T00:42:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.583+0000 7f9f27e00700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30108300 0x7f9f3019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:42:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.583+0000 7f9f34c00700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f30107130 0x7f9f3019cad0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000bd70 tx=0x7f9f2000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:42:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.583+0000 7f9f2e000700 1 -- 172.21.15.88:0/592304181 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f2001d070 con 0x7f9f30107130 2024-03-24T00:42:46.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.583+0000 7f9f2e000700 1 -- 172.21.15.88:0/592304181 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f20005dc0 con 0x7f9f30107130 2024-03-24T00:42:46.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.583+0000 7f9f2e000700 1 -- 172.21.15.88:0/592304181 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f20021410 con 0x7f9f30107130 2024-03-24T00:42:46.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.583+0000 7f9f37600700 1 -- 172.21.15.88:0/592304181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f301a2120 con 0x7f9f30107130 2024-03-24T00:42:46.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.584+0000 7f9f37600700 1 -- 172.21.15.88:0/592304181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f301a2690 con 0x7f9f30107130 2024-03-24T00:42:46.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.585+0000 7f9f37600700 1 -- 172.21.15.88:0/592304181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f30196cc0 con 0x7f9f30107130 2024-03-24T00:42:46.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.586+0000 7f9f2e000700 1 -- 172.21.15.88:0/592304181 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9f2000f820 con 0x7f9f30107130 2024-03-24T00:42:46.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.587+0000 7f9f2e000700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f1406ccf0 0x7f9f1406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:42:46.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.587+0000 7f9f2e000700 1 -- 172.21.15.88:0/592304181 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f9f2008cfb0 con 0x7f9f30107130 2024-03-24T00:42:46.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.588+0000 7f9f27e00700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f1406ccf0 0x7f9f1406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:42:46.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.589+0000 7f9f27e00700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f1406ccf0 0x7f9f1406f1b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005fd0 tx=0x7f9f18005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:42:46.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.592+0000 7f9f2e000700 1 -- 172.21.15.88:0/592304181 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9f2005b300 con 0x7f9f30107130 2024-03-24T00:42:46.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:42:46.820+0000 7f9f37600700 1 -- 172.21.15.88:0/592304181 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f300611f0 con 0x7f9f1406ccf0 2024-03-24T00:42:47.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:46 smithi038 ceph-mon[29271]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:47.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:46 smithi088 ceph-mon[36610]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:47.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:47 smithi088 ceph-mon[36610]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:42:47.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:47 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:42:47.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:47 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:42:47.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:47 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:47 smithi038 ceph-mon[29271]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:42:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T00:42:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T00:42:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:42:49.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:49 smithi038 ceph-mon[29271]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:49.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:49 smithi088 ceph-mon[36610]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:51.011 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:50 smithi038 ceph-mon[29271]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:51.012 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:50 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/3873841826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ea2cfb3-be7a-4f97-9993-8f9001b8b678"}]: dispatch 2024-03-24T00:42:51.012 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:50 smithi038 ceph-mon[29271]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ea2cfb3-be7a-4f97-9993-8f9001b8b678"}]: dispatch 2024-03-24T00:42:51.012 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:50 smithi038 ceph-mon[29271]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ea2cfb3-be7a-4f97-9993-8f9001b8b678"}]': finished 2024-03-24T00:42:51.012 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:50 smithi038 ceph-mon[29271]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T00:42:51.012 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:50 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:42:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:50 smithi088 ceph-mon[36610]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:50 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/3873841826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ea2cfb3-be7a-4f97-9993-8f9001b8b678"}]: dispatch 2024-03-24T00:42:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:50 smithi088 ceph-mon[36610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ea2cfb3-be7a-4f97-9993-8f9001b8b678"}]: dispatch 2024-03-24T00:42:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:50 smithi088 ceph-mon[36610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ea2cfb3-be7a-4f97-9993-8f9001b8b678"}]': finished 2024-03-24T00:42:51.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:50 smithi088 ceph-mon[36610]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T00:42:51.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:50 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:42:52.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:51 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/3486958660' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:42:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:51 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/3486958660' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T00:42:53.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:52 smithi038 ceph-mon[29271]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:53.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:52 smithi088 ceph-mon[36610]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:55.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:54 smithi038 ceph-mon[29271]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:55.144 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:54 smithi088 ceph-mon[36610]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:57.056 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:56 smithi088 ceph-mon[36610]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:57.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:56 smithi038 ceph-mon[29271]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:59.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:59 smithi038 ceph-mon[29271]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:59.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:42:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:42:59.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:59 smithi088 ceph-mon[36610]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:42:59.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:42:59 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:43:00.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:00 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T00:43:00.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:00 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:43:00.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:00 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T00:43:00.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:00 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:43:01.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:01 smithi038 ceph-mon[29271]: Deploying daemon osd.5 on smithi088 2024-03-24T00:43:01.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:01 smithi038 ceph-mon[29271]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:01.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:01 smithi088 ceph-mon[36610]: Deploying daemon osd.5 on smithi088 2024-03-24T00:43:01.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:01 smithi088 ceph-mon[36610]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:03.047 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:02 smithi088 ceph-mon[36610]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:03.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:02 smithi038 ceph-mon[29271]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:05.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:04 smithi038 ceph-mon[29271]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:05.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:04 smithi088 ceph-mon[36610]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:06.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:06 smithi088 ceph-mon[36610]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:06 smithi038 ceph-mon[29271]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:09.775 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:09.775 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:09.776 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:43:09.776 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:09 smithi088 ceph-mon[36610]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:43:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:09 smithi038 ceph-mon[29271]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:10.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:10 smithi088 ceph-mon[36610]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:11.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:10 smithi038 ceph-mon[29271]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:12.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:12 smithi088 ceph-mon[36610]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:12 smithi038 ceph-mon[29271]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:14.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:13 smithi038 ceph-mon[29271]: from='osd.5 [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T00:43:14.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:13 smithi038 ceph-mon[29271]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T00:43:14.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:14.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:14.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:43:14.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:43:14.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:14.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:43:14.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:13 smithi088 ceph-mon[36610]: from='osd.5 [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T00:43:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:13 smithi088 ceph-mon[36610]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T00:43:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:43:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:43:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:43:14.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.292+0000 7f9f2e000700 1 -- 172.21.15.88:0/592304181 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9f300611f0 con 0x7f9f1406ccf0 2024-03-24T00:43:14.294 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 5 on host 'smithi088' 2024-03-24T00:43:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.296+0000 7f9f37600700 1 -- 172.21.15.88:0/592304181 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f1406ccf0 msgr2=0x7f9f1406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.296+0000 7f9f37600700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f1406ccf0 0x7f9f1406f1b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005fd0 tx=0x7f9f18005ee0 comp rx=0 tx=0).stop 2024-03-24T00:43:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.296+0000 7f9f37600700 1 -- 172.21.15.88:0/592304181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f30107130 msgr2=0x7f9f3019cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.296+0000 7f9f37600700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f30107130 0x7f9f3019cad0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000bd70 tx=0x7f9f2000be50 comp rx=0 tx=0).stop 2024-03-24T00:43:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.296+0000 7f9f37600700 1 -- 172.21.15.88:0/592304181 shutdown_connections 2024-03-24T00:43:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.296+0000 7f9f37600700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f1406ccf0 0x7f9f1406f1b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.296+0000 7f9f37600700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f30107130 0x7f9f3019cad0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.296+0000 7f9f37600700 1 --2- 172.21.15.88:0/592304181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f30108300 0x7f9f3019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.296+0000 7f9f37600700 1 -- 172.21.15.88:0/592304181 >> 172.21.15.88:0/592304181 conn(0x7f9f300762d0 msgr2=0x7f9f3010b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.296+0000 7f9f37600700 1 -- 172.21.15.88:0/592304181 shutdown_connections 2024-03-24T00:43:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:14.296+0000 7f9f37600700 1 -- 172.21.15.88:0/592304181 wait complete. 2024-03-24T00:43:15.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:14 smithi088 ceph-mon[36610]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T00:43:15.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:14 smithi088 ceph-mon[36610]: osdmap e38: 6 total, 5 up, 6 in 2024-03-24T00:43:15.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:43:15.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:14 smithi088 ceph-mon[36610]: from='osd.5 [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:43:15.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:14 smithi088 ceph-mon[36610]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:43:15.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:43:15.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:15.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:15.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:14 smithi088 ceph-mon[36610]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:15.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:14 smithi038 ceph-mon[29271]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T00:43:15.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:14 smithi038 ceph-mon[29271]: osdmap e38: 6 total, 5 up, 6 in 2024-03-24T00:43:15.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:43:15.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:14 smithi038 ceph-mon[29271]: from='osd.5 [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:43:15.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:14 smithi038 ceph-mon[29271]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:43:15.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:43:15.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:15.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:15.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:14 smithi038 ceph-mon[29271]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:15.148 DEBUG:teuthology.orchestra.run.smithi088:osd.5> sudo journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.5.service 2024-03-24T00:43:15.155 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-24T00:43:15.155 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-24T00:43:15.587 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:16.020 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:15 smithi038 ceph-mon[29271]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-03-24T00:43:16.020 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:15 smithi038 ceph-mon[29271]: osdmap e39: 6 total, 5 up, 6 in 2024-03-24T00:43:16.020 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:15 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:43:16.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:15 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:43:16.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:15 smithi088 ceph-mon[36610]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-03-24T00:43:16.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:15 smithi088 ceph-mon[36610]: osdmap e39: 6 total, 5 up, 6 in 2024-03-24T00:43:16.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:15 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:43:16.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:15 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:43:16.896 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:16 smithi088 ceph-mon[36610]: purged_snaps scrub starts 2024-03-24T00:43:16.896 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:16 smithi088 ceph-mon[36610]: purged_snaps scrub ok 2024-03-24T00:43:16.896 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:16 smithi088 ceph-mon[36610]: osd.5 [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582] boot 2024-03-24T00:43:16.896 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:16 smithi088 ceph-mon[36610]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T00:43:16.896 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:16 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:43:16.896 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:16 smithi088 ceph-mon[36610]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:17.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:16 smithi038 ceph-mon[29271]: purged_snaps scrub starts 2024-03-24T00:43:17.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:16 smithi038 ceph-mon[29271]: purged_snaps scrub ok 2024-03-24T00:43:17.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:16 smithi038 ceph-mon[29271]: osd.5 [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582] boot 2024-03-24T00:43:17.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:16 smithi038 ceph-mon[29271]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T00:43:17.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:16 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:43:17.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:16 smithi038 ceph-mon[29271]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-24T00:43:17.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.673+0000 7f2743800700 1 -- 172.21.15.38:0/2472361443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f273c100bd0 msgr2=0x7f273c101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.673+0000 7f2743800700 1 --2- 172.21.15.38:0/2472361443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f273c100bd0 0x7f273c101030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2728009910 tx=0x7f2728009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.673+0000 7f2743800700 1 -- 172.21.15.38:0/2472361443 shutdown_connections 2024-03-24T00:43:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.673+0000 7f2743800700 1 --2- 172.21.15.38:0/2472361443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f273c100bd0 0x7f273c101030 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.673+0000 7f2743800700 1 --2- 172.21.15.38:0/2472361443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f273c0ff9d0 0x7f273c0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.673+0000 7f2743800700 1 -- 172.21.15.38:0/2472361443 >> 172.21.15.38:0/2472361443 conn(0x7f273c0faf70 msgr2=0x7f273c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.673+0000 7f2743800700 1 -- 172.21.15.38:0/2472361443 shutdown_connections 2024-03-24T00:43:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.673+0000 7f2743800700 1 -- 172.21.15.38:0/2472361443 wait complete. 2024-03-24T00:43:17.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.674+0000 7f2743800700 1 Processor -- start 2024-03-24T00:43:17.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.674+0000 7f2743800700 1 -- start start 2024-03-24T00:43:17.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.675+0000 7f2743800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f273c0ff9d0 0x7f273c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:17.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.675+0000 7f2743800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f273c100bd0 0x7f273c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:17.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.675+0000 7f2743800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c199210 con 0x7f273c0ff9d0 2024-03-24T00:43:17.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.675+0000 7f2743800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c199350 con 0x7f273c100bd0 2024-03-24T00:43:17.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.675+0000 7f2740e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f273c0ff9d0 0x7f273c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:17.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.675+0000 7f2740e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f273c0ff9d0 0x7f273c1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34444/0 (socket says 172.21.15.38:34444) 2024-03-24T00:43:17.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.675+0000 7f2740e00700 1 -- 172.21.15.38:0/3121960863 learned_addr learned my addr 172.21.15.38:0/3121960863 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:17.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.675+0000 7f273be00700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f273c100bd0 0x7f273c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:17.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.676+0000 7f2740e00700 1 -- 172.21.15.38:0/3121960863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f273c100bd0 msgr2=0x7f273c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:17.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.676+0000 7f2740e00700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f273c100bd0 0x7f273c198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:17.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.676+0000 7f2740e00700 1 -- 172.21.15.38:0/3121960863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27280095f0 con 0x7f273c0ff9d0 2024-03-24T00:43:17.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.676+0000 7f273be00700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f273c100bd0 0x7f273c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:43:17.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.676+0000 7f2740e00700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f273c0ff9d0 0x7f273c1986b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f273000d900 tx=0x7f273000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:17.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.676+0000 7f2739600700 1 -- 172.21.15.38:0/3121960863 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27300041f0 con 0x7f273c0ff9d0 2024-03-24T00:43:17.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.676+0000 7f2739600700 1 -- 172.21.15.38:0/3121960863 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2730004350 con 0x7f273c0ff9d0 2024-03-24T00:43:17.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.676+0000 7f2739600700 1 -- 172.21.15.38:0/3121960863 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2730003dc0 con 0x7f273c0ff9d0 2024-03-24T00:43:17.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.676+0000 7f2743800700 1 -- 172.21.15.38:0/3121960863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c19ddb0 con 0x7f273c0ff9d0 2024-03-24T00:43:17.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.676+0000 7f2743800700 1 -- 172.21.15.38:0/3121960863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c19e2f0 con 0x7f273c0ff9d0 2024-03-24T00:43:17.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.678+0000 7f2743800700 1 -- 172.21.15.38:0/3121960863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273c04ea80 con 0x7f273c0ff9d0 2024-03-24T00:43:17.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.679+0000 7f2739600700 1 -- 172.21.15.38:0/3121960863 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2730009730 con 0x7f273c0ff9d0 2024-03-24T00:43:17.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.680+0000 7f2739600700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f272406cce0 0x7f272406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:17.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.680+0000 7f2739600700 1 -- 172.21.15.38:0/3121960863 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f273008bb20 con 0x7f273c0ff9d0 2024-03-24T00:43:17.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.680+0000 7f273be00700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f272406cce0 0x7f272406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:17.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.681+0000 7f273be00700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f272406cce0 0x7f272406f1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f27280059d0 tx=0x7f2728005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:17.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.685+0000 7f2739600700 1 -- 172.21.15.38:0/3121960863 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2730059d40 con 0x7f273c0ff9d0 2024-03-24T00:43:17.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.907+0000 7f2743800700 1 -- 172.21.15.38:0/3121960863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f273c19ec90 con 0x7f273c0ff9d0 2024-03-24T00:43:17.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.907+0000 7f2739600700 1 -- 172.21.15.38:0/3121960863 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f27300598d0 con 0x7f273c0ff9d0 2024-03-24T00:43:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:43:17.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.910+0000 7f2743800700 1 -- 172.21.15.38:0/3121960863 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f272406cce0 msgr2=0x7f272406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:17.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.910+0000 7f2743800700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f272406cce0 0x7f272406f1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f27280059d0 tx=0x7f2728005960 comp rx=0 tx=0).stop 2024-03-24T00:43:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.910+0000 7f2743800700 1 -- 172.21.15.38:0/3121960863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f273c0ff9d0 msgr2=0x7f273c1986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.910+0000 7f2743800700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f273c0ff9d0 0x7f273c1986b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f273000d900 tx=0x7f273000dcc0 comp rx=0 tx=0).stop 2024-03-24T00:43:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.910+0000 7f2743800700 1 -- 172.21.15.38:0/3121960863 shutdown_connections 2024-03-24T00:43:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.910+0000 7f2743800700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f272406cce0 0x7f272406f1a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.910+0000 7f2743800700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f273c0ff9d0 0x7f273c1986b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.910+0000 7f2743800700 1 --2- 172.21.15.38:0/3121960863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f273c100bd0 0x7f273c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.910+0000 7f2743800700 1 -- 172.21.15.38:0/3121960863 >> 172.21.15.38:0/3121960863 conn(0x7f273c0faf70 msgr2=0x7f273c068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.911+0000 7f2743800700 1 -- 172.21.15.38:0/3121960863 shutdown_connections 2024-03-24T00:43:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:17.911+0000 7f2743800700 1 -- 172.21.15.38:0/3121960863 wait complete. 2024-03-24T00:43:18.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:17 smithi038 ceph-mon[29271]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T00:43:18.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:17 smithi088 ceph-mon[36610]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T00:43:18.690 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1711240995,"num_in_osds":6,"osd_in_since":1711240970,"num_remapped_pgs":0} 2024-03-24T00:43:18.691 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T00:43:18.922 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:18 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3121960863' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T00:43:18.922 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:18 smithi038 ceph-mon[29271]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:19.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:18 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3121960863' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T00:43:19.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:18 smithi088 ceph-mon[36610]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:20.917 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: Detected new or changed devices on smithi088 2024-03-24T00:43:20.917 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:20.917 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:20.917 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:43:20.917 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:43:20.917 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:43:20.917 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: Adjusting osd_memory_target on smithi088 to 4971M 2024-03-24T00:43:20.917 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:20.917 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:43:20.917 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:43:20.918 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:20.918 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:20 smithi038 ceph-mon[29271]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: Detected new or changed devices on smithi088 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: Adjusting osd_memory_target on smithi088 to 4971M 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:43:21.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:20 smithi088 ceph-mon[36610]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.183+0000 7f8a12000700 1 -- 172.21.15.38:0/1154181273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a0c102390 msgr2=0x7f8a0c10a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.183+0000 7f8a12000700 1 --2- 172.21.15.38:0/1154181273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a0c102390 0x7f8a0c10a880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009910 tx=0x7f89f8009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.184+0000 7f8a12000700 1 -- 172.21.15.38:0/1154181273 shutdown_connections 2024-03-24T00:43:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.184+0000 7f8a12000700 1 --2- 172.21.15.38:0/1154181273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a0c102390 0x7f8a0c10a880 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.184+0000 7f8a12000700 1 --2- 172.21.15.38:0/1154181273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a0c101a70 0x7f8a0c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.184+0000 7f8a12000700 1 -- 172.21.15.38:0/1154181273 >> 172.21.15.38:0/1154181273 conn(0x7f8a0c0fb300 msgr2=0x7f8a0c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.184+0000 7f8a12000700 1 -- 172.21.15.38:0/1154181273 shutdown_connections 2024-03-24T00:43:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.184+0000 7f8a12000700 1 -- 172.21.15.38:0/1154181273 wait complete. 2024-03-24T00:43:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.185+0000 7f8a12000700 1 Processor -- start 2024-03-24T00:43:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.185+0000 7f8a12000700 1 -- start start 2024-03-24T00:43:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.186+0000 7f8a12000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a0c101a70 0x7f8a0c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.186+0000 7f8a12000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a0c102390 0x7f8a0c19d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.186+0000 7f8a12000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c19d980 con 0x7f8a0c101a70 2024-03-24T00:43:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.186+0000 7f8a12000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c19dac0 con 0x7f8a0c102390 2024-03-24T00:43:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.186+0000 7f8a10c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a0c101a70 0x7f8a0c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.186+0000 7f8a10c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a0c101a70 0x7f8a0c19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34460/0 (socket says 172.21.15.38:34460) 2024-03-24T00:43:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.186+0000 7f8a10c00700 1 -- 172.21.15.38:0/216005478 learned_addr learned my addr 172.21.15.38:0/216005478 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.186+0000 7f8a0be00700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a0c102390 0x7f8a0c19d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.187+0000 7f8a10c00700 1 -- 172.21.15.38:0/216005478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a0c102390 msgr2=0x7f8a0c19d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.187+0000 7f8a10c00700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a0c102390 0x7f8a0c19d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:21.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.187+0000 7f8a10c00700 1 -- 172.21.15.38:0/216005478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f80095f0 con 0x7f8a0c101a70 2024-03-24T00:43:21.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.187+0000 7f8a0be00700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a0c102390 0x7f8a0c19d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:43:21.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.187+0000 7f8a10c00700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a0c101a70 0x7f8a0c19cdf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c930 tx=0x7f8a0000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:21.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.187+0000 7f8a09600700 1 -- 172.21.15.38:0/216005478 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a00007ad0 con 0x7f8a0c101a70 2024-03-24T00:43:21.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.187+0000 7f8a09600700 1 -- 172.21.15.38:0/216005478 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a00007c30 con 0x7f8a0c101a70 2024-03-24T00:43:21.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.187+0000 7f8a12000700 1 -- 172.21.15.38:0/216005478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c1a1850 con 0x7f8a0c101a70 2024-03-24T00:43:21.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.187+0000 7f8a09600700 1 -- 172.21.15.38:0/216005478 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a00018610 con 0x7f8a0c101a70 2024-03-24T00:43:21.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.187+0000 7f8a12000700 1 -- 172.21.15.38:0/216005478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c1a1da0 con 0x7f8a0c101a70 2024-03-24T00:43:21.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.189+0000 7f8a12000700 1 -- 172.21.15.38:0/216005478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c105fd0 con 0x7f8a0c101a70 2024-03-24T00:43:21.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.190+0000 7f8a09600700 1 -- 172.21.15.38:0/216005478 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8a0000f450 con 0x7f8a0c101a70 2024-03-24T00:43:21.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.191+0000 7f8a09600700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f89f406ccf0 0x7f89f406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:21.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.191+0000 7f8a09600700 1 -- 172.21.15.38:0/216005478 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8a0008c1d0 con 0x7f8a0c101a70 2024-03-24T00:43:21.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.191+0000 7f8a0be00700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f89f406ccf0 0x7f89f406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:21.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.192+0000 7f8a0be00700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f89f406ccf0 0x7f89f406f1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f89f8006010 tx=0x7f89f800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:21.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.196+0000 7f8a09600700 1 -- 172.21.15.38:0/216005478 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8a00056840 con 0x7f8a0c101a70 2024-03-24T00:43:21.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.420+0000 7f8a12000700 1 -- 172.21.15.38:0/216005478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a0c19e100 con 0x7f8a0c101a70 2024-03-24T00:43:21.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.421+0000 7f8a09600700 1 -- 172.21.15.38:0/216005478 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11276 (secure 0 0 0) 0x7f8a00059e60 con 0x7f8a0c101a70 2024-03-24T00:43:21.422 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:43:21.422 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":41,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","created":"2024-03-24T00:35:36.968970+0000","modified":"2024-03-24T00:43:16.800268+0000","last_up_change":"2024-03-24T00:43:15.784148+0000","last_in_change":"2024-03-24T00:42:50.726360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T00:41:31.289149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e5df40af-b5d0-4f89-94b2-59ee2e710bc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6803","nonce":667786666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6805","nonce":667786666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6809","nonce":667786666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6807","nonce":667786666}]},"public_addr":"172.21.15.38:6803/667786666","cluster_addr":"172.21.15.38:6805/667786666","heartbeat_back_addr":"172.21.15.38:6809/667786666","heartbeat_front_addr":"172.21.15.38:6807/667786666","state":["exists","up"]},{"osd":1,"uuid":"ac3c0960-dd6c-44f9-b4b0-234c10f838fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6811","nonce":723151394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6813","nonce":723151394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6817","nonce":723151394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6815","nonce":723151394}]},"public_addr":"172.21.15.38:6811/723151394","cluster_addr":"172.21.15.38:6813/723151394","heartbeat_back_addr":"172.21.15.38:6817/723151394","heartbeat_front_addr":"172.21.15.38:6815/723151394","state":["exists","up"]},{"osd":2,"uuid":"921d9dd3-2471-4b13-b6bd-01e5b6c78648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6819","nonce":643463158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6821","nonce":643463158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6825","nonce":643463158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6823","nonce":643463158}]},"public_addr":"172.21.15.38:6819/643463158","cluster_addr":"172.21.15.38:6821/643463158","heartbeat_back_addr":"172.21.15.38:6825/643463158","heartbeat_front_addr":"172.21.15.38:6823/643463158","state":["exists","up"]},{"osd":3,"uuid":"7e93813a-0036-4cea-8396-41b8b759c257","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6801","nonce":3248144048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6803","nonce":3248144048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6807","nonce":3248144048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6805","nonce":3248144048}]},"public_addr":"172.21.15.88:6801/3248144048","cluster_addr":"172.21.15.88:6803/3248144048","heartbeat_back_addr":"172.21.15.88:6807/3248144048","heartbeat_front_addr":"172.21.15.88:6805/3248144048","state":["exists","up"]},{"osd":4,"uuid":"b73f29f7-134e-4725-b341-e893b4bbff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6809","nonce":1859611126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6811","nonce":1859611126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6815","nonce":1859611126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6813","nonce":1859611126}]},"public_addr":"172.21.15.88:6809/1859611126","cluster_addr":"172.21.15.88:6811/1859611126","heartbeat_back_addr":"172.21.15.88:6815/1859611126","heartbeat_front_addr":"172.21.15.88:6813/1859611126","state":["exists","up"]},{"osd":5,"uuid":"8ea2cfb3-be7a-4f97-9993-8f9001b8b678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6817","nonce":2286313582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6819","nonce":2286313582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6823","nonce":2286313582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6821","nonce":2286313582}]},"public_addr":"172.21.15.88:6817/2286313582","cluster_addr":"172.21.15.88:6819/2286313582","heartbeat_back_addr":"172.21.15.88:6823/2286313582","heartbeat_front_addr":"172.21.15.88:6821/2286313582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:11.140794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:49.877898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:41:28.612279+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:02.338217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:36.760939+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:43:13.811948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/4049972561":"2024-03-25T00:38:28.565032+0000","172.21.15.38:6801/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6800/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6800/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/2868668356":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/3448795319":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/748221873":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/3860887425":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/913707898":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6801/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/1949387173":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6800/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6801/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/884740303":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/3845097007":"2024-03-25T00:36:55.932282+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T00:43:21.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.424+0000 7f8a12000700 1 -- 172.21.15.38:0/216005478 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f89f406ccf0 msgr2=0x7f89f406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:21.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.424+0000 7f8a12000700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f89f406ccf0 0x7f89f406f1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f89f8006010 tx=0x7f89f800b540 comp rx=0 tx=0).stop 2024-03-24T00:43:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.424+0000 7f8a12000700 1 -- 172.21.15.38:0/216005478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a0c101a70 msgr2=0x7f8a0c19cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.424+0000 7f8a12000700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a0c101a70 0x7f8a0c19cdf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c930 tx=0x7f8a0000ccf0 comp rx=0 tx=0).stop 2024-03-24T00:43:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.424+0000 7f8a12000700 1 -- 172.21.15.38:0/216005478 shutdown_connections 2024-03-24T00:43:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.424+0000 7f8a12000700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f89f406ccf0 0x7f89f406f1b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.424+0000 7f8a12000700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a0c101a70 0x7f8a0c19cdf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.424+0000 7f8a12000700 1 --2- 172.21.15.38:0/216005478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a0c102390 0x7f8a0c19d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.425+0000 7f8a12000700 1 -- 172.21.15.38:0/216005478 >> 172.21.15.38:0/216005478 conn(0x7f8a0c0fb300 msgr2=0x7f8a0c0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.425+0000 7f8a12000700 1 -- 172.21.15.38:0/216005478 shutdown_connections 2024-03-24T00:43:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:21.425+0000 7f8a12000700 1 -- 172.21.15.38:0/216005478 wait complete. 2024-03-24T00:43:22.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:21 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/216005478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T00:43:22.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:21 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/216005478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T00:43:22.296 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T00:41:31.289149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T00:43:22.296 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-24T00:43:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:23.178 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:22 smithi038 ceph-mon[29271]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:23.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:22 smithi088 ceph-mon[36610]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:24.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.920+0000 7f4ba4a00700 1 -- 172.21.15.38:0/3200610544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba01038a0 msgr2=0x7f4ba01078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:24.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.920+0000 7f4ba4a00700 1 --2- 172.21.15.38:0/3200610544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba01038a0 0x7f4ba01078f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009910 tx=0x7f4b88009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:24.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.920+0000 7f4ba4a00700 1 -- 172.21.15.38:0/3200610544 shutdown_connections 2024-03-24T00:43:24.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.920+0000 7f4ba4a00700 1 --2- 172.21.15.38:0/3200610544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba01038a0 0x7f4ba01078f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:24.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.920+0000 7f4ba4a00700 1 --2- 172.21.15.38:0/3200610544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba0102ef0 0x7f4ba01032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:24.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.920+0000 7f4ba4a00700 1 -- 172.21.15.38:0/3200610544 >> 172.21.15.38:0/3200610544 conn(0x7f4ba00fe780 msgr2=0x7f4ba0100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:24.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.921+0000 7f4ba4a00700 1 -- 172.21.15.38:0/3200610544 shutdown_connections 2024-03-24T00:43:24.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.921+0000 7f4ba4a00700 1 -- 172.21.15.38:0/3200610544 wait complete. 2024-03-24T00:43:24.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.922+0000 7f4ba4a00700 1 Processor -- start 2024-03-24T00:43:24.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.922+0000 7f4ba4a00700 1 -- start start 2024-03-24T00:43:24.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.923+0000 7f4ba4a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0102ef0 0x7f4ba0198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:24.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.923+0000 7f4ba4a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba0199460 0x7f4ba019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:24.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.923+0000 7f4ba4a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba0199a80 con 0x7f4ba0102ef0 2024-03-24T00:43:24.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.923+0000 7f4ba4a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba0199bf0 con 0x7f4ba0199460 2024-03-24T00:43:24.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.923+0000 7f4b9de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0102ef0 0x7f4ba0198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.923+0000 7f4b9de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0102ef0 0x7f4ba0198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34338/0 (socket says 172.21.15.38:34338) 2024-03-24T00:43:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.923+0000 7f4b9de00700 1 -- 172.21.15.38:0/2063384696 learned_addr learned my addr 172.21.15.38:0/2063384696 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.923+0000 7f4b97e00700 1 --2- 172.21.15.38:0/2063384696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba0199460 0x7f4ba019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.924+0000 7f4b9de00700 1 -- 172.21.15.38:0/2063384696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba0199460 msgr2=0x7f4ba019d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.924+0000 7f4b9de00700 1 --2- 172.21.15.38:0/2063384696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba0199460 0x7f4ba019d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.924+0000 7f4b9de00700 1 -- 172.21.15.38:0/2063384696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b880095f0 con 0x7f4ba0102ef0 2024-03-24T00:43:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.924+0000 7f4b9de00700 1 --2- 172.21.15.38:0/2063384696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0102ef0 0x7f4ba0198f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000b4b0 tx=0x7f4b9000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.924+0000 7f4b96a00700 1 -- 172.21.15.38:0/2063384696 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b90011800 con 0x7f4ba0102ef0 2024-03-24T00:43:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.924+0000 7f4ba4a00700 1 -- 172.21.15.38:0/2063384696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba019de10 con 0x7f4ba0102ef0 2024-03-24T00:43:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.924+0000 7f4b96a00700 1 -- 172.21.15.38:0/2063384696 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b90011e20 con 0x7f4ba0102ef0 2024-03-24T00:43:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.924+0000 7f4b96a00700 1 -- 172.21.15.38:0/2063384696 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b9000f2e0 con 0x7f4ba0102ef0 2024-03-24T00:43:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.924+0000 7f4ba4a00700 1 -- 172.21.15.38:0/2063384696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba019e360 con 0x7f4ba0102ef0 2024-03-24T00:43:24.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.926+0000 7f4ba4a00700 1 -- 172.21.15.38:0/2063384696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba019e630 con 0x7f4ba0102ef0 2024-03-24T00:43:24.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.927+0000 7f4b96a00700 1 -- 172.21.15.38:0/2063384696 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4b90010380 con 0x7f4ba0102ef0 2024-03-24T00:43:24.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.928+0000 7f4b96a00700 1 --2- 172.21.15.38:0/2063384696 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4b8406ccf0 0x7f4b8406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:24.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.928+0000 7f4b96a00700 1 -- 172.21.15.38:0/2063384696 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4b9008bb70 con 0x7f4ba0102ef0 2024-03-24T00:43:24.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.928+0000 7f4b97e00700 1 --2- 172.21.15.38:0/2063384696 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4b8406ccf0 0x7f4b8406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:24.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.929+0000 7f4b97e00700 1 --2- 172.21.15.38:0/2063384696 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4b8406ccf0 0x7f4b8406f1b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b88000c00 tx=0x7f4b8801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:24.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:24.933+0000 7f4b96a00700 1 -- 172.21.15.38:0/2063384696 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4b90056310 con 0x7f4ba0102ef0 2024-03-24T00:43:25.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:24 smithi038 ceph-mon[29271]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:25.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.160+0000 7f4ba4a00700 1 -- 172.21.15.38:0/2063384696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ba019a230 con 0x7f4ba0102ef0 2024-03-24T00:43:25.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.160+0000 7f4b96a00700 1 -- 172.21.15.38:0/2063384696 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b90059930 con 0x7f4ba0102ef0 2024-03-24T00:43:25.161 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-03-24T00:43:25.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.163+0000 7f4ba4a00700 1 -- 172.21.15.38:0/2063384696 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4b8406ccf0 msgr2=0x7f4b8406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:25.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.163+0000 7f4ba4a00700 1 --2- 172.21.15.38:0/2063384696 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4b8406ccf0 0x7f4b8406f1b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b88000c00 tx=0x7f4b8801a040 comp rx=0 tx=0).stop 2024-03-24T00:43:25.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.163+0000 7f4ba4a00700 1 -- 172.21.15.38:0/2063384696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0102ef0 msgr2=0x7f4ba0198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:25.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.163+0000 7f4ba4a00700 1 --2- 172.21.15.38:0/2063384696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0102ef0 0x7f4ba0198f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000b4b0 tx=0x7f4b9000b870 comp rx=0 tx=0).stop 2024-03-24T00:43:25.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.163+0000 7f4ba4a00700 1 -- 172.21.15.38:0/2063384696 shutdown_connections 2024-03-24T00:43:25.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.163+0000 7f4ba4a00700 1 --2- 172.21.15.38:0/2063384696 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4b8406ccf0 0x7f4b8406f1b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:25.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.163+0000 7f4ba4a00700 1 --2- 172.21.15.38:0/2063384696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba0102ef0 0x7f4ba0198f20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:25.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.163+0000 7f4ba4a00700 1 --2- 172.21.15.38:0/2063384696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba0199460 0x7f4ba019d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:25.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.163+0000 7f4ba4a00700 1 -- 172.21.15.38:0/2063384696 >> 172.21.15.38:0/2063384696 conn(0x7f4ba00fe780 msgr2=0x7f4ba00ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:25.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.164+0000 7f4ba4a00700 1 -- 172.21.15.38:0/2063384696 shutdown_connections 2024-03-24T00:43:25.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:25.164+0000 7f4ba4a00700 1 -- 172.21.15.38:0/2063384696 wait complete. 2024-03-24T00:43:25.182 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:24 smithi088 ceph-mon[36610]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:25.869 INFO:tasks.cephadm:Setting up client nodes... 2024-03-24T00:43:25.869 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-24T00:43:26.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:25 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2063384696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T00:43:26.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:25 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2063384696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T00:43:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:27.015 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:26 smithi038 ceph-mon[29271]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:27.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:26 smithi088 ceph-mon[36610]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:28.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.463+0000 7fe39ce00700 1 -- 172.21.15.38:0/539432386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398101ce0 msgr2=0x7fe398102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:28.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.463+0000 7fe39ce00700 1 --2- 172.21.15.38:0/539432386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398101ce0 0x7fe398102140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe380009910 tx=0x7fe380009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.463+0000 7fe39ce00700 1 -- 172.21.15.38:0/539432386 shutdown_connections 2024-03-24T00:43:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.463+0000 7fe39ce00700 1 --2- 172.21.15.38:0/539432386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398101ce0 0x7fe398102140 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.463+0000 7fe39ce00700 1 --2- 172.21.15.38:0/539432386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe398100ae0 0x7fe398100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.463+0000 7fe39ce00700 1 -- 172.21.15.38:0/539432386 >> 172.21.15.38:0/539432386 conn(0x7fe3980fc0c0 msgr2=0x7fe3980fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.463+0000 7fe39ce00700 1 -- 172.21.15.38:0/539432386 shutdown_connections 2024-03-24T00:43:28.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.464+0000 7fe39ce00700 1 -- 172.21.15.38:0/539432386 wait complete. 2024-03-24T00:43:28.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.465+0000 7fe39ce00700 1 Processor -- start 2024-03-24T00:43:28.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.465+0000 7fe39ce00700 1 -- start start 2024-03-24T00:43:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.466+0000 7fe39ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398100ae0 0x7fe3981986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.466+0000 7fe39ce00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe398101ce0 0x7fe398198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.466+0000 7fe39ce00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe398199230 con 0x7fe398100ae0 2024-03-24T00:43:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.466+0000 7fe39ce00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe398199370 con 0x7fe398101ce0 2024-03-24T00:43:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.466+0000 7fe395e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398100ae0 0x7fe3981986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.466+0000 7fe395e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398100ae0 0x7fe3981986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34368/0 (socket says 172.21.15.38:34368) 2024-03-24T00:43:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.466+0000 7fe395e00700 1 -- 172.21.15.38:0/6649354 learned_addr learned my addr 172.21.15.38:0/6649354 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.466+0000 7fe38fe00700 1 --2- 172.21.15.38:0/6649354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe398101ce0 0x7fe398198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.466+0000 7fe395e00700 1 -- 172.21.15.38:0/6649354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe398101ce0 msgr2=0x7fe398198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.466+0000 7fe395e00700 1 --2- 172.21.15.38:0/6649354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe398101ce0 0x7fe398198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.466+0000 7fe395e00700 1 -- 172.21.15.38:0/6649354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3800095f0 con 0x7fe398100ae0 2024-03-24T00:43:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.467+0000 7fe395e00700 1 --2- 172.21.15.38:0/6649354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398100ae0 0x7fe3981986d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c8f0 tx=0x7fe38800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.467+0000 7fe38ea00700 1 -- 172.21.15.38:0/6649354 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe388007a70 con 0x7fe398100ae0 2024-03-24T00:43:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.467+0000 7fe38ea00700 1 -- 172.21.15.38:0/6649354 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe388007bd0 con 0x7fe398100ae0 2024-03-24T00:43:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.467+0000 7fe39ce00700 1 -- 172.21.15.38:0/6649354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39819ddd0 con 0x7fe398100ae0 2024-03-24T00:43:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.467+0000 7fe38ea00700 1 -- 172.21.15.38:0/6649354 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe388018630 con 0x7fe398100ae0 2024-03-24T00:43:28.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.467+0000 7fe39ce00700 1 -- 172.21.15.38:0/6649354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39819e320 con 0x7fe398100ae0 2024-03-24T00:43:28.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.469+0000 7fe39ce00700 1 -- 172.21.15.38:0/6649354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3981066b0 con 0x7fe398100ae0 2024-03-24T00:43:28.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.470+0000 7fe38ea00700 1 -- 172.21.15.38:0/6649354 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe38800f450 con 0x7fe398100ae0 2024-03-24T00:43:28.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.470+0000 7fe38ea00700 1 --2- 172.21.15.38:0/6649354 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe37c06ccf0 0x7fe37c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:28.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.471+0000 7fe38ea00700 1 -- 172.21.15.38:0/6649354 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe38808bff0 con 0x7fe398100ae0 2024-03-24T00:43:28.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.471+0000 7fe38fe00700 1 --2- 172.21.15.38:0/6649354 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe37c06ccf0 0x7fe37c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:28.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.471+0000 7fe38fe00700 1 --2- 172.21.15.38:0/6649354 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe37c06ccf0 0x7fe37c06f1b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3800051c0 tx=0x7fe38000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:28.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.475+0000 7fe38ea00700 1 -- 172.21.15.38:0/6649354 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe3880bf9f0 con 0x7fe398100ae0 2024-03-24T00:43:28.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.786+0000 7fe39ce00700 1 -- 172.21.15.38:0/6649354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fe39804ea80 con 0x7fe398100ae0 2024-03-24T00:43:28.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.798+0000 7fe38ea00700 1 -- 172.21.15.38:0/6649354 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fe38805a220 con 0x7fe398100ae0 2024-03-24T00:43:28.800 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-03-24T00:43:28.800 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQAwd/9l4VkILxAA2VIIPbYYLtqvkBFlzAfEQg== 2024-03-24T00:43:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.801+0000 7fe39ce00700 1 -- 172.21.15.38:0/6649354 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe37c06ccf0 msgr2=0x7fe37c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.801+0000 7fe39ce00700 1 --2- 172.21.15.38:0/6649354 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe37c06ccf0 0x7fe37c06f1b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3800051c0 tx=0x7fe38000b540 comp rx=0 tx=0).stop 2024-03-24T00:43:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.801+0000 7fe39ce00700 1 -- 172.21.15.38:0/6649354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398100ae0 msgr2=0x7fe3981986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.801+0000 7fe39ce00700 1 --2- 172.21.15.38:0/6649354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398100ae0 0x7fe3981986d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c8f0 tx=0x7fe38800ccb0 comp rx=0 tx=0).stop 2024-03-24T00:43:28.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.801+0000 7fe39ce00700 1 -- 172.21.15.38:0/6649354 shutdown_connections 2024-03-24T00:43:28.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.801+0000 7fe39ce00700 1 --2- 172.21.15.38:0/6649354 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe37c06ccf0 0x7fe37c06f1b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:28.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.801+0000 7fe39ce00700 1 --2- 172.21.15.38:0/6649354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398100ae0 0x7fe3981986d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:28.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.801+0000 7fe39ce00700 1 --2- 172.21.15.38:0/6649354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe398101ce0 0x7fe398198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:28.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.801+0000 7fe39ce00700 1 -- 172.21.15.38:0/6649354 >> 172.21.15.38:0/6649354 conn(0x7fe3980fc0c0 msgr2=0x7fe398073310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:28.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.802+0000 7fe39ce00700 1 -- 172.21.15.38:0/6649354 shutdown_connections 2024-03-24T00:43:28.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:28.802+0000 7fe39ce00700 1 -- 172.21.15.38:0/6649354 wait complete. 2024-03-24T00:43:29.664 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:43:29.664 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-24T00:43:29.664 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-24T00:43:29.831 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-24T00:43:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:29 smithi038 ceph-mon[29271]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:43:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:29 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/6649354' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T00:43:29.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:29 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/6649354' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T00:43:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:29 smithi088 ceph-mon[36610]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:43:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:29 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/6649354' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T00:43:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:29 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/6649354' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T00:43:30.266 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi088/config 2024-03-24T00:43:31.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:30 smithi038 ceph-mon[29271]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:31.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:30 smithi088 ceph-mon[36610]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.534+0000 7fbf02800700 1 -- 172.21.15.88:0/1262717892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbefc074de0 msgr2=0x7fbefc073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.534+0000 7fbf02800700 1 --2- 172.21.15.88:0/1262717892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbefc074de0 0x7fbefc073240 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009910 tx=0x7fbeec009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.534+0000 7fbf02800700 1 -- 172.21.15.88:0/1262717892 shutdown_connections 2024-03-24T00:43:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.534+0000 7fbf02800700 1 --2- 172.21.15.88:0/1262717892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc073810 0x7fbefc073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.534+0000 7fbf02800700 1 --2- 172.21.15.88:0/1262717892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbefc074de0 0x7fbefc073240 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.534+0000 7fbf02800700 1 -- 172.21.15.88:0/1262717892 >> 172.21.15.88:0/1262717892 conn(0x7fbefc0fc0b0 msgr2=0x7fbefc0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.534+0000 7fbf02800700 1 -- 172.21.15.88:0/1262717892 shutdown_connections 2024-03-24T00:43:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.534+0000 7fbf02800700 1 -- 172.21.15.88:0/1262717892 wait complete. 2024-03-24T00:43:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.535+0000 7fbf02800700 1 Processor -- start 2024-03-24T00:43:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.535+0000 7fbf02800700 1 -- start start 2024-03-24T00:43:32.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.536+0000 7fbf02800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc073810 0x7fbefc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:32.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.536+0000 7fbf02800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbefc074de0 0x7fbefc19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:32.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.536+0000 7fbf02800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc19d620 con 0x7fbefc073810 2024-03-24T00:43:32.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.536+0000 7fbf02800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc19d760 con 0x7fbefc074de0 2024-03-24T00:43:32.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.536+0000 7fbefaa00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbefc074de0 0x7fbefc19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:32.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.536+0000 7fbefaa00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbefc074de0 0x7fbefc19d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49056/0 (socket says 172.21.15.88:49056) 2024-03-24T00:43:32.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.536+0000 7fbefaa00700 1 -- 172.21.15.88:0/2297893299 learned_addr learned my addr 172.21.15.88:0/2297893299 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-03-24T00:43:32.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.536+0000 7fbefb400700 1 --2- 172.21.15.88:0/2297893299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc073810 0x7fbefc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:32.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.536+0000 7fbefaa00700 1 -- 172.21.15.88:0/2297893299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc073810 msgr2=0x7fbefc19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:32.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.536+0000 7fbefaa00700 1 --2- 172.21.15.88:0/2297893299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc073810 0x7fbefc19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:32.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.536+0000 7fbefaa00700 1 -- 172.21.15.88:0/2297893299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec0095f0 con 0x7fbefc074de0 2024-03-24T00:43:32.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.537+0000 7fbefaa00700 1 --2- 172.21.15.88:0/2297893299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbefc074de0 0x7fbefc19d000 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c8f0 tx=0x7fbee800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:32.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.537+0000 7fbef3e00700 1 -- 172.21.15.88:0/2297893299 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee8007a70 con 0x7fbefc074de0 2024-03-24T00:43:32.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.537+0000 7fbef3e00700 1 -- 172.21.15.88:0/2297893299 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbee8007bd0 con 0x7fbefc074de0 2024-03-24T00:43:32.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.537+0000 7fbef3e00700 1 -- 172.21.15.88:0/2297893299 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee8018630 con 0x7fbefc074de0 2024-03-24T00:43:32.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.537+0000 7fbf02800700 1 -- 172.21.15.88:0/2297893299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbefc1a21c0 con 0x7fbefc074de0 2024-03-24T00:43:32.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.537+0000 7fbf02800700 1 -- 172.21.15.88:0/2297893299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbefc1a2740 con 0x7fbefc074de0 2024-03-24T00:43:32.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.538+0000 7fbf02800700 1 -- 172.21.15.88:0/2297893299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbefc196c80 con 0x7fbefc074de0 2024-03-24T00:43:32.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.539+0000 7fbef3e00700 1 -- 172.21.15.88:0/2297893299 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbee800f450 con 0x7fbefc074de0 2024-03-24T00:43:32.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.540+0000 7fbef3e00700 1 --2- 172.21.15.88:0/2297893299 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbee006ccf0 0x7fbee006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:32.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.540+0000 7fbef3e00700 1 -- 172.21.15.88:0/2297893299 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbee808bfb0 con 0x7fbefc074de0 2024-03-24T00:43:32.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.541+0000 7fbefb400700 1 --2- 172.21.15.88:0/2297893299 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbee006ccf0 0x7fbee006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:32.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.542+0000 7fbefb400700 1 --2- 172.21.15.88:0/2297893299 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbee006ccf0 0x7fbee006f1b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbeec006010 tx=0x7fbeec00b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:32.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.545+0000 7fbef3e00700 1 -- 172.21.15.88:0/2297893299 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbee80bf9f0 con 0x7fbefc074de0 2024-03-24T00:43:32.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.845+0000 7fbf02800700 1 -- 172.21.15.88:0/2297893299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fbefc04ea80 con 0x7fbefc074de0 2024-03-24T00:43:32.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.855+0000 7fbef3e00700 1 -- 172.21.15.88:0/2297893299 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7fbee805a0b0 con 0x7fbefc074de0 2024-03-24T00:43:32.856 INFO:teuthology.orchestra.run.smithi088.stdout:[client.1] 2024-03-24T00:43:32.857 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQA0d/9lIgmHMhAAjpIcPIIwnyatpNoGAgDJgw== 2024-03-24T00:43:32.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.859+0000 7fbf02800700 1 -- 172.21.15.88:0/2297893299 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbee006ccf0 msgr2=0x7fbee006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:32.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.859+0000 7fbf02800700 1 --2- 172.21.15.88:0/2297893299 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbee006ccf0 0x7fbee006f1b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbeec006010 tx=0x7fbeec00b540 comp rx=0 tx=0).stop 2024-03-24T00:43:32.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.859+0000 7fbf02800700 1 -- 172.21.15.88:0/2297893299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbefc074de0 msgr2=0x7fbefc19d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:32.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.859+0000 7fbf02800700 1 --2- 172.21.15.88:0/2297893299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbefc074de0 0x7fbefc19d000 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c8f0 tx=0x7fbee800ccb0 comp rx=0 tx=0).stop 2024-03-24T00:43:32.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.859+0000 7fbf02800700 1 -- 172.21.15.88:0/2297893299 shutdown_connections 2024-03-24T00:43:32.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.859+0000 7fbf02800700 1 --2- 172.21.15.88:0/2297893299 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbee006ccf0 0x7fbee006f1b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:32.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.859+0000 7fbf02800700 1 --2- 172.21.15.88:0/2297893299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc073810 0x7fbefc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:32.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.859+0000 7fbf02800700 1 --2- 172.21.15.88:0/2297893299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbefc074de0 0x7fbefc19d000 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:32.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.859+0000 7fbf02800700 1 -- 172.21.15.88:0/2297893299 >> 172.21.15.88:0/2297893299 conn(0x7fbefc0fc0b0 msgr2=0x7fbefc102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:32.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.859+0000 7fbf02800700 1 -- 172.21.15.88:0/2297893299 shutdown_connections 2024-03-24T00:43:32.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-03-24T00:43:32.859+0000 7fbf02800700 1 -- 172.21.15.88:0/2297893299 wait complete. 2024-03-24T00:43:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:32 smithi038 ceph-mon[29271]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:33.148 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:32 smithi088 ceph-mon[36610]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:33.792 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:43:33.792 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-24T00:43:33.792 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-24T00:43:33.946 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-24T00:43:33.946 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-24T00:43:33.946 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-24T00:43:34.035 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:33 smithi038 ceph-mon[29271]: from='client.? 172.21.15.88:0/2297893299' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T00:43:34.035 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:33 smithi038 ceph-mon[29271]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T00:43:34.035 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:33 smithi038 ceph-mon[29271]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T00:43:34.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:33 smithi088 ceph-mon[36610]: from='client.? 172.21.15.88:0/2297893299' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T00:43:34.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:33 smithi088 ceph-mon[36610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T00:43:34.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:33 smithi088 ceph-mon[36610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T00:43:34.385 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:34.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:34 smithi038 ceph-mon[29271]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:35.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:34 smithi088 ceph-mon[36610]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:36.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.736+0000 7fc52ce00700 1 -- 172.21.15.38:0/2798095142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5280752e0 msgr2=0x7fc5280756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:36.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.736+0000 7fc52ce00700 1 --2- 172.21.15.38:0/2798095142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5280752e0 0x7fc5280756c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc518009910 tx=0x7fc518009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:36.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.737+0000 7fc52ce00700 1 -- 172.21.15.38:0/2798095142 shutdown_connections 2024-03-24T00:43:36.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.737+0000 7fc52ce00700 1 --2- 172.21.15.38:0/2798095142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc528075c00 0x7fc528111410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:36.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.737+0000 7fc52ce00700 1 --2- 172.21.15.38:0/2798095142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5280752e0 0x7fc5280756c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:36.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.737+0000 7fc52ce00700 1 -- 172.21.15.38:0/2798095142 >> 172.21.15.38:0/2798095142 conn(0x7fc5280fe7a0 msgr2=0x7fc528100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:36.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.737+0000 7fc52ce00700 1 -- 172.21.15.38:0/2798095142 shutdown_connections 2024-03-24T00:43:36.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.737+0000 7fc52ce00700 1 -- 172.21.15.38:0/2798095142 wait complete. 2024-03-24T00:43:36.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.738+0000 7fc52ce00700 1 Processor -- start 2024-03-24T00:43:36.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.738+0000 7fc52ce00700 1 -- start start 2024-03-24T00:43:36.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.739+0000 7fc52ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5280752e0 0x7fc52819d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:36.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.739+0000 7fc52ce00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc528075c00 0x7fc52819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:36.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.739+0000 7fc52ce00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52819dd10 con 0x7fc5280752e0 2024-03-24T00:43:36.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.739+0000 7fc52ce00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5281a19e0 con 0x7fc528075c00 2024-03-24T00:43:36.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.739+0000 7fc525e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5280752e0 0x7fc52819d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:36.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.739+0000 7fc525e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5280752e0 0x7fc52819d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58922/0 (socket says 172.21.15.38:58922) 2024-03-24T00:43:36.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.739+0000 7fc525e00700 1 -- 172.21.15.38:0/458664061 learned_addr learned my addr 172.21.15.38:0/458664061 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:36.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.739+0000 7fc525400700 1 --2- 172.21.15.38:0/458664061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc528075c00 0x7fc52819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:36.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.739+0000 7fc525e00700 1 -- 172.21.15.38:0/458664061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc528075c00 msgr2=0x7fc52819d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:36.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.739+0000 7fc525e00700 1 --2- 172.21.15.38:0/458664061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc528075c00 0x7fc52819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:36.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.739+0000 7fc525e00700 1 -- 172.21.15.38:0/458664061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5180095f0 con 0x7fc5280752e0 2024-03-24T00:43:36.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.740+0000 7fc525e00700 1 --2- 172.21.15.38:0/458664061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5280752e0 0x7fc52819d0f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5180092e0 tx=0x7fc5180056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:36.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.740+0000 7fc51ea00700 1 -- 172.21.15.38:0/458664061 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc51801d070 con 0x7fc5280752e0 2024-03-24T00:43:36.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.740+0000 7fc52ce00700 1 -- 172.21.15.38:0/458664061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5281a1cc0 con 0x7fc5280752e0 2024-03-24T00:43:36.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.740+0000 7fc51ea00700 1 -- 172.21.15.38:0/458664061 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc51800bed0 con 0x7fc5280752e0 2024-03-24T00:43:36.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.740+0000 7fc51ea00700 1 -- 172.21.15.38:0/458664061 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc518021410 con 0x7fc5280752e0 2024-03-24T00:43:36.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.740+0000 7fc52ce00700 1 -- 172.21.15.38:0/458664061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5281a2210 con 0x7fc5280752e0 2024-03-24T00:43:36.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.741+0000 7fc52ce00700 1 -- 172.21.15.38:0/458664061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc528076a20 con 0x7fc5280752e0 2024-03-24T00:43:36.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.742+0000 7fc51ea00700 1 -- 172.21.15.38:0/458664061 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc51800f460 con 0x7fc5280752e0 2024-03-24T00:43:36.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.743+0000 7fc51ea00700 1 --2- 172.21.15.38:0/458664061 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc50c06ccf0 0x7fc50c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:36.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.743+0000 7fc51ea00700 1 -- 172.21.15.38:0/458664061 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc51808d070 con 0x7fc5280752e0 2024-03-24T00:43:36.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.744+0000 7fc525400700 1 --2- 172.21.15.38:0/458664061 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc50c06ccf0 0x7fc50c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.744+0000 7fc525400700 1 --2- 172.21.15.38:0/458664061 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc50c06ccf0 0x7fc50c06f1b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc510006fd0 tx=0x7fc510008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:36.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:36.748+0000 7fc51ea00700 1 -- 172.21.15.38:0/458664061 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc518057800 con 0x7fc5280752e0 2024-03-24T00:43:37.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.030+0000 7fc52ce00700 1 -- 172.21.15.38:0/458664061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc52804ea80 con 0x7fc5280752e0 2024-03-24T00:43:37.033 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:36 smithi038 ceph-mon[29271]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:37.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.034+0000 7fc51ea00700 1 -- 172.21.15.38:0/458664061 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+174002 (secure 0 0 0) 0x7fc51805ae20 con 0x7fc5280752e0 2024-03-24T00:43:37.039 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:43:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.039+0000 7fc52ce00700 1 -- 172.21.15.38:0/458664061 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc50c06ccf0 msgr2=0x7fc50c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.039+0000 7fc52ce00700 1 --2- 172.21.15.38:0/458664061 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc50c06ccf0 0x7fc50c06f1b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc510006fd0 tx=0x7fc510008040 comp rx=0 tx=0).stop 2024-03-24T00:43:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.039+0000 7fc52ce00700 1 -- 172.21.15.38:0/458664061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5280752e0 msgr2=0x7fc52819d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.039+0000 7fc52ce00700 1 --2- 172.21.15.38:0/458664061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5280752e0 0x7fc52819d0f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5180092e0 tx=0x7fc5180056a0 comp rx=0 tx=0).stop 2024-03-24T00:43:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.039+0000 7fc52ce00700 1 -- 172.21.15.38:0/458664061 shutdown_connections 2024-03-24T00:43:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.039+0000 7fc52ce00700 1 --2- 172.21.15.38:0/458664061 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc50c06ccf0 0x7fc50c06f1b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.039+0000 7fc52ce00700 1 --2- 172.21.15.38:0/458664061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5280752e0 0x7fc52819d0f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:37.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.039+0000 7fc52ce00700 1 --2- 172.21.15.38:0/458664061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc528075c00 0x7fc52819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:37.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.039+0000 7fc52ce00700 1 -- 172.21.15.38:0/458664061 >> 172.21.15.38:0/458664061 conn(0x7fc5280fe7a0 msgr2=0x7fc5280feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:37.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.039+0000 7fc52ce00700 1 -- 172.21.15.38:0/458664061 shutdown_connections 2024-03-24T00:43:37.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:37.039+0000 7fc52ce00700 1 -- 172.21.15.38:0/458664061 wait complete. 2024-03-24T00:43:37.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:36 smithi088 ceph-mon[36610]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:37.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:37 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/458664061' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T00:43:37.933 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":19,"active_gid":14215,"active_name":"smithi038.huhioz","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3107810736},{"type":"v1","addr":"172.21.15.38:6801","nonce":3107810736}]},"active_addr":"172.21.15.38:6801/3107810736","active_change":"2024-03-24T00:38:28.565296+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14240,"name":"smithi088.njctea","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.38:8443/","prometheus":"http://172.21.15.38:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":1962732004}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":4189386253}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":633787949}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":3536812262}]}]} 2024-03-24T00:43:37.937 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-24T00:43:37.937 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-24T00:43:37.937 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T00:43:38.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:37 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/458664061' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T00:43:38.372 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:39.809 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:39 smithi038 ceph-mon[29271]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:39.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:39 smithi088 ceph-mon[36610]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:40.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.596+0000 7f7022c00700 1 -- 172.21.15.38:0/1751825073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f701c1038a0 msgr2=0x7f701c1078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:40.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.596+0000 7f7022c00700 1 --2- 172.21.15.38:0/1751825073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f701c1038a0 0x7f701c1078f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7004009910 tx=0x7f7004009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:40.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.596+0000 7f7022c00700 1 -- 172.21.15.38:0/1751825073 shutdown_connections 2024-03-24T00:43:40.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.596+0000 7f7022c00700 1 --2- 172.21.15.38:0/1751825073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f701c1038a0 0x7f701c1078f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:40.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.596+0000 7f7022c00700 1 --2- 172.21.15.38:0/1751825073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f701c102ef0 0x7f701c1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:40.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.596+0000 7f7022c00700 1 -- 172.21.15.38:0/1751825073 >> 172.21.15.38:0/1751825073 conn(0x7f701c0fe780 msgr2=0x7f701c100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:40.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.596+0000 7f7022c00700 1 -- 172.21.15.38:0/1751825073 shutdown_connections 2024-03-24T00:43:40.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.596+0000 7f7022c00700 1 -- 172.21.15.38:0/1751825073 wait complete. 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.597+0000 7f7022c00700 1 Processor -- start 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.597+0000 7f7022c00700 1 -- start start 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.598+0000 7f7022c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f701c102ef0 0x7f701c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.598+0000 7f7022c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f701c1038a0 0x7f701c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.598+0000 7f7022c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c19f530 con 0x7f701c1038a0 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.598+0000 7f7022c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c198b10 con 0x7f701c102ef0 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.598+0000 7f7013e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f701c1038a0 0x7f701c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.598+0000 7f7013e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f701c1038a0 0x7f701c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58942/0 (socket says 172.21.15.38:58942) 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.598+0000 7f7013e00700 1 -- 172.21.15.38:0/2485603626 learned_addr learned my addr 172.21.15.38:0/2485603626 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.598+0000 7f701be00700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f701c102ef0 0x7f701c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.598+0000 7f7013e00700 1 -- 172.21.15.38:0/2485603626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f701c102ef0 msgr2=0x7f701c19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.599+0000 7f7013e00700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f701c102ef0 0x7f701c19e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:40.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.599+0000 7f7013e00700 1 -- 172.21.15.38:0/2485603626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70040095f0 con 0x7f701c1038a0 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.599+0000 7f701be00700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f701c102ef0 0x7f701c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.599+0000 7f7013e00700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f701c1038a0 0x7f701c19ef10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70040092e0 tx=0x7f700400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.599+0000 7f7019600700 1 -- 172.21.15.38:0/2485603626 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f700401d070 con 0x7f701c1038a0 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.599+0000 7f7019600700 1 -- 172.21.15.38:0/2485603626 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f700400f460 con 0x7f701c1038a0 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.599+0000 7f7019600700 1 -- 172.21.15.38:0/2485603626 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7004017630 con 0x7f701c1038a0 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.599+0000 7f7022c00700 1 -- 172.21.15.38:0/2485603626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701c198d90 con 0x7f701c1038a0 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.599+0000 7f7022c00700 1 -- 172.21.15.38:0/2485603626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c199280 con 0x7f701c1038a0 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.601+0000 7f7022c00700 1 -- 172.21.15.38:0/2485603626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701c10b240 con 0x7f701c1038a0 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.602+0000 7f7019600700 1 -- 172.21.15.38:0/2485603626 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f700400f5d0 con 0x7f701c1038a0 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.602+0000 7f7019600700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f700006cce0 0x7f700006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.603+0000 7f7019600700 1 -- 172.21.15.38:0/2485603626 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f700408d310 con 0x7f701c1038a0 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.603+0000 7f701be00700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f700006cce0 0x7f700006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.603+0000 7f701be00700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f700006cce0 0x7f700006f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f700c0099d0 tx=0x7f700c008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:40.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.607+0000 7f7019600700 1 -- 172.21.15.38:0/2485603626 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f700405b490 con 0x7f701c1038a0 2024-03-24T00:43:40.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.838+0000 7f7022c00700 1 -- 172.21.15.38:0/2485603626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f701c199fd0 con 0x7f701c1038a0 2024-03-24T00:43:40.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.839+0000 7f7019600700 1 -- 172.21.15.38:0/2485603626 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11276 (secure 0 0 0) 0x7f7004005740 con 0x7f701c1038a0 2024-03-24T00:43:40.840 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:43:40.841 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":41,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","created":"2024-03-24T00:35:36.968970+0000","modified":"2024-03-24T00:43:16.800268+0000","last_up_change":"2024-03-24T00:43:15.784148+0000","last_in_change":"2024-03-24T00:42:50.726360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T00:41:31.289149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e5df40af-b5d0-4f89-94b2-59ee2e710bc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6803","nonce":667786666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6805","nonce":667786666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6809","nonce":667786666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6807","nonce":667786666}]},"public_addr":"172.21.15.38:6803/667786666","cluster_addr":"172.21.15.38:6805/667786666","heartbeat_back_addr":"172.21.15.38:6809/667786666","heartbeat_front_addr":"172.21.15.38:6807/667786666","state":["exists","up"]},{"osd":1,"uuid":"ac3c0960-dd6c-44f9-b4b0-234c10f838fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6811","nonce":723151394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6813","nonce":723151394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6817","nonce":723151394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6815","nonce":723151394}]},"public_addr":"172.21.15.38:6811/723151394","cluster_addr":"172.21.15.38:6813/723151394","heartbeat_back_addr":"172.21.15.38:6817/723151394","heartbeat_front_addr":"172.21.15.38:6815/723151394","state":["exists","up"]},{"osd":2,"uuid":"921d9dd3-2471-4b13-b6bd-01e5b6c78648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6819","nonce":643463158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6821","nonce":643463158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6825","nonce":643463158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6823","nonce":643463158}]},"public_addr":"172.21.15.38:6819/643463158","cluster_addr":"172.21.15.38:6821/643463158","heartbeat_back_addr":"172.21.15.38:6825/643463158","heartbeat_front_addr":"172.21.15.38:6823/643463158","state":["exists","up"]},{"osd":3,"uuid":"7e93813a-0036-4cea-8396-41b8b759c257","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6801","nonce":3248144048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6803","nonce":3248144048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6807","nonce":3248144048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6805","nonce":3248144048}]},"public_addr":"172.21.15.88:6801/3248144048","cluster_addr":"172.21.15.88:6803/3248144048","heartbeat_back_addr":"172.21.15.88:6807/3248144048","heartbeat_front_addr":"172.21.15.88:6805/3248144048","state":["exists","up"]},{"osd":4,"uuid":"b73f29f7-134e-4725-b341-e893b4bbff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6809","nonce":1859611126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6811","nonce":1859611126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6815","nonce":1859611126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6813","nonce":1859611126}]},"public_addr":"172.21.15.88:6809/1859611126","cluster_addr":"172.21.15.88:6811/1859611126","heartbeat_back_addr":"172.21.15.88:6815/1859611126","heartbeat_front_addr":"172.21.15.88:6813/1859611126","state":["exists","up"]},{"osd":5,"uuid":"8ea2cfb3-be7a-4f97-9993-8f9001b8b678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6817","nonce":2286313582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6819","nonce":2286313582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6823","nonce":2286313582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6821","nonce":2286313582}]},"public_addr":"172.21.15.88:6817/2286313582","cluster_addr":"172.21.15.88:6819/2286313582","heartbeat_back_addr":"172.21.15.88:6823/2286313582","heartbeat_front_addr":"172.21.15.88:6821/2286313582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:11.140794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:49.877898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:41:28.612279+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:02.338217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:36.760939+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:43:13.811948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/4049972561":"2024-03-25T00:38:28.565032+0000","172.21.15.38:6801/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6800/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6800/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/2868668356":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/3448795319":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/748221873":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/3860887425":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/913707898":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6801/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/1949387173":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6800/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6801/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/884740303":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/3845097007":"2024-03-25T00:36:55.932282+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T00:43:40.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.842+0000 7f7022c00700 1 -- 172.21.15.38:0/2485603626 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f700006cce0 msgr2=0x7f700006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:40.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.842+0000 7f7022c00700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f700006cce0 0x7f700006f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f700c0099d0 tx=0x7f700c008040 comp rx=0 tx=0).stop 2024-03-24T00:43:40.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.842+0000 7f7022c00700 1 -- 172.21.15.38:0/2485603626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f701c1038a0 msgr2=0x7f701c19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:40.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.842+0000 7f7022c00700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f701c1038a0 0x7f701c19ef10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70040092e0 tx=0x7f700400be20 comp rx=0 tx=0).stop 2024-03-24T00:43:40.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.842+0000 7f7022c00700 1 -- 172.21.15.38:0/2485603626 shutdown_connections 2024-03-24T00:43:40.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.842+0000 7f7022c00700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f700006cce0 0x7f700006f1a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:40.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.842+0000 7f7022c00700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f701c102ef0 0x7f701c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:40.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.842+0000 7f7022c00700 1 --2- 172.21.15.38:0/2485603626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f701c1038a0 0x7f701c19ef10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:40.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.842+0000 7f7022c00700 1 -- 172.21.15.38:0/2485603626 >> 172.21.15.38:0/2485603626 conn(0x7f701c0fe780 msgr2=0x7f701c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:40.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.842+0000 7f7022c00700 1 -- 172.21.15.38:0/2485603626 shutdown_connections 2024-03-24T00:43:40.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:40.843+0000 7f7022c00700 1 -- 172.21.15.38:0/2485603626 wait complete. 2024-03-24T00:43:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:40 smithi088 ceph-mon[36610]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:41.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:40 smithi038 ceph-mon[29271]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:41.573 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-24T00:43:41.574 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T00:43:42.018 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:42.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:41 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2485603626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T00:43:42.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:41 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2485603626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T00:43:43.079 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:42 smithi038 ceph-mon[29271]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:43.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:42 smithi088 ceph-mon[36610]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:44.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.312+0000 7f57a3c00700 1 -- 172.21.15.38:0/3647006731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f579c107cc0 msgr2=0x7f579c10bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:44.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.312+0000 7f57a3c00700 1 --2- 172.21.15.38:0/3647006731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f579c107cc0 0x7f579c10bd10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f578c009910 tx=0x7f578c009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:44.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.312+0000 7f57a3c00700 1 -- 172.21.15.38:0/3647006731 shutdown_connections 2024-03-24T00:43:44.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.312+0000 7f57a3c00700 1 --2- 172.21.15.38:0/3647006731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f579c107cc0 0x7f579c10bd10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:44.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.312+0000 7f57a3c00700 1 --2- 172.21.15.38:0/3647006731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f579c107310 0x7f579c1076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:44.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.312+0000 7f57a3c00700 1 -- 172.21.15.38:0/3647006731 >> 172.21.15.38:0/3647006731 conn(0x7f579c0768e0 msgr2=0x7f579c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:44.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.312+0000 7f57a3c00700 1 -- 172.21.15.38:0/3647006731 shutdown_connections 2024-03-24T00:43:44.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.312+0000 7f57a3c00700 1 -- 172.21.15.38:0/3647006731 wait complete. 2024-03-24T00:43:44.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.313+0000 7f57a3c00700 1 Processor -- start 2024-03-24T00:43:44.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.313+0000 7f57a3c00700 1 -- start start 2024-03-24T00:43:44.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f57a3c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f579c107310 0x7f579c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:44.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f57a3c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f579c107cc0 0x7f579c19d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:44.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f57a3c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c19dbe0 con 0x7f579c107310 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f57a3c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c1a18b0 con 0x7f579c107cc0 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f57a1200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f579c107310 0x7f579c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f57a1200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f579c107310 0x7f579c19cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51120/0 (socket says 172.21.15.38:51120) 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f57a1200700 1 -- 172.21.15.38:0/3726899081 learned_addr learned my addr 172.21.15.38:0/3726899081 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f5793e00700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f579c107cc0 0x7f579c19d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f57a1200700 1 -- 172.21.15.38:0/3726899081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f579c107cc0 msgr2=0x7f579c19d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f57a1200700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f579c107cc0 0x7f579c19d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f57a1200700 1 -- 172.21.15.38:0/3726899081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c0095f0 con 0x7f579c107310 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.314+0000 7f5793e00700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f579c107cc0 0x7f579c19d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.315+0000 7f57a1200700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f579c107310 0x7f579c19cfc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f578400d8d0 tx=0x7f578400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.315+0000 7f579a000700 1 -- 172.21.15.38:0/3726899081 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5784009940 con 0x7f579c107310 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.315+0000 7f57a3c00700 1 -- 172.21.15.38:0/3726899081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c1a1b90 con 0x7f579c107310 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.315+0000 7f579a000700 1 -- 172.21.15.38:0/3726899081 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5784010460 con 0x7f579c107310 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.315+0000 7f579a000700 1 -- 172.21.15.38:0/3726899081 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f578400f5f0 con 0x7f579c107310 2024-03-24T00:43:44.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.315+0000 7f57a3c00700 1 -- 172.21.15.38:0/3726899081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579c1a20e0 con 0x7f579c107310 2024-03-24T00:43:44.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.316+0000 7f57a3c00700 1 -- 172.21.15.38:0/3726899081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579c10f710 con 0x7f579c107310 2024-03-24T00:43:44.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.318+0000 7f579a000700 1 -- 172.21.15.38:0/3726899081 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f57840105d0 con 0x7f579c107310 2024-03-24T00:43:44.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.318+0000 7f579a000700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f577c06ccf0 0x7f577c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:44.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.319+0000 7f579a000700 1 -- 172.21.15.38:0/3726899081 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f578408bfb0 con 0x7f579c107310 2024-03-24T00:43:44.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.319+0000 7f5793e00700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f577c06ccf0 0x7f577c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:44.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.319+0000 7f5793e00700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f577c06ccf0 0x7f577c06f1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f578c006010 tx=0x7f578c00b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:44.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.324+0000 7f579a000700 1 -- 172.21.15.38:0/3726899081 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f57840bf9f0 con 0x7f579c107310 2024-03-24T00:43:44.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.552+0000 7f57a3c00700 1 -- 172.21.15.38:0/3726899081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f579c1087f0 con 0x7f579c107310 2024-03-24T00:43:44.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.553+0000 7f579a000700 1 -- 172.21.15.38:0/3726899081 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11276 (secure 0 0 0) 0x7f5784056620 con 0x7f579c107310 2024-03-24T00:43:44.555 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:43:44.555 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":41,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","created":"2024-03-24T00:35:36.968970+0000","modified":"2024-03-24T00:43:16.800268+0000","last_up_change":"2024-03-24T00:43:15.784148+0000","last_in_change":"2024-03-24T00:42:50.726360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T00:41:31.289149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e5df40af-b5d0-4f89-94b2-59ee2e710bc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6803","nonce":667786666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6805","nonce":667786666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6809","nonce":667786666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6807","nonce":667786666}]},"public_addr":"172.21.15.38:6803/667786666","cluster_addr":"172.21.15.38:6805/667786666","heartbeat_back_addr":"172.21.15.38:6809/667786666","heartbeat_front_addr":"172.21.15.38:6807/667786666","state":["exists","up"]},{"osd":1,"uuid":"ac3c0960-dd6c-44f9-b4b0-234c10f838fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6811","nonce":723151394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6813","nonce":723151394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6817","nonce":723151394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6815","nonce":723151394}]},"public_addr":"172.21.15.38:6811/723151394","cluster_addr":"172.21.15.38:6813/723151394","heartbeat_back_addr":"172.21.15.38:6817/723151394","heartbeat_front_addr":"172.21.15.38:6815/723151394","state":["exists","up"]},{"osd":2,"uuid":"921d9dd3-2471-4b13-b6bd-01e5b6c78648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6819","nonce":643463158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6821","nonce":643463158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6825","nonce":643463158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6823","nonce":643463158}]},"public_addr":"172.21.15.38:6819/643463158","cluster_addr":"172.21.15.38:6821/643463158","heartbeat_back_addr":"172.21.15.38:6825/643463158","heartbeat_front_addr":"172.21.15.38:6823/643463158","state":["exists","up"]},{"osd":3,"uuid":"7e93813a-0036-4cea-8396-41b8b759c257","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6801","nonce":3248144048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6803","nonce":3248144048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6807","nonce":3248144048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6805","nonce":3248144048}]},"public_addr":"172.21.15.88:6801/3248144048","cluster_addr":"172.21.15.88:6803/3248144048","heartbeat_back_addr":"172.21.15.88:6807/3248144048","heartbeat_front_addr":"172.21.15.88:6805/3248144048","state":["exists","up"]},{"osd":4,"uuid":"b73f29f7-134e-4725-b341-e893b4bbff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6809","nonce":1859611126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6811","nonce":1859611126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6815","nonce":1859611126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6813","nonce":1859611126}]},"public_addr":"172.21.15.88:6809/1859611126","cluster_addr":"172.21.15.88:6811/1859611126","heartbeat_back_addr":"172.21.15.88:6815/1859611126","heartbeat_front_addr":"172.21.15.88:6813/1859611126","state":["exists","up"]},{"osd":5,"uuid":"8ea2cfb3-be7a-4f97-9993-8f9001b8b678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6817","nonce":2286313582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6819","nonce":2286313582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6823","nonce":2286313582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6821","nonce":2286313582}]},"public_addr":"172.21.15.88:6817/2286313582","cluster_addr":"172.21.15.88:6819/2286313582","heartbeat_back_addr":"172.21.15.88:6823/2286313582","heartbeat_front_addr":"172.21.15.88:6821/2286313582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:11.140794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:49.877898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:41:28.612279+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:02.338217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:36.760939+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:43:13.811948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/4049972561":"2024-03-25T00:38:28.565032+0000","172.21.15.38:6801/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6800/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6800/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/2868668356":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/3448795319":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/748221873":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/3860887425":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/913707898":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6801/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/1949387173":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6800/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6801/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/884740303":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/3845097007":"2024-03-25T00:36:55.932282+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T00:43:44.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.556+0000 7f57a3c00700 1 -- 172.21.15.38:0/3726899081 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f577c06ccf0 msgr2=0x7f577c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:44.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.556+0000 7f57a3c00700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f577c06ccf0 0x7f577c06f1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f578c006010 tx=0x7f578c00b540 comp rx=0 tx=0).stop 2024-03-24T00:43:44.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.556+0000 7f57a3c00700 1 -- 172.21.15.38:0/3726899081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f579c107310 msgr2=0x7f579c19cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:44.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.556+0000 7f57a3c00700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f579c107310 0x7f579c19cfc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f578400d8d0 tx=0x7f578400dc90 comp rx=0 tx=0).stop 2024-03-24T00:43:44.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.557+0000 7f57a3c00700 1 -- 172.21.15.38:0/3726899081 shutdown_connections 2024-03-24T00:43:44.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.557+0000 7f57a3c00700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f577c06ccf0 0x7f577c06f1b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:44.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.557+0000 7f57a3c00700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f579c107310 0x7f579c19cfc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:44.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.557+0000 7f57a3c00700 1 --2- 172.21.15.38:0/3726899081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f579c107cc0 0x7f579c19d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:44.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.557+0000 7f57a3c00700 1 -- 172.21.15.38:0/3726899081 >> 172.21.15.38:0/3726899081 conn(0x7f579c0768e0 msgr2=0x7f579c076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:44.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.557+0000 7f57a3c00700 1 -- 172.21.15.38:0/3726899081 shutdown_connections 2024-03-24T00:43:44.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:44.557+0000 7f57a3c00700 1 -- 172.21.15.38:0/3726899081 wait complete. 2024-03-24T00:43:44.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:43:44.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:44 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3726899081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T00:43:44.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:43:44.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:44 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3726899081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T00:43:45.331 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-24T00:43:45.332 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-24T00:43:45.332 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-24T00:43:45.333 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-24T00:43:45.333 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-24T00:43:45.334 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-24T00:43:45.870 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:45.872 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:45.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:45 smithi088 ceph-mon[36610]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:45.981 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:46.052 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:45 smithi038 ceph-mon[29271]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:47.061 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:46 smithi038 ceph-mon[29271]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:47.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:46 smithi088 ceph-mon[36610]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:49.260 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:49.851 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:49 smithi038 ceph-mon[29271]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:49.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:49 smithi088 ceph-mon[36610]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:50.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:50 smithi038 ceph-mon[29271]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:50 smithi088 ceph-mon[36610]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:51.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.789+0000 7f8840800700 1 -- 172.21.15.38:0/2321927760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8838100340 msgr2=0x7f8838104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.789+0000 7f8840800700 1 --2- 172.21.15.38:0/2321927760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8838100340 0x7f8838104810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8828009910 tx=0x7f8828009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.790+0000 7f8840800700 1 -- 172.21.15.38:0/2321927760 shutdown_connections 2024-03-24T00:43:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.790+0000 7f8840800700 1 --2- 172.21.15.38:0/2321927760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8838100340 0x7f8838104810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.790+0000 7f8840800700 1 --2- 172.21.15.38:0/2321927760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88380ff990 0x7f88380ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.790+0000 7f8840800700 1 -- 172.21.15.38:0/2321927760 >> 172.21.15.38:0/2321927760 conn(0x7f88380fb480 msgr2=0x7f88380fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.790+0000 7f8840800700 1 -- 172.21.15.38:0/2321927760 shutdown_connections 2024-03-24T00:43:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.790+0000 7f8840800700 1 -- 172.21.15.38:0/2321927760 wait complete. 2024-03-24T00:43:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.791+0000 7f8840800700 1 Processor -- start 2024-03-24T00:43:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.791+0000 7f8840800700 1 -- start start 2024-03-24T00:43:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.791+0000 7f8840800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88380ff990 0x7f883819d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.791+0000 7f8840800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8838100340 0x7f883819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.791+0000 7f8840800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883819dcd0 con 0x7f88380ff990 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.791+0000 7f8840800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88381a19a0 con 0x7f8838100340 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.791+0000 7f883de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88380ff990 0x7f883819d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.792+0000 7f883de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88380ff990 0x7f883819d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51156/0 (socket says 172.21.15.38:51156) 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.792+0000 7f883de00700 1 -- 172.21.15.38:0/4169502329 learned_addr learned my addr 172.21.15.38:0/4169502329 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.792+0000 7f883d400700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8838100340 0x7f883819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.792+0000 7f883de00700 1 -- 172.21.15.38:0/4169502329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8838100340 msgr2=0x7f883819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.792+0000 7f883de00700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8838100340 0x7f883819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.792+0000 7f883de00700 1 -- 172.21.15.38:0/4169502329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88280095f0 con 0x7f88380ff990 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.792+0000 7f883d400700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8838100340 0x7f883819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.792+0000 7f883de00700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88380ff990 0x7f883819d0b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f882c00b4b0 tx=0x7f882c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.792+0000 7f8832a00700 1 -- 172.21.15.38:0/4169502329 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f882c011800 con 0x7f88380ff990 2024-03-24T00:43:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.793+0000 7f8832a00700 1 -- 172.21.15.38:0/4169502329 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f882c011e20 con 0x7f88380ff990 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.793+0000 7f8832a00700 1 -- 172.21.15.38:0/4169502329 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f882c00f2e0 con 0x7f88380ff990 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.793+0000 7f8840800700 1 -- 172.21.15.38:0/4169502329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88381a1c80 con 0x7f88380ff990 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.793+0000 7f8840800700 1 -- 172.21.15.38:0/4169502329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88381a21d0 con 0x7f88380ff990 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.793+0000 7f8840800700 1 -- 172.21.15.38:0/4169502329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8838114600 con 0x7f88380ff990 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.795+0000 7f8832a00700 1 -- 172.21.15.38:0/4169502329 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f882c010380 con 0x7f88380ff990 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.795+0000 7f8832a00700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f882006ccf0 0x7f882006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.796+0000 7f8832a00700 1 -- 172.21.15.38:0/4169502329 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f882c08ba90 con 0x7f88380ff990 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.796+0000 7f883d400700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f882006ccf0 0x7f882006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.796+0000 7f8832a00700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582] conn(0x7f88200729d0 0x7f8820074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.796+0000 7f8832a00700 1 -- 172.21.15.38:0/4169502329 --> [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88200754a0 con 0x7f88200729d0 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.796+0000 7f8832a00700 1 -- 172.21.15.38:0/4169502329 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f882c08dea0 con 0x7f88380ff990 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.796+0000 7f883d400700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f882006ccf0 0x7f882006f1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f883819e610 tx=0x7f882801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.796+0000 7f883e800700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582] conn(0x7f88200729d0 0x7f8820074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:51.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.797+0000 7f883e800700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582] conn(0x7f88200729d0 0x7f8820074df0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:51.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.800+0000 7f8832a00700 1 -- 172.21.15.38:0/4169502329 <== osd.5 v2:172.21.15.88:6816/2286313582 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f88200754a0 con 0x7f88200729d0 2024-03-24T00:43:51.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.816+0000 7f8840800700 1 -- 172.21.15.38:0/4169502329 --> [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f88380695f0 con 0x7f88200729d0 2024-03-24T00:43:51.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.817+0000 7f8832a00700 1 -- 172.21.15.38:0/4169502329 <== osd.5 v2:172.21.15.88:6816/2286313582 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f88380695f0 con 0x7f88200729d0 2024-03-24T00:43:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 -- 172.21.15.38:0/4169502329 >> [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582] conn(0x7f88200729d0 msgr2=0x7f8820074df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582] conn(0x7f88200729d0 0x7f8820074df0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 -- 172.21.15.38:0/4169502329 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f882006ccf0 msgr2=0x7f882006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f882006ccf0 0x7f882006f1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f883819e610 tx=0x7f882801a040 comp rx=0 tx=0).stop 2024-03-24T00:43:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 -- 172.21.15.38:0/4169502329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88380ff990 msgr2=0x7f883819d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88380ff990 0x7f883819d0b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f882c00b4b0 tx=0x7f882c00b870 comp rx=0 tx=0).stop 2024-03-24T00:43:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 -- 172.21.15.38:0/4169502329 shutdown_connections 2024-03-24T00:43:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.88:6816/2286313582,v1:172.21.15.88:6817/2286313582] conn(0x7f88200729d0 0x7f8820074df0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:51.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f882006ccf0 0x7f882006f1b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:51.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88380ff990 0x7f883819d0b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:51.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 --2- 172.21.15.38:0/4169502329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8838100340 0x7f883819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:51.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 -- 172.21.15.38:0/4169502329 >> 172.21.15.38:0/4169502329 conn(0x7f88380fb480 msgr2=0x7f883810e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:51.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 -- 172.21.15.38:0/4169502329 shutdown_connections 2024-03-24T00:43:51.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:51.818+0000 7f8840800700 1 -- 172.21.15.38:0/4169502329 wait complete. 2024-03-24T00:43:52.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.580+0000 7fa230800700 1 -- 172.21.15.38:0/446262842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228101c10 msgr2=0x7fa228101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.580+0000 7fa230800700 1 --2- 172.21.15.38:0/446262842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228101c10 0x7fa228101ff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009910 tx=0x7fa21c009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.580+0000 7fa230800700 1 -- 172.21.15.38:0/446262842 shutdown_connections 2024-03-24T00:43:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.580+0000 7fa230800700 1 --2- 172.21.15.38:0/446262842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa228102530 0x7fa22810aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.580+0000 7fa230800700 1 --2- 172.21.15.38:0/446262842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228101c10 0x7fa228101ff0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.580+0000 7fa230800700 1 -- 172.21.15.38:0/446262842 >> 172.21.15.38:0/446262842 conn(0x7fa228076640 msgr2=0x7fa2280fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.580+0000 7fa230800700 1 -- 172.21.15.38:0/446262842 shutdown_connections 2024-03-24T00:43:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.580+0000 7fa230800700 1 -- 172.21.15.38:0/446262842 wait complete. 2024-03-24T00:43:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.582+0000 7fa230800700 1 Processor -- start 2024-03-24T00:43:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.582+0000 7fa230800700 1 -- start start 2024-03-24T00:43:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.582+0000 7fa230800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa228101c10 0x7fa228196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.582+0000 7fa230800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228102530 0x7fa228196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.583+0000 7fa230800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa228197580 con 0x7fa228102530 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.583+0000 7fa230800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2281976f0 con 0x7fa228101c10 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.583+0000 7fa22d400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228102530 0x7fa228196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.583+0000 7fa22d400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228102530 0x7fa228196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51134/0 (socket says 172.21.15.38:51134) 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.583+0000 7fa22d400700 1 -- 172.21.15.38:0/3064473380 learned_addr learned my addr 172.21.15.38:0/3064473380 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.583+0000 7fa22de00700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa228101c10 0x7fa228196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.583+0000 7fa22d400700 1 -- 172.21.15.38:0/3064473380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa228101c10 msgr2=0x7fa228196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.583+0000 7fa22d400700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa228101c10 0x7fa228196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.584+0000 7fa22d400700 1 -- 172.21.15.38:0/3064473380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0095f0 con 0x7fa228102530 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.584+0000 7fa22d400700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228102530 0x7fa228196f80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa21800c8f0 tx=0x7fa21800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.584+0000 7fa222a00700 1 -- 172.21.15.38:0/3064473380 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2180078a0 con 0x7fa228102530 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.584+0000 7fa222a00700 1 -- 172.21.15.38:0/3064473380 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa21800f450 con 0x7fa228102530 2024-03-24T00:43:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.584+0000 7fa230800700 1 -- 172.21.15.38:0/3064473380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22819b4b0 con 0x7fa228102530 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.584+0000 7fa222a00700 1 -- 172.21.15.38:0/3064473380 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa218018630 con 0x7fa228102530 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.584+0000 7fa230800700 1 -- 172.21.15.38:0/3064473380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22819ba00 con 0x7fa228102530 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.585+0000 7fa230800700 1 -- 172.21.15.38:0/3064473380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa22810df10 con 0x7fa228102530 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.587+0000 7fa222a00700 1 -- 172.21.15.38:0/3064473380 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa21800f5c0 con 0x7fa228102530 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.588+0000 7fa222a00700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fa21006ccf0 0x7fa21006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.588+0000 7fa222a00700 1 -- 172.21.15.38:0/3064473380 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa21808c180 con 0x7fa228102530 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.588+0000 7fa22de00700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fa21006ccf0 0x7fa21006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.588+0000 7fa222a00700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126] conn(0x7fa2100729d0 0x7fa210074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.588+0000 7fa222a00700 1 -- 172.21.15.38:0/3064473380 --> [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2100754a0 con 0x7fa2100729d0 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.588+0000 7fa222a00700 1 -- 172.21.15.38:0/3064473380 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa21808c540 con 0x7fa228102530 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.589+0000 7fa22de00700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fa21006ccf0 0x7fa21006f1b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa21c006010 tx=0x7fa21c01a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.589+0000 7fa22e800700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126] conn(0x7fa2100729d0 0x7fa210074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:52.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.590+0000 7fa22e800700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126] conn(0x7fa2100729d0 0x7fa210074df0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.592+0000 7fa222a00700 1 -- 172.21.15.38:0/3064473380 <== osd.4 v2:172.21.15.88:6808/1859611126 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa2100754a0 con 0x7fa2100729d0 2024-03-24T00:43:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.609+0000 7fa230800700 1 -- 172.21.15.38:0/3064473380 --> [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa22819bec0 con 0x7fa2100729d0 2024-03-24T00:43:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.610+0000 7fa222a00700 1 -- 172.21.15.38:0/3064473380 <== osd.4 v2:172.21.15.88:6808/1859611126 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa22819bec0 con 0x7fa2100729d0 2024-03-24T00:43:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.610+0000 7fa230800700 1 -- 172.21.15.38:0/3064473380 >> [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126] conn(0x7fa2100729d0 msgr2=0x7fa210074df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.610+0000 7fa230800700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126] conn(0x7fa2100729d0 0x7fa210074df0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 -- 172.21.15.38:0/3064473380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fa21006ccf0 msgr2=0x7fa21006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fa21006ccf0 0x7fa21006f1b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa21c006010 tx=0x7fa21c01a040 comp rx=0 tx=0).stop 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 -- 172.21.15.38:0/3064473380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228102530 msgr2=0x7fa228196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228102530 0x7fa228196f80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa21800c8f0 tx=0x7fa21800cc00 comp rx=0 tx=0).stop 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 -- 172.21.15.38:0/3064473380 shutdown_connections 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.88:6808/1859611126,v1:172.21.15.88:6809/1859611126] conn(0x7fa2100729d0 0x7fa210074df0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fa21006ccf0 0x7fa21006f1b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa228101c10 0x7fa228196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 --2- 172.21.15.38:0/3064473380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228102530 0x7fa228196f80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 -- 172.21.15.38:0/3064473380 >> 172.21.15.38:0/3064473380 conn(0x7fa228076640 msgr2=0x7fa228102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 -- 172.21.15.38:0/3064473380 shutdown_connections 2024-03-24T00:43:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:50.611+0000 7fa230800700 1 -- 172.21.15.38:0/3064473380 wait complete. 2024-03-24T00:43:52.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.679+0000 7f1f22c00700 1 -- 172.21.15.38:0/3987558741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c107310 msgr2=0x7f1f1c1076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:52.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.679+0000 7f1f22c00700 1 --2- 172.21.15.38:0/3987558741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c107310 0x7f1f1c1076f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009910 tx=0x7f1f0c009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:52.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.679+0000 7f1f22c00700 1 -- 172.21.15.38:0/3987558741 shutdown_connections 2024-03-24T00:43:52.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.679+0000 7f1f22c00700 1 --2- 172.21.15.38:0/3987558741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f1c107cc0 0x7f1f1c10bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.679+0000 7f1f22c00700 1 --2- 172.21.15.38:0/3987558741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c107310 0x7f1f1c1076f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.679+0000 7f1f22c00700 1 -- 172.21.15.38:0/3987558741 >> 172.21.15.38:0/3987558741 conn(0x7f1f1c0768e0 msgr2=0x7f1f1c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:52.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.679+0000 7f1f22c00700 1 -- 172.21.15.38:0/3987558741 shutdown_connections 2024-03-24T00:43:52.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.679+0000 7f1f22c00700 1 -- 172.21.15.38:0/3987558741 wait complete. 2024-03-24T00:43:52.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.680+0000 7f1f22c00700 1 Processor -- start 2024-03-24T00:43:52.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.680+0000 7f1f22c00700 1 -- start start 2024-03-24T00:43:52.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.681+0000 7f1f22c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f1c107310 0x7f1f1c19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:52.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.681+0000 7f1f22c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c107cc0 0x7f1f1c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:52.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.681+0000 7f1f22c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c19dbc0 con 0x7f1f1c107cc0 2024-03-24T00:43:52.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.681+0000 7f1f22c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c19dd00 con 0x7f1f1c107310 2024-03-24T00:43:52.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.681+0000 7f1f1b400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c107cc0 0x7f1f1c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:52.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.681+0000 7f1f1b400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c107cc0 0x7f1f1c19d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51174/0 (socket says 172.21.15.38:51174) 2024-03-24T00:43:52.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.681+0000 7f1f1b400700 1 -- 172.21.15.38:0/2668258667 learned_addr learned my addr 172.21.15.38:0/2668258667 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:52.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.681+0000 7f1f1be00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f1c107310 0x7f1f1c19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:52.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.682+0000 7f1f1b400700 1 -- 172.21.15.38:0/2668258667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f1c107310 msgr2=0x7f1f1c19d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:52.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.682+0000 7f1f1b400700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f1c107310 0x7f1f1c19d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.682+0000 7f1f1b400700 1 -- 172.21.15.38:0/2668258667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0095f0 con 0x7f1f1c107cc0 2024-03-24T00:43:52.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.682+0000 7f1f1be00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f1c107310 0x7f1f1c19d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:43:52.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.682+0000 7f1f1b400700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c107cc0 0x7f1f1c19d570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800c960 tx=0x7f1f0800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:52.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.682+0000 7f1f18c00700 1 -- 172.21.15.38:0/2668258667 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f08007a70 con 0x7f1f1c107cc0 2024-03-24T00:43:52.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.682+0000 7f1f18c00700 1 -- 172.21.15.38:0/2668258667 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f08007bd0 con 0x7f1f1c107cc0 2024-03-24T00:43:52.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.682+0000 7f1f18c00700 1 -- 172.21.15.38:0/2668258667 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f080186c0 con 0x7f1f1c107cc0 2024-03-24T00:43:52.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.682+0000 7f1f22c00700 1 -- 172.21.15.38:0/2668258667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c1a1a40 con 0x7f1f1c107cc0 2024-03-24T00:43:52.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.683+0000 7f1f22c00700 1 -- 172.21.15.38:0/2668258667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c1a1f90 con 0x7f1f1c107cc0 2024-03-24T00:43:52.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.683+0000 7f1f22c00700 1 -- 172.21.15.38:0/2668258667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f1c1145a0 con 0x7f1f1c107cc0 2024-03-24T00:43:52.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.685+0000 7f1f18c00700 1 -- 172.21.15.38:0/2668258667 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1f0800faa0 con 0x7f1f1c107cc0 2024-03-24T00:43:52.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.686+0000 7f1f18c00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1f0006cce0 0x7f1f0006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:52.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.686+0000 7f1f18c00700 1 -- 172.21.15.38:0/2668258667 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1f0808c2f0 con 0x7f1f1c107cc0 2024-03-24T00:43:52.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.686+0000 7f1f1be00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1f0006cce0 0x7f1f0006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:52.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.686+0000 7f1f18c00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158] conn(0x7f1f000729c0 0x7f1f00074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:52.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.686+0000 7f1f18c00700 1 -- 172.21.15.38:0/2668258667 --> [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f00075490 con 0x7f1f000729c0 2024-03-24T00:43:52.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.686+0000 7f1f18c00700 1 -- 172.21.15.38:0/2668258667 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f0808c6b0 con 0x7f1f1c107cc0 2024-03-24T00:43:52.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.686+0000 7f1f20c00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158] conn(0x7f1f000729c0 0x7f1f00074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:52.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.687+0000 7f1f1be00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1f0006cce0 0x7f1f0006f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0059d0 tx=0x7f1f0c01a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:52.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.687+0000 7f1f20c00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158] conn(0x7f1f000729c0 0x7f1f00074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:52.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.688+0000 7f1f18c00700 1 -- 172.21.15.38:0/2668258667 <== osd.2 v2:172.21.15.38:6818/643463158 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f1f00075490 con 0x7f1f000729c0 2024-03-24T00:43:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.705+0000 7f1f22c00700 1 -- 172.21.15.38:0/2668258667 --> [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1f1c1a2500 con 0x7f1f000729c0 2024-03-24T00:43:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.706+0000 7f1f18c00700 1 -- 172.21.15.38:0/2668258667 <== osd.2 v2:172.21.15.38:6818/643463158 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1f1c1a2500 con 0x7f1f000729c0 2024-03-24T00:43:52.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.706+0000 7f1f22c00700 1 -- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158] conn(0x7f1f000729c0 msgr2=0x7f1f00074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:52.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.706+0000 7f1f22c00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158] conn(0x7f1f000729c0 0x7f1f00074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 -- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1f0006cce0 msgr2=0x7f1f0006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:52.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1f0006cce0 0x7f1f0006f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0059d0 tx=0x7f1f0c01a040 comp rx=0 tx=0).stop 2024-03-24T00:43:52.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 -- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c107cc0 msgr2=0x7f1f1c19d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:52.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c107cc0 0x7f1f1c19d570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800c960 tx=0x7f1f0800cd20 comp rx=0 tx=0).stop 2024-03-24T00:43:52.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 -- 172.21.15.38:0/2668258667 shutdown_connections 2024-03-24T00:43:52.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6818/643463158,v1:172.21.15.38:6819/643463158] conn(0x7f1f000729c0 0x7f1f00074de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1f0006cce0 0x7f1f0006f1a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f1c107310 0x7f1f1c19d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 --2- 172.21.15.38:0/2668258667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c107cc0 0x7f1f1c19d570 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:52.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 -- 172.21.15.38:0/2668258667 >> 172.21.15.38:0/2668258667 conn(0x7f1f1c0768e0 msgr2=0x7f1f1c076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:52.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 -- 172.21.15.38:0/2668258667 shutdown_connections 2024-03-24T00:43:52.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:52.707+0000 7f1f22c00700 1 -- 172.21.15.38:0/2668258667 wait complete. 2024-03-24T00:43:53.055 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:52 smithi038 ceph-mon[29271]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:53.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:52 smithi088 ceph-mon[36610]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:54.188 INFO:teuthology.orchestra.run.smithi038.stdout:150323855376 2024-03-24T00:43:54.188 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-24T00:43:54.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:54 smithi038 ceph-mon[29271]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:55.170 INFO:teuthology.orchestra.run.smithi038.stdout:81604378655 2024-03-24T00:43:55.170 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-24T00:43:55.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:54 smithi088 ceph-mon[36610]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:55.779 INFO:teuthology.orchestra.run.smithi038.stdout:171798691850 2024-03-24T00:43:55.779 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-24T00:43:56.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.810+0000 7fcdcf200700 1 -- 172.21.15.38:0/2692141772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc81038a0 msgr2=0x7fcdc81078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:56.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.810+0000 7fcdcf200700 1 --2- 172.21.15.38:0/2692141772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc81038a0 0x7fcdc81078f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b0a0 tx=0x7fcdb000b3b0 comp rx=0 tx=0).stop 2024-03-24T00:43:56.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.811+0000 7fcdcf200700 1 -- 172.21.15.38:0/2692141772 shutdown_connections 2024-03-24T00:43:56.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.811+0000 7fcdcf200700 1 --2- 172.21.15.38:0/2692141772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc81038a0 0x7fcdc81078f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:56.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.811+0000 7fcdcf200700 1 --2- 172.21.15.38:0/2692141772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc8102ef0 0x7fcdc81032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.811+0000 7fcdcf200700 1 -- 172.21.15.38:0/2692141772 >> 172.21.15.38:0/2692141772 conn(0x7fcdc80fe780 msgr2=0x7fcdc8100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.811+0000 7fcdcf200700 1 -- 172.21.15.38:0/2692141772 shutdown_connections 2024-03-24T00:43:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.811+0000 7fcdcf200700 1 -- 172.21.15.38:0/2692141772 wait complete. 2024-03-24T00:43:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.812+0000 7fcdcf200700 1 Processor -- start 2024-03-24T00:43:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.812+0000 7fcdcf200700 1 -- start start 2024-03-24T00:43:56.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.813+0000 7fcdcf200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc8102ef0 0x7fcdc819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:56.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.813+0000 7fcdcf200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc81038a0 0x7fcdc819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:56.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.813+0000 7fcdcf200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc819f490 con 0x7fcdc81038a0 2024-03-24T00:43:56.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.813+0000 7fcdcf200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc8198a70 con 0x7fcdc8102ef0 2024-03-24T00:43:56.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.813+0000 7fcdbf400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc81038a0 0x7fcdc819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:56.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.813+0000 7fcdbf400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc81038a0 0x7fcdc819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47946/0 (socket says 172.21.15.38:47946) 2024-03-24T00:43:56.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.813+0000 7fcdbf400700 1 -- 172.21.15.38:0/2711562255 learned_addr learned my addr 172.21.15.38:0/2711562255 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:56.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.814+0000 7fcdc7e00700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc8102ef0 0x7fcdc819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.814+0000 7fcdbf400700 1 -- 172.21.15.38:0/2711562255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc8102ef0 msgr2=0x7fcdc819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.814+0000 7fcdbf400700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc8102ef0 0x7fcdc819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.814+0000 7fcdbf400700 1 -- 172.21.15.38:0/2711562255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0009d00 con 0x7fcdc81038a0 2024-03-24T00:43:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.814+0000 7fcdbf400700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc81038a0 0x7fcdc819ee70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b070 tx=0x7fcdb0007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.814+0000 7fcdc5600700 1 -- 172.21.15.38:0/2711562255 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdb000e040 con 0x7fcdc81038a0 2024-03-24T00:43:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.815+0000 7fcdcf200700 1 -- 172.21.15.38:0/2711562255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc8198cf0 con 0x7fcdc81038a0 2024-03-24T00:43:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.815+0000 7fcdc5600700 1 -- 172.21.15.38:0/2711562255 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdb0003de0 con 0x7fcdc81038a0 2024-03-24T00:43:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.815+0000 7fcdc5600700 1 -- 172.21.15.38:0/2711562255 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdb001d6b0 con 0x7fcdc81038a0 2024-03-24T00:43:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.815+0000 7fcdcf200700 1 -- 172.21.15.38:0/2711562255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc81991e0 con 0x7fcdc81038a0 2024-03-24T00:43:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.816+0000 7fcdcf200700 1 -- 172.21.15.38:0/2711562255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcdc8110190 con 0x7fcdc81038a0 2024-03-24T00:43:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.817+0000 7fcdc5600700 1 -- 172.21.15.38:0/2711562255 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcdb0004040 con 0x7fcdc81038a0 2024-03-24T00:43:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.819+0000 7fcdc5600700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcdac06ccf0 0x7fcdac06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.819+0000 7fcdc5600700 1 -- 172.21.15.38:0/2711562255 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fcdb008d750 con 0x7fcdc81038a0 2024-03-24T00:43:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.819+0000 7fcdc7e00700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcdac06ccf0 0x7fcdac06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.819+0000 7fcdc5600700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048] conn(0x7fcdac0729d0 0x7fcdac074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.819+0000 7fcdc5600700 1 -- 172.21.15.38:0/2711562255 --> [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdac0754a0 con 0x7fcdac0729d0 2024-03-24T00:43:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.819+0000 7fcdc5600700 1 -- 172.21.15.38:0/2711562255 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb008fd80 con 0x7fcdc81038a0 2024-03-24T00:43:56.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.819+0000 7fcdcd200700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048] conn(0x7fcdac0729d0 0x7fcdac074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:56.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.819+0000 7fcdc7e00700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcdac06ccf0 0x7fcdac06f1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8005fd0 tx=0x7fcdb800a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.820+0000 7fcdcd200700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048] conn(0x7fcdac0729d0 0x7fcdac074df0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.822+0000 7fcdc5600700 1 -- 172.21.15.38:0/2711562255 <== osd.3 v2:172.21.15.88:6800/3248144048 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fcdac0754a0 con 0x7fcdac0729d0 2024-03-24T00:43:56.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.839+0000 7fcdcf200700 1 -- 172.21.15.38:0/2711562255 --> [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcdc8199640 con 0x7fcdac0729d0 2024-03-24T00:43:56.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.841+0000 7fcdc5600700 1 -- 172.21.15.38:0/2711562255 <== osd.3 v2:172.21.15.88:6800/3248144048 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcdc8199640 con 0x7fcdac0729d0 2024-03-24T00:43:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.841+0000 7fcdcf200700 1 -- 172.21.15.38:0/2711562255 >> [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048] conn(0x7fcdac0729d0 msgr2=0x7fcdac074df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.841+0000 7fcdcf200700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048] conn(0x7fcdac0729d0 0x7fcdac074df0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.841+0000 7fcdcf200700 1 -- 172.21.15.38:0/2711562255 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcdac06ccf0 msgr2=0x7fcdac06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.841+0000 7fcdcf200700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcdac06ccf0 0x7fcdac06f1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8005fd0 tx=0x7fcdb800a560 comp rx=0 tx=0).stop 2024-03-24T00:43:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.841+0000 7fcdcf200700 1 -- 172.21.15.38:0/2711562255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc81038a0 msgr2=0x7fcdc819ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.841+0000 7fcdcf200700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc81038a0 0x7fcdc819ee70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b070 tx=0x7fcdb0007bd0 comp rx=0 tx=0).stop 2024-03-24T00:43:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.841+0000 7fcdcf200700 1 -- 172.21.15.38:0/2711562255 shutdown_connections 2024-03-24T00:43:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.841+0000 7fcdcf200700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcdac06ccf0 0x7fcdac06f1b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.841+0000 7fcdcf200700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc8102ef0 0x7fcdc819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.841+0000 7fcdcf200700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.88:6800/3248144048,v1:172.21.15.88:6801/3248144048] conn(0x7fcdac0729d0 0x7fcdac074df0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.842+0000 7fcdcf200700 1 --2- 172.21.15.38:0/2711562255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdc81038a0 0x7fcdc819ee70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.842+0000 7fcdcf200700 1 -- 172.21.15.38:0/2711562255 >> 172.21.15.38:0/2711562255 conn(0x7fcdc80fe780 msgr2=0x7fcdc80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.842+0000 7fcdcf200700 1 -- 172.21.15.38:0/2711562255 shutdown_connections 2024-03-24T00:43:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:56.842+0000 7fcdcf200700 1 -- 172.21.15.38:0/2711562255 wait complete. 2024-03-24T00:43:57.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:56 smithi038 ceph-mon[29271]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:57.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:56 smithi088 ceph-mon[36610]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.816+0000 7fb608400700 1 -- 172.21.15.38:0/3237121404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb600107cc0 msgr2=0x7fb60010bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.816+0000 7fb608400700 1 --2- 172.21.15.38:0/3237121404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb600107cc0 0x7fb60010bd10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009910 tx=0x7fb5ec009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.816+0000 7fb608400700 1 -- 172.21.15.38:0/3237121404 shutdown_connections 2024-03-24T00:43:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.816+0000 7fb608400700 1 --2- 172.21.15.38:0/3237121404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb600107cc0 0x7fb60010bd10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.816+0000 7fb608400700 1 --2- 172.21.15.38:0/3237121404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb600107310 0x7fb6001076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.816+0000 7fb608400700 1 -- 172.21.15.38:0/3237121404 >> 172.21.15.38:0/3237121404 conn(0x7fb6000768e0 msgr2=0x7fb600078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:57.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.816+0000 7fb608400700 1 -- 172.21.15.38:0/3237121404 shutdown_connections 2024-03-24T00:43:57.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.816+0000 7fb608400700 1 -- 172.21.15.38:0/3237121404 wait complete. 2024-03-24T00:43:57.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.817+0000 7fb608400700 1 Processor -- start 2024-03-24T00:43:57.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.817+0000 7fb608400700 1 -- start start 2024-03-24T00:43:57.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.817+0000 7fb608400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb600107310 0x7fb60019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:57.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.817+0000 7fb608400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb600107cc0 0x7fb60019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:57.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.818+0000 7fb608400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60019dbe0 con 0x7fb600107310 2024-03-24T00:43:57.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.818+0000 7fb608400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6001a18b0 con 0x7fb600107cc0 2024-03-24T00:43:57.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.818+0000 7fb605a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb600107310 0x7fb60019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:57.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.818+0000 7fb605a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb600107310 0x7fb60019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47958/0 (socket says 172.21.15.38:47958) 2024-03-24T00:43:57.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.818+0000 7fb605a00700 1 -- 172.21.15.38:0/3967501591 learned_addr learned my addr 172.21.15.38:0/3967501591 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:57.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.818+0000 7fb605000700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb600107cc0 0x7fb60019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:57.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.818+0000 7fb605a00700 1 -- 172.21.15.38:0/3967501591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb600107cc0 msgr2=0x7fb60019d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:57.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.818+0000 7fb605a00700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb600107cc0 0x7fb60019d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:57.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.818+0000 7fb605a00700 1 -- 172.21.15.38:0/3967501591 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0095f0 con 0x7fb600107310 2024-03-24T00:43:57.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.819+0000 7fb605a00700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb600107310 0x7fb60019cfc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400d8d0 tx=0x7fb5f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:57.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.819+0000 7fb5fe000700 1 -- 172.21.15.38:0/3967501591 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f4009940 con 0x7fb600107310 2024-03-24T00:43:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.819+0000 7fb5fe000700 1 -- 172.21.15.38:0/3967501591 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5f4010460 con 0x7fb600107310 2024-03-24T00:43:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.819+0000 7fb608400700 1 -- 172.21.15.38:0/3967501591 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6001a1b90 con 0x7fb600107310 2024-03-24T00:43:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.819+0000 7fb5fe000700 1 -- 172.21.15.38:0/3967501591 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f400f5f0 con 0x7fb600107310 2024-03-24T00:43:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.819+0000 7fb608400700 1 -- 172.21.15.38:0/3967501591 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6001a20e0 con 0x7fb600107310 2024-03-24T00:43:57.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.820+0000 7fb608400700 1 -- 172.21.15.38:0/3967501591 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb6001145b0 con 0x7fb600107310 2024-03-24T00:43:57.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.821+0000 7fb5fe000700 1 -- 172.21.15.38:0/3967501591 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb5f40105d0 con 0x7fb600107310 2024-03-24T00:43:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.822+0000 7fb5fe000700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb5e806ccf0 0x7fb5e806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.822+0000 7fb5fe000700 1 -- 172.21.15.38:0/3967501591 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb5f408bfb0 con 0x7fb600107310 2024-03-24T00:43:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.822+0000 7fb605000700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb5e806ccf0 0x7fb5e806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.822+0000 7fb5fe000700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666] conn(0x7fb5e80729d0 0x7fb5e8074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.822+0000 7fb5fe000700 1 -- 172.21.15.38:0/3967501591 --> [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5e80754a0 con 0x7fb5e80729d0 2024-03-24T00:43:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.822+0000 7fb5fe000700 1 -- 172.21.15.38:0/3967501591 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5f40915a0 con 0x7fb600107310 2024-03-24T00:43:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.823+0000 7fb606400700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666] conn(0x7fb5e80729d0 0x7fb5e8074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.823+0000 7fb605000700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb5e806ccf0 0x7fb5e806f1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec006010 tx=0x7fb5ec00b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.824+0000 7fb606400700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666] conn(0x7fb5e80729d0 0x7fb5e8074df0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.824+0000 7fb5fe000700 1 -- 172.21.15.38:0/3967501591 <== osd.0 v2:172.21.15.38:6802/667786666 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb5e80754a0 con 0x7fb5e80729d0 2024-03-24T00:43:57.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.842+0000 7fb608400700 1 -- 172.21.15.38:0/3967501591 --> [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb6001087f0 con 0x7fb5e80729d0 2024-03-24T00:43:57.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.842+0000 7fb5fe000700 1 -- 172.21.15.38:0/3967501591 <== osd.0 v2:172.21.15.38:6802/667786666 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb6001087f0 con 0x7fb5e80729d0 2024-03-24T00:43:57.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 -- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666] conn(0x7fb5e80729d0 msgr2=0x7fb5e8074df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:57.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666] conn(0x7fb5e80729d0 0x7fb5e8074df0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:57.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 -- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb5e806ccf0 msgr2=0x7fb5e806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:57.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb5e806ccf0 0x7fb5e806f1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec006010 tx=0x7fb5ec00b540 comp rx=0 tx=0).stop 2024-03-24T00:43:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 -- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb600107310 msgr2=0x7fb60019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb600107310 0x7fb60019cfc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400d8d0 tx=0x7fb5f400dc90 comp rx=0 tx=0).stop 2024-03-24T00:43:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 -- 172.21.15.38:0/3967501591 shutdown_connections 2024-03-24T00:43:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6802/667786666,v1:172.21.15.38:6803/667786666] conn(0x7fb5e80729d0 0x7fb5e8074df0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb5e806ccf0 0x7fb5e806f1b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb600107310 0x7fb60019cfc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 --2- 172.21.15.38:0/3967501591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb600107cc0 0x7fb60019d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.843+0000 7fb608400700 1 -- 172.21.15.38:0/3967501591 >> 172.21.15.38:0/3967501591 conn(0x7fb6000768e0 msgr2=0x7fb600076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.844+0000 7fb608400700 1 -- 172.21.15.38:0/3967501591 shutdown_connections 2024-03-24T00:43:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:57.844+0000 7fb608400700 1 -- 172.21.15.38:0/3967501591 wait complete. 2024-03-24T00:43:58.617 INFO:teuthology.orchestra.run.smithi038.stdout:115964117016 2024-03-24T00:43:58.617 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-24T00:43:58.855 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:58.855 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:43:58.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.894+0000 7fc57d800700 1 -- 172.21.15.38:0/3317260159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5781038a0 msgr2=0x7fc5781078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:58.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.894+0000 7fc57d800700 1 --2- 172.21.15.38:0/3317260159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5781038a0 0x7fc5781078f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc564009910 tx=0x7fc564009c20 comp rx=0 tx=0).stop 2024-03-24T00:43:58.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.894+0000 7fc57d800700 1 -- 172.21.15.38:0/3317260159 shutdown_connections 2024-03-24T00:43:58.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.894+0000 7fc57d800700 1 --2- 172.21.15.38:0/3317260159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5781038a0 0x7fc5781078f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:58.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.894+0000 7fc57d800700 1 --2- 172.21.15.38:0/3317260159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc578102ef0 0x7fc5781032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:58.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.894+0000 7fc57d800700 1 -- 172.21.15.38:0/3317260159 >> 172.21.15.38:0/3317260159 conn(0x7fc5780fe780 msgr2=0x7fc578100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:58.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.894+0000 7fc57d800700 1 -- 172.21.15.38:0/3317260159 shutdown_connections 2024-03-24T00:43:58.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.895+0000 7fc57d800700 1 -- 172.21.15.38:0/3317260159 wait complete. 2024-03-24T00:43:58.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.896+0000 7fc57d800700 1 Processor -- start 2024-03-24T00:43:58.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.897+0000 7fc57d800700 1 -- start start 2024-03-24T00:43:58.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.897+0000 7fc57d800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc578102ef0 0x7fc5780752c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:58.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.897+0000 7fc57d800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5781038a0 0x7fc578075800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:58.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.897+0000 7fc57d800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc578079390 con 0x7fc578102ef0 2024-03-24T00:43:58.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.897+0000 7fc57d800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc578075d40 con 0x7fc5781038a0 2024-03-24T00:43:58.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.898+0000 7fc576a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc578102ef0 0x7fc5780752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:58.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.898+0000 7fc576a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc578102ef0 0x7fc5780752c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47976/0 (socket says 172.21.15.38:47976) 2024-03-24T00:43:58.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.898+0000 7fc576a00700 1 -- 172.21.15.38:0/1958209421 learned_addr learned my addr 172.21.15.38:0/1958209421 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:43:58.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.898+0000 7fc576000700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5781038a0 0x7fc578075800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:58.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.898+0000 7fc576a00700 1 -- 172.21.15.38:0/1958209421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5781038a0 msgr2=0x7fc578075800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:58.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.898+0000 7fc576a00700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5781038a0 0x7fc578075800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:58.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.898+0000 7fc576a00700 1 -- 172.21.15.38:0/1958209421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5640095f0 con 0x7fc578102ef0 2024-03-24T00:43:58.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.898+0000 7fc576000700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5781038a0 0x7fc578075800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:43:58.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.899+0000 7fc576a00700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc578102ef0 0x7fc5780752c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc56800e920 tx=0x7fc56800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:58.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.899+0000 7fc56f400700 1 -- 172.21.15.38:0/1958209421 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc568009790 con 0x7fc578102ef0 2024-03-24T00:43:58.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.899+0000 7fc56f400700 1 -- 172.21.15.38:0/1958209421 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc56800cba0 con 0x7fc578102ef0 2024-03-24T00:43:58.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.899+0000 7fc56f400700 1 -- 172.21.15.38:0/1958209421 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc568018650 con 0x7fc578102ef0 2024-03-24T00:43:58.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.899+0000 7fc57d800700 1 -- 172.21.15.38:0/1958209421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc578076020 con 0x7fc578102ef0 2024-03-24T00:43:58.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.900+0000 7fc57d800700 1 -- 172.21.15.38:0/1958209421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5781a7400 con 0x7fc578102ef0 2024-03-24T00:43:58.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.901+0000 7fc57d800700 1 -- 172.21.15.38:0/1958209421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc578110110 con 0x7fc578102ef0 2024-03-24T00:43:58.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.902+0000 7fc56f400700 1 -- 172.21.15.38:0/1958209421 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc56800cd10 con 0x7fc578102ef0 2024-03-24T00:43:58.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.903+0000 7fc56f400700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc55c06ccf0 0x7fc55c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:58.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.904+0000 7fc56f400700 1 -- 172.21.15.38:0/1958209421 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc568014070 con 0x7fc578102ef0 2024-03-24T00:43:58.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.904+0000 7fc56f400700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394] conn(0x7fc55c0729d0 0x7fc55c074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:43:58.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.904+0000 7fc56f400700 1 -- 172.21.15.38:0/1958209421 --> [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc55c0754a0 con 0x7fc55c0729d0 2024-03-24T00:43:58.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.904+0000 7fc56f400700 1 -- 172.21.15.38:0/1958209421 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc56808cce0 con 0x7fc578102ef0 2024-03-24T00:43:58.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.904+0000 7fc577400700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394] conn(0x7fc55c0729d0 0x7fc55c074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.905+0000 7fc576000700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc55c06ccf0 0x7fc55c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:43:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.905+0000 7fc577400700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394] conn(0x7fc55c0729d0 0x7fc55c074df0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:58.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.906+0000 7fc576000700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc55c06ccf0 0x7fc55c06f1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc578076aa0 tx=0x7fc564005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:43:58.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.906+0000 7fc56f400700 1 -- 172.21.15.38:0/1958209421 <== osd.1 v2:172.21.15.38:6810/723151394 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc55c0754a0 con 0x7fc55c0729d0 2024-03-24T00:43:58.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.925+0000 7fc57d800700 1 -- 172.21.15.38:0/1958209421 --> [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc578061430 con 0x7fc55c0729d0 2024-03-24T00:43:58.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.925+0000 7fc56f400700 1 -- 172.21.15.38:0/1958209421 <== osd.1 v2:172.21.15.38:6810/723151394 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc578061430 con 0x7fc55c0729d0 2024-03-24T00:43:58.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 -- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394] conn(0x7fc55c0729d0 msgr2=0x7fc55c074df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:58.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394] conn(0x7fc55c0729d0 0x7fc55c074df0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:58.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 -- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc55c06ccf0 msgr2=0x7fc55c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:58.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc55c06ccf0 0x7fc55c06f1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc578076aa0 tx=0x7fc564005a90 comp rx=0 tx=0).stop 2024-03-24T00:43:58.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 -- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc578102ef0 msgr2=0x7fc5780752c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:43:58.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc578102ef0 0x7fc5780752c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc56800e920 tx=0x7fc56800ece0 comp rx=0 tx=0).stop 2024-03-24T00:43:58.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 -- 172.21.15.38:0/1958209421 shutdown_connections 2024-03-24T00:43:58.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc55c06ccf0 0x7fc55c06f1b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:58.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc578102ef0 0x7fc5780752c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:58.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.38:6810/723151394,v1:172.21.15.38:6811/723151394] conn(0x7fc55c0729d0 0x7fc55c074df0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:58.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 --2- 172.21.15.38:0/1958209421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5781038a0 0x7fc578075800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:43:58.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 -- 172.21.15.38:0/1958209421 >> 172.21.15.38:0/1958209421 conn(0x7fc5780fe780 msgr2=0x7fc5780ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:43:58.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 -- 172.21.15.38:0/1958209421 shutdown_connections 2024-03-24T00:43:58.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:43:58.926+0000 7fc57d800700 1 -- 172.21.15.38:0/1958209421 wait complete. 2024-03-24T00:43:59.443 INFO:teuthology.orchestra.run.smithi038.stdout:38654705711 2024-03-24T00:43:59.444 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-24T00:43:59.695 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:43:59.696 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:43:59 smithi038 ceph-mon[29271]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:43:59.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:59 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:43:59.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:43:59 smithi088 ceph-mon[36610]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:00.199 INFO:teuthology.orchestra.run.smithi038.stdout:60129542183 2024-03-24T00:44:00.199 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-24T00:44:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:00 smithi038 ceph-mon[29271]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:01.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:00 smithi088 ceph-mon[36610]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:02.384 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:02.501 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:02.502 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:03.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:02 smithi038 ceph-mon[29271]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:03.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:02 smithi088 ceph-mon[36610]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:05.033 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:04 smithi038 ceph-mon[29271]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:04 smithi088 ceph-mon[36610]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:05.826 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:05 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3479905758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T00:44:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:05 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3479905758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T00:44:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.302+0000 7f7f02c00700 1 -- 172.21.15.38:0/305550040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7efc073a70 msgr2=0x7f7efc1113a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.302+0000 7f7f02c00700 1 --2- 172.21.15.38:0/305550040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7efc073a70 0x7f7efc1113a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009910 tx=0x7f7ee8009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.302+0000 7f7f02c00700 1 -- 172.21.15.38:0/305550040 shutdown_connections 2024-03-24T00:44:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.302+0000 7f7f02c00700 1 --2- 172.21.15.38:0/305550040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7efc073a70 0x7f7efc1113a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.302+0000 7f7f02c00700 1 --2- 172.21.15.38:0/305550040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7efc073150 0x7f7efc073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.302+0000 7f7f02c00700 1 -- 172.21.15.38:0/305550040 >> 172.21.15.38:0/305550040 conn(0x7f7efc0fc400 msgr2=0x7f7efc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:06.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.302+0000 7f7f02c00700 1 -- 172.21.15.38:0/305550040 shutdown_connections 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.302+0000 7f7f02c00700 1 -- 172.21.15.38:0/305550040 wait complete. 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.303+0000 7f7f02c00700 1 Processor -- start 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.303+0000 7f7f02c00700 1 -- start start 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.304+0000 7f7f02c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7efc073150 0x7f7efc19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.304+0000 7f7f02c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7efc073a70 0x7f7efc19d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.304+0000 7f7f02c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc19da40 con 0x7f7efc073150 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.304+0000 7f7f02c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1a1710 con 0x7f7efc073a70 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.304+0000 7f7f01800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7efc073150 0x7f7efc19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.304+0000 7f7f01800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7efc073150 0x7f7efc19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53732/0 (socket says 172.21.15.38:53732) 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.304+0000 7f7f01800700 1 -- 172.21.15.38:0/3479905758 learned_addr learned my addr 172.21.15.38:0/3479905758 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.304+0000 7f7f00e00700 1 --2- 172.21.15.38:0/3479905758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7efc073a70 0x7f7efc19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:06.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.304+0000 7f7f01800700 1 -- 172.21.15.38:0/3479905758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7efc073a70 msgr2=0x7f7efc19d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.304+0000 7f7f01800700 1 --2- 172.21.15.38:0/3479905758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7efc073a70 0x7f7efc19d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.304+0000 7f7f01800700 1 -- 172.21.15.38:0/3479905758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee80095f0 con 0x7f7efc073150 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.305+0000 7f7f01800700 1 --2- 172.21.15.38:0/3479905758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7efc073150 0x7f7efc19ce20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000c8a0 tx=0x7f7ef000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.305+0000 7f7efa000700 1 -- 172.21.15.38:0/3479905758 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef000cea0 con 0x7f7efc073150 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.305+0000 7f7efa000700 1 -- 172.21.15.38:0/3479905758 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ef0004830 con 0x7f7efc073150 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.305+0000 7f7f02c00700 1 -- 172.21.15.38:0/3479905758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc1a19f0 con 0x7f7efc073150 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.305+0000 7f7efa000700 1 -- 172.21.15.38:0/3479905758 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef00055b0 con 0x7f7efc073150 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.305+0000 7f7f02c00700 1 -- 172.21.15.38:0/3479905758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc1a1f40 con 0x7f7efc073150 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.306+0000 7f7f02c00700 1 -- 172.21.15.38:0/3479905758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc074890 con 0x7f7efc073150 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.307+0000 7f7efa000700 1 -- 172.21.15.38:0/3479905758 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7ef00049a0 con 0x7f7efc073150 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.309+0000 7f7efa000700 1 --2- 172.21.15.38:0/3479905758 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7ee406ccf0 0x7f7ee406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.309+0000 7f7efa000700 1 -- 172.21.15.38:0/3479905758 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7ef008bc50 con 0x7f7efc073150 2024-03-24T00:44:06.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.309+0000 7f7f00e00700 1 --2- 172.21.15.38:0/3479905758 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7ee406ccf0 0x7f7ee406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:06.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.310+0000 7f7f00e00700 1 --2- 172.21.15.38:0/3479905758 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7ee406ccf0 0x7f7ee406f1b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8006010 tx=0x7f7ee800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:06.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.314+0000 7f7efa000700 1 -- 172.21.15.38:0/3479905758 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7ef00562c0 con 0x7f7efc073150 2024-03-24T00:44:06.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.552+0000 7f7f02c00700 1 -- 172.21.15.38:0/3479905758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7efc04f2d0 con 0x7f7efc073150 2024-03-24T00:44:06.375 INFO:teuthology.orchestra.run.smithi038.stdout:81604378657 2024-03-24T00:44:06.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.553+0000 7f7efa000700 1 -- 172.21.15.38:0/3479905758 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ef0004c50 con 0x7f7efc073150 2024-03-24T00:44:06.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.557+0000 7f7f02c00700 1 -- 172.21.15.38:0/3479905758 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7ee406ccf0 msgr2=0x7f7ee406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:06.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.557+0000 7f7f02c00700 1 --2- 172.21.15.38:0/3479905758 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7ee406ccf0 0x7f7ee406f1b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8006010 tx=0x7f7ee800b540 comp rx=0 tx=0).stop 2024-03-24T00:44:06.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.557+0000 7f7f02c00700 1 -- 172.21.15.38:0/3479905758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7efc073150 msgr2=0x7f7efc19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:06.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.557+0000 7f7f02c00700 1 --2- 172.21.15.38:0/3479905758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7efc073150 0x7f7efc19ce20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000c8a0 tx=0x7f7ef000cc60 comp rx=0 tx=0).stop 2024-03-24T00:44:06.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.557+0000 7f7f02c00700 1 -- 172.21.15.38:0/3479905758 shutdown_connections 2024-03-24T00:44:06.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.557+0000 7f7f02c00700 1 --2- 172.21.15.38:0/3479905758 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7ee406ccf0 0x7f7ee406f1b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:06.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.557+0000 7f7f02c00700 1 --2- 172.21.15.38:0/3479905758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7efc073150 0x7f7efc19ce20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:06.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.557+0000 7f7f02c00700 1 --2- 172.21.15.38:0/3479905758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7efc073a70 0x7f7efc19d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:06.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.557+0000 7f7f02c00700 1 -- 172.21.15.38:0/3479905758 >> 172.21.15.38:0/3479905758 conn(0x7f7efc0fc400 msgr2=0x7f7efc100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:06.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.557+0000 7f7f02c00700 1 -- 172.21.15.38:0/3479905758 shutdown_connections 2024-03-24T00:44:06.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:05.557+0000 7f7f02c00700 1 -- 172.21.15.38:0/3479905758 wait complete. 2024-03-24T00:44:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:06 smithi038 ceph-mon[29271]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:06 smithi088 ceph-mon[36610]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:07.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.609+0000 7fbc04000700 1 -- 172.21.15.38:0/1838429718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbfc102550 msgr2=0x7fbbfc10aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:07.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.609+0000 7fbc04000700 1 --2- 172.21.15.38:0/1838429718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbfc102550 0x7fbbfc10aa40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009910 tx=0x7fbbe8009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:07.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.609+0000 7fbc04000700 1 -- 172.21.15.38:0/1838429718 shutdown_connections 2024-03-24T00:44:07.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.609+0000 7fbc04000700 1 --2- 172.21.15.38:0/1838429718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbfc102550 0x7fbbfc10aa40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:07.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.609+0000 7fbc04000700 1 --2- 172.21.15.38:0/1838429718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbfc101ba0 0x7fbbfc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:07.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.609+0000 7fbc04000700 1 -- 172.21.15.38:0/1838429718 >> 172.21.15.38:0/1838429718 conn(0x7fbbfc0fb480 msgr2=0x7fbbfc0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:07.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.609+0000 7fbc04000700 1 -- 172.21.15.38:0/1838429718 shutdown_connections 2024-03-24T00:44:07.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.609+0000 7fbc04000700 1 -- 172.21.15.38:0/1838429718 wait complete. 2024-03-24T00:44:07.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.610+0000 7fbc04000700 1 Processor -- start 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.610+0000 7fbc04000700 1 -- start start 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.611+0000 7fbc04000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbfc101ba0 0x7fbbfc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.611+0000 7fbc04000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbfc102550 0x7fbbfc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.611+0000 7fbc04000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc1999d0 con 0x7fbbfc101ba0 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.611+0000 7fbc04000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc19d6a0 con 0x7fbbfc102550 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.611+0000 7fbc01600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbfc101ba0 0x7fbbfc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.611+0000 7fbc01600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbfc101ba0 0x7fbbfc198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53764/0 (socket says 172.21.15.38:53764) 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.611+0000 7fbc01600700 1 -- 172.21.15.38:0/3964814997 learned_addr learned my addr 172.21.15.38:0/3964814997 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.611+0000 7fbc01600700 1 -- 172.21.15.38:0/3964814997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbfc102550 msgr2=0x7fbbfc1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.611+0000 7fbc00c00700 1 --2- 172.21.15.38:0/3964814997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbfc102550 0x7fbbfc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.611+0000 7fbc01600700 1 --2- 172.21.15.38:0/3964814997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbfc102550 0x7fbbfc1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.611+0000 7fbc01600700 1 -- 172.21.15.38:0/3964814997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe80095f0 con 0x7fbbfc101ba0 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.612+0000 7fbc01600700 1 --2- 172.21.15.38:0/3964814997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbfc101ba0 0x7fbbfc198db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000b4b0 tx=0x7fbbf000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.612+0000 7fbbfa000700 1 -- 172.21.15.38:0/3964814997 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbf0011800 con 0x7fbbfc101ba0 2024-03-24T00:44:07.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.612+0000 7fbbfa000700 1 -- 172.21.15.38:0/3964814997 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbf0011e20 con 0x7fbbfc101ba0 2024-03-24T00:44:07.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.612+0000 7fbc04000700 1 -- 172.21.15.38:0/3964814997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc19d980 con 0x7fbbfc101ba0 2024-03-24T00:44:07.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.612+0000 7fbbfa000700 1 -- 172.21.15.38:0/3964814997 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbf000f2e0 con 0x7fbbfc101ba0 2024-03-24T00:44:07.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.612+0000 7fbc04000700 1 -- 172.21.15.38:0/3964814997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc19ded0 con 0x7fbbfc101ba0 2024-03-24T00:44:07.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.613+0000 7fbc04000700 1 -- 172.21.15.38:0/3964814997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc10a5a0 con 0x7fbbfc101ba0 2024-03-24T00:44:07.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.614+0000 7fbbfa000700 1 -- 172.21.15.38:0/3964814997 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbbf0010380 con 0x7fbbfc101ba0 2024-03-24T00:44:07.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.615+0000 7fbbfa000700 1 --2- 172.21.15.38:0/3964814997 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbbe406ccf0 0x7fbbe406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:07.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.615+0000 7fbbfa000700 1 -- 172.21.15.38:0/3964814997 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbbf008bb30 con 0x7fbbfc101ba0 2024-03-24T00:44:07.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.616+0000 7fbc00c00700 1 --2- 172.21.15.38:0/3964814997 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbbe406ccf0 0x7fbbe406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:07.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.616+0000 7fbc00c00700 1 --2- 172.21.15.38:0/3964814997 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbbe406ccf0 0x7fbbe406f1b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8006010 tx=0x7fbbe800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:07.866 INFO:teuthology.orchestra.run.smithi038.stdout:171798691852 2024-03-24T00:44:07.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.620+0000 7fbbfa000700 1 -- 172.21.15.38:0/3964814997 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbbf00562d0 con 0x7fbbfc101ba0 2024-03-24T00:44:07.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.854+0000 7fbc04000700 1 -- 172.21.15.38:0/3964814997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fbbfc0695f0 con 0x7fbbfc101ba0 2024-03-24T00:44:07.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.854+0000 7fbbfa000700 1 -- 172.21.15.38:0/3964814997 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbbf00598f0 con 0x7fbbfc101ba0 2024-03-24T00:44:07.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.857+0000 7fbc04000700 1 -- 172.21.15.38:0/3964814997 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbbe406ccf0 msgr2=0x7fbbe406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:07.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.857+0000 7fbc04000700 1 --2- 172.21.15.38:0/3964814997 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbbe406ccf0 0x7fbbe406f1b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8006010 tx=0x7fbbe800b540 comp rx=0 tx=0).stop 2024-03-24T00:44:07.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.857+0000 7fbc04000700 1 -- 172.21.15.38:0/3964814997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbfc101ba0 msgr2=0x7fbbfc198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:07.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.857+0000 7fbc04000700 1 --2- 172.21.15.38:0/3964814997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbfc101ba0 0x7fbbfc198db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000b4b0 tx=0x7fbbf000b870 comp rx=0 tx=0).stop 2024-03-24T00:44:07.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.857+0000 7fbc04000700 1 -- 172.21.15.38:0/3964814997 shutdown_connections 2024-03-24T00:44:07.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.857+0000 7fbc04000700 1 --2- 172.21.15.38:0/3964814997 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbbe406ccf0 0x7fbbe406f1b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:07.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.857+0000 7fbc04000700 1 --2- 172.21.15.38:0/3964814997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbfc101ba0 0x7fbbfc198db0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:07.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.857+0000 7fbc04000700 1 --2- 172.21.15.38:0/3964814997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbfc102550 0x7fbbfc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:07.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.857+0000 7fbc04000700 1 -- 172.21.15.38:0/3964814997 >> 172.21.15.38:0/3964814997 conn(0x7fbbfc0fb480 msgr2=0x7fbbfc101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:07.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.858+0000 7fbc04000700 1 -- 172.21.15.38:0/3964814997 shutdown_connections 2024-03-24T00:44:07.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:07.858+0000 7fbc04000700 1 -- 172.21.15.38:0/3964814997 wait complete. 2024-03-24T00:44:08.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:07 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3964814997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T00:44:08.233 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:07 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3964814997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T00:44:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.346+0000 7f0092400700 1 -- 172.21.15.38:0/3292430348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f008c0752e0 msgr2=0x7f008c0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.346+0000 7f0092400700 1 --2- 172.21.15.38:0/3292430348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f008c0752e0 0x7f008c0756c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0074009910 tx=0x7f0074009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.346+0000 7f0092400700 1 -- 172.21.15.38:0/3292430348 shutdown_connections 2024-03-24T00:44:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.346+0000 7f0092400700 1 --2- 172.21.15.38:0/3292430348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f008c075c00 0x7f008c111230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.346+0000 7f0092400700 1 --2- 172.21.15.38:0/3292430348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f008c0752e0 0x7f008c0756c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.346+0000 7f0092400700 1 -- 172.21.15.38:0/3292430348 >> 172.21.15.38:0/3292430348 conn(0x7f008c0fe5e0 msgr2=0x7f008c100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.346+0000 7f0092400700 1 -- 172.21.15.38:0/3292430348 shutdown_connections 2024-03-24T00:44:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.347+0000 7f0092400700 1 -- 172.21.15.38:0/3292430348 wait complete. 2024-03-24T00:44:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.348+0000 7f0092400700 1 Processor -- start 2024-03-24T00:44:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.348+0000 7f0092400700 1 -- start start 2024-03-24T00:44:09.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.348+0000 7f0092400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f008c0752e0 0x7f008c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:09.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.349+0000 7f0092400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f008c075c00 0x7f008c19d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:09.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.349+0000 7f0092400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c19d9f0 con 0x7f008c075c00 2024-03-24T00:44:09.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.349+0000 7f0092400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c1a16c0 con 0x7f008c0752e0 2024-03-24T00:44:09.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.349+0000 7f0083e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f008c075c00 0x7f008c19d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:09.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.349+0000 7f0083e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f008c075c00 0x7f008c19d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53804/0 (socket says 172.21.15.38:53804) 2024-03-24T00:44:09.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.349+0000 7f0083e00700 1 -- 172.21.15.38:0/3374052313 learned_addr learned my addr 172.21.15.38:0/3374052313 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:09.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.349+0000 7f0091000700 1 --2- 172.21.15.38:0/3374052313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f008c0752e0 0x7f008c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:09.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.349+0000 7f0083e00700 1 -- 172.21.15.38:0/3374052313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f008c0752e0 msgr2=0x7f008c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:09.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.349+0000 7f0083e00700 1 --2- 172.21.15.38:0/3374052313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f008c0752e0 0x7f008c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:09.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.349+0000 7f0083e00700 1 -- 172.21.15.38:0/3374052313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740095f0 con 0x7f008c075c00 2024-03-24T00:44:09.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.350+0000 7f0083e00700 1 --2- 172.21.15.38:0/3374052313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f008c075c00 0x7f008c19d310 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f007c00d8d0 tx=0x7f007c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:09.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.350+0000 7f008a000700 1 -- 172.21.15.38:0/3374052313 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f007c0098a0 con 0x7f008c075c00 2024-03-24T00:44:09.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.350+0000 7f008a000700 1 -- 172.21.15.38:0/3374052313 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f007c010460 con 0x7f008c075c00 2024-03-24T00:44:09.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.350+0000 7f0092400700 1 -- 172.21.15.38:0/3374052313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c1a19a0 con 0x7f008c075c00 2024-03-24T00:44:09.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.350+0000 7f008a000700 1 -- 172.21.15.38:0/3374052313 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f007c00f5f0 con 0x7f008c075c00 2024-03-24T00:44:09.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.350+0000 7f0092400700 1 -- 172.21.15.38:0/3374052313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c1a1ef0 con 0x7f008c075c00 2024-03-24T00:44:09.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.352+0000 7f0092400700 1 -- 172.21.15.38:0/3374052313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c076a20 con 0x7f008c075c00 2024-03-24T00:44:09.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.352+0000 7f008a000700 1 -- 172.21.15.38:0/3374052313 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f007c009a00 con 0x7f008c075c00 2024-03-24T00:44:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.353+0000 7f008a000700 1 --2- 172.21.15.38:0/3374052313 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f006c06ccf0 0x7f006c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.354+0000 7f008a000700 1 -- 172.21.15.38:0/3374052313 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f007c08bdc0 con 0x7f008c075c00 2024-03-24T00:44:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.354+0000 7f0091000700 1 --2- 172.21.15.38:0/3374052313 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f006c06ccf0 0x7f006c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.354+0000 7f0091000700 1 --2- 172.21.15.38:0/3374052313 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f006c06ccf0 0x7f006c06f1b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0074009fd0 tx=0x7f0074005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.358+0000 7f008a000700 1 -- 172.21.15.38:0/3374052313 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f007c090070 con 0x7f008c075c00 2024-03-24T00:44:09.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.583+0000 7f0092400700 1 -- 172.21.15.38:0/3374052313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f008c1a2160 con 0x7f008c075c00 2024-03-24T00:44:09.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.584+0000 7f008a000700 1 -- 172.21.15.38:0/3374052313 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f007c0564b0 con 0x7f008c075c00 2024-03-24T00:44:09.584 INFO:teuthology.orchestra.run.smithi038.stdout:115964117018 2024-03-24T00:44:09.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.587+0000 7f0092400700 1 -- 172.21.15.38:0/3374052313 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f006c06ccf0 msgr2=0x7f006c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:09.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.587+0000 7f0092400700 1 --2- 172.21.15.38:0/3374052313 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f006c06ccf0 0x7f006c06f1b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0074009fd0 tx=0x7f0074005a90 comp rx=0 tx=0).stop 2024-03-24T00:44:09.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.587+0000 7f0092400700 1 -- 172.21.15.38:0/3374052313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f008c075c00 msgr2=0x7f008c19d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:09.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.587+0000 7f0092400700 1 --2- 172.21.15.38:0/3374052313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f008c075c00 0x7f008c19d310 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f007c00d8d0 tx=0x7f007c00dbe0 comp rx=0 tx=0).stop 2024-03-24T00:44:09.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.587+0000 7f0092400700 1 -- 172.21.15.38:0/3374052313 shutdown_connections 2024-03-24T00:44:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.587+0000 7f0092400700 1 --2- 172.21.15.38:0/3374052313 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f006c06ccf0 0x7f006c06f1b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.587+0000 7f0092400700 1 --2- 172.21.15.38:0/3374052313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f008c0752e0 0x7f008c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.587+0000 7f0092400700 1 --2- 172.21.15.38:0/3374052313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f008c075c00 0x7f008c19d310 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.587+0000 7f0092400700 1 -- 172.21.15.38:0/3374052313 >> 172.21.15.38:0/3374052313 conn(0x7f008c0fe5e0 msgr2=0x7f008c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.587+0000 7f0092400700 1 -- 172.21.15.38:0/3374052313 shutdown_connections 2024-03-24T00:44:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.587+0000 7f0092400700 1 -- 172.21.15.38:0/3374052313 wait complete. 2024-03-24T00:44:09.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:09 smithi088 ceph-mon[36610]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:09.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:09 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2032870680' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T00:44:09.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:09 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3374052313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T00:44:09.769 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378655 got 81604378657 for osd.2 2024-03-24T00:44:09.769 DEBUG:teuthology.parallel:result is None 2024-03-24T00:44:09.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:09 smithi038 ceph-mon[29271]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:09.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:09 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2032870680' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T00:44:09.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:09 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3374052313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T00:44:10.466 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117016 got 115964117018 for osd.3 2024-03-24T00:44:10.466 DEBUG:teuthology.parallel:result is None 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.769+0000 7fcdb8a00700 1 -- 172.21.15.38:0/4149870374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdb4102530 msgr2=0x7fcdb410aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.769+0000 7fcdb8a00700 1 --2- 172.21.15.38:0/4149870374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdb4102530 0x7fcdb410aa20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009910 tx=0x7fcda0009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.769+0000 7fcdb8a00700 1 -- 172.21.15.38:0/4149870374 shutdown_connections 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.769+0000 7fcdb8a00700 1 --2- 172.21.15.38:0/4149870374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdb4102530 0x7fcdb410aa20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.769+0000 7fcdb8a00700 1 --2- 172.21.15.38:0/4149870374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb4101c10 0x7fcdb4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.769+0000 7fcdb8a00700 1 -- 172.21.15.38:0/4149870374 >> 172.21.15.38:0/4149870374 conn(0x7fcdb4076640 msgr2=0x7fcdb40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.769+0000 7fcdb8a00700 1 -- 172.21.15.38:0/4149870374 shutdown_connections 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.769+0000 7fcdb8a00700 1 -- 172.21.15.38:0/4149870374 wait complete. 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.770+0000 7fcdb8a00700 1 Processor -- start 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.770+0000 7fcdb8a00700 1 -- start start 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.771+0000 7fcdb8a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdb4101c10 0x7fcdb419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:10.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.771+0000 7fcdb8a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb4102530 0x7fcdb419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.771+0000 7fcdb8a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb419d2c0 con 0x7fcdb4101c10 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.771+0000 7fcdb8a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4196830 con 0x7fcdb4102530 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.771+0000 7fcdb1e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdb4101c10 0x7fcdb419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.771+0000 7fcdb1e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdb4101c10 0x7fcdb419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53794/0 (socket says 172.21.15.38:53794) 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.771+0000 7fcdb1e00700 1 -- 172.21.15.38:0/2032870680 learned_addr learned my addr 172.21.15.38:0/2032870680 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.771+0000 7fcdb1400700 1 --2- 172.21.15.38:0/2032870680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb4102530 0x7fcdb419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.772+0000 7fcdb1e00700 1 -- 172.21.15.38:0/2032870680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb4102530 msgr2=0x7fcdb419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.772+0000 7fcdb1e00700 1 --2- 172.21.15.38:0/2032870680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb4102530 0x7fcdb419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.772+0000 7fcdb1e00700 1 -- 172.21.15.38:0/2032870680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda00095f0 con 0x7fcdb4101c10 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.772+0000 7fcdb1e00700 1 --2- 172.21.15.38:0/2032870680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdb4101c10 0x7fcdb419c6f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcda400d8d0 tx=0x7fcda400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.772+0000 7fcdaaa00700 1 -- 172.21.15.38:0/2032870680 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda4009940 con 0x7fcdb4101c10 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.772+0000 7fcdaaa00700 1 -- 172.21.15.38:0/2032870680 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcda4010460 con 0x7fcdb4101c10 2024-03-24T00:44:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.772+0000 7fcdaaa00700 1 -- 172.21.15.38:0/2032870680 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda400f5f0 con 0x7fcdb4101c10 2024-03-24T00:44:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.772+0000 7fcdb8a00700 1 -- 172.21.15.38:0/2032870680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb4196b10 con 0x7fcdb4101c10 2024-03-24T00:44:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.773+0000 7fcdb8a00700 1 -- 172.21.15.38:0/2032870680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb4197060 con 0x7fcdb4101c10 2024-03-24T00:44:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.774+0000 7fcdb8a00700 1 -- 172.21.15.38:0/2032870680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb4105330 con 0x7fcdb4101c10 2024-03-24T00:44:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.775+0000 7fcdaaa00700 1 -- 172.21.15.38:0/2032870680 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcda40105d0 con 0x7fcdb4101c10 2024-03-24T00:44:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.776+0000 7fcdaaa00700 1 --2- 172.21.15.38:0/2032870680 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcd9806cce0 0x7fcd9806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.776+0000 7fcdaaa00700 1 -- 172.21.15.38:0/2032870680 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fcda408bee0 con 0x7fcdb4101c10 2024-03-24T00:44:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.776+0000 7fcdb1400700 1 --2- 172.21.15.38:0/2032870680 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcd9806cce0 0x7fcd9806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.777+0000 7fcdb1400700 1 --2- 172.21.15.38:0/2032870680 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcd9806cce0 0x7fcd9806f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4197c50 tx=0x7fcda001a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:08.782+0000 7fcdaaa00700 1 -- 172.21.15.38:0/2032870680 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcda405a060 con 0x7fcdb4101c10 2024-03-24T00:44:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.014+0000 7fcdb8a00700 1 -- 172.21.15.38:0/2032870680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcdb4197cf0 con 0x7fcdb4101c10 2024-03-24T00:44:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.014+0000 7fcdaaa00700 1 -- 172.21.15.38:0/2032870680 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcda4059bf0 con 0x7fcdb4101c10 2024-03-24T00:44:10.535 INFO:teuthology.orchestra.run.smithi038.stdout:38654705713 2024-03-24T00:44:10.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.017+0000 7fcdb8a00700 1 -- 172.21.15.38:0/2032870680 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcd9806cce0 msgr2=0x7fcd9806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:10.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.017+0000 7fcdb8a00700 1 --2- 172.21.15.38:0/2032870680 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcd9806cce0 0x7fcd9806f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4197c50 tx=0x7fcda001a040 comp rx=0 tx=0).stop 2024-03-24T00:44:10.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.017+0000 7fcdb8a00700 1 -- 172.21.15.38:0/2032870680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdb4101c10 msgr2=0x7fcdb419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:10.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.017+0000 7fcdb8a00700 1 --2- 172.21.15.38:0/2032870680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdb4101c10 0x7fcdb419c6f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcda400d8d0 tx=0x7fcda400dc90 comp rx=0 tx=0).stop 2024-03-24T00:44:10.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.017+0000 7fcdb8a00700 1 -- 172.21.15.38:0/2032870680 shutdown_connections 2024-03-24T00:44:10.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.017+0000 7fcdb8a00700 1 --2- 172.21.15.38:0/2032870680 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fcd9806cce0 0x7fcd9806f1a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:10.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.017+0000 7fcdb8a00700 1 --2- 172.21.15.38:0/2032870680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdb4101c10 0x7fcdb419c6f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:10.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.017+0000 7fcdb8a00700 1 --2- 172.21.15.38:0/2032870680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb4102530 0x7fcdb419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:10.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.017+0000 7fcdb8a00700 1 -- 172.21.15.38:0/2032870680 >> 172.21.15.38:0/2032870680 conn(0x7fcdb4076640 msgr2=0x7fcdb4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:10.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.018+0000 7fcdb8a00700 1 -- 172.21.15.38:0/2032870680 shutdown_connections 2024-03-24T00:44:10.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:09.018+0000 7fcdb8a00700 1 -- 172.21.15.38:0/2032870680 wait complete. 2024-03-24T00:44:10.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:10 smithi088 ceph-mon[36610]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:11.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:10 smithi038 ceph-mon[29271]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:11.338 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705711 got 38654705713 for osd.0 2024-03-24T00:44:11.339 DEBUG:teuthology.parallel:result is None 2024-03-24T00:44:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.124+0000 7f0bf1c00700 1 -- 172.21.15.38:0/1894616310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bec073180 msgr2=0x7f0bec073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.124+0000 7f0bf1c00700 1 --2- 172.21.15.38:0/1894616310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bec073180 0x7f0bec073560 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009910 tx=0x7f0bdc009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.125+0000 7f0bf1c00700 1 -- 172.21.15.38:0/1894616310 shutdown_connections 2024-03-24T00:44:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.125+0000 7f0bf1c00700 1 --2- 172.21.15.38:0/1894616310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bec073aa0 0x7f0bec10cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.125+0000 7f0bf1c00700 1 --2- 172.21.15.38:0/1894616310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bec073180 0x7f0bec073560 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.125+0000 7f0bf1c00700 1 -- 172.21.15.38:0/1894616310 >> 172.21.15.38:0/1894616310 conn(0x7f0bec0fc5a0 msgr2=0x7f0bec0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.125+0000 7f0bf1c00700 1 -- 172.21.15.38:0/1894616310 shutdown_connections 2024-03-24T00:44:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.125+0000 7f0bf1c00700 1 -- 172.21.15.38:0/1894616310 wait complete. 2024-03-24T00:44:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.126+0000 7f0bf1c00700 1 Processor -- start 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.126+0000 7f0bf1c00700 1 -- start start 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.126+0000 7f0bf1c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bec073180 0x7f0bec19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.126+0000 7f0bf1c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bec073aa0 0x7f0bec19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.126+0000 7f0bf1c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec19f420 con 0x7f0bec073180 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.126+0000 7f0bf1c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec198a00 con 0x7f0bec073aa0 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.127+0000 7f0beaa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bec073180 0x7f0bec19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.127+0000 7f0bea000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bec073aa0 0x7f0bec19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.127+0000 7f0beaa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bec073180 0x7f0bec19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53822/0 (socket says 172.21.15.38:53822) 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.127+0000 7f0beaa00700 1 -- 172.21.15.38:0/3956318380 learned_addr learned my addr 172.21.15.38:0/3956318380 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.127+0000 7f0beaa00700 1 -- 172.21.15.38:0/3956318380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bec073aa0 msgr2=0x7f0bec19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.127+0000 7f0beaa00700 1 --2- 172.21.15.38:0/3956318380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bec073aa0 0x7f0bec19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.127+0000 7f0beaa00700 1 -- 172.21.15.38:0/3956318380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc0095f0 con 0x7f0bec073180 2024-03-24T00:44:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.127+0000 7f0bea000700 1 --2- 172.21.15.38:0/3956318380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bec073aa0 0x7f0bec19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.128+0000 7f0beaa00700 1 --2- 172.21.15.38:0/3956318380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bec073180 0x7f0bec19e8c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0092e0 tx=0x7f0bdc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.128+0000 7f0be3400700 1 -- 172.21.15.38:0/3956318380 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bdc01d070 con 0x7f0bec073180 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.128+0000 7f0be3400700 1 -- 172.21.15.38:0/3956318380 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bdc0056e0 con 0x7f0bec073180 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.128+0000 7f0bf1c00700 1 -- 172.21.15.38:0/3956318380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bec198c80 con 0x7f0bec073180 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.128+0000 7f0be3400700 1 -- 172.21.15.38:0/3956318380 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bdc00fe80 con 0x7f0bec073180 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.128+0000 7f0bf1c00700 1 -- 172.21.15.38:0/3956318380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bec199170 con 0x7f0bec073180 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.129+0000 7f0bf1c00700 1 -- 172.21.15.38:0/3956318380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bec1076e0 con 0x7f0bec073180 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.130+0000 7f0be3400700 1 -- 172.21.15.38:0/3956318380 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0bdc00f6f0 con 0x7f0bec073180 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.131+0000 7f0be3400700 1 --2- 172.21.15.38:0/3956318380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bd006ccf0 0x7f0bd006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.131+0000 7f0be3400700 1 -- 172.21.15.38:0/3956318380 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0bdc08d3b0 con 0x7f0bec073180 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.132+0000 7f0bea000700 1 --2- 172.21.15.38:0/3956318380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bd006ccf0 0x7f0bd006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.132+0000 7f0bea000700 1 --2- 172.21.15.38:0/3956318380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bd006ccf0 0x7f0bd006f1b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8005950 tx=0x7f0bd80058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.136+0000 7f0be3400700 1 -- 172.21.15.38:0/3956318380 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0bdc05b550 con 0x7f0bec073180 2024-03-24T00:44:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.367+0000 7f0bf1c00700 1 -- 172.21.15.38:0/3956318380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0bec074830 con 0x7f0bec073180 2024-03-24T00:44:11.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.368+0000 7f0be3400700 1 -- 172.21.15.38:0/3956318380 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0bdc05b0e0 con 0x7f0bec073180 2024-03-24T00:44:11.624 INFO:teuthology.orchestra.run.smithi038.stdout:60129542185 2024-03-24T00:44:11.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.372+0000 7f0bf1c00700 1 -- 172.21.15.38:0/3956318380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bd006ccf0 msgr2=0x7f0bd006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:11.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.372+0000 7f0bf1c00700 1 --2- 172.21.15.38:0/3956318380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bd006ccf0 0x7f0bd006f1b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8005950 tx=0x7f0bd80058e0 comp rx=0 tx=0).stop 2024-03-24T00:44:11.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.372+0000 7f0bf1c00700 1 -- 172.21.15.38:0/3956318380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bec073180 msgr2=0x7f0bec19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:11.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.372+0000 7f0bf1c00700 1 --2- 172.21.15.38:0/3956318380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bec073180 0x7f0bec19e8c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0092e0 tx=0x7f0bdc00be20 comp rx=0 tx=0).stop 2024-03-24T00:44:11.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.372+0000 7f0bf1c00700 1 -- 172.21.15.38:0/3956318380 shutdown_connections 2024-03-24T00:44:11.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.372+0000 7f0bf1c00700 1 --2- 172.21.15.38:0/3956318380 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bd006ccf0 0x7f0bd006f1b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:11.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.372+0000 7f0bf1c00700 1 --2- 172.21.15.38:0/3956318380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bec073180 0x7f0bec19e8c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:11.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.372+0000 7f0bf1c00700 1 --2- 172.21.15.38:0/3956318380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bec073aa0 0x7f0bec19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:11.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.372+0000 7f0bf1c00700 1 -- 172.21.15.38:0/3956318380 >> 172.21.15.38:0/3956318380 conn(0x7f0bec0fc5a0 msgr2=0x7f0bec1052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:11.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.372+0000 7f0bf1c00700 1 -- 172.21.15.38:0/3956318380 shutdown_connections 2024-03-24T00:44:11.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:11.372+0000 7f0bf1c00700 1 -- 172.21.15.38:0/3956318380 wait complete. 2024-03-24T00:44:12.092 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:11 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3956318380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T00:44:12.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:11 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3956318380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.265+0000 7f683e600700 1 -- 172.21.15.38:0/4276101985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6838107310 msgr2=0x7f68381076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.265+0000 7f683e600700 1 --2- 172.21.15.38:0/4276101985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6838107310 0x7f68381076f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6828009910 tx=0x7f6828009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.265+0000 7f683e600700 1 -- 172.21.15.38:0/4276101985 shutdown_connections 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.265+0000 7f683e600700 1 --2- 172.21.15.38:0/4276101985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6838107cc0 0x7f683810bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.265+0000 7f683e600700 1 --2- 172.21.15.38:0/4276101985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6838107310 0x7f68381076f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.265+0000 7f683e600700 1 -- 172.21.15.38:0/4276101985 >> 172.21.15.38:0/4276101985 conn(0x7f68380768e0 msgr2=0x7f6838078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.266+0000 7f683e600700 1 -- 172.21.15.38:0/4276101985 shutdown_connections 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.266+0000 7f683e600700 1 -- 172.21.15.38:0/4276101985 wait complete. 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.267+0000 7f683e600700 1 Processor -- start 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.267+0000 7f683e600700 1 -- start start 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f683e600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6838107310 0x7f683819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:12.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f683e600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6838107cc0 0x7f683819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f683e600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683819dd20 con 0x7f6838107310 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f683e600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68381a19a0 con 0x7f6838107cc0 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f6837400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6838107310 0x7f683819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f6837400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6838107310 0x7f683819d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53832/0 (socket says 172.21.15.38:53832) 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f6837400700 1 -- 172.21.15.38:0/198266122 learned_addr learned my addr 172.21.15.38:0/198266122 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f682fe00700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6838107cc0 0x7f683819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f6837400700 1 -- 172.21.15.38:0/198266122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6838107cc0 msgr2=0x7f683819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f6837400700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6838107cc0 0x7f683819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f6837400700 1 -- 172.21.15.38:0/198266122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68280095f0 con 0x7f6838107310 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.268+0000 7f682fe00700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6838107cc0 0x7f683819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.269+0000 7f6837400700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6838107310 0x7f683819d100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f68280092e0 tx=0x7f682800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.269+0000 7f6834c00700 1 -- 172.21.15.38:0/198266122 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f682801d070 con 0x7f6838107310 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.269+0000 7f6834c00700 1 -- 172.21.15.38:0/198266122 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f682800f460 con 0x7f6838107310 2024-03-24T00:44:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.269+0000 7f6834c00700 1 -- 172.21.15.38:0/198266122 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6828017630 con 0x7f6838107310 2024-03-24T00:44:12.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.269+0000 7f683e600700 1 -- 172.21.15.38:0/198266122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68381a1c20 con 0x7f6838107310 2024-03-24T00:44:12.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.269+0000 7f683e600700 1 -- 172.21.15.38:0/198266122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68381a2110 con 0x7f6838107310 2024-03-24T00:44:12.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.271+0000 7f683e600700 1 -- 172.21.15.38:0/198266122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683810f6b0 con 0x7f6838107310 2024-03-24T00:44:12.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.272+0000 7f6834c00700 1 -- 172.21.15.38:0/198266122 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f682800f5d0 con 0x7f6838107310 2024-03-24T00:44:12.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.273+0000 7f6834c00700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f681c06ccf0 0x7f681c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:12.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.273+0000 7f6834c00700 1 -- 172.21.15.38:0/198266122 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f682808d4b0 con 0x7f6838107310 2024-03-24T00:44:12.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.273+0000 7f682fe00700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f681c06ccf0 0x7f681c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:12.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.274+0000 7f682fe00700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f681c06ccf0 0x7f681c06f1b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f683819e610 tx=0x7f6820008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:12.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.278+0000 7f6834c00700 1 -- 172.21.15.38:0/198266122 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6828057b20 con 0x7f6838107310 2024-03-24T00:44:12.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.506+0000 7f683e600700 1 -- 172.21.15.38:0/198266122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f683819e3c0 con 0x7f6838107310 2024-03-24T00:44:12.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.506+0000 7f6834c00700 1 -- 172.21.15.38:0/198266122 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f682805b140 con 0x7f6838107310 2024-03-24T00:44:12.506 INFO:teuthology.orchestra.run.smithi038.stdout:150323855380 2024-03-24T00:44:12.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.509+0000 7f683e600700 1 -- 172.21.15.38:0/198266122 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f681c06ccf0 msgr2=0x7f681c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:12.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.509+0000 7f683e600700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f681c06ccf0 0x7f681c06f1b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f683819e610 tx=0x7f6820008040 comp rx=0 tx=0).stop 2024-03-24T00:44:12.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.509+0000 7f683e600700 1 -- 172.21.15.38:0/198266122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6838107310 msgr2=0x7f683819d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:12.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.509+0000 7f683e600700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6838107310 0x7f683819d100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f68280092e0 tx=0x7f682800be20 comp rx=0 tx=0).stop 2024-03-24T00:44:12.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.509+0000 7f683e600700 1 -- 172.21.15.38:0/198266122 shutdown_connections 2024-03-24T00:44:12.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.509+0000 7f683e600700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f681c06ccf0 0x7f681c06f1b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:12.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.509+0000 7f683e600700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6838107310 0x7f683819d100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:12.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.509+0000 7f683e600700 1 --2- 172.21.15.38:0/198266122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6838107cc0 0x7f683819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:12.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.509+0000 7f683e600700 1 -- 172.21.15.38:0/198266122 >> 172.21.15.38:0/198266122 conn(0x7f68380768e0 msgr2=0x7f6838077c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:12.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.509+0000 7f683e600700 1 -- 172.21.15.38:0/198266122 shutdown_connections 2024-03-24T00:44:12.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:12.509+0000 7f683e600700 1 -- 172.21.15.38:0/198266122 wait complete. 2024-03-24T00:44:12.845 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691850 got 171798691852 for osd.5 2024-03-24T00:44:12.845 DEBUG:teuthology.parallel:result is None 2024-03-24T00:44:12.929 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542183 got 60129542185 for osd.1 2024-03-24T00:44:12.929 DEBUG:teuthology.parallel:result is None 2024-03-24T00:44:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:12 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/198266122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T00:44:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:12 smithi038 ceph-mon[29271]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:13.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:12 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/198266122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T00:44:13.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:12 smithi088 ceph-mon[36610]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:13.718 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855376 got 150323855380 for osd.4 2024-03-24T00:44:13.718 DEBUG:teuthology.parallel:result is None 2024-03-24T00:44:13.718 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-24T00:44:13.718 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-24T00:44:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:14.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:44:14.952 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:44:15.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:15 smithi088 ceph-mon[36610]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:15.938 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:15 smithi038 ceph-mon[29271]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:16.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.283+0000 7f0997400700 1 -- 172.21.15.38:0/299094959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0990102ef0 msgr2=0x7f09901032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:16.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.283+0000 7f0997400700 1 --2- 172.21.15.38:0/299094959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0990102ef0 0x7f09901032d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0980009910 tx=0x7f0980009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:16.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.284+0000 7f0997400700 1 -- 172.21.15.38:0/299094959 shutdown_connections 2024-03-24T00:44:16.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.284+0000 7f0997400700 1 --2- 172.21.15.38:0/299094959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09901038a0 0x7f09901078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:16.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.284+0000 7f0997400700 1 --2- 172.21.15.38:0/299094959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0990102ef0 0x7f09901032d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:16.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.284+0000 7f0997400700 1 -- 172.21.15.38:0/299094959 >> 172.21.15.38:0/299094959 conn(0x7f09900fe780 msgr2=0x7f0990100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:16.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.284+0000 7f0997400700 1 -- 172.21.15.38:0/299094959 shutdown_connections 2024-03-24T00:44:16.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.284+0000 7f0997400700 1 -- 172.21.15.38:0/299094959 wait complete. 2024-03-24T00:44:16.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.285+0000 7f0997400700 1 Processor -- start 2024-03-24T00:44:16.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.285+0000 7f0997400700 1 -- start start 2024-03-24T00:44:16.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.285+0000 7f0997400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0990102ef0 0x7f099019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:16.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.285+0000 7f0997400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09901038a0 0x7f099019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:16.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.285+0000 7f0997400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099019f550 con 0x7f0990102ef0 2024-03-24T00:44:16.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.285+0000 7f0997400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0990198b30 con 0x7f09901038a0 2024-03-24T00:44:16.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.286+0000 7f0994a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0990102ef0 0x7f099019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:16.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.286+0000 7f0994a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0990102ef0 0x7f099019e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52354/0 (socket says 172.21.15.38:52354) 2024-03-24T00:44:16.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.286+0000 7f0994a00700 1 -- 172.21.15.38:0/705391093 learned_addr learned my addr 172.21.15.38:0/705391093 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:16.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.286+0000 7f0987e00700 1 --2- 172.21.15.38:0/705391093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09901038a0 0x7f099019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:16.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.286+0000 7f0994a00700 1 -- 172.21.15.38:0/705391093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09901038a0 msgr2=0x7f099019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:16.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.286+0000 7f0994a00700 1 --2- 172.21.15.38:0/705391093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09901038a0 0x7f099019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:16.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.286+0000 7f0994a00700 1 -- 172.21.15.38:0/705391093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09800095f0 con 0x7f0990102ef0 2024-03-24T00:44:16.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.286+0000 7f0994a00700 1 --2- 172.21.15.38:0/705391093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0990102ef0 0x7f099019e9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0980006010 tx=0x7f098000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:16.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.287+0000 7f098e000700 1 -- 172.21.15.38:0/705391093 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f098001d070 con 0x7f0990102ef0 2024-03-24T00:44:16.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.287+0000 7f098e000700 1 -- 172.21.15.38:0/705391093 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f098000f460 con 0x7f0990102ef0 2024-03-24T00:44:16.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.287+0000 7f0997400700 1 -- 172.21.15.38:0/705391093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0990198db0 con 0x7f0990102ef0 2024-03-24T00:44:16.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.287+0000 7f098e000700 1 -- 172.21.15.38:0/705391093 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0980017630 con 0x7f0990102ef0 2024-03-24T00:44:16.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.287+0000 7f0997400700 1 -- 172.21.15.38:0/705391093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09901992a0 con 0x7f0990102ef0 2024-03-24T00:44:16.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.288+0000 7f0997400700 1 -- 172.21.15.38:0/705391093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099010b290 con 0x7f0990102ef0 2024-03-24T00:44:16.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.289+0000 7f098e000700 1 -- 172.21.15.38:0/705391093 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f098000f5d0 con 0x7f0990102ef0 2024-03-24T00:44:16.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.290+0000 7f098e000700 1 --2- 172.21.15.38:0/705391093 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f097406ccf0 0x7f097406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:16.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.291+0000 7f098e000700 1 -- 172.21.15.38:0/705391093 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f098008d4f0 con 0x7f0990102ef0 2024-03-24T00:44:16.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.291+0000 7f0987e00700 1 --2- 172.21.15.38:0/705391093 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f097406ccf0 0x7f097406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:16.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.292+0000 7f0987e00700 1 --2- 172.21.15.38:0/705391093 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f097406ccf0 0x7f097406f1b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0978006fd0 tx=0x7f0978008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:16.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.296+0000 7f098e000700 1 -- 172.21.15.38:0/705391093 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f098005b6b0 con 0x7f0990102ef0 2024-03-24T00:44:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.514+0000 7f0997400700 1 -- 172.21.15.38:0/705391093 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0990199ff0 con 0x7f097406ccf0 2024-03-24T00:44:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.518+0000 7f098e000700 1 -- 172.21.15.38:0/705391093 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27324 (secure 0 0 0) 0x7f0990199ff0 con 0x7f097406ccf0 2024-03-24T00:44:16.519 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:44:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.521+0000 7f0997400700 1 -- 172.21.15.38:0/705391093 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f097406ccf0 msgr2=0x7f097406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.521+0000 7f0997400700 1 --2- 172.21.15.38:0/705391093 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f097406ccf0 0x7f097406f1b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0978006fd0 tx=0x7f0978008040 comp rx=0 tx=0).stop 2024-03-24T00:44:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.521+0000 7f0997400700 1 -- 172.21.15.38:0/705391093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0990102ef0 msgr2=0x7f099019e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.521+0000 7f0997400700 1 --2- 172.21.15.38:0/705391093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0990102ef0 0x7f099019e9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0980006010 tx=0x7f098000be20 comp rx=0 tx=0).stop 2024-03-24T00:44:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.521+0000 7f0997400700 1 -- 172.21.15.38:0/705391093 shutdown_connections 2024-03-24T00:44:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.521+0000 7f0997400700 1 --2- 172.21.15.38:0/705391093 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f097406ccf0 0x7f097406f1b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.521+0000 7f0997400700 1 --2- 172.21.15.38:0/705391093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0990102ef0 0x7f099019e9f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.521+0000 7f0997400700 1 --2- 172.21.15.38:0/705391093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09901038a0 0x7f099019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.521+0000 7f0997400700 1 -- 172.21.15.38:0/705391093 >> 172.21.15.38:0/705391093 conn(0x7f09900fe780 msgr2=0x7f09900feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.521+0000 7f0997400700 1 -- 172.21.15.38:0/705391093 shutdown_connections 2024-03-24T00:44:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:16.521+0000 7f0997400700 1 -- 172.21.15.38:0/705391093 wait complete. 2024-03-24T00:44:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-03-24T00:44:17.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:16 smithi038 ceph-mon[29271]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T00:44:17.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:16 smithi038 ceph-mon[29271]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:17.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:16 smithi088 ceph-mon[36610]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T00:44:17.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:16 smithi088 ceph-mon[36610]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:17.385 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-03-24T00:44:14.670720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":166048,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":562304864,"statfs":{"total":575970213888,"available":575800180736,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":165927590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003642"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-24T00:43:15.793771+0000","last_change":"2024-03-24T00:42:37.859318+0000","last_active":"2024-03-24T00:43:15.793771+0000","last_peered":"2024-03-24T00:43:15.793771+0000","last_clean":"2024-03-24T00:43:15.793771+0000","last_became_active":"2024-03-24T00:42:37.858420+0000","last_became_peered":"2024-03-24T00:42:37.858420+0000","last_unstale":"2024-03-24T00:43:15.793771+0000","last_undegraded":"2024-03-24T00:43:15.793771+0000","last_fullsized":"2024-03-24T00:43:15.793771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T00:41:31.721282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T00:41:31.721282+0000","last_clean_scrub_stamp":"2024-03-24T00:41:31.721282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:34:41.600123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]}]},{"osd":4,"up_from":35,"seq":150323855381,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 00:43:38 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.88},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1}]},{"osd":1,"last update":"Sun Mar 24 00:43:38 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.105}]},{"osd":2,"last update":"Sun Mar 24 00:43:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.92200000000000004}]},{"osd":3,"last update":"Sun Mar 24 00:43:38 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.044},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.22}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":3,"up_from":27,"seq":115964117020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 00:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.90500000000000003,"15min":0.89100000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.363,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.95299999999999996,"15min":0.96699999999999997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.389,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.81799999999999995}]},{"osd":1,"last update":"Sun Mar 24 00:44:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.96299999999999997,"15min":0.95599999999999996},"min":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98899999999999999,"15min":0.98999999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":1.226}]},{"osd":2,"last update":"Sun Mar 24 00:44:11 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.75,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3149999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.008,"15min":0.99399999999999999},"min":{"1min":0.78000000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0069999999999999}]},{"osd":4,"last update":"Sun Mar 24 00:43:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]}]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 00:43:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.79900000000000004,"15min":0.77800000000000002},"min":{"1min":0.73399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.81999999999999995,"15min":0.80200000000000005},"min":{"1min":0.74199999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.0029999999999999}]},{"osd":1,"last update":"Sun Mar 24 00:43:32 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.90900000000000003,"15min":0.89000000000000001},"min":{"1min":0.76500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.93799999999999994,"15min":0.91200000000000003},"min":{"1min":0.79900000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":1.1040000000000001}]},{"osd":3,"last update":"Sun Mar 24 00:44:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0680000000000001,"15min":1.0640000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.577},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.1639999999999999,"15min":1.147},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.262}]},{"osd":4,"last update":"Sun Mar 24 00:43:39 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.532}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]}]},{"osd":0,"up_from":9,"seq":38654705715,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Mar 24 00:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.72899999999999998,"15min":0.67500000000000004},"min":{"1min":0.498,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.117,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.042,"5min":0.83799999999999997,"15min":0.77400000000000002},"min":{"1min":0.48599999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.68400000000000005}]},{"osd":2,"last update":"Sun Mar 24 00:43:35 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.94599999999999995,"15min":0.93000000000000005},"min":{"1min":0.72899999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.052,"5min":0.97199999999999998,"15min":0.95899999999999996},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.76900000000000002}]},{"osd":3,"last update":"Sun Mar 24 00:44:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0209999999999999,"15min":1.014},"min":{"1min":0.751,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.135,"15min":1.1100000000000001},"min":{"1min":0.93100000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.0589999999999999}]},{"osd":4,"last update":"Sun Mar 24 00:43:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.127},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 00:44:01 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.80400000000000005,"15min":0.76700000000000002},"min":{"1min":0.498,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3839999999999999,"5min":1.694,"15min":1.694},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.81499999999999995,"15min":0.75900000000000001},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.2370000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Sun Mar 24 00:43:34 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":0.98299999999999998,"15min":0.96299999999999997},"min":{"1min":0.65700000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.147},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.99299999999999999,"15min":0.98799999999999999},"min":{"1min":0.57799999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.395,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.103}]},{"osd":3,"last update":"Sun Mar 24 00:44:03 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.0780000000000001,"15min":1.0680000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.202},{"interface":"front","average":{"1min":1.248,"5min":1.177,"15min":1.165},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":6.4459999999999997}]},{"osd":4,"last update":"Sun Mar 24 00:43:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.25},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":6.5049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T00:44:17.385 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-24T00:44:17.823 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:19.764 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:19 smithi038 ceph-mon[29271]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:19.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:19 smithi088 ceph-mon[36610]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.939+0000 7fdeca200700 1 -- 172.21.15.38:0/2620369423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdec4073b00 msgr2=0x7fdec410cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.939+0000 7fdeca200700 1 --2- 172.21.15.38:0/2620369423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdec4073b00 0x7fdec410cd70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009910 tx=0x7fdeb0009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.939+0000 7fdeca200700 1 -- 172.21.15.38:0/2620369423 shutdown_connections 2024-03-24T00:44:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.939+0000 7fdeca200700 1 --2- 172.21.15.38:0/2620369423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdec4073b00 0x7fdec410cd70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.939+0000 7fdeca200700 1 --2- 172.21.15.38:0/2620369423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40731e0 0x7fdec40735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.939+0000 7fdeca200700 1 -- 172.21.15.38:0/2620369423 >> 172.21.15.38:0/2620369423 conn(0x7fdec40fc580 msgr2=0x7fdec40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.940+0000 7fdeca200700 1 -- 172.21.15.38:0/2620369423 shutdown_connections 2024-03-24T00:44:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.940+0000 7fdeca200700 1 -- 172.21.15.38:0/2620369423 wait complete. 2024-03-24T00:44:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.941+0000 7fdeca200700 1 Processor -- start 2024-03-24T00:44:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.941+0000 7fdeca200700 1 -- start start 2024-03-24T00:44:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.942+0000 7fdeca200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40731e0 0x7fdec4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.942+0000 7fdeca200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdec4073b00 0x7fdec41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.942+0000 7fdeca200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4199880 con 0x7fdec4073b00 2024-03-24T00:44:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.942+0000 7fdeca200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec419d490 con 0x7fdec40731e0 2024-03-24T00:44:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.942+0000 7fdec2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdec4073b00 0x7fdec41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.942+0000 7fdec2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdec4073b00 0x7fdec41991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52372/0 (socket says 172.21.15.38:52372) 2024-03-24T00:44:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.942+0000 7fdec2a00700 1 -- 172.21.15.38:0/3869440149 learned_addr learned my addr 172.21.15.38:0/3869440149 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.942+0000 7fdec3400700 1 --2- 172.21.15.38:0/3869440149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40731e0 0x7fdec4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.943+0000 7fdec2a00700 1 -- 172.21.15.38:0/3869440149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40731e0 msgr2=0x7fdec4198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.943+0000 7fdec2a00700 1 --2- 172.21.15.38:0/3869440149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40731e0 0x7fdec4198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.943+0000 7fdec2a00700 1 -- 172.21.15.38:0/3869440149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb00095f0 con 0x7fdec4073b00 2024-03-24T00:44:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.943+0000 7fdec2a00700 1 --2- 172.21.15.38:0/3869440149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdec4073b00 0x7fdec41991a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0006010 tx=0x7fdeb000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:19.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.943+0000 7fdebbe00700 1 -- 172.21.15.38:0/3869440149 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeb001d070 con 0x7fdec4073b00 2024-03-24T00:44:19.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.943+0000 7fdebbe00700 1 -- 172.21.15.38:0/3869440149 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdeb000f460 con 0x7fdec4073b00 2024-03-24T00:44:19.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.943+0000 7fdeca200700 1 -- 172.21.15.38:0/3869440149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec419d710 con 0x7fdec4073b00 2024-03-24T00:44:19.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.943+0000 7fdebbe00700 1 -- 172.21.15.38:0/3869440149 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeb0017630 con 0x7fdec4073b00 2024-03-24T00:44:19.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.944+0000 7fdeca200700 1 -- 172.21.15.38:0/3869440149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec419dc00 con 0x7fdec4073b00 2024-03-24T00:44:19.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.945+0000 7fdeca200700 1 -- 172.21.15.38:0/3869440149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec410a5b0 con 0x7fdec4073b00 2024-03-24T00:44:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.946+0000 7fdebbe00700 1 -- 172.21.15.38:0/3869440149 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdeb000f5d0 con 0x7fdec4073b00 2024-03-24T00:44:19.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.946+0000 7fdebbe00700 1 --2- 172.21.15.38:0/3869440149 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fdea806ccf0 0x7fdea806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:19.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.947+0000 7fdebbe00700 1 -- 172.21.15.38:0/3869440149 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdeb008d4d0 con 0x7fdec4073b00 2024-03-24T00:44:19.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.947+0000 7fdec3400700 1 --2- 172.21.15.38:0/3869440149 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fdea806ccf0 0x7fdea806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:19.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.948+0000 7fdec3400700 1 --2- 172.21.15.38:0/3869440149 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fdea806ccf0 0x7fdea806f1b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4005950 tx=0x7fdeb4005ed0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:19.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:19.952+0000 7fdebbe00700 1 -- 172.21.15.38:0/3869440149 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdeb0057c00 con 0x7fdec4073b00 2024-03-24T00:44:20.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.178+0000 7fdeca200700 1 -- 172.21.15.38:0/3869440149 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdec4199eb0 con 0x7fdea806ccf0 2024-03-24T00:44:20.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.181+0000 7fdebbe00700 1 -- 172.21.15.38:0/3869440149 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+29708 (secure 0 0 0) 0x7fdec4199eb0 con 0x7fdea806ccf0 2024-03-24T00:44:20.181 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:44:20.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.183+0000 7fdeca200700 1 -- 172.21.15.38:0/3869440149 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fdea806ccf0 msgr2=0x7fdea806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:20.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.183+0000 7fdeca200700 1 --2- 172.21.15.38:0/3869440149 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fdea806ccf0 0x7fdea806f1b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4005950 tx=0x7fdeb4005ed0 comp rx=0 tx=0).stop 2024-03-24T00:44:20.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.183+0000 7fdeca200700 1 -- 172.21.15.38:0/3869440149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdec4073b00 msgr2=0x7fdec41991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:20.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.183+0000 7fdeca200700 1 --2- 172.21.15.38:0/3869440149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdec4073b00 0x7fdec41991a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0006010 tx=0x7fdeb000be20 comp rx=0 tx=0).stop 2024-03-24T00:44:20.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.184+0000 7fdeca200700 1 -- 172.21.15.38:0/3869440149 shutdown_connections 2024-03-24T00:44:20.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.184+0000 7fdeca200700 1 --2- 172.21.15.38:0/3869440149 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fdea806ccf0 0x7fdea806f1b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:20.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.184+0000 7fdeca200700 1 --2- 172.21.15.38:0/3869440149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40731e0 0x7fdec4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:20.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.184+0000 7fdeca200700 1 --2- 172.21.15.38:0/3869440149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdec4073b00 0x7fdec41991a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:20.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.184+0000 7fdeca200700 1 -- 172.21.15.38:0/3869440149 >> 172.21.15.38:0/3869440149 conn(0x7fdec40fc580 msgr2=0x7fdec41076f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:20.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.184+0000 7fdeca200700 1 -- 172.21.15.38:0/3869440149 shutdown_connections 2024-03-24T00:44:20.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:20.184+0000 7fdeca200700 1 -- 172.21.15.38:0/3869440149 wait complete. 2024-03-24T00:44:20.185 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-03-24T00:44:20.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:44:20.992 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:44:21.056 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-03-24T00:44:18.671683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":166048,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":562304864,"statfs":{"total":575970213888,"available":575800180736,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":165927590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003266"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-24T00:43:15.793771+0000","last_change":"2024-03-24T00:42:37.859318+0000","last_active":"2024-03-24T00:43:15.793771+0000","last_peered":"2024-03-24T00:43:15.793771+0000","last_clean":"2024-03-24T00:43:15.793771+0000","last_became_active":"2024-03-24T00:42:37.858420+0000","last_became_peered":"2024-03-24T00:42:37.858420+0000","last_unstale":"2024-03-24T00:43:15.793771+0000","last_undegraded":"2024-03-24T00:43:15.793771+0000","last_fullsized":"2024-03-24T00:43:15.793771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T00:41:31.721282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T00:41:31.721282+0000","last_clean_scrub_stamp":"2024-03-24T00:41:31.721282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:34:41.600123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 00:44:16 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.042}]},{"osd":1,"last update":"Sun Mar 24 00:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.91600000000000004}]},{"osd":2,"last update":"Sun Mar 24 00:44:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.115},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.099}]},{"osd":3,"last update":"Sun Mar 24 00:44:16 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.1819999999999999}]},{"osd":4,"last update":"Sun Mar 24 00:44:16 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.216},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.244}]}]},{"osd":4,"up_from":35,"seq":150323855382,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 00:43:38 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.1459999999999999}]},{"osd":1,"last update":"Sun Mar 24 00:43:38 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.95499999999999996}]},{"osd":2,"last update":"Sun Mar 24 00:43:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.238},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.02}]},{"osd":3,"last update":"Sun Mar 24 00:43:38 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.284},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.0660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]}]},{"osd":3,"up_from":27,"seq":115964117020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 00:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.90500000000000003,"15min":0.89100000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.363,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.95299999999999996,"15min":0.96699999999999997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.389,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.81799999999999995}]},{"osd":1,"last update":"Sun Mar 24 00:44:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.96299999999999997,"15min":0.95599999999999996},"min":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98899999999999999,"15min":0.98999999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":1.226}]},{"osd":2,"last update":"Sun Mar 24 00:44:11 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.75,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3149999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.008,"15min":0.99399999999999999},"min":{"1min":0.78000000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0069999999999999}]},{"osd":4,"last update":"Sun Mar 24 00:43:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]}]},{"osd":2,"up_from":19,"seq":81604378660,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 00:43:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.79900000000000004,"15min":0.77800000000000002},"min":{"1min":0.73399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.81999999999999995,"15min":0.80200000000000005},"min":{"1min":0.74199999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.76800000000000002}]},{"osd":1,"last update":"Sun Mar 24 00:43:32 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.90900000000000003,"15min":0.89000000000000001},"min":{"1min":0.76500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.93799999999999994,"15min":0.91200000000000003},"min":{"1min":0.79900000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.82199999999999995}]},{"osd":3,"last update":"Sun Mar 24 00:44:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0680000000000001,"15min":1.0640000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.212},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.1639999999999999,"15min":1.147},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.111}]},{"osd":4,"last update":"Sun Mar 24 00:43:39 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.375}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]}]},{"osd":0,"up_from":9,"seq":38654705716,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Mar 24 00:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.72899999999999998,"15min":0.67500000000000004},"min":{"1min":0.498,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.117,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.042,"5min":0.83799999999999997,"15min":0.77400000000000002},"min":{"1min":0.48599999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.873}]},{"osd":2,"last update":"Sun Mar 24 00:43:35 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.94599999999999995,"15min":0.93000000000000005},"min":{"1min":0.72899999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.052,"5min":0.97199999999999998,"15min":0.95899999999999996},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.98899999999999999}]},{"osd":3,"last update":"Sun Mar 24 00:44:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0209999999999999,"15min":1.014},"min":{"1min":0.751,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.135,"15min":1.1100000000000001},"min":{"1min":0.93100000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.413}]},{"osd":4,"last update":"Sun Mar 24 00:43:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":1.2330000000000001}]},{"osd":5,"last update":"Sun Mar 24 00:44:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":1.05,"5min":1.05,"15min":1.05},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.284},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.1579999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 00:44:01 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.80400000000000005,"15min":0.76700000000000002},"min":{"1min":0.498,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3839999999999999,"5min":1.694,"15min":1.694},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.81499999999999995,"15min":0.75900000000000001},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.2370000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.88200000000000001}]},{"osd":2,"last update":"Sun Mar 24 00:43:34 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":0.98299999999999998,"15min":0.96299999999999997},"min":{"1min":0.65700000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.99299999999999999,"15min":0.98799999999999999},"min":{"1min":0.57799999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.395,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.1160000000000001}]},{"osd":3,"last update":"Sun Mar 24 00:44:03 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.0780000000000001,"15min":1.0680000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.22},{"interface":"front","average":{"1min":1.248,"5min":1.177,"15min":1.165},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.095}]},{"osd":4,"last update":"Sun Mar 24 00:43:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":1.361}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T00:44:21.057 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-24T00:44:21.057 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-24T00:44:21.057 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-24T00:44:21.057 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-24T00:44:21.475 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:21.911 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:21 smithi038 ceph-mon[29271]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T00:44:21.911 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:21 smithi038 ceph-mon[29271]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:21.911 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:21 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:44:21.911 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:21 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:44:21.911 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:21 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:21 smithi088 ceph-mon[36610]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T00:44:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:21 smithi088 ceph-mon[36610]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:21 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:44:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:21 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:44:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:21 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:23.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:22 smithi038 ceph-mon[29271]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:23.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:22 smithi088 ceph-mon[36610]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.680+0000 7f2891400700 1 -- 172.21.15.38:0/4081710377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f288c102590 msgr2=0x7f288c10aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.680+0000 7f2891400700 1 --2- 172.21.15.38:0/4081710377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f288c102590 0x7f288c10aa80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2874009910 tx=0x7f2874009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.681+0000 7f2891400700 1 -- 172.21.15.38:0/4081710377 shutdown_connections 2024-03-24T00:44:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.681+0000 7f2891400700 1 --2- 172.21.15.38:0/4081710377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f288c102590 0x7f288c10aa80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.681+0000 7f2891400700 1 --2- 172.21.15.38:0/4081710377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288c101be0 0x7f288c101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.681+0000 7f2891400700 1 -- 172.21.15.38:0/4081710377 >> 172.21.15.38:0/4081710377 conn(0x7f288c0fb500 msgr2=0x7f288c0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.681+0000 7f2891400700 1 -- 172.21.15.38:0/4081710377 shutdown_connections 2024-03-24T00:44:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.681+0000 7f2891400700 1 -- 172.21.15.38:0/4081710377 wait complete. 2024-03-24T00:44:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.682+0000 7f2891400700 1 Processor -- start 2024-03-24T00:44:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.682+0000 7f2891400700 1 -- start start 2024-03-24T00:44:23.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.682+0000 7f2891400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288c101be0 0x7f288c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:23.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.682+0000 7f2891400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f288c199370 0x7f288c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:23.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.682+0000 7f2891400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c199990 con 0x7f288c199370 2024-03-24T00:44:23.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.682+0000 7f2891400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c199b00 con 0x7f288c101be0 2024-03-24T00:44:23.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.683+0000 7f2883e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f288c199370 0x7f288c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.683+0000 7f2883e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f288c199370 0x7f288c19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51562/0 (socket says 172.21.15.38:51562) 2024-03-24T00:44:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.683+0000 7f2883e00700 1 -- 172.21.15.38:0/543484253 learned_addr learned my addr 172.21.15.38:0/543484253 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.683+0000 7f288a800700 1 --2- 172.21.15.38:0/543484253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288c101be0 0x7f288c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.683+0000 7f2883e00700 1 -- 172.21.15.38:0/543484253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288c101be0 msgr2=0x7f288c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.683+0000 7f2883e00700 1 --2- 172.21.15.38:0/543484253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288c101be0 0x7f288c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.683+0000 7f2883e00700 1 -- 172.21.15.38:0/543484253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28740095f0 con 0x7f288c199370 2024-03-24T00:44:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.683+0000 7f2883e00700 1 --2- 172.21.15.38:0/543484253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f288c199370 0x7f288c19d720 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2874005b40 tx=0x7f287400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.684+0000 7f2883400700 1 -- 172.21.15.38:0/543484253 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f287401d070 con 0x7f288c199370 2024-03-24T00:44:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.684+0000 7f2883400700 1 -- 172.21.15.38:0/543484253 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2874005700 con 0x7f288c199370 2024-03-24T00:44:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.684+0000 7f2883400700 1 -- 172.21.15.38:0/543484253 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f287400fd20 con 0x7f288c199370 2024-03-24T00:44:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.684+0000 7f2891400700 1 -- 172.21.15.38:0/543484253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288c19dcc0 con 0x7f288c199370 2024-03-24T00:44:23.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.684+0000 7f2891400700 1 -- 172.21.15.38:0/543484253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c19e210 con 0x7f288c199370 2024-03-24T00:44:23.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.685+0000 7f2891400700 1 -- 172.21.15.38:0/543484253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288c10a5e0 con 0x7f288c199370 2024-03-24T00:44:23.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.686+0000 7f2883400700 1 -- 172.21.15.38:0/543484253 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f287400f560 con 0x7f288c199370 2024-03-24T00:44:23.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.687+0000 7f2883400700 1 --2- 172.21.15.38:0/543484253 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f287006ccf0 0x7f287006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:23.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.687+0000 7f2883400700 1 -- 172.21.15.38:0/543484253 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f287408d0c0 con 0x7f288c199370 2024-03-24T00:44:23.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.687+0000 7f288a800700 1 --2- 172.21.15.38:0/543484253 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f287006ccf0 0x7f287006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:23.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.688+0000 7f288a800700 1 --2- 172.21.15.38:0/543484253 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f287006ccf0 0x7f287006f1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f287c005950 tx=0x7f287c0058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:23.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.692+0000 7f2883400700 1 -- 172.21.15.38:0/543484253 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f28740577b0 con 0x7f288c199370 2024-03-24T00:44:23.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.964+0000 7f2891400700 1 -- 172.21.15.38:0/543484253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f288c19a140 con 0x7f288c199370 2024-03-24T00:44:23.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.965+0000 7f2883400700 1 -- 172.21.15.38:0/543484253 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f287405add0 con 0x7f288c199370 2024-03-24T00:44:23.966 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:44:23.966 INFO:teuthology.orchestra.run.smithi038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-24T00:44:23.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.968+0000 7f2891400700 1 -- 172.21.15.38:0/543484253 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f287006ccf0 msgr2=0x7f287006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:23.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.968+0000 7f2891400700 1 --2- 172.21.15.38:0/543484253 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f287006ccf0 0x7f287006f1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f287c005950 tx=0x7f287c0058e0 comp rx=0 tx=0).stop 2024-03-24T00:44:23.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.968+0000 7f2891400700 1 -- 172.21.15.38:0/543484253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f288c199370 msgr2=0x7f288c19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:23.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.968+0000 7f2891400700 1 --2- 172.21.15.38:0/543484253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f288c199370 0x7f288c19d720 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2874005b40 tx=0x7f287400be20 comp rx=0 tx=0).stop 2024-03-24T00:44:23.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.968+0000 7f2891400700 1 -- 172.21.15.38:0/543484253 shutdown_connections 2024-03-24T00:44:23.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.968+0000 7f2891400700 1 --2- 172.21.15.38:0/543484253 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f287006ccf0 0x7f287006f1b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:23.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.968+0000 7f2891400700 1 --2- 172.21.15.38:0/543484253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288c101be0 0x7f288c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:23.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.968+0000 7f2891400700 1 --2- 172.21.15.38:0/543484253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f288c199370 0x7f288c19d720 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:23.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.968+0000 7f2891400700 1 -- 172.21.15.38:0/543484253 >> 172.21.15.38:0/543484253 conn(0x7f288c0fb500 msgr2=0x7f288c101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:23.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.968+0000 7f2891400700 1 -- 172.21.15.38:0/543484253 shutdown_connections 2024-03-24T00:44:23.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:23.968+0000 7f2891400700 1 -- 172.21.15.38:0/543484253 wait complete. 2024-03-24T00:44:24.789 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-24T00:44:24.790 INFO:tasks.cephadm:Setup complete, yielding 2024-03-24T00:44:24.790 INFO:teuthology.run_tasks:Running task print... 2024-03-24T00:44:24.800 INFO:teuthology.task.print:**** done end installing reef cephadm ... 2024-03-24T00:44:24.800 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T00:44:24.808 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:44:24.809 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-24T00:44:24.890 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:24 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/543484253' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T00:44:24.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:24 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/543484253' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T00:44:25.243 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:25.738 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:25 smithi038 ceph-mon[29271]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:25.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:25 smithi088 ceph-mon[36610]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:26.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:26 smithi038 ceph-mon[29271]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:27.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:26 smithi088 ceph-mon[36610]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:27.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.430+0000 7fccdd800700 1 -- 172.21.15.38:0/2531299621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccd8073810 msgr2=0x7fccd8073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:27.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.430+0000 7fccdd800700 1 --2- 172.21.15.38:0/2531299621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccd8073810 0x7fccd8073c90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009910 tx=0x7fccc4009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:27.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.430+0000 7fccdd800700 1 -- 172.21.15.38:0/2531299621 shutdown_connections 2024-03-24T00:44:27.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.430+0000 7fccdd800700 1 --2- 172.21.15.38:0/2531299621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccd8073810 0x7fccd8073c90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:27.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.430+0000 7fccdd800700 1 --2- 172.21.15.38:0/2531299621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd8074de0 0x7fccd8073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:27.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.430+0000 7fccdd800700 1 -- 172.21.15.38:0/2531299621 >> 172.21.15.38:0/2531299621 conn(0x7fccd80fc090 msgr2=0x7fccd80fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:27.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.430+0000 7fccdd800700 1 -- 172.21.15.38:0/2531299621 shutdown_connections 2024-03-24T00:44:27.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.430+0000 7fccdd800700 1 -- 172.21.15.38:0/2531299621 wait complete. 2024-03-24T00:44:27.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.431+0000 7fccdd800700 1 Processor -- start 2024-03-24T00:44:27.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.431+0000 7fccdd800700 1 -- start start 2024-03-24T00:44:27.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.431+0000 7fccdd800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd8073810 0x7fccd819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:27.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.431+0000 7fccdd800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccd8074de0 0x7fccd819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:27.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.431+0000 7fccdd800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd819d630 con 0x7fccd8074de0 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.431+0000 7fccdd800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd819d770 con 0x7fccd8073810 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.432+0000 7fccd6000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccd8074de0 0x7fccd819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.432+0000 7fccd6000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccd8074de0 0x7fccd819d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51574/0 (socket says 172.21.15.38:51574) 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.432+0000 7fccd6000700 1 -- 172.21.15.38:0/2500183278 learned_addr learned my addr 172.21.15.38:0/2500183278 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.432+0000 7fccd6a00700 1 --2- 172.21.15.38:0/2500183278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd8073810 0x7fccd819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.432+0000 7fccd6000700 1 -- 172.21.15.38:0/2500183278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd8073810 msgr2=0x7fccd819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.432+0000 7fccd6000700 1 --2- 172.21.15.38:0/2500183278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd8073810 0x7fccd819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.432+0000 7fccd6000700 1 -- 172.21.15.38:0/2500183278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc40095f0 con 0x7fccd8074de0 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.433+0000 7fccd6000700 1 --2- 172.21.15.38:0/2500183278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccd8074de0 0x7fccd819d010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fccc400bd70 tx=0x7fccc400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.433+0000 7fcccf400700 1 -- 172.21.15.38:0/2500183278 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccc401d070 con 0x7fccd8074de0 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.433+0000 7fcccf400700 1 -- 172.21.15.38:0/2500183278 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccc4005dc0 con 0x7fccd8074de0 2024-03-24T00:44:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.433+0000 7fcccf400700 1 -- 172.21.15.38:0/2500183278 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccc4021410 con 0x7fccd8074de0 2024-03-24T00:44:27.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.433+0000 7fccdd800700 1 -- 172.21.15.38:0/2500183278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd81a2170 con 0x7fccd8074de0 2024-03-24T00:44:27.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.433+0000 7fccdd800700 1 -- 172.21.15.38:0/2500183278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd81a26e0 con 0x7fccd8074de0 2024-03-24T00:44:27.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.434+0000 7fccdd800700 1 -- 172.21.15.38:0/2500183278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd804ea80 con 0x7fccd8074de0 2024-03-24T00:44:27.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.435+0000 7fcccf400700 1 -- 172.21.15.38:0/2500183278 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fccc400f820 con 0x7fccd8074de0 2024-03-24T00:44:27.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.436+0000 7fcccf400700 1 --2- 172.21.15.38:0/2500183278 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fccbc06ccf0 0x7fccbc06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:27.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.436+0000 7fcccf400700 1 -- 172.21.15.38:0/2500183278 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fccc408d070 con 0x7fccd8074de0 2024-03-24T00:44:27.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.437+0000 7fccd6a00700 1 --2- 172.21.15.38:0/2500183278 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fccbc06ccf0 0x7fccbc06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:27.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.437+0000 7fccd6a00700 1 --2- 172.21.15.38:0/2500183278 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fccbc06ccf0 0x7fccbc06f1b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fccc8005950 tx=0x7fccc80058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:27.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.441+0000 7fcccf400700 1 -- 172.21.15.38:0/2500183278 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fccc405b290 con 0x7fccd8074de0 2024-03-24T00:44:27.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.664+0000 7fccdd800700 1 -- 172.21.15.38:0/2500183278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fccd81a2a10 con 0x7fccd8074de0 2024-03-24T00:44:27.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.675+0000 7fcccf400700 1 -- 172.21.15.38:0/2500183278 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fccc405ae20 con 0x7fccd8074de0 2024-03-24T00:44:27.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.678+0000 7fccdd800700 1 -- 172.21.15.38:0/2500183278 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fccbc06ccf0 msgr2=0x7fccbc06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:27.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.678+0000 7fccdd800700 1 --2- 172.21.15.38:0/2500183278 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fccbc06ccf0 0x7fccbc06f1b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fccc8005950 tx=0x7fccc80058e0 comp rx=0 tx=0).stop 2024-03-24T00:44:27.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.678+0000 7fccdd800700 1 -- 172.21.15.38:0/2500183278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccd8074de0 msgr2=0x7fccd819d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:27.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.678+0000 7fccdd800700 1 --2- 172.21.15.38:0/2500183278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccd8074de0 0x7fccd819d010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fccc400bd70 tx=0x7fccc400be50 comp rx=0 tx=0).stop 2024-03-24T00:44:27.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.678+0000 7fccdd800700 1 -- 172.21.15.38:0/2500183278 shutdown_connections 2024-03-24T00:44:27.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.678+0000 7fccdd800700 1 --2- 172.21.15.38:0/2500183278 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fccbc06ccf0 0x7fccbc06f1b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:27.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.678+0000 7fccdd800700 1 --2- 172.21.15.38:0/2500183278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd8073810 0x7fccd819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:27.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.678+0000 7fccdd800700 1 --2- 172.21.15.38:0/2500183278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccd8074de0 0x7fccd819d010 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:27.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.678+0000 7fccdd800700 1 -- 172.21.15.38:0/2500183278 >> 172.21.15.38:0/2500183278 conn(0x7fccd80fc090 msgr2=0x7fccd81023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:27.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.679+0000 7fccdd800700 1 -- 172.21.15.38:0/2500183278 shutdown_connections 2024-03-24T00:44:27.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:27.679+0000 7fccdd800700 1 -- 172.21.15.38:0/2500183278 wait complete. 2024-03-24T00:44:28.582 INFO:teuthology.run_tasks:Running task print... 2024-03-24T00:44:28.591 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-24T00:44:28.592 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T00:44:28.599 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:44:28.599 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-24T00:44:28.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:28 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2500183278' entity='client.admin' 2024-03-24T00:44:28.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:44:28.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:44:28.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:44:28.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:28.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:44:28.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:28 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2500183278' entity='client.admin' 2024-03-24T00:44:28.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:44:28.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:44:28.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:44:28.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:28.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:44:29.038 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:29.856 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:29 smithi038 ceph-mon[29271]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:29 smithi088 ceph-mon[36610]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.164+0000 7f5ac1200700 1 -- 172.21.15.38:0/2627409846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5abc0ff9d0 msgr2=0x7f5abc0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.164+0000 7f5ac1200700 1 --2- 172.21.15.38:0/2627409846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5abc0ff9d0 0x7f5abc0ffdf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009910 tx=0x7f5aac009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.164+0000 7f5ac1200700 1 -- 172.21.15.38:0/2627409846 shutdown_connections 2024-03-24T00:44:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.164+0000 7f5ac1200700 1 --2- 172.21.15.38:0/2627409846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5abc100bd0 0x7f5abc101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.164+0000 7f5ac1200700 1 --2- 172.21.15.38:0/2627409846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5abc0ff9d0 0x7f5abc0ffdf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.164+0000 7f5ac1200700 1 -- 172.21.15.38:0/2627409846 >> 172.21.15.38:0/2627409846 conn(0x7f5abc0faf70 msgr2=0x7f5abc0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:31.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.164+0000 7f5ac1200700 1 -- 172.21.15.38:0/2627409846 shutdown_connections 2024-03-24T00:44:31.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.164+0000 7f5ac1200700 1 -- 172.21.15.38:0/2627409846 wait complete. 2024-03-24T00:44:31.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.165+0000 7f5ac1200700 1 Processor -- start 2024-03-24T00:44:31.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.165+0000 7f5ac1200700 1 -- start start 2024-03-24T00:44:31.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.166+0000 7f5ac1200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5abc0ff9d0 0x7f5abc195440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:31.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.166+0000 7f5ac1200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5abc100bd0 0x7f5abc195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:31.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.166+0000 7f5ac1200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc195f10 con 0x7f5abc0ff9d0 2024-03-24T00:44:31.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.166+0000 7f5ac1200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc196050 con 0x7f5abc100bd0 2024-03-24T00:44:31.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.166+0000 7f5ab9e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5abc0ff9d0 0x7f5abc195440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:31.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.166+0000 7f5ab9e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5abc0ff9d0 0x7f5abc195440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51594/0 (socket says 172.21.15.38:51594) 2024-03-24T00:44:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.166+0000 7f5ab9e00700 1 -- 172.21.15.38:0/4157518430 learned_addr learned my addr 172.21.15.38:0/4157518430 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.166+0000 7f5ab9400700 1 --2- 172.21.15.38:0/4157518430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5abc100bd0 0x7f5abc195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.167+0000 7f5ab9e00700 1 -- 172.21.15.38:0/4157518430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5abc100bd0 msgr2=0x7f5abc195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.167+0000 7f5ab9e00700 1 --2- 172.21.15.38:0/4157518430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5abc100bd0 0x7f5abc195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.167+0000 7f5ab9e00700 1 -- 172.21.15.38:0/4157518430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac0095f0 con 0x7f5abc0ff9d0 2024-03-24T00:44:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.167+0000 7f5ab9e00700 1 --2- 172.21.15.38:0/4157518430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5abc0ff9d0 0x7f5abc195440 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0059d0 tx=0x7f5aac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.167+0000 7f5ab2a00700 1 -- 172.21.15.38:0/4157518430 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aac01f070 con 0x7f5abc0ff9d0 2024-03-24T00:44:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.167+0000 7f5ab2a00700 1 -- 172.21.15.38:0/4157518430 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5aac019610 con 0x7f5abc0ff9d0 2024-03-24T00:44:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.167+0000 7f5ac1200700 1 -- 172.21.15.38:0/4157518430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc068480 con 0x7f5abc0ff9d0 2024-03-24T00:44:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.167+0000 7f5ab2a00700 1 -- 172.21.15.38:0/4157518430 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aac00f690 con 0x7f5abc0ff9d0 2024-03-24T00:44:31.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.167+0000 7f5ac1200700 1 -- 172.21.15.38:0/4157518430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc0689c0 con 0x7f5abc0ff9d0 2024-03-24T00:44:31.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.168+0000 7f5ac1200700 1 -- 172.21.15.38:0/4157518430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc108e90 con 0x7f5abc0ff9d0 2024-03-24T00:44:31.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.169+0000 7f5ab2a00700 1 -- 172.21.15.38:0/4157518430 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5aac019780 con 0x7f5abc0ff9d0 2024-03-24T00:44:31.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.170+0000 7f5ab2a00700 1 --2- 172.21.15.38:0/4157518430 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f5aa006cce0 0x7f5aa006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:31.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.171+0000 7f5ab2a00700 1 -- 172.21.15.38:0/4157518430 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5aac08f560 con 0x7f5abc0ff9d0 2024-03-24T00:44:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.171+0000 7f5ab9400700 1 --2- 172.21.15.38:0/4157518430 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f5aa006cce0 0x7f5aa006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.171+0000 7f5ab9400700 1 --2- 172.21.15.38:0/4157518430 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f5aa006cce0 0x7f5aa006f1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005ea0 tx=0x7f5aa8005e30 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:31.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.176+0000 7f5ab2a00700 1 -- 172.21.15.38:0/4157518430 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5aac05d660 con 0x7f5abc0ff9d0 2024-03-24T00:44:31.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.418+0000 7f5ac1200700 1 -- 172.21.15.38:0/4157518430 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5abc18f460 con 0x7f5aa006cce0 2024-03-24T00:44:31.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.419+0000 7f5ab2a00700 1 -- 172.21.15.38:0/4157518430 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5abc18f460 con 0x7f5aa006cce0 2024-03-24T00:44:31.421 INFO:teuthology.orchestra.run.smithi038.stdout:Backend: cephadm 2024-03-24T00:44:31.421 INFO:teuthology.orchestra.run.smithi038.stdout:Available: Yes 2024-03-24T00:44:31.421 INFO:teuthology.orchestra.run.smithi038.stdout:Paused: No 2024-03-24T00:44:31.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.422+0000 7f5ac1200700 1 -- 172.21.15.38:0/4157518430 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f5aa006cce0 msgr2=0x7f5aa006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:31.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.422+0000 7f5ac1200700 1 --2- 172.21.15.38:0/4157518430 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f5aa006cce0 0x7f5aa006f1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005ea0 tx=0x7f5aa8005e30 comp rx=0 tx=0).stop 2024-03-24T00:44:31.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.422+0000 7f5ac1200700 1 -- 172.21.15.38:0/4157518430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5abc0ff9d0 msgr2=0x7f5abc195440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:31.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.422+0000 7f5ac1200700 1 --2- 172.21.15.38:0/4157518430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5abc0ff9d0 0x7f5abc195440 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0059d0 tx=0x7f5aac00be20 comp rx=0 tx=0).stop 2024-03-24T00:44:31.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.422+0000 7f5ac1200700 1 -- 172.21.15.38:0/4157518430 shutdown_connections 2024-03-24T00:44:31.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.422+0000 7f5ac1200700 1 --2- 172.21.15.38:0/4157518430 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f5aa006cce0 0x7f5aa006f1a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:31.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.422+0000 7f5ac1200700 1 --2- 172.21.15.38:0/4157518430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5abc0ff9d0 0x7f5abc195440 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:31.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.422+0000 7f5ac1200700 1 --2- 172.21.15.38:0/4157518430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5abc100bd0 0x7f5abc195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:31.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.422+0000 7f5ac1200700 1 -- 172.21.15.38:0/4157518430 >> 172.21.15.38:0/4157518430 conn(0x7f5abc0faf70 msgr2=0x7f5abc104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:31.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.423+0000 7f5ac1200700 1 -- 172.21.15.38:0/4157518430 shutdown_connections 2024-03-24T00:44:31.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:31.423+0000 7f5ac1200700 1 -- 172.21.15.38:0/4157518430 wait complete. 2024-03-24T00:44:32.076 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:31 smithi038 ceph-mon[29271]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:32.140 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-24T00:44:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:31 smithi088 ceph-mon[36610]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:33.013 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:32 smithi038 ceph-mon[29271]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:33.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:32 smithi088 ceph-mon[36610]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:33.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:33 smithi038 ceph-mon[29271]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:34.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:33 smithi088 ceph-mon[36610]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:34.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.744+0000 7f1b5be00700 1 -- 172.21.15.38:0/4048694053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c102b30 msgr2=0x7f1b5c102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:34.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.744+0000 7f1b5be00700 1 --2- 172.21.15.38:0/4048694053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c102b30 0x7f1b5c102f50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009910 tx=0x7f1b4c009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:34.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.744+0000 7f1b5be00700 1 -- 172.21.15.38:0/4048694053 shutdown_connections 2024-03-24T00:44:34.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.744+0000 7f1b5be00700 1 --2- 172.21.15.38:0/4048694053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c103d30 0x7f1b5c104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:34.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.744+0000 7f1b5be00700 1 --2- 172.21.15.38:0/4048694053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c102b30 0x7f1b5c102f50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:34.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.744+0000 7f1b5be00700 1 -- 172.21.15.38:0/4048694053 >> 172.21.15.38:0/4048694053 conn(0x7f1b5c0fe110 msgr2=0x7f1b5c100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:34.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.744+0000 7f1b5be00700 1 -- 172.21.15.38:0/4048694053 shutdown_connections 2024-03-24T00:44:34.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.744+0000 7f1b5be00700 1 -- 172.21.15.38:0/4048694053 wait complete. 2024-03-24T00:44:34.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.745+0000 7f1b5be00700 1 Processor -- start 2024-03-24T00:44:34.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.746+0000 7f1b5be00700 1 -- start start 2024-03-24T00:44:34.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.746+0000 7f1b5be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c102b30 0x7f1b5c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:34.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.746+0000 7f1b5be00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c103d30 0x7f1b5c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:34.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.746+0000 7f1b5be00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c199070 con 0x7f1b5c102b30 2024-03-24T00:44:34.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.746+0000 7f1b5be00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c1991b0 con 0x7f1b5c103d30 2024-03-24T00:44:34.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.746+0000 7f1b5aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c102b30 0x7f1b5c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:34.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.746+0000 7f1b5aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c102b30 0x7f1b5c198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47464/0 (socket says 172.21.15.38:47464) 2024-03-24T00:44:34.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.746+0000 7f1b5aa00700 1 -- 172.21.15.38:0/2784201337 learned_addr learned my addr 172.21.15.38:0/2784201337 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:34.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.747+0000 7f1b52000700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c103d30 0x7f1b5c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:34.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.747+0000 7f1b5aa00700 1 -- 172.21.15.38:0/2784201337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c103d30 msgr2=0x7f1b5c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:34.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.747+0000 7f1b5aa00700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c103d30 0x7f1b5c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:34.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.747+0000 7f1b5aa00700 1 -- 172.21.15.38:0/2784201337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c0095f0 con 0x7f1b5c102b30 2024-03-24T00:44:34.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.747+0000 7f1b52000700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c103d30 0x7f1b5c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:44:34.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.747+0000 7f1b5aa00700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c102b30 0x7f1b5c198510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00bd70 tx=0x7f1b4c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:34.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.747+0000 7f1b53e00700 1 -- 172.21.15.38:0/2784201337 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b4c01d070 con 0x7f1b5c102b30 2024-03-24T00:44:34.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.747+0000 7f1b53e00700 1 -- 172.21.15.38:0/2784201337 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b4c005dc0 con 0x7f1b5c102b30 2024-03-24T00:44:34.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.747+0000 7f1b53e00700 1 -- 172.21.15.38:0/2784201337 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b4c021410 con 0x7f1b5c102b30 2024-03-24T00:44:34.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.747+0000 7f1b5be00700 1 -- 172.21.15.38:0/2784201337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c19dbb0 con 0x7f1b5c102b30 2024-03-24T00:44:34.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.748+0000 7f1b5be00700 1 -- 172.21.15.38:0/2784201337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c19e0a0 con 0x7f1b5c102b30 2024-03-24T00:44:34.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.749+0000 7f1b5be00700 1 -- 172.21.15.38:0/2784201337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c1926b0 con 0x7f1b5c102b30 2024-03-24T00:44:34.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.750+0000 7f1b53e00700 1 -- 172.21.15.38:0/2784201337 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1b4c00f820 con 0x7f1b5c102b30 2024-03-24T00:44:34.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.751+0000 7f1b53e00700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1b4006ccf0 0x7f1b4006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:34.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.751+0000 7f1b53e00700 1 -- 172.21.15.38:0/2784201337 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1b4c08d130 con 0x7f1b5c102b30 2024-03-24T00:44:34.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.751+0000 7f1b52000700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1b4006ccf0 0x7f1b4006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:34.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.752+0000 7f1b52000700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1b4006ccf0 0x7f1b4006f1b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1b44005fd0 tx=0x7f1b44005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:34.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.756+0000 7f1b53e00700 1 -- 172.21.15.38:0/2784201337 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1b4c05b2b0 con 0x7f1b5c102b30 2024-03-24T00:44:34.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:34.990+0000 7f1b5be00700 1 -- 172.21.15.38:0/2784201337 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b5c0611f0 con 0x7f1b4006ccf0 2024-03-24T00:44:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.018+0000 7f1b53e00700 1 -- 172.21.15.38:0/2784201337 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3080 (secure 0 0 0) 0x7f1b5c0611f0 con 0x7f1b4006ccf0 2024-03-24T00:44:35.019 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:44:35.019 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (4m) 2m ago 6m 14.1M - 0.25.0 c8568f914cd2 9f1243fbcb3e 2024-03-24T00:44:35.019 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (7m) 2m ago 7m 7092k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:44:35.019 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (5m) 81s ago 5m 7008k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (7m) 2m ago 7m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (5m) 81s ago 5m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (4m) 2m ago 6m 71.7M - 9.4.7 954c08fa6188 e6aa268d69e8 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:9283,8765,8443 running (8m) 2m ago 8m 490M - 18.2.2-952-g738c2fd6 68aff9aff3b2 bf370183b69d 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (5m) 81s ago 5m 448M - 18.2.2-952-g738c2fd6 68aff9aff3b2 acebd34a44e9 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (8m) 2m ago 8m 43.5M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (5m) 81s ago 5m 33.1M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (7m) 2m ago 7m 7520k - 1.5.0 0da6a335fe13 c34cc077c93f 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (5m) 81s ago 5m 7566k - 1.5.0 0da6a335fe13 189e840ee2f1 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (4m) 2m ago 4m 39.4M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (3m) 2m ago 3m 39.0M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (3m) 2m ago 3m 38.8M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:44:35.020 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (2m) 81s ago 2m 40.3M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:44:35.021 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (2m) 81s ago 2m 38.2M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:44:35.021 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (87s) 81s ago 86s 11.1M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:44:35.021 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (4m) 2m ago 6m 38.8M - 2.43.0 a07b618ecd1d c59e5c711ace 2024-03-24T00:44:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.021+0000 7f1b5be00700 1 -- 172.21.15.38:0/2784201337 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1b4006ccf0 msgr2=0x7f1b4006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.021+0000 7f1b5be00700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1b4006ccf0 0x7f1b4006f1b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1b44005fd0 tx=0x7f1b44005ee0 comp rx=0 tx=0).stop 2024-03-24T00:44:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.021+0000 7f1b5be00700 1 -- 172.21.15.38:0/2784201337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c102b30 msgr2=0x7f1b5c198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.021+0000 7f1b5be00700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c102b30 0x7f1b5c198510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00bd70 tx=0x7f1b4c00be50 comp rx=0 tx=0).stop 2024-03-24T00:44:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.021+0000 7f1b5be00700 1 -- 172.21.15.38:0/2784201337 shutdown_connections 2024-03-24T00:44:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.021+0000 7f1b5be00700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f1b4006ccf0 0x7f1b4006f1b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.021+0000 7f1b5be00700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b5c102b30 0x7f1b5c198510 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.021+0000 7f1b5be00700 1 --2- 172.21.15.38:0/2784201337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c103d30 0x7f1b5c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.021+0000 7f1b5be00700 1 -- 172.21.15.38:0/2784201337 >> 172.21.15.38:0/2784201337 conn(0x7f1b5c0fe110 msgr2=0x7f1b5c107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.021+0000 7f1b5be00700 1 -- 172.21.15.38:0/2784201337 shutdown_connections 2024-03-24T00:44:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:35.021+0000 7f1b5be00700 1 -- 172.21.15.38:0/2784201337 wait complete. 2024-03-24T00:44:35.883 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-24T00:44:35.979 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:35 smithi038 ceph-mon[29271]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:36.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:35 smithi088 ceph-mon[36610]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:36.336 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:36.810 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:36 smithi038 ceph-mon[29271]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:37.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:36 smithi088 ceph-mon[36610]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:37.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:37 smithi038 ceph-mon[29271]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:38.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:37 smithi088 ceph-mon[36610]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:38.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.387+0000 7f396ce00700 1 -- 172.21.15.38:0/3683803281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3968103f00 msgr2=0x7f3968104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:38.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.387+0000 7f396ce00700 1 --2- 172.21.15.38:0/3683803281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3968103f00 0x7f3968104360 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3954009910 tx=0x7f3954009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:38.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.387+0000 7f396ce00700 1 -- 172.21.15.38:0/3683803281 shutdown_connections 2024-03-24T00:44:38.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.387+0000 7f396ce00700 1 --2- 172.21.15.38:0/3683803281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3968103f00 0x7f3968104360 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:38.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.387+0000 7f396ce00700 1 --2- 172.21.15.38:0/3683803281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3968102d00 0x7f3968103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.387+0000 7f396ce00700 1 -- 172.21.15.38:0/3683803281 >> 172.21.15.38:0/3683803281 conn(0x7f39680fe2a0 msgr2=0x7f39681006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.387+0000 7f396ce00700 1 -- 172.21.15.38:0/3683803281 shutdown_connections 2024-03-24T00:44:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.387+0000 7f396ce00700 1 -- 172.21.15.38:0/3683803281 wait complete. 2024-03-24T00:44:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.388+0000 7f396ce00700 1 Processor -- start 2024-03-24T00:44:38.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.388+0000 7f396ce00700 1 -- start start 2024-03-24T00:44:38.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f396ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3968102d00 0x7f3968198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:38.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f396ce00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3968103f00 0x7f3968198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:38.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f396ce00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3968199270 con 0x7f3968102d00 2024-03-24T00:44:38.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f396ce00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39681993b0 con 0x7f3968103f00 2024-03-24T00:44:38.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f3965e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3968102d00 0x7f3968198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:38.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f3965e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3968102d00 0x7f3968198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47484/0 (socket says 172.21.15.38:47484) 2024-03-24T00:44:38.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f3965e00700 1 -- 172.21.15.38:0/985446459 learned_addr learned my addr 172.21.15.38:0/985446459 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:38.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f3965400700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3968103f00 0x7f3968198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:38.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f3965e00700 1 -- 172.21.15.38:0/985446459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3968103f00 msgr2=0x7f3968198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:38.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f3965e00700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3968103f00 0x7f3968198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:38.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f3965e00700 1 -- 172.21.15.38:0/985446459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39540095f0 con 0x7f3968102d00 2024-03-24T00:44:38.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.389+0000 7f3965400700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3968103f00 0x7f3968198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:44:38.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.390+0000 7f3965e00700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3968102d00 0x7f3968198710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f395800d900 tx=0x7f395800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:38.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.390+0000 7f395ea00700 1 -- 172.21.15.38:0/985446459 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39580041f0 con 0x7f3968102d00 2024-03-24T00:44:38.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.390+0000 7f395ea00700 1 -- 172.21.15.38:0/985446459 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3958004350 con 0x7f3968102d00 2024-03-24T00:44:38.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.390+0000 7f396ce00700 1 -- 172.21.15.38:0/985446459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396819ddc0 con 0x7f3968102d00 2024-03-24T00:44:38.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.390+0000 7f395ea00700 1 -- 172.21.15.38:0/985446459 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3958003dc0 con 0x7f3968102d00 2024-03-24T00:44:38.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.390+0000 7f396ce00700 1 -- 172.21.15.38:0/985446459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396819e310 con 0x7f3968102d00 2024-03-24T00:44:38.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.392+0000 7f396ce00700 1 -- 172.21.15.38:0/985446459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3968192880 con 0x7f3968102d00 2024-03-24T00:44:38.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.393+0000 7f395ea00700 1 -- 172.21.15.38:0/985446459 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3958009730 con 0x7f3968102d00 2024-03-24T00:44:38.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.393+0000 7f395ea00700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f394c06ccf0 0x7f394c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:38.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.394+0000 7f395ea00700 1 -- 172.21.15.38:0/985446459 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f395808c100 con 0x7f3968102d00 2024-03-24T00:44:38.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.394+0000 7f3965400700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f394c06ccf0 0x7f394c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:38.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.394+0000 7f3965400700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f394c06ccf0 0x7f394c06f1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f395400b5c0 tx=0x7f3954005160 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:38.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.398+0000 7f395ea00700 1 -- 172.21.15.38:0/985446459 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f395805a280 con 0x7f3968102d00 2024-03-24T00:44:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.637+0000 7f396ce00700 1 -- 172.21.15.38:0/985446459 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f39680611f0 con 0x7f394c06ccf0 2024-03-24T00:44:38.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.644+0000 7f395ea00700 1 -- 172.21.15.38:0/985446459 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1290 (secure 0 0 0) 0x7f39680611f0 con 0x7f394c06ccf0 2024-03-24T00:44:38.645 INFO:teuthology.orchestra.run.smithi038.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-24T00:44:38.645 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager ?:9093,9094 1/1 2m ago 7m count:1 2024-03-24T00:44:38.646 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter 2/2 2m ago 8m * 2024-03-24T00:44:38.646 INFO:teuthology.orchestra.run.smithi038.stdout:crash 2/2 2m ago 8m * 2024-03-24T00:44:38.646 INFO:teuthology.orchestra.run.smithi038.stdout:grafana ?:3000 1/1 2m ago 8m count:1 2024-03-24T00:44:38.646 INFO:teuthology.orchestra.run.smithi038.stdout:mgr 2/2 2m ago 8m count:2 2024-03-24T00:44:38.646 INFO:teuthology.orchestra.run.smithi038.stdout:mon 2/2 2m ago 7m smithi038:172.21.15.38=smithi038;smithi088:172.21.15.88=smithi088;count:2 2024-03-24T00:44:38.646 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter ?:9100 2/2 2m ago 7m * 2024-03-24T00:44:38.646 INFO:teuthology.orchestra.run.smithi038.stdout:osd 6 2m ago - 2024-03-24T00:44:38.646 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus ?:9095 1/1 2m ago 8m count:1 2024-03-24T00:44:38.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.647+0000 7f396ce00700 1 -- 172.21.15.38:0/985446459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f394c06ccf0 msgr2=0x7f394c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:38.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.647+0000 7f396ce00700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f394c06ccf0 0x7f394c06f1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f395400b5c0 tx=0x7f3954005160 comp rx=0 tx=0).stop 2024-03-24T00:44:38.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.647+0000 7f396ce00700 1 -- 172.21.15.38:0/985446459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3968102d00 msgr2=0x7f3968198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:38.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.647+0000 7f396ce00700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3968102d00 0x7f3968198710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f395800d900 tx=0x7f395800dcc0 comp rx=0 tx=0).stop 2024-03-24T00:44:38.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.647+0000 7f396ce00700 1 -- 172.21.15.38:0/985446459 shutdown_connections 2024-03-24T00:44:38.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.647+0000 7f396ce00700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f394c06ccf0 0x7f394c06f1b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:38.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.647+0000 7f396ce00700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3968102d00 0x7f3968198710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:38.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.647+0000 7f396ce00700 1 --2- 172.21.15.38:0/985446459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3968103f00 0x7f3968198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:38.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.647+0000 7f396ce00700 1 -- 172.21.15.38:0/985446459 >> 172.21.15.38:0/985446459 conn(0x7f39680fe2a0 msgr2=0x7f3968107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:38.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.647+0000 7f396ce00700 1 -- 172.21.15.38:0/985446459 shutdown_connections 2024-03-24T00:44:38.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:38.647+0000 7f396ce00700 1 -- 172.21.15.38:0/985446459 wait complete. 2024-03-24T00:44:39.384 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-24T00:44:39.829 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:39.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:39 smithi088 ceph-mon[36610]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:39.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:39 smithi088 ceph-mon[36610]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:39.978 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:39 smithi038 ceph-mon[29271]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:39.978 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:39 smithi038 ceph-mon[29271]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:41.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:40 smithi038 ceph-mon[29271]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:41.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:40 smithi088 ceph-mon[36610]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.959+0000 7f4dad200700 1 -- 172.21.15.38:0/3675982997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8101570 msgr2=0x7f4da8103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.959+0000 7f4dad200700 1 --2- 172.21.15.38:0/3675982997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8101570 0x7f4da8103960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800b0a0 tx=0x7f4d9800b3b0 comp rx=0 tx=0).stop 2024-03-24T00:44:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.960+0000 7f4dad200700 1 -- 172.21.15.38:0/3675982997 shutdown_connections 2024-03-24T00:44:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.960+0000 7f4dad200700 1 --2- 172.21.15.38:0/3675982997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4da8103ea0 0x7f4da8106290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.960+0000 7f4dad200700 1 --2- 172.21.15.38:0/3675982997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8101570 0x7f4da8103960 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.960+0000 7f4dad200700 1 -- 172.21.15.38:0/3675982997 >> 172.21.15.38:0/3675982997 conn(0x7f4da80faf80 msgr2=0x7f4da80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.960+0000 7f4dad200700 1 -- 172.21.15.38:0/3675982997 shutdown_connections 2024-03-24T00:44:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.960+0000 7f4dad200700 1 -- 172.21.15.38:0/3675982997 wait complete. 2024-03-24T00:44:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.961+0000 7f4dad200700 1 Processor -- start 2024-03-24T00:44:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.961+0000 7f4dad200700 1 -- start start 2024-03-24T00:44:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.962+0000 7f4dad200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4da8101570 0x7f4da8198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.962+0000 7f4dad200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8103ea0 0x7f4da8198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.962+0000 7f4dad200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da8199360 con 0x7f4da8103ea0 2024-03-24T00:44:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.962+0000 7f4dad200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da81994a0 con 0x7f4da8101570 2024-03-24T00:44:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.962+0000 7f4da5400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8103ea0 0x7f4da8198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.962+0000 7f4da5400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8103ea0 0x7f4da8198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47504/0 (socket says 172.21.15.38:47504) 2024-03-24T00:44:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.962+0000 7f4da5400700 1 -- 172.21.15.38:0/1116670712 learned_addr learned my addr 172.21.15.38:0/1116670712 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.962+0000 7f4da5e00700 1 --2- 172.21.15.38:0/1116670712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4da8101570 0x7f4da8198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.963+0000 7f4da5400700 1 -- 172.21.15.38:0/1116670712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4da8101570 msgr2=0x7f4da8198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.963+0000 7f4da5400700 1 --2- 172.21.15.38:0/1116670712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4da8101570 0x7f4da8198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.963+0000 7f4da5400700 1 -- 172.21.15.38:0/1116670712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d98009d00 con 0x7f4da8103ea0 2024-03-24T00:44:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.963+0000 7f4da5400700 1 --2- 172.21.15.38:0/1116670712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8103ea0 0x7f4da8198d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400b4b0 tx=0x7f4d9400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.963+0000 7f4d9ea00700 1 -- 172.21.15.38:0/1116670712 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d94011800 con 0x7f4da8103ea0 2024-03-24T00:44:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.963+0000 7f4d9ea00700 1 -- 172.21.15.38:0/1116670712 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d94011e20 con 0x7f4da8103ea0 2024-03-24T00:44:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.963+0000 7f4d9ea00700 1 -- 172.21.15.38:0/1116670712 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d9400f2e0 con 0x7f4da8103ea0 2024-03-24T00:44:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.963+0000 7f4dad200700 1 -- 172.21.15.38:0/1116670712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da819df00 con 0x7f4da8103ea0 2024-03-24T00:44:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.963+0000 7f4dad200700 1 -- 172.21.15.38:0/1116670712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da819e4a0 con 0x7f4da8103ea0 2024-03-24T00:44:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.965+0000 7f4dad200700 1 -- 172.21.15.38:0/1116670712 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da80fcb50 con 0x7f4da8103ea0 2024-03-24T00:44:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.966+0000 7f4d9ea00700 1 -- 172.21.15.38:0/1116670712 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4d94010380 con 0x7f4da8103ea0 2024-03-24T00:44:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.967+0000 7f4d9ea00700 1 --2- 172.21.15.38:0/1116670712 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4d8c06ccf0 0x7f4d8c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.967+0000 7f4d9ea00700 1 -- 172.21.15.38:0/1116670712 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4d9408ba60 con 0x7f4da8103ea0 2024-03-24T00:44:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.967+0000 7f4da5e00700 1 --2- 172.21.15.38:0/1116670712 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4d8c06ccf0 0x7f4d8c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.968+0000 7f4da5e00700 1 --2- 172.21.15.38:0/1116670712 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4d8c06ccf0 0x7f4d8c06f1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d98000f80 tx=0x7f4d9800b7b0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:41.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:41.972+0000 7f4d9ea00700 1 -- 172.21.15.38:0/1116670712 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4d94059be0 con 0x7f4da8103ea0 2024-03-24T00:44:42.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.207+0000 7f4dad200700 1 -- 172.21.15.38:0/1116670712 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4da80611f0 con 0x7f4d8c06ccf0 2024-03-24T00:44:42.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.209+0000 7f4d9ea00700 1 -- 172.21.15.38:0/1116670712 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7f4da80611f0 con 0x7f4d8c06ccf0 2024-03-24T00:44:42.211 INFO:teuthology.orchestra.run.smithi038.stdout:HOST ADDR LABELS STATUS 2024-03-24T00:44:42.211 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 172.21.15.38 2024-03-24T00:44:42.211 INFO:teuthology.orchestra.run.smithi038.stdout:smithi088 172.21.15.88 2024-03-24T00:44:42.211 INFO:teuthology.orchestra.run.smithi038.stdout:2 hosts in cluster 2024-03-24T00:44:42.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.212+0000 7f4dad200700 1 -- 172.21.15.38:0/1116670712 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4d8c06ccf0 msgr2=0x7f4d8c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:42.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.212+0000 7f4dad200700 1 --2- 172.21.15.38:0/1116670712 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4d8c06ccf0 0x7f4d8c06f1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d98000f80 tx=0x7f4d9800b7b0 comp rx=0 tx=0).stop 2024-03-24T00:44:42.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.213+0000 7f4dad200700 1 -- 172.21.15.38:0/1116670712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8103ea0 msgr2=0x7f4da8198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:42.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.213+0000 7f4dad200700 1 --2- 172.21.15.38:0/1116670712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8103ea0 0x7f4da8198d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400b4b0 tx=0x7f4d9400b870 comp rx=0 tx=0).stop 2024-03-24T00:44:42.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.213+0000 7f4dad200700 1 -- 172.21.15.38:0/1116670712 shutdown_connections 2024-03-24T00:44:42.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.213+0000 7f4dad200700 1 --2- 172.21.15.38:0/1116670712 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4d8c06ccf0 0x7f4d8c06f1b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:42.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.213+0000 7f4dad200700 1 --2- 172.21.15.38:0/1116670712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4da8101570 0x7f4da8198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:42.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.213+0000 7f4dad200700 1 --2- 172.21.15.38:0/1116670712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8103ea0 0x7f4da8198d40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:42.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.213+0000 7f4dad200700 1 -- 172.21.15.38:0/1116670712 >> 172.21.15.38:0/1116670712 conn(0x7f4da80faf80 msgr2=0x7f4da8104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:42.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.213+0000 7f4dad200700 1 -- 172.21.15.38:0/1116670712 shutdown_connections 2024-03-24T00:44:42.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:42.213+0000 7f4dad200700 1 -- 172.21.15.38:0/1116670712 wait complete. 2024-03-24T00:44:42.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:42 smithi038 ceph-mon[29271]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:42.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:42 smithi038 ceph-mon[29271]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:42.972 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-24T00:44:43.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:42 smithi088 ceph-mon[36610]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:43.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:42 smithi088 ceph-mon[36610]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:43.403 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:44.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:44:45.017 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:44:45.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.503+0000 7fc1bf600700 1 -- 172.21.15.38:0/403525062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b8103e90 msgr2=0x7fc1b8106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:45.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.503+0000 7fc1bf600700 1 --2- 172.21.15.38:0/403525062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b8103e90 0x7fc1b8106280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009910 tx=0x7fc1a4009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:45.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.504+0000 7fc1bf600700 1 -- 172.21.15.38:0/403525062 shutdown_connections 2024-03-24T00:44:45.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.504+0000 7fc1bf600700 1 --2- 172.21.15.38:0/403525062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b8103e90 0x7fc1b8106280 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:45.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.504+0000 7fc1bf600700 1 --2- 172.21.15.38:0/403525062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1b8101560 0x7fc1b8103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:45.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.504+0000 7fc1bf600700 1 -- 172.21.15.38:0/403525062 >> 172.21.15.38:0/403525062 conn(0x7fc1b80faf70 msgr2=0x7fc1b80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:45.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.504+0000 7fc1bf600700 1 -- 172.21.15.38:0/403525062 shutdown_connections 2024-03-24T00:44:45.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.504+0000 7fc1bf600700 1 -- 172.21.15.38:0/403525062 wait complete. 2024-03-24T00:44:45.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.505+0000 7fc1bf600700 1 Processor -- start 2024-03-24T00:44:45.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.505+0000 7fc1bf600700 1 -- start start 2024-03-24T00:44:45.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.505+0000 7fc1bf600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1b8101560 0x7fc1b81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:45.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.506+0000 7fc1bf600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b8103e90 0x7fc1b8198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:45.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.506+0000 7fc1bf600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8199210 con 0x7fc1b8103e90 2024-03-24T00:44:45.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.506+0000 7fc1bf600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8199350 con 0x7fc1b8101560 2024-03-24T00:44:45.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.506+0000 7fc1b7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b8103e90 0x7fc1b8198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:45.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.506+0000 7fc1b7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b8103e90 0x7fc1b8198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46470/0 (socket says 172.21.15.38:46470) 2024-03-24T00:44:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.506+0000 7fc1b7e00700 1 -- 172.21.15.38:0/1997648438 learned_addr learned my addr 172.21.15.38:0/1997648438 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.506+0000 7fc1bcc00700 1 --2- 172.21.15.38:0/1997648438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1b8101560 0x7fc1b81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.506+0000 7fc1b7e00700 1 -- 172.21.15.38:0/1997648438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1b8101560 msgr2=0x7fc1b81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.506+0000 7fc1b7e00700 1 --2- 172.21.15.38:0/1997648438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1b8101560 0x7fc1b81986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.506+0000 7fc1b7e00700 1 -- 172.21.15.38:0/1997648438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40095f0 con 0x7fc1b8103e90 2024-03-24T00:44:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.506+0000 7fc1b7e00700 1 --2- 172.21.15.38:0/1997648438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b8103e90 0x7fc1b8198bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400bd70 tx=0x7fc1a400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.507+0000 7fc1b5600700 1 -- 172.21.15.38:0/1997648438 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a401d070 con 0x7fc1b8103e90 2024-03-24T00:44:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.507+0000 7fc1b5600700 1 -- 172.21.15.38:0/1997648438 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1a4005dc0 con 0x7fc1b8103e90 2024-03-24T00:44:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.507+0000 7fc1b5600700 1 -- 172.21.15.38:0/1997648438 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a4021410 con 0x7fc1b8103e90 2024-03-24T00:44:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.507+0000 7fc1bf600700 1 -- 172.21.15.38:0/1997648438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b819dd50 con 0x7fc1b8103e90 2024-03-24T00:44:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.507+0000 7fc1bf600700 1 -- 172.21.15.38:0/1997648438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b819e2c0 con 0x7fc1b8103e90 2024-03-24T00:44:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.508+0000 7fc1bf600700 1 -- 172.21.15.38:0/1997648438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b80fcb40 con 0x7fc1b8103e90 2024-03-24T00:44:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.509+0000 7fc1b5600700 1 -- 172.21.15.38:0/1997648438 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc1a400f820 con 0x7fc1b8103e90 2024-03-24T00:44:45.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.510+0000 7fc1b5600700 1 --2- 172.21.15.38:0/1997648438 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc1a006ccf0 0x7fc1a006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:45.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.510+0000 7fc1b5600700 1 -- 172.21.15.38:0/1997648438 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc1a408cfb0 con 0x7fc1b8103e90 2024-03-24T00:44:45.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.511+0000 7fc1bcc00700 1 --2- 172.21.15.38:0/1997648438 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc1a006ccf0 0x7fc1a006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:45.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.511+0000 7fc1bcc00700 1 --2- 172.21.15.38:0/1997648438 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc1a006ccf0 0x7fc1a006f1b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac005950 tx=0x7fc1ac0058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:45.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.515+0000 7fc1b5600700 1 -- 172.21.15.38:0/1997648438 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc1a405b1d0 con 0x7fc1b8103e90 2024-03-24T00:44:45.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.759+0000 7fc1bf600700 1 -- 172.21.15.38:0/1997648438 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1b80611f0 con 0x7fc1a006ccf0 2024-03-24T00:44:45.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.763+0000 7fc1b5600700 1 -- 172.21.15.38:0/1997648438 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fc1b80611f0 con 0x7fc1a006ccf0 2024-03-24T00:44:45.765 INFO:teuthology.orchestra.run.smithi038.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-24T00:44:45.765 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN 3726G No 3m ago Has a FileSystem, LVM detected 2024-03-24T00:44:45.765 INFO:teuthology.orchestra.run.smithi038.stdout:smithi088 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XD400BGN 372G No 85s ago Has a FileSystem, LVM detected 2024-03-24T00:44:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.766+0000 7fc1bf600700 1 -- 172.21.15.38:0/1997648438 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc1a006ccf0 msgr2=0x7fc1a006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.766+0000 7fc1bf600700 1 --2- 172.21.15.38:0/1997648438 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc1a006ccf0 0x7fc1a006f1b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac005950 tx=0x7fc1ac0058e0 comp rx=0 tx=0).stop 2024-03-24T00:44:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.766+0000 7fc1bf600700 1 -- 172.21.15.38:0/1997648438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b8103e90 msgr2=0x7fc1b8198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.766+0000 7fc1bf600700 1 --2- 172.21.15.38:0/1997648438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b8103e90 0x7fc1b8198bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400bd70 tx=0x7fc1a400be50 comp rx=0 tx=0).stop 2024-03-24T00:44:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.766+0000 7fc1bf600700 1 -- 172.21.15.38:0/1997648438 shutdown_connections 2024-03-24T00:44:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.766+0000 7fc1bf600700 1 --2- 172.21.15.38:0/1997648438 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc1a006ccf0 0x7fc1a006f1b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.766+0000 7fc1bf600700 1 --2- 172.21.15.38:0/1997648438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1b8101560 0x7fc1b81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.766+0000 7fc1bf600700 1 --2- 172.21.15.38:0/1997648438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b8103e90 0x7fc1b8198bf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.766+0000 7fc1bf600700 1 -- 172.21.15.38:0/1997648438 >> 172.21.15.38:0/1997648438 conn(0x7fc1b80faf70 msgr2=0x7fc1b8100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.767+0000 7fc1bf600700 1 -- 172.21.15.38:0/1997648438 shutdown_connections 2024-03-24T00:44:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:45.767+0000 7fc1bf600700 1 -- 172.21.15.38:0/1997648438 wait complete. 2024-03-24T00:44:45.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:45 smithi088 ceph-mon[36610]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:46.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:45 smithi038 ceph-mon[29271]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:46.612 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T00:44:46.622 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:44:46.622 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-24T00:44:46.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:46 smithi038 ceph-mon[29271]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:46.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:46 smithi038 ceph-mon[29271]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:47.050 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:47.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:46 smithi088 ceph-mon[36610]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:47.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:46 smithi088 ceph-mon[36610]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:49.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.249+0000 7f6c49200700 1 -- 172.21.15.38:0/2057243948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c44101560 msgr2=0x7f6c44103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:49.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.249+0000 7f6c49200700 1 --2- 172.21.15.38:0/2057243948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c44101560 0x7f6c44103950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009910 tx=0x7f6c34009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:49.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.250+0000 7f6c49200700 1 -- 172.21.15.38:0/2057243948 shutdown_connections 2024-03-24T00:44:49.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.250+0000 7f6c49200700 1 --2- 172.21.15.38:0/2057243948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c44103e90 0x7f6c44106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:49.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.250+0000 7f6c49200700 1 --2- 172.21.15.38:0/2057243948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c44101560 0x7f6c44103950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:49.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.250+0000 7f6c49200700 1 -- 172.21.15.38:0/2057243948 >> 172.21.15.38:0/2057243948 conn(0x7f6c440faf70 msgr2=0x7f6c440fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:49.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.250+0000 7f6c49200700 1 -- 172.21.15.38:0/2057243948 shutdown_connections 2024-03-24T00:44:49.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.250+0000 7f6c49200700 1 -- 172.21.15.38:0/2057243948 wait complete. 2024-03-24T00:44:49.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.251+0000 7f6c49200700 1 Processor -- start 2024-03-24T00:44:49.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.251+0000 7f6c49200700 1 -- start start 2024-03-24T00:44:49.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.252+0000 7f6c49200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c44101560 0x7f6c441986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:49.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.252+0000 7f6c49200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c44103e90 0x7f6c44198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:49.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.252+0000 7f6c49200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44199230 con 0x7f6c44101560 2024-03-24T00:44:49.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.252+0000 7f6c49200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44199370 con 0x7f6c44103e90 2024-03-24T00:44:49.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.252+0000 7f6c41e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c44101560 0x7f6c441986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:49.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.252+0000 7f6c41e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c44101560 0x7f6c441986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46500/0 (socket says 172.21.15.38:46500) 2024-03-24T00:44:49.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.252+0000 7f6c41e00700 1 -- 172.21.15.38:0/3497585688 learned_addr learned my addr 172.21.15.38:0/3497585688 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:49.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.253+0000 7f6c41400700 1 --2- 172.21.15.38:0/3497585688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c44103e90 0x7f6c44198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:49.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.253+0000 7f6c41e00700 1 -- 172.21.15.38:0/3497585688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c44103e90 msgr2=0x7f6c44198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:49.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.253+0000 7f6c41e00700 1 --2- 172.21.15.38:0/3497585688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c44103e90 0x7f6c44198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:49.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.253+0000 7f6c41e00700 1 -- 172.21.15.38:0/3497585688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c340095f0 con 0x7f6c44101560 2024-03-24T00:44:49.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.253+0000 7f6c41e00700 1 --2- 172.21.15.38:0/3497585688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c44101560 0x7f6c441986d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c340092e0 tx=0x7f6c3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:49.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.253+0000 7f6c3aa00700 1 -- 172.21.15.38:0/3497585688 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c3401d070 con 0x7f6c44101560 2024-03-24T00:44:49.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.253+0000 7f6c3aa00700 1 -- 172.21.15.38:0/3497585688 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c3400f460 con 0x7f6c44101560 2024-03-24T00:44:49.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.253+0000 7f6c49200700 1 -- 172.21.15.38:0/3497585688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4419dd70 con 0x7f6c44101560 2024-03-24T00:44:49.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.254+0000 7f6c3aa00700 1 -- 172.21.15.38:0/3497585688 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c34021640 con 0x7f6c44101560 2024-03-24T00:44:49.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.254+0000 7f6c49200700 1 -- 172.21.15.38:0/3497585688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4419e280 con 0x7f6c44101560 2024-03-24T00:44:49.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.255+0000 7f6c49200700 1 -- 172.21.15.38:0/3497585688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4404eaf0 con 0x7f6c44101560 2024-03-24T00:44:49.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.256+0000 7f6c3aa00700 1 -- 172.21.15.38:0/3497585688 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6c3400fad0 con 0x7f6c44101560 2024-03-24T00:44:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.257+0000 7f6c3aa00700 1 --2- 172.21.15.38:0/3497585688 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6c2806cce0 0x7f6c2806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.257+0000 7f6c3aa00700 1 -- 172.21.15.38:0/3497585688 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f6c3408d1f0 con 0x7f6c44101560 2024-03-24T00:44:49.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.257+0000 7f6c41400700 1 --2- 172.21.15.38:0/3497585688 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6c2806cce0 0x7f6c2806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:49.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.258+0000 7f6c41400700 1 --2- 172.21.15.38:0/3497585688 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6c2806cce0 0x7f6c2806f1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c30005950 tx=0x7f6c300058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:49.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.262+0000 7f6c3aa00700 1 -- 172.21.15.38:0/3497585688 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6c4404eaf0 con 0x7f6c44101560 2024-03-24T00:44:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:49.520+0000 7f6c49200700 1 -- 172.21.15.38:0/3497585688 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c4404ea80 con 0x7f6c2806cce0 2024-03-24T00:44:50.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:50 smithi038 ceph-mon[29271]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:50 smithi088 ceph-mon[36610]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.171+0000 7f6c3aa00700 1 -- 172.21.15.38:0/3497585688 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6c4404ea80 con 0x7f6c2806cce0 2024-03-24T00:44:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.175+0000 7f6c49200700 1 -- 172.21.15.38:0/3497585688 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6c2806cce0 msgr2=0x7f6c2806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.175+0000 7f6c49200700 1 --2- 172.21.15.38:0/3497585688 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6c2806cce0 0x7f6c2806f1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c30005950 tx=0x7f6c300058e0 comp rx=0 tx=0).stop 2024-03-24T00:44:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.175+0000 7f6c49200700 1 -- 172.21.15.38:0/3497585688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c44101560 msgr2=0x7f6c441986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.175+0000 7f6c49200700 1 --2- 172.21.15.38:0/3497585688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c44101560 0x7f6c441986d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c340092e0 tx=0x7f6c3400be20 comp rx=0 tx=0).stop 2024-03-24T00:44:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.175+0000 7f6c49200700 1 -- 172.21.15.38:0/3497585688 shutdown_connections 2024-03-24T00:44:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.175+0000 7f6c49200700 1 --2- 172.21.15.38:0/3497585688 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f6c2806cce0 0x7f6c2806f1a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:51.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.175+0000 7f6c49200700 1 --2- 172.21.15.38:0/3497585688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c44101560 0x7f6c441986d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.175+0000 7f6c49200700 1 --2- 172.21.15.38:0/3497585688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c44103e90 0x7f6c44198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.175+0000 7f6c49200700 1 -- 172.21.15.38:0/3497585688 >> 172.21.15.38:0/3497585688 conn(0x7f6c440faf70 msgr2=0x7f6c440ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.175+0000 7f6c49200700 1 -- 172.21.15.38:0/3497585688 shutdown_connections 2024-03-24T00:44:51.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:51.175+0000 7f6c49200700 1 -- 172.21.15.38:0/3497585688 wait complete. 2024-03-24T00:44:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:51 smithi038 ceph-mon[29271]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:51 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-24T00:44:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:51 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-24T00:44:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:51 smithi038 ceph-mon[29271]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T00:44:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:51 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-24T00:44:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:51 smithi038 ceph-mon[29271]: pgmap v221: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:51.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:51 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[29267]: 2024-03-24T00:44:51.137+0000 7f3823000700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T00:44:51.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:51 smithi088 ceph-mon[36610]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:44:51.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:51 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-24T00:44:51.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:51 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-24T00:44:51.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:51 smithi088 ceph-mon[36610]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T00:44:51.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:51 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-24T00:44:51.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:51 smithi088 ceph-mon[36610]: pgmap v221: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:52.094 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-24T00:44:52.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T00:44:52.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: osdmap e43: 6 total, 6 up, 6 in 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: osdmap e44: 6 total, 6 up, 6 in 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: fsmap cephfs:0 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: Saving service mds.cephfs spec with placement count:4 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:44:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:52.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.yhkmsb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:44:52.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.yhkmsb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T00:44:52.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:44:52.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:52 smithi038 ceph-mon[29271]: Deploying daemon mds.cephfs.smithi038.yhkmsb on smithi038 2024-03-24T00:44:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T00:44:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-24T00:44:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: osdmap e43: 6 total, 6 up, 6 in 2024-03-24T00:44:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-24T00:44:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T00:44:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-24T00:44:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: osdmap e44: 6 total, 6 up, 6 in 2024-03-24T00:44:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-24T00:44:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: fsmap cephfs:0 2024-03-24T00:44:52.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: Saving service mds.cephfs spec with placement count:4 2024-03-24T00:44:52.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:52.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:44:52.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:44:52.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:44:52.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:52.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.yhkmsb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:44:52.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.yhkmsb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T00:44:52.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:44:52.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:52 smithi088 ceph-mon[36610]: Deploying daemon mds.cephfs.smithi038.yhkmsb on smithi038 2024-03-24T00:44:53.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:53 smithi088 ceph-mon[36610]: osdmap e45: 6 total, 6 up, 6 in 2024-03-24T00:44:53.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:53 smithi088 ceph-mon[36610]: pgmap v225: 65 pgs: 13 creating+peering, 6 active+clean, 46 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:53.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:53 smithi038 ceph-mon[29271]: osdmap e45: 6 total, 6 up, 6 in 2024-03-24T00:44:53.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:53 smithi038 ceph-mon[29271]: pgmap v225: 65 pgs: 13 creating+peering, 6 active+clean, 46 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:53.762 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:54.428 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:54 smithi038 ceph-mon[29271]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T00:44:54.429 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:54 smithi038 ceph-mon[29271]: osdmap e46: 6 total, 6 up, 6 in 2024-03-24T00:44:54.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:54 smithi088 ceph-mon[36610]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T00:44:54.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:54 smithi088 ceph-mon[36610]: osdmap e46: 6 total, 6 up, 6 in 2024-03-24T00:44:55.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:55 smithi088 ceph-mon[36610]: pgmap v227: 65 pgs: 13 creating+peering, 48 active+clean, 4 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:55.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:55 smithi038 ceph-mon[29271]: pgmap v227: 65 pgs: 13 creating+peering, 48 active+clean, 4 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.337+0000 7f0c12000700 1 -- 172.21.15.38:0/4153486559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c0c075760 msgr2=0x7f0c0c075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.337+0000 7f0c12000700 1 --2- 172.21.15.38:0/4153486559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c0c075760 0x7f0c0c075b80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009910 tx=0x7f0bfc009c20 comp rx=0 tx=0).stop 2024-03-24T00:44:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.337+0000 7f0c12000700 1 -- 172.21.15.38:0/4153486559 shutdown_connections 2024-03-24T00:44:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.337+0000 7f0c12000700 1 --2- 172.21.15.38:0/4153486559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c0c0769b0 0x7f0c0c076e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.337+0000 7f0c12000700 1 --2- 172.21.15.38:0/4153486559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c0c075760 0x7f0c0c075b80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.337+0000 7f0c12000700 1 -- 172.21.15.38:0/4153486559 >> 172.21.15.38:0/4153486559 conn(0x7f0c0c0fe2a0 msgr2=0x7f0c0c1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.337+0000 7f0c12000700 1 -- 172.21.15.38:0/4153486559 shutdown_connections 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.337+0000 7f0c12000700 1 -- 172.21.15.38:0/4153486559 wait complete. 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.339+0000 7f0c12000700 1 Processor -- start 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.339+0000 7f0c12000700 1 -- start start 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.340+0000 7f0c12000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c0c075760 0x7f0c0c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.340+0000 7f0c12000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c0c0769b0 0x7f0c0c19d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.340+0000 7f0c12000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c19d660 con 0x7f0c0c075760 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.340+0000 7f0c12000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c19d7a0 con 0x7f0c0c0769b0 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.340+0000 7f0c0b400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c0c075760 0x7f0c0c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.340+0000 7f0c0b400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c0c075760 0x7f0c0c19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39550/0 (socket says 172.21.15.38:39550) 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.340+0000 7f0c0b400700 1 -- 172.21.15.38:0/3438319552 learned_addr learned my addr 172.21.15.38:0/3438319552 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:44:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.341+0000 7f0c0aa00700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c0c0769b0 0x7f0c0c19d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.341+0000 7f0c0b400700 1 -- 172.21.15.38:0/3438319552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c0c0769b0 msgr2=0x7f0c0c19d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.341+0000 7f0c0b400700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c0c0769b0 0x7f0c0c19d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.341+0000 7f0c0b400700 1 -- 172.21.15.38:0/3438319552 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc0095f0 con 0x7f0c0c075760 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.341+0000 7f0c0aa00700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c0c0769b0 0x7f0c0c19d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.341+0000 7f0c0b400700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c0c075760 0x7f0c0c19cb00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00bd70 tx=0x7f0bfc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.341+0000 7f0c03e00700 1 -- 172.21.15.38:0/3438319552 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bfc01d070 con 0x7f0c0c075760 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.341+0000 7f0c03e00700 1 -- 172.21.15.38:0/3438319552 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bfc00f460 con 0x7f0c0c075760 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.342+0000 7f0c12000700 1 -- 172.21.15.38:0/3438319552 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c1a2150 con 0x7f0c0c075760 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.342+0000 7f0c03e00700 1 -- 172.21.15.38:0/3438319552 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bfc021620 con 0x7f0c0c075760 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.342+0000 7f0c12000700 1 -- 172.21.15.38:0/3438319552 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c1a26c0 con 0x7f0c0c075760 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.343+0000 7f0c12000700 1 -- 172.21.15.38:0/3438319552 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c196c70 con 0x7f0c0c075760 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.344+0000 7f0c03e00700 1 -- 172.21.15.38:0/3438319552 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0bfc00fa70 con 0x7f0c0c075760 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.345+0000 7f0c03e00700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bf006ccf0 0x7f0bf006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:44:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.345+0000 7f0c03e00700 1 -- 172.21.15.38:0/3438319552 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0bfc08d520 con 0x7f0c0c075760 2024-03-24T00:44:56.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.345+0000 7f0c0aa00700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bf006ccf0 0x7f0bf006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:44:56.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.346+0000 7f0c0aa00700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bf006ccf0 0x7f0bf006f1b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8005fd0 tx=0x7f0bf8005dc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:44:56.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.350+0000 7f0c03e00700 1 -- 172.21.15.38:0/3438319552 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0bfc05b400 con 0x7f0c0c075760 2024-03-24T00:44:56.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.631+0000 7f0c12000700 1 -- 172.21.15.38:0/3438319552 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0c0c0619c0 con 0x7f0c0c075760 2024-03-24T00:44:56.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.633+0000 7f0c03e00700 1 -- 172.21.15.38:0/3438319552 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1285 (secure 0 0 0) 0x7f0bfc05af90 con 0x7f0c0c075760 2024-03-24T00:44:56.633 INFO:teuthology.orchestra.run.smithi038.stdout:e4 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 4 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:44:56.189270+0000 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:44:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 1 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:in 0 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462} 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:44:56.635 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:44:56.636 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:44:56.636 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:44:56.636 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:44:56.636 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 0 2024-03-24T00:44:56.636 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:creating seq 1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:44:56.636 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:44:56.636 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:44:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.635+0000 7f0c12000700 1 -- 172.21.15.38:0/3438319552 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bf006ccf0 msgr2=0x7f0bf006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.635+0000 7f0c12000700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bf006ccf0 0x7f0bf006f1b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8005fd0 tx=0x7f0bf8005dc0 comp rx=0 tx=0).stop 2024-03-24T00:44:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.635+0000 7f0c12000700 1 -- 172.21.15.38:0/3438319552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c0c075760 msgr2=0x7f0c0c19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:44:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.635+0000 7f0c12000700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c0c075760 0x7f0c0c19cb00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00bd70 tx=0x7f0bfc00be50 comp rx=0 tx=0).stop 2024-03-24T00:44:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.636+0000 7f0c12000700 1 -- 172.21.15.38:0/3438319552 shutdown_connections 2024-03-24T00:44:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.636+0000 7f0c12000700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f0bf006ccf0 0x7f0bf006f1b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.636+0000 7f0c12000700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c0c075760 0x7f0c0c19cb00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.636+0000 7f0c12000700 1 --2- 172.21.15.38:0/3438319552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c0c0769b0 0x7f0c0c19d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:44:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.636+0000 7f0c12000700 1 -- 172.21.15.38:0/3438319552 >> 172.21.15.38:0/3438319552 conn(0x7f0c0c0fe2a0 msgr2=0x7f0c0c10d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:44:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.636+0000 7f0c12000700 1 -- 172.21.15.38:0/3438319552 shutdown_connections 2024-03-24T00:44:56.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:44:56.636+0000 7f0c12000700 1 -- 172.21.15.38:0/3438319552 wait complete. 2024-03-24T00:44:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 4 2024-03-24T00:44:57.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] up:boot 2024-03-24T00:44:57.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: daemon mds.cephfs.smithi038.yhkmsb assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-24T00:44:57.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T00:44:57.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: Cluster is now healthy 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: fsmap cephfs:0 1 up:standby 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: fsmap cephfs:1 {0=cephfs.smithi038.yhkmsb=up:creating} 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: daemon mds.cephfs.smithi038.yhkmsb is now active in filesystem cephfs as rank 0 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.cbfvvp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.cbfvvp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: Deploying daemon mds.cephfs.smithi088.cbfvvp on smithi088 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3438319552' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:44:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:57 smithi038 ceph-mon[29271]: pgmap v228: 65 pgs: 8 creating+peering, 57 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:57.479 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] up:boot 2024-03-24T00:44:57.479 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: daemon mds.cephfs.smithi038.yhkmsb assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-24T00:44:57.479 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T00:44:57.479 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-24T00:44:57.479 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: Cluster is now healthy 2024-03-24T00:44:57.479 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: fsmap cephfs:0 1 up:standby 2024-03-24T00:44:57.479 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:44:57.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: fsmap cephfs:1 {0=cephfs.smithi038.yhkmsb=up:creating} 2024-03-24T00:44:57.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: daemon mds.cephfs.smithi038.yhkmsb is now active in filesystem cephfs as rank 0 2024-03-24T00:44:57.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:57.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:57.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:44:57.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.cbfvvp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:44:57.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.cbfvvp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T00:44:57.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:44:57.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: Deploying daemon mds.cephfs.smithi088.cbfvvp on smithi088 2024-03-24T00:44:57.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3438319552' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:44:57.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:57 smithi088 ceph-mon[36610]: pgmap v228: 65 pgs: 8 creating+peering, 57 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:44:57.530 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T00:44:57.540 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:44:57.541 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-03-24T00:44:57.980 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:44:58.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:58 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] up:active 2024-03-24T00:44:58.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:58 smithi088 ceph-mon[36610]: fsmap cephfs:1 {0=cephfs.smithi038.yhkmsb=up:active} 2024-03-24T00:44:58.463 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:58 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] up:active 2024-03-24T00:44:58.464 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:58 smithi038 ceph-mon[29271]: fsmap cephfs:1 {0=cephfs.smithi038.yhkmsb=up:active} 2024-03-24T00:44:59.726 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:44:59.726 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:44:59 smithi038 ceph-mon[29271]: pgmap v229: 65 pgs: 65 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 2 op/s 2024-03-24T00:44:59.801 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:59 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:44:59.801 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:44:59 smithi088 ceph-mon[36610]: pgmap v229: 65 pgs: 65 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 2 op/s 2024-03-24T00:45:00.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.307+0000 7f7d76800700 1 -- 172.21.15.38:0/1380955968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d701080f0 msgr2=0x7f7d70108570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:00.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.307+0000 7f7d76800700 1 --2- 172.21.15.38:0/1380955968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d701080f0 0x7f7d70108570 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009910 tx=0x7f7d5c009c20 comp rx=0 tx=0).stop 2024-03-24T00:45:00.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.307+0000 7f7d76800700 1 -- 172.21.15.38:0/1380955968 shutdown_connections 2024-03-24T00:45:00.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.307+0000 7f7d76800700 1 --2- 172.21.15.38:0/1380955968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d701080f0 0x7f7d70108570 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:00.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.307+0000 7f7d76800700 1 --2- 172.21.15.38:0/1380955968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d70106f70 0x7f7d70107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:00.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.307+0000 7f7d76800700 1 -- 172.21.15.38:0/1380955968 >> 172.21.15.38:0/1380955968 conn(0x7f7d700760a0 msgr2=0x7f7d700784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:00.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.307+0000 7f7d76800700 1 -- 172.21.15.38:0/1380955968 shutdown_connections 2024-03-24T00:45:00.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.307+0000 7f7d76800700 1 -- 172.21.15.38:0/1380955968 wait complete. 2024-03-24T00:45:00.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.308+0000 7f7d76800700 1 Processor -- start 2024-03-24T00:45:00.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.309+0000 7f7d76800700 1 -- start start 2024-03-24T00:45:00.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.309+0000 7f7d76800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d70106f70 0x7f7d7019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:00.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.309+0000 7f7d76800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d701080f0 0x7f7d7019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:00.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.309+0000 7f7d76800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7019d490 con 0x7f7d701080f0 2024-03-24T00:45:00.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.309+0000 7f7d76800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7019d5d0 con 0x7f7d70106f70 2024-03-24T00:45:00.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.309+0000 7f7d74a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d701080f0 0x7f7d7019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:00.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.309+0000 7f7d74a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d701080f0 0x7f7d7019ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39562/0 (socket says 172.21.15.38:39562) 2024-03-24T00:45:00.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.309+0000 7f7d74a00700 1 -- 172.21.15.38:0/3396762459 learned_addr learned my addr 172.21.15.38:0/3396762459 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:00.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.309+0000 7f7d75400700 1 --2- 172.21.15.38:0/3396762459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d70106f70 0x7f7d7019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:00.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.310+0000 7f7d74a00700 1 -- 172.21.15.38:0/3396762459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d70106f70 msgr2=0x7f7d7019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:00.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.310+0000 7f7d74a00700 1 --2- 172.21.15.38:0/3396762459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d70106f70 0x7f7d7019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:00.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.310+0000 7f7d74a00700 1 -- 172.21.15.38:0/3396762459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c0095f0 con 0x7f7d701080f0 2024-03-24T00:45:00.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.310+0000 7f7d74a00700 1 --2- 172.21.15.38:0/3396762459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d701080f0 0x7f7d7019ce70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c006010 tx=0x7f7d5c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:00.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.310+0000 7f7d6e000700 1 -- 172.21.15.38:0/3396762459 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d5c01d070 con 0x7f7d701080f0 2024-03-24T00:45:00.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.310+0000 7f7d6e000700 1 -- 172.21.15.38:0/3396762459 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d5c005dc0 con 0x7f7d701080f0 2024-03-24T00:45:00.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.310+0000 7f7d76800700 1 -- 172.21.15.38:0/3396762459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d701a1fd0 con 0x7f7d701080f0 2024-03-24T00:45:00.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.310+0000 7f7d6e000700 1 -- 172.21.15.38:0/3396762459 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d5c021640 con 0x7f7d701080f0 2024-03-24T00:45:00.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.310+0000 7f7d76800700 1 -- 172.21.15.38:0/3396762459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d701a2510 con 0x7f7d701080f0 2024-03-24T00:45:00.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.312+0000 7f7d76800700 1 -- 172.21.15.38:0/3396762459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d70196a40 con 0x7f7d701080f0 2024-03-24T00:45:00.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.313+0000 7f7d6e000700 1 -- 172.21.15.38:0/3396762459 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7d5c00f880 con 0x7f7d701080f0 2024-03-24T00:45:00.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.313+0000 7f7d6e000700 1 --2- 172.21.15.38:0/3396762459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7d5806ccf0 0x7f7d5806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:00.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.314+0000 7f7d6e000700 1 -- 172.21.15.38:0/3396762459 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f7d5c08d520 con 0x7f7d701080f0 2024-03-24T00:45:00.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.314+0000 7f7d75400700 1 --2- 172.21.15.38:0/3396762459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7d5806ccf0 0x7f7d5806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:00.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.315+0000 7f7d75400700 1 --2- 172.21.15.38:0/3396762459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7d5806ccf0 0x7f7d5806f1b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d64005950 tx=0x7f7d640058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:00.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.319+0000 7f7d6e000700 1 -- 172.21.15.38:0/3396762459 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7d5c05b2e0 con 0x7f7d701080f0 2024-03-24T00:45:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.604+0000 7f7d76800700 1 -- 172.21.15.38:0/3396762459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f7d700619c0 con 0x7f7d701080f0 2024-03-24T00:45:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.650+0000 7f7d6e000700 1 -- 172.21.15.38:0/3396762459 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f7d5c05ae70 con 0x7f7d701080f0 2024-03-24T00:45:00.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.653+0000 7f7d76800700 1 -- 172.21.15.38:0/3396762459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7d5806ccf0 msgr2=0x7f7d5806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.653+0000 7f7d76800700 1 --2- 172.21.15.38:0/3396762459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7d5806ccf0 0x7f7d5806f1b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d64005950 tx=0x7f7d640058e0 comp rx=0 tx=0).stop 2024-03-24T00:45:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.653+0000 7f7d76800700 1 -- 172.21.15.38:0/3396762459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d701080f0 msgr2=0x7f7d7019ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.653+0000 7f7d76800700 1 --2- 172.21.15.38:0/3396762459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d701080f0 0x7f7d7019ce70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c006010 tx=0x7f7d5c00be20 comp rx=0 tx=0).stop 2024-03-24T00:45:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.653+0000 7f7d76800700 1 -- 172.21.15.38:0/3396762459 shutdown_connections 2024-03-24T00:45:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.654+0000 7f7d76800700 1 --2- 172.21.15.38:0/3396762459 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f7d5806ccf0 0x7f7d5806f1b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.654+0000 7f7d76800700 1 --2- 172.21.15.38:0/3396762459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d70106f70 0x7f7d7019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.654+0000 7f7d76800700 1 --2- 172.21.15.38:0/3396762459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d701080f0 0x7f7d7019ce70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.654+0000 7f7d76800700 1 -- 172.21.15.38:0/3396762459 >> 172.21.15.38:0/3396762459 conn(0x7f7d700760a0 msgr2=0x7f7d7010b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.654+0000 7f7d76800700 1 -- 172.21.15.38:0/3396762459 shutdown_connections 2024-03-24T00:45:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:00.654+0000 7f7d76800700 1 -- 172.21.15.38:0/3396762459 wait complete. 2024-03-24T00:45:01.517 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:01.518 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:01.518 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:01.518 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.xnpqhp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:45:01.518 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.xnpqhp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T00:45:01.518 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:45:01.518 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: Deploying daemon mds.cephfs.smithi038.xnpqhp on smithi038 2024-03-24T00:45:01.518 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3396762459' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-24T00:45:01.518 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-24T00:45:01.518 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.88:6824/1531706761,v1:172.21.15.88:6825/1531706761] up:boot 2024-03-24T00:45:01.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3396762459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-24T00:45:01.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: daemon mds.cephfs.smithi088.cbfvvp assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-24T00:45:01.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-24T00:45:01.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-24T00:45:01.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: fsmap cephfs:1 {0=cephfs.smithi038.yhkmsb=up:active} 1 up:standby 2024-03-24T00:45:01.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:45:01.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:creating} 2024-03-24T00:45:01.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: daemon mds.cephfs.smithi088.cbfvvp is now active in filesystem cephfs as rank 1 2024-03-24T00:45:01.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:01 smithi038 ceph-mon[29271]: pgmap v230: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-24T00:45:01.595 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T00:45:01.605 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:45:01.605 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-03-24T00:45:01.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:01.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:01.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:01.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.xnpqhp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:45:01.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.xnpqhp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T00:45:01.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:45:01.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: Deploying daemon mds.cephfs.smithi038.xnpqhp on smithi038 2024-03-24T00:45:01.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3396762459' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-24T00:45:01.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-24T00:45:01.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.88:6824/1531706761,v1:172.21.15.88:6825/1531706761] up:boot 2024-03-24T00:45:01.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3396762459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-24T00:45:01.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: daemon mds.cephfs.smithi088.cbfvvp assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-24T00:45:01.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-24T00:45:01.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-24T00:45:01.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: fsmap cephfs:1 {0=cephfs.smithi038.yhkmsb=up:active} 1 up:standby 2024-03-24T00:45:01.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:45:01.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:creating} 2024-03-24T00:45:01.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: daemon mds.cephfs.smithi088.cbfvvp is now active in filesystem cephfs as rank 1 2024-03-24T00:45:01.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:01 smithi088 ceph-mon[36610]: pgmap v230: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-24T00:45:02.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:02 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.88:6824/1531706761,v1:172.21.15.88:6825/1531706761] up:active 2024-03-24T00:45:02.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:02 smithi038 ceph-mon[29271]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 2024-03-24T00:45:02.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:02 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.88:6824/1531706761,v1:172.21.15.88:6825/1531706761] up:active 2024-03-24T00:45:02.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:02 smithi088 ceph-mon[36610]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 2024-03-24T00:45:03.051 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:45:03.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:03 smithi038 ceph-mon[29271]: pgmap v231: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 5 op/s 2024-03-24T00:45:03.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:03 smithi088 ceph-mon[36610]: pgmap v231: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 5 op/s 2024-03-24T00:45:04.979 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:04 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:05.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:04 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:05.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.364+0000 7f9084400700 1 -- 172.21.15.38:0/3528995782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f907c100ad0 msgr2=0x7f907c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.364+0000 7f9084400700 1 --2- 172.21.15.38:0/3528995782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f907c100ad0 0x7f907c100ef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9070009910 tx=0x7f9070009c20 comp rx=0 tx=0).stop 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.364+0000 7f9084400700 1 -- 172.21.15.38:0/3528995782 shutdown_connections 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.364+0000 7f9084400700 1 --2- 172.21.15.38:0/3528995782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c101cd0 0x7f907c102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.364+0000 7f9084400700 1 --2- 172.21.15.38:0/3528995782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f907c100ad0 0x7f907c100ef0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.364+0000 7f9084400700 1 -- 172.21.15.38:0/3528995782 >> 172.21.15.38:0/3528995782 conn(0x7f907c0fc090 msgr2=0x7f907c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.364+0000 7f9084400700 1 -- 172.21.15.38:0/3528995782 shutdown_connections 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.364+0000 7f9084400700 1 -- 172.21.15.38:0/3528995782 wait complete. 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.365+0000 7f9084400700 1 Processor -- start 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.365+0000 7f9084400700 1 -- start start 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9084400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c100ad0 0x7f907c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9084400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f907c101cd0 0x7f907c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9084400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c199350 con 0x7f907c101cd0 2024-03-24T00:45:05.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9084400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c199490 con 0x7f907c100ad0 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9081000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f907c101cd0 0x7f907c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9081000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f907c101cd0 0x7f907c198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36066/0 (socket says 172.21.15.38:36066) 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9081000700 1 -- 172.21.15.38:0/1703680571 learned_addr learned my addr 172.21.15.38:0/1703680571 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9081a00700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c100ad0 0x7f907c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9081000700 1 -- 172.21.15.38:0/1703680571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c100ad0 msgr2=0x7f907c1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9081000700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c100ad0 0x7f907c1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9081000700 1 -- 172.21.15.38:0/1703680571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700095f0 con 0x7f907c101cd0 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.366+0000 7f9081a00700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c100ad0 0x7f907c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.367+0000 7f9081000700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f907c101cd0 0x7f907c198d30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f906800b4b0 tx=0x7f906800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.367+0000 7f907a000700 1 -- 172.21.15.38:0/1703680571 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9068011800 con 0x7f907c101cd0 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.367+0000 7f907a000700 1 -- 172.21.15.38:0/1703680571 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9068011e20 con 0x7f907c101cd0 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.367+0000 7f9084400700 1 -- 172.21.15.38:0/1703680571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907c19def0 con 0x7f907c101cd0 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.367+0000 7f907a000700 1 -- 172.21.15.38:0/1703680571 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f906800f2e0 con 0x7f907c101cd0 2024-03-24T00:45:05.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.367+0000 7f9084400700 1 -- 172.21.15.38:0/1703680571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907c19e490 con 0x7f907c101cd0 2024-03-24T00:45:05.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.368+0000 7f9084400700 1 -- 172.21.15.38:0/1703680571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907c1066a0 con 0x7f907c101cd0 2024-03-24T00:45:05.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.370+0000 7f907a000700 1 -- 172.21.15.38:0/1703680571 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9068010380 con 0x7f907c101cd0 2024-03-24T00:45:05.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.371+0000 7f907a000700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f906406ccf0 0x7f906406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:05.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.371+0000 7f907a000700 1 -- 172.21.15.38:0/1703680571 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f906808bdf0 con 0x7f907c101cd0 2024-03-24T00:45:05.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.371+0000 7f9081a00700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f906406ccf0 0x7f906406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:05.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.371+0000 7f9081a00700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f906406ccf0 0x7f906406f1b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9070006010 tx=0x7f907000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:05.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.377+0000 7f907a000700 1 -- 172.21.15.38:0/1703680571 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9068059c50 con 0x7f907c101cd0 2024-03-24T00:45:05.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.663+0000 7f9084400700 1 -- 172.21.15.38:0/1703680571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f907c04ea80 con 0x7f907c101cd0 2024-03-24T00:45:05.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.748+0000 7f907a000700 1 -- 172.21.15.38:0/1703680571 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7f90680597e0 con 0x7f907c101cd0 2024-03-24T00:45:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.753+0000 7f9084400700 1 -- 172.21.15.38:0/1703680571 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f906406ccf0 msgr2=0x7f906406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.753+0000 7f9084400700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f906406ccf0 0x7f906406f1b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9070006010 tx=0x7f907000b540 comp rx=0 tx=0).stop 2024-03-24T00:45:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.754+0000 7f9084400700 1 -- 172.21.15.38:0/1703680571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f907c101cd0 msgr2=0x7f907c198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.754+0000 7f9084400700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f907c101cd0 0x7f907c198d30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f906800b4b0 tx=0x7f906800b870 comp rx=0 tx=0).stop 2024-03-24T00:45:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.754+0000 7f9084400700 1 -- 172.21.15.38:0/1703680571 shutdown_connections 2024-03-24T00:45:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.754+0000 7f9084400700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f906406ccf0 0x7f906406f1b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.754+0000 7f9084400700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c100ad0 0x7f907c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:05.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.754+0000 7f9084400700 1 --2- 172.21.15.38:0/1703680571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f907c101cd0 0x7f907c198d30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.754+0000 7f9084400700 1 -- 172.21.15.38:0/1703680571 >> 172.21.15.38:0/1703680571 conn(0x7f907c0fc090 msgr2=0x7f907c073430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.754+0000 7f9084400700 1 -- 172.21.15.38:0/1703680571 shutdown_connections 2024-03-24T00:45:05.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:05.754+0000 7f9084400700 1 -- 172.21.15.38:0/1703680571 wait complete. 2024-03-24T00:45:06.001 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:05 smithi038 ceph-mon[29271]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-24T00:45:06.002 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:06.002 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:06.002 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:05 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1703680571' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-24T00:45:06.002 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:06.002 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.yhksbr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:45:06.002 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.yhksbr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T00:45:06.002 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:05 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:45:06.002 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:05 smithi038 ceph-mon[29271]: Deploying daemon mds.cephfs.smithi088.yhksbr on smithi088 2024-03-24T00:45:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:05 smithi088 ceph-mon[36610]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-24T00:45:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:05 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1703680571' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-24T00:45:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.yhksbr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:45:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.yhksbr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T00:45:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:05 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:45:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:05 smithi088 ceph-mon[36610]: Deploying daemon mds.cephfs.smithi088.yhksbr on smithi088 2024-03-24T00:45:06.573 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T00:45:07.222 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:45:07.222 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-03-24T00:45:07.223 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:06 smithi088 ceph-mon[36610]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:06 smithi088 ceph-mon[36610]: Cluster is now healthy 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:06 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] up:boot 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:06 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1703680571' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:06 smithi088 ceph-mon[36610]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 1 up:standby 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:06 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:06 smithi088 ceph-mon[36610]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 1 up:standby-replay 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:06 smithi088 ceph-mon[36610]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:06 smithi038 ceph-mon[29271]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:06 smithi038 ceph-mon[29271]: Cluster is now healthy 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:06 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] up:boot 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:06 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1703680571' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-24T00:45:07.224 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:06 smithi038 ceph-mon[29271]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 1 up:standby 2024-03-24T00:45:07.225 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:06 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T00:45:07.225 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:06 smithi038 ceph-mon[29271]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 1 up:standby-replay 2024-03-24T00:45:07.225 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:06 smithi038 ceph-mon[29271]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-03-24T00:45:07.676 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:45:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.869+0000 7f47bfe00700 1 -- 172.21.15.38:0/2495754520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b8107130 msgr2=0x7f47b8107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.869+0000 7f47bfe00700 1 --2- 172.21.15.38:0/2495754520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b8107130 0x7f47b8107550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009910 tx=0x7f47ac009c20 comp rx=0 tx=0).stop 2024-03-24T00:45:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.869+0000 7f47bfe00700 1 -- 172.21.15.38:0/2495754520 shutdown_connections 2024-03-24T00:45:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.869+0000 7f47bfe00700 1 --2- 172.21.15.38:0/2495754520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47b8108300 0x7f47b8108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.869+0000 7f47bfe00700 1 --2- 172.21.15.38:0/2495754520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b8107130 0x7f47b8107550 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.869+0000 7f47bfe00700 1 -- 172.21.15.38:0/2495754520 >> 172.21.15.38:0/2495754520 conn(0x7f47b80762d0 msgr2=0x7f47b8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.869+0000 7f47bfe00700 1 -- 172.21.15.38:0/2495754520 shutdown_connections 2024-03-24T00:45:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.869+0000 7f47bfe00700 1 -- 172.21.15.38:0/2495754520 wait complete. 2024-03-24T00:45:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.870+0000 7f47bfe00700 1 Processor -- start 2024-03-24T00:45:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.870+0000 7f47bfe00700 1 -- start start 2024-03-24T00:45:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bfe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b8107130 0x7f47b819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bfe00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47b8108300 0x7f47b819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bfe00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b819d5e0 con 0x7f47b8107130 2024-03-24T00:45:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bfe00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b819d720 con 0x7f47b8108300 2024-03-24T00:45:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bd400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b8107130 0x7f47b819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bd400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b8107130 0x7f47b819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36086/0 (socket says 172.21.15.38:36086) 2024-03-24T00:45:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bd400700 1 -- 172.21.15.38:0/4112588488 learned_addr learned my addr 172.21.15.38:0/4112588488 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bca00700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47b8108300 0x7f47b819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bd400700 1 -- 172.21.15.38:0/4112588488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47b8108300 msgr2=0x7f47b819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bd400700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47b8108300 0x7f47b819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bd400700 1 -- 172.21.15.38:0/4112588488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0095f0 con 0x7f47b8107130 2024-03-24T00:45:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.871+0000 7f47bca00700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47b8108300 0x7f47b819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:45:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.872+0000 7f47bd400700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b8107130 0x7f47b819ca80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00b800 tx=0x7f47ac00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.872+0000 7f47b6000700 1 -- 172.21.15.38:0/4112588488 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47ac01d070 con 0x7f47b8107130 2024-03-24T00:45:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.872+0000 7f47b6000700 1 -- 172.21.15.38:0/4112588488 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47ac0052a0 con 0x7f47b8107130 2024-03-24T00:45:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.872+0000 7f47bfe00700 1 -- 172.21.15.38:0/4112588488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b81a2120 con 0x7f47b8107130 2024-03-24T00:45:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.872+0000 7f47b6000700 1 -- 172.21.15.38:0/4112588488 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47ac017400 con 0x7f47b8107130 2024-03-24T00:45:09.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.872+0000 7f47bfe00700 1 -- 172.21.15.38:0/4112588488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b81a2690 con 0x7f47b8107130 2024-03-24T00:45:09.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.874+0000 7f47bfe00700 1 -- 172.21.15.38:0/4112588488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b8196ca0 con 0x7f47b8107130 2024-03-24T00:45:09.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.874+0000 7f47b6000700 1 -- 172.21.15.38:0/4112588488 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f47ac0037f0 con 0x7f47b8107130 2024-03-24T00:45:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.875+0000 7f47b6000700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f47a006cce0 0x7f47a006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.876+0000 7f47b6000700 1 -- 172.21.15.38:0/4112588488 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f47ac08d790 con 0x7f47b8107130 2024-03-24T00:45:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.876+0000 7f47bca00700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f47a006cce0 0x7f47a006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.877+0000 7f47bca00700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f47a006cce0 0x7f47a006f1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009cb0 tx=0x7f47a4009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:09.881+0000 7f47b6000700 1 -- 172.21.15.38:0/4112588488 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f47ac05b5f0 con 0x7f47b8107130 2024-03-24T00:45:09.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:09 smithi088 ceph-mon[36610]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 341 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-03-24T00:45:09.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:09.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:09.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:09.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:09.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:09 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:09 smithi038 ceph-mon[29271]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 341 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-03-24T00:45:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:09 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:10.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.163+0000 7f47bfe00700 1 -- 172.21.15.38:0/4112588488 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f47b802cf60 con 0x7f47b8107130 2024-03-24T00:45:10.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.671+0000 7f47b6000700 1 -- 172.21.15.38:0/4112588488 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v13) v1 ==== 133+0+0 (secure 0 0 0) 0x7f47ac05b180 con 0x7f47b8107130 2024-03-24T00:45:10.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.674+0000 7f47bfe00700 1 -- 172.21.15.38:0/4112588488 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f47a006cce0 msgr2=0x7f47a006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:10.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.674+0000 7f47bfe00700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f47a006cce0 0x7f47a006f1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009cb0 tx=0x7f47a4009260 comp rx=0 tx=0).stop 2024-03-24T00:45:10.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.674+0000 7f47bfe00700 1 -- 172.21.15.38:0/4112588488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b8107130 msgr2=0x7f47b819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:10.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.674+0000 7f47bfe00700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b8107130 0x7f47b819ca80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00b800 tx=0x7f47ac00f510 comp rx=0 tx=0).stop 2024-03-24T00:45:10.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.674+0000 7f47bfe00700 1 -- 172.21.15.38:0/4112588488 shutdown_connections 2024-03-24T00:45:10.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.674+0000 7f47bfe00700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f47a006cce0 0x7f47a006f1a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:10.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.674+0000 7f47bfe00700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47b8107130 0x7f47b819ca80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:10.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.674+0000 7f47bfe00700 1 --2- 172.21.15.38:0/4112588488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47b8108300 0x7f47b819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:10.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.674+0000 7f47bfe00700 1 -- 172.21.15.38:0/4112588488 >> 172.21.15.38:0/4112588488 conn(0x7f47b80762d0 msgr2=0x7f47b810b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:10.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.674+0000 7f47bfe00700 1 -- 172.21.15.38:0/4112588488 shutdown_connections 2024-03-24T00:45:10.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:10.674+0000 7f47bfe00700 1 -- 172.21.15.38:0/4112588488 wait complete. 2024-03-24T00:45:10.678 INFO:teuthology.orchestra.run.smithi038.stderr:inline data disabled 2024-03-24T00:45:10.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:10 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:boot 2024-03-24T00:45:10.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:10 smithi088 ceph-mon[36610]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 1 up:standby-replay 1 up:standby 2024-03-24T00:45:10.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:10 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T00:45:10.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:10 smithi088 ceph-mon[36610]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 2 up:standby-replay 2024-03-24T00:45:10.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:10 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:45:10.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:10 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/4112588488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-24T00:45:11.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:10 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:boot 2024-03-24T00:45:11.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:10 smithi038 ceph-mon[29271]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 1 up:standby-replay 1 up:standby 2024-03-24T00:45:11.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:10 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T00:45:11.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:10 smithi038 ceph-mon[29271]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 2 up:standby-replay 2024-03-24T00:45:11.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:10 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:45:11.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:10 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/4112588488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-24T00:45:11.702 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T00:45:11.712 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:45:11.712 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-24T00:45:11.933 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:11 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/4112588488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-24T00:45:11.933 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:11 smithi038 ceph-mon[29271]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 2 up:standby-replay 2024-03-24T00:45:11.933 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:11 smithi038 ceph-mon[29271]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-24T00:45:12.046 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:11 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/4112588488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-24T00:45:12.046 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:11 smithi088 ceph-mon[36610]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 2 up:standby-replay 2024-03-24T00:45:12.046 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:11 smithi088 ceph-mon[36610]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-24T00:45:12.275 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:45:13.960 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] up:active 2024-03-24T00:45:13.960 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 2 up:standby-replay 2024-03-24T00:45:13.960 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 1023 B/s wr, 6 op/s 2024-03-24T00:45:13.960 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:13.960 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:13.960 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: Dropping low affinity active daemon mds.cephfs.smithi088.cbfvvp in favor of higher affinity standby. 2024-03-24T00:45:13.960 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: Replacing daemon mds.cephfs.smithi088.cbfvvp as rank 1 with standby daemon mds.cephfs.smithi088.yhksbr 2024-03-24T00:45:13.960 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T00:45:13.960 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: osdmap e47: 6 total, 6 up, 6 in 2024-03-24T00:45:13.960 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:standby-replay 2024-03-24T00:45:13.961 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] up:standby-replay 2024-03-24T00:45:13.961 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:replay} 1 up:standby-replay 2024-03-24T00:45:13.961 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:13.961 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] up:active 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.cbfvvp=up:active} 2 up:standby-replay 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 1023 B/s wr, 6 op/s 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: Dropping low affinity active daemon mds.cephfs.smithi088.cbfvvp in favor of higher affinity standby. 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: Replacing daemon mds.cephfs.smithi088.cbfvvp as rank 1 with standby daemon mds.cephfs.smithi088.yhksbr 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: osdmap e47: 6 total, 6 up, 6 in 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:standby-replay 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] up:standby-replay 2024-03-24T00:45:14.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:replay} 1 up:standby-replay 2024-03-24T00:45:14.182 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:14.182 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:45:14.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:14.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:14 smithi038 ceph-mon[29271]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T00:45:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:14 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:resolve 2024-03-24T00:45:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:14 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] up:boot 2024-03-24T00:45:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:14 smithi038 ceph-mon[29271]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:resolve} 1 up:standby-replay 1 up:standby 2024-03-24T00:45:14.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:14 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:45:14.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.891+0000 7f3aee000700 1 -- 172.21.15.38:0/2921578359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8101cd0 msgr2=0x7f3ae8102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:14.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.891+0000 7f3aee000700 1 --2- 172.21.15.38:0/2921578359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8101cd0 0x7f3ae8102130 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009910 tx=0x7f3ad4009c20 comp rx=0 tx=0).stop 2024-03-24T00:45:14.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.891+0000 7f3aee000700 1 -- 172.21.15.38:0/2921578359 shutdown_connections 2024-03-24T00:45:14.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.891+0000 7f3aee000700 1 --2- 172.21.15.38:0/2921578359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8101cd0 0x7f3ae8102130 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:14.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.891+0000 7f3aee000700 1 --2- 172.21.15.38:0/2921578359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ae8100ad0 0x7f3ae8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:14.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.891+0000 7f3aee000700 1 -- 172.21.15.38:0/2921578359 >> 172.21.15.38:0/2921578359 conn(0x7f3ae80fc090 msgr2=0x7f3ae80fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:14.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.891+0000 7f3aee000700 1 -- 172.21.15.38:0/2921578359 shutdown_connections 2024-03-24T00:45:14.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.891+0000 7f3aee000700 1 -- 172.21.15.38:0/2921578359 wait complete. 2024-03-24T00:45:14.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.892+0000 7f3aee000700 1 Processor -- start 2024-03-24T00:45:14.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.892+0000 7f3aee000700 1 -- start start 2024-03-24T00:45:14.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.893+0000 7f3aee000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100ad0 0x7f3ae81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:14.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.893+0000 7f3aee000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ae8101cd0 0x7f3ae8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:14.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.893+0000 7f3aee000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae8199340 con 0x7f3ae8100ad0 2024-03-24T00:45:14.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.893+0000 7f3aee000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae8199480 con 0x7f3ae8101cd0 2024-03-24T00:45:14.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.893+0000 7f3ae7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100ad0 0x7f3ae81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:14.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.893+0000 7f3ae7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100ad0 0x7f3ae81987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50048/0 (socket says 172.21.15.38:50048) 2024-03-24T00:45:14.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.893+0000 7f3ae7400700 1 -- 172.21.15.38:0/2154818364 learned_addr learned my addr 172.21.15.38:0/2154818364 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:14.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.894+0000 7f3ae6a00700 1 --2- 172.21.15.38:0/2154818364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ae8101cd0 0x7f3ae8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:14.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.894+0000 7f3ae7400700 1 -- 172.21.15.38:0/2154818364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ae8101cd0 msgr2=0x7f3ae8198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:14.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.894+0000 7f3ae7400700 1 --2- 172.21.15.38:0/2154818364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ae8101cd0 0x7f3ae8198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:14.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.894+0000 7f3ae7400700 1 -- 172.21.15.38:0/2154818364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad40095f0 con 0x7f3ae8100ad0 2024-03-24T00:45:14.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.894+0000 7f3ae7400700 1 --2- 172.21.15.38:0/2154818364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100ad0 0x7f3ae81987e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800b480 tx=0x7f3ad800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:14.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.894+0000 7f3adfe00700 1 -- 172.21.15.38:0/2154818364 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ad80118c0 con 0x7f3ae8100ad0 2024-03-24T00:45:14.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.894+0000 7f3adfe00700 1 -- 172.21.15.38:0/2154818364 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ad8010380 con 0x7f3ae8100ad0 2024-03-24T00:45:14.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.894+0000 7f3adfe00700 1 -- 172.21.15.38:0/2154818364 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ad800f4f0 con 0x7f3ae8100ad0 2024-03-24T00:45:14.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.895+0000 7f3aee000700 1 -- 172.21.15.38:0/2154818364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae819dee0 con 0x7f3ae8100ad0 2024-03-24T00:45:14.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.895+0000 7f3aee000700 1 -- 172.21.15.38:0/2154818364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae819e420 con 0x7f3ae8100ad0 2024-03-24T00:45:14.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.896+0000 7f3aee000700 1 -- 172.21.15.38:0/2154818364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae81066a0 con 0x7f3ae8100ad0 2024-03-24T00:45:14.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.897+0000 7f3adfe00700 1 -- 172.21.15.38:0/2154818364 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3ad80104f0 con 0x7f3ae8100ad0 2024-03-24T00:45:14.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.899+0000 7f3adfe00700 1 --2- 172.21.15.38:0/2154818364 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3acc06ccf0 0x7f3acc06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:14.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.899+0000 7f3adfe00700 1 -- 172.21.15.38:0/2154818364 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3ad808bed0 con 0x7f3ae8100ad0 2024-03-24T00:45:14.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.899+0000 7f3ae6a00700 1 --2- 172.21.15.38:0/2154818364 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3acc06ccf0 0x7f3acc06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:14.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.900+0000 7f3ae6a00700 1 --2- 172.21.15.38:0/2154818364 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3acc06ccf0 0x7f3acc06f1b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40051c0 tx=0x7f3ad400b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:14.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:14.904+0000 7f3adfe00700 1 -- 172.21.15.38:0/2154818364 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3ad8059cb0 con 0x7f3ae8100ad0 2024-03-24T00:45:15.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:15.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:14 smithi088 ceph-mon[36610]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T00:45:15.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:14 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:resolve 2024-03-24T00:45:15.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:14 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] up:boot 2024-03-24T00:45:15.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:14 smithi088 ceph-mon[36610]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:resolve} 1 up:standby-replay 1 up:standby 2024-03-24T00:45:15.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:14 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:45:15.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.193+0000 7f3aee000700 1 -- 172.21.15.38:0/2154818364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3ae804ea80 con 0x7f3ae8100ad0 2024-03-24T00:45:15.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.194+0000 7f3adfe00700 1 -- 172.21.15.38:0/2154818364 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1882 (secure 0 0 0) 0x7f3ad8015770 con 0x7f3ae8100ad0 2024-03-24T00:45:15.197 INFO:teuthology.orchestra.run.smithi038.stdout:e16 2024-03-24T00:45:15.197 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:45:15.197 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:45:15.197 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:45:15.197 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:45:15.197 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:45:15.197 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:45:15.197 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 16 2024-03-24T00:45:15.197 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:45:15.197 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:14.698414+0000 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:45:15.198 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:resolve seq 3 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:45:15.199 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:45:15.200 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-03-24T00:45:15.200 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:45:15.200 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{-1:24279} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:45:15.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.198+0000 7f3aee000700 1 -- 172.21.15.38:0/2154818364 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3acc06ccf0 msgr2=0x7f3acc06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:15.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.198+0000 7f3aee000700 1 --2- 172.21.15.38:0/2154818364 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3acc06ccf0 0x7f3acc06f1b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40051c0 tx=0x7f3ad400b540 comp rx=0 tx=0).stop 2024-03-24T00:45:15.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.198+0000 7f3aee000700 1 -- 172.21.15.38:0/2154818364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100ad0 msgr2=0x7f3ae81987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:15.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.198+0000 7f3aee000700 1 --2- 172.21.15.38:0/2154818364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100ad0 0x7f3ae81987e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800b480 tx=0x7f3ad800b840 comp rx=0 tx=0).stop 2024-03-24T00:45:15.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.199+0000 7f3aee000700 1 -- 172.21.15.38:0/2154818364 shutdown_connections 2024-03-24T00:45:15.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.199+0000 7f3aee000700 1 --2- 172.21.15.38:0/2154818364 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f3acc06ccf0 0x7f3acc06f1b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:15.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.199+0000 7f3aee000700 1 --2- 172.21.15.38:0/2154818364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100ad0 0x7f3ae81987e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:15.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.199+0000 7f3aee000700 1 --2- 172.21.15.38:0/2154818364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ae8101cd0 0x7f3ae8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:15.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.199+0000 7f3aee000700 1 -- 172.21.15.38:0/2154818364 >> 172.21.15.38:0/2154818364 conn(0x7f3ae80fc090 msgr2=0x7f3ae8073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:15.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.199+0000 7f3aee000700 1 -- 172.21.15.38:0/2154818364 shutdown_connections 2024-03-24T00:45:15.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:15.199+0000 7f3aee000700 1 -- 172.21.15.38:0/2154818364 wait complete. 2024-03-24T00:45:15.201 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 16 2024-03-24T00:45:15.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:15 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2154818364' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:45:15.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:15 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:reconnect 2024-03-24T00:45:15.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:15 smithi038 ceph-mon[29271]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:reconnect} 1 up:standby-replay 1 up:standby 2024-03-24T00:45:16.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:15 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2154818364' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:45:16.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:15 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:reconnect 2024-03-24T00:45:16.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:15 smithi088 ceph-mon[36610]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:reconnect} 1 up:standby-replay 1 up:standby 2024-03-24T00:45:16.232 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-24T00:45:17.049 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:45:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:17 smithi038 ceph-mon[29271]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T00:45:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:17 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:rejoin 2024-03-24T00:45:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:17 smithi038 ceph-mon[29271]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:rejoin} 1 up:standby-replay 1 up:standby 2024-03-24T00:45:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:17 smithi038 ceph-mon[29271]: daemon mds.cephfs.smithi088.yhksbr is now active in filesystem cephfs as rank 1 2024-03-24T00:45:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:17 smithi088 ceph-mon[36610]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T00:45:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:17 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:rejoin 2024-03-24T00:45:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:17 smithi088 ceph-mon[36610]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:rejoin} 1 up:standby-replay 1 up:standby 2024-03-24T00:45:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:17 smithi088 ceph-mon[36610]: daemon mds.cephfs.smithi088.yhksbr is now active in filesystem cephfs as rank 1 2024-03-24T00:45:18.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:18 smithi038 ceph-mon[29271]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T00:45:18.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:18 smithi038 ceph-mon[29271]: Cluster is now healthy 2024-03-24T00:45:18.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:18 smithi038 ceph-mon[29271]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:active 2024-03-24T00:45:18.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:18 smithi038 ceph-mon[29271]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:active} 1 up:standby-replay 1 up:standby 2024-03-24T00:45:18.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:18 smithi038 ceph-mon[29271]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:active} 2 up:standby-replay 2024-03-24T00:45:19.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:18 smithi088 ceph-mon[36610]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T00:45:19.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:18 smithi088 ceph-mon[36610]: Cluster is now healthy 2024-03-24T00:45:19.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:18 smithi088 ceph-mon[36610]: mds.? [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] up:active 2024-03-24T00:45:19.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:18 smithi088 ceph-mon[36610]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:active} 1 up:standby-replay 1 up:standby 2024-03-24T00:45:19.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:18 smithi088 ceph-mon[36610]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:active} 2 up:standby-replay 2024-03-24T00:45:19.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.673+0000 7fb669200700 1 -- 172.21.15.38:0/445592139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb664102d00 msgr2=0x7fb664103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:19.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.673+0000 7fb669200700 1 --2- 172.21.15.38:0/445592139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb664102d00 0x7fb664103120 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb654009910 tx=0x7fb654009c20 comp rx=0 tx=0).stop 2024-03-24T00:45:19.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.673+0000 7fb669200700 1 -- 172.21.15.38:0/445592139 shutdown_connections 2024-03-24T00:45:19.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.673+0000 7fb669200700 1 --2- 172.21.15.38:0/445592139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb664103f00 0x7fb664104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:19.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.673+0000 7fb669200700 1 --2- 172.21.15.38:0/445592139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb664102d00 0x7fb664103120 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:19.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.673+0000 7fb669200700 1 -- 172.21.15.38:0/445592139 >> 172.21.15.38:0/445592139 conn(0x7fb6640fe2a0 msgr2=0x7fb6641006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:19.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.673+0000 7fb669200700 1 -- 172.21.15.38:0/445592139 shutdown_connections 2024-03-24T00:45:19.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.674+0000 7fb669200700 1 -- 172.21.15.38:0/445592139 wait complete. 2024-03-24T00:45:19.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.674+0000 7fb669200700 1 Processor -- start 2024-03-24T00:45:19.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.675+0000 7fb669200700 1 -- start start 2024-03-24T00:45:19.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.675+0000 7fb669200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb664102d00 0x7fb6641986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:19.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.675+0000 7fb669200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb664103f00 0x7fb664198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:19.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.675+0000 7fb669200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb664199190 con 0x7fb664103f00 2024-03-24T00:45:19.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.675+0000 7fb669200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641992d0 con 0x7fb664102d00 2024-03-24T00:45:19.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.675+0000 7fb661400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb664103f00 0x7fb664198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:19.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.675+0000 7fb661400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb664103f00 0x7fb664198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50070/0 (socket says 172.21.15.38:50070) 2024-03-24T00:45:19.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.675+0000 7fb661400700 1 -- 172.21.15.38:0/3103136404 learned_addr learned my addr 172.21.15.38:0/3103136404 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:19.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.676+0000 7fb661e00700 1 --2- 172.21.15.38:0/3103136404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb664102d00 0x7fb6641986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:19.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.676+0000 7fb661400700 1 -- 172.21.15.38:0/3103136404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb664102d00 msgr2=0x7fb6641986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:19.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.676+0000 7fb661400700 1 --2- 172.21.15.38:0/3103136404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb664102d00 0x7fb6641986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:19.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.676+0000 7fb661400700 1 -- 172.21.15.38:0/3103136404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6540095f0 con 0x7fb664103f00 2024-03-24T00:45:19.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.676+0000 7fb661400700 1 --2- 172.21.15.38:0/3103136404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb664103f00 0x7fb664198c00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb65000c930 tx=0x7fb65000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:19.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.677+0000 7fb65aa00700 1 -- 172.21.15.38:0/3103136404 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb650007ad0 con 0x7fb664103f00 2024-03-24T00:45:19.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.677+0000 7fb65aa00700 1 -- 172.21.15.38:0/3103136404 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb65000ce80 con 0x7fb664103f00 2024-03-24T00:45:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.677+0000 7fb65aa00700 1 -- 172.21.15.38:0/3103136404 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb650018630 con 0x7fb664103f00 2024-03-24T00:45:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.677+0000 7fb669200700 1 -- 172.21.15.38:0/3103136404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66419dd40 con 0x7fb664103f00 2024-03-24T00:45:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.677+0000 7fb669200700 1 -- 172.21.15.38:0/3103136404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66419e290 con 0x7fb664103f00 2024-03-24T00:45:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.678+0000 7fb669200700 1 -- 172.21.15.38:0/3103136404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb664192880 con 0x7fb664103f00 2024-03-24T00:45:19.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.679+0000 7fb65aa00700 1 -- 172.21.15.38:0/3103136404 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb650007c30 con 0x7fb664103f00 2024-03-24T00:45:19.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.681+0000 7fb65aa00700 1 --2- 172.21.15.38:0/3103136404 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb64806cce0 0x7fb64806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:19.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.681+0000 7fb65aa00700 1 -- 172.21.15.38:0/3103136404 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb65008c270 con 0x7fb664103f00 2024-03-24T00:45:19.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.681+0000 7fb661e00700 1 --2- 172.21.15.38:0/3103136404 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb64806cce0 0x7fb64806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:19.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.682+0000 7fb661e00700 1 --2- 172.21.15.38:0/3103136404 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb64806cce0 0x7fb64806f1a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb654000c00 tx=0x7fb654005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:19.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.686+0000 7fb65aa00700 1 -- 172.21.15.38:0/3103136404 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb65005a070 con 0x7fb664103f00 2024-03-24T00:45:19.854 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:19 smithi038 ceph-mon[29271]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 4.2 KiB/s rd, 5 op/s 2024-03-24T00:45:19.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.994+0000 7fb669200700 1 -- 172.21.15.38:0/3103136404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb66402cee0 con 0x7fb664103f00 2024-03-24T00:45:19.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.995+0000 7fb65aa00700 1 -- 172.21.15.38:0/3103136404 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4856 (secure 0 0 0) 0x7fb650059c00 con 0x7fb664103f00 2024-03-24T00:45:19.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.998+0000 7fb669200700 1 -- 172.21.15.38:0/3103136404 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb64806cce0 msgr2=0x7fb64806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:19.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.998+0000 7fb669200700 1 --2- 172.21.15.38:0/3103136404 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb64806cce0 0x7fb64806f1a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb654000c00 tx=0x7fb654005960 comp rx=0 tx=0).stop 2024-03-24T00:45:20.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.998+0000 7fb669200700 1 -- 172.21.15.38:0/3103136404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb664103f00 msgr2=0x7fb664198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:20.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.998+0000 7fb669200700 1 --2- 172.21.15.38:0/3103136404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb664103f00 0x7fb664198c00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb65000c930 tx=0x7fb65000cc40 comp rx=0 tx=0).stop 2024-03-24T00:45:20.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.998+0000 7fb669200700 1 -- 172.21.15.38:0/3103136404 shutdown_connections 2024-03-24T00:45:20.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.998+0000 7fb669200700 1 --2- 172.21.15.38:0/3103136404 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb64806cce0 0x7fb64806f1a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:20.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.998+0000 7fb669200700 1 --2- 172.21.15.38:0/3103136404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb664102d00 0x7fb6641986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:20.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.998+0000 7fb669200700 1 --2- 172.21.15.38:0/3103136404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb664103f00 0x7fb664198c00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:20.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.998+0000 7fb669200700 1 -- 172.21.15.38:0/3103136404 >> 172.21.15.38:0/3103136404 conn(0x7fb6640fe2a0 msgr2=0x7fb6641074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:20.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.999+0000 7fb669200700 1 -- 172.21.15.38:0/3103136404 shutdown_connections 2024-03-24T00:45:20.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:19.999+0000 7fb669200700 1 -- 172.21.15.38:0/3103136404 wait complete. 2024-03-24T00:45:20.001 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:45:20.015 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:45:20.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:19 smithi088 ceph-mon[36610]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 4.2 KiB/s rd, 5 op/s 2024-03-24T00:45:20.859 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-24T00:45:20.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:20.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:20.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:45:20.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:45:20.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:20.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:20 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3103136404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T00:45:20.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:20 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:45:20.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:20 smithi088 ceph-mon[36610]: pgmap v241: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 7.1 KiB/s rd, 8 op/s 2024-03-24T00:45:21.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:21.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:21.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:45:21.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:45:21.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:21.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:20 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3103136404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T00:45:21.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:20 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:45:21.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:20 smithi038 ceph-mon[29271]: pgmap v241: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 7.1 KiB/s rd, 8 op/s 2024-03-24T00:45:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:45:23.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:22 smithi038 ceph-mon[29271]: pgmap v242: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 9.8 KiB/s rd, 11 op/s 2024-03-24T00:45:23.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:22 smithi088 ceph-mon[36610]: pgmap v242: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 9.8 KiB/s rd, 11 op/s 2024-03-24T00:45:24.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:24 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:24.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:24 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:24.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.519+0000 7fb757800700 1 -- 172.21.15.38:0/3954626633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb750102d00 msgr2=0x7fb750103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.519+0000 7fb757800700 1 --2- 172.21.15.38:0/3954626633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb750102d00 0x7fb750103120 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb744009910 tx=0x7fb744009c20 comp rx=0 tx=0).stop 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.519+0000 7fb757800700 1 -- 172.21.15.38:0/3954626633 shutdown_connections 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.519+0000 7fb757800700 1 --2- 172.21.15.38:0/3954626633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb750103f00 0x7fb750104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.519+0000 7fb757800700 1 --2- 172.21.15.38:0/3954626633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb750102d00 0x7fb750103120 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.519+0000 7fb757800700 1 -- 172.21.15.38:0/3954626633 >> 172.21.15.38:0/3954626633 conn(0x7fb7500fe2c0 msgr2=0x7fb7501006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.519+0000 7fb757800700 1 -- 172.21.15.38:0/3954626633 shutdown_connections 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.519+0000 7fb757800700 1 -- 172.21.15.38:0/3954626633 wait complete. 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.520+0000 7fb757800700 1 Processor -- start 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.521+0000 7fb757800700 1 -- start start 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.521+0000 7fb757800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb750102d00 0x7fb750198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.521+0000 7fb757800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb750103f00 0x7fb750198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.521+0000 7fb757800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb750199280 con 0x7fb750102d00 2024-03-24T00:45:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.521+0000 7fb757800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7501993c0 con 0x7fb750103f00 2024-03-24T00:45:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.521+0000 7fb754e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb750102d00 0x7fb750198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.521+0000 7fb754e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb750102d00 0x7fb750198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51222/0 (socket says 172.21.15.38:51222) 2024-03-24T00:45:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.521+0000 7fb754e00700 1 -- 172.21.15.38:0/2360813105 learned_addr learned my addr 172.21.15.38:0/2360813105 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.522+0000 7fb74fe00700 1 --2- 172.21.15.38:0/2360813105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb750103f00 0x7fb750198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.522+0000 7fb754e00700 1 -- 172.21.15.38:0/2360813105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb750103f00 msgr2=0x7fb750198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.522+0000 7fb754e00700 1 --2- 172.21.15.38:0/2360813105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb750103f00 0x7fb750198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.522+0000 7fb754e00700 1 -- 172.21.15.38:0/2360813105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7440095f0 con 0x7fb750102d00 2024-03-24T00:45:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.522+0000 7fb754e00700 1 --2- 172.21.15.38:0/2360813105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb750102d00 0x7fb750198720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb74400bd70 tx=0x7fb74400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.522+0000 7fb74d600700 1 -- 172.21.15.38:0/2360813105 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb74401d070 con 0x7fb750102d00 2024-03-24T00:45:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.522+0000 7fb74d600700 1 -- 172.21.15.38:0/2360813105 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb74400f460 con 0x7fb750102d00 2024-03-24T00:45:24.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.522+0000 7fb757800700 1 -- 172.21.15.38:0/2360813105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75019dd70 con 0x7fb750102d00 2024-03-24T00:45:24.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.522+0000 7fb74d600700 1 -- 172.21.15.38:0/2360813105 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb744021620 con 0x7fb750102d00 2024-03-24T00:45:24.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.523+0000 7fb757800700 1 -- 172.21.15.38:0/2360813105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75019e280 con 0x7fb750102d00 2024-03-24T00:45:24.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.524+0000 7fb757800700 1 -- 172.21.15.38:0/2360813105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75004eaf0 con 0x7fb750102d00 2024-03-24T00:45:24.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.525+0000 7fb74d600700 1 -- 172.21.15.38:0/2360813105 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb74400fa70 con 0x7fb750102d00 2024-03-24T00:45:24.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.526+0000 7fb74d600700 1 --2- 172.21.15.38:0/2360813105 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb73806ccf0 0x7fb73806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:24.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.526+0000 7fb74d600700 1 -- 172.21.15.38:0/2360813105 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb74408d4d0 con 0x7fb750102d00 2024-03-24T00:45:24.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.526+0000 7fb74fe00700 1 --2- 172.21.15.38:0/2360813105 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb73806ccf0 0x7fb73806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:24.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.527+0000 7fb74fe00700 1 --2- 172.21.15.38:0/2360813105 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb73806ccf0 0x7fb73806f1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb73c005fd0 tx=0x7fb73c00a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:24.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.531+0000 7fb74d600700 1 -- 172.21.15.38:0/2360813105 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb75004eaf0 con 0x7fb750102d00 2024-03-24T00:45:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:24 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:24 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:24.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.848+0000 7fb757800700 1 -- 172.21.15.38:0/2360813105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fb75004ea80 con 0x7fb750102d00 2024-03-24T00:45:24.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.849+0000 7fb74d600700 1 -- 172.21.15.38:0/2360813105 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v20) v1 ==== 78+0+97 (secure 0 0 0) 0x7fb74405b350 con 0x7fb750102d00 2024-03-24T00:45:24.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.852+0000 7fb757800700 1 -- 172.21.15.38:0/2360813105 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb73806ccf0 msgr2=0x7fb73806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:24.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.852+0000 7fb757800700 1 --2- 172.21.15.38:0/2360813105 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb73806ccf0 0x7fb73806f1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb73c005fd0 tx=0x7fb73c00a560 comp rx=0 tx=0).stop 2024-03-24T00:45:24.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.852+0000 7fb757800700 1 -- 172.21.15.38:0/2360813105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb750102d00 msgr2=0x7fb750198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:24.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.852+0000 7fb757800700 1 --2- 172.21.15.38:0/2360813105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb750102d00 0x7fb750198720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb74400bd70 tx=0x7fb74400be50 comp rx=0 tx=0).stop 2024-03-24T00:45:24.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.852+0000 7fb757800700 1 -- 172.21.15.38:0/2360813105 shutdown_connections 2024-03-24T00:45:24.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.852+0000 7fb757800700 1 --2- 172.21.15.38:0/2360813105 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fb73806ccf0 0x7fb73806f1b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:24.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.852+0000 7fb757800700 1 --2- 172.21.15.38:0/2360813105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb750102d00 0x7fb750198720 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:24.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.852+0000 7fb757800700 1 --2- 172.21.15.38:0/2360813105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb750103f00 0x7fb750198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:24.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.852+0000 7fb757800700 1 -- 172.21.15.38:0/2360813105 >> 172.21.15.38:0/2360813105 conn(0x7fb7500fe2c0 msgr2=0x7fb750107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:24.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.852+0000 7fb757800700 1 -- 172.21.15.38:0/2360813105 shutdown_connections 2024-03-24T00:45:24.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:24.852+0000 7fb757800700 1 -- 172.21.15.38:0/2360813105 wait complete. 2024-03-24T00:45:24.868 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:45:25.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:25 smithi038 ceph-mon[29271]: pgmap v243: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T00:45:25.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:25 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2360813105' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-24T00:45:25.577 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-24T00:45:25.590 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T00:45:25.634 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:25 smithi088 ceph-mon[36610]: pgmap v243: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T00:45:25.634 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:25 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2360813105' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-24T00:45:25.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.832+0000 7f2c6b200640 1 -- 172.21.15.38:0/3949974027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c0ffe10 msgr2=0x7f2c6c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:25.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.832+0000 7f2c6b200640 1 --2- 172.21.15.38:0/3949974027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c0ffe10 0x7f2c6c100230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009880 tx=0x7f2c5c02bce0 comp rx=0 tx=0).stop 2024-03-24T00:45:25.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.832+0000 7f2c6b200640 1 -- 172.21.15.38:0/3949974027 shutdown_connections 2024-03-24T00:45:25.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.832+0000 7f2c6b200640 1 --2- 172.21.15.38:0/3949974027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c6c100f60 0x7f2c6c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:25.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.832+0000 7f2c6b200640 1 --2- 172.21.15.38:0/3949974027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c0ffe10 0x7f2c6c100230 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:25.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.832+0000 7f2c6b200640 1 -- 172.21.15.38:0/3949974027 >> 172.21.15.38:0/3949974027 conn(0x7f2c6c0fb380 msgr2=0x7f2c6c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:25.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.832+0000 7f2c6b200640 1 -- 172.21.15.38:0/3949974027 shutdown_connections 2024-03-24T00:45:25.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.832+0000 7f2c6b200640 1 -- 172.21.15.38:0/3949974027 wait complete. 2024-03-24T00:45:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.833+0000 7f2c6b200640 1 Processor -- start 2024-03-24T00:45:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.834+0000 7f2c6b200640 1 -- start start 2024-03-24T00:45:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.834+0000 7f2c6b200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c0ffe10 0x7f2c6c195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.834+0000 7f2c6b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c6c100f60 0x7f2c6c195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.834+0000 7f2c6b200640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c196410 con 0x7f2c6c0ffe10 2024-03-24T00:45:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.834+0000 7f2c6b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c196580 con 0x7f2c6c100f60 2024-03-24T00:45:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.834+0000 7f2c69e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c0ffe10 0x7f2c6c195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.834+0000 7f2c69e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c0ffe10 0x7f2c6c195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51242/0 (socket says 172.21.15.38:51242) 2024-03-24T00:45:25.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.834+0000 7f2c69e00640 1 -- 172.21.15.38:0/94726846 learned_addr learned my addr 172.21.15.38:0/94726846 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:25.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.834+0000 7f2c63e00640 1 --2- 172.21.15.38:0/94726846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c6c100f60 0x7f2c6c195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:25.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.835+0000 7f2c69e00640 1 -- 172.21.15.38:0/94726846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c6c100f60 msgr2=0x7f2c6c195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:25.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.835+0000 7f2c69e00640 1 --2- 172.21.15.38:0/94726846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c6c100f60 0x7f2c6c195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:25.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.835+0000 7f2c69e00640 1 -- 172.21.15.38:0/94726846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c009560 con 0x7f2c6c0ffe10 2024-03-24T00:45:25.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.835+0000 7f2c69e00640 1 --2- 172.21.15.38:0/94726846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c0ffe10 0x7f2c6c195900 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c02efd0 tx=0x7f2c5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:25.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.835+0000 7f2c62a00640 1 -- 172.21.15.38:0/94726846 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c5c0037c0 con 0x7f2c6c0ffe10 2024-03-24T00:45:25.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.835+0000 7f2c62a00640 1 -- 172.21.15.38:0/94726846 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c5c003de0 con 0x7f2c6c0ffe10 2024-03-24T00:45:25.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.835+0000 7f2c6b200640 1 -- 172.21.15.38:0/94726846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c19af10 con 0x7f2c6c0ffe10 2024-03-24T00:45:25.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.835+0000 7f2c62a00640 1 -- 172.21.15.38:0/94726846 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c5c034440 con 0x7f2c6c0ffe10 2024-03-24T00:45:25.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.836+0000 7f2c6b200640 1 -- 172.21.15.38:0/94726846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c19b410 con 0x7f2c6c0ffe10 2024-03-24T00:45:25.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.837+0000 7f2c6b200640 1 -- 172.21.15.38:0/94726846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c18f910 con 0x7f2c6c0ffe10 2024-03-24T00:45:25.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.838+0000 7f2c62a00640 1 -- 172.21.15.38:0/94726846 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2c5c004ff0 con 0x7f2c6c0ffe10 2024-03-24T00:45:25.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.839+0000 7f2c62a00640 1 --2- 172.21.15.38:0/94726846 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c5006cc70 0x7f2c5006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:25.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.839+0000 7f2c62a00640 1 -- 172.21.15.38:0/94726846 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2c5c0ae800 con 0x7f2c6c0ffe10 2024-03-24T00:45:25.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.839+0000 7f2c63e00640 1 --2- 172.21.15.38:0/94726846 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c5006cc70 0x7f2c5006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:25.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.840+0000 7f2c63e00640 1 --2- 172.21.15.38:0/94726846 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c5006cc70 0x7f2c5006f130 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2c54005fd0 tx=0x7f2c54005850 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:25.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:25.844+0000 7f2c62a00640 1 -- 172.21.15.38:0/94726846 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2c5c078ad0 con 0x7f2c6c0ffe10 2024-03-24T00:45:26.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.050+0000 7f2c6b200640 1 -- 172.21.15.38:0/94726846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c6c02c3f0 con 0x7f2c6c0ffe10 2024-03-24T00:45:26.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.053+0000 7f2c62a00640 1 -- 172.21.15.38:0/94726846 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15079 (secure 0 0 0) 0x7f2c5c07c0f0 con 0x7f2c6c0ffe10 2024-03-24T00:45:26.054 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:45:26.054 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":47,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","created":"2024-03-24T00:35:36.968970+0000","modified":"2024-03-24T00:45:13.640904+0000","last_up_change":"2024-03-24T00:43:15.784148+0000","last_in_change":"2024-03-24T00:42:50.726360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T00:41:31.289149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-24T00:44:50.057892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-24T00:44:50.717526+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e5df40af-b5d0-4f89-94b2-59ee2e710bc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6803","nonce":667786666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6805","nonce":667786666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6809","nonce":667786666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":667786666},{"type":"v1","addr":"172.21.15.38:6807","nonce":667786666}]},"public_addr":"172.21.15.38:6803/667786666","cluster_addr":"172.21.15.38:6805/667786666","heartbeat_back_addr":"172.21.15.38:6809/667786666","heartbeat_front_addr":"172.21.15.38:6807/667786666","state":["exists","up"]},{"osd":1,"uuid":"ac3c0960-dd6c-44f9-b4b0-234c10f838fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6811","nonce":723151394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6813","nonce":723151394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6817","nonce":723151394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":723151394},{"type":"v1","addr":"172.21.15.38:6815","nonce":723151394}]},"public_addr":"172.21.15.38:6811/723151394","cluster_addr":"172.21.15.38:6813/723151394","heartbeat_back_addr":"172.21.15.38:6817/723151394","heartbeat_front_addr":"172.21.15.38:6815/723151394","state":["exists","up"]},{"osd":2,"uuid":"921d9dd3-2471-4b13-b6bd-01e5b6c78648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6819","nonce":643463158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6821","nonce":643463158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6825","nonce":643463158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":643463158},{"type":"v1","addr":"172.21.15.38:6823","nonce":643463158}]},"public_addr":"172.21.15.38:6819/643463158","cluster_addr":"172.21.15.38:6821/643463158","heartbeat_back_addr":"172.21.15.38:6825/643463158","heartbeat_front_addr":"172.21.15.38:6823/643463158","state":["exists","up"]},{"osd":3,"uuid":"7e93813a-0036-4cea-8396-41b8b759c257","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6801","nonce":3248144048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6803","nonce":3248144048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6807","nonce":3248144048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3248144048},{"type":"v1","addr":"172.21.15.88:6805","nonce":3248144048}]},"public_addr":"172.21.15.88:6801/3248144048","cluster_addr":"172.21.15.88:6803/3248144048","heartbeat_back_addr":"172.21.15.88:6807/3248144048","heartbeat_front_addr":"172.21.15.88:6805/3248144048","state":["exists","up"]},{"osd":4,"uuid":"b73f29f7-134e-4725-b341-e893b4bbff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6809","nonce":1859611126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6811","nonce":1859611126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6815","nonce":1859611126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1859611126},{"type":"v1","addr":"172.21.15.88:6813","nonce":1859611126}]},"public_addr":"172.21.15.88:6809/1859611126","cluster_addr":"172.21.15.88:6811/1859611126","heartbeat_back_addr":"172.21.15.88:6815/1859611126","heartbeat_front_addr":"172.21.15.88:6813/1859611126","state":["exists","up"]},{"osd":5,"uuid":"8ea2cfb3-be7a-4f97-9993-8f9001b8b678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6817","nonce":2286313582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6819","nonce":2286313582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6823","nonce":2286313582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2286313582},{"type":"v1","addr":"172.21.15.88:6821","nonce":2286313582}]},"public_addr":"172.21.15.88:6817/2286313582","cluster_addr":"172.21.15.88:6819/2286313582","heartbeat_back_addr":"172.21.15.88:6823/2286313582","heartbeat_front_addr":"172.21.15.88:6821/2286313582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:11.140794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:49.877898+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:41:28.612279+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:02.338217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:36.760939+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:43:13.811948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6825/1531706761":"2024-03-25T00:45:13.640830+0000","172.21.15.38:0/3845097007":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/884740303":"2024-03-25T00:38:28.565032+0000","172.21.15.88:6824/1531706761":"2024-03-25T00:45:13.640830+0000","172.21.15.38:6801/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6800/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6801/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/913707898":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/3860887425":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/1949387173":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/748221873":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6800/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.38:6800/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6801/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/3448795319":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/2868668356":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/4049972561":"2024-03-25T00:38:28.565032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T00:45:26.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.056+0000 7f2c6b200640 1 -- 172.21.15.38:0/94726846 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c5006cc70 msgr2=0x7f2c5006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:26.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.056+0000 7f2c6b200640 1 --2- 172.21.15.38:0/94726846 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c5006cc70 0x7f2c5006f130 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2c54005fd0 tx=0x7f2c54005850 comp rx=0 tx=0).stop 2024-03-24T00:45:26.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.056+0000 7f2c6b200640 1 -- 172.21.15.38:0/94726846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c0ffe10 msgr2=0x7f2c6c195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:26.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.056+0000 7f2c6b200640 1 --2- 172.21.15.38:0/94726846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c0ffe10 0x7f2c6c195900 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c02efd0 tx=0x7f2c5c004270 comp rx=0 tx=0).stop 2024-03-24T00:45:26.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.056+0000 7f2c6b200640 1 -- 172.21.15.38:0/94726846 shutdown_connections 2024-03-24T00:45:26.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.056+0000 7f2c6b200640 1 --2- 172.21.15.38:0/94726846 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f2c5006cc70 0x7f2c5006f130 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.056+0000 7f2c6b200640 1 --2- 172.21.15.38:0/94726846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c6c100f60 0x7f2c6c195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.056+0000 7f2c6b200640 1 --2- 172.21.15.38:0/94726846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c6c0ffe10 0x7f2c6c195900 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.056+0000 7f2c6b200640 1 -- 172.21.15.38:0/94726846 >> 172.21.15.38:0/94726846 conn(0x7f2c6c0fb380 msgr2=0x7f2c6c1045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:26.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.056+0000 7f2c6b200640 1 -- 172.21.15.38:0/94726846 shutdown_connections 2024-03-24T00:45:26.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.056+0000 7f2c6b200640 1 -- 172.21.15.38:0/94726846 wait complete. 2024-03-24T00:45:26.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T00:41:31.289149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-24T00:44:50.057892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-24T00:44:50.717526+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T00:45:26.077 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T00:45:26.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.309+0000 7f5009400640 1 -- 172.21.15.38:0/512760478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50041033b0 msgr2=0x7f5004103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:26.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.309+0000 7f5009400640 1 --2- 172.21.15.38:0/512760478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50041033b0 0x7f5004103830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009880 tx=0x7f4fec02bce0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.309+0000 7f5009400640 1 -- 172.21.15.38:0/512760478 shutdown_connections 2024-03-24T00:45:26.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.309+0000 7f5009400640 1 --2- 172.21.15.38:0/512760478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50041033b0 0x7f5004103830 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.309+0000 7f5009400640 1 --2- 172.21.15.38:0/512760478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50041021c0 0x7f50041025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.309+0000 7f5009400640 1 -- 172.21.15.38:0/512760478 >> 172.21.15.38:0/512760478 conn(0x7f50040fd750 msgr2=0x7f50040ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:26.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.310+0000 7f5009400640 1 -- 172.21.15.38:0/512760478 shutdown_connections 2024-03-24T00:45:26.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.310+0000 7f5009400640 1 -- 172.21.15.38:0/512760478 wait complete. 2024-03-24T00:45:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.311+0000 7f5009400640 1 Processor -- start 2024-03-24T00:45:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.311+0000 7f5009400640 1 -- start start 2024-03-24T00:45:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.311+0000 7f5009400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50041021c0 0x7f5004197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.311+0000 7f5009400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50041033b0 0x7f5004198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.311+0000 7f5009400640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5004198840 con 0x7f50041021c0 2024-03-24T00:45:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.311+0000 7f5009400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50041989b0 con 0x7f50041033b0 2024-03-24T00:45:26.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.311+0000 7f5002800640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50041021c0 0x7f5004197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:26.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.311+0000 7f5002800640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50041021c0 0x7f5004197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51254/0 (socket says 172.21.15.38:51254) 2024-03-24T00:45:26.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.311+0000 7f5002800640 1 -- 172.21.15.38:0/3552726484 learned_addr learned my addr 172.21.15.38:0/3552726484 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.312+0000 7f5002800640 1 -- 172.21.15.38:0/3552726484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50041033b0 msgr2=0x7f5004198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.312+0000 7f4ffbe00640 1 --2- 172.21.15.38:0/3552726484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50041033b0 0x7f5004198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.312+0000 7f5002800640 1 --2- 172.21.15.38:0/3552726484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50041033b0 0x7f5004198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.312+0000 7f5002800640 1 -- 172.21.15.38:0/3552726484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec009560 con 0x7f50041021c0 2024-03-24T00:45:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.312+0000 7f5002800640 1 --2- 172.21.15.38:0/3552726484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50041021c0 0x7f5004197d00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400d8d0 tx=0x7f4ff400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.312+0000 7f4ffb400640 1 -- 172.21.15.38:0/3552726484 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ff4009a30 con 0x7f50041021c0 2024-03-24T00:45:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.313+0000 7f5009400640 1 -- 172.21.15.38:0/3552726484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500419d3e0 con 0x7f50041021c0 2024-03-24T00:45:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.313+0000 7f4ffb400640 1 -- 172.21.15.38:0/3552726484 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ff4009b90 con 0x7f50041021c0 2024-03-24T00:45:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.313+0000 7f5009400640 1 -- 172.21.15.38:0/3552726484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500419d9c0 con 0x7f50041021c0 2024-03-24T00:45:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.313+0000 7f4ffb400640 1 -- 172.21.15.38:0/3552726484 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ff400f670 con 0x7f50041021c0 2024-03-24T00:45:26.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.314+0000 7f5009400640 1 -- 172.21.15.38:0/3552726484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5004191d50 con 0x7f50041021c0 2024-03-24T00:45:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.315+0000 7f4ffb400640 1 -- 172.21.15.38:0/3552726484 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4ff4009d00 con 0x7f50041021c0 2024-03-24T00:45:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.316+0000 7f4ffb400640 1 --2- 172.21.15.38:0/3552726484 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4fe806cc50 0x7f4fe806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.316+0000 7f4ffb400640 1 -- 172.21.15.38:0/3552726484 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4ff408c470 con 0x7f50041021c0 2024-03-24T00:45:26.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.317+0000 7f4ffbe00640 1 --2- 172.21.15.38:0/3552726484 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4fe806cc50 0x7f4fe806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:26.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.317+0000 7f4ffbe00640 1 --2- 172.21.15.38:0/3552726484 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4fe806cc50 0x7f4fe806f110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4fec02c7c0 tx=0x7f4fec005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:26.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.321+0000 7f4ffb400640 1 -- 172.21.15.38:0/3552726484 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4ff40567e0 con 0x7f50041021c0 2024-03-24T00:45:26.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:26 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/94726846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T00:45:26.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.545+0000 7f5009400640 1 -- 172.21.15.38:0/3552726484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f500410b040 con 0x7f50041021c0 2024-03-24T00:45:26.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.546+0000 7f4ffb400640 1 -- 172.21.15.38:0/3552726484 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ff4059e00 con 0x7f50041021c0 2024-03-24T00:45:26.547 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-03-24T00:45:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.549+0000 7f5009400640 1 -- 172.21.15.38:0/3552726484 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4fe806cc50 msgr2=0x7f4fe806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.549+0000 7f5009400640 1 --2- 172.21.15.38:0/3552726484 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4fe806cc50 0x7f4fe806f110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4fec02c7c0 tx=0x7f4fec005fd0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.549+0000 7f5009400640 1 -- 172.21.15.38:0/3552726484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50041021c0 msgr2=0x7f5004197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.549+0000 7f5009400640 1 --2- 172.21.15.38:0/3552726484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50041021c0 0x7f5004197d00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400d8d0 tx=0x7f4ff400dd90 comp rx=0 tx=0).stop 2024-03-24T00:45:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.549+0000 7f5009400640 1 -- 172.21.15.38:0/3552726484 shutdown_connections 2024-03-24T00:45:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.549+0000 7f5009400640 1 --2- 172.21.15.38:0/3552726484 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f4fe806cc50 0x7f4fe806f110 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.549+0000 7f5009400640 1 --2- 172.21.15.38:0/3552726484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50041033b0 0x7f5004198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.549+0000 7f5009400640 1 --2- 172.21.15.38:0/3552726484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50041021c0 0x7f5004197d00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.549+0000 7f5009400640 1 -- 172.21.15.38:0/3552726484 >> 172.21.15.38:0/3552726484 conn(0x7f50040fd750 msgr2=0x7f5004106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:26.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.549+0000 7f5009400640 1 -- 172.21.15.38:0/3552726484 shutdown_connections 2024-03-24T00:45:26.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.549+0000 7f5009400640 1 -- 172.21.15.38:0/3552726484 wait complete. 2024-03-24T00:45:26.572 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-24T00:45:26.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:26 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/94726846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T00:45:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.775+0000 7f8667e00640 1 -- 172.21.15.38:0/3054484073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86601021a0 msgr2=0x7f86601025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.775+0000 7f8667e00640 1 --2- 172.21.15.38:0/3054484073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86601021a0 0x7f86601025c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8654009880 tx=0x7f865402bcf0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.775+0000 7f8667e00640 1 -- 172.21.15.38:0/3054484073 shutdown_connections 2024-03-24T00:45:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.775+0000 7f8667e00640 1 --2- 172.21.15.38:0/3054484073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8660103390 0x7f8660103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.775+0000 7f8667e00640 1 --2- 172.21.15.38:0/3054484073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86601021a0 0x7f86601025c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.775+0000 7f8667e00640 1 -- 172.21.15.38:0/3054484073 >> 172.21.15.38:0/3054484073 conn(0x7f86600fd6d0 msgr2=0x7f86600ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.775+0000 7f8667e00640 1 -- 172.21.15.38:0/3054484073 shutdown_connections 2024-03-24T00:45:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.775+0000 7f8667e00640 1 -- 172.21.15.38:0/3054484073 wait complete. 2024-03-24T00:45:26.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.776+0000 7f8667e00640 1 Processor -- start 2024-03-24T00:45:26.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.776+0000 7f8667e00640 1 -- start start 2024-03-24T00:45:26.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.776+0000 7f8667e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86601021a0 0x7f8660197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:26.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.776+0000 7f8667e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8660103390 0x7f8660198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:26.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.776+0000 7f8667e00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8660198840 con 0x7f86601021a0 2024-03-24T00:45:26.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.776+0000 7f8667e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86601989b0 con 0x7f8660103390 2024-03-24T00:45:26.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.777+0000 7f8665400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86601021a0 0x7f8660197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:26.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.777+0000 7f8665400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86601021a0 0x7f8660197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51266/0 (socket says 172.21.15.38:51266) 2024-03-24T00:45:26.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.777+0000 7f8665400640 1 -- 172.21.15.38:0/1733242322 learned_addr learned my addr 172.21.15.38:0/1733242322 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.777+0000 7f8664a00640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8660103390 0x7f8660198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.777+0000 7f8665400640 1 -- 172.21.15.38:0/1733242322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8660103390 msgr2=0x7f8660198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.777+0000 7f8665400640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8660103390 0x7f8660198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.777+0000 7f8665400640 1 -- 172.21.15.38:0/1733242322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654009560 con 0x7f86601021a0 2024-03-24T00:45:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.777+0000 7f8664a00640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8660103390 0x7f8660198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:45:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.777+0000 7f8665400640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86601021a0 0x7f8660197d00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8654009230 tx=0x7f8654004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.778+0000 7f865e000640 1 -- 172.21.15.38:0/1733242322 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86540037c0 con 0x7f86601021a0 2024-03-24T00:45:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.778+0000 7f865e000640 1 -- 172.21.15.38:0/1733242322 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8654004aa0 con 0x7f86601021a0 2024-03-24T00:45:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.778+0000 7f865e000640 1 -- 172.21.15.38:0/1733242322 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8654034440 con 0x7f86601021a0 2024-03-24T00:45:26.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.778+0000 7f8667e00640 1 -- 172.21.15.38:0/1733242322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866019d380 con 0x7f86601021a0 2024-03-24T00:45:26.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.778+0000 7f8667e00640 1 -- 172.21.15.38:0/1733242322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866019d960 con 0x7f86601021a0 2024-03-24T00:45:26.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.779+0000 7f8667e00640 1 -- 172.21.15.38:0/1733242322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866010ada0 con 0x7f86601021a0 2024-03-24T00:45:26.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.780+0000 7f865e000640 1 -- 172.21.15.38:0/1733242322 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8654005060 con 0x7f86601021a0 2024-03-24T00:45:26.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.781+0000 7f865e000640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f864806cc50 0x7f864806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:26.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.781+0000 7f865e000640 1 -- 172.21.15.38:0/1733242322 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f86540ae650 con 0x7f86601021a0 2024-03-24T00:45:26.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.781+0000 7f8664a00640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f864806cc50 0x7f864806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:26.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.782+0000 7f8664a00640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f864806cc50 0x7f864806f110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86601991b0 tx=0x7f864c005e50 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:26.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.785+0000 7f865e000640 1 -- 172.21.15.38:0/1733242322 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f86540789c0 con 0x7f86601021a0 2024-03-24T00:45:26.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.995+0000 7f8667e00640 1 -- 172.21.15.38:0/1733242322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f866010af30 con 0x7f86601021a0 2024-03-24T00:45:26.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.996+0000 7f865e000640 1 -- 172.21.15.38:0/1733242322 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f865407bfe0 con 0x7f86601021a0 2024-03-24T00:45:26.997 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-03-24T00:45:27.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.998+0000 7f8667e00640 1 -- 172.21.15.38:0/1733242322 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f864806cc50 msgr2=0x7f864806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:27.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.998+0000 7f8667e00640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f864806cc50 0x7f864806f110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86601991b0 tx=0x7f864c005e50 comp rx=0 tx=0).stop 2024-03-24T00:45:27.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.998+0000 7f8667e00640 1 -- 172.21.15.38:0/1733242322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86601021a0 msgr2=0x7f8660197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:27.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.998+0000 7f8667e00640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86601021a0 0x7f8660197d00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8654009230 tx=0x7f8654004270 comp rx=0 tx=0).stop 2024-03-24T00:45:27.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.999+0000 7f8667e00640 1 -- 172.21.15.38:0/1733242322 shutdown_connections 2024-03-24T00:45:27.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.999+0000 7f8667e00640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f864806cc50 0x7f864806f110 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.999+0000 7f8667e00640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8660103390 0x7f8660198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.999+0000 7f8667e00640 1 --2- 172.21.15.38:0/1733242322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86601021a0 0x7f8660197d00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.999+0000 7f8667e00640 1 -- 172.21.15.38:0/1733242322 >> 172.21.15.38:0/1733242322 conn(0x7f86600fd6d0 msgr2=0x7f86601065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:27.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.999+0000 7f8667e00640 1 -- 172.21.15.38:0/1733242322 shutdown_connections 2024-03-24T00:45:27.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:26.999+0000 7f8667e00640 1 -- 172.21.15.38:0/1733242322 wait complete. 2024-03-24T00:45:27.019 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-24T00:45:27.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.238+0000 7fe6b4c00640 1 -- 172.21.15.38:0/3815078690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b01011a0 msgr2=0x7fe6b0101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.238+0000 7fe6b4c00640 1 --2- 172.21.15.38:0/3815078690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b01011a0 0x7fe6b0101620 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe698009880 tx=0x7fe69802bce0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.238+0000 7fe6b4c00640 1 -- 172.21.15.38:0/3815078690 shutdown_connections 2024-03-24T00:45:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.238+0000 7fe6b4c00640 1 --2- 172.21.15.38:0/3815078690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b01011a0 0x7fe6b0101620 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.238+0000 7fe6b4c00640 1 --2- 172.21.15.38:0/3815078690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b00fffb0 0x7fe6b01003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.238+0000 7fe6b4c00640 1 -- 172.21.15.38:0/3815078690 >> 172.21.15.38:0/3815078690 conn(0x7fe6b00fb560 msgr2=0x7fe6b00fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.239+0000 7fe6b4c00640 1 -- 172.21.15.38:0/3815078690 shutdown_connections 2024-03-24T00:45:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.239+0000 7fe6b4c00640 1 -- 172.21.15.38:0/3815078690 wait complete. 2024-03-24T00:45:27.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.240+0000 7fe6b4c00640 1 Processor -- start 2024-03-24T00:45:27.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.240+0000 7fe6b4c00640 1 -- start start 2024-03-24T00:45:27.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.240+0000 7fe6b4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b00fffb0 0x7fe6b0074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:27.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.240+0000 7fe6b4c00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b01011a0 0x7fe6b00726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:27.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.240+0000 7fe6b4c00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0072bf0 con 0x7fe6b01011a0 2024-03-24T00:45:27.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.240+0000 7fe6b4c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0072d60 con 0x7fe6b00fffb0 2024-03-24T00:45:27.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.241+0000 7fe6a7e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b01011a0 0x7fe6b00726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:27.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.241+0000 7fe6a7e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b01011a0 0x7fe6b00726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51284/0 (socket says 172.21.15.38:51284) 2024-03-24T00:45:27.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.241+0000 7fe6a7e00640 1 -- 172.21.15.38:0/2555138281 learned_addr learned my addr 172.21.15.38:0/2555138281 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:27.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.241+0000 7fe6ade00640 1 --2- 172.21.15.38:0/2555138281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b00fffb0 0x7fe6b0074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.241+0000 7fe6a7e00640 1 -- 172.21.15.38:0/2555138281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b00fffb0 msgr2=0x7fe6b0074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.241+0000 7fe6a7e00640 1 --2- 172.21.15.38:0/2555138281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b00fffb0 0x7fe6b0074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.241+0000 7fe6a7e00640 1 -- 172.21.15.38:0/2555138281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe698009560 con 0x7fe6b01011a0 2024-03-24T00:45:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.242+0000 7fe6a7e00640 1 --2- 172.21.15.38:0/2555138281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b01011a0 0x7fe6b00726b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe69802efd0 tx=0x7fe698004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.242+0000 7fe6a6a00640 1 -- 172.21.15.38:0/2555138281 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6980037c0 con 0x7fe6b01011a0 2024-03-24T00:45:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.242+0000 7fe6a6a00640 1 -- 172.21.15.38:0/2555138281 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe698003de0 con 0x7fe6b01011a0 2024-03-24T00:45:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.242+0000 7fe6b4c00640 1 -- 172.21.15.38:0/2555138281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b0072fe0 con 0x7fe6b01011a0 2024-03-24T00:45:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.242+0000 7fe6a6a00640 1 -- 172.21.15.38:0/2555138281 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe698034670 con 0x7fe6b01011a0 2024-03-24T00:45:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.242+0000 7fe6b4c00640 1 -- 172.21.15.38:0/2555138281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b019dc40 con 0x7fe6b01011a0 2024-03-24T00:45:27.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.244+0000 7fe6b4c00640 1 -- 172.21.15.38:0/2555138281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b004e030 con 0x7fe6b01011a0 2024-03-24T00:45:27.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.245+0000 7fe6a6a00640 1 -- 172.21.15.38:0/2555138281 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe698003920 con 0x7fe6b01011a0 2024-03-24T00:45:27.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.245+0000 7fe6a6a00640 1 --2- 172.21.15.38:0/2555138281 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe69406cc50 0x7fe69406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.246+0000 7fe6a6a00640 1 -- 172.21.15.38:0/2555138281 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe6980ae790 con 0x7fe6b01011a0 2024-03-24T00:45:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.246+0000 7fe6ade00640 1 --2- 172.21.15.38:0/2555138281 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe69406cc50 0x7fe69406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:27.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.246+0000 7fe6ade00640 1 --2- 172.21.15.38:0/2555138281 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe69406cc50 0x7fe69406f110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0101000 tx=0x7fe6a0008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:27.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.250+0000 7fe6a6a00640 1 -- 172.21.15.38:0/2555138281 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe698078b00 con 0x7fe6b01011a0 2024-03-24T00:45:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.480+0000 7fe6b4c00640 1 -- 172.21.15.38:0/2555138281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fe6b0108c30 con 0x7fe6b01011a0 2024-03-24T00:45:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.481+0000 7fe6a6a00640 1 -- 172.21.15.38:0/2555138281 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe6980ae020 con 0x7fe6b01011a0 2024-03-24T00:45:27.483 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-03-24T00:45:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.485+0000 7fe6b4c00640 1 -- 172.21.15.38:0/2555138281 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe69406cc50 msgr2=0x7fe69406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.485+0000 7fe6b4c00640 1 --2- 172.21.15.38:0/2555138281 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe69406cc50 0x7fe69406f110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0101000 tx=0x7fe6a0008040 comp rx=0 tx=0).stop 2024-03-24T00:45:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.485+0000 7fe6b4c00640 1 -- 172.21.15.38:0/2555138281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b01011a0 msgr2=0x7fe6b00726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.485+0000 7fe6b4c00640 1 --2- 172.21.15.38:0/2555138281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b01011a0 0x7fe6b00726b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe69802efd0 tx=0x7fe698004270 comp rx=0 tx=0).stop 2024-03-24T00:45:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.485+0000 7fe6b4c00640 1 -- 172.21.15.38:0/2555138281 shutdown_connections 2024-03-24T00:45:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.485+0000 7fe6b4c00640 1 --2- 172.21.15.38:0/2555138281 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe69406cc50 0x7fe69406f110 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.485+0000 7fe6b4c00640 1 --2- 172.21.15.38:0/2555138281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b01011a0 0x7fe6b00726b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.485+0000 7fe6b4c00640 1 --2- 172.21.15.38:0/2555138281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b00fffb0 0x7fe6b0074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.485+0000 7fe6b4c00640 1 -- 172.21.15.38:0/2555138281 >> 172.21.15.38:0/2555138281 conn(0x7fe6b00fb560 msgr2=0x7fe6b0104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.485+0000 7fe6b4c00640 1 -- 172.21.15.38:0/2555138281 shutdown_connections 2024-03-24T00:45:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.485+0000 7fe6b4c00640 1 -- 172.21.15.38:0/2555138281 wait complete. 2024-03-24T00:45:27.506 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T00:45:27.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:27 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3552726484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T00:45:27.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:27 smithi038 ceph-mon[29271]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-24T00:45:27.621 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:27 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1733242322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T00:45:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:27 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3552726484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T00:45:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:27 smithi088 ceph-mon[36610]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-24T00:45:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:27 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1733242322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T00:45:27.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.730+0000 7f1259c00640 1 -- 172.21.15.38:0/2548931929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1254074300 msgr2=0x7f1254072750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:27.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.730+0000 7f1259c00640 1 --2- 172.21.15.38:0/2548931929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1254074300 0x7f1254072750 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1244009880 tx=0x7f124402bce0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.730+0000 7f1259c00640 1 -- 172.21.15.38:0/2548931929 shutdown_connections 2024-03-24T00:45:27.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.730+0000 7f1259c00640 1 --2- 172.21.15.38:0/2548931929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1254072d20 0x7f12540731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.730+0000 7f1259c00640 1 --2- 172.21.15.38:0/2548931929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1254074300 0x7f1254072750 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.730+0000 7f1259c00640 1 -- 172.21.15.38:0/2548931929 >> 172.21.15.38:0/2548931929 conn(0x7f12540fb380 msgr2=0x7f12540fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:27.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.730+0000 7f1259c00640 1 -- 172.21.15.38:0/2548931929 shutdown_connections 2024-03-24T00:45:27.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.730+0000 7f1259c00640 1 -- 172.21.15.38:0/2548931929 wait complete. 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.731+0000 7f1259c00640 1 Processor -- start 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.731+0000 7f1259c00640 1 -- start start 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.731+0000 7f1259c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1254072d20 0x7f1254197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.731+0000 7f1259c00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1254074300 0x7f1254198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.732+0000 7f1259c00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12541986d0 con 0x7f1254074300 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.732+0000 7f1259c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1254198840 con 0x7f1254072d20 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.732+0000 7f1253400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1254074300 0x7f1254198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.732+0000 7f1253400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1254074300 0x7f1254198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51292/0 (socket says 172.21.15.38:51292) 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.732+0000 7f1253400640 1 -- 172.21.15.38:0/654041590 learned_addr learned my addr 172.21.15.38:0/654041590 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.732+0000 7f1253e00640 1 --2- 172.21.15.38:0/654041590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1254072d20 0x7f1254197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.732+0000 7f1253400640 1 -- 172.21.15.38:0/654041590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1254072d20 msgr2=0x7f1254197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:27.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.732+0000 7f1253400640 1 --2- 172.21.15.38:0/654041590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1254072d20 0x7f1254197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.732+0000 7f1253400640 1 -- 172.21.15.38:0/654041590 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1244009560 con 0x7f1254074300 2024-03-24T00:45:27.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.732+0000 7f1253400640 1 --2- 172.21.15.38:0/654041590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1254074300 0x7f1254198100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f124000b420 tx=0x7f124000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:27.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.733+0000 7f1250c00640 1 -- 172.21.15.38:0/654041590 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1240011800 con 0x7f1254074300 2024-03-24T00:45:27.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.733+0000 7f1250c00640 1 -- 172.21.15.38:0/654041590 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1240011e20 con 0x7f1254074300 2024-03-24T00:45:27.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.733+0000 7f1250c00640 1 -- 172.21.15.38:0/654041590 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f124000f2e0 con 0x7f1254074300 2024-03-24T00:45:27.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.733+0000 7f1259c00640 1 -- 172.21.15.38:0/654041590 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125419d280 con 0x7f1254074300 2024-03-24T00:45:27.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.733+0000 7f1259c00640 1 -- 172.21.15.38:0/654041590 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125419d860 con 0x7f1254074300 2024-03-24T00:45:27.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.734+0000 7f1259c00640 1 -- 172.21.15.38:0/654041590 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125410ac50 con 0x7f1254074300 2024-03-24T00:45:27.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.735+0000 7f1250c00640 1 -- 172.21.15.38:0/654041590 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1240010380 con 0x7f1254074300 2024-03-24T00:45:27.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.737+0000 7f1250c00640 1 --2- 172.21.15.38:0/654041590 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123806cc70 0x7f123806f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.737+0000 7f1250c00640 1 -- 172.21.15.38:0/654041590 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f124008bc10 con 0x7f1254074300 2024-03-24T00:45:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.737+0000 7f1253e00640 1 --2- 172.21.15.38:0/654041590 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123806cc70 0x7f123806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:27.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.737+0000 7f1253e00640 1 --2- 172.21.15.38:0/654041590 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123806cc70 0x7f123806f130 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f124402c960 tx=0x7f124402cf90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:27.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.741+0000 7f1250c00640 1 -- 172.21.15.38:0/654041590 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1240055f60 con 0x7f1254074300 2024-03-24T00:45:27.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.993+0000 7f1259c00640 1 -- 172.21.15.38:0/654041590 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f125410ade0 con 0x7f1254074300 2024-03-24T00:45:27.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.995+0000 7f1250c00640 1 -- 172.21.15.38:0/654041590 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4856 (secure 0 0 0) 0x7f1240059580 con 0x7f1254074300 2024-03-24T00:45:27.996 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:45:27.996 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:45:17.720123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14462,"mds_1":24271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14462":{"gid":14462,"name":"cephfs.smithi038.yhkmsb","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6827/2539263555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2539263555},{"type":"v1","addr":"172.21.15.38:6827","nonce":2539263555}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14478":{"gid":14478,"name":"cephfs.smithi038.xnpqhp","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":3,"addr":"172.21.15.38:6829/3798421518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3798421518},{"type":"v1","addr":"172.21.15.38:6829","nonce":3798421518}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24271":{"gid":24271,"name":"cephfs.smithi088.yhksbr","rank":1,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6827/1446532250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1446532250},{"type":"v1","addr":"172.21.15.88:6827","nonce":1446532250}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24279":{"gid":24279,"name":"cephfs.smithi088.cbfvvp","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6825/2705783970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2705783970},{"type":"v1","addr":"172.21.15.88:6825","nonce":2705783970}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-24T00:45:27.996 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:45:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.998+0000 7f1259c00640 1 -- 172.21.15.38:0/654041590 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123806cc70 msgr2=0x7f123806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.998+0000 7f1259c00640 1 --2- 172.21.15.38:0/654041590 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123806cc70 0x7f123806f130 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f124402c960 tx=0x7f124402cf90 comp rx=0 tx=0).stop 2024-03-24T00:45:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.998+0000 7f1259c00640 1 -- 172.21.15.38:0/654041590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1254074300 msgr2=0x7f1254198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.998+0000 7f1259c00640 1 --2- 172.21.15.38:0/654041590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1254074300 0x7f1254198100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f124000b420 tx=0x7f124000b8e0 comp rx=0 tx=0).stop 2024-03-24T00:45:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.998+0000 7f1259c00640 1 -- 172.21.15.38:0/654041590 shutdown_connections 2024-03-24T00:45:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.998+0000 7f1259c00640 1 --2- 172.21.15.38:0/654041590 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123806cc70 0x7f123806f130 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.998+0000 7f1259c00640 1 --2- 172.21.15.38:0/654041590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1254074300 0x7f1254198100 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.998+0000 7f1259c00640 1 --2- 172.21.15.38:0/654041590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1254072d20 0x7f1254197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.998+0000 7f1259c00640 1 -- 172.21.15.38:0/654041590 >> 172.21.15.38:0/654041590 conn(0x7f12540fb380 msgr2=0x7f1254105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.998+0000 7f1259c00640 1 -- 172.21.15.38:0/654041590 shutdown_connections 2024-03-24T00:45:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:27.998+0000 7f1259c00640 1 -- 172.21.15.38:0/654041590 wait complete. 2024-03-24T00:45:28.018 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 20, 'max_mds': 2, 'flags': 50} 2024-03-24T00:45:28.018 INFO:teuthology.run_tasks:Running task kclient... 2024-03-24T00:45:28.043 INFO:tasks.kclient:Mounting kernel clients... 2024-03-24T00:45:28.044 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-24T00:45:28.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:28.044 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-03-24T00:45:28.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:28.101 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-03-24T00:45:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:Cannot find device "ceph-brx" 2024-03-24T00:45:28.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T00:45:28.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:28.192 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-03-24T00:45:28.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:28.251 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-03-24T00:45:28.352 INFO:teuthology.orchestra.run.smithi088.stderr:Cannot find device "ceph-brx" 2024-03-24T00:45:28.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T00:45:28.355 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-24T00:45:28.356 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-24T00:45:28.356 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-24T00:45:28.356 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-24T00:45:28.447 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:28 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2555138281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T00:45:28.447 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:28 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/654041590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T00:45:28.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:28 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2555138281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T00:45:28.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:28 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/654041590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T00:45:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:45:29.963 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:29.963 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:45:29.963 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:29 smithi038 ceph-mon[29271]: pgmap v245: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-24T00:45:29.963 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:29.963 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:30.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:30.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:45:30.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:29 smithi088 ceph-mon[36610]: pgmap v245: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-24T00:45:30.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:30.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:30.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.869+0000 7f76ee400700 1 -- 172.21.15.38:0/3986276614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76e8102ef0 msgr2=0x7f76e81032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:30.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.869+0000 7f76ee400700 1 --2- 172.21.15.38:0/3986276614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76e8102ef0 0x7f76e81032d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009910 tx=0x7f76d8009c20 comp rx=0 tx=0).stop 2024-03-24T00:45:30.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.870+0000 7f76ee400700 1 -- 172.21.15.38:0/3986276614 shutdown_connections 2024-03-24T00:45:30.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.870+0000 7f76ee400700 1 --2- 172.21.15.38:0/3986276614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76e81038a0 0x7f76e81078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:30.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.870+0000 7f76ee400700 1 --2- 172.21.15.38:0/3986276614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76e8102ef0 0x7f76e81032d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:30.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.870+0000 7f76ee400700 1 -- 172.21.15.38:0/3986276614 >> 172.21.15.38:0/3986276614 conn(0x7f76e80fe780 msgr2=0x7f76e8100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:30.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.870+0000 7f76ee400700 1 -- 172.21.15.38:0/3986276614 shutdown_connections 2024-03-24T00:45:30.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.870+0000 7f76ee400700 1 -- 172.21.15.38:0/3986276614 wait complete. 2024-03-24T00:45:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.871+0000 7f76ee400700 1 Processor -- start 2024-03-24T00:45:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.871+0000 7f76ee400700 1 -- start start 2024-03-24T00:45:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.872+0000 7f76ee400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76e8102ef0 0x7f76e819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.872+0000 7f76ee400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76e81038a0 0x7f76e819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.872+0000 7f76ee400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e819f570 con 0x7f76e8102ef0 2024-03-24T00:45:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.872+0000 7f76ee400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e8198ae0 con 0x7f76e81038a0 2024-03-24T00:45:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.872+0000 7f76e7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76e8102ef0 0x7f76e819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.872+0000 7f76e7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76e8102ef0 0x7f76e819e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51316/0 (socket says 172.21.15.38:51316) 2024-03-24T00:45:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.872+0000 7f76e7400700 1 -- 172.21.15.38:0/3601846000 learned_addr learned my addr 172.21.15.38:0/3601846000 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:30.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.873+0000 7f76e7400700 1 -- 172.21.15.38:0/3601846000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76e81038a0 msgr2=0x7f76e819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:30.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.873+0000 7f76e7400700 1 --2- 172.21.15.38:0/3601846000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76e81038a0 0x7f76e819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:30.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.873+0000 7f76e7400700 1 -- 172.21.15.38:0/3601846000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d80095f0 con 0x7f76e8102ef0 2024-03-24T00:45:30.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.873+0000 7f76e7400700 1 --2- 172.21.15.38:0/3601846000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76e8102ef0 0x7f76e819e9a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f76d8006010 tx=0x7f76d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:30.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.873+0000 7f76dfe00700 1 -- 172.21.15.38:0/3601846000 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76d801d070 con 0x7f76e8102ef0 2024-03-24T00:45:30.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.873+0000 7f76dfe00700 1 -- 172.21.15.38:0/3601846000 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76d80056e0 con 0x7f76e8102ef0 2024-03-24T00:45:30.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.873+0000 7f76dfe00700 1 -- 172.21.15.38:0/3601846000 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76d8017400 con 0x7f76e8102ef0 2024-03-24T00:45:30.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.873+0000 7f76ee400700 1 -- 172.21.15.38:0/3601846000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e8198d60 con 0x7f76e8102ef0 2024-03-24T00:45:30.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.873+0000 7f76ee400700 1 -- 172.21.15.38:0/3601846000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e8199250 con 0x7f76e8102ef0 2024-03-24T00:45:30.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.875+0000 7f76dd600700 1 -- 172.21.15.38:0/3601846000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c00052f0 con 0x7f76e8102ef0 2024-03-24T00:45:30.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.876+0000 7f76dfe00700 1 -- 172.21.15.38:0/3601846000 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f76d800f6f0 con 0x7f76e8102ef0 2024-03-24T00:45:30.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.877+0000 7f76dfe00700 1 --2- 172.21.15.38:0/3601846000 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f76cc06cce0 0x7f76cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:30.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.877+0000 7f76dfe00700 1 -- 172.21.15.38:0/3601846000 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f76d808d720 con 0x7f76e8102ef0 2024-03-24T00:45:30.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.878+0000 7f76e6a00700 1 --2- 172.21.15.38:0/3601846000 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f76cc06cce0 0x7f76cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:30.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.878+0000 7f76e6a00700 1 --2- 172.21.15.38:0/3601846000 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f76cc06cce0 0x7f76cc06f1a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f76d4005fd0 tx=0x7f76d4005f60 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:30.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:30.882+0000 7f76dfe00700 1 -- 172.21.15.38:0/3601846000 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f76d805b480 con 0x7f76e8102ef0 2024-03-24T00:45:31.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:30 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:45:31.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:30 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:45:31.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:30 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:31.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.159+0000 7f76dd600700 1 -- 172.21.15.38:0/3601846000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f76c00061d0 con 0x7f76e8102ef0 2024-03-24T00:45:31.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.160+0000 7f76dfe00700 1 -- 172.21.15.38:0/3601846000 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+83 (secure 0 0 0) 0x7f76d805b010 con 0x7f76e8102ef0 2024-03-24T00:45:31.162 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-24T00:45:31.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.164+0000 7f76dd600700 1 -- 172.21.15.38:0/3601846000 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f76cc06cce0 msgr2=0x7f76cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:31.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.164+0000 7f76dd600700 1 --2- 172.21.15.38:0/3601846000 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f76cc06cce0 0x7f76cc06f1a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f76d4005fd0 tx=0x7f76d4005f60 comp rx=0 tx=0).stop 2024-03-24T00:45:31.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.164+0000 7f76dd600700 1 -- 172.21.15.38:0/3601846000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76e8102ef0 msgr2=0x7f76e819e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.164+0000 7f76dd600700 1 --2- 172.21.15.38:0/3601846000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76e8102ef0 0x7f76e819e9a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f76d8006010 tx=0x7f76d800be20 comp rx=0 tx=0).stop 2024-03-24T00:45:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.164+0000 7f76dd600700 1 -- 172.21.15.38:0/3601846000 shutdown_connections 2024-03-24T00:45:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.164+0000 7f76dd600700 1 --2- 172.21.15.38:0/3601846000 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f76cc06cce0 0x7f76cc06f1a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.164+0000 7f76dd600700 1 --2- 172.21.15.38:0/3601846000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76e8102ef0 0x7f76e819e9a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.164+0000 7f76dd600700 1 --2- 172.21.15.38:0/3601846000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76e81038a0 0x7f76e819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.164+0000 7f76dd600700 1 -- 172.21.15.38:0/3601846000 >> 172.21.15.38:0/3601846000 conn(0x7f76e80fe780 msgr2=0x7f76e80ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.164+0000 7f76dd600700 1 -- 172.21.15.38:0/3601846000 shutdown_connections 2024-03-24T00:45:31.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:31.164+0000 7f76dd600700 1 -- 172.21.15.38:0/3601846000 wait complete. 2024-03-24T00:45:31.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:30 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:45:31.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:30 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:45:31.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:30 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:45:31.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:31 smithi038 ceph-mon[29271]: pgmap v246: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-24T00:45:31.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:31 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3601846000' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T00:45:32.082 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-24T00:45:32.082 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-24T00:45:32.082 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-03-24T00:45:32.082 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-24T00:45:32.082 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-24T00:45:32.082 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-24T00:45:32.082 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-24T00:45:32.082 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-24T00:45:32.083 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-24T00:45:32.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:32.083 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-03-24T00:45:32.140 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-24T00:45:32.140 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-24T00:45:32.140 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-03-24T00:45:32.141 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-03-24T00:45:32.141 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-03-24T00:45:32.141 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-03-24T00:45:32.141 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T00:45:32.141 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-03-24T00:45:32.141 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-03-24T00:45:32.141 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T00:45:32.141 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-03-24T00:45:32.141 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-03-24T00:45:32.141 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-24T00:45:32.142 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-03-24T00:45:32.142 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-03-24T00:45:32.142 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-24T00:45:32.142 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42007sec preferred_lft 42007sec 2024-03-24T00:45:32.142 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-03-24T00:45:32.142 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-03-24T00:45:32.142 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T00:45:32.142 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-03-24T00:45:32.142 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-03-24T00:45:32.143 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-24T00:45:32.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:32.144 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-03-24T00:45:32.144 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-03-24T00:45:32.144 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-03-24T00:45:32.144 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-03-24T00:45:32.144 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-03-24T00:45:32.144 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-24T00:45:32.144 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-03-24T00:45:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:31 smithi088 ceph-mon[36610]: pgmap v246: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-24T00:45:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:31 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3601846000' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T00:45:32.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:32.606 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-24T00:45:32.693 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-03-24T00:45:32.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:32.697 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-03-24T00:45:32.752 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-03-24T00:45:32.752 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-24T00:45:32.753 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-24T00:45:32.753 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-24T00:45:32.753 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-24T00:45:32.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:32.753 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-03-24T00:45:32.753 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-03-24T00:45:32.753 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-24T00:45:32.753 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-24T00:45:32.754 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-24T00:45:32.754 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-03-24T00:45:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:32 smithi038 ceph-mon[29271]: pgmap v247: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 4.3 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-24T00:45:33.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:32 smithi088 ceph-mon[36610]: pgmap v247: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 4.3 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-24T00:45:33.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:33.590 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-03-24T00:45:33.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:33.649 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-03-24T00:45:33.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:33.706 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-03-24T00:45:33.706 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-03-24T00:45:33.706 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-24T00:45:33.706 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-24T00:45:33.706 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-03-24T00:45:33.862 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-24T00:45:33.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:33.862 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-03-24T00:45:33.862 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-03-24T00:45:33.862 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-24T00:45:33.862 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-24T00:45:33.862 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-24T00:45:33.862 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-24T00:45:33.862 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-24T00:45:33.862 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-03-24T00:45:34.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:34.151 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-03-24T00:45:34.152 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-03-24T00:45:34.152 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-03-24T00:45:34.152 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-03-24T00:45:34.152 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-03-24T00:45:34.333 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T00:45:34.333 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-24T00:45:34.386 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-24T00:45:34.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T00:45:34.386 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-24T00:45:34.440 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-24T00:45:34.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:34.441 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-24T00:45:34.571 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-24T00:45:34.572 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-24T00:45:34.572 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-03-24T00:45:35.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:34 smithi038 ceph-mon[29271]: pgmap v248: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-24T00:45:35.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:34 smithi088 ceph-mon[36610]: pgmap v248: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-24T00:45:35.244 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-24T00:45:35.244 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-24T00:45:35.244 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-03-24T00:45:35.244 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.88:3300" 2024-03-24T00:45:35.244 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-24T00:45:35.286 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-24T00:45:35.286 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-24T00:45:35.286 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-03-24T00:45:35.287 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.88:3300" 2024-03-24T00:45:35.287 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-24T00:45:35.287 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.88:6789" 2024-03-24T00:45:35.287 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.88:6789:/ 2024-03-24T00:45:35.287 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4796a0d6-e976-11ee-95cd-87774f69a715" will pass to kernel 2024-03-24T00:45:35.292 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-24T00:45:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-03-24T00:45:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-24T00:45:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-03-24T00:45:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-24T00:45:35.298 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-24T00:45:35.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:35.298 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-24T00:45:35.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:35.398 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T00:45:35.467 INFO:teuthology.orchestra.run.smithi038.stdout:client14532 2024-03-24T00:45:35.468 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:45:35.468 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T00:45:35.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:35.527 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T00:45:35.596 INFO:teuthology.orchestra.run.smithi038.stdout:client14532 2024-03-24T00:45:35.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:35.597 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T00:45:35.668 INFO:teuthology.orchestra.run.smithi038.stdout:client14532 2024-03-24T00:45:35.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:35.669 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4796a0d6-e976-11ee-95cd-87774f69a715.client14532/status') 2024-03-24T00:45:35.774 INFO:teuthology.orchestra.run.smithi038.stdout:instance: client.14532 (3)192.168.0.1:0/3187733578 2024-03-24T00:45:35.774 INFO:teuthology.orchestra.run.smithi038.stdout:blocklisted: false 2024-03-24T00:45:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records in 2024-03-24T00:45:35.775 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records out 2024-03-24T00:45:35.775 INFO:teuthology.orchestra.run.smithi038.stderr:70 bytes copied, 7.9389e-05 s, 882 kB/s 2024-03-24T00:45:35.779 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-24T00:45:35.779 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-24T00:45:35.779 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-24T00:45:35.779 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-24T00:45:36.229 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:45:37.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:36 smithi038 ceph-mon[29271]: pgmap v249: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:45:37.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:36 smithi088 ceph-mon[36610]: pgmap v249: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:45:38.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.472+0000 7f38f5e00700 1 -- 172.21.15.38:0/2894637027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f0107cc0 msgr2=0x7f38f010bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.472+0000 7f38f5e00700 1 --2- 172.21.15.38:0/2894637027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f0107cc0 0x7f38f010bd10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009910 tx=0x7f38dc009c20 comp rx=0 tx=0).stop 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.472+0000 7f38f5e00700 1 -- 172.21.15.38:0/2894637027 shutdown_connections 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.472+0000 7f38f5e00700 1 --2- 172.21.15.38:0/2894637027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f0107cc0 0x7f38f010bd10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.472+0000 7f38f5e00700 1 --2- 172.21.15.38:0/2894637027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38f0107310 0x7f38f01076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.472+0000 7f38f5e00700 1 -- 172.21.15.38:0/2894637027 >> 172.21.15.38:0/2894637027 conn(0x7f38f00768e0 msgr2=0x7f38f0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.473+0000 7f38f5e00700 1 -- 172.21.15.38:0/2894637027 shutdown_connections 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.473+0000 7f38f5e00700 1 -- 172.21.15.38:0/2894637027 wait complete. 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.474+0000 7f38f5e00700 1 Processor -- start 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.474+0000 7f38f5e00700 1 -- start start 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.474+0000 7f38f5e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f0107310 0x7f38f019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.474+0000 7f38f5e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38f0107cc0 0x7f38f019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.474+0000 7f38f5e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f019dbe0 con 0x7f38f0107310 2024-03-24T00:45:38.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.474+0000 7f38f5e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f01a18b0 con 0x7f38f0107cc0 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.475+0000 7f38eea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f0107310 0x7f38f019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.475+0000 7f38eea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f0107310 0x7f38f019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59438/0 (socket says 172.21.15.38:59438) 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.475+0000 7f38eea00700 1 -- 172.21.15.38:0/2388752481 learned_addr learned my addr 172.21.15.38:0/2388752481 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.475+0000 7f38ee000700 1 --2- 172.21.15.38:0/2388752481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38f0107cc0 0x7f38f019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.475+0000 7f38eea00700 1 -- 172.21.15.38:0/2388752481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38f0107cc0 msgr2=0x7f38f019d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.475+0000 7f38eea00700 1 --2- 172.21.15.38:0/2388752481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38f0107cc0 0x7f38f019d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.475+0000 7f38eea00700 1 -- 172.21.15.38:0/2388752481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc0095f0 con 0x7f38f0107310 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.475+0000 7f38eea00700 1 --2- 172.21.15.38:0/2388752481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f0107310 0x7f38f019cfc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f38e000d8d0 tx=0x7f38e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.476+0000 7f38f4a00700 1 -- 172.21.15.38:0/2388752481 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38e0009940 con 0x7f38f0107310 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.476+0000 7f38f5e00700 1 -- 172.21.15.38:0/2388752481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f01a1b90 con 0x7f38f0107310 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.476+0000 7f38f4a00700 1 -- 172.21.15.38:0/2388752481 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38e0010460 con 0x7f38f0107310 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.476+0000 7f38f4a00700 1 -- 172.21.15.38:0/2388752481 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38e000f5f0 con 0x7f38f0107310 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.476+0000 7f38f5e00700 1 -- 172.21.15.38:0/2388752481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f01a20e0 con 0x7f38f0107310 2024-03-24T00:45:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.477+0000 7f38f5e00700 1 -- 172.21.15.38:0/2388752481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f010f710 con 0x7f38f0107310 2024-03-24T00:45:38.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.478+0000 7f38f4a00700 1 -- 172.21.15.38:0/2388752481 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f38e00105d0 con 0x7f38f0107310 2024-03-24T00:45:38.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.479+0000 7f38f4a00700 1 --2- 172.21.15.38:0/2388752481 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f38d406ccf0 0x7f38d406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:38.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.479+0000 7f38f4a00700 1 -- 172.21.15.38:0/2388752481 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f38e008c280 con 0x7f38f0107310 2024-03-24T00:45:38.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.479+0000 7f38ee000700 1 --2- 172.21.15.38:0/2388752481 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f38d406ccf0 0x7f38d406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:38.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.480+0000 7f38ee000700 1 --2- 172.21.15.38:0/2388752481 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f38d406ccf0 0x7f38d406f1b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f38f019e520 tx=0x7f38dc01a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:38.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.485+0000 7f38f4a00700 1 -- 172.21.15.38:0/2388752481 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f38e00565d0 con 0x7f38f0107310 2024-03-24T00:45:38.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.763+0000 7f38f5e00700 1 -- 172.21.15.38:0/2388752481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f38f01a2350 con 0x7f38f0107310 2024-03-24T00:45:38.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.764+0000 7f38f4a00700 1 -- 172.21.15.38:0/2388752481 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+83 (secure 0 0 0) 0x7f38e0059bf0 con 0x7f38f0107310 2024-03-24T00:45:38.765 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-24T00:45:38.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.767+0000 7f38f5e00700 1 -- 172.21.15.38:0/2388752481 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f38d406ccf0 msgr2=0x7f38d406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.767+0000 7f38f5e00700 1 --2- 172.21.15.38:0/2388752481 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f38d406ccf0 0x7f38d406f1b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f38f019e520 tx=0x7f38dc01a040 comp rx=0 tx=0).stop 2024-03-24T00:45:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.767+0000 7f38f5e00700 1 -- 172.21.15.38:0/2388752481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f0107310 msgr2=0x7f38f019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.767+0000 7f38f5e00700 1 --2- 172.21.15.38:0/2388752481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f0107310 0x7f38f019cfc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f38e000d8d0 tx=0x7f38e000dc90 comp rx=0 tx=0).stop 2024-03-24T00:45:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.767+0000 7f38f5e00700 1 -- 172.21.15.38:0/2388752481 shutdown_connections 2024-03-24T00:45:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.767+0000 7f38f5e00700 1 --2- 172.21.15.38:0/2388752481 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f38d406ccf0 0x7f38d406f1b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.767+0000 7f38f5e00700 1 --2- 172.21.15.38:0/2388752481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f0107310 0x7f38f019cfc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.767+0000 7f38f5e00700 1 --2- 172.21.15.38:0/2388752481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38f0107cc0 0x7f38f019d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.767+0000 7f38f5e00700 1 -- 172.21.15.38:0/2388752481 >> 172.21.15.38:0/2388752481 conn(0x7f38f00768e0 msgr2=0x7f38f0076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.767+0000 7f38f5e00700 1 -- 172.21.15.38:0/2388752481 shutdown_connections 2024-03-24T00:45:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:38.767+0000 7f38f5e00700 1 -- 172.21.15.38:0/2388752481 wait complete. 2024-03-24T00:45:39.640 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-24T00:45:39.640 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-24T00:45:39.640 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-03-24T00:45:39.640 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-24T00:45:39.641 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-24T00:45:39.641 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-24T00:45:39.641 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-24T00:45:39.641 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-24T00:45:39.641 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-24T00:45:39.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:39.641 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-03-24T00:45:39.698 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-24T00:45:39.698 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-24T00:45:39.698 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-03-24T00:45:39.699 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-24T00:45:39.700 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 41995sec preferred_lft 41995sec 2024-03-24T00:45:39.700 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2024-03-24T00:45:39.700 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-03-24T00:45:39.701 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-24T00:45:39.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:39.701 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2024-03-24T00:45:39.701 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-03-24T00:45:39.701 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add name ceph-brx type bridge 2024-03-24T00:45:39.701 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr flush dev ceph-brx 2024-03-24T00:45:39.701 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set ceph-brx up 2024-03-24T00:45:39.701 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-24T00:45:39.701 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-03-24T00:45:39.857 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:39 smithi038 ceph-mon[29271]: pgmap v250: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:45:39.857 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:39 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2388752481' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T00:45:39.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:39 smithi088 ceph-mon[36610]: pgmap v250: 65 pgs: 65 active+clean; 580 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:45:39.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:39 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2388752481' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T00:45:40.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:40.141 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-24T00:45:40.220 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-03-24T00:45:40.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:40.223 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-03-24T00:45:40.309 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-03-24T00:45:40.310 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-24T00:45:40.310 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-24T00:45:40.310 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-24T00:45:40.310 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-24T00:45:40.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:40.311 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2024-03-24T00:45:40.311 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-03-24T00:45:40.311 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-24T00:45:40.311 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-24T00:45:40.311 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-24T00:45:40.312 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-03-24T00:45:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:40 smithi088 ceph-mon[36610]: pgmap v251: 65 pgs: 65 active+clean; 581 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-03-24T00:45:41.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:41.010 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-03-24T00:45:41.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:41.054 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2024-03-24T00:45:41.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:40 smithi038 ceph-mon[29271]: pgmap v251: 65 pgs: 65 active+clean; 581 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-03-24T00:45:41.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:41.144 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2024-03-24T00:45:41.144 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-03-24T00:45:41.144 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-24T00:45:41.144 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-24T00:45:41.144 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-03-24T00:45:41.307 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-24T00:45:41.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:41.308 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2024-03-24T00:45:41.309 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-03-24T00:45:41.309 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-24T00:45:41.309 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-24T00:45:41.309 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-24T00:45:41.309 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-24T00:45:41.309 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-24T00:45:41.309 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-03-24T00:45:41.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:41.560 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2024-03-24T00:45:41.560 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-03-24T00:45:41.560 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-03-24T00:45:41.560 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.0 master ceph-brx 2024-03-24T00:45:41.560 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-03-24T00:45:41.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T00:45:41.711 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-24T00:45:41.759 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-24T00:45:41.759 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T00:45:41.759 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-24T00:45:41.852 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-24T00:45:41.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:41.852 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-24T00:45:41.995 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-24T00:45:41.995 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-24T00:45:41.995 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-03-24T00:45:42.618 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-24T00:45:42.618 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-24T00:45:42.618 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-03-24T00:45:42.618 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.88:3300" 2024-03-24T00:45:42.618 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-24T00:45:42.652 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-24T00:45:42.652 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-24T00:45:42.653 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-03-24T00:45:42.653 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.88:3300" 2024-03-24T00:45:42.653 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-24T00:45:42.653 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.38:6789,172.21.15.88:6789" 2024-03-24T00:45:42.653 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:6789,172.21.15.88:6789:/ 2024-03-24T00:45:42.653 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=4796a0d6-e976-11ee-95cd-87774f69a715" will pass to kernel 2024-03-24T00:45:42.657 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-24T00:45:42.657 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-03-24T00:45:42.657 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-24T00:45:42.657 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-03-24T00:45:42.658 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-24T00:45:42.663 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-24T00:45:42.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:42.663 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-24T00:45:42.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:42.749 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T00:45:42.807 INFO:teuthology.orchestra.run.smithi088.stdout:client14538 2024-03-24T00:45:42.808 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:45:42.808 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T00:45:42.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:42.854 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T00:45:42.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:42 smithi088 ceph-mon[36610]: pgmap v252: 65 pgs: 65 active+clean; 581 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-03-24T00:45:42.957 INFO:teuthology.orchestra.run.smithi088.stdout:client14538 2024-03-24T00:45:42.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:42.957 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T00:45:43.062 INFO:teuthology.orchestra.run.smithi088.stdout:client14538 2024-03-24T00:45:43.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T00:45:43.063 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4796a0d6-e976-11ee-95cd-87774f69a715.client14538/status') 2024-03-24T00:45:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:42 smithi038 ceph-mon[29271]: pgmap v252: 65 pgs: 65 active+clean; 581 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-03-24T00:45:43.159 INFO:teuthology.orchestra.run.smithi088.stdout:instance: client.14538 (3)172.21.15.88:0/43681896 2024-03-24T00:45:43.160 INFO:teuthology.orchestra.run.smithi088.stdout:blocklisted: false 2024-03-24T00:45:43.160 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records in 2024-03-24T00:45:43.160 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records out 2024-03-24T00:45:43.160 INFO:teuthology.orchestra.run.smithi088.stderr:69 bytes copied, 9.2448e-05 s, 746 kB/s 2024-03-24T00:45:43.164 INFO:teuthology.run_tasks:Running task print... 2024-03-24T00:45:43.176 INFO:teuthology.task.print:**** done client 2024-03-24T00:45:43.176 INFO:teuthology.run_tasks:Running task parallel... 2024-03-24T00:45:43.187 INFO:teuthology.task.parallel:starting parallel... 2024-03-24T00:45:43.188 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-24T00:45:43.188 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-24T00:45:43.189 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:45:43.189 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-24T00:45:43.189 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-24T00:45:43.189 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-24T00:45:43.193 INFO:tasks.workunit:Pulling workunits from ref 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:45:43.193 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-24T00:45:43.193 INFO:tasks.workunit:timeout=3h 2024-03-24T00:45:43.193 INFO:tasks.workunit:cleanup=True 2024-03-24T00:45:43.194 DEBUG:teuthology.orchestra.run.smithi038:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-24T00:45:43.256 INFO:teuthology.orchestra.run.smithi038.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-24T00:45:43.256 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-24T00:45:43.256 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 93h/147d Inode: 1 Links: 2 2024-03-24T00:45:43.256 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T00:45:43.256 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-24T00:45:43.257 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-24T00:45:43.257 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-03-24 00:44:56.198183079 +0000 2024-03-24T00:45:43.257 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-03-24 00:45:35.389614416 +0000 2024-03-24T00:45:43.257 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-03-24 00:44:56.198183079 +0000 2024-03-24T00:45:43.257 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-24T00:45:43.257 DEBUG:teuthology.orchestra.run.smithi038:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-24T00:45:43.394 DEBUG:teuthology.orchestra.run.smithi088:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-24T00:45:43.452 INFO:teuthology.orchestra.run.smithi088.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-24T00:45:43.452 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-03-24T00:45:43.452 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 7dh/125d Inode: 1 Links: 3 2024-03-24T00:45:43.452 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T00:45:43.452 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-24T00:45:43.452 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-24T00:45:43.452 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-03-24 00:45:43.378464034 +0000 2024-03-24T00:45:43.452 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-03-24 00:45:43.378464034 +0000 2024-03-24T00:45:43.453 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-03-24 00:44:56.198183079 +0000 2024-03-24T00:45:43.453 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-24T00:45:43.453 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-24T00:45:43.557 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:45:43.558 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:45:43.608 INFO:tasks.workunit.client.0.smithi038.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-24T00:45:43.612 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:45:43.615 INFO:tasks.workunit.client.1.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-24T00:45:44.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:44 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:45:44.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:44 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:45:45.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.824+0000 7f9f75800700 1 -- 172.21.15.38:0/362573065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f70100a30 msgr2=0x7f9f70100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:45.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.824+0000 7f9f75800700 1 --2- 172.21.15.38:0/362573065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f70100a30 0x7f9f70100e90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009910 tx=0x7f9f58009c20 comp rx=0 tx=0).stop 2024-03-24T00:45:45.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.824+0000 7f9f75800700 1 -- 172.21.15.38:0/362573065 shutdown_connections 2024-03-24T00:45:45.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.824+0000 7f9f75800700 1 --2- 172.21.15.38:0/362573065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f70100a30 0x7f9f70100e90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:45.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.824+0000 7f9f75800700 1 --2- 172.21.15.38:0/362573065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f700ff830 0x7f9f700ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:45.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.824+0000 7f9f75800700 1 -- 172.21.15.38:0/362573065 >> 172.21.15.38:0/362573065 conn(0x7f9f700fadf0 msgr2=0x7f9f700fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:45.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.824+0000 7f9f75800700 1 -- 172.21.15.38:0/362573065 shutdown_connections 2024-03-24T00:45:45.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.825+0000 7f9f75800700 1 -- 172.21.15.38:0/362573065 wait complete. 2024-03-24T00:45:45.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.826+0000 7f9f75800700 1 Processor -- start 2024-03-24T00:45:45.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.826+0000 7f9f75800700 1 -- start start 2024-03-24T00:45:45.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.827+0000 7f9f75800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f700ff830 0x7f9f70068f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:45.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.827+0000 7f9f75800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f70100a30 0x7f9f7019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:45.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.827+0000 7f9f75800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7019b5b0 con 0x7f9f700ff830 2024-03-24T00:45:45.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.827+0000 7f9f75800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7019b6f0 con 0x7f9f70100a30 2024-03-24T00:45:45.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.827+0000 7f9f6fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f700ff830 0x7f9f70068f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:45.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.827+0000 7f9f6fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f700ff830 0x7f9f70068f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40308/0 (socket says 172.21.15.38:40308) 2024-03-24T00:45:45.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.827+0000 7f9f6fe00700 1 -- 172.21.15.38:0/43145556 learned_addr learned my addr 172.21.15.38:0/43145556 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:45.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.827+0000 7f9f67400700 1 --2- 172.21.15.38:0/43145556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f70100a30 0x7f9f7019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:45.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.828+0000 7f9f6fe00700 1 -- 172.21.15.38:0/43145556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f70100a30 msgr2=0x7f9f7019b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:45.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.828+0000 7f9f6fe00700 1 --2- 172.21.15.38:0/43145556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f70100a30 0x7f9f7019b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:45.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.828+0000 7f9f6fe00700 1 -- 172.21.15.38:0/43145556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f580095f0 con 0x7f9f700ff830 2024-03-24T00:45:45.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.828+0000 7f9f6fe00700 1 --2- 172.21.15.38:0/43145556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f700ff830 0x7f9f70068f90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000c910 tx=0x7f9f6000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:45.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.828+0000 7f9f6d600700 1 -- 172.21.15.38:0/43145556 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f600041f0 con 0x7f9f700ff830 2024-03-24T00:45:45.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.828+0000 7f9f6d600700 1 -- 172.21.15.38:0/43145556 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f60007dc0 con 0x7f9f700ff830 2024-03-24T00:45:45.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.828+0000 7f9f75800700 1 -- 172.21.15.38:0/43145556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7019b980 con 0x7f9f700ff830 2024-03-24T00:45:45.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.828+0000 7f9f6d600700 1 -- 172.21.15.38:0/43145556 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f60003e70 con 0x7f9f700ff830 2024-03-24T00:45:45.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.829+0000 7f9f75800700 1 -- 172.21.15.38:0/43145556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f7019bf50 con 0x7f9f700ff830 2024-03-24T00:45:45.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.830+0000 7f9f75800700 1 -- 172.21.15.38:0/43145556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7018f3b0 con 0x7f9f700ff830 2024-03-24T00:45:45.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.831+0000 7f9f6d600700 1 -- 172.21.15.38:0/43145556 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9f60004350 con 0x7f9f700ff830 2024-03-24T00:45:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.832+0000 7f9f6d600700 1 --2- 172.21.15.38:0/43145556 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f5406cce0 0x7f9f5406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.832+0000 7f9f6d600700 1 -- 172.21.15.38:0/43145556 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9f6008bff0 con 0x7f9f700ff830 2024-03-24T00:45:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.832+0000 7f9f67400700 1 --2- 172.21.15.38:0/43145556 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f5406cce0 0x7f9f5406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.833+0000 7f9f67400700 1 --2- 172.21.15.38:0/43145556 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f5406cce0 0x7f9f5406f1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9f58006010 tx=0x7f9f5800b580 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:45.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:45.837+0000 7f9f6d600700 1 -- 172.21.15.38:0/43145556 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9f60059df0 con 0x7f9f700ff830 2024-03-24T00:45:45.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:45 smithi088 ceph-mon[36610]: pgmap v253: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-03-24T00:45:46.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.062+0000 7f9f75800700 1 -- 172.21.15.38:0/43145556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f9f7004f2d0 con 0x7f9f700ff830 2024-03-24T00:45:46.063 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:45 smithi038 ceph-mon[29271]: pgmap v253: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-03-24T00:45:46.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.063+0000 7f9f6d600700 1 -- 172.21.15.38:0/43145556 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f9f60059980 con 0x7f9f700ff830 2024-03-24T00:45:46.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.066+0000 7f9f75800700 1 -- 172.21.15.38:0/43145556 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f5406cce0 msgr2=0x7f9f5406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:46.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.066+0000 7f9f75800700 1 --2- 172.21.15.38:0/43145556 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f5406cce0 0x7f9f5406f1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9f58006010 tx=0x7f9f5800b580 comp rx=0 tx=0).stop 2024-03-24T00:45:46.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.066+0000 7f9f75800700 1 -- 172.21.15.38:0/43145556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f700ff830 msgr2=0x7f9f70068f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:46.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.066+0000 7f9f75800700 1 --2- 172.21.15.38:0/43145556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f700ff830 0x7f9f70068f90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000c910 tx=0x7f9f6000ccd0 comp rx=0 tx=0).stop 2024-03-24T00:45:46.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.067+0000 7f9f75800700 1 -- 172.21.15.38:0/43145556 shutdown_connections 2024-03-24T00:45:46.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.067+0000 7f9f75800700 1 --2- 172.21.15.38:0/43145556 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f9f5406cce0 0x7f9f5406f1a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:46.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.067+0000 7f9f75800700 1 --2- 172.21.15.38:0/43145556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f700ff830 0x7f9f70068f90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:46.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.067+0000 7f9f75800700 1 --2- 172.21.15.38:0/43145556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f70100a30 0x7f9f7019b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:46.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.067+0000 7f9f75800700 1 -- 172.21.15.38:0/43145556 >> 172.21.15.38:0/43145556 conn(0x7f9f700fadf0 msgr2=0x7f9f70103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:46.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.067+0000 7f9f75800700 1 -- 172.21.15.38:0/43145556 shutdown_connections 2024-03-24T00:45:46.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:46.067+0000 7f9f75800700 1 -- 172.21.15.38:0/43145556 wait complete. 2024-03-24T00:45:46.912 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-24T00:45:47.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:46 smithi038 ceph-mon[29271]: pgmap v254: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 852 B/s wr, 4 op/s 2024-03-24T00:45:47.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:46 smithi088 ceph-mon[36610]: pgmap v254: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 852 B/s wr, 4 op/s 2024-03-24T00:45:47.348 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:45:49.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.419+0000 7f007b200700 1 -- 172.21.15.38:0/797002875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00740ffd30 msgr2=0x7f00741001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:49.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.419+0000 7f007b200700 1 --2- 172.21.15.38:0/797002875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00740ffd30 0x7f00741001b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0060009910 tx=0x7f0060009c20 comp rx=0 tx=0).stop 2024-03-24T00:45:49.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.419+0000 7f007b200700 1 -- 172.21.15.38:0/797002875 shutdown_connections 2024-03-24T00:45:49.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.420+0000 7f007b200700 1 --2- 172.21.15.38:0/797002875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00740ffd30 0x7f00741001b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:49.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.420+0000 7f007b200700 1 --2- 172.21.15.38:0/797002875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00740ff3d0 0x7f00740ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:49.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.420+0000 7f007b200700 1 -- 172.21.15.38:0/797002875 >> 172.21.15.38:0/797002875 conn(0x7f00740faf70 msgr2=0x7f00740fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:49.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.420+0000 7f007b200700 1 -- 172.21.15.38:0/797002875 shutdown_connections 2024-03-24T00:45:49.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.420+0000 7f007b200700 1 -- 172.21.15.38:0/797002875 wait complete. 2024-03-24T00:45:49.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.421+0000 7f007b200700 1 Processor -- start 2024-03-24T00:45:49.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.421+0000 7f007b200700 1 -- start start 2024-03-24T00:45:49.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.421+0000 7f007b200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00740ff3d0 0x7f0074198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:49.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.421+0000 7f007b200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00740ffd30 0x7f0074198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:49.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.421+0000 7f007b200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00741991f0 con 0x7f00740ff3d0 2024-03-24T00:45:49.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.421+0000 7f007b200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0074199330 con 0x7f00740ffd30 2024-03-24T00:45:49.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.422+0000 7f0073e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00740ff3d0 0x7f0074198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:49.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.422+0000 7f0073e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00740ff3d0 0x7f0074198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40332/0 (socket says 172.21.15.38:40332) 2024-03-24T00:45:49.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.422+0000 7f0073e00700 1 -- 172.21.15.38:0/1333825421 learned_addr learned my addr 172.21.15.38:0/1333825421 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:49.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.422+0000 7f0073400700 1 --2- 172.21.15.38:0/1333825421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00740ffd30 0x7f0074198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:49.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.422+0000 7f0073e00700 1 -- 172.21.15.38:0/1333825421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00740ffd30 msgr2=0x7f0074198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:49.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.422+0000 7f0073e00700 1 --2- 172.21.15.38:0/1333825421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00740ffd30 0x7f0074198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:49.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.422+0000 7f0073e00700 1 -- 172.21.15.38:0/1333825421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00600095f0 con 0x7f00740ff3d0 2024-03-24T00:45:49.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.422+0000 7f0073e00700 1 --2- 172.21.15.38:0/1333825421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00740ff3d0 0x7f0074198690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f006400d8d0 tx=0x7f006400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:49.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.423+0000 7f0070c00700 1 -- 172.21.15.38:0/1333825421 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0064009940 con 0x7f00740ff3d0 2024-03-24T00:45:49.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.423+0000 7f0070c00700 1 -- 172.21.15.38:0/1333825421 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0064010460 con 0x7f00740ff3d0 2024-03-24T00:45:49.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.423+0000 7f0070c00700 1 -- 172.21.15.38:0/1333825421 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006400f5f0 con 0x7f00740ff3d0 2024-03-24T00:45:49.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.423+0000 7f007b200700 1 -- 172.21.15.38:0/1333825421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007419dd90 con 0x7f00740ff3d0 2024-03-24T00:45:49.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.423+0000 7f007b200700 1 -- 172.21.15.38:0/1333825421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007419e2d0 con 0x7f00740ff3d0 2024-03-24T00:45:49.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.424+0000 7f007b200700 1 -- 172.21.15.38:0/1333825421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007404eaf0 con 0x7f00740ff3d0 2024-03-24T00:45:49.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.426+0000 7f0070c00700 1 -- 172.21.15.38:0/1333825421 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f00640105d0 con 0x7f00740ff3d0 2024-03-24T00:45:49.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.427+0000 7f0070c00700 1 --2- 172.21.15.38:0/1333825421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f005806ccf0 0x7f005806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:49.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.427+0000 7f0070c00700 1 -- 172.21.15.38:0/1333825421 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f006408c1a0 con 0x7f00740ff3d0 2024-03-24T00:45:49.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.428+0000 7f0073400700 1 --2- 172.21.15.38:0/1333825421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f005806ccf0 0x7f005806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:49.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.428+0000 7f0073400700 1 --2- 172.21.15.38:0/1333825421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f005806ccf0 0x7f005806f1b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00600051c0 tx=0x7f006000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:49.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.431+0000 7f0070c00700 1 -- 172.21.15.38:0/1333825421 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f006405a030 con 0x7f00740ff3d0 2024-03-24T00:45:49.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.659+0000 7f007b200700 1 -- 172.21.15.38:0/1333825421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f0074192850 con 0x7f00740ff3d0 2024-03-24T00:45:49.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.661+0000 7f0070c00700 1 -- 172.21.15.38:0/1333825421 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0064059bc0 con 0x7f00740ff3d0 2024-03-24T00:45:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.663+0000 7f007b200700 1 -- 172.21.15.38:0/1333825421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f005806ccf0 msgr2=0x7f005806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.663+0000 7f007b200700 1 --2- 172.21.15.38:0/1333825421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f005806ccf0 0x7f005806f1b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00600051c0 tx=0x7f006000b540 comp rx=0 tx=0).stop 2024-03-24T00:45:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.663+0000 7f007b200700 1 -- 172.21.15.38:0/1333825421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00740ff3d0 msgr2=0x7f0074198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.663+0000 7f007b200700 1 --2- 172.21.15.38:0/1333825421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00740ff3d0 0x7f0074198690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f006400d8d0 tx=0x7f006400dc90 comp rx=0 tx=0).stop 2024-03-24T00:45:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.664+0000 7f007b200700 1 -- 172.21.15.38:0/1333825421 shutdown_connections 2024-03-24T00:45:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.664+0000 7f007b200700 1 --2- 172.21.15.38:0/1333825421 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f005806ccf0 0x7f005806f1b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:49.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.664+0000 7f007b200700 1 --2- 172.21.15.38:0/1333825421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00740ff3d0 0x7f0074198690 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:49.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.664+0000 7f007b200700 1 --2- 172.21.15.38:0/1333825421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00740ffd30 0x7f0074198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:49.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.664+0000 7f007b200700 1 -- 172.21.15.38:0/1333825421 >> 172.21.15.38:0/1333825421 conn(0x7f00740faf70 msgr2=0x7f0074107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:49.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.664+0000 7f007b200700 1 -- 172.21.15.38:0/1333825421 shutdown_connections 2024-03-24T00:45:49.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:49.664+0000 7f007b200700 1 -- 172.21.15.38:0/1333825421 wait complete. 2024-03-24T00:45:49.835 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:49 smithi038 ceph-mon[29271]: pgmap v255: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 938 B/s wr, 4 op/s 2024-03-24T00:45:49.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:49 smithi088 ceph-mon[36610]: pgmap v255: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 938 B/s wr, 4 op/s 2024-03-24T00:45:50.553 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-24T00:45:50.963 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:45:51.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:50 smithi038 ceph-mon[29271]: pgmap v256: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-24T00:45:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:50 smithi088 ceph-mon[36610]: pgmap v256: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-24T00:45:53.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.001+0000 7efc42200700 1 -- 172.21.15.38:0/4156700759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c108300 msgr2=0x7efc3c108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:53.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.001+0000 7efc42200700 1 --2- 172.21.15.38:0/4156700759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c108300 0x7efc3c108780 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efc2400b0a0 tx=0x7efc2400b3b0 comp rx=0 tx=0).stop 2024-03-24T00:45:53.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.001+0000 7efc42200700 1 -- 172.21.15.38:0/4156700759 shutdown_connections 2024-03-24T00:45:53.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.001+0000 7efc42200700 1 --2- 172.21.15.38:0/4156700759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c108300 0x7efc3c108780 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:53.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.001+0000 7efc42200700 1 --2- 172.21.15.38:0/4156700759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c107130 0x7efc3c107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:53.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.001+0000 7efc42200700 1 -- 172.21.15.38:0/4156700759 >> 172.21.15.38:0/4156700759 conn(0x7efc3c0762d0 msgr2=0x7efc3c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:53.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.001+0000 7efc42200700 1 -- 172.21.15.38:0/4156700759 shutdown_connections 2024-03-24T00:45:53.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.001+0000 7efc42200700 1 -- 172.21.15.38:0/4156700759 wait complete. 2024-03-24T00:45:53.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.002+0000 7efc42200700 1 Processor -- start 2024-03-24T00:45:53.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.002+0000 7efc42200700 1 -- start start 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.003+0000 7efc42200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c107130 0x7efc3c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.003+0000 7efc42200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c108300 0x7efc3c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.003+0000 7efc42200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c19d540 con 0x7efc3c108300 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.003+0000 7efc42200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c19d680 con 0x7efc3c107130 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.003+0000 7efc32a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c108300 0x7efc3c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.003+0000 7efc32a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c108300 0x7efc3c19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42698/0 (socket says 172.21.15.38:42698) 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.003+0000 7efc32a00700 1 -- 172.21.15.38:0/2588298256 learned_addr learned my addr 172.21.15.38:0/2588298256 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.003+0000 7efc3b400700 1 --2- 172.21.15.38:0/2588298256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c107130 0x7efc3c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.004+0000 7efc32a00700 1 -- 172.21.15.38:0/2588298256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c107130 msgr2=0x7efc3c19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.004+0000 7efc32a00700 1 --2- 172.21.15.38:0/2588298256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c107130 0x7efc3c19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.004+0000 7efc32a00700 1 -- 172.21.15.38:0/2588298256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc24009d00 con 0x7efc3c108300 2024-03-24T00:45:53.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.004+0000 7efc32a00700 1 --2- 172.21.15.38:0/2588298256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c108300 0x7efc3c19cfb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efc24006130 tx=0x7efc24003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:53.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.004+0000 7efc38c00700 1 -- 172.21.15.38:0/2588298256 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc24021070 con 0x7efc3c108300 2024-03-24T00:45:53.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.004+0000 7efc38c00700 1 -- 172.21.15.38:0/2588298256 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc2400ce60 con 0x7efc3c108300 2024-03-24T00:45:53.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.004+0000 7efc42200700 1 -- 172.21.15.38:0/2588298256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c1a2090 con 0x7efc3c108300 2024-03-24T00:45:53.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.004+0000 7efc38c00700 1 -- 172.21.15.38:0/2588298256 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc2400e040 con 0x7efc3c108300 2024-03-24T00:45:53.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.005+0000 7efc42200700 1 -- 172.21.15.38:0/2588298256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c1a25d0 con 0x7efc3c108300 2024-03-24T00:45:53.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.006+0000 7efc42200700 1 -- 172.21.15.38:0/2588298256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c196ca0 con 0x7efc3c108300 2024-03-24T00:45:53.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.007+0000 7efc38c00700 1 -- 172.21.15.38:0/2588298256 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7efc24004250 con 0x7efc3c108300 2024-03-24T00:45:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.008+0000 7efc38c00700 1 --2- 172.21.15.38:0/2588298256 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7efc2006cce0 0x7efc2006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.008+0000 7efc38c00700 1 -- 172.21.15.38:0/2588298256 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7efc2408d560 con 0x7efc3c108300 2024-03-24T00:45:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.008+0000 7efc3b400700 1 --2- 172.21.15.38:0/2588298256 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7efc2006cce0 0x7efc2006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:53.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.009+0000 7efc3b400700 1 --2- 172.21.15.38:0/2588298256 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7efc2006cce0 0x7efc2006f1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc2c005ea0 tx=0x7efc2c005e30 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:53.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.013+0000 7efc38c00700 1 -- 172.21.15.38:0/2588298256 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efc2405b340 con 0x7efc3c108300 2024-03-24T00:45:53.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:52 smithi038 ceph-mon[29271]: pgmap v257: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-03-24T00:45:53.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:52 smithi088 ceph-mon[36610]: pgmap v257: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-03-24T00:45:53.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.238+0000 7efc42200700 1 -- 172.21.15.38:0/2588298256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7efc3c04ea80 con 0x7efc3c108300 2024-03-24T00:45:53.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.239+0000 7efc38c00700 1 -- 172.21.15.38:0/2588298256 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7efc2405aed0 con 0x7efc3c108300 2024-03-24T00:45:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.242+0000 7efc42200700 1 -- 172.21.15.38:0/2588298256 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7efc2006cce0 msgr2=0x7efc2006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.242+0000 7efc42200700 1 --2- 172.21.15.38:0/2588298256 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7efc2006cce0 0x7efc2006f1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc2c005ea0 tx=0x7efc2c005e30 comp rx=0 tx=0).stop 2024-03-24T00:45:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.242+0000 7efc42200700 1 -- 172.21.15.38:0/2588298256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c108300 msgr2=0x7efc3c19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.242+0000 7efc42200700 1 --2- 172.21.15.38:0/2588298256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c108300 0x7efc3c19cfb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efc24006130 tx=0x7efc24003ce0 comp rx=0 tx=0).stop 2024-03-24T00:45:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.242+0000 7efc42200700 1 -- 172.21.15.38:0/2588298256 shutdown_connections 2024-03-24T00:45:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.242+0000 7efc42200700 1 --2- 172.21.15.38:0/2588298256 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7efc2006cce0 0x7efc2006f1a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.242+0000 7efc42200700 1 --2- 172.21.15.38:0/2588298256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c107130 0x7efc3c19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.242+0000 7efc42200700 1 --2- 172.21.15.38:0/2588298256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c108300 0x7efc3c19cfb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.242+0000 7efc42200700 1 -- 172.21.15.38:0/2588298256 >> 172.21.15.38:0/2588298256 conn(0x7efc3c0762d0 msgr2=0x7efc3c10b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.242+0000 7efc42200700 1 -- 172.21.15.38:0/2588298256 shutdown_connections 2024-03-24T00:45:53.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:53.242+0000 7efc42200700 1 -- 172.21.15.38:0/2588298256 wait complete. 2024-03-24T00:45:54.111 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-03-24T00:45:54.558 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:45:55.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:54 smithi038 ceph-mon[29271]: pgmap v258: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-24T00:45:55.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:54 smithi088 ceph-mon[36610]: pgmap v258: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-24T00:45:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.836+0000 7fe5f6400700 1 -- 172.21.15.38:0/675210048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5f0100ae0 msgr2=0x7fe5f0100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.836+0000 7fe5f6400700 1 --2- 172.21.15.38:0/675210048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5f0100ae0 0x7fe5f0100f00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000b0a0 tx=0x7fe5e000b3b0 comp rx=0 tx=0).stop 2024-03-24T00:45:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.836+0000 7fe5f6400700 1 -- 172.21.15.38:0/675210048 shutdown_connections 2024-03-24T00:45:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.836+0000 7fe5f6400700 1 --2- 172.21.15.38:0/675210048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f0101ce0 0x7fe5f0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.836+0000 7fe5f6400700 1 --2- 172.21.15.38:0/675210048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5f0100ae0 0x7fe5f0100f00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.836+0000 7fe5f6400700 1 -- 172.21.15.38:0/675210048 >> 172.21.15.38:0/675210048 conn(0x7fe5f00fc0a0 msgr2=0x7fe5f00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:45:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.836+0000 7fe5f6400700 1 -- 172.21.15.38:0/675210048 shutdown_connections 2024-03-24T00:45:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.836+0000 7fe5f6400700 1 -- 172.21.15.38:0/675210048 wait complete. 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.837+0000 7fe5f6400700 1 Processor -- start 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.838+0000 7fe5f6400700 1 -- start start 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.838+0000 7fe5f6400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f0100ae0 0x7fe5f01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.838+0000 7fe5f6400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5f0101ce0 0x7fe5f0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.838+0000 7fe5f6400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0199310 con 0x7fe5f0101ce0 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.838+0000 7fe5f6400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0199450 con 0x7fe5f0100ae0 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.838+0000 7fe5e6a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5f0101ce0 0x7fe5f0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.838+0000 7fe5e6a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5f0101ce0 0x7fe5f0198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42726/0 (socket says 172.21.15.38:42726) 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.838+0000 7fe5e6a00700 1 -- 172.21.15.38:0/2245182693 learned_addr learned my addr 172.21.15.38:0/2245182693 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.839+0000 7fe5ef400700 1 --2- 172.21.15.38:0/2245182693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f0100ae0 0x7fe5f01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.839+0000 7fe5e6a00700 1 -- 172.21.15.38:0/2245182693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f0100ae0 msgr2=0x7fe5f01987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:45:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.839+0000 7fe5e6a00700 1 --2- 172.21.15.38:0/2245182693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f0100ae0 0x7fe5f01987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:45:56.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.839+0000 7fe5e6a00700 1 -- 172.21.15.38:0/2245182693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0009d00 con 0x7fe5f0101ce0 2024-03-24T00:45:56.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.839+0000 7fe5e6a00700 1 --2- 172.21.15.38:0/2245182693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5f0101ce0 0x7fe5f0198cf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800e920 tx=0x7fe5d800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:56.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.840+0000 7fe5ecc00700 1 -- 172.21.15.38:0/2245182693 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5d8009790 con 0x7fe5f0101ce0 2024-03-24T00:45:56.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.840+0000 7fe5f6400700 1 -- 172.21.15.38:0/2245182693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f019de60 con 0x7fe5f0101ce0 2024-03-24T00:45:56.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.840+0000 7fe5f6400700 1 -- 172.21.15.38:0/2245182693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f019e400 con 0x7fe5f0101ce0 2024-03-24T00:45:56.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.840+0000 7fe5ecc00700 1 -- 172.21.15.38:0/2245182693 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5d800cba0 con 0x7fe5f0101ce0 2024-03-24T00:45:56.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.840+0000 7fe5ecc00700 1 -- 172.21.15.38:0/2245182693 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5d8018670 con 0x7fe5f0101ce0 2024-03-24T00:45:56.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.841+0000 7fe5f6400700 1 -- 172.21.15.38:0/2245182693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5cc005320 con 0x7fe5f0101ce0 2024-03-24T00:45:56.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.842+0000 7fe5ecc00700 1 -- 172.21.15.38:0/2245182693 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe5d800cd10 con 0x7fe5f0101ce0 2024-03-24T00:45:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.843+0000 7fe5ecc00700 1 --2- 172.21.15.38:0/2245182693 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe5d406ccf0 0x7fe5d406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:45:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.843+0000 7fe5ecc00700 1 -- 172.21.15.38:0/2245182693 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe5d8014070 con 0x7fe5f0101ce0 2024-03-24T00:45:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.844+0000 7fe5ef400700 1 --2- 172.21.15.38:0/2245182693 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe5d406ccf0 0x7fe5d406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:45:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.844+0000 7fe5ef400700 1 --2- 172.21.15.38:0/2245182693 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe5d406ccf0 0x7fe5d406f1b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009fd0 tx=0x7fe5e000b700 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:45:56.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:56.848+0000 7fe5ecc00700 1 -- 172.21.15.38:0/2245182693 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe5d80529f0 con 0x7fe5f0101ce0 2024-03-24T00:45:57.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:45:57.093+0000 7fe5f6400700 1 -- 172.21.15.38:0/2245182693 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5cc000ca0 con 0x7fe5d406ccf0 2024-03-24T00:45:57.094 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:56 smithi038 ceph-mon[29271]: pgmap v259: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 85 B/s wr, 3 op/s 2024-03-24T00:45:57.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:56 smithi088 ceph-mon[36610]: pgmap v259: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 85 B/s wr, 3 op/s 2024-03-24T00:45:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:57 smithi038 ceph-mon[29271]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:45:58.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:57 smithi088 ceph-mon[36610]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:45:59.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:59 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:45:59.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:45:59 smithi088 ceph-mon[36610]: pgmap v260: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-03-24T00:46:00.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:59 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:46:00.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:45:59 smithi038 ceph-mon[29271]: pgmap v260: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-03-24T00:46:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:00 smithi038 ceph-mon[29271]: pgmap v261: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:46:01.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:00 smithi088 ceph-mon[36610]: pgmap v261: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:46:03.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:02 smithi038 ceph-mon[29271]: pgmap v262: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:46:03.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:02 smithi088 ceph-mon[36610]: pgmap v262: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:46:05.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:04 smithi038 ceph-mon[29271]: pgmap v263: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:04 smithi088 ceph-mon[36610]: pgmap v263: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:06 smithi038 ceph-mon[29271]: pgmap v264: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:46:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:06 smithi088 ceph-mon[36610]: pgmap v264: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:46:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:09 smithi038 ceph-mon[29271]: pgmap v265: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:09 smithi088 ceph-mon[36610]: pgmap v265: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:11.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:10 smithi038 ceph-mon[29271]: pgmap v266: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:11.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:10 smithi088 ceph-mon[36610]: pgmap v266: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:12 smithi038 ceph-mon[29271]: pgmap v267: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:46:13.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:12 smithi088 ceph-mon[36610]: pgmap v267: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:46:14.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:46:14.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:46:15.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:14 smithi038 ceph-mon[29271]: pgmap v268: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:15.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:14 smithi088 ceph-mon[36610]: pgmap v268: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:17.078 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:16 smithi088 ceph-mon[36610]: pgmap v269: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:46:17.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:16 smithi038 ceph-mon[29271]: pgmap v269: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:46:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:19 smithi038 ceph-mon[29271]: pgmap v270: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:20.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:19 smithi088 ceph-mon[36610]: pgmap v270: 65 pgs: 65 active+clean; 586 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:21.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:20 smithi038 ceph-mon[29271]: pgmap v271: 65 pgs: 65 active+clean; 587 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-03-24T00:46:21.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:20 smithi088 ceph-mon[36610]: pgmap v271: 65 pgs: 65 active+clean; 587 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-03-24T00:46:23.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:22 smithi038 ceph-mon[29271]: pgmap v272: 65 pgs: 65 active+clean; 587 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-24T00:46:23.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:22 smithi088 ceph-mon[36610]: pgmap v272: 65 pgs: 65 active+clean; 587 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-24T00:46:25.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:24 smithi038 ceph-mon[29271]: pgmap v273: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-24T00:46:25.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:24 smithi088 ceph-mon[36610]: pgmap v273: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-24T00:46:27.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:26 smithi038 ceph-mon[29271]: pgmap v274: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T00:46:27.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:26 smithi088 ceph-mon[36610]: pgmap v274: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T00:46:29.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:46:29.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:46:30.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:29 smithi038 ceph-mon[29271]: pgmap v275: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-24T00:46:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:29 smithi088 ceph-mon[36610]: pgmap v275: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-24T00:46:31.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:30 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:46:31.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:30 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:46:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:31 smithi038 ceph-mon[29271]: pgmap v276: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-24T00:46:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:31 smithi088 ceph-mon[36610]: pgmap v276: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-24T00:46:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:32 smithi038 ceph-mon[29271]: pgmap v277: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-03-24T00:46:33.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:32 smithi088 ceph-mon[36610]: pgmap v277: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-03-24T00:46:35.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:34 smithi038 ceph-mon[29271]: pgmap v278: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-24T00:46:35.182 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:34 smithi088 ceph-mon[36610]: pgmap v278: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-24T00:46:37.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:36 smithi038 ceph-mon[29271]: pgmap v279: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-03-24T00:46:37.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:36 smithi088 ceph-mon[36610]: pgmap v279: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-03-24T00:46:39.847 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:39 smithi038 ceph-mon[29271]: pgmap v280: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-03-24T00:46:39.847 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:46:39.847 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:46:39.847 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:39 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:39.982 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:39 smithi088 ceph-mon[36610]: pgmap v280: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-03-24T00:46:39.982 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:39 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:46:39.982 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:39 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:46:39.982 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:39 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:40.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.778+0000 7fe5ecc00700 1 -- 172.21.15.38:0/2245182693 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fe5cc000ca0 con 0x7fe5d406ccf0 2024-03-24T00:46:40.779 INFO:teuthology.orchestra.run.smithi038.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:46:40.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.782+0000 7fe5f6400700 1 -- 172.21.15.38:0/2245182693 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe5d406ccf0 msgr2=0x7fe5d406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:40.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.782+0000 7fe5f6400700 1 --2- 172.21.15.38:0/2245182693 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe5d406ccf0 0x7fe5d406f1b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009fd0 tx=0x7fe5e000b700 comp rx=0 tx=0).stop 2024-03-24T00:46:40.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.782+0000 7fe5f6400700 1 -- 172.21.15.38:0/2245182693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5f0101ce0 msgr2=0x7fe5f0198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:40.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.782+0000 7fe5f6400700 1 --2- 172.21.15.38:0/2245182693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5f0101ce0 0x7fe5f0198cf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800e920 tx=0x7fe5d800ece0 comp rx=0 tx=0).stop 2024-03-24T00:46:40.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.782+0000 7fe5f6400700 1 -- 172.21.15.38:0/2245182693 shutdown_connections 2024-03-24T00:46:40.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.782+0000 7fe5f6400700 1 --2- 172.21.15.38:0/2245182693 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fe5d406ccf0 0x7fe5d406f1b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:40.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.782+0000 7fe5f6400700 1 --2- 172.21.15.38:0/2245182693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f0100ae0 0x7fe5f01987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:40.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.782+0000 7fe5f6400700 1 --2- 172.21.15.38:0/2245182693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5f0101ce0 0x7fe5f0198cf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:40.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.782+0000 7fe5f6400700 1 -- 172.21.15.38:0/2245182693 >> 172.21.15.38:0/2245182693 conn(0x7fe5f00fc0a0 msgr2=0x7fe5f0073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:46:40.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.782+0000 7fe5f6400700 1 -- 172.21.15.38:0/2245182693 shutdown_connections 2024-03-24T00:46:40.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:40.783+0000 7fe5f6400700 1 -- 172.21.15.38:0/2245182693 wait complete. 2024-03-24T00:46:41.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:40 smithi038 ceph-mon[29271]: pgmap v281: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:41.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:40 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:41.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:40 smithi088 ceph-mon[36610]: pgmap v281: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:41.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:40 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:41.522 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-03-24T00:46:42.034 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:46:42.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:41 smithi038 ceph-mon[29271]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:46:42.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:41 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:46:42.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:41 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:42.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:41 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:42.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:41 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:46:42.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:41 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:46:42.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:41 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:42.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:41 smithi088 ceph-mon[36610]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:46:42.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:41 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:46:42.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:41 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:42.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:41 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:42.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:41 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:46:42.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:41 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:46:42.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:41 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:42 smithi038 ceph-mon[29271]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:46:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:42 smithi038 ceph-mon[29271]: pgmap v282: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:46:43.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:42 smithi088 ceph-mon[36610]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:46:43.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:42 smithi088 ceph-mon[36610]: pgmap v282: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:46:44.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:43 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:46:44.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:43 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:46:45.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:44 smithi038 ceph-mon[29271]: pgmap v283: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:45.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:44 smithi088 ceph-mon[36610]: pgmap v283: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:47.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:46 smithi038 ceph-mon[29271]: pgmap v284: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:46:47.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:46 smithi088 ceph-mon[36610]: pgmap v284: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:46:49.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:49 smithi038 ceph-mon[29271]: pgmap v285: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:49 smithi088 ceph-mon[36610]: pgmap v285: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:50.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.921+0000 7fbc46400700 1 -- 172.21.15.38:0/97697988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40103ef0 msgr2=0x7fbc40104350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:50.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.921+0000 7fbc46400700 1 --2- 172.21.15.38:0/97697988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40103ef0 0x7fbc40104350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009910 tx=0x7fbc2c009c20 comp rx=0 tx=0).stop 2024-03-24T00:46:50.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.921+0000 7fbc46400700 1 -- 172.21.15.38:0/97697988 shutdown_connections 2024-03-24T00:46:50.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.921+0000 7fbc46400700 1 --2- 172.21.15.38:0/97697988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40103ef0 0x7fbc40104350 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:50.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.921+0000 7fbc46400700 1 --2- 172.21.15.38:0/97697988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc40102cf0 0x7fbc40103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:50.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.921+0000 7fbc46400700 1 -- 172.21.15.38:0/97697988 >> 172.21.15.38:0/97697988 conn(0x7fbc400fe2b0 msgr2=0x7fbc401006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:46:50.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.922+0000 7fbc46400700 1 -- 172.21.15.38:0/97697988 shutdown_connections 2024-03-24T00:46:50.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.922+0000 7fbc46400700 1 -- 172.21.15.38:0/97697988 wait complete. 2024-03-24T00:46:50.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.924+0000 7fbc46400700 1 Processor -- start 2024-03-24T00:46:50.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.924+0000 7fbc46400700 1 -- start start 2024-03-24T00:46:50.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.924+0000 7fbc46400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc40102cf0 0x7fbc40194320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:50.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.924+0000 7fbc46400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40103ef0 0x7fbc40194860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:50.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.924+0000 7fbc46400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc40194e30 con 0x7fbc40103ef0 2024-03-24T00:46:50.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.924+0000 7fbc46400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc40194f70 con 0x7fbc40102cf0 2024-03-24T00:46:50.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.925+0000 7fbc3ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40103ef0 0x7fbc40194860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:50.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.925+0000 7fbc3ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40103ef0 0x7fbc40194860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38580/0 (socket says 172.21.15.38:38580) 2024-03-24T00:46:50.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.925+0000 7fbc3ea00700 1 -- 172.21.15.38:0/2554079286 learned_addr learned my addr 172.21.15.38:0/2554079286 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:46:50.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.925+0000 7fbc3f400700 1 --2- 172.21.15.38:0/2554079286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc40102cf0 0x7fbc40194320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:50.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.925+0000 7fbc3ea00700 1 -- 172.21.15.38:0/2554079286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc40102cf0 msgr2=0x7fbc40194320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:50.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.925+0000 7fbc3ea00700 1 --2- 172.21.15.38:0/2554079286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc40102cf0 0x7fbc40194320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:50.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.925+0000 7fbc3ea00700 1 -- 172.21.15.38:0/2554079286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c0095f0 con 0x7fbc40103ef0 2024-03-24T00:46:50.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.926+0000 7fbc3ea00700 1 --2- 172.21.15.38:0/2554079286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40103ef0 0x7fbc40194860 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00b650 tx=0x7fbc2c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:46:50.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.926+0000 7fbc37e00700 1 -- 172.21.15.38:0/2554079286 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2c01d070 con 0x7fbc40103ef0 2024-03-24T00:46:50.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.926+0000 7fbc37e00700 1 -- 172.21.15.38:0/2554079286 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc2c00fe70 con 0x7fbc40103ef0 2024-03-24T00:46:50.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.926+0000 7fbc46400700 1 -- 172.21.15.38:0/2554079286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc401aa9b0 con 0x7fbc40103ef0 2024-03-24T00:46:50.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.926+0000 7fbc37e00700 1 -- 172.21.15.38:0/2554079286 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2c017630 con 0x7fbc40103ef0 2024-03-24T00:46:50.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.926+0000 7fbc46400700 1 -- 172.21.15.38:0/2554079286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc401aae90 con 0x7fbc40103ef0 2024-03-24T00:46:50.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.928+0000 7fbc46400700 1 -- 172.21.15.38:0/2554079286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc40076990 con 0x7fbc40103ef0 2024-03-24T00:46:50.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.931+0000 7fbc37e00700 1 -- 172.21.15.38:0/2554079286 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbc2c003810 con 0x7fbc40103ef0 2024-03-24T00:46:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.932+0000 7fbc37e00700 1 --2- 172.21.15.38:0/2554079286 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbc2406cd30 0x7fbc2406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.932+0000 7fbc37e00700 1 -- 172.21.15.38:0/2554079286 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbc2c08db20 con 0x7fbc40103ef0 2024-03-24T00:46:50.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.933+0000 7fbc3f400700 1 --2- 172.21.15.38:0/2554079286 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbc2406cd30 0x7fbc2406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.934+0000 7fbc3f400700 1 --2- 172.21.15.38:0/2554079286 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbc2406cd30 0x7fbc2406f1f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc40103d50 tx=0x7fbc30009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:46:50.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:50.939+0000 7fbc37e00700 1 -- 172.21.15.38:0/2554079286 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbc2c05b980 con 0x7fbc40103ef0 2024-03-24T00:46:51.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:50 smithi038 ceph-mon[29271]: pgmap v286: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:50 smithi088 ceph-mon[36610]: pgmap v286: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:51.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.413+0000 7fbc46400700 1 -- 172.21.15.38:0/2554079286 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc400611f0 con 0x7fbc2406cd30 2024-03-24T00:46:51.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.415+0000 7fbc37e00700 1 -- 172.21.15.38:0/2554079286 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fbc400611f0 con 0x7fbc2406cd30 2024-03-24T00:46:51.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.418+0000 7fbc46400700 1 -- 172.21.15.38:0/2554079286 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbc2406cd30 msgr2=0x7fbc2406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:51.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.418+0000 7fbc46400700 1 --2- 172.21.15.38:0/2554079286 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbc2406cd30 0x7fbc2406f1f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc40103d50 tx=0x7fbc30009260 comp rx=0 tx=0).stop 2024-03-24T00:46:51.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.418+0000 7fbc46400700 1 -- 172.21.15.38:0/2554079286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40103ef0 msgr2=0x7fbc40194860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:51.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.418+0000 7fbc46400700 1 --2- 172.21.15.38:0/2554079286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40103ef0 0x7fbc40194860 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00b650 tx=0x7fbc2c00f510 comp rx=0 tx=0).stop 2024-03-24T00:46:51.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.419+0000 7fbc46400700 1 -- 172.21.15.38:0/2554079286 shutdown_connections 2024-03-24T00:46:51.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.419+0000 7fbc46400700 1 --2- 172.21.15.38:0/2554079286 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fbc2406cd30 0x7fbc2406f1f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:51.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.419+0000 7fbc46400700 1 --2- 172.21.15.38:0/2554079286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc40102cf0 0x7fbc40194320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:51.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.419+0000 7fbc46400700 1 --2- 172.21.15.38:0/2554079286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40103ef0 0x7fbc40194860 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:51.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.419+0000 7fbc46400700 1 -- 172.21.15.38:0/2554079286 >> 172.21.15.38:0/2554079286 conn(0x7fbc400fe2b0 msgr2=0x7fbc401074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:46:51.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.419+0000 7fbc46400700 1 -- 172.21.15.38:0/2554079286 shutdown_connections 2024-03-24T00:46:51.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.419+0000 7fbc46400700 1 -- 172.21.15.38:0/2554079286 wait complete. 2024-03-24T00:46:51.441 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:46:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.600+0000 7f326da00700 1 -- 172.21.15.38:0/3551108994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32680ff9d0 msgr2=0x7f32680ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.600+0000 7f326da00700 1 --2- 172.21.15.38:0/3551108994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32680ff9d0 0x7f32680ffdf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3258009910 tx=0x7f3258009c20 comp rx=0 tx=0).stop 2024-03-24T00:46:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.601+0000 7f326da00700 1 -- 172.21.15.38:0/3551108994 shutdown_connections 2024-03-24T00:46:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.601+0000 7f326da00700 1 --2- 172.21.15.38:0/3551108994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3268100bd0 0x7f3268101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.601+0000 7f326da00700 1 --2- 172.21.15.38:0/3551108994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32680ff9d0 0x7f32680ffdf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.601+0000 7f326da00700 1 -- 172.21.15.38:0/3551108994 >> 172.21.15.38:0/3551108994 conn(0x7f32680faf70 msgr2=0x7f32680fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:46:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.601+0000 7f326da00700 1 -- 172.21.15.38:0/3551108994 shutdown_connections 2024-03-24T00:46:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.602+0000 7f326da00700 1 -- 172.21.15.38:0/3551108994 wait complete. 2024-03-24T00:46:51.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.603+0000 7f326da00700 1 Processor -- start 2024-03-24T00:46:51.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.604+0000 7f326da00700 1 -- start start 2024-03-24T00:46:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.604+0000 7f326da00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32680ff9d0 0x7f32681942e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.604+0000 7f326da00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3268100bd0 0x7f3268194820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.605+0000 7f326da00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268194e40 con 0x7f3268100bd0 2024-03-24T00:46:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.605+0000 7f326da00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268194f80 con 0x7f32680ff9d0 2024-03-24T00:46:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.605+0000 7f3266a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32680ff9d0 0x7f32681942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.605+0000 7f3266a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32680ff9d0 0x7f32681942e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:46750/0 (socket says 172.21.15.38:46750) 2024-03-24T00:46:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.605+0000 7f3266a00700 1 -- 172.21.15.38:0/3765648545 learned_addr learned my addr 172.21.15.38:0/3765648545 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:46:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.605+0000 7f3266000700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3268100bd0 0x7f3268194820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.606+0000 7f3266a00700 1 -- 172.21.15.38:0/3765648545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3268100bd0 msgr2=0x7f3268194820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.606+0000 7f3266a00700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3268100bd0 0x7f3268194820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.606+0000 7f3266a00700 1 -- 172.21.15.38:0/3765648545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32580095f0 con 0x7f32680ff9d0 2024-03-24T00:46:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.606+0000 7f3266000700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3268100bd0 0x7f3268194820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T00:46:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.606+0000 7f3266a00700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32680ff9d0 0x7f32681942e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f325800b800 tx=0x7f3258005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:46:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.607+0000 7f325f400700 1 -- 172.21.15.38:0/3765648545 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f325801d070 con 0x7f32680ff9d0 2024-03-24T00:46:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.607+0000 7f325f400700 1 -- 172.21.15.38:0/3765648545 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f325800fdd0 con 0x7f32680ff9d0 2024-03-24T00:46:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.607+0000 7f325f400700 1 -- 172.21.15.38:0/3765648545 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3258017400 con 0x7f32680ff9d0 2024-03-24T00:46:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.607+0000 7f326da00700 1 -- 172.21.15.38:0/3765648545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3268199980 con 0x7f32680ff9d0 2024-03-24T00:46:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.608+0000 7f326da00700 1 -- 172.21.15.38:0/3765648545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3268199ef0 con 0x7f32680ff9d0 2024-03-24T00:46:51.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.609+0000 7f326da00700 1 -- 172.21.15.38:0/3765648545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326818e440 con 0x7f32680ff9d0 2024-03-24T00:46:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.611+0000 7f325f400700 1 -- 172.21.15.38:0/3765648545 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3258003810 con 0x7f32680ff9d0 2024-03-24T00:46:51.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.612+0000 7f325f400700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f324c06ccf0 0x7f324c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:51.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.612+0000 7f325f400700 1 -- 172.21.15.38:0/3765648545 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f325808d480 con 0x7f32680ff9d0 2024-03-24T00:46:51.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.612+0000 7f3266000700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f324c06ccf0 0x7f324c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:51.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.613+0000 7f3266000700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f324c06ccf0 0x7f324c06f1b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f325400bfd0 tx=0x7f325400d040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:46:51.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.622+0000 7f325f400700 1 -- 172.21.15.38:0/3765648545 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f325805b300 con 0x7f32680ff9d0 2024-03-24T00:46:51.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.874+0000 7f326da00700 1 -- 172.21.15.38:0/3765648545 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32680611f0 con 0x7f324c06ccf0 2024-03-24T00:46:51.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.876+0000 7f325f400700 1 -- 172.21.15.38:0/3765648545 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f32680611f0 con 0x7f324c06ccf0 2024-03-24T00:46:51.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.879+0000 7f326da00700 1 -- 172.21.15.38:0/3765648545 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f324c06ccf0 msgr2=0x7f324c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:51.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.879+0000 7f326da00700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f324c06ccf0 0x7f324c06f1b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f325400bfd0 tx=0x7f325400d040 comp rx=0 tx=0).stop 2024-03-24T00:46:51.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.879+0000 7f326da00700 1 -- 172.21.15.38:0/3765648545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32680ff9d0 msgr2=0x7f32681942e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:51.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.879+0000 7f326da00700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32680ff9d0 0x7f32681942e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f325800b800 tx=0x7f3258005f00 comp rx=0 tx=0).stop 2024-03-24T00:46:51.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.879+0000 7f326da00700 1 -- 172.21.15.38:0/3765648545 shutdown_connections 2024-03-24T00:46:51.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.879+0000 7f326da00700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f324c06ccf0 0x7f324c06f1b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:51.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.879+0000 7f326da00700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32680ff9d0 0x7f32681942e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:51.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.879+0000 7f326da00700 1 --2- 172.21.15.38:0/3765648545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3268100bd0 0x7f3268194820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:51.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.879+0000 7f326da00700 1 -- 172.21.15.38:0/3765648545 >> 172.21.15.38:0/3765648545 conn(0x7f32680faf70 msgr2=0x7f3268104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:46:51.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.880+0000 7f326da00700 1 -- 172.21.15.38:0/3765648545 shutdown_connections 2024-03-24T00:46:51.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:51.880+0000 7f326da00700 1 -- 172.21.15.38:0/3765648545 wait complete. 2024-03-24T00:46:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.035+0000 7f124be00700 1 -- 172.21.15.38:0/1208592001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f124c101b20 msgr2=0x7f124c103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:52.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.035+0000 7f124be00700 1 --2- 172.21.15.38:0/1208592001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f124c101b20 0x7f124c103fb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1238009910 tx=0x7f1238009c20 comp rx=0 tx=0).stop 2024-03-24T00:46:52.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.035+0000 7f124be00700 1 -- 172.21.15.38:0/1208592001 shutdown_connections 2024-03-24T00:46:52.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.035+0000 7f124be00700 1 --2- 172.21.15.38:0/1208592001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f124c101b20 0x7f124c103fb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:52.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.035+0000 7f124be00700 1 --2- 172.21.15.38:0/1208592001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f124c0ff1b0 0x7f124c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:52.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.035+0000 7f124be00700 1 -- 172.21.15.38:0/1208592001 >> 172.21.15.38:0/1208592001 conn(0x7f124c0fadf0 msgr2=0x7f124c0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:46:52.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.035+0000 7f124be00700 1 -- 172.21.15.38:0/1208592001 shutdown_connections 2024-03-24T00:46:52.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.035+0000 7f124be00700 1 -- 172.21.15.38:0/1208592001 wait complete. 2024-03-24T00:46:52.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.037+0000 7f124be00700 1 Processor -- start 2024-03-24T00:46:52.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.037+0000 7f124be00700 1 -- start start 2024-03-24T00:46:52.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.037+0000 7f124be00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f124c0ff1b0 0x7f124c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:52.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.037+0000 7f124be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f124c101b20 0x7f124c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:52.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.037+0000 7f124be00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c19d480 con 0x7f124c101b20 2024-03-24T00:46:52.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.037+0000 7f124be00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c19d5c0 con 0x7f124c0ff1b0 2024-03-24T00:46:52.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.038+0000 7f124aa00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f124c0ff1b0 0x7f124c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:52.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.038+0000 7f124aa00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f124c0ff1b0 0x7f124c19c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:46778/0 (socket says 172.21.15.38:46778) 2024-03-24T00:46:52.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.038+0000 7f124aa00700 1 -- 172.21.15.38:0/1684299328 learned_addr learned my addr 172.21.15.38:0/1684299328 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:46:52.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.038+0000 7f124a000700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f124c101b20 0x7f124c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:52.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.038+0000 7f124aa00700 1 -- 172.21.15.38:0/1684299328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f124c101b20 msgr2=0x7f124c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:52.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.038+0000 7f124aa00700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f124c101b20 0x7f124c19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:52.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.038+0000 7f124aa00700 1 -- 172.21.15.38:0/1684299328 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12380095f0 con 0x7f124c0ff1b0 2024-03-24T00:46:52.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.038+0000 7f124a000700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f124c101b20 0x7f124c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T00:46:52.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.039+0000 7f124aa00700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f124c0ff1b0 0x7f124c19c920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f123c00e920 tx=0x7f123c00ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:46:52.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.039+0000 7f1243400700 1 -- 172.21.15.38:0/1684299328 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123c00c9d0 con 0x7f124c0ff1b0 2024-03-24T00:46:52.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.039+0000 7f1243400700 1 -- 172.21.15.38:0/1684299328 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f123c010430 con 0x7f124c0ff1b0 2024-03-24T00:46:52.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.039+0000 7f1243400700 1 -- 172.21.15.38:0/1684299328 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123c018630 con 0x7f124c0ff1b0 2024-03-24T00:46:52.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.040+0000 7f124be00700 1 -- 172.21.15.38:0/1684299328 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c1a1fc0 con 0x7f124c0ff1b0 2024-03-24T00:46:52.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.040+0000 7f124be00700 1 -- 172.21.15.38:0/1684299328 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c1a2560 con 0x7f124c0ff1b0 2024-03-24T00:46:52.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.041+0000 7f124be00700 1 -- 172.21.15.38:0/1684299328 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124c04f2d0 con 0x7f124c0ff1b0 2024-03-24T00:46:52.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.044+0000 7f1243400700 1 -- 172.21.15.38:0/1684299328 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f123c0105a0 con 0x7f124c0ff1b0 2024-03-24T00:46:52.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.044+0000 7f1243400700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123006ccf0 0x7f123006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:52.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.045+0000 7f1243400700 1 -- 172.21.15.38:0/1684299328 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f123c014070 con 0x7f124c0ff1b0 2024-03-24T00:46:52.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.045+0000 7f124a000700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123006ccf0 0x7f123006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:52.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.046+0000 7f124a000700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123006ccf0 0x7f123006f1b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f123800b6f0 tx=0x7f1238005fb0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:46:52.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.052+0000 7f1243400700 1 -- 172.21.15.38:0/1684299328 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f123c05aa60 con 0x7f124c0ff1b0 2024-03-24T00:46:52.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.296+0000 7f124be00700 1 -- 172.21.15.38:0/1684299328 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f124c1a2810 con 0x7f123006ccf0 2024-03-24T00:46:52.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.310+0000 7f1243400700 1 -- 172.21.15.38:0/1684299328 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f124c1a2810 con 0x7f123006ccf0 2024-03-24T00:46:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:46:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (7m) 83s ago 9m 13.4M - 0.25.0 c8568f914cd2 9f1243fbcb3e 2024-03-24T00:46:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (9m) 83s ago 9m 7658k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:46:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (8m) 88s ago 8m 7482k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:46:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (9m) 83s ago 9m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:46:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (8m) 88s ago 8m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:46:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (7m) 83s ago 8m 72.7M - 9.4.7 954c08fa6188 e6aa268d69e8 2024-03-24T00:46:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (107s) 83s ago 106s 13.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:46:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (116s) 83s ago 116s 15.0M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:46:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (112s) 88s ago 112s 12.8M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:46:52.311 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (103s) 88s ago 102s 14.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:9283,8765,8443 running (11m) 83s ago 10m 498M - 18.2.2-952-g738c2fd6 68aff9aff3b2 bf370183b69d 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (7m) 88s ago 7m 449M - 18.2.2-952-g738c2fd6 68aff9aff3b2 acebd34a44e9 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (11m) 83s ago 11m 45.0M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (7m) 88s ago 7m 33.9M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (9m) 83s ago 9m 8108k - 1.5.0 0da6a335fe13 c34cc077c93f 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (7m) 88s ago 7m 7884k - 1.5.0 0da6a335fe13 189e840ee2f1 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (6m) 83s ago 6m 46.7M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (6m) 83s ago 6m 47.1M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (5m) 83s ago 5m 45.5M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (4m) 88s ago 4m 45.5M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (4m) 88s ago 4m 43.8M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (3m) 88s ago 3m 42.7M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:46:52.312 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (6m) 83s ago 8m 49.7M - 2.43.0 a07b618ecd1d c59e5c711ace 2024-03-24T00:46:52.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.313+0000 7f124be00700 1 -- 172.21.15.38:0/1684299328 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123006ccf0 msgr2=0x7f123006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:52.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.313+0000 7f124be00700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123006ccf0 0x7f123006f1b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f123800b6f0 tx=0x7f1238005fb0 comp rx=0 tx=0).stop 2024-03-24T00:46:52.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.313+0000 7f124be00700 1 -- 172.21.15.38:0/1684299328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f124c0ff1b0 msgr2=0x7f124c19c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:52.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.313+0000 7f124be00700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f124c0ff1b0 0x7f124c19c920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f123c00e920 tx=0x7f123c00ec30 comp rx=0 tx=0).stop 2024-03-24T00:46:52.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.313+0000 7f124be00700 1 -- 172.21.15.38:0/1684299328 shutdown_connections 2024-03-24T00:46:52.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.313+0000 7f124be00700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f123006ccf0 0x7f123006f1b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:52.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.313+0000 7f124be00700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f124c0ff1b0 0x7f124c19c920 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:52.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.313+0000 7f124be00700 1 --2- 172.21.15.38:0/1684299328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f124c101b20 0x7f124c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:52.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.313+0000 7f124be00700 1 -- 172.21.15.38:0/1684299328 >> 172.21.15.38:0/1684299328 conn(0x7f124c0fadf0 msgr2=0x7f124c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:46:52.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.314+0000 7f124be00700 1 -- 172.21.15.38:0/1684299328 shutdown_connections 2024-03-24T00:46:52.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.314+0000 7f124be00700 1 -- 172.21.15.38:0/1684299328 wait complete. 2024-03-24T00:46:52.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.454+0000 7ff729c00700 1 -- 172.21.15.38:0/2588370314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7240ff350 msgr2=0x7ff724101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:52.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.454+0000 7ff729c00700 1 --2- 172.21.15.38:0/2588370314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7240ff350 0x7ff724101780 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff714009910 tx=0x7ff714009c20 comp rx=0 tx=0).stop 2024-03-24T00:46:52.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.455+0000 7ff729c00700 1 -- 172.21.15.38:0/2588370314 shutdown_connections 2024-03-24T00:46:52.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.455+0000 7ff729c00700 1 --2- 172.21.15.38:0/2588370314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff724101cc0 0x7ff724104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:52.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.455+0000 7ff729c00700 1 --2- 172.21.15.38:0/2588370314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7240ff350 0x7ff724101780 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:52.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.455+0000 7ff729c00700 1 -- 172.21.15.38:0/2588370314 >> 172.21.15.38:0/2588370314 conn(0x7ff7240faf70 msgr2=0x7ff7240fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:46:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.455+0000 7ff729c00700 1 -- 172.21.15.38:0/2588370314 shutdown_connections 2024-03-24T00:46:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.455+0000 7ff729c00700 1 -- 172.21.15.38:0/2588370314 wait complete. 2024-03-24T00:46:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.456+0000 7ff729c00700 1 Processor -- start 2024-03-24T00:46:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.456+0000 7ff729c00700 1 -- start start 2024-03-24T00:46:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.457+0000 7ff729c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240ff350 0x7ff72419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.457+0000 7ff729c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff724101cc0 0x7ff72419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.457+0000 7ff729c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72419d5e0 con 0x7ff724101cc0 2024-03-24T00:46:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.457+0000 7ff729c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72419d720 con 0x7ff7240ff350 2024-03-24T00:46:52.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.457+0000 7ff722000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff724101cc0 0x7ff72419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:52.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.457+0000 7ff722000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff724101cc0 0x7ff72419cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38652/0 (socket says 172.21.15.38:38652) 2024-03-24T00:46:52.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.457+0000 7ff722000700 1 -- 172.21.15.38:0/1676738204 learned_addr learned my addr 172.21.15.38:0/1676738204 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:46:52.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.457+0000 7ff722a00700 1 --2- 172.21.15.38:0/1676738204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240ff350 0x7ff72419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:52.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.458+0000 7ff722000700 1 -- 172.21.15.38:0/1676738204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240ff350 msgr2=0x7ff72419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:52.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.458+0000 7ff722000700 1 --2- 172.21.15.38:0/1676738204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240ff350 0x7ff72419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:52.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.458+0000 7ff722000700 1 -- 172.21.15.38:0/1676738204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7140095f0 con 0x7ff724101cc0 2024-03-24T00:46:52.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.458+0000 7ff722000700 1 --2- 172.21.15.38:0/1676738204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff724101cc0 0x7ff72419cfc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff71000e920 tx=0x7ff71000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:46:52.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.459+0000 7ff71b400700 1 -- 172.21.15.38:0/1676738204 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff71000c9d0 con 0x7ff724101cc0 2024-03-24T00:46:52.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.459+0000 7ff71b400700 1 -- 172.21.15.38:0/1676738204 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff710010430 con 0x7ff724101cc0 2024-03-24T00:46:52.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.459+0000 7ff71b400700 1 -- 172.21.15.38:0/1676738204 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff710018650 con 0x7ff724101cc0 2024-03-24T00:46:52.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.459+0000 7ff729c00700 1 -- 172.21.15.38:0/1676738204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7241a2180 con 0x7ff724101cc0 2024-03-24T00:46:52.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.460+0000 7ff729c00700 1 -- 172.21.15.38:0/1676738204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7241a2720 con 0x7ff724101cc0 2024-03-24T00:46:52.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.462+0000 7ff718c00700 1 -- 172.21.15.38:0/1676738204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72404ea80 con 0x7ff724101cc0 2024-03-24T00:46:52.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.463+0000 7ff71b400700 1 -- 172.21.15.38:0/1676738204 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff7100105a0 con 0x7ff724101cc0 2024-03-24T00:46:52.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.464+0000 7ff71b400700 1 --2- 172.21.15.38:0/1676738204 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7ff70806ccf0 0x7ff70806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:52.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.464+0000 7ff71b400700 1 -- 172.21.15.38:0/1676738204 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff710014070 con 0x7ff724101cc0 2024-03-24T00:46:52.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.465+0000 7ff722a00700 1 --2- 172.21.15.38:0/1676738204 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7ff70806ccf0 0x7ff70806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:52.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.466+0000 7ff722a00700 1 --2- 172.21.15.38:0/1676738204 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7ff70806ccf0 0x7ff70806f1b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff71400b6f0 tx=0x7ff714005fb0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:46:52.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:52.473+0000 7ff71b400700 1 -- 172.21.15.38:0/1676738204 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff71005a540 con 0x7ff724101cc0 2024-03-24T00:46:52.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:52 smithi088 ceph-mon[36610]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:46:52.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:52.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:52 smithi088 ceph-mon[36610]: Upgrade: Target is version 19.0.0-2145-g3be9a5c9 (unknown) 2024-03-24T00:46:52.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:52 smithi088 ceph-mon[36610]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc'] 2024-03-24T00:46:52.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:46:52.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:52 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:52.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:52 smithi088 ceph-mon[36610]: Upgrade: Need to upgrade myself (mgr.smithi038.huhioz) 2024-03-24T00:46:53.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.078+0000 7ff718c00700 1 -- 172.21.15.38:0/1676738204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff7240619c0 con 0x7ff724101cc0 2024-03-24T00:46:53.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.079+0000 7ff71b400700 1 -- 172.21.15.38:0/1676738204 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7ff71005a0d0 con 0x7ff724101cc0 2024-03-24T00:46:53.079 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:52 smithi038 ceph-mon[29271]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:46:53.079 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:53.079 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:52 smithi038 ceph-mon[29271]: Upgrade: Target is version 19.0.0-2145-g3be9a5c9 (unknown) 2024-03-24T00:46:53.079 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:52 smithi038 ceph-mon[29271]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc'] 2024-03-24T00:46:53.079 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:46:53.079 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:52 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:46:53.079 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:52 smithi038 ceph-mon[29271]: Upgrade: Need to upgrade myself (mgr.smithi038.huhioz) 2024-03-24T00:46:53.079 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:46:53.079 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 14 2024-03-24T00:46:53.080 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:46:53.081 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:46:53.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.082+0000 7ff718c00700 1 -- 172.21.15.38:0/1676738204 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7ff70806ccf0 msgr2=0x7ff70806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:53.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.082+0000 7ff718c00700 1 --2- 172.21.15.38:0/1676738204 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7ff70806ccf0 0x7ff70806f1b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff71400b6f0 tx=0x7ff714005fb0 comp rx=0 tx=0).stop 2024-03-24T00:46:53.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.082+0000 7ff718c00700 1 -- 172.21.15.38:0/1676738204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff724101cc0 msgr2=0x7ff72419cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:53.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.082+0000 7ff718c00700 1 --2- 172.21.15.38:0/1676738204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff724101cc0 0x7ff72419cfc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff71000e920 tx=0x7ff71000ec30 comp rx=0 tx=0).stop 2024-03-24T00:46:53.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.082+0000 7ff718c00700 1 -- 172.21.15.38:0/1676738204 shutdown_connections 2024-03-24T00:46:53.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.082+0000 7ff718c00700 1 --2- 172.21.15.38:0/1676738204 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7ff70806ccf0 0x7ff70806f1b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:53.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.082+0000 7ff718c00700 1 --2- 172.21.15.38:0/1676738204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240ff350 0x7ff72419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:53.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.082+0000 7ff718c00700 1 --2- 172.21.15.38:0/1676738204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff724101cc0 0x7ff72419cfc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:53.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.082+0000 7ff718c00700 1 -- 172.21.15.38:0/1676738204 >> 172.21.15.38:0/1676738204 conn(0x7ff7240faf70 msgr2=0x7ff7240fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:46:53.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.082+0000 7ff718c00700 1 -- 172.21.15.38:0/1676738204 shutdown_connections 2024-03-24T00:46:53.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.083+0000 7ff718c00700 1 -- 172.21.15.38:0/1676738204 wait complete. 2024-03-24T00:46:53.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.241+0000 7f717ce00700 1 -- 172.21.15.38:0/3075872090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7178101b30 msgr2=0x7f7178101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.241+0000 7f717ce00700 1 --2- 172.21.15.38:0/3075872090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7178101b30 0x7f7178101f90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7164009910 tx=0x7f7164009c20 comp rx=0 tx=0).stop 2024-03-24T00:46:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.241+0000 7f717ce00700 1 -- 172.21.15.38:0/3075872090 shutdown_connections 2024-03-24T00:46:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.241+0000 7f717ce00700 1 --2- 172.21.15.38:0/3075872090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7178101b30 0x7f7178101f90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.241+0000 7f717ce00700 1 --2- 172.21.15.38:0/3075872090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7178100930 0x7f7178100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.241+0000 7f717ce00700 1 -- 172.21.15.38:0/3075872090 >> 172.21.15.38:0/3075872090 conn(0x7f71780fbed0 msgr2=0x7f71780fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:46:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.241+0000 7f717ce00700 1 -- 172.21.15.38:0/3075872090 shutdown_connections 2024-03-24T00:46:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.241+0000 7f717ce00700 1 -- 172.21.15.38:0/3075872090 wait complete. 2024-03-24T00:46:53.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.242+0000 7f717ce00700 1 Processor -- start 2024-03-24T00:46:53.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.243+0000 7f717ce00700 1 -- start start 2024-03-24T00:46:53.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.243+0000 7f717ce00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7178100930 0x7f7178074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:53.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.243+0000 7f717ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7178101b30 0x7f71780731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:53.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.243+0000 7f717ce00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71780736e0 con 0x7f7178101b30 2024-03-24T00:46:53.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.243+0000 7f717ce00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7178073820 con 0x7f7178100930 2024-03-24T00:46:53.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.243+0000 7f7177400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7178100930 0x7f7178074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:53.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.243+0000 7f7176a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7178101b30 0x7f71780731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:53.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.243+0000 7f7177400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7178100930 0x7f7178074ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:56664/0 (socket says 172.21.15.38:56664) 2024-03-24T00:46:53.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.243+0000 7f7177400700 1 -- 172.21.15.38:0/629480011 learned_addr learned my addr 172.21.15.38:0/629480011 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:46:53.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.244+0000 7f7176a00700 1 -- 172.21.15.38:0/629480011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7178100930 msgr2=0x7f7178074ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:53.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.244+0000 7f7176a00700 1 --2- 172.21.15.38:0/629480011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7178100930 0x7f7178074ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:53.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.244+0000 7f7176a00700 1 -- 172.21.15.38:0/629480011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640095f0 con 0x7f7178101b30 2024-03-24T00:46:53.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.244+0000 7f7177400700 1 --2- 172.21.15.38:0/629480011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7178100930 0x7f7178074ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T00:46:53.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.245+0000 7f7176a00700 1 --2- 172.21.15.38:0/629480011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7178101b30 0x7f71780731a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7164000c00 tx=0x7f7164005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:46:53.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.245+0000 7f716fe00700 1 -- 172.21.15.38:0/629480011 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f716401d070 con 0x7f7178101b30 2024-03-24T00:46:53.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.245+0000 7f716fe00700 1 -- 172.21.15.38:0/629480011 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71640052a0 con 0x7f7178101b30 2024-03-24T00:46:53.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.245+0000 7f717ce00700 1 -- 172.21.15.38:0/629480011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7178073aa0 con 0x7f7178101b30 2024-03-24T00:46:53.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.245+0000 7f716fe00700 1 -- 172.21.15.38:0/629480011 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7164017400 con 0x7f7178101b30 2024-03-24T00:46:53.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.245+0000 7f717ce00700 1 -- 172.21.15.38:0/629480011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7178074010 con 0x7f7178101b30 2024-03-24T00:46:53.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.247+0000 7f717ce00700 1 -- 172.21.15.38:0/629480011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71781904b0 con 0x7f7178101b30 2024-03-24T00:46:53.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.252+0000 7f716fe00700 1 -- 172.21.15.38:0/629480011 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7164003780 con 0x7f7178101b30 2024-03-24T00:46:53.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.253+0000 7f716fe00700 1 --2- 172.21.15.38:0/629480011 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f715c06ce10 0x7f715c06f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:46:53.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.253+0000 7f716fe00700 1 -- 172.21.15.38:0/629480011 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f716408d5c0 con 0x7f7178101b30 2024-03-24T00:46:53.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.253+0000 7f716fe00700 1 -- 172.21.15.38:0/629480011 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f716408d9a0 con 0x7f7178101b30 2024-03-24T00:46:53.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.253+0000 7f7177400700 1 --2- 172.21.15.38:0/629480011 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f715c06ce10 0x7f715c06f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:46:53.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.254+0000 7f7177400700 1 --2- 172.21.15.38:0/629480011 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f715c06ce10 0x7f715c06f2d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7168005950 tx=0x7f7168009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:46:53.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.490+0000 7f717ce00700 1 -- 172.21.15.38:0/629480011 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7178074010 con 0x7f715c06ce10 2024-03-24T00:46:53.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.492+0000 7f716fe00700 1 -- 172.21.15.38:0/629480011 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f7178074010 con 0x7f715c06ce10 2024-03-24T00:46:53.492 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:46:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:46:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:46:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-24T00:46:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [], 2024-03-24T00:46:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "0/2 daemons upgraded", 2024-03-24T00:46:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mgr daemons", 2024-03-24T00:46:53.492 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:46:53.493 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:46:53.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.494+0000 7f717ce00700 1 -- 172.21.15.38:0/629480011 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f715c06ce10 msgr2=0x7f715c06f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.494+0000 7f717ce00700 1 --2- 172.21.15.38:0/629480011 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f715c06ce10 0x7f715c06f2d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7168005950 tx=0x7f7168009260 comp rx=0 tx=0).stop 2024-03-24T00:46:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.494+0000 7f717ce00700 1 -- 172.21.15.38:0/629480011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7178101b30 msgr2=0x7f71780731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:46:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.494+0000 7f717ce00700 1 --2- 172.21.15.38:0/629480011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7178101b30 0x7f71780731a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7164000c00 tx=0x7f7164005f00 comp rx=0 tx=0).stop 2024-03-24T00:46:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.495+0000 7f717ce00700 1 -- 172.21.15.38:0/629480011 shutdown_connections 2024-03-24T00:46:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.495+0000 7f717ce00700 1 --2- 172.21.15.38:0/629480011 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f715c06ce10 0x7f715c06f2d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.495+0000 7f717ce00700 1 --2- 172.21.15.38:0/629480011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7178100930 0x7f7178074ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.495+0000 7f717ce00700 1 --2- 172.21.15.38:0/629480011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7178101b30 0x7f71780731a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:46:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.495+0000 7f717ce00700 1 -- 172.21.15.38:0/629480011 >> 172.21.15.38:0/629480011 conn(0x7f71780fbed0 msgr2=0x7f71781050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:46:53.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.495+0000 7f717ce00700 1 -- 172.21.15.38:0/629480011 shutdown_connections 2024-03-24T00:46:53.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:46:53.495+0000 7f717ce00700 1 -- 172.21.15.38:0/629480011 wait complete. 2024-03-24T00:46:53.849 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:53 smithi038 ceph-mon[29271]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:46:53.849 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:53 smithi038 ceph-mon[29271]: from='client.24327 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:46:53.849 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:53 smithi038 ceph-mon[29271]: pgmap v287: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:46:53.849 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:53 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1676738204' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:46:53.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:53 smithi088 ceph-mon[36610]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:46:53.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:53 smithi088 ceph-mon[36610]: from='client.24327 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:46:53.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:53 smithi088 ceph-mon[36610]: pgmap v287: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:46:53.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:53 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1676738204' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:46:55.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:54 smithi038 ceph-mon[29271]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:46:55.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:54 smithi088 ceph-mon[36610]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:46:56.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:55 smithi038 ceph-mon[29271]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc on smithi088 2024-03-24T00:46:56.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:55 smithi038 ceph-mon[29271]: pgmap v288: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:56.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:55 smithi088 ceph-mon[36610]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc on smithi088 2024-03-24T00:46:56.182 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:55 smithi088 ceph-mon[36610]: pgmap v288: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:46:57.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:56 smithi038 ceph-mon[29271]: pgmap v289: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:46:57.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:56 smithi088 ceph-mon[36610]: pgmap v289: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:46:59.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:58 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:46:59.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:58 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:47:00.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:46:59 smithi038 ceph-mon[29271]: pgmap v290: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:47:00.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:46:59 smithi088 ceph-mon[36610]: pgmap v290: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:47:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:00 smithi038 ceph-mon[29271]: pgmap v291: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:47:01.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:00 smithi088 ceph-mon[36610]: pgmap v291: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:47:03.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:02 smithi038 ceph-mon[29271]: pgmap v292: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:47:03.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:02 smithi088 ceph-mon[36610]: pgmap v292: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:47:05.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:04 smithi038 ceph-mon[29271]: pgmap v293: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:47:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:04 smithi088 ceph-mon[36610]: pgmap v293: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:47:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:06 smithi038 ceph-mon[29271]: pgmap v294: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:47:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:06 smithi088 ceph-mon[36610]: pgmap v294: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:47:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:09 smithi038 ceph-mon[29271]: pgmap v295: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:47:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:09 smithi088 ceph-mon[36610]: pgmap v295: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:47:10.762 INFO:tasks.workunit.client.1.smithi088.stderr:Updating files: 60% (7394/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 98% (12059/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-24T00:47:11.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:10 smithi038 ceph-mon[29271]: pgmap v296: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:47:11.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:10 smithi088 ceph-mon[36610]: pgmap v296: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:47:11.544 INFO:tasks.workunit.client.1.smithi088.stderr:Note: switching to '3be9a5c9bc793e11e2800a8c0c696e8b46742033'. 2024-03-24T00:47:11.544 INFO:tasks.workunit.client.1.smithi088.stderr: 2024-03-24T00:47:11.544 INFO:tasks.workunit.client.1.smithi088.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-24T00:47:11.544 INFO:tasks.workunit.client.1.smithi088.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-24T00:47:11.544 INFO:tasks.workunit.client.1.smithi088.stderr:state without impacting any branches by switching back to a branch. 2024-03-24T00:47:11.544 INFO:tasks.workunit.client.1.smithi088.stderr: 2024-03-24T00:47:11.544 INFO:tasks.workunit.client.1.smithi088.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-24T00:47:11.545 INFO:tasks.workunit.client.1.smithi088.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-24T00:47:11.545 INFO:tasks.workunit.client.1.smithi088.stderr: 2024-03-24T00:47:11.545 INFO:tasks.workunit.client.1.smithi088.stderr: git switch -c 2024-03-24T00:47:11.545 INFO:tasks.workunit.client.1.smithi088.stderr: 2024-03-24T00:47:11.545 INFO:tasks.workunit.client.1.smithi088.stderr:Or undo this operation with: 2024-03-24T00:47:11.545 INFO:tasks.workunit.client.1.smithi088.stderr: 2024-03-24T00:47:11.545 INFO:tasks.workunit.client.1.smithi088.stderr: git switch - 2024-03-24T00:47:11.545 INFO:tasks.workunit.client.1.smithi088.stderr: 2024-03-24T00:47:11.545 INFO:tasks.workunit.client.1.smithi088.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-24T00:47:11.545 INFO:tasks.workunit.client.1.smithi088.stderr: 2024-03-24T00:47:11.545 INFO:tasks.workunit.client.1.smithi088.stderr:HEAD is now at 3be9a5c9bc7 dummy commit #1 2024-03-24T00:47:11.553 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-24T00:47:11.608 INFO:tasks.workunit.client.1.smithi088.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-24T00:47:11.613 INFO:tasks.workunit.client.1.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-24T00:47:11.613 INFO:tasks.workunit.client.1.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-24T00:47:12.508 INFO:tasks.workunit.client.0.smithi038.stderr:Updating files: 59% (7337/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-24T00:47:12.569 INFO:tasks.workunit.client.1.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-24T00:47:12.697 INFO:tasks.workunit.client.1.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-24T00:47:12.778 INFO:tasks.workunit.client.1.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-24T00:47:12.783 INFO:tasks.workunit.client.1.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-24T00:47:12.783 INFO:tasks.workunit.client.1.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-24T00:47:12.870 INFO:tasks.workunit.client.1.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-24T00:47:12.881 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T00:47:12.881 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-24T00:47:12.932 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-24T00:47:12.935 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-24T00:47:12.935 DEBUG:teuthology.orchestra.run.smithi088:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-24T00:47:13.038 INFO:tasks.workunit.client.1.smithi088.stderr:+ mkdir -p fsstress 2024-03-24T00:47:13.042 INFO:tasks.workunit.client.1.smithi088.stderr:+ pushd fsstress 2024-03-24T00:47:13.042 INFO:tasks.workunit.client.1.smithi088.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-24T00:47:13.042 INFO:tasks.workunit.client.1.smithi088.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-24T00:47:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:12 smithi038 ceph-mon[29271]: pgmap v297: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:47:13.114 INFO:tasks.workunit.client.0.smithi038.stderr:Note: switching to '3be9a5c9bc793e11e2800a8c0c696e8b46742033'. 2024-03-24T00:47:13.114 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-03-24T00:47:13.114 INFO:tasks.workunit.client.0.smithi038.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-24T00:47:13.114 INFO:tasks.workunit.client.0.smithi038.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-24T00:47:13.114 INFO:tasks.workunit.client.0.smithi038.stderr:state without impacting any branches by switching back to a branch. 2024-03-24T00:47:13.114 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-03-24T00:47:13.114 INFO:tasks.workunit.client.0.smithi038.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-24T00:47:13.114 INFO:tasks.workunit.client.0.smithi038.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-24T00:47:13.114 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-03-24T00:47:13.115 INFO:tasks.workunit.client.0.smithi038.stderr: git switch -c 2024-03-24T00:47:13.115 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-03-24T00:47:13.115 INFO:tasks.workunit.client.0.smithi038.stderr:Or undo this operation with: 2024-03-24T00:47:13.115 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-03-24T00:47:13.115 INFO:tasks.workunit.client.0.smithi038.stderr: git switch - 2024-03-24T00:47:13.115 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-03-24T00:47:13.115 INFO:tasks.workunit.client.0.smithi038.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-24T00:47:13.115 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-03-24T00:47:13.115 INFO:tasks.workunit.client.0.smithi038.stderr:HEAD is now at 3be9a5c9bc7 dummy commit #1 2024-03-24T00:47:13.125 DEBUG:teuthology.orchestra.run.smithi038:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-24T00:47:13.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:12 smithi088 ceph-mon[36610]: pgmap v297: 65 pgs: 65 active+clean; 589 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:47:13.185 INFO:tasks.workunit.client.0.smithi038.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-24T00:47:13.192 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-24T00:47:13.192 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-24T00:47:13.855 INFO:tasks.workunit.client.1.smithi088.stderr:+ tar xzf ltp-full.tgz 2024-03-24T00:47:14.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:13 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:47:14.154 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-24T00:47:14.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:13 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:47:14.284 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-24T00:47:14.374 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-24T00:47:14.379 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-24T00:47:14.379 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-24T00:47:14.491 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-24T00:47:14.501 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T00:47:14.501 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-24T00:47:14.541 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-24T00:47:14.542 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-24T00:47:14.542 DEBUG:teuthology.orchestra.run.smithi038:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-24T00:47:14.646 INFO:tasks.workunit.client.0.smithi038.stderr:+ mkdir -p fsstress 2024-03-24T00:47:14.649 INFO:tasks.workunit.client.0.smithi038.stderr:+ pushd fsstress 2024-03-24T00:47:14.650 INFO:tasks.workunit.client.0.smithi038.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-24T00:47:14.650 INFO:tasks.workunit.client.0.smithi038.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-24T00:47:15.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:14 smithi038 ceph-mon[29271]: pgmap v298: 65 pgs: 65 active+clean; 600 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-03-24T00:47:15.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:14 smithi088 ceph-mon[36610]: pgmap v298: 65 pgs: 65 active+clean; 600 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-03-24T00:47:17.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:16 smithi038 ceph-mon[29271]: pgmap v299: 65 pgs: 65 active+clean; 600 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-24T00:47:17.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:16 smithi088 ceph-mon[36610]: pgmap v299: 65 pgs: 65 active+clean; 600 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-24T00:47:18.452 INFO:tasks.workunit.client.0.smithi038.stderr:+ tar xzf ltp-full.tgz 2024-03-24T00:47:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:19 smithi038 ceph-mon[29271]: pgmap v300: 65 pgs: 65 active+clean; 2.2 MiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s rd, 145 KiB/s wr, 5 op/s 2024-03-24T00:47:20.522 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:19 smithi088 ceph-mon[36610]: pgmap v300: 65 pgs: 65 active+clean; 2.2 MiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s rd, 145 KiB/s wr, 5 op/s 2024-03-24T00:47:21.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:20 smithi038 ceph-mon[29271]: pgmap v301: 65 pgs: 65 active+clean; 7.2 MiB data, 177 MiB used, 536 GiB / 536 GiB avail; 446 KiB/s rd, 584 KiB/s wr, 28 op/s 2024-03-24T00:47:21.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:20 smithi088 ceph-mon[36610]: pgmap v301: 65 pgs: 65 active+clean; 7.2 MiB data, 177 MiB used, 536 GiB / 536 GiB avail; 446 KiB/s rd, 584 KiB/s wr, 28 op/s 2024-03-24T00:47:23.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:22 smithi038 ceph-mon[29271]: pgmap v302: 65 pgs: 65 active+clean; 17 MiB data, 211 MiB used, 536 GiB / 536 GiB avail; 446 KiB/s rd, 1.4 MiB/s wr, 76 op/s 2024-03-24T00:47:23.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:22 smithi088 ceph-mon[36610]: pgmap v302: 65 pgs: 65 active+clean; 17 MiB data, 211 MiB used, 536 GiB / 536 GiB avail; 446 KiB/s rd, 1.4 MiB/s wr, 76 op/s 2024-03-24T00:47:23.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.677+0000 7fd409200700 1 -- 172.21.15.38:0/2791846919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd404073790 msgr2=0x7fd404073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:23.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.677+0000 7fd409200700 1 --2- 172.21.15.38:0/2791846919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd404073790 0x7fd404073c10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009910 tx=0x7fd3f0009c20 comp rx=0 tx=0).stop 2024-03-24T00:47:23.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.677+0000 7fd409200700 1 -- 172.21.15.38:0/2791846919 shutdown_connections 2024-03-24T00:47:23.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.677+0000 7fd409200700 1 --2- 172.21.15.38:0/2791846919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd404073790 0x7fd404073c10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:23.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.677+0000 7fd409200700 1 --2- 172.21.15.38:0/2791846919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd404074d60 0x7fd4040731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:23.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.677+0000 7fd409200700 1 -- 172.21.15.38:0/2791846919 >> 172.21.15.38:0/2791846919 conn(0x7fd4040fc090 msgr2=0x7fd4040fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:23.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.678+0000 7fd409200700 1 -- 172.21.15.38:0/2791846919 shutdown_connections 2024-03-24T00:47:23.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.678+0000 7fd409200700 1 -- 172.21.15.38:0/2791846919 wait complete. 2024-03-24T00:47:23.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.679+0000 7fd409200700 1 Processor -- start 2024-03-24T00:47:23.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.679+0000 7fd409200700 1 -- start start 2024-03-24T00:47:23.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.680+0000 7fd409200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd404073790 0x7fd404071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:23.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.680+0000 7fd409200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd404074d60 0x7fd404072380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:23.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.680+0000 7fd409200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040729a0 con 0x7fd404074d60 2024-03-24T00:47:23.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.680+0000 7fd409200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd404072ae0 con 0x7fd404073790 2024-03-24T00:47:23.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.681+0000 7fd401e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd404073790 0x7fd404071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.681+0000 7fd401400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd404074d60 0x7fd404072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.681+0000 7fd401400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd404074d60 0x7fd404072380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57238/0 (socket says 172.21.15.38:57238) 2024-03-24T00:47:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.681+0000 7fd401400700 1 -- 172.21.15.38:0/3120007186 learned_addr learned my addr 172.21.15.38:0/3120007186 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:47:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.682+0000 7fd401400700 1 -- 172.21.15.38:0/3120007186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd404073790 msgr2=0x7fd404071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.682+0000 7fd401400700 1 --2- 172.21.15.38:0/3120007186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd404073790 0x7fd404071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.682+0000 7fd401400700 1 -- 172.21.15.38:0/3120007186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f00095f0 con 0x7fd404074d60 2024-03-24T00:47:23.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.682+0000 7fd401400700 1 --2- 172.21.15.38:0/3120007186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd404074d60 0x7fd404072380 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000b650 tx=0x7fd3f0005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:23.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.683+0000 7fd3faa00700 1 -- 172.21.15.38:0/3120007186 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3f001d070 con 0x7fd404074d60 2024-03-24T00:47:23.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.683+0000 7fd3faa00700 1 -- 172.21.15.38:0/3120007186 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3f00052a0 con 0x7fd404074d60 2024-03-24T00:47:23.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.683+0000 7fd409200700 1 -- 172.21.15.38:0/3120007186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4041a6540 con 0x7fd404074d60 2024-03-24T00:47:23.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.683+0000 7fd409200700 1 -- 172.21.15.38:0/3120007186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4041a6ab0 con 0x7fd404074d60 2024-03-24T00:47:23.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.684+0000 7fd3faa00700 1 -- 172.21.15.38:0/3120007186 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3f0017400 con 0x7fd404074d60 2024-03-24T00:47:23.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.685+0000 7fd409200700 1 -- 172.21.15.38:0/3120007186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40406bf70 con 0x7fd404074d60 2024-03-24T00:47:23.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.688+0000 7fd3faa00700 1 -- 172.21.15.38:0/3120007186 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd3f0003810 con 0x7fd404074d60 2024-03-24T00:47:23.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.689+0000 7fd3faa00700 1 --2- 172.21.15.38:0/3120007186 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd3e806cd40 0x7fd3e806f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:23.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.689+0000 7fd3faa00700 1 -- 172.21.15.38:0/3120007186 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd3f008d850 con 0x7fd404074d60 2024-03-24T00:47:23.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.691+0000 7fd401e00700 1 --2- 172.21.15.38:0/3120007186 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd3e806cd40 0x7fd3e806f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:23.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.693+0000 7fd401e00700 1 --2- 172.21.15.38:0/3120007186 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd3e806cd40 0x7fd3e806f200 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009c60 tx=0x7fd3f4009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:23.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.696+0000 7fd3faa00700 1 -- 172.21.15.38:0/3120007186 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd3f005b6b0 con 0x7fd404074d60 2024-03-24T00:47:23.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:23.998+0000 7fd409200700 1 -- 172.21.15.38:0/3120007186 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4040611f0 con 0x7fd3e806cd40 2024-03-24T00:47:24.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.000+0000 7fd3faa00700 1 -- 172.21.15.38:0/3120007186 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fd4040611f0 con 0x7fd3e806cd40 2024-03-24T00:47:24.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.004+0000 7fd409200700 1 -- 172.21.15.38:0/3120007186 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd3e806cd40 msgr2=0x7fd3e806f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:24.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.004+0000 7fd409200700 1 --2- 172.21.15.38:0/3120007186 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd3e806cd40 0x7fd3e806f200 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009c60 tx=0x7fd3f4009260 comp rx=0 tx=0).stop 2024-03-24T00:47:24.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.004+0000 7fd409200700 1 -- 172.21.15.38:0/3120007186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd404074d60 msgr2=0x7fd404072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:24.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.004+0000 7fd409200700 1 --2- 172.21.15.38:0/3120007186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd404074d60 0x7fd404072380 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000b650 tx=0x7fd3f0005f00 comp rx=0 tx=0).stop 2024-03-24T00:47:24.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.004+0000 7fd409200700 1 -- 172.21.15.38:0/3120007186 shutdown_connections 2024-03-24T00:47:24.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.004+0000 7fd409200700 1 --2- 172.21.15.38:0/3120007186 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd3e806cd40 0x7fd3e806f200 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.004+0000 7fd409200700 1 --2- 172.21.15.38:0/3120007186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd404073790 0x7fd404071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.004+0000 7fd409200700 1 --2- 172.21.15.38:0/3120007186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd404074d60 0x7fd404072380 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.004+0000 7fd409200700 1 -- 172.21.15.38:0/3120007186 >> 172.21.15.38:0/3120007186 conn(0x7fd4040fc090 msgr2=0x7fd404106ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:24.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.005+0000 7fd409200700 1 -- 172.21.15.38:0/3120007186 shutdown_connections 2024-03-24T00:47:24.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.005+0000 7fd409200700 1 -- 172.21.15.38:0/3120007186 wait complete. 2024-03-24T00:47:24.028 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:47:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.184+0000 7fc910a00700 1 -- 172.21.15.38:0/2499585604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0ff210 msgr2=0x7fc90c0ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.184+0000 7fc910a00700 1 --2- 172.21.15.38:0/2499585604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0ff210 0x7fc90c0ff630 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009910 tx=0x7fc8fc009c20 comp rx=0 tx=0).stop 2024-03-24T00:47:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.185+0000 7fc910a00700 1 -- 172.21.15.38:0/2499585604 shutdown_connections 2024-03-24T00:47:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.185+0000 7fc910a00700 1 --2- 172.21.15.38:0/2499585604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc90c0ffb70 0x7fc90c0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.185+0000 7fc910a00700 1 --2- 172.21.15.38:0/2499585604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0ff210 0x7fc90c0ff630 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.185+0000 7fc910a00700 1 -- 172.21.15.38:0/2499585604 >> 172.21.15.38:0/2499585604 conn(0x7fc90c0fadf0 msgr2=0x7fc90c0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.185+0000 7fc910a00700 1 -- 172.21.15.38:0/2499585604 shutdown_connections 2024-03-24T00:47:24.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.185+0000 7fc910a00700 1 -- 172.21.15.38:0/2499585604 wait complete. 2024-03-24T00:47:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.187+0000 7fc910a00700 1 Processor -- start 2024-03-24T00:47:24.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.187+0000 7fc910a00700 1 -- start start 2024-03-24T00:47:24.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.187+0000 7fc910a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0ff210 0x7fc90c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:24.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.187+0000 7fc910a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc90c0ffb70 0x7fc90c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:24.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.188+0000 7fc910a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c198ff0 con 0x7fc90c0ff210 2024-03-24T00:47:24.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.188+0000 7fc910a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c199130 con 0x7fc90c0ffb70 2024-03-24T00:47:24.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.188+0000 7fc909e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc90c0ffb70 0x7fc90c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:24.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.188+0000 7fc909e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc90c0ffb70 0x7fc90c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:49568/0 (socket says 172.21.15.38:49568) 2024-03-24T00:47:24.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.188+0000 7fc909e00700 1 -- 172.21.15.38:0/2380342858 learned_addr learned my addr 172.21.15.38:0/2380342858 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:47:24.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.189+0000 7fc90a800700 1 --2- 172.21.15.38:0/2380342858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0ff210 0x7fc90c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:24.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.189+0000 7fc909e00700 1 -- 172.21.15.38:0/2380342858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0ff210 msgr2=0x7fc90c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:24.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.189+0000 7fc909e00700 1 --2- 172.21.15.38:0/2380342858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0ff210 0x7fc90c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.189+0000 7fc909e00700 1 -- 172.21.15.38:0/2380342858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc0095f0 con 0x7fc90c0ffb70 2024-03-24T00:47:24.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.190+0000 7fc909e00700 1 --2- 172.21.15.38:0/2380342858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc90c0ffb70 0x7fc90c198a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800c930 tx=0x7fc8f800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:24.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.191+0000 7fc903400700 1 -- 172.21.15.38:0/2380342858 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f8007ad0 con 0x7fc90c0ffb70 2024-03-24T00:47:24.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.191+0000 7fc910a00700 1 -- 172.21.15.38:0/2380342858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c19dba0 con 0x7fc90c0ffb70 2024-03-24T00:47:24.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.191+0000 7fc903400700 1 -- 172.21.15.38:0/2380342858 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8f800ce80 con 0x7fc90c0ffb70 2024-03-24T00:47:24.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.191+0000 7fc903400700 1 -- 172.21.15.38:0/2380342858 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f8018630 con 0x7fc90c0ffb70 2024-03-24T00:47:24.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.191+0000 7fc910a00700 1 -- 172.21.15.38:0/2380342858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c19e140 con 0x7fc90c0ffb70 2024-03-24T00:47:24.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.193+0000 7fc910a00700 1 -- 172.21.15.38:0/2380342858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c1926b0 con 0x7fc90c0ffb70 2024-03-24T00:47:24.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.196+0000 7fc903400700 1 -- 172.21.15.38:0/2380342858 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc8f8007c30 con 0x7fc90c0ffb70 2024-03-24T00:47:24.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.197+0000 7fc903400700 1 --2- 172.21.15.38:0/2380342858 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc8f006ccf0 0x7fc8f006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:24.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.197+0000 7fc903400700 1 -- 172.21.15.38:0/2380342858 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc8f808c450 con 0x7fc90c0ffb70 2024-03-24T00:47:24.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.197+0000 7fc90a800700 1 --2- 172.21.15.38:0/2380342858 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc8f006ccf0 0x7fc8f006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:24.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.198+0000 7fc90a800700 1 --2- 172.21.15.38:0/2380342858 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc8f006ccf0 0x7fc8f006f1b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc006010 tx=0x7fc8fc005180 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:24.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.213+0000 7fc903400700 1 -- 172.21.15.38:0/2380342858 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc8f805a2b0 con 0x7fc90c0ffb70 2024-03-24T00:47:24.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.521+0000 7fc910a00700 1 -- 172.21.15.38:0/2380342858 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc90c0611f0 con 0x7fc8f006ccf0 2024-03-24T00:47:24.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.523+0000 7fc903400700 1 -- 172.21.15.38:0/2380342858 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fc90c0611f0 con 0x7fc8f006ccf0 2024-03-24T00:47:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.527+0000 7fc910a00700 1 -- 172.21.15.38:0/2380342858 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc8f006ccf0 msgr2=0x7fc8f006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.527+0000 7fc910a00700 1 --2- 172.21.15.38:0/2380342858 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc8f006ccf0 0x7fc8f006f1b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc006010 tx=0x7fc8fc005180 comp rx=0 tx=0).stop 2024-03-24T00:47:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.527+0000 7fc910a00700 1 -- 172.21.15.38:0/2380342858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc90c0ffb70 msgr2=0x7fc90c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.527+0000 7fc910a00700 1 --2- 172.21.15.38:0/2380342858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc90c0ffb70 0x7fc90c198a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800c930 tx=0x7fc8f800cc40 comp rx=0 tx=0).stop 2024-03-24T00:47:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.527+0000 7fc910a00700 1 -- 172.21.15.38:0/2380342858 shutdown_connections 2024-03-24T00:47:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.527+0000 7fc910a00700 1 --2- 172.21.15.38:0/2380342858 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc8f006ccf0 0x7fc8f006f1b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.527+0000 7fc910a00700 1 --2- 172.21.15.38:0/2380342858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0ff210 0x7fc90c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.527+0000 7fc910a00700 1 --2- 172.21.15.38:0/2380342858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc90c0ffb70 0x7fc90c198a60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.527+0000 7fc910a00700 1 -- 172.21.15.38:0/2380342858 >> 172.21.15.38:0/2380342858 conn(0x7fc90c0fadf0 msgr2=0x7fc90c107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.527+0000 7fc910a00700 1 -- 172.21.15.38:0/2380342858 shutdown_connections 2024-03-24T00:47:24.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.527+0000 7fc910a00700 1 -- 172.21.15.38:0/2380342858 wait complete. 2024-03-24T00:47:24.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.714+0000 7fc5bb800700 1 -- 172.21.15.38:0/3384673860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5b4100ad0 msgr2=0x7fc5b4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:24.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.714+0000 7fc5bb800700 1 --2- 172.21.15.38:0/3384673860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5b4100ad0 0x7fc5b4100ef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009910 tx=0x7fc5a8009c20 comp rx=0 tx=0).stop 2024-03-24T00:47:24.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.714+0000 7fc5bb800700 1 -- 172.21.15.38:0/3384673860 shutdown_connections 2024-03-24T00:47:24.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.714+0000 7fc5bb800700 1 --2- 172.21.15.38:0/3384673860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5b4101cd0 0x7fc5b4102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.714+0000 7fc5bb800700 1 --2- 172.21.15.38:0/3384673860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5b4100ad0 0x7fc5b4100ef0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.714+0000 7fc5bb800700 1 -- 172.21.15.38:0/3384673860 >> 172.21.15.38:0/3384673860 conn(0x7fc5b40fc090 msgr2=0x7fc5b40fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:24.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.714+0000 7fc5bb800700 1 -- 172.21.15.38:0/3384673860 shutdown_connections 2024-03-24T00:47:24.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.714+0000 7fc5bb800700 1 -- 172.21.15.38:0/3384673860 wait complete. 2024-03-24T00:47:24.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.715+0000 7fc5bb800700 1 Processor -- start 2024-03-24T00:47:24.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.716+0000 7fc5bb800700 1 -- start start 2024-03-24T00:47:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.716+0000 7fc5bb800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5b4100ad0 0x7fc5b419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.716+0000 7fc5bb800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5b4101cd0 0x7fc5b419d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.716+0000 7fc5bb800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b419d760 con 0x7fc5b4100ad0 2024-03-24T00:47:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.716+0000 7fc5bb800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b419d8a0 con 0x7fc5b4101cd0 2024-03-24T00:47:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.716+0000 7fc5b8e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5b4100ad0 0x7fc5b419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.717+0000 7fc5b8e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5b4100ad0 0x7fc5b419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57272/0 (socket says 172.21.15.38:57272) 2024-03-24T00:47:24.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.717+0000 7fc5b8e00700 1 -- 172.21.15.38:0/438113605 learned_addr learned my addr 172.21.15.38:0/438113605 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:47:24.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.717+0000 7fc5b3e00700 1 --2- 172.21.15.38:0/438113605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5b4101cd0 0x7fc5b419d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:24.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.717+0000 7fc5b8e00700 1 -- 172.21.15.38:0/438113605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5b4101cd0 msgr2=0x7fc5b419d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:24.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.717+0000 7fc5b8e00700 1 --2- 172.21.15.38:0/438113605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5b4101cd0 0x7fc5b419d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:24.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.717+0000 7fc5b8e00700 1 -- 172.21.15.38:0/438113605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a80095f0 con 0x7fc5b4100ad0 2024-03-24T00:47:24.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.717+0000 7fc5b8e00700 1 --2- 172.21.15.38:0/438113605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5b4100ad0 0x7fc5b419cc00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800bd70 tx=0x7fc5a800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:24.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.718+0000 7fc5b1600700 1 -- 172.21.15.38:0/438113605 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5a801d070 con 0x7fc5b4100ad0 2024-03-24T00:47:24.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.718+0000 7fc5bb800700 1 -- 172.21.15.38:0/438113605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b41a22a0 con 0x7fc5b4100ad0 2024-03-24T00:47:24.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.718+0000 7fc5bb800700 1 -- 172.21.15.38:0/438113605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b41a2810 con 0x7fc5b4100ad0 2024-03-24T00:47:24.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.719+0000 7fc5b1600700 1 -- 172.21.15.38:0/438113605 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5a800f460 con 0x7fc5b4100ad0 2024-03-24T00:47:24.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.719+0000 7fc5b1600700 1 -- 172.21.15.38:0/438113605 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5a8021620 con 0x7fc5b4100ad0 2024-03-24T00:47:24.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.720+0000 7fc5bb800700 1 -- 172.21.15.38:0/438113605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc594005320 con 0x7fc5b4100ad0 2024-03-24T00:47:24.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.721+0000 7fc5b1600700 1 -- 172.21.15.38:0/438113605 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc5a8003a60 con 0x7fc5b4100ad0 2024-03-24T00:47:24.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.722+0000 7fc5b1600700 1 --2- 172.21.15.38:0/438113605 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc59c06ccf0 0x7fc59c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:24.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.722+0000 7fc5b1600700 1 -- 172.21.15.38:0/438113605 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc5a808d4b0 con 0x7fc5b4100ad0 2024-03-24T00:47:24.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.722+0000 7fc5b3e00700 1 --2- 172.21.15.38:0/438113605 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc59c06ccf0 0x7fc59c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:24.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.723+0000 7fc5b3e00700 1 --2- 172.21.15.38:0/438113605 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc59c06ccf0 0x7fc59c06f1b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0006fd0 tx=0x7fc5a0008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:24.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:24.730+0000 7fc5b1600700 1 -- 172.21.15.38:0/438113605 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc5a805b290 con 0x7fc5b4100ad0 2024-03-24T00:47:25.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.068+0000 7fc5bb800700 1 -- 172.21.15.38:0/438113605 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc594000bf0 con 0x7fc59c06ccf0 2024-03-24T00:47:25.070 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:24 smithi038 ceph-mon[29271]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:25.070 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:24 smithi038 ceph-mon[29271]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:25.070 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:24 smithi038 ceph-mon[29271]: pgmap v303: 65 pgs: 65 active+clean; 39 MiB data, 286 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 147 op/s 2024-03-24T00:47:25.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.080+0000 7fc5b1600700 1 -- 172.21.15.38:0/438113605 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fc594000bf0 con 0x7fc59c06ccf0 2024-03-24T00:47:25.081 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:47:25.081 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (7m) 116s ago 9m 13.4M - 0.25.0 c8568f914cd2 9f1243fbcb3e 2024-03-24T00:47:25.081 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (10m) 116s ago 10m 7658k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:47:25.081 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (8m) 2m ago 8m 7482k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:47:25.081 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (10m) 116s ago 10m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (8m) 2m ago 8m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (7m) 116s ago 9m 72.7M - 9.4.7 954c08fa6188 e6aa268d69e8 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (2m) 116s ago 2m 13.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (2m) 116s ago 2m 15.0M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (2m) 2m ago 2m 12.8M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (2m) 2m ago 2m 14.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:9283,8765,8443 running (11m) 116s ago 11m 498M - 18.2.2-952-g738c2fd6 68aff9aff3b2 bf370183b69d 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (8m) 2m ago 8m 449M - 18.2.2-952-g738c2fd6 68aff9aff3b2 acebd34a44e9 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (11m) 116s ago 11m 45.0M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (8m) 2m ago 8m 33.9M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (9m) 116s ago 9m 8108k - 1.5.0 0da6a335fe13 c34cc077c93f 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (8m) 2m ago 8m 7884k - 1.5.0 0da6a335fe13 189e840ee2f1 2024-03-24T00:47:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (7m) 116s ago 7m 46.7M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:47:25.083 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (6m) 116s ago 6m 47.1M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:47:25.083 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (6m) 116s ago 6m 45.5M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:47:25.083 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (5m) 2m ago 5m 45.5M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:47:25.083 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (4m) 2m ago 4m 43.8M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:47:25.083 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (4m) 2m ago 4m 42.7M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:47:25.083 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (7m) 116s ago 9m 49.7M - 2.43.0 a07b618ecd1d c59e5c711ace 2024-03-24T00:47:25.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.082+0000 7fc5bb800700 1 -- 172.21.15.38:0/438113605 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc59c06ccf0 msgr2=0x7fc59c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:25.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.082+0000 7fc5bb800700 1 --2- 172.21.15.38:0/438113605 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc59c06ccf0 0x7fc59c06f1b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0006fd0 tx=0x7fc5a0008040 comp rx=0 tx=0).stop 2024-03-24T00:47:25.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.083+0000 7fc5bb800700 1 -- 172.21.15.38:0/438113605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5b4100ad0 msgr2=0x7fc5b419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:25.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.083+0000 7fc5bb800700 1 --2- 172.21.15.38:0/438113605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5b4100ad0 0x7fc5b419cc00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800bd70 tx=0x7fc5a800be50 comp rx=0 tx=0).stop 2024-03-24T00:47:25.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.083+0000 7fc5bb800700 1 -- 172.21.15.38:0/438113605 shutdown_connections 2024-03-24T00:47:25.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.083+0000 7fc5bb800700 1 --2- 172.21.15.38:0/438113605 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fc59c06ccf0 0x7fc59c06f1b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.083+0000 7fc5bb800700 1 --2- 172.21.15.38:0/438113605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5b4100ad0 0x7fc5b419cc00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.083+0000 7fc5bb800700 1 --2- 172.21.15.38:0/438113605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5b4101cd0 0x7fc5b419d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.083+0000 7fc5bb800700 1 -- 172.21.15.38:0/438113605 >> 172.21.15.38:0/438113605 conn(0x7fc5b40fc090 msgr2=0x7fc5b410a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:25.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.083+0000 7fc5bb800700 1 -- 172.21.15.38:0/438113605 shutdown_connections 2024-03-24T00:47:25.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.083+0000 7fc5bb800700 1 -- 172.21.15.38:0/438113605 wait complete. 2024-03-24T00:47:25.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:24 smithi088 ceph-mon[36610]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:25.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:24 smithi088 ceph-mon[36610]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:25.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:24 smithi088 ceph-mon[36610]: pgmap v303: 65 pgs: 65 active+clean; 39 MiB data, 286 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 147 op/s 2024-03-24T00:47:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.246+0000 7fd731000700 1 -- 172.21.15.38:0/1480587702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c0737b0 msgr2=0x7fd72c073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.246+0000 7fd731000700 1 --2- 172.21.15.38:0/1480587702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c0737b0 0x7fd72c073c30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd718009910 tx=0x7fd718009c20 comp rx=0 tx=0).stop 2024-03-24T00:47:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.246+0000 7fd731000700 1 -- 172.21.15.38:0/1480587702 shutdown_connections 2024-03-24T00:47:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.246+0000 7fd731000700 1 --2- 172.21.15.38:0/1480587702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c0737b0 0x7fd72c073c30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.246+0000 7fd731000700 1 --2- 172.21.15.38:0/1480587702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c074d80 0x7fd72c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.246+0000 7fd731000700 1 -- 172.21.15.38:0/1480587702 >> 172.21.15.38:0/1480587702 conn(0x7fd72c0fc090 msgr2=0x7fd72c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.246+0000 7fd731000700 1 -- 172.21.15.38:0/1480587702 shutdown_connections 2024-03-24T00:47:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.247+0000 7fd731000700 1 -- 172.21.15.38:0/1480587702 wait complete. 2024-03-24T00:47:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.248+0000 7fd731000700 1 Processor -- start 2024-03-24T00:47:25.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.248+0000 7fd731000700 1 -- start start 2024-03-24T00:47:25.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.248+0000 7fd731000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c0737b0 0x7fd72c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:25.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.248+0000 7fd731000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c074d80 0x7fd72c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:25.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.248+0000 7fd731000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c1991d0 con 0x7fd72c0737b0 2024-03-24T00:47:25.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.248+0000 7fd731000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c199310 con 0x7fd72c074d80 2024-03-24T00:47:25.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.249+0000 7fd729e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c0737b0 0x7fd72c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:25.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.249+0000 7fd729e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c0737b0 0x7fd72c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57300/0 (socket says 172.21.15.38:57300) 2024-03-24T00:47:25.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.249+0000 7fd729e00700 1 -- 172.21.15.38:0/2306573338 learned_addr learned my addr 172.21.15.38:0/2306573338 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:47:25.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.249+0000 7fd729400700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c074d80 0x7fd72c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:25.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.249+0000 7fd729e00700 1 -- 172.21.15.38:0/2306573338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c074d80 msgr2=0x7fd72c198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:25.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.250+0000 7fd729e00700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c074d80 0x7fd72c198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.250+0000 7fd729e00700 1 -- 172.21.15.38:0/2306573338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7180095f0 con 0x7fd72c0737b0 2024-03-24T00:47:25.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.250+0000 7fd729400700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c074d80 0x7fd72c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:47:25.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.250+0000 7fd729e00700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c0737b0 0x7fd72c198670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00d900 tx=0x7fd71c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:25.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.250+0000 7fd722a00700 1 -- 172.21.15.38:0/2306573338 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71c004a00 con 0x7fd72c0737b0 2024-03-24T00:47:25.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.251+0000 7fd722a00700 1 -- 172.21.15.38:0/2306573338 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd71c0054e0 con 0x7fd72c0737b0 2024-03-24T00:47:25.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.251+0000 7fd731000700 1 -- 172.21.15.38:0/2306573338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72c19dd70 con 0x7fd72c0737b0 2024-03-24T00:47:25.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.251+0000 7fd722a00700 1 -- 172.21.15.38:0/2306573338 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71c009d60 con 0x7fd72c0737b0 2024-03-24T00:47:25.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.251+0000 7fd731000700 1 -- 172.21.15.38:0/2306573338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72c19e2e0 con 0x7fd72c0737b0 2024-03-24T00:47:25.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.253+0000 7fd731000700 1 -- 172.21.15.38:0/2306573338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72c192860 con 0x7fd72c0737b0 2024-03-24T00:47:25.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.254+0000 7fd722a00700 1 -- 172.21.15.38:0/2306573338 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd71c005020 con 0x7fd72c0737b0 2024-03-24T00:47:25.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.255+0000 7fd722a00700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd71006ccf0 0x7fd71006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:25.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.255+0000 7fd722a00700 1 -- 172.21.15.38:0/2306573338 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd71c08c510 con 0x7fd72c0737b0 2024-03-24T00:47:25.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.256+0000 7fd729400700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd71006ccf0 0x7fd71006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:25.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.257+0000 7fd729400700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd71006ccf0 0x7fd71006f1b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7180051c0 tx=0x7fd71800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:25.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.260+0000 7fd722a00700 1 -- 172.21.15.38:0/2306573338 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd71c05a270 con 0x7fd72c0737b0 2024-03-24T00:47:25.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.702+0000 7fd731000700 1 -- 172.21.15.38:0/2306573338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd72c0619c0 con 0x7fd72c0737b0 2024-03-24T00:47:25.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.703+0000 7fd722a00700 1 -- 172.21.15.38:0/2306573338 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7fd71c059e00 con 0x7fd72c0737b0 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:47:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:47:25.705 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:47:25.705 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 14 2024-03-24T00:47:25.705 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:47:25.705 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:47:25.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.706+0000 7fd731000700 1 -- 172.21.15.38:0/2306573338 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd71006ccf0 msgr2=0x7fd71006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:25.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.706+0000 7fd731000700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd71006ccf0 0x7fd71006f1b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7180051c0 tx=0x7fd71800b540 comp rx=0 tx=0).stop 2024-03-24T00:47:25.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.706+0000 7fd731000700 1 -- 172.21.15.38:0/2306573338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c0737b0 msgr2=0x7fd72c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:25.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.706+0000 7fd731000700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c0737b0 0x7fd72c198670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00d900 tx=0x7fd71c00dcc0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.706+0000 7fd731000700 1 -- 172.21.15.38:0/2306573338 shutdown_connections 2024-03-24T00:47:25.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.706+0000 7fd731000700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7fd71006ccf0 0x7fd71006f1b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.706+0000 7fd731000700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c0737b0 0x7fd72c198670 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.706+0000 7fd731000700 1 --2- 172.21.15.38:0/2306573338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c074d80 0x7fd72c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.706+0000 7fd731000700 1 -- 172.21.15.38:0/2306573338 >> 172.21.15.38:0/2306573338 conn(0x7fd72c0fc090 msgr2=0x7fd72c106ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:25.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.707+0000 7fd731000700 1 -- 172.21.15.38:0/2306573338 shutdown_connections 2024-03-24T00:47:25.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.707+0000 7fd731000700 1 -- 172.21.15.38:0/2306573338 wait complete. 2024-03-24T00:47:25.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.884+0000 7f6610c00700 1 -- 172.21.15.38:0/2759471572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f660c073810 msgr2=0x7f660c073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:25.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.884+0000 7f6610c00700 1 --2- 172.21.15.38:0/2759471572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f660c073810 0x7f660c073c90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009910 tx=0x7f65f8009c20 comp rx=0 tx=0).stop 2024-03-24T00:47:25.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.884+0000 7f6610c00700 1 -- 172.21.15.38:0/2759471572 shutdown_connections 2024-03-24T00:47:25.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.884+0000 7f6610c00700 1 --2- 172.21.15.38:0/2759471572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f660c073810 0x7f660c073c90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.884+0000 7f6610c00700 1 --2- 172.21.15.38:0/2759471572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f660c074de0 0x7f660c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.884+0000 7f6610c00700 1 -- 172.21.15.38:0/2759471572 >> 172.21.15.38:0/2759471572 conn(0x7f660c0fc090 msgr2=0x7f660c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.884+0000 7f6610c00700 1 -- 172.21.15.38:0/2759471572 shutdown_connections 2024-03-24T00:47:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.884+0000 7f6610c00700 1 -- 172.21.15.38:0/2759471572 wait complete. 2024-03-24T00:47:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.885+0000 7f6610c00700 1 Processor -- start 2024-03-24T00:47:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.886+0000 7f6610c00700 1 -- start start 2024-03-24T00:47:25.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.886+0000 7f6610c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f660c073810 0x7f660c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:25.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.886+0000 7f6610c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f660c074de0 0x7f660c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:25.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.886+0000 7f6610c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c199390 con 0x7f660c073810 2024-03-24T00:47:25.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.886+0000 7f6610c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c1994d0 con 0x7f660c074de0 2024-03-24T00:47:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.886+0000 7f6609e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f660c073810 0x7f660c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.886+0000 7f6609e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f660c073810 0x7f660c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57320/0 (socket says 172.21.15.38:57320) 2024-03-24T00:47:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.886+0000 7f6609e00700 1 -- 172.21.15.38:0/1586236574 learned_addr learned my addr 172.21.15.38:0/1586236574 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:47:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.887+0000 7f6609400700 1 --2- 172.21.15.38:0/1586236574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f660c074de0 0x7f660c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.887+0000 7f6609e00700 1 -- 172.21.15.38:0/1586236574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f660c074de0 msgr2=0x7f660c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.887+0000 7f6609e00700 1 --2- 172.21.15.38:0/1586236574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f660c074de0 0x7f660c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.887+0000 7f6609e00700 1 -- 172.21.15.38:0/1586236574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80095f0 con 0x7f660c073810 2024-03-24T00:47:25.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.887+0000 7f6609e00700 1 --2- 172.21.15.38:0/1586236574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f660c073810 0x7f660c198830 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00b480 tx=0x7f65fc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:25.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.888+0000 7f6602a00700 1 -- 172.21.15.38:0/1586236574 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65fc0118c0 con 0x7f660c073810 2024-03-24T00:47:25.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.888+0000 7f6602a00700 1 -- 172.21.15.38:0/1586236574 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65fc010380 con 0x7f660c073810 2024-03-24T00:47:25.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.888+0000 7f6602a00700 1 -- 172.21.15.38:0/1586236574 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65fc00f4f0 con 0x7f660c073810 2024-03-24T00:47:25.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.888+0000 7f6610c00700 1 -- 172.21.15.38:0/1586236574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c19df30 con 0x7f660c073810 2024-03-24T00:47:25.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.888+0000 7f6610c00700 1 -- 172.21.15.38:0/1586236574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c19e420 con 0x7f660c073810 2024-03-24T00:47:25.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.889+0000 7f6610c00700 1 -- 172.21.15.38:0/1586236574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660c04ea80 con 0x7f660c073810 2024-03-24T00:47:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.891+0000 7f6602a00700 1 -- 172.21.15.38:0/1586236574 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f65fc0104f0 con 0x7f660c073810 2024-03-24T00:47:25.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.892+0000 7f6602a00700 1 --2- 172.21.15.38:0/1586236574 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f65f006ccf0 0x7f65f006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:25.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.893+0000 7f6602a00700 1 -- 172.21.15.38:0/1586236574 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f65fc08beb0 con 0x7f660c073810 2024-03-24T00:47:25.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.893+0000 7f6609400700 1 --2- 172.21.15.38:0/1586236574 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f65f006ccf0 0x7f65f006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:25.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.894+0000 7f6609400700 1 --2- 172.21.15.38:0/1586236574 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f65f006ccf0 0x7f65f006f1b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f65f8006010 tx=0x7f65f800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:25.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:25.897+0000 7f6602a00700 1 -- 172.21.15.38:0/1586236574 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f65fc059cb0 con 0x7f660c073810 2024-03-24T00:47:26.071 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:25 smithi088 ceph-mon[36610]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:26.072 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:25 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2306573338' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:47:26.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:25 smithi038 ceph-mon[29271]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:26.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:25 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2306573338' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:47:26.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.164+0000 7f6610c00700 1 -- 172.21.15.38:0/1586236574 --> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f660c107fd0 con 0x7f65f006ccf0 2024-03-24T00:47:26.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.167+0000 7f6602a00700 1 -- 172.21.15.38:0/1586236574 <== mgr.14215 v2:172.21.15.38:6800/3107810736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f660c107fd0 con 0x7f65f006ccf0 2024-03-24T00:47:26.168 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:47:26.169 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:47:26.169 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:47:26.169 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-24T00:47:26.169 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [], 2024-03-24T00:47:26.169 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "0/2 daemons upgraded", 2024-03-24T00:47:26.169 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc image on host smithi088", 2024-03-24T00:47:26.169 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:47:26.169 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:47:26.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.170+0000 7f6610c00700 1 -- 172.21.15.38:0/1586236574 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f65f006ccf0 msgr2=0x7f65f006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:26.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.170+0000 7f6610c00700 1 --2- 172.21.15.38:0/1586236574 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f65f006ccf0 0x7f65f006f1b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f65f8006010 tx=0x7f65f800b540 comp rx=0 tx=0).stop 2024-03-24T00:47:26.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.170+0000 7f6610c00700 1 -- 172.21.15.38:0/1586236574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f660c073810 msgr2=0x7f660c198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:26.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.170+0000 7f6610c00700 1 --2- 172.21.15.38:0/1586236574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f660c073810 0x7f660c198830 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00b480 tx=0x7f65fc00b840 comp rx=0 tx=0).stop 2024-03-24T00:47:26.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.171+0000 7f6610c00700 1 -- 172.21.15.38:0/1586236574 shutdown_connections 2024-03-24T00:47:26.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.171+0000 7f6610c00700 1 --2- 172.21.15.38:0/1586236574 >> [v2:172.21.15.38:6800/3107810736,v1:172.21.15.38:6801/3107810736] conn(0x7f65f006ccf0 0x7f65f006f1b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:26.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.171+0000 7f6610c00700 1 --2- 172.21.15.38:0/1586236574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f660c073810 0x7f660c198830 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:26.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.171+0000 7f6610c00700 1 --2- 172.21.15.38:0/1586236574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f660c074de0 0x7f660c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:26.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.171+0000 7f6610c00700 1 -- 172.21.15.38:0/1586236574 >> 172.21.15.38:0/1586236574 conn(0x7f660c0fc090 msgr2=0x7f660c1068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:26.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.171+0000 7f6610c00700 1 -- 172.21.15.38:0/1586236574 shutdown_connections 2024-03-24T00:47:26.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:26.171+0000 7f6610c00700 1 -- 172.21.15.38:0/1586236574 wait complete. 2024-03-24T00:47:27.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:26 smithi038 ceph-mon[29271]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:27.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:26 smithi038 ceph-mon[29271]: pgmap v304: 65 pgs: 65 active+clean; 40 MiB data, 287 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 147 op/s 2024-03-24T00:47:27.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:26 smithi088 ceph-mon[36610]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:27.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:26 smithi088 ceph-mon[36610]: pgmap v304: 65 pgs: 65 active+clean; 40 MiB data, 287 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 147 op/s 2024-03-24T00:47:28.290 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:28 smithi088 ceph-mon[36610]: Upgrade: Updating mgr.smithi088.njctea 2024-03-24T00:47:28.290 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:28.290 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:47:28.290 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:47:28.290 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:28 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:47:28.290 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:28 smithi088 ceph-mon[36610]: Deploying daemon mgr.smithi088.njctea on smithi088 2024-03-24T00:47:28.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:28 smithi038 ceph-mon[29271]: Upgrade: Updating mgr.smithi088.njctea 2024-03-24T00:47:28.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:28.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:47:28.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:47:28.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:28 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:47:28.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:28 smithi038 ceph-mon[29271]: Deploying daemon mgr.smithi088.njctea on smithi088 2024-03-24T00:47:29.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:29 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:47:29.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:29 smithi038 ceph-mon[29271]: pgmap v305: 65 pgs: 65 active+clean; 44 MiB data, 309 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 165 op/s 2024-03-24T00:47:29.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:29 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:47:29.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:29 smithi088 ceph-mon[36610]: pgmap v305: 65 pgs: 65 active+clean; 44 MiB data, 309 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 165 op/s 2024-03-24T00:47:31.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:30 smithi038 ceph-mon[29271]: pgmap v306: 65 pgs: 65 active+clean; 48 MiB data, 317 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 170 op/s 2024-03-24T00:47:31.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:30 smithi088 ceph-mon[36610]: pgmap v306: 65 pgs: 65 active+clean; 48 MiB data, 317 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 170 op/s 2024-03-24T00:47:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:32 smithi038 ceph-mon[29271]: pgmap v307: 65 pgs: 65 active+clean; 52 MiB data, 326 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 149 op/s 2024-03-24T00:47:33.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:32 smithi088 ceph-mon[36610]: pgmap v307: 65 pgs: 65 active+clean; 52 MiB data, 326 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 149 op/s 2024-03-24T00:47:35.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:34 smithi038 ceph-mon[29271]: pgmap v308: 65 pgs: 65 active+clean; 56 MiB data, 335 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 110 op/s 2024-03-24T00:47:35.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:34 smithi088 ceph-mon[36610]: pgmap v308: 65 pgs: 65 active+clean; 56 MiB data, 335 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 110 op/s 2024-03-24T00:47:37.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:36 smithi038 ceph-mon[29271]: pgmap v309: 65 pgs: 65 active+clean; 58 MiB data, 350 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-03-24T00:47:37.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:36 smithi088 ceph-mon[36610]: pgmap v309: 65 pgs: 65 active+clean; 58 MiB data, 350 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 85 op/s 2024-03-24T00:47:40.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:39 smithi088 ceph-mon[36610]: pgmap v310: 65 pgs: 65 active+clean; 76 MiB data, 451 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 370 op/s 2024-03-24T00:47:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:39 smithi038 ceph-mon[29271]: pgmap v310: 65 pgs: 65 active+clean; 76 MiB data, 451 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 370 op/s 2024-03-24T00:47:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:40 smithi088 ceph-mon[36610]: pgmap v311: 65 pgs: 65 active+clean; 89 MiB data, 482 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 430 op/s 2024-03-24T00:47:41.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:40 smithi038 ceph-mon[29271]: pgmap v311: 65 pgs: 65 active+clean; 89 MiB data, 482 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 430 op/s 2024-03-24T00:47:43.301 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:42 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:43.302 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:42 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:43.302 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:42 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:47:43.302 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:42 smithi088 ceph-mon[36610]: pgmap v312: 65 pgs: 65 active+clean; 90 MiB data, 490 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 446 op/s 2024-03-24T00:47:43.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:42 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:43.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:42 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:43.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:42 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:47:43.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:42 smithi038 ceph-mon[29271]: pgmap v312: 65 pgs: 65 active+clean; 90 MiB data, 490 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 446 op/s 2024-03-24T00:47:44.265 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:43 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:47:44.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:43 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:47:45.056 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:44 smithi088 ceph-mon[36610]: pgmap v313: 65 pgs: 65 active+clean; 95 MiB data, 515 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 477 op/s 2024-03-24T00:47:45.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:44 smithi038 ceph-mon[29271]: pgmap v313: 65 pgs: 65 active+clean; 95 MiB data, 515 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 477 op/s 2024-03-24T00:47:47.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:47 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:47.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:47 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:47.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:47 smithi088 ceph-mon[36610]: pgmap v314: 65 pgs: 65 active+clean; 95 MiB data, 519 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 471 op/s 2024-03-24T00:47:47.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:47.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:47 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:47.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:47 smithi038 ceph-mon[29271]: pgmap v314: 65 pgs: 65 active+clean; 95 MiB data, 519 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 471 op/s 2024-03-24T00:47:48.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:48 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:48.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:48 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:48.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:48 smithi088 ceph-mon[36610]: pgmap v315: 65 pgs: 65 active+clean; 100 MiB data, 538 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 437 op/s 2024-03-24T00:47:49.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:48 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:49.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:48 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:49.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:48 smithi038 ceph-mon[29271]: pgmap v315: 65 pgs: 65 active+clean; 100 MiB data, 538 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 437 op/s 2024-03-24T00:47:50.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:50 smithi088 ceph-mon[36610]: pgmap v316: 65 pgs: 65 active+clean; 104 MiB data, 542 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 153 op/s 2024-03-24T00:47:51.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:50 smithi038 ceph-mon[29271]: pgmap v316: 65 pgs: 65 active+clean; 104 MiB data, 542 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 153 op/s 2024-03-24T00:47:52.060 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:51 smithi088 ceph-mon[36610]: Standby manager daemon smithi088.njctea restarted 2024-03-24T00:47:52.060 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:51 smithi088 ceph-mon[36610]: Standby manager daemon smithi088.njctea started 2024-03-24T00:47:52.060 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:51 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/crt"}]: dispatch 2024-03-24T00:47:52.060 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:51 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:47:52.060 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:51 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/key"}]: dispatch 2024-03-24T00:47:52.060 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:51 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:47:52.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:51 smithi038 ceph-mon[29271]: Standby manager daemon smithi088.njctea restarted 2024-03-24T00:47:52.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:51 smithi038 ceph-mon[29271]: Standby manager daemon smithi088.njctea started 2024-03-24T00:47:52.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:51 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/crt"}]: dispatch 2024-03-24T00:47:52.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:51 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:47:52.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:51 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/key"}]: dispatch 2024-03-24T00:47:52.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:51 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:47:53.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:52 smithi038 ceph-mon[29271]: mgrmap e20: smithi038.huhioz(active, since 9m), standbys: smithi088.njctea 2024-03-24T00:47:53.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:52 smithi038 ceph-mon[29271]: pgmap v317: 65 pgs: 65 active+clean; 105 MiB data, 546 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-03-24T00:47:53.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:52 smithi088 ceph-mon[36610]: mgrmap e20: smithi038.huhioz(active, since 9m), standbys: smithi088.njctea 2024-03-24T00:47:53.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:52 smithi088 ceph-mon[36610]: pgmap v317: 65 pgs: 65 active+clean; 105 MiB data, 546 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-03-24T00:47:53.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:53.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:53.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:47:53.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:47:53.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:53.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:47:53.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr fail", "who": "smithi038.huhioz"}]: dispatch 2024-03-24T00:47:53.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: osdmap e48: 6 total, 6 up, 6 in 2024-03-24T00:47:53.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "mgr fail", "who": "smithi038.huhioz"}]': finished 2024-03-24T00:47:53.987 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: mgrmap e21: smithi088.njctea(active, starting, since 0.0184858s) 2024-03-24T00:47:54.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:47:54.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:47:54.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:47:54.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T00:47:54.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T00:47:54.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:47:54.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mgr metadata", "who": "smithi088.njctea", "id": "smithi088.njctea"}]: dispatch 2024-03-24T00:47:54.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:47:54.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:47:54.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: Manager daemon smithi088.njctea is now available 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.njctea/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.njctea/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.njctea/trash_purge_schedule"}]: dispatch 2024-03-24T00:47:54.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:53 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.njctea/trash_purge_schedule"}]: dispatch 2024-03-24T00:47:54.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:54.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:54.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:47:54.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:47:54.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' 2024-03-24T00:47:54.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:47:54.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr fail", "who": "smithi038.huhioz"}]: dispatch 2024-03-24T00:47:54.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: osdmap e48: 6 total, 6 up, 6 in 2024-03-24T00:47:54.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.14215 172.21.15.38:0/4116531253' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "mgr fail", "who": "smithi038.huhioz"}]': finished 2024-03-24T00:47:54.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: mgrmap e21: smithi088.njctea(active, starting, since 0.0184858s) 2024-03-24T00:47:54.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mgr metadata", "who": "smithi088.njctea", "id": "smithi088.njctea"}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:47:54.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T00:47:54.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T00:47:54.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T00:47:54.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: Manager daemon smithi088.njctea is now available 2024-03-24T00:47:54.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:47:54.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:47:54.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:47:54.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.njctea/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:47:54.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.njctea/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:47:54.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.njctea/trash_purge_schedule"}]: dispatch 2024-03-24T00:47:54.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:53 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.njctea/trash_purge_schedule"}]: dispatch 2024-03-24T00:47:55.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:55 smithi038 ceph-mon[29271]: [24/Mar/2024:00:47:53] ENGINE Bus STARTING 2024-03-24T00:47:55.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:55 smithi038 ceph-mon[29271]: [24/Mar/2024:00:47:53] ENGINE Serving on https://172.21.15.88:7150 2024-03-24T00:47:55.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:55 smithi038 ceph-mon[29271]: [24/Mar/2024:00:47:54] ENGINE Serving on http://172.21.15.88:8765 2024-03-24T00:47:55.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:55 smithi038 ceph-mon[29271]: [24/Mar/2024:00:47:54] ENGINE Bus STARTED 2024-03-24T00:47:55.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:55 smithi038 ceph-mon[29271]: Deploying cephadm binary to smithi088 2024-03-24T00:47:55.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:55 smithi038 ceph-mon[29271]: Deploying cephadm binary to smithi038 2024-03-24T00:47:55.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:55 smithi038 ceph-mon[29271]: mgrmap e22: smithi088.njctea(active, since 1.02201s) 2024-03-24T00:47:55.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:55 smithi038 ceph-mon[29271]: pgmap v3: 65 pgs: 65 active+clean; 111 MiB data, 563 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:47:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:55 smithi088 ceph-mon[36610]: [24/Mar/2024:00:47:53] ENGINE Bus STARTING 2024-03-24T00:47:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:55 smithi088 ceph-mon[36610]: [24/Mar/2024:00:47:53] ENGINE Serving on https://172.21.15.88:7150 2024-03-24T00:47:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:55 smithi088 ceph-mon[36610]: [24/Mar/2024:00:47:54] ENGINE Serving on http://172.21.15.88:8765 2024-03-24T00:47:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:55 smithi088 ceph-mon[36610]: [24/Mar/2024:00:47:54] ENGINE Bus STARTED 2024-03-24T00:47:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:55 smithi088 ceph-mon[36610]: Deploying cephadm binary to smithi088 2024-03-24T00:47:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:55 smithi088 ceph-mon[36610]: Deploying cephadm binary to smithi038 2024-03-24T00:47:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:55 smithi088 ceph-mon[36610]: mgrmap e22: smithi088.njctea(active, since 1.02201s) 2024-03-24T00:47:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:55 smithi088 ceph-mon[36610]: pgmap v3: 65 pgs: 65 active+clean; 111 MiB data, 563 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:47:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.365+0000 7f6d0f400700 1 -- 172.21.15.38:0/3379594052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d08103f00 msgr2=0x7f6d08104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.365+0000 7f6d0f400700 1 --2- 172.21.15.38:0/3379594052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d08103f00 0x7f6d08104360 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009910 tx=0x7f6cf4009c20 comp rx=0 tx=0).stop 2024-03-24T00:47:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.366+0000 7f6d0f400700 1 -- 172.21.15.38:0/3379594052 shutdown_connections 2024-03-24T00:47:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.366+0000 7f6d0f400700 1 --2- 172.21.15.38:0/3379594052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d08103f00 0x7f6d08104360 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.366+0000 7f6d0f400700 1 --2- 172.21.15.38:0/3379594052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d08102d00 0x7f6d08103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.366+0000 7f6d0f400700 1 -- 172.21.15.38:0/3379594052 >> 172.21.15.38:0/3379594052 conn(0x7f6d080fe2a0 msgr2=0x7f6d081006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.367+0000 7f6d0f400700 1 -- 172.21.15.38:0/3379594052 shutdown_connections 2024-03-24T00:47:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.367+0000 7f6d0f400700 1 -- 172.21.15.38:0/3379594052 wait complete. 2024-03-24T00:47:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.368+0000 7f6d0f400700 1 Processor -- start 2024-03-24T00:47:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.368+0000 7f6d0f400700 1 -- start start 2024-03-24T00:47:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.368+0000 7f6d0f400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d08102d00 0x7f6d0819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.368+0000 7f6d0f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d08103f00 0x7f6d0819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.368+0000 7f6d0f400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0819d620 con 0x7f6d08103f00 2024-03-24T00:47:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.368+0000 7f6d0f400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0819d760 con 0x7f6d08102d00 2024-03-24T00:47:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.369+0000 7f6d07e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d08103f00 0x7f6d0819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.369+0000 7f6d07e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d08103f00 0x7f6d0819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45260/0 (socket says 172.21.15.38:45260) 2024-03-24T00:47:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.369+0000 7f6d07e00700 1 -- 172.21.15.38:0/4281705861 learned_addr learned my addr 172.21.15.38:0/4281705861 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:47:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.369+0000 7f6d0ca00700 1 --2- 172.21.15.38:0/4281705861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d08102d00 0x7f6d0819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.369+0000 7f6d07e00700 1 -- 172.21.15.38:0/4281705861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d08102d00 msgr2=0x7f6d0819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.369+0000 7f6d07e00700 1 --2- 172.21.15.38:0/4281705861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d08102d00 0x7f6d0819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.369+0000 7f6d07e00700 1 -- 172.21.15.38:0/4281705861 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf40095f0 con 0x7f6d08103f00 2024-03-24T00:47:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.370+0000 7f6d07e00700 1 --2- 172.21.15.38:0/4281705861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d08103f00 0x7f6d0819d000 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4006010 tx=0x7f6cf400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.370+0000 7f6d05600700 1 -- 172.21.15.38:0/4281705861 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cf401d070 con 0x7f6d08103f00 2024-03-24T00:47:56.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.370+0000 7f6d05600700 1 -- 172.21.15.38:0/4281705861 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cf4005dc0 con 0x7f6d08103f00 2024-03-24T00:47:56.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.370+0000 7f6d0f400700 1 -- 172.21.15.38:0/4281705861 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d081a2160 con 0x7f6d08103f00 2024-03-24T00:47:56.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.370+0000 7f6d05600700 1 -- 172.21.15.38:0/4281705861 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cf4021640 con 0x7f6d08103f00 2024-03-24T00:47:56.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.371+0000 7f6d0f400700 1 -- 172.21.15.38:0/4281705861 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d081a26d0 con 0x7f6d08103f00 2024-03-24T00:47:56.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.372+0000 7f6d05600700 1 -- 172.21.15.38:0/4281705861 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 23) v1 ==== 46189+0+0 (secure 0 0 0) 0x7f6cf400f880 con 0x7f6d08103f00 2024-03-24T00:47:56.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.372+0000 7f6d0f400700 1 -- 172.21.15.38:0/4281705861 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d08076990 con 0x7f6d08103f00 2024-03-24T00:47:56.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.373+0000 7f6d05600700 1 --2- 172.21.15.38:0/4281705861 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f6cf00391b0 0x7f6cf003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:56.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.373+0000 7f6d05600700 1 -- 172.21.15.38:0/4281705861 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f6cf404e5b0 con 0x7f6d08103f00 2024-03-24T00:47:56.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.374+0000 7f6d0ca00700 1 --2- 172.21.15.38:0/4281705861 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f6cf00391b0 0x7f6cf003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:56.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.375+0000 7f6d0ca00700 1 --2- 172.21.15.38:0/4281705861 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f6cf00391b0 0x7f6cf003b670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc005950 tx=0x7f6cfc0058e0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:56.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.379+0000 7f6d05600700 1 -- 172.21.15.38:0/4281705861 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f6cf400fe50 con 0x7f6d08103f00 2024-03-24T00:47:56.577 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:56 smithi038 ceph-mon[29271]: pgmap v4: 65 pgs: 65 active+clean; 111 MiB data, 563 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:47:56.577 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:56 smithi038 ceph-mon[29271]: mgrmap e23: smithi088.njctea(active, since 2s) 2024-03-24T00:47:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:56 smithi088 ceph-mon[36610]: pgmap v4: 65 pgs: 65 active+clean; 111 MiB data, 563 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:47:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:56 smithi088 ceph-mon[36610]: mgrmap e23: smithi088.njctea(active, since 2s) 2024-03-24T00:47:56.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.685+0000 7f6d0f400700 1 -- 172.21.15.38:0/4281705861 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d080611f0 con 0x7f6cf00391b0 2024-03-24T00:47:56.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.695+0000 7f6d05600700 1 -- 172.21.15.38:0/4281705861 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f6d080611f0 con 0x7f6cf00391b0 2024-03-24T00:47:56.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.697+0000 7f6d0f400700 1 -- 172.21.15.38:0/4281705861 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f6cf00391b0 msgr2=0x7f6cf003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:56.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.697+0000 7f6d0f400700 1 --2- 172.21.15.38:0/4281705861 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f6cf00391b0 0x7f6cf003b670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc005950 tx=0x7f6cfc0058e0 comp rx=0 tx=0).stop 2024-03-24T00:47:56.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.697+0000 7f6d0f400700 1 -- 172.21.15.38:0/4281705861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d08103f00 msgr2=0x7f6d0819d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:56.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.697+0000 7f6d0f400700 1 --2- 172.21.15.38:0/4281705861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d08103f00 0x7f6d0819d000 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4006010 tx=0x7f6cf400be20 comp rx=0 tx=0).stop 2024-03-24T00:47:56.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.697+0000 7f6d0f400700 1 -- 172.21.15.38:0/4281705861 shutdown_connections 2024-03-24T00:47:56.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.697+0000 7f6d0f400700 1 --2- 172.21.15.38:0/4281705861 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f6cf00391b0 0x7f6cf003b670 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:56.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.697+0000 7f6d0f400700 1 --2- 172.21.15.38:0/4281705861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d08102d00 0x7f6d0819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:56.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.697+0000 7f6d0f400700 1 --2- 172.21.15.38:0/4281705861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d08103f00 0x7f6d0819d000 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:56.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.697+0000 7f6d0f400700 1 -- 172.21.15.38:0/4281705861 >> 172.21.15.38:0/4281705861 conn(0x7f6d080fe2a0 msgr2=0x7f6d08107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:56.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.698+0000 7f6d0f400700 1 -- 172.21.15.38:0/4281705861 shutdown_connections 2024-03-24T00:47:56.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.698+0000 7f6d0f400700 1 -- 172.21.15.38:0/4281705861 wait complete. 2024-03-24T00:47:56.714 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:47:56.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.906+0000 7fdcc1000700 1 -- 172.21.15.38:0/2229667741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcbc102d00 msgr2=0x7fdcbc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:56.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.906+0000 7fdcc1000700 1 --2- 172.21.15.38:0/2229667741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcbc102d00 0x7fdcbc103120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009910 tx=0x7fdcac009c20 comp rx=0 tx=0).stop 2024-03-24T00:47:56.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.907+0000 7fdcc1000700 1 -- 172.21.15.38:0/2229667741 shutdown_connections 2024-03-24T00:47:56.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.907+0000 7fdcc1000700 1 --2- 172.21.15.38:0/2229667741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcbc103f00 0x7fdcbc104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:56.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.907+0000 7fdcc1000700 1 --2- 172.21.15.38:0/2229667741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcbc102d00 0x7fdcbc103120 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:56.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.907+0000 7fdcc1000700 1 -- 172.21.15.38:0/2229667741 >> 172.21.15.38:0/2229667741 conn(0x7fdcbc0fe2a0 msgr2=0x7fdcbc1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:56.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.907+0000 7fdcc1000700 1 -- 172.21.15.38:0/2229667741 shutdown_connections 2024-03-24T00:47:56.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.907+0000 7fdcc1000700 1 -- 172.21.15.38:0/2229667741 wait complete. 2024-03-24T00:47:56.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.908+0000 7fdcc1000700 1 Processor -- start 2024-03-24T00:47:56.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.908+0000 7fdcc1000700 1 -- start start 2024-03-24T00:47:56.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.908+0000 7fdcc1000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcbc102d00 0x7fdcbc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:56.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.909+0000 7fdcc1000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcbc103f00 0x7fdcbc198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:56.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.909+0000 7fdcc1000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc199240 con 0x7fdcbc102d00 2024-03-24T00:47:56.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.909+0000 7fdcc1000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc199380 con 0x7fdcbc103f00 2024-03-24T00:47:56.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.909+0000 7fdcb9e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcbc102d00 0x7fdcbc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:56.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.909+0000 7fdcb9e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcbc102d00 0x7fdcbc1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45274/0 (socket says 172.21.15.38:45274) 2024-03-24T00:47:56.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.909+0000 7fdcb9e00700 1 -- 172.21.15.38:0/560194106 learned_addr learned my addr 172.21.15.38:0/560194106 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:47:56.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.909+0000 7fdcb9400700 1 --2- 172.21.15.38:0/560194106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcbc103f00 0x7fdcbc198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:56.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.909+0000 7fdcb9e00700 1 -- 172.21.15.38:0/560194106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcbc103f00 msgr2=0x7fdcbc198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:56.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.909+0000 7fdcb9e00700 1 --2- 172.21.15.38:0/560194106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcbc103f00 0x7fdcbc198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:56.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.909+0000 7fdcb9e00700 1 -- 172.21.15.38:0/560194106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac0095f0 con 0x7fdcbc102d00 2024-03-24T00:47:56.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.910+0000 7fdcb9e00700 1 --2- 172.21.15.38:0/560194106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcbc102d00 0x7fdcbc1986e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00bd70 tx=0x7fdcac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:56.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.910+0000 7fdcb2a00700 1 -- 172.21.15.38:0/560194106 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcac01d070 con 0x7fdcbc102d00 2024-03-24T00:47:56.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.910+0000 7fdcb2a00700 1 -- 172.21.15.38:0/560194106 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcac005dc0 con 0x7fdcbc102d00 2024-03-24T00:47:56.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.910+0000 7fdcc1000700 1 -- 172.21.15.38:0/560194106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcbc19dd80 con 0x7fdcbc102d00 2024-03-24T00:47:56.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.910+0000 7fdcb2a00700 1 -- 172.21.15.38:0/560194106 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcac021410 con 0x7fdcbc102d00 2024-03-24T00:47:56.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.911+0000 7fdcc1000700 1 -- 172.21.15.38:0/560194106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcbc19e290 con 0x7fdcbc102d00 2024-03-24T00:47:56.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.912+0000 7fdcc1000700 1 -- 172.21.15.38:0/560194106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9c005320 con 0x7fdcbc102d00 2024-03-24T00:47:56.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.912+0000 7fdcb2a00700 1 -- 172.21.15.38:0/560194106 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 23) v1 ==== 46189+0+0 (secure 0 0 0) 0x7fdcac02b440 con 0x7fdcbc102d00 2024-03-24T00:47:56.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.913+0000 7fdcb2a00700 1 --2- 172.21.15.38:0/560194106 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fdca00391b0 0x7fdca003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:56.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.913+0000 7fdcb2a00700 1 -- 172.21.15.38:0/560194106 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fdcac04e4d0 con 0x7fdcbc102d00 2024-03-24T00:47:56.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.913+0000 7fdcb9400700 1 --2- 172.21.15.38:0/560194106 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fdca00391b0 0x7fdca003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:56.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.914+0000 7fdcb9400700 1 --2- 172.21.15.38:0/560194106 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fdca00391b0 0x7fdca003b670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdca8005950 tx=0x7fdca80058e0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:56.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:56.921+0000 7fdcb2a00700 1 -- 172.21.15.38:0/560194106 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fdcac029950 con 0x7fdcbc102d00 2024-03-24T00:47:57.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.355+0000 7fdcc1000700 1 -- 172.21.15.38:0/560194106 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc9c000bf0 con 0x7fdca00391b0 2024-03-24T00:47:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.362+0000 7fdcb2a00700 1 -- 172.21.15.38:0/560194106 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fdc9c000bf0 con 0x7fdca00391b0 2024-03-24T00:47:57.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.367+0000 7fdcc1000700 1 -- 172.21.15.38:0/560194106 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fdca00391b0 msgr2=0x7fdca003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:57.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.367+0000 7fdcc1000700 1 --2- 172.21.15.38:0/560194106 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fdca00391b0 0x7fdca003b670 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdca8005950 tx=0x7fdca80058e0 comp rx=0 tx=0).stop 2024-03-24T00:47:57.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.367+0000 7fdcc1000700 1 -- 172.21.15.38:0/560194106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcbc102d00 msgr2=0x7fdcbc1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:57.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.367+0000 7fdcc1000700 1 --2- 172.21.15.38:0/560194106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcbc102d00 0x7fdcbc1986e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00bd70 tx=0x7fdcac00be50 comp rx=0 tx=0).stop 2024-03-24T00:47:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.368+0000 7fdcc1000700 1 -- 172.21.15.38:0/560194106 shutdown_connections 2024-03-24T00:47:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.368+0000 7fdcc1000700 1 --2- 172.21.15.38:0/560194106 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fdca00391b0 0x7fdca003b670 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdca8005950 tx=0x7fdca80058e0 comp rx=0 tx=0).stop 2024-03-24T00:47:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.368+0000 7fdcc1000700 1 --2- 172.21.15.38:0/560194106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcbc102d00 0x7fdcbc1986e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.368+0000 7fdcc1000700 1 --2- 172.21.15.38:0/560194106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcbc103f00 0x7fdcbc198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.368+0000 7fdcc1000700 1 -- 172.21.15.38:0/560194106 >> 172.21.15.38:0/560194106 conn(0x7fdcbc0fe2a0 msgr2=0x7fdcbc107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:57.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.371+0000 7fdcc1000700 1 -- 172.21.15.38:0/560194106 shutdown_connections 2024-03-24T00:47:57.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.371+0000 7fdcc1000700 1 -- 172.21.15.38:0/560194106 wait complete. 2024-03-24T00:47:57.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:57 smithi038 ceph-mon[29271]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:57.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.637+0000 7fa3ba200700 1 -- 172.21.15.38:0/158347606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3b4072970 msgr2=0x7fa3b4107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:57.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.637+0000 7fa3ba200700 1 --2- 172.21.15.38:0/158347606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3b4072970 0x7fa3b4107db0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009870 tx=0x7fa3a4009b80 comp rx=0 tx=0).stop 2024-03-24T00:47:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.638+0000 7fa3ba200700 1 -- 172.21.15.38:0/158347606 shutdown_connections 2024-03-24T00:47:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.638+0000 7fa3ba200700 1 --2- 172.21.15.38:0/158347606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3b41082f0 0x7fa3b4108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.638+0000 7fa3ba200700 1 --2- 172.21.15.38:0/158347606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3b4072970 0x7fa3b4107db0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.638+0000 7fa3ba200700 1 -- 172.21.15.38:0/158347606 >> 172.21.15.38:0/158347606 conn(0x7fa3b406da40 msgr2=0x7fa3b406fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.638+0000 7fa3ba200700 1 -- 172.21.15.38:0/158347606 shutdown_connections 2024-03-24T00:47:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.638+0000 7fa3ba200700 1 -- 172.21.15.38:0/158347606 wait complete. 2024-03-24T00:47:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.640+0000 7fa3ba200700 1 Processor -- start 2024-03-24T00:47:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.640+0000 7fa3ba200700 1 -- start start 2024-03-24T00:47:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.642+0000 7fa3ba200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3b4072970 0x7fa3b419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:57.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.642+0000 7fa3ba200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3b41082f0 0x7fa3b419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:57.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.643+0000 7fa3ba200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b419f7f0 con 0x7fa3b41082f0 2024-03-24T00:47:57.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.643+0000 7fa3ba200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b419f960 con 0x7fa3b4072970 2024-03-24T00:47:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.642+0000 7fa3b3400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3b4072970 0x7fa3b419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.643+0000 7fa3b3400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3b4072970 0x7fa3b419ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:47040/0 (socket says 172.21.15.38:47040) 2024-03-24T00:47:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.643+0000 7fa3b3400700 1 -- 172.21.15.38:0/2254845545 learned_addr learned my addr 172.21.15.38:0/2254845545 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:47:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.643+0000 7fa3b2a00700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3b41082f0 0x7fa3b419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.645+0000 7fa3b3400700 1 -- 172.21.15.38:0/2254845545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3b41082f0 msgr2=0x7fa3b419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.645+0000 7fa3b3400700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3b41082f0 0x7fa3b419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.645+0000 7fa3b3400700 1 -- 172.21.15.38:0/2254845545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4009520 con 0x7fa3b4072970 2024-03-24T00:47:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.645+0000 7fa3b2a00700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3b41082f0 0x7fa3b419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T00:47:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.645+0000 7fa3b3400700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3b4072970 0x7fa3b419ecb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400b820 tx=0x7fa3a400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:57.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.649+0000 7fa3abe00700 1 -- 172.21.15.38:0/2254845545 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3a401c070 con 0x7fa3b4072970 2024-03-24T00:47:57.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.649+0000 7fa3abe00700 1 -- 172.21.15.38:0/2254845545 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3a400fcf0 con 0x7fa3b4072970 2024-03-24T00:47:57.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.649+0000 7fa3abe00700 1 -- 172.21.15.38:0/2254845545 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3a4022e30 con 0x7fa3b4072970 2024-03-24T00:47:57.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.649+0000 7fa3ba200700 1 -- 172.21.15.38:0/2254845545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b41a4340 con 0x7fa3b4072970 2024-03-24T00:47:57.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.649+0000 7fa3ba200700 1 -- 172.21.15.38:0/2254845545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b41a4860 con 0x7fa3b4072970 2024-03-24T00:47:57.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.651+0000 7fa3ba200700 1 -- 172.21.15.38:0/2254845545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b4198e70 con 0x7fa3b4072970 2024-03-24T00:47:57.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.652+0000 7fa3abe00700 1 -- 172.21.15.38:0/2254845545 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) v1 ==== 46245+0+0 (secure 0 0 0) 0x7fa3a4020070 con 0x7fa3b4072970 2024-03-24T00:47:57.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.653+0000 7fa3abe00700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fa3980392d0 0x7fa39803b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:57.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.653+0000 7fa3abe00700 1 -- 172.21.15.38:0/2254845545 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa3a404e6d0 con 0x7fa3b4072970 2024-03-24T00:47:57.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.653+0000 7fa3b2a00700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fa3980392d0 0x7fa39803b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:57.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.657+0000 7fa3b2a00700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fa3980392d0 0x7fa39803b790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3b41a0170 tx=0x7fa39c00b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:57.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:57.660+0000 7fa3abe00700 1 -- 172.21.15.38:0/2254845545 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fa3a4016970 con 0x7fa3b4072970 2024-03-24T00:47:57.679 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:57 smithi088 ceph-mon[36610]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:58.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.103+0000 7fa3ba200700 1 -- 172.21.15.38:0/2254845545 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3b40611f0 con 0x7fa3980392d0 2024-03-24T00:47:58.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.171+0000 7fa3abe00700 1 -- 172.21.15.38:0/2254845545 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa3b40611f0 con 0x7fa3980392d0 2024-03-24T00:47:58.172 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:47:58.172 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (8m) 2m ago 10m 13.4M - 0.25.0 c8568f914cd2 9f1243fbcb3e 2024-03-24T00:47:58.172 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (10m) 2m ago 10m 7658k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:47:58.172 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (9m) 12s ago 9m 7704k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:47:58.172 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (10m) 2m ago 10m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:47:58.172 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (9m) 12s ago 9m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:47:58.172 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (8m) 2m ago 9m 72.7M - 9.4.7 954c08fa6188 e6aa268d69e8 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (2m) 2m ago 2m 13.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (3m) 2m ago 3m 15.0M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (2m) 12s ago 2m 19.2M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (2m) 12s ago 2m 18.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:9283,8765,8443 running (12m) 2m ago 12m 498M - 18.2.2-952-g738c2fd6 68aff9aff3b2 bf370183b69d 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (16s) 12s ago 8m 61.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b21fee18bc67 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (12m) 2m ago 12m 45.0M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (8m) 12s ago 8m 34.1M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (10m) 2m ago 10m 8108k - 1.5.0 0da6a335fe13 c34cc077c93f 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (9m) 12s ago 9m 8090k - 1.5.0 0da6a335fe13 189e840ee2f1 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (7m) 2m ago 7m 46.7M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (7m) 2m ago 7m 47.1M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (6m) 2m ago 6m 45.5M 3949M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:47:58.173 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (6m) 12s ago 6m 90.9M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:47:58.174 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (5m) 12s ago 5m 87.4M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:47:58.174 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (4m) 12s ago 4m 85.0M 4971M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:47:58.174 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (8m) 2m ago 9m 49.7M - 2.43.0 a07b618ecd1d c59e5c711ace 2024-03-24T00:47:58.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.174+0000 7fa3ba200700 1 -- 172.21.15.38:0/2254845545 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fa3980392d0 msgr2=0x7fa39803b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.174+0000 7fa3ba200700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fa3980392d0 0x7fa39803b790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3b41a0170 tx=0x7fa39c00b040 comp rx=0 tx=0).stop 2024-03-24T00:47:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.174+0000 7fa3ba200700 1 -- 172.21.15.38:0/2254845545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3b4072970 msgr2=0x7fa3b419ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.174+0000 7fa3ba200700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3b4072970 0x7fa3b419ecb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400b820 tx=0x7fa3a400f690 comp rx=0 tx=0).stop 2024-03-24T00:47:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.175+0000 7fa3ba200700 1 -- 172.21.15.38:0/2254845545 shutdown_connections 2024-03-24T00:47:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.175+0000 7fa3ba200700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fa3980392d0 0x7fa39803b790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.175+0000 7fa3ba200700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3b4072970 0x7fa3b419ecb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.175+0000 7fa3ba200700 1 --2- 172.21.15.38:0/2254845545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3b41082f0 0x7fa3b419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.175+0000 7fa3ba200700 1 -- 172.21.15.38:0/2254845545 >> 172.21.15.38:0/2254845545 conn(0x7fa3b406da40 msgr2=0x7fa3b410d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.175+0000 7fa3ba200700 1 -- 172.21.15.38:0/2254845545 shutdown_connections 2024-03-24T00:47:58.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.175+0000 7fa3ba200700 1 -- 172.21.15.38:0/2254845545 wait complete. 2024-03-24T00:47:58.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.358+0000 7f025e200700 1 -- 172.21.15.38:0/4219480727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0258103cf0 msgr2=0x7f02581060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:58.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.358+0000 7f025e200700 1 --2- 172.21.15.38:0/4219480727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0258103cf0 0x7f02581060e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0244009910 tx=0x7f0244009c20 comp rx=0 tx=0).stop 2024-03-24T00:47:58.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.358+0000 7f025e200700 1 -- 172.21.15.38:0/4219480727 shutdown_connections 2024-03-24T00:47:58.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.358+0000 7f025e200700 1 --2- 172.21.15.38:0/4219480727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0258103cf0 0x7f02581060e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:58.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.358+0000 7f025e200700 1 --2- 172.21.15.38:0/4219480727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02581013c0 0x7f02581037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:58.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.358+0000 7f025e200700 1 -- 172.21.15.38:0/4219480727 >> 172.21.15.38:0/4219480727 conn(0x7f02580fadf0 msgr2=0x7f02580fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:58.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.359+0000 7f025e200700 1 -- 172.21.15.38:0/4219480727 shutdown_connections 2024-03-24T00:47:58.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.359+0000 7f025e200700 1 -- 172.21.15.38:0/4219480727 wait complete. 2024-03-24T00:47:58.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.360+0000 7f025e200700 1 Processor -- start 2024-03-24T00:47:58.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.361+0000 7f025e200700 1 -- start start 2024-03-24T00:47:58.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.361+0000 7f025e200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02581013c0 0x7f0258071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:58.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.361+0000 7f025e200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258103cf0 0x7f0258072310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:58.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.361+0000 7f025e200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0258072930 con 0x7f02581013c0 2024-03-24T00:47:58.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.361+0000 7f025e200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0258072a70 con 0x7f0258103cf0 2024-03-24T00:47:58.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.362+0000 7f025ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02581013c0 0x7f0258071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:58.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.362+0000 7f0257e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258103cf0 0x7f0258072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:58.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.362+0000 7f025ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02581013c0 0x7f0258071dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45308/0 (socket says 172.21.15.38:45308) 2024-03-24T00:47:58.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.362+0000 7f025ce00700 1 -- 172.21.15.38:0/1951868284 learned_addr learned my addr 172.21.15.38:0/1951868284 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:47:58.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.363+0000 7f025ce00700 1 -- 172.21.15.38:0/1951868284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258103cf0 msgr2=0x7f0258072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:58.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.363+0000 7f025ce00700 1 --2- 172.21.15.38:0/1951868284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258103cf0 0x7f0258072310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:58.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.363+0000 7f025ce00700 1 -- 172.21.15.38:0/1951868284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02440095f0 con 0x7f02581013c0 2024-03-24T00:47:58.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.363+0000 7f025ce00700 1 --2- 172.21.15.38:0/1951868284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02581013c0 0x7f0258071dd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f024c00d8d0 tx=0x7f024c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:58.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.364+0000 7f0255600700 1 -- 172.21.15.38:0/1951868284 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f024c0098a0 con 0x7f02581013c0 2024-03-24T00:47:58.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.364+0000 7f0255600700 1 -- 172.21.15.38:0/1951868284 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f024c010460 con 0x7f02581013c0 2024-03-24T00:47:58.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.364+0000 7f025e200700 1 -- 172.21.15.38:0/1951868284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02580ff680 con 0x7f02581013c0 2024-03-24T00:47:58.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.364+0000 7f0255600700 1 -- 172.21.15.38:0/1951868284 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f024c00f5f0 con 0x7f02581013c0 2024-03-24T00:47:58.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.365+0000 7f025e200700 1 -- 172.21.15.38:0/1951868284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02580ffc50 con 0x7f02581013c0 2024-03-24T00:47:58.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.367+0000 7f0255600700 1 -- 172.21.15.38:0/1951868284 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 24) v1 ==== 46245+0+0 (secure 0 0 0) 0x7f024c009a00 con 0x7f02581013c0 2024-03-24T00:47:58.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.368+0000 7f025e200700 1 -- 172.21.15.38:0/1951868284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025804f2d0 con 0x7f02581013c0 2024-03-24T00:47:58.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.369+0000 7f0255600700 1 --2- 172.21.15.38:0/1951868284 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f02400392d0 0x7f024003b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:58.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.369+0000 7f0255600700 1 -- 172.21.15.38:0/1951868284 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f024c04c930 con 0x7f02581013c0 2024-03-24T00:47:58.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.371+0000 7f0257e00700 1 --2- 172.21.15.38:0/1951868284 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f02400392d0 0x7f024003b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:58.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.373+0000 7f0257e00700 1 --2- 172.21.15.38:0/1951868284 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f02400392d0 0x7f024003b790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0244006010 tx=0x7f0244005180 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:58.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.384+0000 7f0255600700 1 -- 172.21.15.38:0/1951868284 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f024c017260 con 0x7f02581013c0 2024-03-24T00:47:58.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:58 smithi088 ceph-mon[36610]: pgmap v5: 65 pgs: 65 active+clean; 111 MiB data, 563 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:47:58.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:58 smithi088 ceph-mon[36610]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:58.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:58 smithi088 ceph-mon[36610]: mgrmap e24: smithi088.njctea(active, since 4s) 2024-03-24T00:47:58.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:58 smithi038 ceph-mon[29271]: pgmap v5: 65 pgs: 65 active+clean; 111 MiB data, 563 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:47:58.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:58 smithi038 ceph-mon[29271]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:58.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:58 smithi038 ceph-mon[29271]: mgrmap e24: smithi088.njctea(active, since 4s) 2024-03-24T00:47:58.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.875+0000 7f025e200700 1 -- 172.21.15.38:0/1951868284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f02581000e0 con 0x7f02581013c0 2024-03-24T00:47:58.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.876+0000 7f0255600700 1 -- 172.21.15.38:0/1951868284 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f024c014110 con 0x7f02581013c0 2024-03-24T00:47:58.877 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:47:58.877 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:47:58.877 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:47:58.877 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:47:58.877 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:47:58.877 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 12, 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:47:58.878 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:47:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.879+0000 7f025e200700 1 -- 172.21.15.38:0/1951868284 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f02400392d0 msgr2=0x7f024003b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.879+0000 7f025e200700 1 --2- 172.21.15.38:0/1951868284 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f02400392d0 0x7f024003b790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0244006010 tx=0x7f0244005180 comp rx=0 tx=0).stop 2024-03-24T00:47:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.879+0000 7f025e200700 1 -- 172.21.15.38:0/1951868284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02581013c0 msgr2=0x7f0258071dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.879+0000 7f025e200700 1 --2- 172.21.15.38:0/1951868284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02581013c0 0x7f0258071dd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f024c00d8d0 tx=0x7f024c00dbe0 comp rx=0 tx=0).stop 2024-03-24T00:47:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.880+0000 7f025e200700 1 -- 172.21.15.38:0/1951868284 shutdown_connections 2024-03-24T00:47:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.880+0000 7f025e200700 1 --2- 172.21.15.38:0/1951868284 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f02400392d0 0x7f024003b790 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.880+0000 7f025e200700 1 --2- 172.21.15.38:0/1951868284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02581013c0 0x7f0258071dd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.880+0000 7f025e200700 1 --2- 172.21.15.38:0/1951868284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258103cf0 0x7f0258072310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.880+0000 7f025e200700 1 -- 172.21.15.38:0/1951868284 >> 172.21.15.38:0/1951868284 conn(0x7f02580fadf0 msgr2=0x7f02580fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.880+0000 7f025e200700 1 -- 172.21.15.38:0/1951868284 shutdown_connections 2024-03-24T00:47:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:58.880+0000 7f025e200700 1 -- 172.21.15.38:0/1951868284 wait complete. 2024-03-24T00:47:59.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.054+0000 7fbeebc00700 1 -- 172.21.15.38:0/2652678087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4073810 msgr2=0x7fbee4073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:59.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.054+0000 7fbeebc00700 1 --2- 172.21.15.38:0/2652678087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4073810 0x7fbee4073c90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009910 tx=0x7fbed0009c20 comp rx=0 tx=0).stop 2024-03-24T00:47:59.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.054+0000 7fbeebc00700 1 -- 172.21.15.38:0/2652678087 shutdown_connections 2024-03-24T00:47:59.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.054+0000 7fbeebc00700 1 --2- 172.21.15.38:0/2652678087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4073810 0x7fbee4073c90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:59.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.054+0000 7fbeebc00700 1 --2- 172.21.15.38:0/2652678087 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee4074de0 0x7fbee4073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:59.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.054+0000 7fbeebc00700 1 -- 172.21.15.38:0/2652678087 >> 172.21.15.38:0/2652678087 conn(0x7fbee40fc090 msgr2=0x7fbee40fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:59.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.054+0000 7fbeebc00700 1 -- 172.21.15.38:0/2652678087 shutdown_connections 2024-03-24T00:47:59.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.054+0000 7fbeebc00700 1 -- 172.21.15.38:0/2652678087 wait complete. 2024-03-24T00:47:59.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.055+0000 7fbeebc00700 1 Processor -- start 2024-03-24T00:47:59.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.056+0000 7fbeebc00700 1 -- start start 2024-03-24T00:47:59.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.056+0000 7fbeebc00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee4073810 0x7fbee419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:59.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.056+0000 7fbeebc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4074de0 0x7fbee419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:59.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.056+0000 7fbeebc00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee419d630 con 0x7fbee4074de0 2024-03-24T00:47:59.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.056+0000 7fbeebc00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee419d770 con 0x7fbee4073810 2024-03-24T00:47:59.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.056+0000 7fbee3e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4074de0 0x7fbee419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:59.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.056+0000 7fbee3e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4074de0 0x7fbee419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45336/0 (socket says 172.21.15.38:45336) 2024-03-24T00:47:59.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.056+0000 7fbee3e00700 1 -- 172.21.15.38:0/206434906 learned_addr learned my addr 172.21.15.38:0/206434906 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:47:59.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.057+0000 7fbee9200700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee4073810 0x7fbee419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:59.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.057+0000 7fbee3e00700 1 -- 172.21.15.38:0/206434906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee4073810 msgr2=0x7fbee419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:59.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.057+0000 7fbee3e00700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee4073810 0x7fbee419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:59.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.057+0000 7fbee3e00700 1 -- 172.21.15.38:0/206434906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed00095f0 con 0x7fbee4074de0 2024-03-24T00:47:59.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.057+0000 7fbee9200700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee4073810 0x7fbee419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:47:59.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.057+0000 7fbee3e00700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4074de0 0x7fbee419d010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbed000bd70 tx=0x7fbed000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:59.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.058+0000 7fbee1600700 1 -- 172.21.15.38:0/206434906 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbed001d070 con 0x7fbee4074de0 2024-03-24T00:47:59.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.058+0000 7fbee1600700 1 -- 172.21.15.38:0/206434906 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbed0005dc0 con 0x7fbee4074de0 2024-03-24T00:47:59.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.058+0000 7fbee1600700 1 -- 172.21.15.38:0/206434906 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbed0021410 con 0x7fbee4074de0 2024-03-24T00:47:59.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.058+0000 7fbeebc00700 1 -- 172.21.15.38:0/206434906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee41a2170 con 0x7fbee4074de0 2024-03-24T00:47:59.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.058+0000 7fbeebc00700 1 -- 172.21.15.38:0/206434906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee41a26e0 con 0x7fbee4074de0 2024-03-24T00:47:59.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.059+0000 7fbeebc00700 1 -- 172.21.15.38:0/206434906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee404ea80 con 0x7fbee4074de0 2024-03-24T00:47:59.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.059+0000 7fbee1600700 1 -- 172.21.15.38:0/206434906 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 24) v1 ==== 46245+0+0 (secure 0 0 0) 0x7fbed000f820 con 0x7fbee4074de0 2024-03-24T00:47:59.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.060+0000 7fbee1600700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fbecc039230 0x7fbecc03b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:47:59.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.060+0000 7fbee1600700 1 -- 172.21.15.38:0/206434906 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbed004e590 con 0x7fbee4074de0 2024-03-24T00:47:59.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.061+0000 7fbee9200700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fbecc039230 0x7fbecc03b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:47:59.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.062+0000 7fbee9200700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fbecc039230 0x7fbecc03b6f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbed8005950 tx=0x7fbed80058e0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:47:59.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.070+0000 7fbee1600700 1 -- 172.21.15.38:0/206434906 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fbed0029540 con 0x7fbee4074de0 2024-03-24T00:47:59.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.384+0000 7fbeebc00700 1 -- 172.21.15.38:0/206434906 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbee4103ad0 con 0x7fbecc039230 2024-03-24T00:47:59.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.387+0000 7fbee1600700 1 -- 172.21.15.38:0/206434906 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fbee4103ad0 con 0x7fbecc039230 2024-03-24T00:47:59.388 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:47:59.388 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:47:59.388 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:47:59.388 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-24T00:47:59.388 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [], 2024-03-24T00:47:59.389 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "1/2 daemons upgraded", 2024-03-24T00:47:59.389 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "", 2024-03-24T00:47:59.389 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:47:59.389 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:47:59.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.391+0000 7fbeebc00700 1 -- 172.21.15.38:0/206434906 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fbecc039230 msgr2=0x7fbecc03b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:59.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.391+0000 7fbeebc00700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fbecc039230 0x7fbecc03b6f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbed8005950 tx=0x7fbed80058e0 comp rx=0 tx=0).stop 2024-03-24T00:47:59.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.391+0000 7fbeebc00700 1 -- 172.21.15.38:0/206434906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4074de0 msgr2=0x7fbee419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:47:59.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.391+0000 7fbeebc00700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4074de0 0x7fbee419d010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbed000bd70 tx=0x7fbed000be50 comp rx=0 tx=0).stop 2024-03-24T00:47:59.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.391+0000 7fbeebc00700 1 -- 172.21.15.38:0/206434906 shutdown_connections 2024-03-24T00:47:59.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.391+0000 7fbeebc00700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fbecc039230 0x7fbecc03b6f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:59.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.391+0000 7fbeebc00700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee4073810 0x7fbee419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:47:59.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.391+0000 7fbeebc00700 1 --2- 172.21.15.38:0/206434906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4074de0 0x7fbee419d010 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbed000bd70 tx=0x7fbed000be50 comp rx=0 tx=0).stop 2024-03-24T00:47:59.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.391+0000 7fbeebc00700 1 -- 172.21.15.38:0/206434906 >> 172.21.15.38:0/206434906 conn(0x7fbee40fc090 msgr2=0x7fbee41023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:47:59.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.391+0000 7fbeebc00700 1 -- 172.21.15.38:0/206434906 shutdown_connections 2024-03-24T00:47:59.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:47:59.392+0000 7fbeebc00700 1 -- 172.21.15.38:0/206434906 wait complete. 2024-03-24T00:47:59.988 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:59 smithi038 ceph-mon[29271]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:47:59.988 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:47:59 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1951868284' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:48:00.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:59 smithi088 ceph-mon[36610]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:48:00.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:47:59 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1951868284' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:48:00.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:00 smithi038 ceph-mon[29271]: pgmap v6: 65 pgs: 65 active+clean; 123 MiB data, 617 MiB used, 536 GiB / 536 GiB avail; 599 KiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-03-24T00:48:00.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:00 smithi038 ceph-mon[29271]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:48:01.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:00 smithi088 ceph-mon[36610]: pgmap v6: 65 pgs: 65 active+clean; 123 MiB data, 617 MiB used, 536 GiB / 536 GiB avail; 599 KiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-03-24T00:48:01.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:00 smithi088 ceph-mon[36610]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:48:03.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:02 smithi038 ceph-mon[29271]: pgmap v7: 65 pgs: 65 active+clean; 137 MiB data, 671 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 624 op/s 2024-03-24T00:48:03.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:02 smithi088 ceph-mon[36610]: pgmap v7: 65 pgs: 65 active+clean; 137 MiB data, 671 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 624 op/s 2024-03-24T00:48:05.001 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:04 smithi038 ceph-mon[29271]: pgmap v8: 65 pgs: 65 active+clean; 137 MiB data, 671 MiB used, 536 GiB / 536 GiB avail; 863 KiB/s rd, 3.2 MiB/s wr, 485 op/s 2024-03-24T00:48:05.001 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:04 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:05.001 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:04 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:04 smithi088 ceph-mon[36610]: pgmap v8: 65 pgs: 65 active+clean; 137 MiB data, 671 MiB used, 536 GiB / 536 GiB avail; 863 KiB/s rd, 3.2 MiB/s wr, 485 op/s 2024-03-24T00:48:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:04 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:04 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:05.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:05 smithi038 ceph-mon[29271]: Standby manager daemon smithi038.huhioz started 2024-03-24T00:48:05.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:05 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.38:0/2390665075' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.huhioz/crt"}]: dispatch 2024-03-24T00:48:05.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:05 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.38:0/2390665075' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:48:05.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:05 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.38:0/2390665075' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.huhioz/key"}]: dispatch 2024-03-24T00:48:05.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:05 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.38:0/2390665075' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:48:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:05 smithi088 ceph-mon[36610]: Standby manager daemon smithi038.huhioz started 2024-03-24T00:48:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:05 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.38:0/2390665075' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.huhioz/crt"}]: dispatch 2024-03-24T00:48:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:05 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.38:0/2390665075' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:48:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:05 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.38:0/2390665075' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.huhioz/key"}]: dispatch 2024-03-24T00:48:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:05 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.38:0/2390665075' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:48:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:06 smithi038 ceph-mon[29271]: pgmap v9: 65 pgs: 65 active+clean; 145 MiB data, 702 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 430 op/s 2024-03-24T00:48:07.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:06 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mgr metadata", "who": "smithi038.huhioz", "id": "smithi038.huhioz"}]: dispatch 2024-03-24T00:48:07.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:06 smithi038 ceph-mon[29271]: mgrmap e25: smithi088.njctea(active, since 12s), standbys: smithi038.huhioz 2024-03-24T00:48:07.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:06 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:07.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:06 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:06 smithi088 ceph-mon[36610]: pgmap v9: 65 pgs: 65 active+clean; 145 MiB data, 702 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 430 op/s 2024-03-24T00:48:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:06 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mgr metadata", "who": "smithi038.huhioz", "id": "smithi038.huhioz"}]: dispatch 2024-03-24T00:48:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:06 smithi088 ceph-mon[36610]: mgrmap e25: smithi088.njctea(active, since 12s), standbys: smithi038.huhioz 2024-03-24T00:48:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:06 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:06 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:09.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:08 smithi038 ceph-mon[29271]: pgmap v10: 65 pgs: 65 active+clean; 145 MiB data, 702 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 394 op/s 2024-03-24T00:48:09.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:08 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:48:09.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:08 smithi088 ceph-mon[36610]: pgmap v10: 65 pgs: 65 active+clean; 145 MiB data, 702 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 394 op/s 2024-03-24T00:48:09.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:08 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:48:10.786 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:10 smithi038 ceph-mon[29271]: pgmap v11: 65 pgs: 65 active+clean; 148 MiB data, 718 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 400 op/s 2024-03-24T00:48:11.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:10 smithi088 ceph-mon[36610]: pgmap v11: 65 pgs: 65 active+clean; 148 MiB data, 718 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 400 op/s 2024-03-24T00:48:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:12 smithi038 ceph-mon[29271]: pgmap v12: 65 pgs: 65 active+clean; 152 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 237 op/s 2024-03-24T00:48:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:12 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:12 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:12 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:12 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:12 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:12 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:12 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:12 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:12 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:13.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:12 smithi088 ceph-mon[36610]: pgmap v12: 65 pgs: 65 active+clean; 152 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 237 op/s 2024-03-24T00:48:13.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:12 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:13.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:12 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:13.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:12 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:12 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:12 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:12 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:12 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:12 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:13.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:12 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:14.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:13 smithi038 ceph-mon[29271]: Adjusting osd_memory_target on smithi088 to 2241M 2024-03-24T00:48:14.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:13 smithi038 ceph-mon[29271]: pgmap v13: 65 pgs: 65 active+clean; 152 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 37 op/s 2024-03-24T00:48:14.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:13 smithi088 ceph-mon[36610]: Adjusting osd_memory_target on smithi088 to 2241M 2024-03-24T00:48:14.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:13 smithi088 ceph-mon[36610]: pgmap v13: 65 pgs: 65 active+clean; 152 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 37 op/s 2024-03-24T00:48:15.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:15 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:15.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:15 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:15.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:15 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:15.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:15 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:16.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:16 smithi038 ceph-mon[29271]: pgmap v14: 65 pgs: 65 active+clean; 180 MiB data, 844 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 366 op/s 2024-03-24T00:48:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:16 smithi088 ceph-mon[36610]: pgmap v14: 65 pgs: 65 active+clean; 180 MiB data, 844 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 366 op/s 2024-03-24T00:48:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: Adjusting osd_memory_target on smithi038 to 1218M 2024-03-24T00:48:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:48:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:48:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: Updating smithi038:/etc/ceph/ceph.conf 2024-03-24T00:48:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:17 smithi038 ceph-mon[29271]: Updating smithi088:/etc/ceph/ceph.conf 2024-03-24T00:48:17.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:17.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:17.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T00:48:17.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: Adjusting osd_memory_target on smithi038 to 1218M 2024-03-24T00:48:17.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:17.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:48:17.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:48:17.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: Updating smithi038:/etc/ceph/ceph.conf 2024-03-24T00:48:17.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:17 smithi088 ceph-mon[36610]: Updating smithi088:/etc/ceph/ceph.conf 2024-03-24T00:48:18.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:18 smithi038 ceph-mon[29271]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:48:18.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:18 smithi038 ceph-mon[29271]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:48:18.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:18 smithi038 ceph-mon[29271]: pgmap v15: 65 pgs: 65 active+clean; 180 MiB data, 844 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 335 op/s 2024-03-24T00:48:18.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:18 smithi088 ceph-mon[36610]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:48:18.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:18 smithi088 ceph-mon[36610]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:48:18.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:18 smithi088 ceph-mon[36610]: pgmap v15: 65 pgs: 65 active+clean; 180 MiB data, 844 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 335 op/s 2024-03-24T00:48:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:19 smithi038 ceph-mon[29271]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:48:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:19 smithi038 ceph-mon[29271]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:48:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:19 smithi038 ceph-mon[29271]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:48:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:19 smithi038 ceph-mon[29271]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:48:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:19 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:19 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:19 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:19 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:19 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:20.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:19 smithi088 ceph-mon[36610]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:48:20.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:19 smithi088 ceph-mon[36610]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:48:20.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:19 smithi088 ceph-mon[36610]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:48:20.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:19 smithi088 ceph-mon[36610]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:48:20.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:19 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:20.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:19 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:20.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:19 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:20.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:19 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:20.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:19 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:20.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:20 smithi088 ceph-mon[36610]: pgmap v16: 65 pgs: 65 active+clean; 186 MiB data, 887 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 378 op/s 2024-03-24T00:48:20.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:20 smithi088 ceph-mon[36610]: Reconfiguring prometheus.smithi038 (dependencies changed)... 2024-03-24T00:48:21.043 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:20 smithi038 ceph-mon[29271]: pgmap v16: 65 pgs: 65 active+clean; 186 MiB data, 887 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 378 op/s 2024-03-24T00:48:21.043 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:20 smithi038 ceph-mon[29271]: Reconfiguring prometheus.smithi038 (dependencies changed)... 2024-03-24T00:48:22.096 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:21 smithi038 ceph-mon[29271]: Reconfiguring daemon prometheus.smithi038 on smithi038 2024-03-24T00:48:22.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:21 smithi088 ceph-mon[36610]: Reconfiguring daemon prometheus.smithi038 on smithi038 2024-03-24T00:48:22.955 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:22 smithi038 ceph-mon[29271]: pgmap v17: 65 pgs: 65 active+clean; 197 MiB data, 904 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 389 op/s 2024-03-24T00:48:23.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:22 smithi088 ceph-mon[36610]: pgmap v17: 65 pgs: 65 active+clean; 197 MiB data, 904 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 389 op/s 2024-03-24T00:48:23.959 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:23 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:48:24.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:23 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:48:25.032 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:24 smithi038 ceph-mon[29271]: pgmap v18: 65 pgs: 65 active+clean; 197 MiB data, 904 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 387 op/s 2024-03-24T00:48:25.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:24 smithi088 ceph-mon[36610]: pgmap v18: 65 pgs: 65 active+clean; 197 MiB data, 904 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 387 op/s 2024-03-24T00:48:25.655 INFO:tasks.workunit.client.0.smithi038.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-24T00:48:25.656 INFO:tasks.workunit.client.0.smithi038.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-24T00:48:25.656 INFO:tasks.workunit.client.0.smithi038.stderr:+ make 2024-03-24T00:48:27.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:26 smithi038 ceph-mon[29271]: pgmap v19: 65 pgs: 65 active+clean; 205 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 400 op/s 2024-03-24T00:48:27.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:26 smithi088 ceph-mon[36610]: pgmap v19: 65 pgs: 65 active+clean; 205 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 400 op/s 2024-03-24T00:48:28.130 INFO:tasks.workunit.client.1.smithi088.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-24T00:48:28.164 INFO:tasks.workunit.client.1.smithi088.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-24T00:48:28.165 INFO:tasks.workunit.client.1.smithi088.stderr:+ make 2024-03-24T00:48:28.874 INFO:tasks.workunit.client.1.smithi088.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-24T00:48:28.875 INFO:tasks.workunit.client.0.smithi038.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-24T00:48:29.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:28 smithi038 ceph-mon[29271]: pgmap v20: 65 pgs: 65 active+clean; 205 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 72 op/s 2024-03-24T00:48:29.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:28 smithi088 ceph-mon[36610]: pgmap v20: 65 pgs: 65 active+clean; 205 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 72 op/s 2024-03-24T00:48:29.228 INFO:tasks.workunit.client.0.smithi038.stderr:++ readlink -f fsstress 2024-03-24T00:48:29.230 INFO:tasks.workunit.client.0.smithi038.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-24T00:48:29.230 INFO:tasks.workunit.client.0.smithi038.stderr:+ popd 2024-03-24T00:48:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-24T00:48:29.231 INFO:tasks.workunit.client.0.smithi038.stderr:+ popd 2024-03-24T00:48:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-24T00:48:29.231 INFO:tasks.workunit.client.0.smithi038.stderr:++ mktemp -d -p . 2024-03-24T00:48:29.234 INFO:tasks.workunit.client.0.smithi038.stderr:+ T=./tmp.zG281EyAMe 2024-03-24T00:48:29.234 INFO:tasks.workunit.client.0.smithi038.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.zG281EyAMe -l 1 -n 1000 -p 10 -v 2024-03-24T00:48:29.237 INFO:tasks.workunit.client.0.smithi038.stdout:seed = 1711070230 2024-03-24T00:48:29.241 INFO:tasks.workunit.client.0.smithi038.stdout:0/0: dread - no filename 2024-03-24T00:48:29.241 INFO:tasks.workunit.client.0.smithi038.stdout:0/1: fdatasync - no filename 2024-03-24T00:48:29.243 INFO:tasks.workunit.client.0.smithi038.stdout:1/0: dwrite - no filename 2024-03-24T00:48:29.243 INFO:tasks.workunit.client.0.smithi038.stdout:1/1: dwrite - no filename 2024-03-24T00:48:29.243 INFO:tasks.workunit.client.0.smithi038.stdout:1/2: write - no filename 2024-03-24T00:48:29.243 INFO:tasks.workunit.client.0.smithi038.stdout:1/3: unlink - no file 2024-03-24T00:48:29.243 INFO:tasks.workunit.client.0.smithi038.stdout:1/4: write - no filename 2024-03-24T00:48:29.243 INFO:tasks.workunit.client.0.smithi038.stdout:1/5: write - no filename 2024-03-24T00:48:29.243 INFO:tasks.workunit.client.0.smithi038.stdout:1/6: dwrite - no filename 2024-03-24T00:48:29.243 INFO:tasks.workunit.client.0.smithi038.stdout:1/7: dread - no filename 2024-03-24T00:48:29.243 INFO:tasks.workunit.client.0.smithi038.stdout:1/8: readlink - no filename 2024-03-24T00:48:29.243 INFO:tasks.workunit.client.0.smithi038.stdout:1/9: fdatasync - no filename 2024-03-24T00:48:29.244 INFO:tasks.workunit.client.0.smithi038.stdout:0/2: mkdir d0 0 2024-03-24T00:48:29.244 INFO:tasks.workunit.client.0.smithi038.stdout:2/0: write - no filename 2024-03-24T00:48:29.245 INFO:tasks.workunit.client.0.smithi038.stdout:1/10: symlink l0 0 2024-03-24T00:48:29.246 INFO:tasks.workunit.client.0.smithi038.stdout:0/3: creat d0/f1 x:0 0 0 2024-03-24T00:48:29.247 INFO:tasks.workunit.client.0.smithi038.stdout:3/0: rename - no filename 2024-03-24T00:48:29.248 INFO:tasks.workunit.client.0.smithi038.stdout:2/1: creat f0 x:0 0 0 2024-03-24T00:48:29.249 INFO:tasks.workunit.client.0.smithi038.stdout:1/11: creat f1 x:0 0 0 2024-03-24T00:48:29.250 INFO:tasks.workunit.client.0.smithi038.stdout:4/0: fdatasync - no filename 2024-03-24T00:48:29.251 INFO:tasks.workunit.client.0.smithi038.stdout:4/1: chown . 5 1 2024-03-24T00:48:29.251 INFO:tasks.workunit.client.0.smithi038.stdout:3/1: mknod c0 0 2024-03-24T00:48:29.251 INFO:tasks.workunit.client.0.smithi038.stdout:3/2: read - no filename 2024-03-24T00:48:29.251 INFO:tasks.workunit.client.0.smithi038.stdout:3/3: dwrite - no filename 2024-03-24T00:48:29.251 INFO:tasks.workunit.client.0.smithi038.stdout:3/4: fdatasync - no filename 2024-03-24T00:48:29.251 INFO:tasks.workunit.client.0.smithi038.stdout:1/12: link l0 l2 0 2024-03-24T00:48:29.251 INFO:tasks.workunit.client.0.smithi038.stdout:1/13: write f1 [1043130,114106] 0 2024-03-24T00:48:29.253 INFO:tasks.workunit.client.0.smithi038.stdout:4/2: mknod c0 0 2024-03-24T00:48:29.253 INFO:tasks.workunit.client.0.smithi038.stdout:4/3: dread - no filename 2024-03-24T00:48:29.254 INFO:tasks.workunit.client.0.smithi038.stdout:3/5: creat f1 x:0 0 0 2024-03-24T00:48:29.254 INFO:tasks.workunit.client.0.smithi038.stdout:3/6: dread - f1 zero size 2024-03-24T00:48:29.254 INFO:tasks.workunit.client.0.smithi038.stdout:3/7: creat f2 x:0 0 0 2024-03-24T00:48:29.254 INFO:tasks.workunit.client.0.smithi038.stdout:1/14: write f1 [544171,21922] 0 2024-03-24T00:48:29.255 INFO:tasks.workunit.client.0.smithi038.stdout:5/0: creat f0 x:0 0 0 2024-03-24T00:48:29.255 INFO:tasks.workunit.client.0.smithi038.stdout:5/1: dread - f0 zero size 2024-03-24T00:48:29.256 INFO:tasks.workunit.client.0.smithi038.stdout:6/0: dwrite - no filename 2024-03-24T00:48:29.258 INFO:tasks.workunit.client.0.smithi038.stdout:4/4: rename c0 to c1 0 2024-03-24T00:48:29.258 INFO:tasks.workunit.client.0.smithi038.stdout:4/5: dread - no filename 2024-03-24T00:48:29.258 INFO:tasks.workunit.client.0.smithi038.stdout:4/6: dread - no filename 2024-03-24T00:48:29.261 INFO:tasks.workunit.client.0.smithi038.stdout:3/8: mkdir d3 0 2024-03-24T00:48:29.261 INFO:tasks.workunit.client.0.smithi038.stdout:3/9: chown d3 79077 1 2024-03-24T00:48:29.261 INFO:tasks.workunit.client.0.smithi038.stdout:3/10: truncate f1 948575 0 2024-03-24T00:48:29.261 INFO:tasks.workunit.client.0.smithi038.stdout:3/11: rename d3 to d3/d4 22 2024-03-24T00:48:29.262 INFO:tasks.workunit.client.0.smithi038.stdout:1/15: fdatasync f1 0 2024-03-24T00:48:29.262 INFO:tasks.workunit.client.0.smithi038.stdout:1/16: chown f1 194 1 2024-03-24T00:48:29.263 INFO:tasks.workunit.client.0.smithi038.stdout:7/0: read - no filename 2024-03-24T00:48:29.264 INFO:tasks.workunit.client.0.smithi038.stdout:6/1: symlink l0 0 2024-03-24T00:48:29.267 INFO:tasks.workunit.client.0.smithi038.stdout:0/4: fsync d0/f1 0 2024-03-24T00:48:29.267 INFO:tasks.workunit.client.0.smithi038.stdout:0/5: creat d0/f2 x:0 0 0 2024-03-24T00:48:29.267 INFO:tasks.workunit.client.0.smithi038.stdout:0/6: chown d0/f2 23218 1 2024-03-24T00:48:29.267 INFO:tasks.workunit.client.0.smithi038.stdout:0/7: creat d0/f3 x:0 0 0 2024-03-24T00:48:29.267 INFO:tasks.workunit.client.0.smithi038.stdout:0/8: write d0/f3 [86566,107938] 0 2024-03-24T00:48:29.268 INFO:tasks.workunit.client.0.smithi038.stdout:7/1: mknod c0 0 2024-03-24T00:48:29.270 INFO:tasks.workunit.client.0.smithi038.stdout:9/0: dread - no filename 2024-03-24T00:48:29.270 INFO:tasks.workunit.client.0.smithi038.stdout:9/1: truncate - no filename 2024-03-24T00:48:29.270 INFO:tasks.workunit.client.0.smithi038.stdout:9/2: truncate - no filename 2024-03-24T00:48:29.270 INFO:tasks.workunit.client.0.smithi038.stdout:9/3: write - no filename 2024-03-24T00:48:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:8/0: mkdir d0 0 2024-03-24T00:48:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:8/1: truncate - no filename 2024-03-24T00:48:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:8/2: dread - no filename 2024-03-24T00:48:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:8/3: write - no filename 2024-03-24T00:48:29.278 INFO:tasks.workunit.client.0.smithi038.stdout:3/12: write f1 [65684,74338] 0 2024-03-24T00:48:29.278 INFO:tasks.workunit.client.0.smithi038.stdout:7/2: symlink l1 0 2024-03-24T00:48:29.278 INFO:tasks.workunit.client.0.smithi038.stdout:7/3: fsync - no filename 2024-03-24T00:48:29.278 INFO:tasks.workunit.client.0.smithi038.stdout:3/13: write f1 [1334918,129534] 0 2024-03-24T00:48:29.278 INFO:tasks.workunit.client.0.smithi038.stdout:9/4: symlink l0 0 2024-03-24T00:48:29.278 INFO:tasks.workunit.client.0.smithi038.stdout:7/4: mkdir d2 0 2024-03-24T00:48:29.278 INFO:tasks.workunit.client.0.smithi038.stdout:7/5: truncate - no filename 2024-03-24T00:48:29.279 INFO:tasks.workunit.client.0.smithi038.stdout:9/5: rename l0 to l1 0 2024-03-24T00:48:29.279 INFO:tasks.workunit.client.0.smithi038.stdout:9/6: write - no filename 2024-03-24T00:48:29.279 INFO:tasks.workunit.client.0.smithi038.stdout:9/7: rmdir - no directory 2024-03-24T00:48:29.280 INFO:tasks.workunit.client.0.smithi038.stdout:7/6: mknod d2/c3 0 2024-03-24T00:48:29.280 INFO:tasks.workunit.client.0.smithi038.stdout:7/7: write - no filename 2024-03-24T00:48:29.280 INFO:tasks.workunit.client.0.smithi038.stdout:7/8: dread - no filename 2024-03-24T00:48:29.280 INFO:tasks.workunit.client.0.smithi038.stdout:7/9: dread - no filename 2024-03-24T00:48:29.281 INFO:tasks.workunit.client.0.smithi038.stdout:7/10: creat d2/f4 x:0 0 0 2024-03-24T00:48:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:2/2: dwrite f0 [0,4194304] 0 2024-03-24T00:48:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:2/3: chown f0 12 1 2024-03-24T00:48:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:2/4: fsync f0 0 2024-03-24T00:48:29.332 INFO:tasks.workunit.client.0.smithi038.stdout:4/7: link c1 c2 0 2024-03-24T00:48:29.332 INFO:tasks.workunit.client.0.smithi038.stdout:4/8: write - no filename 2024-03-24T00:48:29.333 INFO:tasks.workunit.client.0.smithi038.stdout:4/9: rmdir - no directory 2024-03-24T00:48:29.333 INFO:tasks.workunit.client.0.smithi038.stdout:4/10: dread - no filename 2024-03-24T00:48:29.333 INFO:tasks.workunit.client.0.smithi038.stdout:4/11: fdatasync - no filename 2024-03-24T00:48:29.336 INFO:tasks.workunit.client.0.smithi038.stdout:4/12: creat f3 x:0 0 0 2024-03-24T00:48:29.340 INFO:tasks.workunit.client.0.smithi038.stdout:6/2: getdents . 0 2024-03-24T00:48:29.340 INFO:tasks.workunit.client.0.smithi038.stdout:6/3: fdatasync - no filename 2024-03-24T00:48:29.341 INFO:tasks.workunit.client.0.smithi038.stdout:4/13: mknod c4 0 2024-03-24T00:48:29.341 INFO:tasks.workunit.client.0.smithi038.stdout:4/14: chown f3 51380387 1 2024-03-24T00:48:29.343 INFO:tasks.workunit.client.0.smithi038.stdout:6/4: mkdir d1 0 2024-03-24T00:48:29.343 INFO:tasks.workunit.client.0.smithi038.stdout:6/5: read - no filename 2024-03-24T00:48:29.343 INFO:tasks.workunit.client.0.smithi038.stdout:6/6: dread - no filename 2024-03-24T00:48:29.343 INFO:tasks.workunit.client.0.smithi038.stdout:6/7: read - no filename 2024-03-24T00:48:29.343 INFO:tasks.workunit.client.0.smithi038.stdout:6/8: dwrite - no filename 2024-03-24T00:48:29.344 INFO:tasks.workunit.client.0.smithi038.stdout:6/9: write - no filename 2024-03-24T00:48:29.344 INFO:tasks.workunit.client.0.smithi038.stdout:6/10: dread - no filename 2024-03-24T00:48:29.351 INFO:tasks.workunit.client.0.smithi038.stdout:6/11: creat d1/f2 x:0 0 0 2024-03-24T00:48:29.351 INFO:tasks.workunit.client.0.smithi038.stdout:0/9: rmdir d0 39 2024-03-24T00:48:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:6/12: unlink l0 0 2024-03-24T00:48:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:0/10: rmdir d0 39 2024-03-24T00:48:29.363 INFO:tasks.workunit.client.0.smithi038.stdout:9/8: unlink l1 0 2024-03-24T00:48:29.363 INFO:tasks.workunit.client.0.smithi038.stdout:9/9: rmdir - no directory 2024-03-24T00:48:29.363 INFO:tasks.workunit.client.0.smithi038.stdout:9/10: unlink - no file 2024-03-24T00:48:29.367 INFO:tasks.workunit.client.0.smithi038.stdout:0/11: rename d0/f3 to d0/f4 0 2024-03-24T00:48:29.368 INFO:tasks.workunit.client.0.smithi038.stdout:0/12: fsync d0/f1 0 2024-03-24T00:48:29.368 INFO:tasks.workunit.client.0.smithi038.stdout:0/13: chown d0/f1 18806995 1 2024-03-24T00:48:29.368 INFO:tasks.workunit.client.0.smithi038.stdout:0/14: write d0/f4 [687729,95616] 0 2024-03-24T00:48:29.376 INFO:tasks.workunit.client.0.smithi038.stdout:3/14: dread f1 [0,4194304] 0 2024-03-24T00:48:29.376 INFO:tasks.workunit.client.0.smithi038.stdout:9/11: symlink l2 0 2024-03-24T00:48:29.376 INFO:tasks.workunit.client.0.smithi038.stdout:3/15: fdatasync f2 0 2024-03-24T00:48:29.376 INFO:tasks.workunit.client.0.smithi038.stdout:3/16: fdatasync f2 0 2024-03-24T00:48:29.376 INFO:tasks.workunit.client.0.smithi038.stdout:3/17: fdatasync f2 0 2024-03-24T00:48:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:9/12: creat f3 x:0 0 0 2024-03-24T00:48:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:3/18: creat d3/f5 x:0 0 0 2024-03-24T00:48:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:3/19: creat d3/f6 x:0 0 0 2024-03-24T00:48:29.382 INFO:tasks.workunit.client.0.smithi038.stdout:9/13: link f3 f4 0 2024-03-24T00:48:29.382 INFO:tasks.workunit.client.0.smithi038.stdout:9/14: chown l2 1439382 1 2024-03-24T00:48:29.389 INFO:tasks.workunit.client.0.smithi038.stdout:3/20: mkdir d3/d7 0 2024-03-24T00:48:29.390 INFO:tasks.workunit.client.0.smithi038.stdout:3/21: dread - f2 zero size 2024-03-24T00:48:29.390 INFO:tasks.workunit.client.0.smithi038.stdout:3/22: dread - d3/f6 zero size 2024-03-24T00:48:29.390 INFO:tasks.workunit.client.0.smithi038.stdout:3/23: chown c0 325 1 2024-03-24T00:48:29.390 INFO:tasks.workunit.client.0.smithi038.stdout:3/24: fdatasync d3/f5 0 2024-03-24T00:48:29.390 INFO:tasks.workunit.client.0.smithi038.stdout:9/15: mknod c5 0 2024-03-24T00:48:29.390 INFO:tasks.workunit.client.0.smithi038.stdout:3/25: creat d3/f8 x:0 0 0 2024-03-24T00:48:29.394 INFO:tasks.workunit.client.0.smithi038.stdout:9/16: link f3 f6 0 2024-03-24T00:48:29.394 INFO:tasks.workunit.client.0.smithi038.stdout:9/17: truncate f4 57981 0 2024-03-24T00:48:29.443 INFO:tasks.workunit.client.0.smithi038.stdout:5/2: dwrite f0 [0,4194304] 0 2024-03-24T00:48:29.444 INFO:tasks.workunit.client.0.smithi038.stdout:5/3: chown f0 11392014 1 2024-03-24T00:48:29.511 INFO:tasks.workunit.client.1.smithi088.stderr:++ readlink -f fsstress 2024-03-24T00:48:29.511 INFO:tasks.workunit.client.1.smithi088.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-24T00:48:29.511 INFO:tasks.workunit.client.1.smithi088.stderr:+ popd 2024-03-24T00:48:29.512 INFO:tasks.workunit.client.1.smithi088.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-24T00:48:29.512 INFO:tasks.workunit.client.1.smithi088.stderr:+ popd 2024-03-24T00:48:29.512 INFO:tasks.workunit.client.1.smithi088.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-24T00:48:29.515 INFO:tasks.workunit.client.1.smithi088.stderr:++ mktemp -d -p . 2024-03-24T00:48:29.521 INFO:tasks.workunit.client.1.smithi088.stderr:+ T=./tmp.K9rthFc5O1 2024-03-24T00:48:29.522 INFO:tasks.workunit.client.1.smithi088.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.K9rthFc5O1 -l 1 -n 1000 -p 10 -v 2024-03-24T00:48:29.529 INFO:tasks.workunit.client.1.smithi088.stdout:seed = 1710713865 2024-03-24T00:48:29.541 INFO:tasks.workunit.client.1.smithi088.stdout:2/0: stat - no entries 2024-03-24T00:48:29.542 INFO:tasks.workunit.client.1.smithi088.stdout:2/1: write - no filename 2024-03-24T00:48:29.542 INFO:tasks.workunit.client.1.smithi088.stdout:2/2: truncate - no filename 2024-03-24T00:48:29.543 INFO:tasks.workunit.client.1.smithi088.stdout:0/0: mknod c0 0 2024-03-24T00:48:29.543 INFO:tasks.workunit.client.1.smithi088.stdout:0/1: dread - no filename 2024-03-24T00:48:29.545 INFO:tasks.workunit.client.1.smithi088.stdout:2/3: symlink l0 0 2024-03-24T00:48:29.545 INFO:tasks.workunit.client.1.smithi088.stdout:2/4: write - no filename 2024-03-24T00:48:29.547 INFO:tasks.workunit.client.1.smithi088.stdout:3/0: getdents . 0 2024-03-24T00:48:29.548 INFO:tasks.workunit.client.1.smithi088.stdout:3/1: truncate - no filename 2024-03-24T00:48:29.548 INFO:tasks.workunit.client.1.smithi088.stdout:3/2: stat - no entries 2024-03-24T00:48:29.548 INFO:tasks.workunit.client.1.smithi088.stdout:3/3: dread - no filename 2024-03-24T00:48:29.548 INFO:tasks.workunit.client.1.smithi088.stdout:3/4: write - no filename 2024-03-24T00:48:29.550 INFO:tasks.workunit.client.1.smithi088.stdout:0/2: link c0 c1 0 2024-03-24T00:48:29.551 INFO:tasks.workunit.client.0.smithi038.stdout:5/4: dread f0 [0,4194304] 0 2024-03-24T00:48:29.552 INFO:tasks.workunit.client.1.smithi088.stdout:2/5: unlink l0 0 2024-03-24T00:48:29.554 INFO:tasks.workunit.client.1.smithi088.stdout:3/5: mknod c0 0 2024-03-24T00:48:29.558 INFO:tasks.workunit.client.1.smithi088.stdout:4/0: dread - no filename 2024-03-24T00:48:29.558 INFO:tasks.workunit.client.1.smithi088.stdout:4/1: stat - no entries 2024-03-24T00:48:29.558 INFO:tasks.workunit.client.1.smithi088.stdout:4/2: fdatasync - no filename 2024-03-24T00:48:29.558 INFO:tasks.workunit.client.1.smithi088.stdout:4/3: fdatasync - no filename 2024-03-24T00:48:29.560 INFO:tasks.workunit.client.1.smithi088.stdout:0/3: rename c1 to c2 0 2024-03-24T00:48:29.566 INFO:tasks.workunit.client.1.smithi088.stdout:2/6: creat f1 x:0 0 0 2024-03-24T00:48:29.568 INFO:tasks.workunit.client.1.smithi088.stdout:3/6: creat f1 x:0 0 0 2024-03-24T00:48:29.568 INFO:tasks.workunit.client.1.smithi088.stdout:3/7: fdatasync f1 0 2024-03-24T00:48:29.568 INFO:tasks.workunit.client.1.smithi088.stdout:3/8: creat f2 x:0 0 0 2024-03-24T00:48:29.568 INFO:tasks.workunit.client.1.smithi088.stdout:3/9: dread - f1 zero size 2024-03-24T00:48:29.569 INFO:tasks.workunit.client.1.smithi088.stdout:5/0: dread - no filename 2024-03-24T00:48:29.569 INFO:tasks.workunit.client.1.smithi088.stdout:5/1: write - no filename 2024-03-24T00:48:29.569 INFO:tasks.workunit.client.1.smithi088.stdout:5/2: getdents . 0 2024-03-24T00:48:29.570 INFO:tasks.workunit.client.1.smithi088.stdout:4/4: mknod c0 0 2024-03-24T00:48:29.571 INFO:tasks.workunit.client.1.smithi088.stdout:4/5: write - no filename 2024-03-24T00:48:29.571 INFO:tasks.workunit.client.1.smithi088.stdout:4/6: dread - no filename 2024-03-24T00:48:29.571 INFO:tasks.workunit.client.1.smithi088.stdout:4/7: dread - no filename 2024-03-24T00:48:29.571 INFO:tasks.workunit.client.1.smithi088.stdout:4/8: dread - no filename 2024-03-24T00:48:29.571 INFO:tasks.workunit.client.1.smithi088.stdout:4/9: chown c0 3137165 1 2024-03-24T00:48:29.571 INFO:tasks.workunit.client.1.smithi088.stdout:4/10: readlink - no filename 2024-03-24T00:48:29.573 INFO:tasks.workunit.client.1.smithi088.stdout:0/4: creat f3 x:0 0 0 2024-03-24T00:48:29.574 INFO:tasks.workunit.client.1.smithi088.stdout:0/5: readlink - no filename 2024-03-24T00:48:29.574 INFO:tasks.workunit.client.1.smithi088.stdout:0/6: read - f3 zero size 2024-03-24T00:48:29.575 INFO:tasks.workunit.client.1.smithi088.stdout:2/7: symlink l2 0 2024-03-24T00:48:29.577 INFO:tasks.workunit.client.1.smithi088.stdout:3/10: mkdir d3 0 2024-03-24T00:48:29.580 INFO:tasks.workunit.client.1.smithi088.stdout:6/0: dread - no filename 2024-03-24T00:48:29.582 INFO:tasks.workunit.client.1.smithi088.stdout:5/3: mknod c0 0 2024-03-24T00:48:29.583 INFO:tasks.workunit.client.1.smithi088.stdout:5/4: chown c0 127485 1 2024-03-24T00:48:29.583 INFO:tasks.workunit.client.1.smithi088.stdout:5/5: chown c0 7 1 2024-03-24T00:48:29.583 INFO:tasks.workunit.client.1.smithi088.stdout:5/6: dread - no filename 2024-03-24T00:48:29.583 INFO:tasks.workunit.client.1.smithi088.stdout:5/7: chown c0 1438 1 2024-03-24T00:48:29.583 INFO:tasks.workunit.client.1.smithi088.stdout:5/8: fdatasync - no filename 2024-03-24T00:48:29.584 INFO:tasks.workunit.client.1.smithi088.stdout:4/11: creat f1 x:0 0 0 2024-03-24T00:48:29.585 INFO:tasks.workunit.client.1.smithi088.stdout:0/7: mknod c4 0 2024-03-24T00:48:29.585 INFO:tasks.workunit.client.1.smithi088.stdout:0/8: readlink - no filename 2024-03-24T00:48:29.585 INFO:tasks.workunit.client.1.smithi088.stdout:0/9: write f3 [377966,87154] 0 2024-03-24T00:48:29.585 INFO:tasks.workunit.client.1.smithi088.stdout:0/10: rmdir - no directory 2024-03-24T00:48:29.586 INFO:tasks.workunit.client.1.smithi088.stdout:0/11: truncate f3 912067 0 2024-03-24T00:48:29.587 INFO:tasks.workunit.client.1.smithi088.stdout:2/8: mknod c3 0 2024-03-24T00:48:29.587 INFO:tasks.workunit.client.1.smithi088.stdout:2/9: truncate f1 593724 0 2024-03-24T00:48:29.587 INFO:tasks.workunit.client.1.smithi088.stdout:2/10: fdatasync f1 0 2024-03-24T00:48:29.591 INFO:tasks.workunit.client.1.smithi088.stdout:7/0: truncate - no filename 2024-03-24T00:48:29.591 INFO:tasks.workunit.client.1.smithi088.stdout:7/1: write - no filename 2024-03-24T00:48:29.592 INFO:tasks.workunit.client.1.smithi088.stdout:7/2: dwrite - no filename 2024-03-24T00:48:29.596 INFO:tasks.workunit.client.1.smithi088.stdout:6/1: symlink l0 0 2024-03-24T00:48:29.596 INFO:tasks.workunit.client.1.smithi088.stdout:6/2: dwrite - no filename 2024-03-24T00:48:29.596 INFO:tasks.workunit.client.1.smithi088.stdout:6/3: dread - no filename 2024-03-24T00:48:29.602 INFO:tasks.workunit.client.1.smithi088.stdout:5/9: mkdir d1 0 2024-03-24T00:48:29.602 INFO:tasks.workunit.client.1.smithi088.stdout:5/10: getdents d1 0 2024-03-24T00:48:29.602 INFO:tasks.workunit.client.1.smithi088.stdout:5/11: read - no filename 2024-03-24T00:48:29.602 INFO:tasks.workunit.client.1.smithi088.stdout:5/12: chown d1 0 1 2024-03-24T00:48:29.609 INFO:tasks.workunit.client.1.smithi088.stdout:7/3: creat f0 x:0 0 0 2024-03-24T00:48:29.611 INFO:tasks.workunit.client.1.smithi088.stdout:8/0: rmdir - no directory 2024-03-24T00:48:29.615 INFO:tasks.workunit.client.1.smithi088.stdout:5/13: mknod d1/c2 0 2024-03-24T00:48:29.615 INFO:tasks.workunit.client.1.smithi088.stdout:5/14: dread - no filename 2024-03-24T00:48:29.615 INFO:tasks.workunit.client.1.smithi088.stdout:6/4: creat f1 x:0 0 0 2024-03-24T00:48:29.618 INFO:tasks.workunit.client.1.smithi088.stdout:7/4: rename f0 to f1 0 2024-03-24T00:48:29.620 INFO:tasks.workunit.client.1.smithi088.stdout:9/0: chown . 492 1 2024-03-24T00:48:29.620 INFO:tasks.workunit.client.1.smithi088.stdout:9/1: write - no filename 2024-03-24T00:48:29.620 INFO:tasks.workunit.client.1.smithi088.stdout:9/2: dread - no filename 2024-03-24T00:48:29.620 INFO:tasks.workunit.client.1.smithi088.stdout:9/3: rename - no filename 2024-03-24T00:48:29.620 INFO:tasks.workunit.client.1.smithi088.stdout:9/4: read - no filename 2024-03-24T00:48:29.620 INFO:tasks.workunit.client.1.smithi088.stdout:9/5: rmdir - no directory 2024-03-24T00:48:29.620 INFO:tasks.workunit.client.1.smithi088.stdout:9/6: link - no file 2024-03-24T00:48:29.620 INFO:tasks.workunit.client.1.smithi088.stdout:9/7: dwrite - no filename 2024-03-24T00:48:29.622 INFO:tasks.workunit.client.1.smithi088.stdout:8/1: symlink l0 0 2024-03-24T00:48:29.622 INFO:tasks.workunit.client.1.smithi088.stdout:8/2: dwrite - no filename 2024-03-24T00:48:29.625 INFO:tasks.workunit.client.1.smithi088.stdout:6/5: unlink l0 0 2024-03-24T00:48:29.625 INFO:tasks.workunit.client.1.smithi088.stdout:6/6: truncate f1 155193 0 2024-03-24T00:48:29.627 INFO:tasks.workunit.client.1.smithi088.stdout:5/15: link c0 d1/c3 0 2024-03-24T00:48:29.631 INFO:tasks.workunit.client.1.smithi088.stdout:7/5: mknod c2 0 2024-03-24T00:48:29.631 INFO:tasks.workunit.client.1.smithi088.stdout:7/6: dread - f1 zero size 2024-03-24T00:48:29.631 INFO:tasks.workunit.client.1.smithi088.stdout:7/7: read - f1 zero size 2024-03-24T00:48:29.631 INFO:tasks.workunit.client.1.smithi088.stdout:7/8: chown c2 1 1 2024-03-24T00:48:29.635 INFO:tasks.workunit.client.1.smithi088.stdout:9/8: mknod c0 0 2024-03-24T00:48:29.635 INFO:tasks.workunit.client.1.smithi088.stdout:9/9: dread - no filename 2024-03-24T00:48:29.639 INFO:tasks.workunit.client.1.smithi088.stdout:8/3: creat f1 x:0 0 0 2024-03-24T00:48:29.639 INFO:tasks.workunit.client.1.smithi088.stdout:8/4: creat f2 x:0 0 0 2024-03-24T00:48:29.640 INFO:tasks.workunit.client.1.smithi088.stdout:5/16: rmdir d1 39 2024-03-24T00:48:29.640 INFO:tasks.workunit.client.1.smithi088.stdout:5/17: write - no filename 2024-03-24T00:48:29.640 INFO:tasks.workunit.client.1.smithi088.stdout:1/0: creat f0 x:0 0 0 2024-03-24T00:48:29.640 INFO:tasks.workunit.client.1.smithi088.stdout:1/1: fdatasync f0 0 2024-03-24T00:48:29.647 INFO:tasks.workunit.client.1.smithi088.stdout:1/2: link f0 f1 0 2024-03-24T00:48:29.648 INFO:tasks.workunit.client.1.smithi088.stdout:1/3: rmdir - no directory 2024-03-24T00:48:29.648 INFO:tasks.workunit.client.1.smithi088.stdout:1/4: dread - f1 zero size 2024-03-24T00:48:29.649 INFO:tasks.workunit.client.1.smithi088.stdout:1/5: write f1 [1021945,127219] 0 2024-03-24T00:48:29.649 INFO:tasks.workunit.client.1.smithi088.stdout:1/6: creat f2 x:0 0 0 2024-03-24T00:48:29.649 INFO:tasks.workunit.client.1.smithi088.stdout:1/7: chown f2 10 1 2024-03-24T00:48:29.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.678+0000 7f5a4be00700 1 -- 172.21.15.38:0/1022761031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a4409d340 msgr2=0x7f5a4409d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:29.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.678+0000 7f5a4be00700 1 --2- 172.21.15.38:0/1022761031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a4409d340 0x7f5a4409d7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009870 tx=0x7f5a38009b80 comp rx=0 tx=0).stop 2024-03-24T00:48:29.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.678+0000 7f5a4be00700 1 -- 172.21.15.38:0/1022761031 shutdown_connections 2024-03-24T00:48:29.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.678+0000 7f5a4be00700 1 --2- 172.21.15.38:0/1022761031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a4409d340 0x7f5a4409d7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:29.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.678+0000 7f5a4be00700 1 --2- 172.21.15.38:0/1022761031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4409c140 0x7f5a4409c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:29.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.678+0000 7f5a4be00700 1 -- 172.21.15.38:0/1022761031 >> 172.21.15.38:0/1022761031 conn(0x7f5a4400b100 msgr2=0x7f5a44099d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:48:29.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.678+0000 7f5a4be00700 1 -- 172.21.15.38:0/1022761031 shutdown_connections 2024-03-24T00:48:29.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.678+0000 7f5a4be00700 1 -- 172.21.15.38:0/1022761031 wait complete. 2024-03-24T00:48:29.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.680+0000 7f5a4be00700 1 Processor -- start 2024-03-24T00:48:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.682+0000 7f5a4be00700 1 -- start start 2024-03-24T00:48:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.683+0000 7f5a4be00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4409c140 0x7f5a44142a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.683+0000 7f5a4be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a4409d340 0x7f5a44142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.683+0000 7f5a4be00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a441435e0 con 0x7f5a4409d340 2024-03-24T00:48:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.683+0000 7f5a4be00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a44143720 con 0x7f5a4409c140 2024-03-24T00:48:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.684+0000 7f5a4a000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a4409d340 0x7f5a44142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.688+0000 7f5a4aa00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4409c140 0x7f5a44142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.688+0000 7f5a4a000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a4409d340 0x7f5a44142fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45068/0 (socket says 172.21.15.38:45068) 2024-03-24T00:48:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.688+0000 7f5a4a000700 1 -- 172.21.15.38:0/1292085177 learned_addr learned my addr 172.21.15.38:0/1292085177 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:48:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.689+0000 7f5a4a000700 1 -- 172.21.15.38:0/1292085177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4409c140 msgr2=0x7f5a44142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:29.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.689+0000 7f5a4a000700 1 --2- 172.21.15.38:0/1292085177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4409c140 0x7f5a44142a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:29.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.689+0000 7f5a4a000700 1 -- 172.21.15.38:0/1292085177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38009520 con 0x7f5a4409d340 2024-03-24T00:48:29.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.689+0000 7f5a4a000700 1 --2- 172.21.15.38:0/1292085177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a4409d340 0x7f5a44142fc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c000 tx=0x7f5a380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:48:29.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.691+0000 7f5a43400700 1 -- 172.21.15.38:0/1292085177 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a3801d070 con 0x7f5a4409d340 2024-03-24T00:48:29.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.691+0000 7f5a43400700 1 -- 172.21.15.38:0/1292085177 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a3800f460 con 0x7f5a4409d340 2024-03-24T00:48:29.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.691+0000 7f5a43400700 1 -- 172.21.15.38:0/1292085177 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a380175e0 con 0x7f5a4409d340 2024-03-24T00:48:29.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.691+0000 7f5a4be00700 1 -- 172.21.15.38:0/1292085177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4400d8a0 con 0x7f5a4409d340 2024-03-24T00:48:29.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.691+0000 7f5a4be00700 1 -- 172.21.15.38:0/1292085177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4400dd30 con 0x7f5a4409d340 2024-03-24T00:48:29.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.694+0000 7f5a4be00700 1 -- 172.21.15.38:0/1292085177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a440055b0 con 0x7f5a4409d340 2024-03-24T00:48:29.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.699+0000 7f5a43400700 1 -- 172.21.15.38:0/1292085177 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 25) v1 ==== 91370+0+0 (secure 0 0 0) 0x7f5a3800b9a0 con 0x7f5a4409d340 2024-03-24T00:48:29.703 INFO:tasks.workunit.client.0.smithi038.stdout:9/18: dwrite f3 [0,4194304] 0 2024-03-24T00:48:29.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.702+0000 7f5a43400700 1 --2- 172.21.15.38:0/1292085177 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f5a3006d640 0x7f5a3006fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:29.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.702+0000 7f5a43400700 1 -- 172.21.15.38:0/1292085177 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f5a3808e4d0 con 0x7f5a4409d340 2024-03-24T00:48:29.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.706+0000 7f5a4aa00700 1 --2- 172.21.15.38:0/1292085177 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f5a3006d640 0x7f5a3006fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:29.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.707+0000 7f5a4aa00700 1 --2- 172.21.15.38:0/1292085177 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f5a3006d640 0x7f5a3006fb00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c005e50 tx=0x7f5a3c009430 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:48:29.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:29.713+0000 7f5a43400700 1 -- 172.21.15.38:0/1292085177 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f5a3805b9a0 con 0x7f5a4409d340 2024-03-24T00:48:29.714 INFO:tasks.workunit.client.1.smithi088.stdout:3/11: dwrite f2 [0,4194304] 0 2024-03-24T00:48:29.718 INFO:tasks.workunit.client.0.smithi038.stdout:4/15: fsync f3 0 2024-03-24T00:48:29.724 INFO:tasks.workunit.client.0.smithi038.stdout:6/13: rmdir d1 39 2024-03-24T00:48:29.724 INFO:tasks.workunit.client.0.smithi038.stdout:6/14: chown d1/f2 4355 1 2024-03-24T00:48:29.724 INFO:tasks.workunit.client.0.smithi038.stdout:6/15: dread - d1/f2 zero size 2024-03-24T00:48:29.724 INFO:tasks.workunit.client.0.smithi038.stdout:6/16: readlink - no filename 2024-03-24T00:48:29.742 INFO:tasks.workunit.client.1.smithi088.stdout:4/12: dwrite f1 [0,4194304] 0 2024-03-24T00:48:29.756 INFO:tasks.workunit.client.0.smithi038.stdout:3/26: dwrite d3/f5 [0,4194304] 0 2024-03-24T00:48:29.766 INFO:tasks.workunit.client.0.smithi038.stdout:7/11: dwrite d2/f4 [0,4194304] 0 2024-03-24T00:48:29.781 INFO:tasks.workunit.client.1.smithi088.stdout:3/12: creat d3/f4 x:0 0 0 2024-03-24T00:48:29.815 INFO:tasks.workunit.client.1.smithi088.stdout:0/12: dwrite f3 [0,4194304] 0 2024-03-24T00:48:29.842 INFO:tasks.workunit.client.1.smithi088.stdout:2/11: dwrite f1 [0,4194304] 0 2024-03-24T00:48:29.842 INFO:tasks.workunit.client.1.smithi088.stdout:2/12: chown l2 3512350 1 2024-03-24T00:48:29.938 INFO:tasks.workunit.client.0.smithi038.stdout:1/17: dwrite f1 [0,4194304] 0 2024-03-24T00:48:29.941 INFO:tasks.workunit.client.1.smithi088.stdout:4/13: write f1 [1902168,70580] 0 2024-03-24T00:48:29.975 INFO:tasks.workunit.client.0.smithi038.stdout:0/15: dwrite d0/f2 [0,4194304] 0 2024-03-24T00:48:29.999 INFO:tasks.workunit.client.0.smithi038.stdout:4/16: dwrite f3 [0,4194304] 0 2024-03-24T00:48:29.999 INFO:tasks.workunit.client.0.smithi038.stdout:4/17: stat c4 0 2024-03-24T00:48:30.019 INFO:tasks.workunit.client.1.smithi088.stdout:6/7: dwrite f1 [0,4194304] 0 2024-03-24T00:48:30.093 INFO:tasks.workunit.client.0.smithi038.stdout:2/5: dwrite f0 [4194304,4194304] 0 2024-03-24T00:48:30.093 INFO:tasks.workunit.client.0.smithi038.stdout:2/6: chown f0 18 1 2024-03-24T00:48:30.140 INFO:tasks.workunit.client.1.smithi088.stdout:9/10: getdents . 0 2024-03-24T00:48:30.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.210+0000 7f5a4be00700 1 -- 172.21.15.38:0/1292085177 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a4413b530 con 0x7f5a3006d640 2024-03-24T00:48:30.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.214+0000 7f5a43400700 1 -- 172.21.15.38:0/1292085177 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f5a4413b530 con 0x7f5a3006d640 2024-03-24T00:48:30.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.219+0000 7f5a4be00700 1 -- 172.21.15.38:0/1292085177 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f5a3006d640 msgr2=0x7f5a3006fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:30.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.219+0000 7f5a4be00700 1 --2- 172.21.15.38:0/1292085177 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f5a3006d640 0x7f5a3006fb00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c005e50 tx=0x7f5a3c009430 comp rx=0 tx=0).stop 2024-03-24T00:48:30.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.219+0000 7f5a4be00700 1 -- 172.21.15.38:0/1292085177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a4409d340 msgr2=0x7f5a44142fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:30.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.219+0000 7f5a4be00700 1 --2- 172.21.15.38:0/1292085177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a4409d340 0x7f5a44142fc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c000 tx=0x7f5a380057f0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.219+0000 7f5a4be00700 1 -- 172.21.15.38:0/1292085177 shutdown_connections 2024-03-24T00:48:30.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.219+0000 7f5a4be00700 1 --2- 172.21.15.38:0/1292085177 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f5a3006d640 0x7f5a3006fb00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.219+0000 7f5a4be00700 1 --2- 172.21.15.38:0/1292085177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4409c140 0x7f5a44142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.219+0000 7f5a4be00700 1 --2- 172.21.15.38:0/1292085177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a4409d340 0x7f5a44142fc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.219+0000 7f5a4be00700 1 -- 172.21.15.38:0/1292085177 >> 172.21.15.38:0/1292085177 conn(0x7f5a4400b100 msgr2=0x7f5a440a0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:48:30.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.219+0000 7f5a4be00700 1 -- 172.21.15.38:0/1292085177 shutdown_connections 2024-03-24T00:48:30.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.219+0000 7f5a4be00700 1 -- 172.21.15.38:0/1292085177 wait complete. 2024-03-24T00:48:30.225 INFO:tasks.workunit.client.1.smithi088.stdout:1/8: fsync f1 0 2024-03-24T00:48:30.226 INFO:tasks.workunit.client.1.smithi088.stdout:1/9: chown f1 7 1 2024-03-24T00:48:30.226 INFO:tasks.workunit.client.1.smithi088.stdout:1/10: fsync f1 0 2024-03-24T00:48:30.226 INFO:tasks.workunit.client.1.smithi088.stdout:1/11: chown f1 2687 1 2024-03-24T00:48:30.226 INFO:tasks.workunit.client.1.smithi088.stdout:1/12: write f2 [840821,86946] 0 2024-03-24T00:48:30.247 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:48:30.265 INFO:tasks.workunit.client.1.smithi088.stdout:2/13: symlink l4 0 2024-03-24T00:48:30.277 INFO:tasks.workunit.client.0.smithi038.stdout:2/7: dwrite f0 [0,4194304] 0 2024-03-24T00:48:30.309 INFO:tasks.workunit.client.1.smithi088.stdout:2/14: dread f1 [0,4194304] 0 2024-03-24T00:48:30.329 INFO:tasks.workunit.client.1.smithi088.stdout:2/15: write f1 [1086480,116226] 0 2024-03-24T00:48:30.399 INFO:tasks.workunit.client.1.smithi088.stdout:4/14: dwrite f1 [0,4194304] 0 2024-03-24T00:48:30.399 INFO:tasks.workunit.client.1.smithi088.stdout:4/15: chown f1 357261 1 2024-03-24T00:48:30.400 INFO:tasks.workunit.client.1.smithi088.stdout:4/16: creat f2 x:0 0 0 2024-03-24T00:48:30.403 INFO:tasks.workunit.client.0.smithi038.stdout:2/8: dread f0 [4194304,4194304] 0 2024-03-24T00:48:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.448+0000 7f400cc00700 1 -- 172.21.15.38:0/236687945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4008102d00 msgr2=0x7f4008103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.448+0000 7f400cc00700 1 --2- 172.21.15.38:0/236687945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4008102d00 0x7f4008103120 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f400000b0a0 tx=0x7f400000b3b0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.448+0000 7f400cc00700 1 -- 172.21.15.38:0/236687945 shutdown_connections 2024-03-24T00:48:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.448+0000 7f400cc00700 1 --2- 172.21.15.38:0/236687945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4008103f00 0x7f4008104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.448+0000 7f400cc00700 1 --2- 172.21.15.38:0/236687945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4008102d00 0x7f4008103120 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.448+0000 7f400cc00700 1 -- 172.21.15.38:0/236687945 >> 172.21.15.38:0/236687945 conn(0x7f40080fe2a0 msgr2=0x7f40081006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:48:30.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.450+0000 7f400cc00700 1 -- 172.21.15.38:0/236687945 shutdown_connections 2024-03-24T00:48:30.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.450+0000 7f400cc00700 1 -- 172.21.15.38:0/236687945 wait complete. 2024-03-24T00:48:30.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.453+0000 7f400cc00700 1 Processor -- start 2024-03-24T00:48:30.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.453+0000 7f400cc00700 1 -- start start 2024-03-24T00:48:30.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.454+0000 7f400cc00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4008103f00 0x7f4008078ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.454+0000 7f400cc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4008079000 0x7f40080755c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.454+0000 7f400cc00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4008075b00 con 0x7f4008079000 2024-03-24T00:48:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.454+0000 7f400cc00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4008075c70 con 0x7f4008103f00 2024-03-24T00:48:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.454+0000 7f4005e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4008103f00 0x7f4008078ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.454+0000 7f4005e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4008103f00 0x7f4008078ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:49216/0 (socket says 172.21.15.38:49216) 2024-03-24T00:48:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.454+0000 7f4005e00700 1 -- 172.21.15.38:0/2498336733 learned_addr learned my addr 172.21.15.38:0/2498336733 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:48:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.454+0000 7f4005400700 1 --2- 172.21.15.38:0/2498336733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4008079000 0x7f40080755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.455+0000 7f4005e00700 1 -- 172.21.15.38:0/2498336733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4008079000 msgr2=0x7f40080755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.455+0000 7f4005e00700 1 --2- 172.21.15.38:0/2498336733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4008079000 0x7f40080755c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.455+0000 7f4005e00700 1 -- 172.21.15.38:0/2498336733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4000009d00 con 0x7f4008103f00 2024-03-24T00:48:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.456+0000 7f4005e00700 1 --2- 172.21.15.38:0/2498336733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4008103f00 0x7f4008078ac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4000015040 tx=0x7f40000096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:48:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.457+0000 7f3ffea00700 1 -- 172.21.15.38:0/2498336733 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4000021070 con 0x7f4008103f00 2024-03-24T00:48:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.457+0000 7f3ffea00700 1 -- 172.21.15.38:0/2498336733 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4000003c10 con 0x7f4008103f00 2024-03-24T00:48:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.457+0000 7f3ffea00700 1 -- 172.21.15.38:0/2498336733 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4000021070 con 0x7f4008103f00 2024-03-24T00:48:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.458+0000 7f400cc00700 1 -- 172.21.15.38:0/2498336733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4008075ef0 con 0x7f4008103f00 2024-03-24T00:48:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.459+0000 7f400cc00700 1 -- 172.21.15.38:0/2498336733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4008076210 con 0x7f4008103f00 2024-03-24T00:48:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.461+0000 7f400cc00700 1 -- 172.21.15.38:0/2498336733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4008192880 con 0x7f4008103f00 2024-03-24T00:48:30.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.465+0000 7f3ffea00700 1 -- 172.21.15.38:0/2498336733 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) v1 ==== 91370+0+0 (secure 0 0 0) 0x7f4000019070 con 0x7f4008103f00 2024-03-24T00:48:30.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.466+0000 7f3ffea00700 1 --2- 172.21.15.38:0/2498336733 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f3fec06d620 0x7f3fec06fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:30.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.467+0000 7f3ffea00700 1 -- 172.21.15.38:0/2498336733 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f400008d690 con 0x7f4008103f00 2024-03-24T00:48:30.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.467+0000 7f4005400700 1 --2- 172.21.15.38:0/2498336733 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f3fec06d620 0x7f3fec06fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:30.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.468+0000 7f4005400700 1 --2- 172.21.15.38:0/2498336733 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f3fec06d620 0x7f3fec06fae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009bf0 tx=0x7f3ff0009260 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:48:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.476+0000 7f3ffea00700 1 -- 172.21.15.38:0/2498336733 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f400005aac0 con 0x7f4008103f00 2024-03-24T00:48:30.498 INFO:tasks.workunit.client.1.smithi088.stdout:4/17: dwrite f1 [0,4194304] 0 2024-03-24T00:48:30.499 INFO:tasks.workunit.client.1.smithi088.stdout:4/18: chown c0 16 1 2024-03-24T00:48:30.517 INFO:tasks.workunit.client.1.smithi088.stdout:4/19: dread f1 [0,4194304] 0 2024-03-24T00:48:30.517 INFO:tasks.workunit.client.1.smithi088.stdout:4/20: dread - f2 zero size 2024-03-24T00:48:30.517 INFO:tasks.workunit.client.1.smithi088.stdout:4/21: chown c0 145 1 2024-03-24T00:48:30.552 INFO:tasks.workunit.client.1.smithi088.stdout:3/13: getdents d3 0 2024-03-24T00:48:30.554 INFO:tasks.workunit.client.1.smithi088.stdout:8/5: dwrite f1 [0,4194304] 0 2024-03-24T00:48:30.656 INFO:tasks.workunit.client.1.smithi088.stdout:4/22: dwrite f2 [0,4194304] 0 2024-03-24T00:48:30.678 INFO:tasks.workunit.client.1.smithi088.stdout:4/23: dread f2 [0,4194304] 0 2024-03-24T00:48:30.712 INFO:tasks.workunit.client.0.smithi038.stdout:6/17: link d1/f2 d1/f3 0 2024-03-24T00:48:30.712 INFO:tasks.workunit.client.0.smithi038.stdout:6/18: rename d1 to d1/d4 22 2024-03-24T00:48:30.717 INFO:tasks.workunit.client.1.smithi088.stdout:1/13: write f0 [981279,50261] 0 2024-03-24T00:48:30.766 INFO:tasks.workunit.client.0.smithi038.stdout:3/27: mkdir d3/d9 0 2024-03-24T00:48:30.807 INFO:tasks.workunit.client.0.smithi038.stdout:3/28: dread d3/f5 [0,4194304] 0 2024-03-24T00:48:30.873 INFO:tasks.workunit.client.0.smithi038.stdout:3/29: dwrite d3/f8 [0,4194304] 0 2024-03-24T00:48:30.873 INFO:tasks.workunit.client.0.smithi038.stdout:3/30: chown d3/f6 784 1 2024-03-24T00:48:30.917 INFO:tasks.workunit.client.0.smithi038.stdout:3/31: dwrite f1 [0,4194304] 0 2024-03-24T00:48:30.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:30 smithi088 ceph-mon[36610]: pgmap v21: 65 pgs: 65 active+clean; 206 MiB data, 937 MiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 90 op/s 2024-03-24T00:48:30.934 INFO:tasks.workunit.client.0.smithi038.stdout:9/19: write f6 [4435077,17447] 0 2024-03-24T00:48:30.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.936+0000 7f400cc00700 1 -- 172.21.15.38:0/2498336733 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40080611f0 con 0x7f3fec06d620 2024-03-24T00:48:30.937 INFO:tasks.workunit.client.0.smithi038.stdout:3/32: write d3/f8 [3665136,58977] 0 2024-03-24T00:48:30.937 INFO:tasks.workunit.client.0.smithi038.stdout:3/33: fdatasync d3/f5 0 2024-03-24T00:48:30.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.939+0000 7f3ffea00700 1 -- 172.21.15.38:0/2498336733 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f40080611f0 con 0x7f3fec06d620 2024-03-24T00:48:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.942+0000 7f400cc00700 1 -- 172.21.15.38:0/2498336733 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f3fec06d620 msgr2=0x7f3fec06fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.942+0000 7f400cc00700 1 --2- 172.21.15.38:0/2498336733 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f3fec06d620 0x7f3fec06fae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009bf0 tx=0x7f3ff0009260 comp rx=0 tx=0).stop 2024-03-24T00:48:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.942+0000 7f400cc00700 1 -- 172.21.15.38:0/2498336733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4008103f00 msgr2=0x7f4008078ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.942+0000 7f400cc00700 1 --2- 172.21.15.38:0/2498336733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4008103f00 0x7f4008078ac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4000015040 tx=0x7f40000096e0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.942+0000 7f400cc00700 1 -- 172.21.15.38:0/2498336733 shutdown_connections 2024-03-24T00:48:30.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.942+0000 7f400cc00700 1 --2- 172.21.15.38:0/2498336733 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f3fec06d620 0x7f3fec06fae0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.942+0000 7f400cc00700 1 --2- 172.21.15.38:0/2498336733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4008103f00 0x7f4008078ac0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.942+0000 7f400cc00700 1 --2- 172.21.15.38:0/2498336733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4008079000 0x7f40080755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:30.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.942+0000 7f400cc00700 1 -- 172.21.15.38:0/2498336733 >> 172.21.15.38:0/2498336733 conn(0x7f40080fe2a0 msgr2=0x7f4008107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:48:30.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.942+0000 7f400cc00700 1 -- 172.21.15.38:0/2498336733 shutdown_connections 2024-03-24T00:48:30.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:30.942+0000 7f400cc00700 1 -- 172.21.15.38:0/2498336733 wait complete. 2024-03-24T00:48:31.050 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:30 smithi038 ceph-mon[29271]: pgmap v21: 65 pgs: 65 active+clean; 206 MiB data, 937 MiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 90 op/s 2024-03-24T00:48:31.090 INFO:tasks.workunit.client.0.smithi038.stdout:1/18: symlink l3 0 2024-03-24T00:48:31.090 INFO:tasks.workunit.client.0.smithi038.stdout:1/19: chown l2 1164 1 2024-03-24T00:48:31.090 INFO:tasks.workunit.client.0.smithi038.stdout:1/20: creat f4 x:0 0 0 2024-03-24T00:48:31.090 INFO:tasks.workunit.client.0.smithi038.stdout:1/21: write f4 [831108,111092] 0 2024-03-24T00:48:31.091 INFO:tasks.workunit.client.0.smithi038.stdout:1/22: creat f5 x:0 0 0 2024-03-24T00:48:31.119 INFO:tasks.workunit.client.1.smithi088.stdout:6/8: symlink l2 0 2024-03-24T00:48:31.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.143+0000 7ff50e800700 1 -- 172.21.15.38:0/2091450540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508103ef0 msgr2=0x7ff5081062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:31.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.143+0000 7ff50e800700 1 --2- 172.21.15.38:0/2091450540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508103ef0 0x7ff5081062e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009910 tx=0x7ff4f4009c20 comp rx=0 tx=0).stop 2024-03-24T00:48:31.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.143+0000 7ff50e800700 1 -- 172.21.15.38:0/2091450540 shutdown_connections 2024-03-24T00:48:31.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.143+0000 7ff50e800700 1 --2- 172.21.15.38:0/2091450540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508103ef0 0x7ff5081062e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:31.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.143+0000 7ff50e800700 1 --2- 172.21.15.38:0/2091450540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5081015c0 0x7ff5081039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:31.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.143+0000 7ff50e800700 1 -- 172.21.15.38:0/2091450540 >> 172.21.15.38:0/2091450540 conn(0x7ff5080faf70 msgr2=0x7ff5080fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:48:31.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.143+0000 7ff50e800700 1 -- 172.21.15.38:0/2091450540 shutdown_connections 2024-03-24T00:48:31.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.143+0000 7ff50e800700 1 -- 172.21.15.38:0/2091450540 wait complete. 2024-03-24T00:48:31.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.144+0000 7ff50e800700 1 Processor -- start 2024-03-24T00:48:31.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.144+0000 7ff50e800700 1 -- start start 2024-03-24T00:48:31.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.145+0000 7ff50e800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5081015c0 0x7ff508196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:31.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.145+0000 7ff50e800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508103ef0 0x7ff5081969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:31.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.145+0000 7ff50e800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff508196fd0 con 0x7ff508103ef0 2024-03-24T00:48:31.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.145+0000 7ff50e800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff508197110 con 0x7ff5081015c0 2024-03-24T00:48:31.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.145+0000 7ff506a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508103ef0 0x7ff5081969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:31.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.145+0000 7ff506a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508103ef0 0x7ff5081969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45106/0 (socket says 172.21.15.38:45106) 2024-03-24T00:48:31.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.145+0000 7ff506a00700 1 -- 172.21.15.38:0/4125849097 learned_addr learned my addr 172.21.15.38:0/4125849097 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:48:31.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.145+0000 7ff507400700 1 --2- 172.21.15.38:0/4125849097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5081015c0 0x7ff508196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:31.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.145+0000 7ff506a00700 1 -- 172.21.15.38:0/4125849097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5081015c0 msgr2=0x7ff508196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:31.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.145+0000 7ff506a00700 1 --2- 172.21.15.38:0/4125849097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5081015c0 0x7ff508196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:31.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.145+0000 7ff506a00700 1 -- 172.21.15.38:0/4125849097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f40095f0 con 0x7ff508103ef0 2024-03-24T00:48:31.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.146+0000 7ff506a00700 1 --2- 172.21.15.38:0/4125849097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508103ef0 0x7ff5081969b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400bd70 tx=0x7ff4f400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:48:31.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.146+0000 7ff4ffe00700 1 -- 172.21.15.38:0/4125849097 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4f401d070 con 0x7ff508103ef0 2024-03-24T00:48:31.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.146+0000 7ff4ffe00700 1 -- 172.21.15.38:0/4125849097 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4f4005dc0 con 0x7ff508103ef0 2024-03-24T00:48:31.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.146+0000 7ff50e800700 1 -- 172.21.15.38:0/4125849097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50819bb10 con 0x7ff508103ef0 2024-03-24T00:48:31.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.146+0000 7ff4ffe00700 1 -- 172.21.15.38:0/4125849097 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4f4021410 con 0x7ff508103ef0 2024-03-24T00:48:31.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.147+0000 7ff50e800700 1 -- 172.21.15.38:0/4125849097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50819c080 con 0x7ff508103ef0 2024-03-24T00:48:31.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.148+0000 7ff50e800700 1 -- 172.21.15.38:0/4125849097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50804ea80 con 0x7ff508103ef0 2024-03-24T00:48:31.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.151+0000 7ff4ffe00700 1 -- 172.21.15.38:0/4125849097 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 25) v1 ==== 91370+0+0 (secure 0 0 0) 0x7ff4f400f820 con 0x7ff508103ef0 2024-03-24T00:48:31.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.152+0000 7ff4ffe00700 1 --2- 172.21.15.38:0/4125849097 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7ff4ec06d5b0 0x7ff4ec06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:31.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.152+0000 7ff4ffe00700 1 -- 172.21.15.38:0/4125849097 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff4f408e710 con 0x7ff508103ef0 2024-03-24T00:48:31.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.153+0000 7ff507400700 1 --2- 172.21.15.38:0/4125849097 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7ff4ec06d5b0 0x7ff4ec06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:31.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.154+0000 7ff507400700 1 --2- 172.21.15.38:0/4125849097 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7ff4ec06d5b0 0x7ff4ec06fa70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8005950 tx=0x7ff4f80058e0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:48:31.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.155+0000 7ff4ffe00700 1 -- 172.21.15.38:0/4125849097 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7ff4f405bc40 con 0x7ff508103ef0 2024-03-24T00:48:31.199 INFO:tasks.workunit.client.0.smithi038.stdout:0/16: unlink d0/f4 0 2024-03-24T00:48:31.234 INFO:tasks.workunit.client.0.smithi038.stdout:0/17: write d0/f2 [3961269,71342] 0 2024-03-24T00:48:31.234 INFO:tasks.workunit.client.0.smithi038.stdout:0/18: chown d0/f2 734086 1 2024-03-24T00:48:31.409 INFO:tasks.workunit.client.0.smithi038.stdout:5/5: unlink f0 0 2024-03-24T00:48:31.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.453+0000 7ff50e800700 1 -- 172.21.15.38:0/4125849097 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff508105db0 con 0x7ff4ec06d5b0 2024-03-24T00:48:31.457 INFO:tasks.workunit.client.0.smithi038.stdout:2/9: link f0 f1 0 2024-03-24T00:48:31.480 INFO:tasks.workunit.client.1.smithi088.stdout:6/9: write f1 [4434707,41330] 0 2024-03-24T00:48:31.487 INFO:tasks.workunit.client.0.smithi038.stdout:2/10: write f1 [1254420,40170] 0 2024-03-24T00:48:31.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.499+0000 7ff4ffe00700 1 -- 172.21.15.38:0/4125849097 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff508105db0 con 0x7ff4ec06d5b0 2024-03-24T00:48:31.499 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:48:31.499 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (8m) 17s ago 10m 13.8M - 0.25.0 c8568f914cd2 9f1243fbcb3e 2024-03-24T00:48:31.499 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (11m) 17s ago 11m 7922k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (9m) 26s ago 9m 7730k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (11m) 17s ago 11m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (9m) 26s ago 9m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (8m) 17s ago 10m 74.5M - 9.4.7 954c08fa6188 e6aa268d69e8 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (3m) 17s ago 3m 78.9M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (3m) 17s ago 3m 133M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (3m) 26s ago 3m 20.5M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (3m) 26s ago 3m 19.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:9283,8765,8443 running (12m) 17s ago 12m 297M - 18.2.2-952-g738c2fd6 68aff9aff3b2 bf370183b69d 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (50s) 26s ago 9m 490M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b21fee18bc67 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (12m) 17s ago 12m 49.6M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (9m) 26s ago 9m 37.8M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (11m) 17s ago 11m 8006k - 1.5.0 0da6a335fe13 c34cc077c93f 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (9m) 26s ago 9m 7998k - 1.5.0 0da6a335fe13 189e840ee2f1 2024-03-24T00:48:31.500 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (8m) 17s ago 8m 103M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:48:31.501 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (7m) 17s ago 7m 111M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:48:31.501 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (7m) 17s ago 7m 89.9M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:48:31.501 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (6m) 26s ago 6m 109M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:48:31.501 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (6m) 26s ago 5m 99.5M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:48:31.501 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (5m) 26s ago 5m 97.5M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:48:31.501 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 starting - - - - 2024-03-24T00:48:31.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.502+0000 7ff50e800700 1 -- 172.21.15.38:0/4125849097 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7ff4ec06d5b0 msgr2=0x7ff4ec06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:31.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.502+0000 7ff50e800700 1 --2- 172.21.15.38:0/4125849097 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7ff4ec06d5b0 0x7ff4ec06fa70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8005950 tx=0x7ff4f80058e0 comp rx=0 tx=0).stop 2024-03-24T00:48:31.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.502+0000 7ff50e800700 1 -- 172.21.15.38:0/4125849097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508103ef0 msgr2=0x7ff5081969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:31.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.502+0000 7ff50e800700 1 --2- 172.21.15.38:0/4125849097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508103ef0 0x7ff5081969b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400bd70 tx=0x7ff4f400be50 comp rx=0 tx=0).stop 2024-03-24T00:48:31.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.503+0000 7ff50e800700 1 -- 172.21.15.38:0/4125849097 shutdown_connections 2024-03-24T00:48:31.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.503+0000 7ff50e800700 1 --2- 172.21.15.38:0/4125849097 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7ff4ec06d5b0 0x7ff4ec06fa70 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8005950 tx=0x7ff4f80058e0 comp rx=0 tx=0).stop 2024-03-24T00:48:31.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.503+0000 7ff50e800700 1 --2- 172.21.15.38:0/4125849097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5081015c0 0x7ff508196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:31.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.503+0000 7ff50e800700 1 --2- 172.21.15.38:0/4125849097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508103ef0 0x7ff5081969b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:31.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.503+0000 7ff50e800700 1 -- 172.21.15.38:0/4125849097 >> 172.21.15.38:0/4125849097 conn(0x7ff5080faf70 msgr2=0x7ff5080fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:48:31.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.505+0000 7ff50e800700 1 -- 172.21.15.38:0/4125849097 shutdown_connections 2024-03-24T00:48:31.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.506+0000 7ff50e800700 1 -- 172.21.15.38:0/4125849097 wait complete. 2024-03-24T00:48:31.605 INFO:tasks.workunit.client.0.smithi038.stdout:2/11: dwrite f0 [0,4194304] 0 2024-03-24T00:48:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.740+0000 7f8868e00700 1 -- 172.21.15.38:0/4244771480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8864103cf0 msgr2=0x7f88641060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.740+0000 7f8868e00700 1 --2- 172.21.15.38:0/4244771480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8864103cf0 0x7f88641060e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8854009910 tx=0x7f8854009c20 comp rx=0 tx=0).stop 2024-03-24T00:48:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.740+0000 7f8868e00700 1 -- 172.21.15.38:0/4244771480 shutdown_connections 2024-03-24T00:48:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.740+0000 7f8868e00700 1 --2- 172.21.15.38:0/4244771480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8864103cf0 0x7f88641060e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.740+0000 7f8868e00700 1 --2- 172.21.15.38:0/4244771480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88641013c0 0x7f88641037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.740+0000 7f8868e00700 1 -- 172.21.15.38:0/4244771480 >> 172.21.15.38:0/4244771480 conn(0x7f88640fadf0 msgr2=0x7f88640fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:48:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.740+0000 7f8868e00700 1 -- 172.21.15.38:0/4244771480 shutdown_connections 2024-03-24T00:48:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.741+0000 7f8868e00700 1 -- 172.21.15.38:0/4244771480 wait complete. 2024-03-24T00:48:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.742+0000 7f8868e00700 1 Processor -- start 2024-03-24T00:48:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.742+0000 7f8868e00700 1 -- start start 2024-03-24T00:48:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.742+0000 7f8868e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88641013c0 0x7f8864198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.742+0000 7f8868e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8864103cf0 0x7f8864198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.742+0000 7f8868e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88641990e0 con 0x7f88641013c0 2024-03-24T00:48:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.742+0000 7f8868e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8864199220 con 0x7f8864103cf0 2024-03-24T00:48:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.743+0000 7f8863400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88641013c0 0x7f8864198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.743+0000 7f8863400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88641013c0 0x7f8864198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45136/0 (socket says 172.21.15.38:45136) 2024-03-24T00:48:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.743+0000 7f8863400700 1 -- 172.21.15.38:0/3500958582 learned_addr learned my addr 172.21.15.38:0/3500958582 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:48:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.743+0000 7f8862a00700 1 --2- 172.21.15.38:0/3500958582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8864103cf0 0x7f8864198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:31.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.743+0000 7f8863400700 1 -- 172.21.15.38:0/3500958582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8864103cf0 msgr2=0x7f8864198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:31.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.743+0000 7f8863400700 1 --2- 172.21.15.38:0/3500958582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8864103cf0 0x7f8864198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:31.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.743+0000 7f8863400700 1 -- 172.21.15.38:0/3500958582 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88540095f0 con 0x7f88641013c0 2024-03-24T00:48:31.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.744+0000 7f8863400700 1 --2- 172.21.15.38:0/3500958582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88641013c0 0x7f8864198580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f884c00c8a0 tx=0x7f884c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:48:31.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.744+0000 7f885be00700 1 -- 172.21.15.38:0/3500958582 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f884c00cea0 con 0x7f88641013c0 2024-03-24T00:48:31.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.744+0000 7f885be00700 1 -- 172.21.15.38:0/3500958582 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f884c004830 con 0x7f88641013c0 2024-03-24T00:48:31.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.744+0000 7f8868e00700 1 -- 172.21.15.38:0/3500958582 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886419dbe0 con 0x7f88641013c0 2024-03-24T00:48:31.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.744+0000 7f885be00700 1 -- 172.21.15.38:0/3500958582 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f884c0055b0 con 0x7f88641013c0 2024-03-24T00:48:31.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.744+0000 7f8868e00700 1 -- 172.21.15.38:0/3500958582 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886419e130 con 0x7f88641013c0 2024-03-24T00:48:31.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.746+0000 7f8868e00700 1 -- 172.21.15.38:0/3500958582 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886404f2d0 con 0x7f88641013c0 2024-03-24T00:48:31.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.747+0000 7f885be00700 1 -- 172.21.15.38:0/3500958582 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 25) v1 ==== 91370+0+0 (secure 0 0 0) 0x7f884c0049a0 con 0x7f88641013c0 2024-03-24T00:48:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.748+0000 7f885be00700 1 --2- 172.21.15.38:0/3500958582 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f884806d5b0 0x7f884806fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.748+0000 7f885be00700 1 -- 172.21.15.38:0/3500958582 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f884c08cc30 con 0x7f88641013c0 2024-03-24T00:48:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.749+0000 7f8862a00700 1 --2- 172.21.15.38:0/3500958582 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f884806d5b0 0x7f884806fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.751+0000 7f8862a00700 1 --2- 172.21.15.38:0/3500958582 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f884806d5b0 0x7f884806fa70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f88540051c0 tx=0x7f885400b540 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:48:31.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:31.753+0000 7f885be00700 1 -- 172.21.15.38:0/3500958582 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f884c059fc0 con 0x7f88641013c0 2024-03-24T00:48:32.018 INFO:tasks.workunit.client.1.smithi088.stdout:9/11: mkdir d1 0 2024-03-24T00:48:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:31 smithi038 ceph-mon[29271]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:48:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:31 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:31 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:31 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:48:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:31 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:48:32.138 INFO:tasks.workunit.client.1.smithi088.stdout:0/13: getdents . 0 2024-03-24T00:48:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:31 smithi088 ceph-mon[36610]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:48:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:31 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:31 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:32.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:31 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:48:32.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:31 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:48:32.204 INFO:tasks.workunit.client.1.smithi088.stdout:0/14: dwrite f3 [0,4194304] 0 2024-03-24T00:48:32.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.251+0000 7f8868e00700 1 -- 172.21.15.38:0/3500958582 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f886419e5a0 con 0x7f88641013c0 2024-03-24T00:48:32.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.252+0000 7f885be00700 1 -- 172.21.15.38:0/3500958582 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f884c01d070 con 0x7f88641013c0 2024-03-24T00:48:32.252 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:48:32.252 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:48:32.252 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:48:32.252 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:48:32.252 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:48:32.252 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 1, 2024-03-24T00:48:32.252 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-24T00:48:32.252 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:48:32.252 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:48:32.253 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:48:32.253 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:48:32.253 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:48:32.253 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:48:32.253 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:48:32.253 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:48:32.253 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 13, 2024-03-24T00:48:32.253 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-24T00:48:32.253 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:48:32.253 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:48:32.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.255+0000 7f8868e00700 1 -- 172.21.15.38:0/3500958582 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f884806d5b0 msgr2=0x7f884806fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:32.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.255+0000 7f8868e00700 1 --2- 172.21.15.38:0/3500958582 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f884806d5b0 0x7f884806fa70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f88540051c0 tx=0x7f885400b540 comp rx=0 tx=0).stop 2024-03-24T00:48:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.255+0000 7f8868e00700 1 -- 172.21.15.38:0/3500958582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88641013c0 msgr2=0x7f8864198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.255+0000 7f8868e00700 1 --2- 172.21.15.38:0/3500958582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88641013c0 0x7f8864198580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f884c00c8a0 tx=0x7f884c00cc60 comp rx=0 tx=0).stop 2024-03-24T00:48:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.255+0000 7f8868e00700 1 -- 172.21.15.38:0/3500958582 shutdown_connections 2024-03-24T00:48:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.255+0000 7f8868e00700 1 --2- 172.21.15.38:0/3500958582 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f884806d5b0 0x7f884806fa70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.255+0000 7f8868e00700 1 --2- 172.21.15.38:0/3500958582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88641013c0 0x7f8864198580 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.255+0000 7f8868e00700 1 --2- 172.21.15.38:0/3500958582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8864103cf0 0x7f8864198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.255+0000 7f8868e00700 1 -- 172.21.15.38:0/3500958582 >> 172.21.15.38:0/3500958582 conn(0x7f88640fadf0 msgr2=0x7f88640ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:48:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.256+0000 7f8868e00700 1 -- 172.21.15.38:0/3500958582 shutdown_connections 2024-03-24T00:48:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.256+0000 7f8868e00700 1 -- 172.21.15.38:0/3500958582 wait complete. 2024-03-24T00:48:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.445+0000 7f4713200700 1 -- 172.21.15.38:0/3550038979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f470c101cd0 msgr2=0x7f470c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.445+0000 7f4713200700 1 --2- 172.21.15.38:0/3550038979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f470c101cd0 0x7f470c104160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009910 tx=0x7f46f8009c20 comp rx=0 tx=0).stop 2024-03-24T00:48:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.445+0000 7f4713200700 1 -- 172.21.15.38:0/3550038979 shutdown_connections 2024-03-24T00:48:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.445+0000 7f4713200700 1 --2- 172.21.15.38:0/3550038979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f470c101cd0 0x7f470c104160 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.445+0000 7f4713200700 1 --2- 172.21.15.38:0/3550038979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f470c0ff360 0x7f470c101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.445+0000 7f4713200700 1 -- 172.21.15.38:0/3550038979 >> 172.21.15.38:0/3550038979 conn(0x7f470c0faf80 msgr2=0x7f470c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:48:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.446+0000 7f4713200700 1 -- 172.21.15.38:0/3550038979 shutdown_connections 2024-03-24T00:48:32.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.446+0000 7f4713200700 1 -- 172.21.15.38:0/3550038979 wait complete. 2024-03-24T00:48:32.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.447+0000 7f4713200700 1 Processor -- start 2024-03-24T00:48:32.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.447+0000 7f4713200700 1 -- start start 2024-03-24T00:48:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.447+0000 7f4713200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f470c0ff360 0x7f470c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.447+0000 7f4713200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f470c101cd0 0x7f470c19d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.447+0000 7f4713200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c19d700 con 0x7f470c0ff360 2024-03-24T00:48:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.447+0000 7f4713200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c19d840 con 0x7f470c101cd0 2024-03-24T00:48:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.447+0000 7f470be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f470c0ff360 0x7f470c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.447+0000 7f470be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f470c0ff360 0x7f470c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45164/0 (socket says 172.21.15.38:45164) 2024-03-24T00:48:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.447+0000 7f470be00700 1 -- 172.21.15.38:0/3341139377 learned_addr learned my addr 172.21.15.38:0/3341139377 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:48:32.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.448+0000 7f470b400700 1 --2- 172.21.15.38:0/3341139377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f470c101cd0 0x7f470c19d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.448+0000 7f470be00700 1 -- 172.21.15.38:0/3341139377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f470c101cd0 msgr2=0x7f470c19d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.448+0000 7f470be00700 1 --2- 172.21.15.38:0/3341139377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f470c101cd0 0x7f470c19d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.448+0000 7f470be00700 1 -- 172.21.15.38:0/3341139377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f80095f0 con 0x7f470c0ff360 2024-03-24T00:48:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.448+0000 7f470be00700 1 --2- 172.21.15.38:0/3341139377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f470c0ff360 0x7f470c19cba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00b4b0 tx=0x7f46fc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:48:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.448+0000 7f4708c00700 1 -- 172.21.15.38:0/3341139377 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46fc011800 con 0x7f470c0ff360 2024-03-24T00:48:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.449+0000 7f4708c00700 1 -- 172.21.15.38:0/3341139377 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46fc011e20 con 0x7f470c0ff360 2024-03-24T00:48:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.449+0000 7f4708c00700 1 -- 172.21.15.38:0/3341139377 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46fc00f2e0 con 0x7f470c0ff360 2024-03-24T00:48:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.449+0000 7f4713200700 1 -- 172.21.15.38:0/3341139377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c1a22a0 con 0x7f470c0ff360 2024-03-24T00:48:32.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.449+0000 7f4713200700 1 -- 172.21.15.38:0/3341139377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c1a2870 con 0x7f470c0ff360 2024-03-24T00:48:32.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.450+0000 7f4713200700 1 -- 172.21.15.38:0/3341139377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c04ea80 con 0x7f470c0ff360 2024-03-24T00:48:32.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.452+0000 7f4708c00700 1 -- 172.21.15.38:0/3341139377 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 25) v1 ==== 91370+0+0 (secure 0 0 0) 0x7f46fc010380 con 0x7f470c0ff360 2024-03-24T00:48:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.453+0000 7f4708c00700 1 --2- 172.21.15.38:0/3341139377 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f46f006d5b0 0x7f46f006fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:48:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.453+0000 7f4708c00700 1 -- 172.21.15.38:0/3341139377 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f46fc08cac0 con 0x7f470c0ff360 2024-03-24T00:48:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.454+0000 7f470b400700 1 --2- 172.21.15.38:0/3341139377 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f46f006d5b0 0x7f46f006fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:48:32.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.455+0000 7f470b400700 1 --2- 172.21.15.38:0/3341139377 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f46f006d5b0 0x7f46f006fa70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f46f80051c0 tx=0x7f46f800b540 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:48:32.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.461+0000 7f4708c00700 1 -- 172.21.15.38:0/3341139377 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f46fc05a010 con 0x7f470c0ff360 2024-03-24T00:48:32.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.763+0000 7f4713200700 1 -- 172.21.15.38:0/3341139377 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f470c1a2b90 con 0x7f46f006d5b0 2024-03-24T00:48:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.766+0000 7f4708c00700 1 -- 172.21.15.38:0/3341139377 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f470c1a2b90 con 0x7f46f006d5b0 2024-03-24T00:48:32.766 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:48:32.766 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:48:32.766 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:48:32.766 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-24T00:48:32.766 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [], 2024-03-24T00:48:32.766 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "1/2 daemons upgraded", 2024-03-24T00:48:32.766 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mgr daemons", 2024-03-24T00:48:32.767 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:48:32.767 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:48:32.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.768+0000 7f4713200700 1 -- 172.21.15.38:0/3341139377 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f46f006d5b0 msgr2=0x7f46f006fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:32.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.768+0000 7f4713200700 1 --2- 172.21.15.38:0/3341139377 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f46f006d5b0 0x7f46f006fa70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f46f80051c0 tx=0x7f46f800b540 comp rx=0 tx=0).stop 2024-03-24T00:48:32.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.768+0000 7f4713200700 1 -- 172.21.15.38:0/3341139377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f470c0ff360 msgr2=0x7f470c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:48:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.768+0000 7f4713200700 1 --2- 172.21.15.38:0/3341139377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f470c0ff360 0x7f470c19cba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00b4b0 tx=0x7f46fc00b870 comp rx=0 tx=0).stop 2024-03-24T00:48:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.769+0000 7f4713200700 1 -- 172.21.15.38:0/3341139377 shutdown_connections 2024-03-24T00:48:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.769+0000 7f4713200700 1 --2- 172.21.15.38:0/3341139377 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f46f006d5b0 0x7f46f006fa70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.769+0000 7f4713200700 1 --2- 172.21.15.38:0/3341139377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f470c0ff360 0x7f470c19cba0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.769+0000 7f4713200700 1 --2- 172.21.15.38:0/3341139377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f470c101cd0 0x7f470c19d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:48:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.769+0000 7f4713200700 1 -- 172.21.15.38:0/3341139377 >> 172.21.15.38:0/3341139377 conn(0x7f470c0faf80 msgr2=0x7f470c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:48:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.769+0000 7f4713200700 1 -- 172.21.15.38:0/3341139377 shutdown_connections 2024-03-24T00:48:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:48:32.769+0000 7f4713200700 1 -- 172.21.15.38:0/3341139377 wait complete. 2024-03-24T00:48:32.990 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:32 smithi038 ceph-mon[29271]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:48:32.990 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:32 smithi038 ceph-mon[29271]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:48:32.991 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:32 smithi038 ceph-mon[29271]: Upgrade: Need to upgrade myself (mgr.smithi088.njctea) 2024-03-24T00:48:32.991 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:32 smithi038 ceph-mon[29271]: Upgrade: Need to upgrade myself (mgr.smithi088.njctea) 2024-03-24T00:48:32.991 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:32 smithi038 ceph-mon[29271]: pgmap v22: 65 pgs: 65 active+clean; 227 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 158 op/s 2024-03-24T00:48:32.991 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:32 smithi038 ceph-mon[29271]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:48:32.991 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:32 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3500958582' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:48:33.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:32 smithi088 ceph-mon[36610]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:48:33.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:32 smithi088 ceph-mon[36610]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:48:33.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:32 smithi088 ceph-mon[36610]: Upgrade: Need to upgrade myself (mgr.smithi088.njctea) 2024-03-24T00:48:33.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:32 smithi088 ceph-mon[36610]: Upgrade: Need to upgrade myself (mgr.smithi088.njctea) 2024-03-24T00:48:33.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:32 smithi088 ceph-mon[36610]: pgmap v22: 65 pgs: 65 active+clean; 227 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 158 op/s 2024-03-24T00:48:33.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:32 smithi088 ceph-mon[36610]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:48:33.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:32 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3500958582' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:48:33.484 INFO:tasks.workunit.client.0.smithi038.stdout:6/19: link d1/f2 d1/f5 0 2024-03-24T00:48:33.484 INFO:tasks.workunit.client.0.smithi038.stdout:6/20: chown d1 1182028 1 2024-03-24T00:48:33.488 INFO:tasks.workunit.client.0.smithi038.stdout:9/20: link l2 l7 0 2024-03-24T00:48:33.489 INFO:tasks.workunit.client.0.smithi038.stdout:3/34: link d3/f6 d3/d9/fa 0 2024-03-24T00:48:33.493 INFO:tasks.workunit.client.0.smithi038.stdout:1/23: symlink l6 0 2024-03-24T00:48:33.493 INFO:tasks.workunit.client.0.smithi038.stdout:1/24: rmdir - no directory 2024-03-24T00:48:33.494 INFO:tasks.workunit.client.0.smithi038.stdout:4/18: truncate f3 2543101 0 2024-03-24T00:48:33.497 INFO:tasks.workunit.client.0.smithi038.stdout:0/19: creat d0/f5 x:0 0 0 2024-03-24T00:48:33.497 INFO:tasks.workunit.client.0.smithi038.stdout:0/20: creat d0/f6 x:0 0 0 2024-03-24T00:48:33.497 INFO:tasks.workunit.client.0.smithi038.stdout:0/21: creat d0/f7 x:0 0 0 2024-03-24T00:48:33.497 INFO:tasks.workunit.client.0.smithi038.stdout:0/22: write d0/f2 [1568995,116698] 0 2024-03-24T00:48:33.497 INFO:tasks.workunit.client.0.smithi038.stdout:0/23: creat d0/f8 x:0 0 0 2024-03-24T00:48:33.539 INFO:tasks.workunit.client.0.smithi038.stdout:9/21: read f3 [2810295,99264] 0 2024-03-24T00:48:33.539 INFO:tasks.workunit.client.0.smithi038.stdout:9/22: creat f8 x:0 0 0 2024-03-24T00:48:33.555 INFO:tasks.workunit.client.0.smithi038.stdout:5/6: symlink l1 0 2024-03-24T00:48:33.555 INFO:tasks.workunit.client.0.smithi038.stdout:5/7: dread - no filename 2024-03-24T00:48:33.555 INFO:tasks.workunit.client.0.smithi038.stdout:5/8: write - no filename 2024-03-24T00:48:33.555 INFO:tasks.workunit.client.0.smithi038.stdout:5/9: write - no filename 2024-03-24T00:48:33.555 INFO:tasks.workunit.client.0.smithi038.stdout:5/10: dwrite - no filename 2024-03-24T00:48:33.555 INFO:tasks.workunit.client.0.smithi038.stdout:5/11: creat f2 x:0 0 0 2024-03-24T00:48:33.555 INFO:tasks.workunit.client.0.smithi038.stdout:5/12: readlink l1 0 2024-03-24T00:48:33.555 INFO:tasks.workunit.client.0.smithi038.stdout:5/13: read - f2 zero size 2024-03-24T00:48:33.570 INFO:tasks.workunit.client.0.smithi038.stdout:3/35: rmdir d3/d7 0 2024-03-24T00:48:33.576 INFO:tasks.workunit.client.0.smithi038.stdout:1/25: link f4 f7 0 2024-03-24T00:48:33.576 INFO:tasks.workunit.client.0.smithi038.stdout:1/26: write f7 [1846018,69413] 0 2024-03-24T00:48:33.576 INFO:tasks.workunit.client.0.smithi038.stdout:1/27: chown f1 128130 1 2024-03-24T00:48:33.576 INFO:tasks.workunit.client.0.smithi038.stdout:1/28: creat f8 x:0 0 0 2024-03-24T00:48:33.579 INFO:tasks.workunit.client.0.smithi038.stdout:4/19: symlink l5 0 2024-03-24T00:48:33.585 INFO:tasks.workunit.client.0.smithi038.stdout:0/24: symlink d0/l9 0 2024-03-24T00:48:33.588 INFO:tasks.workunit.client.0.smithi038.stdout:9/23: stat l7 0 2024-03-24T00:48:33.593 INFO:tasks.workunit.client.0.smithi038.stdout:3/36: creat d3/d9/fb x:0 0 0 2024-03-24T00:48:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:3/37: dwrite d3/f6 [0,4194304] 0 2024-03-24T00:48:33.676 INFO:tasks.workunit.client.0.smithi038.stdout:0/25: dwrite d0/f7 [0,4194304] 0 2024-03-24T00:48:33.681 INFO:tasks.workunit.client.0.smithi038.stdout:0/26: unlink d0/f8 0 2024-03-24T00:48:33.683 INFO:tasks.workunit.client.0.smithi038.stdout:3/38: dread d3/f6 [0,4194304] 0 2024-03-24T00:48:33.683 INFO:tasks.workunit.client.0.smithi038.stdout:1/29: dwrite f7 [0,4194304] 0 2024-03-24T00:48:33.683 INFO:tasks.workunit.client.0.smithi038.stdout:1/30: chown l3 16341176 1 2024-03-24T00:48:33.683 INFO:tasks.workunit.client.0.smithi038.stdout:9/24: dwrite f8 [0,4194304] 0 2024-03-24T00:48:33.683 INFO:tasks.workunit.client.0.smithi038.stdout:9/25: chown f8 49 1 2024-03-24T00:48:33.690 INFO:tasks.workunit.client.0.smithi038.stdout:3/39: mkdir d3/d9/dc 0 2024-03-24T00:48:33.708 INFO:tasks.workunit.client.0.smithi038.stdout:1/31: read f7 [1878667,31337] 0 2024-03-24T00:48:33.710 INFO:tasks.workunit.client.0.smithi038.stdout:3/40: dread d3/f6 [0,4194304] 0 2024-03-24T00:48:33.715 INFO:tasks.workunit.client.0.smithi038.stdout:3/41: rename d3/d9/fa to d3/fd 0 2024-03-24T00:48:33.752 INFO:tasks.workunit.client.0.smithi038.stdout:1/32: dwrite f5 [0,4194304] 0 2024-03-24T00:48:33.799 INFO:tasks.workunit.client.1.smithi088.stdout:3/14: creat d3/f5 x:0 0 0 2024-03-24T00:48:33.810 INFO:tasks.workunit.client.0.smithi038.stdout:1/33: dwrite f5 [0,4194304] 0 2024-03-24T00:48:33.909 INFO:tasks.workunit.client.1.smithi088.stdout:3/15: dwrite f2 [4194304,4194304] 0 2024-03-24T00:48:34.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:33 smithi038 ceph-mon[29271]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:48:34.163 INFO:tasks.workunit.client.1.smithi088.stdout:1/14: mknod c3 0 2024-03-24T00:48:34.169 INFO:tasks.workunit.client.1.smithi088.stdout:1/15: fsync f0 0 2024-03-24T00:48:34.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:33 smithi088 ceph-mon[36610]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:48:34.183 INFO:tasks.workunit.client.1.smithi088.stdout:1/16: dread f1 [0,4194304] 0 2024-03-24T00:48:34.183 INFO:tasks.workunit.client.1.smithi088.stdout:1/17: chown f2 13122 1 2024-03-24T00:48:34.194 INFO:tasks.workunit.client.0.smithi038.stdout:6/21: getdents d1 0 2024-03-24T00:48:34.194 INFO:tasks.workunit.client.0.smithi038.stdout:6/22: chown d1 5 1 2024-03-24T00:48:34.196 INFO:tasks.workunit.client.0.smithi038.stdout:6/23: unlink d1/f2 0 2024-03-24T00:48:34.199 INFO:tasks.workunit.client.0.smithi038.stdout:6/24: creat d1/f6 x:0 0 0 2024-03-24T00:48:34.199 INFO:tasks.workunit.client.0.smithi038.stdout:6/25: creat d1/f7 x:0 0 0 2024-03-24T00:48:34.200 INFO:tasks.workunit.client.0.smithi038.stdout:9/26: truncate f6 1244848 0 2024-03-24T00:48:34.208 INFO:tasks.workunit.client.0.smithi038.stdout:6/26: truncate d1/f5 545551 0 2024-03-24T00:48:34.208 INFO:tasks.workunit.client.0.smithi038.stdout:6/27: truncate d1/f7 524696 0 2024-03-24T00:48:34.208 INFO:tasks.workunit.client.0.smithi038.stdout:6/28: creat d1/f8 x:0 0 0 2024-03-24T00:48:34.208 INFO:tasks.workunit.client.0.smithi038.stdout:6/29: creat d1/f9 x:0 0 0 2024-03-24T00:48:34.216 INFO:tasks.workunit.client.0.smithi038.stdout:6/30: mkdir d1/da 0 2024-03-24T00:48:34.216 INFO:tasks.workunit.client.0.smithi038.stdout:6/31: chown d1/f5 547 1 2024-03-24T00:48:34.220 INFO:tasks.workunit.client.0.smithi038.stdout:9/27: dread f8 [0,4194304] 0 2024-03-24T00:48:34.225 INFO:tasks.workunit.client.0.smithi038.stdout:9/28: link c5 c9 0 2024-03-24T00:48:34.226 INFO:tasks.workunit.client.0.smithi038.stdout:9/29: creat fa x:0 0 0 2024-03-24T00:48:34.293 INFO:tasks.workunit.client.0.smithi038.stdout:6/32: dwrite d1/f7 [0,4194304] 0 2024-03-24T00:48:34.306 INFO:tasks.workunit.client.0.smithi038.stdout:6/33: creat d1/da/fb x:0 0 0 2024-03-24T00:48:34.328 INFO:tasks.workunit.client.1.smithi088.stdout:6/10: rename f1 to f3 0 2024-03-24T00:48:34.332 INFO:tasks.workunit.client.1.smithi088.stdout:0/15: mknod c5 0 2024-03-24T00:48:34.334 INFO:tasks.workunit.client.0.smithi038.stdout:6/34: dread d1/f7 [0,4194304] 0 2024-03-24T00:48:34.334 INFO:tasks.workunit.client.0.smithi038.stdout:6/35: dread - d1/da/fb zero size 2024-03-24T00:48:34.335 INFO:tasks.workunit.client.0.smithi038.stdout:6/36: write d1/f9 [1047645,110667] 0 2024-03-24T00:48:34.336 INFO:tasks.workunit.client.1.smithi088.stdout:2/16: truncate f1 1173790 0 2024-03-24T00:48:34.338 INFO:tasks.workunit.client.0.smithi038.stdout:6/37: symlink d1/da/lc 0 2024-03-24T00:48:34.340 INFO:tasks.workunit.client.0.smithi038.stdout:6/38: symlink d1/ld 0 2024-03-24T00:48:34.341 INFO:tasks.workunit.client.0.smithi038.stdout:6/39: write d1/f9 [556237,115905] 0 2024-03-24T00:48:34.372 INFO:tasks.workunit.client.1.smithi088.stdout:0/16: dread f3 [0,4194304] 0 2024-03-24T00:48:34.372 INFO:tasks.workunit.client.0.smithi038.stdout:1/34: getdents . 0 2024-03-24T00:48:34.404 INFO:tasks.workunit.client.0.smithi038.stdout:8/4: sync 2024-03-24T00:48:34.404 INFO:tasks.workunit.client.0.smithi038.stdout:7/12: sync 2024-03-24T00:48:34.404 INFO:tasks.workunit.client.0.smithi038.stdout:8/5: stat d0 0 2024-03-24T00:48:34.404 INFO:tasks.workunit.client.0.smithi038.stdout:2/12: sync 2024-03-24T00:48:34.407 INFO:tasks.workunit.client.0.smithi038.stdout:8/6: mknod d0/c1 0 2024-03-24T00:48:34.407 INFO:tasks.workunit.client.1.smithi088.stdout:0/17: dread f3 [0,4194304] 0 2024-03-24T00:48:34.409 INFO:tasks.workunit.client.0.smithi038.stdout:2/13: unlink f1 0 2024-03-24T00:48:34.410 INFO:tasks.workunit.client.0.smithi038.stdout:6/40: dwrite d1/f6 [0,4194304] 0 2024-03-24T00:48:34.410 INFO:tasks.workunit.client.0.smithi038.stdout:6/41: truncate d1/f9 2001717 0 2024-03-24T00:48:34.410 INFO:tasks.workunit.client.0.smithi038.stdout:6/42: truncate d1/f7 4941788 0 2024-03-24T00:48:34.411 INFO:tasks.workunit.client.0.smithi038.stdout:6/43: truncate d1/f5 1184315 0 2024-03-24T00:48:34.411 INFO:tasks.workunit.client.0.smithi038.stdout:6/44: creat d1/fe x:0 0 0 2024-03-24T00:48:34.411 INFO:tasks.workunit.client.0.smithi038.stdout:8/7: symlink d0/l2 0 2024-03-24T00:48:34.425 INFO:tasks.workunit.client.0.smithi038.stdout:2/14: rename f0 to f2 0 2024-03-24T00:48:34.426 INFO:tasks.workunit.client.1.smithi088.stdout:0/18: dread f3 [0,4194304] 0 2024-03-24T00:48:34.433 INFO:tasks.workunit.client.0.smithi038.stdout:6/45: symlink d1/da/lf 0 2024-03-24T00:48:34.434 INFO:tasks.workunit.client.0.smithi038.stdout:8/8: mknod d0/c3 0 2024-03-24T00:48:34.438 INFO:tasks.workunit.client.0.smithi038.stdout:2/15: mknod c3 0 2024-03-24T00:48:34.439 INFO:tasks.workunit.client.0.smithi038.stdout:0/27: getdents d0 0 2024-03-24T00:48:34.439 INFO:tasks.workunit.client.0.smithi038.stdout:0/28: dread - d0/f6 zero size 2024-03-24T00:48:34.439 INFO:tasks.workunit.client.0.smithi038.stdout:1/35: dwrite f8 [0,4194304] 0 2024-03-24T00:48:34.440 INFO:tasks.workunit.client.0.smithi038.stdout:8/9: link d0/c1 d0/c4 0 2024-03-24T00:48:34.442 INFO:tasks.workunit.client.0.smithi038.stdout:2/16: mkdir d4 0 2024-03-24T00:48:34.449 INFO:tasks.workunit.client.0.smithi038.stdout:0/29: creat d0/fa x:0 0 0 2024-03-24T00:48:34.452 INFO:tasks.workunit.client.0.smithi038.stdout:1/36: creat f9 x:0 0 0 2024-03-24T00:48:34.455 INFO:tasks.workunit.client.0.smithi038.stdout:8/10: mknod d0/c5 0 2024-03-24T00:48:34.455 INFO:tasks.workunit.client.0.smithi038.stdout:8/11: dread - no filename 2024-03-24T00:48:34.455 INFO:tasks.workunit.client.0.smithi038.stdout:8/12: dwrite - no filename 2024-03-24T00:48:34.455 INFO:tasks.workunit.client.0.smithi038.stdout:8/13: dwrite - no filename 2024-03-24T00:48:34.455 INFO:tasks.workunit.client.0.smithi038.stdout:8/14: chown d0/c4 4100830 1 2024-03-24T00:48:34.459 INFO:tasks.workunit.client.0.smithi038.stdout:2/17: truncate f2 2010014 0 2024-03-24T00:48:34.461 INFO:tasks.workunit.client.0.smithi038.stdout:0/30: mknod d0/cb 0 2024-03-24T00:48:34.464 INFO:tasks.workunit.client.0.smithi038.stdout:1/37: mkdir da 0 2024-03-24T00:48:34.464 INFO:tasks.workunit.client.0.smithi038.stdout:1/38: chown f5 240911 1 2024-03-24T00:48:34.466 INFO:tasks.workunit.client.0.smithi038.stdout:8/15: stat d0/c1 0 2024-03-24T00:48:34.466 INFO:tasks.workunit.client.0.smithi038.stdout:8/16: fsync - no filename 2024-03-24T00:48:34.468 INFO:tasks.workunit.client.0.smithi038.stdout:8/17: symlink d0/l6 0 2024-03-24T00:48:34.471 INFO:tasks.workunit.client.0.smithi038.stdout:7/13: dwrite d2/f4 [0,4194304] 0 2024-03-24T00:48:34.471 INFO:tasks.workunit.client.0.smithi038.stdout:7/14: creat d2/f5 x:0 0 0 2024-03-24T00:48:34.471 INFO:tasks.workunit.client.0.smithi038.stdout:8/18: link d0/c4 d0/c7 0 2024-03-24T00:48:34.476 INFO:tasks.workunit.client.0.smithi038.stdout:7/15: link d2/f4 d2/f6 0 2024-03-24T00:48:34.476 INFO:tasks.workunit.client.0.smithi038.stdout:7/16: chown d2/f5 65503 1 2024-03-24T00:48:34.477 INFO:tasks.workunit.client.0.smithi038.stdout:7/17: creat d2/f7 x:0 0 0 2024-03-24T00:48:34.478 INFO:tasks.workunit.client.0.smithi038.stdout:3/42: getdents d3/d9 0 2024-03-24T00:48:34.480 INFO:tasks.workunit.client.0.smithi038.stdout:8/19: mknod d0/c8 0 2024-03-24T00:48:34.480 INFO:tasks.workunit.client.0.smithi038.stdout:8/20: chown d0/c5 14396 1 2024-03-24T00:48:34.483 INFO:tasks.workunit.client.0.smithi038.stdout:0/31: dread d0/f2 [0,4194304] 0 2024-03-24T00:48:34.489 INFO:tasks.workunit.client.0.smithi038.stdout:7/18: rename d2/f7 to d2/f8 0 2024-03-24T00:48:34.492 INFO:tasks.workunit.client.0.smithi038.stdout:7/19: unlink d2/f6 0 2024-03-24T00:48:34.492 INFO:tasks.workunit.client.0.smithi038.stdout:7/20: chown d2/f4 62594 1 2024-03-24T00:48:34.492 INFO:tasks.workunit.client.0.smithi038.stdout:7/21: readlink l1 0 2024-03-24T00:48:34.493 INFO:tasks.workunit.client.0.smithi038.stdout:7/22: chown d2 0 1 2024-03-24T00:48:34.495 INFO:tasks.workunit.client.0.smithi038.stdout:6/46: dwrite d1/da/fb [0,4194304] 0 2024-03-24T00:48:34.495 INFO:tasks.workunit.client.0.smithi038.stdout:3/43: dread d3/f5 [0,4194304] 0 2024-03-24T00:48:34.496 INFO:tasks.workunit.client.0.smithi038.stdout:7/23: mkdir d2/d9 0 2024-03-24T00:48:34.496 INFO:tasks.workunit.client.0.smithi038.stdout:7/24: chown c0 947 1 2024-03-24T00:48:34.496 INFO:tasks.workunit.client.0.smithi038.stdout:7/25: read - d2/f8 zero size 2024-03-24T00:48:34.496 INFO:tasks.workunit.client.0.smithi038.stdout:7/26: chown c0 231 1 2024-03-24T00:48:34.503 INFO:tasks.workunit.client.0.smithi038.stdout:3/44: mknod d3/ce 0 2024-03-24T00:48:34.509 INFO:tasks.workunit.client.0.smithi038.stdout:3/45: mkdir d3/d9/dc/df 0 2024-03-24T00:48:34.511 INFO:tasks.workunit.client.0.smithi038.stdout:3/46: mknod d3/c10 0 2024-03-24T00:48:34.513 INFO:tasks.workunit.client.0.smithi038.stdout:3/47: link d3/fd d3/d9/dc/f11 0 2024-03-24T00:48:34.527 INFO:tasks.workunit.client.0.smithi038.stdout:7/27: dread d2/f4 [0,4194304] 0 2024-03-24T00:48:34.529 INFO:tasks.workunit.client.0.smithi038.stdout:7/28: mkdir d2/d9/da 0 2024-03-24T00:48:35.030 INFO:tasks.workunit.client.0.smithi038.stdout:9/30: truncate f3 1351573 0 2024-03-24T00:48:35.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:34 smithi038 ceph-mon[29271]: pgmap v23: 65 pgs: 65 active+clean; 227 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 141 op/s 2024-03-24T00:48:35.130 INFO:tasks.workunit.client.0.smithi038.stdout:9/31: dwrite fa [0,4194304] 0 2024-03-24T00:48:35.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:34 smithi088 ceph-mon[36610]: pgmap v23: 65 pgs: 65 active+clean; 227 MiB data, 994 MiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 141 op/s 2024-03-24T00:48:35.201 INFO:tasks.workunit.client.1.smithi088.stdout:6/11: fsync f3 0 2024-03-24T00:48:35.281 INFO:tasks.workunit.client.0.smithi038.stdout:6/47: rmdir d1 39 2024-03-24T00:48:35.285 INFO:tasks.workunit.client.0.smithi038.stdout:6/48: creat d1/f10 x:0 0 0 2024-03-24T00:48:35.291 INFO:tasks.workunit.client.0.smithi038.stdout:6/49: chown d1/fe 1 1 2024-03-24T00:48:35.303 INFO:tasks.workunit.client.1.smithi088.stdout:2/17: dwrite f1 [0,4194304] 0 2024-03-24T00:48:35.314 INFO:tasks.workunit.client.0.smithi038.stdout:0/32: getdents d0 0 2024-03-24T00:48:35.320 INFO:tasks.workunit.client.0.smithi038.stdout:0/33: creat d0/fc x:0 0 0 2024-03-24T00:48:35.320 INFO:tasks.workunit.client.0.smithi038.stdout:0/34: truncate d0/fc 1047294 0 2024-03-24T00:48:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:7/29: rename d2/f8 to d2/d9/fb 0 2024-03-24T00:48:35.357 INFO:tasks.workunit.client.0.smithi038.stdout:0/35: write d0/fc [326439,101060] 0 2024-03-24T00:48:35.357 INFO:tasks.workunit.client.0.smithi038.stdout:0/36: write d0/fa [950784,70393] 0 2024-03-24T00:48:35.357 INFO:tasks.workunit.client.0.smithi038.stdout:0/37: fsync d0/f5 0 2024-03-24T00:48:35.363 INFO:tasks.workunit.client.0.smithi038.stdout:0/38: mknod d0/cd 0 2024-03-24T00:48:35.366 INFO:tasks.workunit.client.0.smithi038.stdout:0/39: symlink d0/le 0 2024-03-24T00:48:35.366 INFO:tasks.workunit.client.0.smithi038.stdout:0/40: creat d0/ff x:0 0 0 2024-03-24T00:48:35.368 INFO:tasks.workunit.client.0.smithi038.stdout:6/50: dwrite d1/f8 [0,4194304] 0 2024-03-24T00:48:35.372 INFO:tasks.workunit.client.0.smithi038.stdout:0/41: link d0/le d0/l10 0 2024-03-24T00:48:35.381 INFO:tasks.workunit.client.0.smithi038.stdout:3/48: getdents d3 0 2024-03-24T00:48:35.381 INFO:tasks.workunit.client.0.smithi038.stdout:3/49: stat d3/d9/dc/f11 0 2024-03-24T00:48:35.381 INFO:tasks.workunit.client.0.smithi038.stdout:3/50: dread - d3/d9/fb zero size 2024-03-24T00:48:35.382 INFO:tasks.workunit.client.0.smithi038.stdout:6/51: stat d1/f6 0 2024-03-24T00:48:35.385 INFO:tasks.workunit.client.0.smithi038.stdout:3/51: creat d3/d9/dc/df/f12 x:0 0 0 2024-03-24T00:48:35.385 INFO:tasks.workunit.client.0.smithi038.stdout:3/52: read - d3/d9/fb zero size 2024-03-24T00:48:35.406 INFO:tasks.workunit.client.1.smithi088.stdout:2/18: dwrite f1 [0,4194304] 0 2024-03-24T00:48:35.427 INFO:tasks.workunit.client.0.smithi038.stdout:2/18: dwrite f2 [0,4194304] 0 2024-03-24T00:48:35.427 INFO:tasks.workunit.client.0.smithi038.stdout:2/19: stat d4 0 2024-03-24T00:48:35.430 INFO:tasks.workunit.client.0.smithi038.stdout:7/30: dwrite d2/f4 [0,4194304] 0 2024-03-24T00:48:35.437 INFO:tasks.workunit.client.0.smithi038.stdout:0/42: dread d0/fc [0,4194304] 0 2024-03-24T00:48:35.449 INFO:tasks.workunit.client.0.smithi038.stdout:3/53: dwrite f1 [0,4194304] 0 2024-03-24T00:48:35.753 INFO:tasks.workunit.client.0.smithi038.stdout:7/31: unlink d2/f5 0 2024-03-24T00:48:35.759 INFO:tasks.workunit.client.0.smithi038.stdout:0/43: mknod d0/c11 0 2024-03-24T00:48:35.769 INFO:tasks.workunit.client.0.smithi038.stdout:3/54: rename d3/c10 to d3/d9/dc/c13 0 2024-03-24T00:48:35.769 INFO:tasks.workunit.client.0.smithi038.stdout:3/55: chown d3/f6 286 1 2024-03-24T00:48:35.778 INFO:tasks.workunit.client.0.smithi038.stdout:0/44: mkdir d0/d12 0 2024-03-24T00:48:35.782 INFO:tasks.workunit.client.0.smithi038.stdout:3/56: mkdir d3/d9/dc/df/d14 0 2024-03-24T00:48:35.790 INFO:tasks.workunit.client.0.smithi038.stdout:0/45: mkdir d0/d12/d13 0 2024-03-24T00:48:35.795 INFO:tasks.workunit.client.0.smithi038.stdout:3/57: mknod d3/c15 0 2024-03-24T00:48:35.800 INFO:tasks.workunit.client.0.smithi038.stdout:0/46: unlink d0/f1 0 2024-03-24T00:48:35.804 INFO:tasks.workunit.client.0.smithi038.stdout:3/58: creat d3/d9/f16 x:0 0 0 2024-03-24T00:48:35.808 INFO:tasks.workunit.client.0.smithi038.stdout:0/47: mkdir d0/d14 0 2024-03-24T00:48:35.808 INFO:tasks.workunit.client.0.smithi038.stdout:0/48: getdents d0/d14 0 2024-03-24T00:48:35.812 INFO:tasks.workunit.client.0.smithi038.stdout:3/59: creat d3/d9/dc/df/d14/f17 x:0 0 0 2024-03-24T00:48:35.812 INFO:tasks.workunit.client.0.smithi038.stdout:3/60: fdatasync d3/d9/f16 0 2024-03-24T00:48:35.812 INFO:tasks.workunit.client.0.smithi038.stdout:3/61: fsync f2 0 2024-03-24T00:48:35.919 INFO:tasks.workunit.client.0.smithi038.stdout:0/49: dwrite d0/fa [0,4194304] 0 2024-03-24T00:48:35.927 INFO:tasks.workunit.client.0.smithi038.stdout:0/50: mkdir d0/d12/d15 0 2024-03-24T00:48:35.927 INFO:tasks.workunit.client.0.smithi038.stdout:0/51: readlink d0/l9 0 2024-03-24T00:48:36.169 INFO:tasks.workunit.client.1.smithi088.stdout:8/6: getdents . 0 2024-03-24T00:48:36.172 INFO:tasks.workunit.client.1.smithi088.stdout:3/16: mkdir d3/d6 0 2024-03-24T00:48:36.172 INFO:tasks.workunit.client.1.smithi088.stdout:3/17: dread - f1 zero size 2024-03-24T00:48:36.172 INFO:tasks.workunit.client.1.smithi088.stdout:3/18: chown f1 85573705 1 2024-03-24T00:48:36.172 INFO:tasks.workunit.client.1.smithi088.stdout:3/19: dread - f1 zero size 2024-03-24T00:48:36.172 INFO:tasks.workunit.client.1.smithi088.stdout:3/20: chown d3/f4 3114 1 2024-03-24T00:48:36.172 INFO:tasks.workunit.client.1.smithi088.stdout:3/21: dread - d3/f4 zero size 2024-03-24T00:48:36.173 INFO:tasks.workunit.client.1.smithi088.stdout:3/22: write d3/f5 [591876,79054] 0 2024-03-24T00:48:36.173 INFO:tasks.workunit.client.1.smithi088.stdout:3/23: creat d3/f7 x:0 0 0 2024-03-24T00:48:36.173 INFO:tasks.workunit.client.1.smithi088.stdout:4/24: truncate f1 1353781 0 2024-03-24T00:48:36.174 INFO:tasks.workunit.client.1.smithi088.stdout:4/25: creat f3 x:0 0 0 2024-03-24T00:48:36.174 INFO:tasks.workunit.client.1.smithi088.stdout:1/18: symlink l4 0 2024-03-24T00:48:36.176 INFO:tasks.workunit.client.1.smithi088.stdout:9/12: rename c0 to d1/c2 0 2024-03-24T00:48:36.176 INFO:tasks.workunit.client.1.smithi088.stdout:9/13: stat d1 0 2024-03-24T00:48:36.176 INFO:tasks.workunit.client.1.smithi088.stdout:9/14: write - no filename 2024-03-24T00:48:36.177 INFO:tasks.workunit.client.1.smithi088.stdout:6/12: symlink l4 0 2024-03-24T00:48:36.181 INFO:tasks.workunit.client.1.smithi088.stdout:3/24: unlink c0 0 2024-03-24T00:48:36.181 INFO:tasks.workunit.client.1.smithi088.stdout:3/25: chown d3 20 1 2024-03-24T00:48:36.186 INFO:tasks.workunit.client.1.smithi088.stdout:0/19: truncate f3 398568 0 2024-03-24T00:48:36.186 INFO:tasks.workunit.client.1.smithi088.stdout:9/15: stat d1/c2 0 2024-03-24T00:48:36.186 INFO:tasks.workunit.client.1.smithi088.stdout:9/16: rename d1 to d1/d3 22 2024-03-24T00:48:36.186 INFO:tasks.workunit.client.1.smithi088.stdout:9/17: chown d1 5334 1 2024-03-24T00:48:36.198 INFO:tasks.workunit.client.0.smithi038.stdout:9/32: rename f3 to fb 0 2024-03-24T00:48:36.201 INFO:tasks.workunit.client.1.smithi088.stdout:6/13: symlink l5 0 2024-03-24T00:48:36.201 INFO:tasks.workunit.client.1.smithi088.stdout:2/19: getdents . 0 2024-03-24T00:48:36.201 INFO:tasks.workunit.client.1.smithi088.stdout:9/18: creat d1/f4 x:0 0 0 2024-03-24T00:48:36.201 INFO:tasks.workunit.client.1.smithi088.stdout:3/26: unlink d3/f5 0 2024-03-24T00:48:36.201 INFO:tasks.workunit.client.1.smithi088.stdout:9/19: creat d1/f5 x:0 0 0 2024-03-24T00:48:36.201 INFO:tasks.workunit.client.1.smithi088.stdout:9/20: write d1/f5 [58674,113641] 0 2024-03-24T00:48:36.201 INFO:tasks.workunit.client.1.smithi088.stdout:9/21: chown d1/c2 140051 1 2024-03-24T00:48:36.203 INFO:tasks.workunit.client.1.smithi088.stdout:3/27: mknod d3/c8 0 2024-03-24T00:48:36.204 INFO:tasks.workunit.client.1.smithi088.stdout:3/28: truncate d3/f4 311055 0 2024-03-24T00:48:36.206 INFO:tasks.workunit.client.1.smithi088.stdout:9/22: link d1/f4 d1/f6 0 2024-03-24T00:48:36.210 INFO:tasks.workunit.client.1.smithi088.stdout:9/23: link d1/f6 d1/f7 0 2024-03-24T00:48:36.210 INFO:tasks.workunit.client.1.smithi088.stdout:9/24: creat d1/f8 x:0 0 0 2024-03-24T00:48:36.215 INFO:tasks.workunit.client.1.smithi088.stdout:8/7: dread f1 [0,4194304] 0 2024-03-24T00:48:36.228 INFO:tasks.workunit.client.1.smithi088.stdout:2/20: dread f1 [0,4194304] 0 2024-03-24T00:48:36.249 INFO:tasks.workunit.client.1.smithi088.stdout:8/8: write f1 [1126403,81424] 0 2024-03-24T00:48:36.249 INFO:tasks.workunit.client.1.smithi088.stdout:8/9: write f2 [697252,60793] 0 2024-03-24T00:48:36.268 INFO:tasks.workunit.client.1.smithi088.stdout:8/10: fsync f1 0 2024-03-24T00:48:36.268 INFO:tasks.workunit.client.1.smithi088.stdout:8/11: write f1 [903427,129675] 0 2024-03-24T00:48:36.269 INFO:tasks.workunit.client.1.smithi088.stdout:6/14: dread f3 [0,4194304] 0 2024-03-24T00:48:36.271 INFO:tasks.workunit.client.1.smithi088.stdout:6/15: symlink l6 0 2024-03-24T00:48:36.272 INFO:tasks.workunit.client.1.smithi088.stdout:6/16: write f3 [4314081,35055] 0 2024-03-24T00:48:36.272 INFO:tasks.workunit.client.1.smithi088.stdout:9/25: dread d1/f5 [0,4194304] 0 2024-03-24T00:48:36.273 INFO:tasks.workunit.client.1.smithi088.stdout:6/17: write f3 [4589693,114811] 0 2024-03-24T00:48:36.275 INFO:tasks.workunit.client.0.smithi038.stdout:9/33: dwrite f8 [0,4194304] 0 2024-03-24T00:48:36.275 INFO:tasks.workunit.client.0.smithi038.stdout:9/34: chown c5 2170 1 2024-03-24T00:48:36.275 INFO:tasks.workunit.client.0.smithi038.stdout:9/35: write fb [2327841,38048] 0 2024-03-24T00:48:36.275 INFO:tasks.workunit.client.1.smithi088.stdout:1/19: dwrite f1 [0,4194304] 0 2024-03-24T00:48:36.275 INFO:tasks.workunit.client.1.smithi088.stdout:1/20: stat f1 0 2024-03-24T00:48:36.283 INFO:tasks.workunit.client.1.smithi088.stdout:9/26: dread d1/f5 [0,4194304] 0 2024-03-24T00:48:36.288 INFO:tasks.workunit.client.1.smithi088.stdout:9/27: mknod d1/c9 0 2024-03-24T00:48:36.288 INFO:tasks.workunit.client.1.smithi088.stdout:9/28: truncate d1/f8 308300 0 2024-03-24T00:48:36.291 INFO:tasks.workunit.client.0.smithi038.stdout:9/36: write f8 [1809490,16882] 0 2024-03-24T00:48:36.291 INFO:tasks.workunit.client.0.smithi038.stdout:9/37: creat fc x:0 0 0 2024-03-24T00:48:36.295 INFO:tasks.workunit.client.0.smithi038.stdout:9/38: rename c5 to cd 0 2024-03-24T00:48:36.296 INFO:tasks.workunit.client.0.smithi038.stdout:9/39: creat fe x:0 0 0 2024-03-24T00:48:36.298 INFO:tasks.workunit.client.0.smithi038.stdout:9/40: mknod cf 0 2024-03-24T00:48:36.302 INFO:tasks.workunit.client.1.smithi088.stdout:6/18: dread f3 [0,4194304] 0 2024-03-24T00:48:36.306 INFO:tasks.workunit.client.1.smithi088.stdout:1/21: dread f0 [0,4194304] 0 2024-03-24T00:48:36.313 INFO:tasks.workunit.client.0.smithi038.stdout:2/20: truncate f2 318063 0 2024-03-24T00:48:36.313 INFO:tasks.workunit.client.0.smithi038.stdout:7/32: truncate d2/f4 1986589 0 2024-03-24T00:48:36.313 INFO:tasks.workunit.client.0.smithi038.stdout:7/33: dread - d2/d9/fb zero size 2024-03-24T00:48:36.318 INFO:tasks.workunit.client.1.smithi088.stdout:6/19: dread f3 [4194304,4194304] 0 2024-03-24T00:48:36.321 INFO:tasks.workunit.client.0.smithi038.stdout:1/39: sync 2024-03-24T00:48:36.321 INFO:tasks.workunit.client.0.smithi038.stdout:5/14: sync 2024-03-24T00:48:36.321 INFO:tasks.workunit.client.0.smithi038.stdout:4/20: sync 2024-03-24T00:48:36.321 INFO:tasks.workunit.client.0.smithi038.stdout:5/15: readlink l1 0 2024-03-24T00:48:36.321 INFO:tasks.workunit.client.0.smithi038.stdout:8/21: sync 2024-03-24T00:48:36.322 INFO:tasks.workunit.client.0.smithi038.stdout:8/22: dread - no filename 2024-03-24T00:48:36.322 INFO:tasks.workunit.client.0.smithi038.stdout:8/23: dwrite - no filename 2024-03-24T00:48:36.322 INFO:tasks.workunit.client.0.smithi038.stdout:8/24: truncate - no filename 2024-03-24T00:48:36.322 INFO:tasks.workunit.client.0.smithi038.stdout:5/16: creat f3 x:0 0 0 2024-03-24T00:48:36.322 INFO:tasks.workunit.client.0.smithi038.stdout:5/17: stat f2 0 2024-03-24T00:48:36.322 INFO:tasks.workunit.client.0.smithi038.stdout:5/18: read - f2 zero size 2024-03-24T00:48:36.322 INFO:tasks.workunit.client.0.smithi038.stdout:2/21: rename c3 to d4/c5 0 2024-03-24T00:48:36.323 INFO:tasks.workunit.client.1.smithi088.stdout:6/20: write f3 [4449270,93281] 0 2024-03-24T00:48:36.324 INFO:tasks.workunit.client.0.smithi038.stdout:4/21: mknod c6 0 2024-03-24T00:48:36.326 INFO:tasks.workunit.client.1.smithi088.stdout:1/22: dread f2 [0,4194304] 0 2024-03-24T00:48:36.330 INFO:tasks.workunit.client.0.smithi038.stdout:8/25: creat d0/f9 x:0 0 0 2024-03-24T00:48:36.330 INFO:tasks.workunit.client.0.smithi038.stdout:8/26: creat d0/fa x:0 0 0 2024-03-24T00:48:36.333 INFO:tasks.workunit.client.1.smithi088.stdout:6/21: rename l4 to l7 0 2024-03-24T00:48:36.341 INFO:tasks.workunit.client.1.smithi088.stdout:6/22: fsync f3 0 2024-03-24T00:48:36.341 INFO:tasks.workunit.client.1.smithi088.stdout:6/23: chown l6 8178 1 2024-03-24T00:48:36.341 INFO:tasks.workunit.client.1.smithi088.stdout:6/24: fsync f3 0 2024-03-24T00:48:36.346 INFO:tasks.workunit.client.1.smithi088.stdout:9/29: dwrite d1/f7 [0,4194304] 0 2024-03-24T00:48:36.349 INFO:tasks.workunit.client.1.smithi088.stdout:6/25: unlink f3 0 2024-03-24T00:48:36.349 INFO:tasks.workunit.client.1.smithi088.stdout:6/26: dread - no filename 2024-03-24T00:48:36.349 INFO:tasks.workunit.client.1.smithi088.stdout:6/27: write - no filename 2024-03-24T00:48:36.349 INFO:tasks.workunit.client.1.smithi088.stdout:6/28: creat f8 x:0 0 0 2024-03-24T00:48:36.349 INFO:tasks.workunit.client.1.smithi088.stdout:6/29: write f8 [243777,58432] 0 2024-03-24T00:48:36.350 INFO:tasks.workunit.client.0.smithi038.stdout:8/27: rename d0/fa to d0/fb 0 2024-03-24T00:48:36.350 INFO:tasks.workunit.client.0.smithi038.stdout:8/28: fdatasync d0/fb 0 2024-03-24T00:48:36.356 INFO:tasks.workunit.client.1.smithi088.stdout:9/30: symlink d1/la 0 2024-03-24T00:48:36.356 INFO:tasks.workunit.client.1.smithi088.stdout:9/31: fdatasync d1/f8 0 2024-03-24T00:48:36.383 INFO:tasks.workunit.client.0.smithi038.stdout:9/41: dwrite fe [0,4194304] 0 2024-03-24T00:48:36.395 INFO:tasks.workunit.client.0.smithi038.stdout:4/22: dread f3 [0,4194304] 0 2024-03-24T00:48:36.395 INFO:tasks.workunit.client.0.smithi038.stdout:9/42: link cf c10 0 2024-03-24T00:48:36.397 INFO:tasks.workunit.client.0.smithi038.stdout:1/40: dwrite f1 [0,4194304] 0 2024-03-24T00:48:36.399 INFO:tasks.workunit.client.0.smithi038.stdout:4/23: symlink l7 0 2024-03-24T00:48:36.399 INFO:tasks.workunit.client.0.smithi038.stdout:4/24: creat f8 x:0 0 0 2024-03-24T00:48:36.402 INFO:tasks.workunit.client.0.smithi038.stdout:1/41: creat da/fb x:0 0 0 2024-03-24T00:48:36.404 INFO:tasks.workunit.client.0.smithi038.stdout:4/25: unlink c2 0 2024-03-24T00:48:36.410 INFO:tasks.workunit.client.0.smithi038.stdout:9/43: dread f8 [0,4194304] 0 2024-03-24T00:48:36.412 INFO:tasks.workunit.client.0.smithi038.stdout:4/26: write f3 [977042,116013] 0 2024-03-24T00:48:36.432 INFO:tasks.workunit.client.0.smithi038.stdout:9/44: dread f4 [0,4194304] 0 2024-03-24T00:48:36.432 INFO:tasks.workunit.client.0.smithi038.stdout:9/45: rmdir - no directory 2024-03-24T00:48:36.434 INFO:tasks.workunit.client.0.smithi038.stdout:9/46: rename c10 to c11 0 2024-03-24T00:48:36.434 INFO:tasks.workunit.client.0.smithi038.stdout:9/47: truncate fb 2569242 0 2024-03-24T00:48:36.434 INFO:tasks.workunit.client.0.smithi038.stdout:9/48: fdatasync fe 0 2024-03-24T00:48:36.436 INFO:tasks.workunit.client.0.smithi038.stdout:9/49: unlink c9 0 2024-03-24T00:48:36.436 INFO:tasks.workunit.client.0.smithi038.stdout:9/50: chown fc 2 1 2024-03-24T00:48:36.439 INFO:tasks.workunit.client.0.smithi038.stdout:9/51: mkdir d12 0 2024-03-24T00:48:36.443 INFO:tasks.workunit.client.1.smithi088.stdout:8/12: dwrite f1 [0,4194304] 0 2024-03-24T00:48:36.444 INFO:tasks.workunit.client.1.smithi088.stdout:8/13: truncate f2 1006631 0 2024-03-24T00:48:36.444 INFO:tasks.workunit.client.1.smithi088.stdout:8/14: truncate f2 1695632 0 2024-03-24T00:48:36.446 INFO:tasks.workunit.client.0.smithi038.stdout:9/52: symlink d12/l13 0 2024-03-24T00:48:36.449 INFO:tasks.workunit.client.0.smithi038.stdout:5/19: dwrite f3 [0,4194304] 0 2024-03-24T00:48:36.449 INFO:tasks.workunit.client.0.smithi038.stdout:5/20: dread - f2 zero size 2024-03-24T00:48:36.449 INFO:tasks.workunit.client.0.smithi038.stdout:5/21: stat f3 0 2024-03-24T00:48:36.450 INFO:tasks.workunit.client.1.smithi088.stdout:8/15: read f2 [637739,111066] 0 2024-03-24T00:48:36.453 INFO:tasks.workunit.client.1.smithi088.stdout:8/16: symlink l3 0 2024-03-24T00:48:36.453 INFO:tasks.workunit.client.1.smithi088.stdout:8/17: fdatasync f2 0 2024-03-24T00:48:36.455 INFO:tasks.workunit.client.1.smithi088.stdout:8/18: link l3 l4 0 2024-03-24T00:48:36.456 INFO:tasks.workunit.client.1.smithi088.stdout:8/19: link l4 l5 0 2024-03-24T00:48:36.476 INFO:tasks.workunit.client.1.smithi088.stdout:8/20: dread f2 [0,4194304] 0 2024-03-24T00:48:36.478 INFO:tasks.workunit.client.1.smithi088.stdout:8/21: symlink l6 0 2024-03-24T00:48:36.480 INFO:tasks.workunit.client.1.smithi088.stdout:8/22: creat f7 x:0 0 0 2024-03-24T00:48:36.481 INFO:tasks.workunit.client.1.smithi088.stdout:8/23: rename l0 to l8 0 2024-03-24T00:48:36.483 INFO:tasks.workunit.client.1.smithi088.stdout:8/24: mkdir d9 0 2024-03-24T00:48:36.483 INFO:tasks.workunit.client.1.smithi088.stdout:8/25: truncate f7 220117 0 2024-03-24T00:48:36.488 INFO:tasks.workunit.client.0.smithi038.stdout:5/22: write f3 [1978785,78429] 0 2024-03-24T00:48:36.488 INFO:tasks.workunit.client.0.smithi038.stdout:4/27: dwrite f3 [0,4194304] 0 2024-03-24T00:48:36.489 INFO:tasks.workunit.client.0.smithi038.stdout:5/23: mknod c4 0 2024-03-24T00:48:36.495 INFO:tasks.workunit.client.1.smithi088.stdout:5/18: sync 2024-03-24T00:48:36.495 INFO:tasks.workunit.client.1.smithi088.stdout:7/9: sync 2024-03-24T00:48:36.497 INFO:tasks.workunit.client.1.smithi088.stdout:5/19: symlink d1/l4 0 2024-03-24T00:48:36.497 INFO:tasks.workunit.client.1.smithi088.stdout:5/20: chown d1/l4 16505655 1 2024-03-24T00:48:36.497 INFO:tasks.workunit.client.1.smithi088.stdout:5/21: chown d1/c2 36422775 1 2024-03-24T00:48:36.497 INFO:tasks.workunit.client.1.smithi088.stdout:5/22: dwrite - no filename 2024-03-24T00:48:36.499 INFO:tasks.workunit.client.1.smithi088.stdout:5/23: unlink d1/c3 0 2024-03-24T00:48:36.500 INFO:tasks.workunit.client.1.smithi088.stdout:5/24: link c0 d1/c5 0 2024-03-24T00:48:36.502 INFO:tasks.workunit.client.1.smithi088.stdout:5/25: symlink d1/l6 0 2024-03-24T00:48:36.503 INFO:tasks.workunit.client.1.smithi088.stdout:5/26: readlink d1/l6 0 2024-03-24T00:48:36.503 INFO:tasks.workunit.client.1.smithi088.stdout:5/27: read - no filename 2024-03-24T00:48:36.512 INFO:tasks.workunit.client.1.smithi088.stdout:5/28: stat c0 0 2024-03-24T00:48:36.514 INFO:tasks.workunit.client.1.smithi088.stdout:4/26: write f1 [865912,32772] 0 2024-03-24T00:48:36.518 INFO:tasks.workunit.client.1.smithi088.stdout:2/21: truncate f1 484147 0 2024-03-24T00:48:36.518 INFO:tasks.workunit.client.1.smithi088.stdout:5/29: creat d1/f7 x:0 0 0 2024-03-24T00:48:36.519 INFO:tasks.workunit.client.1.smithi088.stdout:5/30: write d1/f7 [202749,70785] 0 2024-03-24T00:48:36.523 INFO:tasks.workunit.client.0.smithi038.stdout:9/53: link cd d12/c14 0 2024-03-24T00:48:36.523 INFO:tasks.workunit.client.0.smithi038.stdout:9/54: chown d12/c14 111414926 1 2024-03-24T00:48:36.527 INFO:tasks.workunit.client.0.smithi038.stdout:9/55: creat d12/f15 x:0 0 0 2024-03-24T00:48:36.531 INFO:tasks.workunit.client.0.smithi038.stdout:2/22: link d4/c5 d4/c6 0 2024-03-24T00:48:36.531 INFO:tasks.workunit.client.0.smithi038.stdout:0/52: sync 2024-03-24T00:48:36.531 INFO:tasks.workunit.client.0.smithi038.stdout:6/52: sync 2024-03-24T00:48:36.531 INFO:tasks.workunit.client.0.smithi038.stdout:3/62: sync 2024-03-24T00:48:36.534 INFO:tasks.workunit.client.0.smithi038.stdout:9/56: write f4 [1335711,67324] 0 2024-03-24T00:48:36.539 INFO:tasks.workunit.client.0.smithi038.stdout:2/23: creat d4/f7 x:0 0 0 2024-03-24T00:48:36.539 INFO:tasks.workunit.client.0.smithi038.stdout:2/24: dread - d4/f7 zero size 2024-03-24T00:48:36.539 INFO:tasks.workunit.client.0.smithi038.stdout:2/25: chown d4/c6 2899968 1 2024-03-24T00:48:36.539 INFO:tasks.workunit.client.0.smithi038.stdout:2/26: creat d4/f8 x:0 0 0 2024-03-24T00:48:36.540 INFO:tasks.workunit.client.0.smithi038.stdout:0/53: creat d0/d12/d15/f16 x:0 0 0 2024-03-24T00:48:36.541 INFO:tasks.workunit.client.1.smithi088.stdout:4/27: dread f1 [0,4194304] 0 2024-03-24T00:48:36.541 INFO:tasks.workunit.client.1.smithi088.stdout:4/28: chown c0 52 1 2024-03-24T00:48:36.541 INFO:tasks.workunit.client.1.smithi088.stdout:4/29: truncate f3 962676 0 2024-03-24T00:48:36.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:36 smithi038 ceph-mon[29271]: Upgrade: Updating mgr.smithi038.huhioz 2024-03-24T00:48:36.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:36 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:36.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:36 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.huhioz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:48:36.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:36 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.huhioz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:48:36.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:36 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:48:36.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:36 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:48:36.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:36 smithi038 ceph-mon[29271]: Deploying daemon mgr.smithi038.huhioz on smithi038 2024-03-24T00:48:36.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:36 smithi038 ceph-mon[29271]: pgmap v24: 65 pgs: 65 active+clean; 368 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 462 op/s 2024-03-24T00:48:36.548 INFO:tasks.workunit.client.0.smithi038.stdout:6/53: mknod d1/c11 0 2024-03-24T00:48:36.554 INFO:tasks.workunit.client.0.smithi038.stdout:9/57: dread f8 [0,4194304] 0 2024-03-24T00:48:36.560 INFO:tasks.workunit.client.0.smithi038.stdout:5/24: dwrite f2 [0,4194304] 0 2024-03-24T00:48:36.561 INFO:tasks.workunit.client.0.smithi038.stdout:8/29: getdents d0 0 2024-03-24T00:48:36.561 INFO:tasks.workunit.client.0.smithi038.stdout:2/27: symlink d4/l9 0 2024-03-24T00:48:36.561 INFO:tasks.workunit.client.0.smithi038.stdout:2/28: creat d4/fa x:0 0 0 2024-03-24T00:48:36.563 INFO:tasks.workunit.client.1.smithi088.stdout:9/32: rmdir d1 39 2024-03-24T00:48:36.563 INFO:tasks.workunit.client.1.smithi088.stdout:9/33: rename d1 to d1/db 22 2024-03-24T00:48:36.565 INFO:tasks.workunit.client.1.smithi088.stdout:6/30: fsync f8 0 2024-03-24T00:48:36.565 INFO:tasks.workunit.client.1.smithi088.stdout:6/31: stat l2 0 2024-03-24T00:48:36.570 INFO:tasks.workunit.client.0.smithi038.stdout:9/58: mknod d12/c16 0 2024-03-24T00:48:36.570 INFO:tasks.workunit.client.0.smithi038.stdout:9/59: chown d12/l13 231 1 2024-03-24T00:48:36.574 INFO:tasks.workunit.client.1.smithi088.stdout:8/26: dwrite f2 [0,4194304] 0 2024-03-24T00:48:36.575 INFO:tasks.workunit.client.1.smithi088.stdout:6/32: unlink l6 0 2024-03-24T00:48:36.577 INFO:tasks.workunit.client.1.smithi088.stdout:6/33: write f8 [1222506,37389] 0 2024-03-24T00:48:36.577 INFO:tasks.workunit.client.1.smithi088.stdout:6/34: chown l5 411811 1 2024-03-24T00:48:36.581 INFO:tasks.workunit.client.1.smithi088.stdout:8/27: mknod d9/ca 0 2024-03-24T00:48:36.588 INFO:tasks.workunit.client.0.smithi038.stdout:8/30: link d0/c5 d0/cc 0 2024-03-24T00:48:36.588 INFO:tasks.workunit.client.1.smithi088.stdout:8/28: dread f7 [0,4194304] 0 2024-03-24T00:48:36.589 INFO:tasks.workunit.client.0.smithi038.stdout:6/54: truncate d1/da/fb 2492185 0 2024-03-24T00:48:36.592 INFO:tasks.workunit.client.1.smithi088.stdout:0/20: dwrite f3 [0,4194304] 0 2024-03-24T00:48:36.592 INFO:tasks.workunit.client.1.smithi088.stdout:0/21: chown c2 123633816 1 2024-03-24T00:48:36.595 INFO:tasks.workunit.client.1.smithi088.stdout:9/34: read d1/f8 [219265,83754] 0 2024-03-24T00:48:36.595 INFO:tasks.workunit.client.0.smithi038.stdout:6/55: fdatasync d1/f5 0 2024-03-24T00:48:36.596 INFO:tasks.workunit.client.0.smithi038.stdout:6/56: chown d1/fe 114972 1 2024-03-24T00:48:36.596 INFO:tasks.workunit.client.0.smithi038.stdout:6/57: dread - d1/f10 zero size 2024-03-24T00:48:36.601 INFO:tasks.workunit.client.0.smithi038.stdout:6/58: truncate d1/f6 2210833 0 2024-03-24T00:48:36.604 INFO:tasks.workunit.client.0.smithi038.stdout:8/31: rename d0/l6 to d0/ld 0 2024-03-24T00:48:36.605 INFO:tasks.workunit.client.1.smithi088.stdout:4/30: dwrite f1 [0,4194304] 0 2024-03-24T00:48:36.605 INFO:tasks.workunit.client.1.smithi088.stdout:4/31: chown c0 65852 1 2024-03-24T00:48:36.609 INFO:tasks.workunit.client.0.smithi038.stdout:8/32: dread - d0/f9 zero size 2024-03-24T00:48:36.618 INFO:tasks.workunit.client.0.smithi038.stdout:0/54: dwrite d0/ff [0,4194304] 0 2024-03-24T00:48:36.619 INFO:tasks.workunit.client.1.smithi088.stdout:8/29: dread f2 [0,4194304] 0 2024-03-24T00:48:36.620 INFO:tasks.workunit.client.1.smithi088.stdout:8/30: chown d9 60380 1 2024-03-24T00:48:36.626 INFO:tasks.workunit.client.1.smithi088.stdout:4/32: dread f2 [0,4194304] 0 2024-03-24T00:48:36.632 INFO:tasks.workunit.client.0.smithi038.stdout:9/60: link c11 d12/c17 0 2024-03-24T00:48:36.636 INFO:tasks.workunit.client.0.smithi038.stdout:8/33: mknod d0/ce 0 2024-03-24T00:48:36.639 INFO:tasks.workunit.client.1.smithi088.stdout:7/10: dwrite f1 [0,4194304] 0 2024-03-24T00:48:36.643 INFO:tasks.workunit.client.0.smithi038.stdout:5/25: getdents . 0 2024-03-24T00:48:36.643 INFO:tasks.workunit.client.0.smithi038.stdout:4/28: truncate f3 3877158 0 2024-03-24T00:48:36.643 INFO:tasks.workunit.client.0.smithi038.stdout:4/29: truncate f8 791107 0 2024-03-24T00:48:36.644 INFO:tasks.workunit.client.1.smithi088.stdout:6/35: dwrite f8 [0,4194304] 0 2024-03-24T00:48:36.651 INFO:tasks.workunit.client.1.smithi088.stdout:4/33: dread f3 [0,4194304] 0 2024-03-24T00:48:36.651 INFO:tasks.workunit.client.1.smithi088.stdout:4/34: fdatasync f3 0 2024-03-24T00:48:36.651 INFO:tasks.workunit.client.1.smithi088.stdout:4/35: fdatasync f1 0 2024-03-24T00:48:36.651 INFO:tasks.workunit.client.1.smithi088.stdout:4/36: creat f4 x:0 0 0 2024-03-24T00:48:36.652 INFO:tasks.workunit.client.0.smithi038.stdout:5/26: creat f5 x:0 0 0 2024-03-24T00:48:36.654 INFO:tasks.workunit.client.1.smithi088.stdout:5/31: truncate d1/f7 135626 0 2024-03-24T00:48:36.656 INFO:tasks.workunit.client.0.smithi038.stdout:2/29: fsync d4/f7 0 2024-03-24T00:48:36.658 INFO:tasks.workunit.client.0.smithi038.stdout:9/61: dread f4 [0,4194304] 0 2024-03-24T00:48:36.659 INFO:tasks.workunit.client.1.smithi088.stdout:2/22: dread f1 [0,4194304] 0 2024-03-24T00:48:36.661 INFO:tasks.workunit.client.1.smithi088.stdout:9/35: dwrite d1/f5 [0,4194304] 0 2024-03-24T00:48:36.661 INFO:tasks.workunit.client.1.smithi088.stdout:9/36: chown d1/f4 6693 1 2024-03-24T00:48:36.661 INFO:tasks.workunit.client.0.smithi038.stdout:2/30: rename d4/f8 to d4/fb 0 2024-03-24T00:48:36.662 INFO:tasks.workunit.client.0.smithi038.stdout:2/31: creat d4/fc x:0 0 0 2024-03-24T00:48:36.662 INFO:tasks.workunit.client.0.smithi038.stdout:2/32: dread - d4/fb zero size 2024-03-24T00:48:36.662 INFO:tasks.workunit.client.0.smithi038.stdout:2/33: dread - d4/fb zero size 2024-03-24T00:48:36.665 INFO:tasks.workunit.client.1.smithi088.stdout:6/36: mknod c9 0 2024-03-24T00:48:36.674 INFO:tasks.workunit.client.0.smithi038.stdout:2/34: link d4/fa d4/fd 0 2024-03-24T00:48:36.674 INFO:tasks.workunit.client.0.smithi038.stdout:2/35: chown f2 0 1 2024-03-24T00:48:36.675 INFO:tasks.workunit.client.1.smithi088.stdout:2/23: creat f5 x:0 0 0 2024-03-24T00:48:36.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:36 smithi088 ceph-mon[36610]: Upgrade: Updating mgr.smithi038.huhioz 2024-03-24T00:48:36.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:36 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:36.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:36 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.huhioz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:48:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:36 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.huhioz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:48:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:36 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:48:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:36 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:48:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:36 smithi088 ceph-mon[36610]: Deploying daemon mgr.smithi038.huhioz on smithi038 2024-03-24T00:48:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:36 smithi088 ceph-mon[36610]: pgmap v24: 65 pgs: 65 active+clean; 368 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 462 op/s 2024-03-24T00:48:36.681 INFO:tasks.workunit.client.1.smithi088.stdout:4/37: dread f2 [0,4194304] 0 2024-03-24T00:48:36.683 INFO:tasks.workunit.client.1.smithi088.stdout:9/37: getdents d1 0 2024-03-24T00:48:36.683 INFO:tasks.workunit.client.1.smithi088.stdout:9/38: chown d1/la 5 1 2024-03-24T00:48:36.683 INFO:tasks.workunit.client.1.smithi088.stdout:9/39: readlink d1/la 0 2024-03-24T00:48:36.683 INFO:tasks.workunit.client.1.smithi088.stdout:9/40: stat d1/la 0 2024-03-24T00:48:36.683 INFO:tasks.workunit.client.1.smithi088.stdout:9/41: getdents d1 0 2024-03-24T00:48:36.686 INFO:tasks.workunit.client.1.smithi088.stdout:6/37: unlink l2 0 2024-03-24T00:48:36.689 INFO:tasks.workunit.client.1.smithi088.stdout:7/11: dwrite f1 [0,4194304] 0 2024-03-24T00:48:36.692 INFO:tasks.workunit.client.1.smithi088.stdout:4/38: mkdir d5 0 2024-03-24T00:48:36.692 INFO:tasks.workunit.client.1.smithi088.stdout:4/39: dread - f4 zero size 2024-03-24T00:48:36.692 INFO:tasks.workunit.client.1.smithi088.stdout:9/42: mknod d1/cc 0 2024-03-24T00:48:36.698 INFO:tasks.workunit.client.1.smithi088.stdout:9/43: mknod d1/cd 0 2024-03-24T00:48:36.703 INFO:tasks.workunit.client.1.smithi088.stdout:4/40: truncate f1 4165763 0 2024-03-24T00:48:36.703 INFO:tasks.workunit.client.1.smithi088.stdout:9/44: write d1/f4 [4699403,8195] 0 2024-03-24T00:48:36.705 INFO:tasks.workunit.client.1.smithi088.stdout:8/31: truncate f1 3387850 0 2024-03-24T00:48:36.707 INFO:tasks.workunit.client.1.smithi088.stdout:9/45: symlink d1/le 0 2024-03-24T00:48:36.708 INFO:tasks.workunit.client.0.smithi038.stdout:6/59: dwrite d1/f10 [0,4194304] 0 2024-03-24T00:48:36.713 INFO:tasks.workunit.client.1.smithi088.stdout:6/38: dread f8 [0,4194304] 0 2024-03-24T00:48:36.713 INFO:tasks.workunit.client.1.smithi088.stdout:8/32: creat d9/fb x:0 0 0 2024-03-24T00:48:36.713 INFO:tasks.workunit.client.1.smithi088.stdout:9/46: chown d1/f7 0 1 2024-03-24T00:48:36.714 INFO:tasks.workunit.client.1.smithi088.stdout:7/12: write f1 [1783266,49593] 0 2024-03-24T00:48:36.716 INFO:tasks.workunit.client.0.smithi038.stdout:6/60: chown d1/f9 61246 1 2024-03-24T00:48:36.716 INFO:tasks.workunit.client.0.smithi038.stdout:6/61: truncate d1/fe 844822 0 2024-03-24T00:48:36.719 INFO:tasks.workunit.client.0.smithi038.stdout:4/30: unlink f3 0 2024-03-24T00:48:36.719 INFO:tasks.workunit.client.0.smithi038.stdout:4/31: chown l7 15303 1 2024-03-24T00:48:36.719 INFO:tasks.workunit.client.0.smithi038.stdout:4/32: write f8 [1294319,45704] 0 2024-03-24T00:48:36.720 INFO:tasks.workunit.client.1.smithi088.stdout:9/47: truncate d1/f8 875910 0 2024-03-24T00:48:36.722 INFO:tasks.workunit.client.0.smithi038.stdout:6/62: rmdir d1/da 39 2024-03-24T00:48:36.722 INFO:tasks.workunit.client.0.smithi038.stdout:6/63: chown d1/da 30021564 1 2024-03-24T00:48:36.722 INFO:tasks.workunit.client.0.smithi038.stdout:6/64: fsync d1/f6 0 2024-03-24T00:48:36.722 INFO:tasks.workunit.client.0.smithi038.stdout:6/65: stat d1/da/lc 0 2024-03-24T00:48:36.723 INFO:tasks.workunit.client.1.smithi088.stdout:6/39: write f8 [3426668,7873] 0 2024-03-24T00:48:36.723 INFO:tasks.workunit.client.1.smithi088.stdout:6/40: chown l7 219726 1 2024-03-24T00:48:36.724 INFO:tasks.workunit.client.1.smithi088.stdout:7/13: mkdir d3 0 2024-03-24T00:48:36.725 INFO:tasks.workunit.client.0.smithi038.stdout:0/55: dwrite d0/fa [0,4194304] 0 2024-03-24T00:48:36.725 INFO:tasks.workunit.client.0.smithi038.stdout:5/27: dread f3 [0,4194304] 0 2024-03-24T00:48:36.727 INFO:tasks.workunit.client.0.smithi038.stdout:2/36: fsync d4/fb 0 2024-03-24T00:48:36.727 INFO:tasks.workunit.client.0.smithi038.stdout:2/37: fsync d4/f7 0 2024-03-24T00:48:36.730 INFO:tasks.workunit.client.0.smithi038.stdout:0/56: unlink d0/c11 0 2024-03-24T00:48:36.730 INFO:tasks.workunit.client.0.smithi038.stdout:0/57: creat d0/d12/d15/f17 x:0 0 0 2024-03-24T00:48:36.731 INFO:tasks.workunit.client.1.smithi088.stdout:6/41: mknod ca 0 2024-03-24T00:48:36.734 INFO:tasks.workunit.client.1.smithi088.stdout:2/24: fsync f5 0 2024-03-24T00:48:36.742 INFO:tasks.workunit.client.0.smithi038.stdout:2/38: rename d4/l9 to d4/le 0 2024-03-24T00:48:36.742 INFO:tasks.workunit.client.0.smithi038.stdout:6/66: dread d1/f3 [0,4194304] 0 2024-03-24T00:48:36.742 INFO:tasks.workunit.client.0.smithi038.stdout:6/67: chown d1/da/fb 1742515260 1 2024-03-24T00:48:36.742 INFO:tasks.workunit.client.0.smithi038.stdout:6/68: stat d1/da/lc 0 2024-03-24T00:48:36.743 INFO:tasks.workunit.client.0.smithi038.stdout:5/28: truncate f3 475140 0 2024-03-24T00:48:36.743 INFO:tasks.workunit.client.0.smithi038.stdout:9/62: dread f6 [0,4194304] 0 2024-03-24T00:48:36.743 INFO:tasks.workunit.client.0.smithi038.stdout:9/63: dread - fc zero size 2024-03-24T00:48:36.743 INFO:tasks.workunit.client.0.smithi038.stdout:4/33: dread f8 [0,4194304] 0 2024-03-24T00:48:36.749 INFO:tasks.workunit.client.1.smithi088.stdout:9/48: link d1/f4 d1/ff 0 2024-03-24T00:48:36.754 INFO:tasks.workunit.client.0.smithi038.stdout:0/58: link d0/cd d0/d12/d15/c18 0 2024-03-24T00:48:36.754 INFO:tasks.workunit.client.1.smithi088.stdout:4/41: dwrite f3 [0,4194304] 0 2024-03-24T00:48:36.756 INFO:tasks.workunit.client.0.smithi038.stdout:4/34: unlink c6 0 2024-03-24T00:48:36.756 INFO:tasks.workunit.client.1.smithi088.stdout:6/42: dread f8 [0,4194304] 0 2024-03-24T00:48:36.756 INFO:tasks.workunit.client.1.smithi088.stdout:9/49: chown d1/c9 10 1 2024-03-24T00:48:36.759 INFO:tasks.workunit.client.0.smithi038.stdout:8/34: dwrite d0/f9 [0,4194304] 0 2024-03-24T00:48:36.769 INFO:tasks.workunit.client.1.smithi088.stdout:1/23: sync 2024-03-24T00:48:36.769 INFO:tasks.workunit.client.1.smithi088.stdout:0/22: sync 2024-03-24T00:48:36.769 INFO:tasks.workunit.client.1.smithi088.stdout:3/29: sync 2024-03-24T00:48:36.769 INFO:tasks.workunit.client.1.smithi088.stdout:3/30: creat d3/f9 x:0 0 0 2024-03-24T00:48:36.769 INFO:tasks.workunit.client.1.smithi088.stdout:3/31: fdatasync f1 0 2024-03-24T00:48:36.773 INFO:tasks.workunit.client.1.smithi088.stdout:8/33: rmdir d9 39 2024-03-24T00:48:36.780 INFO:tasks.workunit.client.0.smithi038.stdout:0/59: write d0/f7 [1077024,2750] 0 2024-03-24T00:48:36.782 INFO:tasks.workunit.client.1.smithi088.stdout:4/42: write f2 [3502144,102859] 0 2024-03-24T00:48:36.789 INFO:tasks.workunit.client.1.smithi088.stdout:1/24: getdents . 0 2024-03-24T00:48:36.790 INFO:tasks.workunit.client.0.smithi038.stdout:8/35: mknod d0/cf 0 2024-03-24T00:48:36.791 INFO:tasks.workunit.client.1.smithi088.stdout:9/50: fdatasync d1/f6 0 2024-03-24T00:48:36.793 INFO:tasks.workunit.client.1.smithi088.stdout:8/34: getdents d9 0 2024-03-24T00:48:36.797 INFO:tasks.workunit.client.1.smithi088.stdout:4/43: creat d5/f6 x:0 0 0 2024-03-24T00:48:36.799 INFO:tasks.workunit.client.0.smithi038.stdout:5/29: dread f3 [0,4194304] 0 2024-03-24T00:48:36.801 INFO:tasks.workunit.client.0.smithi038.stdout:0/60: mknod d0/d12/c19 0 2024-03-24T00:48:36.802 INFO:tasks.workunit.client.1.smithi088.stdout:7/14: truncate f1 2396358 0 2024-03-24T00:48:36.802 INFO:tasks.workunit.client.1.smithi088.stdout:7/15: chown f1 443 1 2024-03-24T00:48:36.802 INFO:tasks.workunit.client.1.smithi088.stdout:7/16: chown f1 1189612 1 2024-03-24T00:48:36.807 INFO:tasks.workunit.client.0.smithi038.stdout:8/36: write d0/fb [825077,54881] 0 2024-03-24T00:48:36.809 INFO:tasks.workunit.client.1.smithi088.stdout:6/43: truncate f8 3710491 0 2024-03-24T00:48:36.811 INFO:tasks.workunit.client.0.smithi038.stdout:5/30: mkdir d6 0 2024-03-24T00:48:36.811 INFO:tasks.workunit.client.0.smithi038.stdout:5/31: rename d6 to d6/d7 22 2024-03-24T00:48:36.811 INFO:tasks.workunit.client.0.smithi038.stdout:5/32: fsync f3 0 2024-03-24T00:48:36.812 INFO:tasks.workunit.client.1.smithi088.stdout:1/25: symlink l5 0 2024-03-24T00:48:36.813 INFO:tasks.workunit.client.1.smithi088.stdout:2/25: dwrite f1 [0,4194304] 0 2024-03-24T00:48:36.814 INFO:tasks.workunit.client.0.smithi038.stdout:5/33: rename c4 to d6/c8 0 2024-03-24T00:48:36.816 INFO:tasks.workunit.client.1.smithi088.stdout:8/35: mkdir d9/dc 0 2024-03-24T00:48:36.818 INFO:tasks.workunit.client.0.smithi038.stdout:8/37: mkdir d0/d10 0 2024-03-24T00:48:36.818 INFO:tasks.workunit.client.0.smithi038.stdout:8/38: write d0/fb [1328830,60347] 0 2024-03-24T00:48:36.818 INFO:tasks.workunit.client.0.smithi038.stdout:6/69: dwrite d1/da/fb [0,4194304] 0 2024-03-24T00:48:36.818 INFO:tasks.workunit.client.0.smithi038.stdout:6/70: fdatasync d1/f8 0 2024-03-24T00:48:36.819 INFO:tasks.workunit.client.1.smithi088.stdout:7/17: creat d3/f4 x:0 0 0 2024-03-24T00:48:36.819 INFO:tasks.workunit.client.1.smithi088.stdout:7/18: creat d3/f5 x:0 0 0 2024-03-24T00:48:36.821 INFO:tasks.workunit.client.0.smithi038.stdout:4/35: dwrite f8 [0,4194304] 0 2024-03-24T00:48:36.821 INFO:tasks.workunit.client.0.smithi038.stdout:4/36: creat f9 x:0 0 0 2024-03-24T00:48:36.821 INFO:tasks.workunit.client.0.smithi038.stdout:9/64: dwrite f4 [0,4194304] 0 2024-03-24T00:48:36.821 INFO:tasks.workunit.client.0.smithi038.stdout:9/65: creat d12/f18 x:0 0 0 2024-03-24T00:48:36.821 INFO:tasks.workunit.client.0.smithi038.stdout:9/66: dread - d12/f18 zero size 2024-03-24T00:48:36.828 INFO:tasks.workunit.client.1.smithi088.stdout:0/23: read f3 [3799312,56092] 0 2024-03-24T00:48:36.828 INFO:tasks.workunit.client.1.smithi088.stdout:0/24: write f3 [5235129,88133] 0 2024-03-24T00:48:36.830 INFO:tasks.workunit.client.1.smithi088.stdout:0/25: write f3 [5224551,42454] 0 2024-03-24T00:48:36.830 INFO:tasks.workunit.client.0.smithi038.stdout:8/39: symlink d0/d10/l11 0 2024-03-24T00:48:36.832 INFO:tasks.workunit.client.1.smithi088.stdout:2/26: link f5 f6 0 2024-03-24T00:48:36.832 INFO:tasks.workunit.client.1.smithi088.stdout:3/32: rmdir d3 39 2024-03-24T00:48:36.832 INFO:tasks.workunit.client.1.smithi088.stdout:8/36: mknod d9/dc/cd 0 2024-03-24T00:48:36.834 INFO:tasks.workunit.client.0.smithi038.stdout:0/61: dread d0/f7 [0,4194304] 0 2024-03-24T00:48:36.835 INFO:tasks.workunit.client.1.smithi088.stdout:9/51: dread d1/f5 [0,4194304] 0 2024-03-24T00:48:36.835 INFO:tasks.workunit.client.1.smithi088.stdout:1/26: mkdir d6 0 2024-03-24T00:48:36.837 INFO:tasks.workunit.client.0.smithi038.stdout:4/37: mkdir da 0 2024-03-24T00:48:36.838 INFO:tasks.workunit.client.0.smithi038.stdout:4/38: readlink l7 0 2024-03-24T00:48:36.841 INFO:tasks.workunit.client.1.smithi088.stdout:5/32: sync 2024-03-24T00:48:36.841 INFO:tasks.workunit.client.1.smithi088.stdout:5/33: readlink d1/l6 0 2024-03-24T00:48:36.844 INFO:tasks.workunit.client.1.smithi088.stdout:2/27: unlink l4 0 2024-03-24T00:48:36.846 INFO:tasks.workunit.client.0.smithi038.stdout:8/40: creat d0/f12 x:0 0 0 2024-03-24T00:48:36.847 INFO:tasks.workunit.client.0.smithi038.stdout:6/71: dread d1/f8 [0,4194304] 0 2024-03-24T00:48:36.849 INFO:tasks.workunit.client.1.smithi088.stdout:3/33: rename d3/f4 to d3/fa 0 2024-03-24T00:48:36.849 INFO:tasks.workunit.client.1.smithi088.stdout:3/34: dread - d3/f9 zero size 2024-03-24T00:48:36.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/39: rename l7 to da/lb 0 2024-03-24T00:48:36.855 INFO:tasks.workunit.client.1.smithi088.stdout:9/52: truncate d1/f4 670836 0 2024-03-24T00:48:36.856 INFO:tasks.workunit.client.0.smithi038.stdout:2/39: dwrite d4/f7 [0,4194304] 0 2024-03-24T00:48:36.857 INFO:tasks.workunit.client.1.smithi088.stdout:4/44: dread f2 [0,4194304] 0 2024-03-24T00:48:36.857 INFO:tasks.workunit.client.0.smithi038.stdout:8/41: stat d0/c7 0 2024-03-24T00:48:36.860 INFO:tasks.workunit.client.1.smithi088.stdout:8/37: write f2 [2149826,47487] 0 2024-03-24T00:48:36.865 INFO:tasks.workunit.client.1.smithi088.stdout:5/34: symlink d1/l8 0 2024-03-24T00:48:36.865 INFO:tasks.workunit.client.0.smithi038.stdout:5/34: dwrite f2 [0,4194304] 0 2024-03-24T00:48:36.865 INFO:tasks.workunit.client.0.smithi038.stdout:5/35: fdatasync f3 0 2024-03-24T00:48:36.868 INFO:tasks.workunit.client.0.smithi038.stdout:2/40: link d4/fb d4/ff 0 2024-03-24T00:48:36.869 INFO:tasks.workunit.client.0.smithi038.stdout:2/41: creat d4/f10 x:0 0 0 2024-03-24T00:48:36.869 INFO:tasks.workunit.client.0.smithi038.stdout:2/42: write d4/f10 [699052,30225] 0 2024-03-24T00:48:36.869 INFO:tasks.workunit.client.0.smithi038.stdout:2/43: read - d4/fd zero size 2024-03-24T00:48:36.873 INFO:tasks.workunit.client.0.smithi038.stdout:6/72: rename d1/fe to d1/f12 0 2024-03-24T00:48:36.874 INFO:tasks.workunit.client.1.smithi088.stdout:4/45: mkdir d5/d7 0 2024-03-24T00:48:36.878 INFO:tasks.workunit.client.1.smithi088.stdout:0/26: dread f3 [0,4194304] 0 2024-03-24T00:48:36.878 INFO:tasks.workunit.client.1.smithi088.stdout:7/19: fsync d3/f5 0 2024-03-24T00:48:36.879 INFO:tasks.workunit.client.1.smithi088.stdout:7/20: write d3/f4 [568870,73383] 0 2024-03-24T00:48:36.879 INFO:tasks.workunit.client.0.smithi038.stdout:4/40: write f8 [3390127,2263] 0 2024-03-24T00:48:36.879 INFO:tasks.workunit.client.0.smithi038.stdout:4/41: chown c4 254 1 2024-03-24T00:48:36.880 INFO:tasks.workunit.client.1.smithi088.stdout:8/38: unlink f7 0 2024-03-24T00:48:36.882 INFO:tasks.workunit.client.0.smithi038.stdout:2/44: unlink d4/le 0 2024-03-24T00:48:36.886 INFO:tasks.workunit.client.1.smithi088.stdout:1/27: dread f0 [0,4194304] 0 2024-03-24T00:48:36.887 INFO:tasks.workunit.client.1.smithi088.stdout:5/35: symlink d1/l9 0 2024-03-24T00:48:36.887 INFO:tasks.workunit.client.1.smithi088.stdout:5/36: creat d1/fa x:0 0 0 2024-03-24T00:48:36.890 INFO:tasks.workunit.client.1.smithi088.stdout:9/53: mkdir d1/d10 0 2024-03-24T00:48:36.890 INFO:tasks.workunit.client.0.smithi038.stdout:8/42: fsync d0/f12 0 2024-03-24T00:48:36.891 INFO:tasks.workunit.client.0.smithi038.stdout:4/42: mkdir da/dc 0 2024-03-24T00:48:36.891 INFO:tasks.workunit.client.0.smithi038.stdout:4/43: dread - f9 zero size 2024-03-24T00:48:36.894 INFO:tasks.workunit.client.1.smithi088.stdout:4/46: mknod d5/d7/c8 0 2024-03-24T00:48:36.894 INFO:tasks.workunit.client.1.smithi088.stdout:4/47: stat c0 0 2024-03-24T00:48:36.894 INFO:tasks.workunit.client.0.smithi038.stdout:0/62: dwrite d0/fa [0,4194304] 0 2024-03-24T00:48:36.894 INFO:tasks.workunit.client.0.smithi038.stdout:0/63: write d0/fa [4606723,9833] 0 2024-03-24T00:48:36.896 INFO:tasks.workunit.client.1.smithi088.stdout:8/39: dread f1 [0,4194304] 0 2024-03-24T00:48:36.897 INFO:tasks.workunit.client.0.smithi038.stdout:2/45: mknod d4/c11 0 2024-03-24T00:48:36.898 INFO:tasks.workunit.client.0.smithi038.stdout:4/44: fdatasync f8 0 2024-03-24T00:48:36.898 INFO:tasks.workunit.client.0.smithi038.stdout:4/45: chown f9 60 1 2024-03-24T00:48:36.900 INFO:tasks.workunit.client.1.smithi088.stdout:0/27: mknod c6 0 2024-03-24T00:48:36.900 INFO:tasks.workunit.client.1.smithi088.stdout:0/28: readlink - no filename 2024-03-24T00:48:36.902 INFO:tasks.workunit.client.1.smithi088.stdout:7/21: dread d3/f4 [0,4194304] 0 2024-03-24T00:48:36.902 INFO:tasks.workunit.client.1.smithi088.stdout:7/22: chown d3/f4 220133658 1 2024-03-24T00:48:36.902 INFO:tasks.workunit.client.1.smithi088.stdout:7/23: chown c2 2 1 2024-03-24T00:48:36.902 INFO:tasks.workunit.client.1.smithi088.stdout:7/24: chown d3 398376908 1 2024-03-24T00:48:36.902 INFO:tasks.workunit.client.1.smithi088.stdout:7/25: write d3/f5 [694346,51051] 0 2024-03-24T00:48:36.903 INFO:tasks.workunit.client.0.smithi038.stdout:8/43: link d0/cc d0/d10/c13 0 2024-03-24T00:48:36.914 INFO:tasks.workunit.client.0.smithi038.stdout:4/46: mkdir da/dc/dd 0 2024-03-24T00:48:36.914 INFO:tasks.workunit.client.0.smithi038.stdout:4/47: rename da to da/dc/de 22 2024-03-24T00:48:36.916 INFO:tasks.workunit.client.1.smithi088.stdout:2/28: truncate f1 1642460 0 2024-03-24T00:48:36.916 INFO:tasks.workunit.client.1.smithi088.stdout:2/29: stat f1 0 2024-03-24T00:48:36.917 INFO:tasks.workunit.client.1.smithi088.stdout:9/54: dread d1/ff [0,4194304] 0 2024-03-24T00:48:36.922 INFO:tasks.workunit.client.0.smithi038.stdout:5/36: dwrite f5 [0,4194304] 0 2024-03-24T00:48:36.927 INFO:tasks.workunit.client.0.smithi038.stdout:8/44: link d0/cc d0/c14 0 2024-03-24T00:48:36.927 INFO:tasks.workunit.client.0.smithi038.stdout:8/45: fsync d0/f12 0 2024-03-24T00:48:36.928 INFO:tasks.workunit.client.0.smithi038.stdout:5/37: creat d6/f9 x:0 0 0 2024-03-24T00:48:36.929 INFO:tasks.workunit.client.1.smithi088.stdout:1/28: unlink c3 0 2024-03-24T00:48:36.929 INFO:tasks.workunit.client.1.smithi088.stdout:1/29: fdatasync f0 0 2024-03-24T00:48:36.930 INFO:tasks.workunit.client.1.smithi088.stdout:0/29: truncate f3 3299775 0 2024-03-24T00:48:36.932 INFO:tasks.workunit.client.0.smithi038.stdout:5/38: rename f2 to d6/fa 0 2024-03-24T00:48:36.936 INFO:tasks.workunit.client.1.smithi088.stdout:0/30: rename c0 to c7 0 2024-03-24T00:48:36.941 INFO:tasks.workunit.client.1.smithi088.stdout:4/48: getdents d5/d7 0 2024-03-24T00:48:36.941 INFO:tasks.workunit.client.1.smithi088.stdout:4/49: creat d5/f9 x:0 0 0 2024-03-24T00:48:36.942 INFO:tasks.workunit.client.0.smithi038.stdout:4/48: dread f8 [0,4194304] 0 2024-03-24T00:48:36.944 INFO:tasks.workunit.client.1.smithi088.stdout:4/50: mknod d5/d7/ca 0 2024-03-24T00:48:36.945 INFO:tasks.workunit.client.1.smithi088.stdout:4/51: chown d5/d7/c8 3903852 1 2024-03-24T00:48:36.945 INFO:tasks.workunit.client.1.smithi088.stdout:4/52: chown f4 1015314 1 2024-03-24T00:48:36.946 INFO:tasks.workunit.client.0.smithi038.stdout:4/49: mknod da/cf 0 2024-03-24T00:48:36.947 INFO:tasks.workunit.client.1.smithi088.stdout:4/53: truncate f2 302262 0 2024-03-24T00:48:36.948 INFO:tasks.workunit.client.1.smithi088.stdout:4/54: dread - d5/f9 zero size 2024-03-24T00:48:36.958 INFO:tasks.workunit.client.0.smithi038.stdout:8/46: rmdir d0 39 2024-03-24T00:48:36.958 INFO:tasks.workunit.client.0.smithi038.stdout:5/39: write f5 [4224234,42523] 0 2024-03-24T00:48:36.960 INFO:tasks.workunit.client.0.smithi038.stdout:5/40: rename d6/c8 to d6/cb 0 2024-03-24T00:48:36.964 INFO:tasks.workunit.client.0.smithi038.stdout:8/47: mkdir d0/d15 0 2024-03-24T00:48:36.964 INFO:tasks.workunit.client.0.smithi038.stdout:8/48: fdatasync d0/f9 0 2024-03-24T00:48:36.964 INFO:tasks.workunit.client.0.smithi038.stdout:8/49: chown d0 5 1 2024-03-24T00:48:36.971 INFO:tasks.workunit.client.1.smithi088.stdout:9/55: dread d1/f8 [0,4194304] 0 2024-03-24T00:48:36.976 INFO:tasks.workunit.client.1.smithi088.stdout:3/35: dwrite d3/fa [0,4194304] 0 2024-03-24T00:48:36.980 INFO:tasks.workunit.client.1.smithi088.stdout:9/56: dread d1/f8 [0,4194304] 0 2024-03-24T00:48:36.982 INFO:tasks.workunit.client.0.smithi038.stdout:2/46: dwrite f2 [0,4194304] 0 2024-03-24T00:48:36.982 INFO:tasks.workunit.client.0.smithi038.stdout:2/47: creat d4/f12 x:0 0 0 2024-03-24T00:48:36.982 INFO:tasks.workunit.client.0.smithi038.stdout:2/48: stat d4/f12 0 2024-03-24T00:48:36.982 INFO:tasks.workunit.client.0.smithi038.stdout:2/49: fdatasync d4/ff 0 2024-03-24T00:48:36.982 INFO:tasks.workunit.client.0.smithi038.stdout:2/50: creat d4/f13 x:0 0 0 2024-03-24T00:48:36.982 INFO:tasks.workunit.client.0.smithi038.stdout:2/51: readlink - no filename 2024-03-24T00:48:36.982 INFO:tasks.workunit.client.0.smithi038.stdout:2/52: creat d4/f14 x:0 0 0 2024-03-24T00:48:36.982 INFO:tasks.workunit.client.0.smithi038.stdout:2/53: readlink - no filename 2024-03-24T00:48:36.983 INFO:tasks.workunit.client.0.smithi038.stdout:2/54: creat d4/f15 x:0 0 0 2024-03-24T00:48:36.983 INFO:tasks.workunit.client.0.smithi038.stdout:2/55: creat d4/f16 x:0 0 0 2024-03-24T00:48:36.987 INFO:tasks.workunit.client.0.smithi038.stdout:0/64: dwrite d0/d12/d15/f16 [0,4194304] 0 2024-03-24T00:48:36.987 INFO:tasks.workunit.client.1.smithi088.stdout:9/57: dread d1/f8 [0,4194304] 0 2024-03-24T00:48:36.989 INFO:tasks.workunit.client.0.smithi038.stdout:2/56: symlink d4/l17 0 2024-03-24T00:48:36.990 INFO:tasks.workunit.client.1.smithi088.stdout:1/30: dread f2 [0,4194304] 0 2024-03-24T00:48:36.990 INFO:tasks.workunit.client.1.smithi088.stdout:1/31: chown l5 602700858 1 2024-03-24T00:48:36.990 INFO:tasks.workunit.client.1.smithi088.stdout:1/32: getdents d6 0 2024-03-24T00:48:36.994 INFO:tasks.workunit.client.1.smithi088.stdout:2/30: dwrite f6 [0,4194304] 0 2024-03-24T00:48:36.994 INFO:tasks.workunit.client.1.smithi088.stdout:2/31: readlink l2 0 2024-03-24T00:48:36.994 INFO:tasks.workunit.client.1.smithi088.stdout:2/32: truncate f6 5239609 0 2024-03-24T00:48:36.996 INFO:tasks.workunit.client.1.smithi088.stdout:5/37: dwrite d1/f7 [0,4194304] 0 2024-03-24T00:48:36.996 INFO:tasks.workunit.client.1.smithi088.stdout:5/38: chown d1/l9 2040752657 1 2024-03-24T00:48:36.996 INFO:tasks.workunit.client.1.smithi088.stdout:5/39: creat d1/fb x:0 0 0 2024-03-24T00:48:37.002 INFO:tasks.workunit.client.0.smithi038.stdout:2/57: dread d4/f10 [0,4194304] 0 2024-03-24T00:48:37.004 INFO:tasks.workunit.client.1.smithi088.stdout:4/55: truncate f2 789573 0 2024-03-24T00:48:37.005 INFO:tasks.workunit.client.1.smithi088.stdout:5/40: symlink d1/lc 0 2024-03-24T00:48:37.006 INFO:tasks.workunit.client.1.smithi088.stdout:7/26: dwrite f1 [0,4194304] 0 2024-03-24T00:48:37.007 INFO:tasks.workunit.client.0.smithi038.stdout:2/58: mkdir d4/d18 0 2024-03-24T00:48:37.008 INFO:tasks.workunit.client.0.smithi038.stdout:2/59: creat d4/f19 x:0 0 0 2024-03-24T00:48:37.009 INFO:tasks.workunit.client.1.smithi088.stdout:8/40: dwrite d9/fb [0,4194304] 0 2024-03-24T00:48:37.010 INFO:tasks.workunit.client.0.smithi038.stdout:4/50: dwrite f8 [4194304,4194304] 0 2024-03-24T00:48:37.010 INFO:tasks.workunit.client.0.smithi038.stdout:4/51: getdents da/dc/dd 0 2024-03-24T00:48:37.010 INFO:tasks.workunit.client.0.smithi038.stdout:4/52: chown c1 2501 1 2024-03-24T00:48:37.010 INFO:tasks.workunit.client.0.smithi038.stdout:4/53: chown l5 8113407 1 2024-03-24T00:48:37.011 INFO:tasks.workunit.client.1.smithi088.stdout:7/27: fsync d3/f5 0 2024-03-24T00:48:37.011 INFO:tasks.workunit.client.1.smithi088.stdout:4/56: mknod d5/d7/cb 0 2024-03-24T00:48:37.011 INFO:tasks.workunit.client.1.smithi088.stdout:7/28: chown d3/f5 0 1 2024-03-24T00:48:37.011 INFO:tasks.workunit.client.1.smithi088.stdout:4/57: stat d5/f9 0 2024-03-24T00:48:37.011 INFO:tasks.workunit.client.1.smithi088.stdout:4/58: dread - f4 zero size 2024-03-24T00:48:37.011 INFO:tasks.workunit.client.1.smithi088.stdout:4/59: chown c0 75631 1 2024-03-24T00:48:37.013 INFO:tasks.workunit.client.1.smithi088.stdout:5/41: mkdir d1/dd 0 2024-03-24T00:48:37.013 INFO:tasks.workunit.client.0.smithi038.stdout:5/41: truncate f5 3152451 0 2024-03-24T00:48:37.014 INFO:tasks.workunit.client.1.smithi088.stdout:5/42: write d1/fa [30911,70058] 0 2024-03-24T00:48:37.014 INFO:tasks.workunit.client.1.smithi088.stdout:5/43: creat d1/fe x:0 0 0 2024-03-24T00:48:37.014 INFO:tasks.workunit.client.1.smithi088.stdout:5/44: stat d1/c2 0 2024-03-24T00:48:37.020 INFO:tasks.workunit.client.1.smithi088.stdout:3/36: write d3/fa [4579932,50573] 0 2024-03-24T00:48:37.020 INFO:tasks.workunit.client.1.smithi088.stdout:3/37: dread - f1 zero size 2024-03-24T00:48:37.020 INFO:tasks.workunit.client.1.smithi088.stdout:3/38: write d3/f7 [903201,106126] 0 2024-03-24T00:48:37.020 INFO:tasks.workunit.client.1.smithi088.stdout:1/33: dread f1 [0,4194304] 0 2024-03-24T00:48:37.020 INFO:tasks.workunit.client.1.smithi088.stdout:0/31: dread f3 [0,4194304] 0 2024-03-24T00:48:37.020 INFO:tasks.workunit.client.1.smithi088.stdout:0/32: write f3 [2224539,85547] 0 2024-03-24T00:48:37.021 INFO:tasks.workunit.client.0.smithi038.stdout:2/60: symlink d4/d18/l1a 0 2024-03-24T00:48:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:4/54: creat da/dc/dd/f10 x:0 0 0 2024-03-24T00:48:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:4/55: chown f8 1979274865 1 2024-03-24T00:48:37.023 INFO:tasks.workunit.client.1.smithi088.stdout:8/41: rename d9/dc/cd to d9/dc/ce 0 2024-03-24T00:48:37.027 INFO:tasks.workunit.client.1.smithi088.stdout:2/33: dread f5 [0,4194304] 0 2024-03-24T00:48:37.028 INFO:tasks.workunit.client.0.smithi038.stdout:5/42: unlink d6/cb 0 2024-03-24T00:48:37.030 INFO:tasks.workunit.client.0.smithi038.stdout:2/61: dread d4/f10 [0,4194304] 0 2024-03-24T00:48:37.031 INFO:tasks.workunit.client.0.smithi038.stdout:4/56: mknod da/dc/dd/c11 0 2024-03-24T00:48:37.031 INFO:tasks.workunit.client.0.smithi038.stdout:2/62: write d4/f19 [74481,63806] 0 2024-03-24T00:48:37.033 INFO:tasks.workunit.client.1.smithi088.stdout:2/34: dread f6 [4194304,4194304] 0 2024-03-24T00:48:37.033 INFO:tasks.workunit.client.0.smithi038.stdout:2/63: write d4/fc [935809,47151] 0 2024-03-24T00:48:37.033 INFO:tasks.workunit.client.0.smithi038.stdout:2/64: dread - d4/fb zero size 2024-03-24T00:48:37.033 INFO:tasks.workunit.client.0.smithi038.stdout:2/65: creat d4/f1b x:0 0 0 2024-03-24T00:48:37.033 INFO:tasks.workunit.client.0.smithi038.stdout:2/66: dread - d4/fb zero size 2024-03-24T00:48:37.034 INFO:tasks.workunit.client.0.smithi038.stdout:5/43: mknod d6/cc 0 2024-03-24T00:48:37.034 INFO:tasks.workunit.client.1.smithi088.stdout:7/29: mknod d3/c6 0 2024-03-24T00:48:37.034 INFO:tasks.workunit.client.1.smithi088.stdout:4/60: rmdir d5/d7 39 2024-03-24T00:48:37.035 INFO:tasks.workunit.client.1.smithi088.stdout:4/61: chown c0 17866 1 2024-03-24T00:48:37.035 INFO:tasks.workunit.client.1.smithi088.stdout:7/30: creat d3/f7 x:0 0 0 2024-03-24T00:48:37.039 INFO:tasks.workunit.client.1.smithi088.stdout:3/39: write f2 [3590342,129650] 0 2024-03-24T00:48:37.040 INFO:tasks.workunit.client.1.smithi088.stdout:9/58: dwrite d1/f7 [0,4194304] 0 2024-03-24T00:48:37.042 INFO:tasks.workunit.client.0.smithi038.stdout:4/57: mkdir da/d12 0 2024-03-24T00:48:37.042 INFO:tasks.workunit.client.0.smithi038.stdout:4/58: readlink da/lb 0 2024-03-24T00:48:37.042 INFO:tasks.workunit.client.0.smithi038.stdout:6/73: dwrite d1/f9 [0,4194304] 0 2024-03-24T00:48:37.048 INFO:tasks.workunit.client.0.smithi038.stdout:5/44: link f3 d6/fd 0 2024-03-24T00:48:37.057 INFO:tasks.workunit.client.0.smithi038.stdout:6/74: creat d1/f13 x:0 0 0 2024-03-24T00:48:37.057 INFO:tasks.workunit.client.0.smithi038.stdout:6/75: chown d1/f10 560445 1 2024-03-24T00:48:37.059 INFO:tasks.workunit.client.1.smithi088.stdout:0/33: mknod c8 0 2024-03-24T00:48:37.061 INFO:tasks.workunit.client.0.smithi038.stdout:5/45: mknod d6/ce 0 2024-03-24T00:48:37.066 INFO:tasks.workunit.client.1.smithi088.stdout:8/42: symlink d9/dc/lf 0 2024-03-24T00:48:37.070 INFO:tasks.workunit.client.1.smithi088.stdout:1/34: dwrite f0 [0,4194304] 0 2024-03-24T00:48:37.074 INFO:tasks.workunit.client.0.smithi038.stdout:5/46: unlink f3 0 2024-03-24T00:48:37.081 INFO:tasks.workunit.client.0.smithi038.stdout:0/65: dwrite d0/f7 [0,4194304] 0 2024-03-24T00:48:37.082 INFO:tasks.workunit.client.1.smithi088.stdout:3/40: rename d3/f7 to d3/d6/fb 0 2024-03-24T00:48:37.082 INFO:tasks.workunit.client.1.smithi088.stdout:3/41: chown d3/d6 3604 1 2024-03-24T00:48:37.082 INFO:tasks.workunit.client.1.smithi088.stdout:3/42: chown d3/f9 68771 1 2024-03-24T00:48:37.084 INFO:tasks.workunit.client.1.smithi088.stdout:5/45: rename d1/f7 to d1/ff 0 2024-03-24T00:48:37.084 INFO:tasks.workunit.client.1.smithi088.stdout:5/46: write d1/fb [877302,43845] 0 2024-03-24T00:48:37.085 INFO:tasks.workunit.client.0.smithi038.stdout:2/67: dwrite d4/f1b [0,4194304] 0 2024-03-24T00:48:37.086 INFO:tasks.workunit.client.0.smithi038.stdout:2/68: stat d4/fc 0 2024-03-24T00:48:37.087 INFO:tasks.workunit.client.1.smithi088.stdout:0/34: mkdir d9 0 2024-03-24T00:48:37.088 INFO:tasks.workunit.client.0.smithi038.stdout:5/47: mkdir d6/df 0 2024-03-24T00:48:37.092 INFO:tasks.workunit.client.1.smithi088.stdout:1/35: write f0 [2258509,77437] 0 2024-03-24T00:48:37.095 INFO:tasks.workunit.client.0.smithi038.stdout:6/76: rmdir d1 39 2024-03-24T00:48:37.095 INFO:tasks.workunit.client.1.smithi088.stdout:2/35: dwrite f6 [4194304,4194304] 0 2024-03-24T00:48:37.096 INFO:tasks.workunit.client.1.smithi088.stdout:2/36: rmdir - no directory 2024-03-24T00:48:37.098 INFO:tasks.workunit.client.1.smithi088.stdout:4/62: getdents d5 0 2024-03-24T00:48:37.099 INFO:tasks.workunit.client.1.smithi088.stdout:8/43: getdents d9/dc 0 2024-03-24T00:48:37.099 INFO:tasks.workunit.client.1.smithi088.stdout:8/44: write d9/fb [4520239,24539] 0 2024-03-24T00:48:37.100 INFO:tasks.workunit.client.0.smithi038.stdout:6/77: rename d1/f3 to d1/da/f14 0 2024-03-24T00:48:37.102 INFO:tasks.workunit.client.1.smithi088.stdout:7/31: fsync d3/f7 0 2024-03-24T00:48:37.103 INFO:tasks.workunit.client.0.smithi038.stdout:6/78: stat d1 0 2024-03-24T00:48:37.103 INFO:tasks.workunit.client.0.smithi038.stdout:2/69: read d4/f7 [961367,44039] 0 2024-03-24T00:48:37.103 INFO:tasks.workunit.client.0.smithi038.stdout:6/79: read d1/f5 [55485,53652] 0 2024-03-24T00:48:37.104 INFO:tasks.workunit.client.1.smithi088.stdout:3/43: creat d3/fc x:0 0 0 2024-03-24T00:48:37.110 INFO:tasks.workunit.client.0.smithi038.stdout:4/59: dwrite f8 [4194304,4194304] 0 2024-03-24T00:48:37.110 INFO:tasks.workunit.client.1.smithi088.stdout:5/47: mkdir d1/d10 0 2024-03-24T00:48:37.110 INFO:tasks.workunit.client.1.smithi088.stdout:5/48: fsync d1/ff 0 2024-03-24T00:48:37.111 INFO:tasks.workunit.client.0.smithi038.stdout:2/70: rmdir d4/d18 39 2024-03-24T00:48:37.113 INFO:tasks.workunit.client.1.smithi088.stdout:0/35: symlink d9/la 0 2024-03-24T00:48:37.113 INFO:tasks.workunit.client.1.smithi088.stdout:0/36: chown c7 524 1 2024-03-24T00:48:37.116 INFO:tasks.workunit.client.0.smithi038.stdout:6/80: truncate d1/f5 955067 0 2024-03-24T00:48:37.116 INFO:tasks.workunit.client.1.smithi088.stdout:7/32: dread d3/f5 [0,4194304] 0 2024-03-24T00:48:37.116 INFO:tasks.workunit.client.1.smithi088.stdout:7/33: stat c2 0 2024-03-24T00:48:37.117 INFO:tasks.workunit.client.0.smithi038.stdout:5/48: dread d6/fa [0,4194304] 0 2024-03-24T00:48:37.118 INFO:tasks.workunit.client.1.smithi088.stdout:4/63: unlink f3 0 2024-03-24T00:48:37.120 INFO:tasks.workunit.client.1.smithi088.stdout:3/44: mknod d3/cd 0 2024-03-24T00:48:37.124 INFO:tasks.workunit.client.0.smithi038.stdout:4/60: rename da/lb to da/l13 0 2024-03-24T00:48:37.124 INFO:tasks.workunit.client.0.smithi038.stdout:4/61: fdatasync f8 0 2024-03-24T00:48:37.125 INFO:tasks.workunit.client.1.smithi088.stdout:5/49: mknod d1/d10/c11 0 2024-03-24T00:48:37.125 INFO:tasks.workunit.client.1.smithi088.stdout:5/50: getdents d1/dd 0 2024-03-24T00:48:37.125 INFO:tasks.workunit.client.1.smithi088.stdout:9/59: dwrite d1/ff [0,4194304] 0 2024-03-24T00:48:37.126 INFO:tasks.workunit.client.1.smithi088.stdout:9/60: read d1/f8 [411167,69571] 0 2024-03-24T00:48:37.129 INFO:tasks.workunit.client.1.smithi088.stdout:7/34: symlink d3/l8 0 2024-03-24T00:48:37.133 INFO:tasks.workunit.client.1.smithi088.stdout:4/64: unlink d5/d7/c8 0 2024-03-24T00:48:37.133 INFO:tasks.workunit.client.1.smithi088.stdout:4/65: dread - f4 zero size 2024-03-24T00:48:37.135 INFO:tasks.workunit.client.1.smithi088.stdout:3/45: mkdir d3/de 0 2024-03-24T00:48:37.137 INFO:tasks.workunit.client.1.smithi088.stdout:9/61: mkdir d1/d10/d11 0 2024-03-24T00:48:37.139 INFO:tasks.workunit.client.0.smithi038.stdout:2/71: dread f2 [0,4194304] 0 2024-03-24T00:48:37.140 INFO:tasks.workunit.client.0.smithi038.stdout:2/72: write d4/ff [710767,39834] 0 2024-03-24T00:48:37.141 INFO:tasks.workunit.client.1.smithi088.stdout:0/37: dread f3 [0,4194304] 0 2024-03-24T00:48:37.141 INFO:tasks.workunit.client.1.smithi088.stdout:0/38: readlink d9/la 0 2024-03-24T00:48:37.141 INFO:tasks.workunit.client.1.smithi088.stdout:0/39: write f3 [2264761,16737] 0 2024-03-24T00:48:37.143 INFO:tasks.workunit.client.1.smithi088.stdout:7/35: symlink d3/l9 0 2024-03-24T00:48:37.143 INFO:tasks.workunit.client.1.smithi088.stdout:7/36: chown d3/f5 12553 1 2024-03-24T00:48:37.143 INFO:tasks.workunit.client.1.smithi088.stdout:7/37: creat d3/fa x:0 0 0 2024-03-24T00:48:37.150 INFO:tasks.workunit.client.0.smithi038.stdout:6/81: mknod d1/c15 0 2024-03-24T00:48:37.151 INFO:tasks.workunit.client.1.smithi088.stdout:9/62: creat d1/f12 x:0 0 0 2024-03-24T00:48:37.151 INFO:tasks.workunit.client.1.smithi088.stdout:4/66: creat d5/fc x:0 0 0 2024-03-24T00:48:37.151 INFO:tasks.workunit.client.1.smithi088.stdout:4/67: dread - d5/f6 zero size 2024-03-24T00:48:37.151 INFO:tasks.workunit.client.1.smithi088.stdout:4/68: dread - d5/f6 zero size 2024-03-24T00:48:37.151 INFO:tasks.workunit.client.1.smithi088.stdout:2/37: dwrite f6 [0,4194304] 0 2024-03-24T00:48:37.151 INFO:tasks.workunit.client.1.smithi088.stdout:5/51: dread d1/ff [0,4194304] 0 2024-03-24T00:48:37.151 INFO:tasks.workunit.client.1.smithi088.stdout:5/52: truncate d1/fe 257130 0 2024-03-24T00:48:37.152 INFO:tasks.workunit.client.1.smithi088.stdout:8/45: dwrite f1 [0,4194304] 0 2024-03-24T00:48:37.153 INFO:tasks.workunit.client.1.smithi088.stdout:7/38: read d3/f4 [81958,86673] 0 2024-03-24T00:48:37.154 INFO:tasks.workunit.client.1.smithi088.stdout:6/44: sync 2024-03-24T00:48:37.161 INFO:tasks.workunit.client.1.smithi088.stdout:5/53: rename d1/lc to d1/dd/l12 0 2024-03-24T00:48:37.162 INFO:tasks.workunit.client.1.smithi088.stdout:2/38: getdents . 0 2024-03-24T00:48:37.163 INFO:tasks.workunit.client.0.smithi038.stdout:4/62: write f8 [5611219,97852] 0 2024-03-24T00:48:37.166 INFO:tasks.workunit.client.1.smithi088.stdout:1/36: dwrite f2 [0,4194304] 0 2024-03-24T00:48:37.176 INFO:tasks.workunit.client.1.smithi088.stdout:5/54: creat d1/dd/f13 x:0 0 0 2024-03-24T00:48:37.177 INFO:tasks.workunit.client.1.smithi088.stdout:5/55: write d1/ff [4822971,110759] 0 2024-03-24T00:48:37.177 INFO:tasks.workunit.client.1.smithi088.stdout:5/56: chown d1/fa 856989 1 2024-03-24T00:48:37.180 INFO:tasks.workunit.client.1.smithi088.stdout:5/57: rename d1/fe to d1/f14 0 2024-03-24T00:48:37.184 INFO:tasks.workunit.client.1.smithi088.stdout:8/46: dread f1 [0,4194304] 0 2024-03-24T00:48:37.186 INFO:tasks.workunit.client.1.smithi088.stdout:8/47: creat d9/f10 x:0 0 0 2024-03-24T00:48:37.189 INFO:tasks.workunit.client.1.smithi088.stdout:8/48: mkdir d9/dc/d11 0 2024-03-24T00:48:37.192 INFO:tasks.workunit.client.0.smithi038.stdout:2/73: dread d4/fc [0,4194304] 0 2024-03-24T00:48:37.195 INFO:tasks.workunit.client.0.smithi038.stdout:2/74: link d4/f12 d4/f1c 0 2024-03-24T00:48:37.195 INFO:tasks.workunit.client.0.smithi038.stdout:2/75: write d4/ff [1244447,36610] 0 2024-03-24T00:48:37.196 INFO:tasks.workunit.client.1.smithi088.stdout:3/46: dwrite f1 [0,4194304] 0 2024-03-24T00:48:37.207 INFO:tasks.workunit.client.1.smithi088.stdout:9/63: dread d1/f4 [0,4194304] 0 2024-03-24T00:48:37.207 INFO:tasks.workunit.client.1.smithi088.stdout:8/49: fsync d9/fb 0 2024-03-24T00:48:37.210 INFO:tasks.workunit.client.1.smithi088.stdout:9/64: mknod d1/d10/d11/c13 0 2024-03-24T00:48:37.211 INFO:tasks.workunit.client.1.smithi088.stdout:8/50: truncate f2 2634462 0 2024-03-24T00:48:37.212 INFO:tasks.workunit.client.1.smithi088.stdout:8/51: stat f2 0 2024-03-24T00:48:37.212 INFO:tasks.workunit.client.1.smithi088.stdout:8/52: chown l3 781031299 1 2024-03-24T00:48:37.212 INFO:tasks.workunit.client.1.smithi088.stdout:8/53: dread - d9/f10 zero size 2024-03-24T00:48:37.212 INFO:tasks.workunit.client.1.smithi088.stdout:8/54: rename d9 to d9/dc/d11/d12 22 2024-03-24T00:48:37.212 INFO:tasks.workunit.client.1.smithi088.stdout:9/65: mknod d1/d10/d11/c14 0 2024-03-24T00:48:37.212 INFO:tasks.workunit.client.1.smithi088.stdout:9/66: chown d1/le 34410433 1 2024-03-24T00:48:37.213 INFO:tasks.workunit.client.1.smithi088.stdout:8/55: rmdir d9/dc 39 2024-03-24T00:48:37.214 INFO:tasks.workunit.client.1.smithi088.stdout:9/67: mkdir d1/d15 0 2024-03-24T00:48:37.214 INFO:tasks.workunit.client.1.smithi088.stdout:9/68: fsync d1/ff 0 2024-03-24T00:48:37.215 INFO:tasks.workunit.client.1.smithi088.stdout:9/69: symlink d1/l16 0 2024-03-24T00:48:37.221 INFO:tasks.workunit.client.0.smithi038.stdout:2/76: read d4/f19 [66193,84710] 0 2024-03-24T00:48:37.221 INFO:tasks.workunit.client.0.smithi038.stdout:2/77: creat d4/f1d x:0 0 0 2024-03-24T00:48:37.222 INFO:tasks.workunit.client.1.smithi088.stdout:7/39: dread d3/f5 [0,4194304] 0 2024-03-24T00:48:37.222 INFO:tasks.workunit.client.1.smithi088.stdout:7/40: truncate d3/f7 725559 0 2024-03-24T00:48:37.223 INFO:tasks.workunit.client.1.smithi088.stdout:9/70: rename d1/f12 to d1/d10/d11/f17 0 2024-03-24T00:48:37.223 INFO:tasks.workunit.client.1.smithi088.stdout:1/37: dread f2 [0,4194304] 0 2024-03-24T00:48:37.227 INFO:tasks.workunit.client.0.smithi038.stdout:5/49: dwrite d6/fd [0,4194304] 0 2024-03-24T00:48:37.228 INFO:tasks.workunit.client.1.smithi088.stdout:5/58: write d1/fb [52581,67724] 0 2024-03-24T00:48:37.228 INFO:tasks.workunit.client.1.smithi088.stdout:5/59: creat d1/dd/f15 x:0 0 0 2024-03-24T00:48:37.228 INFO:tasks.workunit.client.1.smithi088.stdout:5/60: truncate d1/fa 624003 0 2024-03-24T00:48:37.231 INFO:tasks.workunit.client.1.smithi088.stdout:7/41: mknod d3/cb 0 2024-03-24T00:48:37.234 INFO:tasks.workunit.client.0.smithi038.stdout:5/50: fdatasync d6/fa 0 2024-03-24T00:48:37.234 INFO:tasks.workunit.client.0.smithi038.stdout:5/51: fsync d6/fd 0 2024-03-24T00:48:37.235 INFO:tasks.workunit.client.1.smithi088.stdout:1/38: link f1 d6/f7 0 2024-03-24T00:48:37.242 INFO:tasks.workunit.client.0.smithi038.stdout:4/63: dwrite f9 [0,4194304] 0 2024-03-24T00:48:37.242 INFO:tasks.workunit.client.0.smithi038.stdout:4/64: dread - da/dc/dd/f10 zero size 2024-03-24T00:48:37.251 INFO:tasks.workunit.client.1.smithi088.stdout:0/40: dread f3 [0,4194304] 0 2024-03-24T00:48:37.251 INFO:tasks.workunit.client.1.smithi088.stdout:0/41: write f3 [3635107,13899] 0 2024-03-24T00:48:37.251 INFO:tasks.workunit.client.1.smithi088.stdout:4/69: dwrite f2 [0,4194304] 0 2024-03-24T00:48:37.251 INFO:tasks.workunit.client.1.smithi088.stdout:4/70: write d5/f9 [462843,104916] 0 2024-03-24T00:48:37.252 INFO:tasks.workunit.client.1.smithi088.stdout:2/39: dwrite f1 [0,4194304] 0 2024-03-24T00:48:37.256 INFO:tasks.workunit.client.1.smithi088.stdout:8/56: read f2 [929175,37875] 0 2024-03-24T00:48:37.257 INFO:tasks.workunit.client.1.smithi088.stdout:4/71: mkdir d5/d7/dd 0 2024-03-24T00:48:37.257 INFO:tasks.workunit.client.1.smithi088.stdout:1/39: dread f0 [0,4194304] 0 2024-03-24T00:48:37.258 INFO:tasks.workunit.client.1.smithi088.stdout:3/47: dwrite f1 [0,4194304] 0 2024-03-24T00:48:37.259 INFO:tasks.workunit.client.1.smithi088.stdout:2/40: mknod c7 0 2024-03-24T00:48:37.260 INFO:tasks.workunit.client.0.smithi038.stdout:4/65: fsync f8 0 2024-03-24T00:48:37.262 INFO:tasks.workunit.client.1.smithi088.stdout:8/57: mknod d9/dc/d11/c13 0 2024-03-24T00:48:37.263 INFO:tasks.workunit.client.1.smithi088.stdout:1/40: readlink l4 0 2024-03-24T00:48:37.265 INFO:tasks.workunit.client.1.smithi088.stdout:4/72: creat d5/d7/fe x:0 0 0 2024-03-24T00:48:37.269 INFO:tasks.workunit.client.1.smithi088.stdout:9/71: link d1/d10/d11/f17 d1/d10/f18 0 2024-03-24T00:48:37.272 INFO:tasks.workunit.client.0.smithi038.stdout:5/52: dwrite f5 [0,4194304] 0 2024-03-24T00:48:37.276 INFO:tasks.workunit.client.1.smithi088.stdout:3/48: unlink d3/f9 0 2024-03-24T00:48:37.279 INFO:tasks.workunit.client.1.smithi088.stdout:8/58: symlink d9/dc/l14 0 2024-03-24T00:48:37.279 INFO:tasks.workunit.client.1.smithi088.stdout:8/59: chown l8 681472091 1 2024-03-24T00:48:37.280 INFO:tasks.workunit.client.1.smithi088.stdout:1/41: truncate f2 2102438 0 2024-03-24T00:48:37.281 INFO:tasks.workunit.client.1.smithi088.stdout:4/73: dread f1 [0,4194304] 0 2024-03-24T00:48:37.282 INFO:tasks.workunit.client.0.smithi038.stdout:5/53: mkdir d6/d10 0 2024-03-24T00:48:37.283 INFO:tasks.workunit.client.1.smithi088.stdout:9/72: mknod d1/d15/c19 0 2024-03-24T00:48:37.283 INFO:tasks.workunit.client.1.smithi088.stdout:9/73: write d1/d10/d11/f17 [302828,115179] 0 2024-03-24T00:48:37.283 INFO:tasks.workunit.client.1.smithi088.stdout:7/42: dwrite f1 [0,4194304] 0 2024-03-24T00:48:37.286 INFO:tasks.workunit.client.1.smithi088.stdout:3/49: mkdir d3/de/df 0 2024-03-24T00:48:37.286 INFO:tasks.workunit.client.1.smithi088.stdout:3/50: write f2 [1943993,4243] 0 2024-03-24T00:48:37.289 INFO:tasks.workunit.client.0.smithi038.stdout:5/54: creat d6/f11 x:0 0 0 2024-03-24T00:48:37.291 INFO:tasks.workunit.client.1.smithi088.stdout:8/60: rename d9/ca to d9/dc/c15 0 2024-03-24T00:48:37.292 INFO:tasks.workunit.client.1.smithi088.stdout:8/61: dread - d9/f10 zero size 2024-03-24T00:48:37.295 INFO:tasks.workunit.client.1.smithi088.stdout:1/42: rmdir d6 39 2024-03-24T00:48:37.296 INFO:tasks.workunit.client.1.smithi088.stdout:4/74: link f1 d5/d7/dd/ff 0 2024-03-24T00:48:37.298 INFO:tasks.workunit.client.1.smithi088.stdout:4/75: write d5/f9 [353968,96311] 0 2024-03-24T00:48:37.305 INFO:tasks.workunit.client.1.smithi088.stdout:2/41: getdents . 0 2024-03-24T00:48:37.309 INFO:tasks.workunit.client.1.smithi088.stdout:0/42: truncate f3 3123218 0 2024-03-24T00:48:37.309 INFO:tasks.workunit.client.0.smithi038.stdout:3/63: sync 2024-03-24T00:48:37.309 INFO:tasks.workunit.client.0.smithi038.stdout:7/34: sync 2024-03-24T00:48:37.309 INFO:tasks.workunit.client.0.smithi038.stdout:1/42: sync 2024-03-24T00:48:37.310 INFO:tasks.workunit.client.0.smithi038.stdout:7/35: chown l1 0 1 2024-03-24T00:48:37.315 INFO:tasks.workunit.client.1.smithi088.stdout:9/74: fsync d1/d10/d11/f17 0 2024-03-24T00:48:37.320 INFO:tasks.workunit.client.1.smithi088.stdout:1/43: rename f1 to d6/f8 0 2024-03-24T00:48:37.324 INFO:tasks.workunit.client.0.smithi038.stdout:7/36: mkdir d2/d9/da/dc 0 2024-03-24T00:48:37.328 INFO:tasks.workunit.client.0.smithi038.stdout:7/37: write d2/d9/fb [684319,126892] 0 2024-03-24T00:48:37.328 INFO:tasks.workunit.client.0.smithi038.stdout:2/78: dwrite d4/ff [0,4194304] 0 2024-03-24T00:48:37.332 INFO:tasks.workunit.client.1.smithi088.stdout:4/76: mkdir d5/d7/d10 0 2024-03-24T00:48:37.332 INFO:tasks.workunit.client.1.smithi088.stdout:4/77: chown f1 99 1 2024-03-24T00:48:37.338 INFO:tasks.workunit.client.1.smithi088.stdout:2/42: creat f8 x:0 0 0 2024-03-24T00:48:37.340 INFO:tasks.workunit.client.1.smithi088.stdout:9/75: dread d1/f5 [0,4194304] 0 2024-03-24T00:48:37.340 INFO:tasks.workunit.client.1.smithi088.stdout:7/43: dwrite d3/f5 [0,4194304] 0 2024-03-24T00:48:37.340 INFO:tasks.workunit.client.0.smithi038.stdout:2/79: unlink d4/f14 0 2024-03-24T00:48:37.340 INFO:tasks.workunit.client.0.smithi038.stdout:2/80: readlink d4/l17 0 2024-03-24T00:48:37.342 INFO:tasks.workunit.client.1.smithi088.stdout:1/44: symlink d6/l9 0 2024-03-24T00:48:37.345 INFO:tasks.workunit.client.1.smithi088.stdout:2/43: mknod c9 0 2024-03-24T00:48:37.345 INFO:tasks.workunit.client.1.smithi088.stdout:2/44: chown f6 2 1 2024-03-24T00:48:37.347 INFO:tasks.workunit.client.0.smithi038.stdout:2/81: truncate f2 3483976 0 2024-03-24T00:48:37.352 INFO:tasks.workunit.client.1.smithi088.stdout:6/45: sync 2024-03-24T00:48:37.352 INFO:tasks.workunit.client.1.smithi088.stdout:6/46: chown f8 178632 1 2024-03-24T00:48:37.354 INFO:tasks.workunit.client.0.smithi038.stdout:2/82: symlink d4/d18/l1e 0 2024-03-24T00:48:37.355 INFO:tasks.workunit.client.1.smithi088.stdout:9/76: mknod d1/d15/c1a 0 2024-03-24T00:48:37.355 INFO:tasks.workunit.client.1.smithi088.stdout:9/77: truncate d1/d10/d11/f17 524249 0 2024-03-24T00:48:37.355 INFO:tasks.workunit.client.1.smithi088.stdout:1/45: symlink d6/la 0 2024-03-24T00:48:37.356 INFO:tasks.workunit.client.1.smithi088.stdout:7/44: dread d3/f4 [0,4194304] 0 2024-03-24T00:48:37.356 INFO:tasks.workunit.client.1.smithi088.stdout:7/45: creat d3/fc x:0 0 0 2024-03-24T00:48:37.360 INFO:tasks.workunit.client.1.smithi088.stdout:6/47: mkdir db 0 2024-03-24T00:48:37.364 INFO:tasks.workunit.client.1.smithi088.stdout:6/48: symlink db/lc 0 2024-03-24T00:48:37.364 INFO:tasks.workunit.client.0.smithi038.stdout:3/64: read d3/d9/dc/f11 [3948745,102694] 0 2024-03-24T00:48:37.364 INFO:tasks.workunit.client.1.smithi088.stdout:6/49: chown db 1 1 2024-03-24T00:48:37.364 INFO:tasks.workunit.client.1.smithi088.stdout:6/50: fdatasync f8 0 2024-03-24T00:48:37.365 INFO:tasks.workunit.client.0.smithi038.stdout:4/66: dwrite f8 [0,4194304] 0 2024-03-24T00:48:37.367 INFO:tasks.workunit.client.0.smithi038.stdout:2/83: dread d4/f10 [0,4194304] 0 2024-03-24T00:48:37.369 INFO:tasks.workunit.client.1.smithi088.stdout:8/62: dwrite f1 [0,4194304] 0 2024-03-24T00:48:37.371 INFO:tasks.workunit.client.0.smithi038.stdout:3/65: getdents d3/d9/dc 0 2024-03-24T00:48:37.371 INFO:tasks.workunit.client.0.smithi038.stdout:3/66: truncate f2 497752 0 2024-03-24T00:48:37.371 INFO:tasks.workunit.client.0.smithi038.stdout:3/67: truncate f1 4679657 0 2024-03-24T00:48:37.371 INFO:tasks.workunit.client.0.smithi038.stdout:3/68: truncate f2 1225416 0 2024-03-24T00:48:37.372 INFO:tasks.workunit.client.1.smithi088.stdout:6/51: rmdir db 39 2024-03-24T00:48:37.374 INFO:tasks.workunit.client.0.smithi038.stdout:7/38: dwrite d2/f4 [0,4194304] 0 2024-03-24T00:48:37.374 INFO:tasks.workunit.client.0.smithi038.stdout:7/39: chown l1 45098732 1 2024-03-24T00:48:37.375 INFO:tasks.workunit.client.0.smithi038.stdout:4/67: symlink da/dc/dd/l14 0 2024-03-24T00:48:37.378 INFO:tasks.workunit.client.1.smithi088.stdout:6/52: mkdir db/dd 0 2024-03-24T00:48:37.379 INFO:tasks.workunit.client.0.smithi038.stdout:1/43: dwrite da/fb [0,4194304] 0 2024-03-24T00:48:37.379 INFO:tasks.workunit.client.0.smithi038.stdout:1/44: rename da to da/dc 22 2024-03-24T00:48:37.379 INFO:tasks.workunit.client.0.smithi038.stdout:1/45: read f7 [75235,110132] 0 2024-03-24T00:48:37.391 INFO:tasks.workunit.client.0.smithi038.stdout:7/40: creat d2/d9/da/dc/fd x:0 0 0 2024-03-24T00:48:37.393 INFO:tasks.workunit.client.0.smithi038.stdout:4/68: mkdir da/dc/dd/d15 0 2024-03-24T00:48:37.396 INFO:tasks.workunit.client.1.smithi088.stdout:6/53: rename c9 to db/dd/ce 0 2024-03-24T00:48:37.400 INFO:tasks.workunit.client.0.smithi038.stdout:8/50: sync 2024-03-24T00:48:37.401 INFO:tasks.workunit.client.0.smithi038.stdout:9/67: sync 2024-03-24T00:48:37.401 INFO:tasks.workunit.client.0.smithi038.stdout:9/68: truncate f8 4640041 0 2024-03-24T00:48:37.402 INFO:tasks.workunit.client.0.smithi038.stdout:1/46: truncate f7 3010549 0 2024-03-24T00:48:37.403 INFO:tasks.workunit.client.0.smithi038.stdout:7/41: mknod d2/ce 0 2024-03-24T00:48:37.406 INFO:tasks.workunit.client.0.smithi038.stdout:9/69: symlink d12/l19 0 2024-03-24T00:48:37.406 INFO:tasks.workunit.client.0.smithi038.stdout:1/47: mkdir da/dd 0 2024-03-24T00:48:37.407 INFO:tasks.workunit.client.0.smithi038.stdout:7/42: mkdir d2/d9/df 0 2024-03-24T00:48:37.409 INFO:tasks.workunit.client.0.smithi038.stdout:8/51: rename d0/fb to d0/f16 0 2024-03-24T00:48:37.410 INFO:tasks.workunit.client.0.smithi038.stdout:9/70: mkdir d12/d1a 0 2024-03-24T00:48:37.410 INFO:tasks.workunit.client.0.smithi038.stdout:9/71: chown fc 49023674 1 2024-03-24T00:48:37.414 INFO:tasks.workunit.client.0.smithi038.stdout:1/48: mknod da/ce 0 2024-03-24T00:48:37.415 INFO:tasks.workunit.client.0.smithi038.stdout:8/52: creat d0/d15/f17 x:0 0 0 2024-03-24T00:48:37.415 INFO:tasks.workunit.client.0.smithi038.stdout:9/72: link d12/c17 d12/c1b 0 2024-03-24T00:48:37.415 INFO:tasks.workunit.client.0.smithi038.stdout:9/73: getdents d12/d1a 0 2024-03-24T00:48:37.415 INFO:tasks.workunit.client.0.smithi038.stdout:9/74: chown fc 125724922 1 2024-03-24T00:48:37.415 INFO:tasks.workunit.client.0.smithi038.stdout:9/75: chown d12/c16 42760 1 2024-03-24T00:48:37.418 INFO:tasks.workunit.client.0.smithi038.stdout:8/53: rename d0/c7 to d0/d10/c18 0 2024-03-24T00:48:37.420 INFO:tasks.workunit.client.0.smithi038.stdout:9/76: mknod d12/d1a/c1c 0 2024-03-24T00:48:37.420 INFO:tasks.workunit.client.0.smithi038.stdout:9/77: truncate d12/f18 125047 0 2024-03-24T00:48:37.420 INFO:tasks.workunit.client.0.smithi038.stdout:0/66: sync 2024-03-24T00:48:37.420 INFO:tasks.workunit.client.0.smithi038.stdout:9/78: write f4 [4671886,88580] 0 2024-03-24T00:48:37.421 INFO:tasks.workunit.client.0.smithi038.stdout:8/54: mkdir d0/d10/d19 0 2024-03-24T00:48:37.421 INFO:tasks.workunit.client.0.smithi038.stdout:8/55: truncate d0/f16 2304964 0 2024-03-24T00:48:37.421 INFO:tasks.workunit.client.0.smithi038.stdout:8/56: creat d0/d15/f1a x:0 0 0 2024-03-24T00:48:37.421 INFO:tasks.workunit.client.0.smithi038.stdout:8/57: fdatasync d0/d15/f1a 0 2024-03-24T00:48:37.421 INFO:tasks.workunit.client.0.smithi038.stdout:9/79: mknod d12/d1a/c1d 0 2024-03-24T00:48:37.422 INFO:tasks.workunit.client.0.smithi038.stdout:9/80: fdatasync d12/f15 0 2024-03-24T00:48:37.422 INFO:tasks.workunit.client.0.smithi038.stdout:9/81: creat d12/f1e x:0 0 0 2024-03-24T00:48:37.425 INFO:tasks.workunit.client.0.smithi038.stdout:4/69: dread f9 [0,4194304] 0 2024-03-24T00:48:37.428 INFO:tasks.workunit.client.1.smithi088.stdout:9/78: dwrite d1/f5 [4194304,4194304] 0 2024-03-24T00:48:37.428 INFO:tasks.workunit.client.1.smithi088.stdout:9/79: fdatasync d1/f6 0 2024-03-24T00:48:37.441 INFO:tasks.workunit.client.0.smithi038.stdout:7/43: write d2/f4 [1918770,129135] 0 2024-03-24T00:48:37.442 INFO:tasks.workunit.client.0.smithi038.stdout:3/69: dwrite d3/d9/fb [0,4194304] 0 2024-03-24T00:48:37.443 INFO:tasks.workunit.client.1.smithi088.stdout:8/63: dwrite f2 [0,4194304] 0 2024-03-24T00:48:37.443 INFO:tasks.workunit.client.1.smithi088.stdout:8/64: truncate d9/fb 4757375 0 2024-03-24T00:48:37.444 INFO:tasks.workunit.client.1.smithi088.stdout:8/65: creat d9/dc/f16 x:0 0 0 2024-03-24T00:48:37.447 INFO:tasks.workunit.client.1.smithi088.stdout:4/78: dwrite d5/f6 [0,4194304] 0 2024-03-24T00:48:37.447 INFO:tasks.workunit.client.1.smithi088.stdout:4/79: creat d5/d7/f11 x:0 0 0 2024-03-24T00:48:37.447 INFO:tasks.workunit.client.1.smithi088.stdout:4/80: stat d5/d7/fe 0 2024-03-24T00:48:37.448 INFO:tasks.workunit.client.1.smithi088.stdout:4/81: chown d5/d7/dd 2191 1 2024-03-24T00:48:37.449 INFO:tasks.workunit.client.0.smithi038.stdout:7/44: link c0 d2/d9/df/c10 0 2024-03-24T00:48:37.449 INFO:tasks.workunit.client.0.smithi038.stdout:7/45: truncate d2/f4 4293154 0 2024-03-24T00:48:37.452 INFO:tasks.workunit.client.0.smithi038.stdout:7/46: rename c0 to d2/d9/c11 0 2024-03-24T00:48:37.453 INFO:tasks.workunit.client.0.smithi038.stdout:8/58: read d0/f9 [3126227,67192] 0 2024-03-24T00:48:37.468 INFO:tasks.workunit.client.1.smithi088.stdout:4/82: creat d5/d7/d10/f12 x:0 0 0 2024-03-24T00:48:37.468 INFO:tasks.workunit.client.1.smithi088.stdout:4/83: dread - d5/fc zero size 2024-03-24T00:48:37.471 INFO:tasks.workunit.client.1.smithi088.stdout:6/54: dwrite f8 [0,4194304] 0 2024-03-24T00:48:37.471 INFO:tasks.workunit.client.1.smithi088.stdout:6/55: fdatasync f8 0 2024-03-24T00:48:37.471 INFO:tasks.workunit.client.1.smithi088.stdout:0/43: dwrite f3 [0,4194304] 0 2024-03-24T00:48:37.476 INFO:tasks.workunit.client.1.smithi088.stdout:1/46: dwrite f2 [0,4194304] 0 2024-03-24T00:48:37.478 INFO:tasks.workunit.client.0.smithi038.stdout:7/47: rename d2/c3 to d2/d9/da/dc/c12 0 2024-03-24T00:48:37.478 INFO:tasks.workunit.client.0.smithi038.stdout:8/59: creat d0/d10/d19/f1b x:0 0 0 2024-03-24T00:48:37.478 INFO:tasks.workunit.client.0.smithi038.stdout:2/84: dwrite d4/f16 [0,4194304] 0 2024-03-24T00:48:37.478 INFO:tasks.workunit.client.0.smithi038.stdout:2/85: dread - d4/f12 zero size 2024-03-24T00:48:37.478 INFO:tasks.workunit.client.0.smithi038.stdout:8/60: write d0/d15/f17 [524639,20780] 0 2024-03-24T00:48:37.479 INFO:tasks.workunit.client.0.smithi038.stdout:8/61: chown d0/d15 15 1 2024-03-24T00:48:37.479 INFO:tasks.workunit.client.0.smithi038.stdout:8/62: fsync d0/f16 0 2024-03-24T00:48:37.479 INFO:tasks.workunit.client.0.smithi038.stdout:8/63: write d0/d15/f1a [258142,59889] 0 2024-03-24T00:48:37.479 INFO:tasks.workunit.client.0.smithi038.stdout:3/70: dread f1 [0,4194304] 0 2024-03-24T00:48:37.479 INFO:tasks.workunit.client.0.smithi038.stdout:3/71: chown d3/d9 3 1 2024-03-24T00:48:37.479 INFO:tasks.workunit.client.0.smithi038.stdout:3/72: write d3/fd [2706005,67483] 0 2024-03-24T00:48:37.479 INFO:tasks.workunit.client.0.smithi038.stdout:3/73: write d3/d9/dc/df/d14/f17 [913121,72402] 0 2024-03-24T00:48:37.479 INFO:tasks.workunit.client.0.smithi038.stdout:3/74: readlink - no filename 2024-03-24T00:48:37.479 INFO:tasks.workunit.client.0.smithi038.stdout:9/82: dread fe [0,4194304] 0 2024-03-24T00:48:37.482 INFO:tasks.workunit.client.1.smithi088.stdout:4/84: dread f1 [0,4194304] 0 2024-03-24T00:48:37.482 INFO:tasks.workunit.client.1.smithi088.stdout:4/85: fsync d5/f6 0 2024-03-24T00:48:37.484 INFO:tasks.workunit.client.0.smithi038.stdout:1/49: dwrite f4 [0,4194304] 0 2024-03-24T00:48:37.485 INFO:tasks.workunit.client.1.smithi088.stdout:9/80: truncate d1/f5 8228050 0 2024-03-24T00:48:37.489 INFO:tasks.workunit.client.1.smithi088.stdout:6/56: mknod db/cf 0 2024-03-24T00:48:37.493 INFO:tasks.workunit.client.1.smithi088.stdout:0/44: mknod d9/cb 0 2024-03-24T00:48:37.494 INFO:tasks.workunit.client.1.smithi088.stdout:0/45: chown c2 0 1 2024-03-24T00:48:37.496 INFO:tasks.workunit.client.0.smithi038.stdout:8/64: getdents d0 0 2024-03-24T00:48:37.496 INFO:tasks.workunit.client.0.smithi038.stdout:8/65: chown d0/ce 13972 1 2024-03-24T00:48:37.496 INFO:tasks.workunit.client.0.smithi038.stdout:8/66: chown d0/c5 207364 1 2024-03-24T00:48:37.497 INFO:tasks.workunit.client.0.smithi038.stdout:8/67: dread - d0/d10/d19/f1b zero size 2024-03-24T00:48:37.503 INFO:tasks.workunit.client.0.smithi038.stdout:9/83: dread fe [0,4194304] 0 2024-03-24T00:48:37.505 INFO:tasks.workunit.client.1.smithi088.stdout:8/66: dwrite f1 [0,4194304] 0 2024-03-24T00:48:37.505 INFO:tasks.workunit.client.1.smithi088.stdout:6/57: dread f8 [0,4194304] 0 2024-03-24T00:48:37.505 INFO:tasks.workunit.client.0.smithi038.stdout:8/68: symlink d0/d10/d19/l1c 0 2024-03-24T00:48:37.505 INFO:tasks.workunit.client.0.smithi038.stdout:8/69: getdents d0 0 2024-03-24T00:48:37.508 INFO:tasks.workunit.client.0.smithi038.stdout:0/67: dwrite d0/f6 [0,4194304] 0 2024-03-24T00:48:37.510 INFO:tasks.workunit.client.0.smithi038.stdout:9/84: mkdir d12/d1f 0 2024-03-24T00:48:37.512 INFO:tasks.workunit.client.0.smithi038.stdout:8/70: mkdir d0/d1d 0 2024-03-24T00:48:37.514 INFO:tasks.workunit.client.1.smithi088.stdout:6/58: read f8 [3448576,109147] 0 2024-03-24T00:48:37.517 INFO:tasks.workunit.client.1.smithi088.stdout:1/47: symlink d6/lb 0 2024-03-24T00:48:37.523 INFO:tasks.workunit.client.1.smithi088.stdout:9/81: getdents d1/d15 0 2024-03-24T00:48:37.527 INFO:tasks.workunit.client.1.smithi088.stdout:0/46: mkdir d9/dc 0 2024-03-24T00:48:37.531 INFO:tasks.workunit.client.0.smithi038.stdout:0/68: write d0/f7 [3624992,40806] 0 2024-03-24T00:48:37.537 INFO:tasks.workunit.client.1.smithi088.stdout:8/67: truncate d9/fb 2510155 0 2024-03-24T00:48:37.537 INFO:tasks.workunit.client.1.smithi088.stdout:3/51: sync 2024-03-24T00:48:37.537 INFO:tasks.workunit.client.1.smithi088.stdout:8/68: dread - d9/f10 zero size 2024-03-24T00:48:37.537 INFO:tasks.workunit.client.1.smithi088.stdout:5/61: sync 2024-03-24T00:48:37.538 INFO:tasks.workunit.client.1.smithi088.stdout:8/69: creat d9/f17 x:0 0 0 2024-03-24T00:48:37.538 INFO:tasks.workunit.client.0.smithi038.stdout:8/71: dread d0/f16 [0,4194304] 0 2024-03-24T00:48:37.538 INFO:tasks.workunit.client.0.smithi038.stdout:1/50: dread f4 [0,4194304] 0 2024-03-24T00:48:37.538 INFO:tasks.workunit.client.0.smithi038.stdout:1/51: chown da/dd 240195881 1 2024-03-24T00:48:37.539 INFO:tasks.workunit.client.1.smithi088.stdout:4/86: rename d5/d7/d10 to d5/d13 0 2024-03-24T00:48:37.539 INFO:tasks.workunit.client.1.smithi088.stdout:4/87: fsync d5/f6 0 2024-03-24T00:48:37.539 INFO:tasks.workunit.client.1.smithi088.stdout:4/88: write f4 [512919,5130] 0 2024-03-24T00:48:37.539 INFO:tasks.workunit.client.1.smithi088.stdout:1/48: dread f2 [0,4194304] 0 2024-03-24T00:48:37.550 INFO:tasks.workunit.client.1.smithi088.stdout:3/52: mknod d3/de/c10 0 2024-03-24T00:48:37.551 INFO:tasks.workunit.client.1.smithi088.stdout:0/47: dread f3 [0,4194304] 0 2024-03-24T00:48:37.555 INFO:tasks.workunit.client.1.smithi088.stdout:9/82: dread d1/ff [0,4194304] 0 2024-03-24T00:48:37.557 INFO:tasks.workunit.client.1.smithi088.stdout:4/89: dread d5/d7/dd/ff [0,4194304] 0 2024-03-24T00:48:37.561 INFO:tasks.workunit.client.1.smithi088.stdout:5/62: symlink d1/l16 0 2024-03-24T00:48:37.561 INFO:tasks.workunit.client.1.smithi088.stdout:5/63: write d1/dd/f13 [32399,89454] 0 2024-03-24T00:48:37.568 INFO:tasks.workunit.client.1.smithi088.stdout:1/49: unlink d6/l9 0 2024-03-24T00:48:37.568 INFO:tasks.workunit.client.1.smithi088.stdout:1/50: truncate d6/f7 5134029 0 2024-03-24T00:48:37.569 INFO:tasks.workunit.client.1.smithi088.stdout:3/53: write d3/fa [2597795,98128] 0 2024-03-24T00:48:37.571 INFO:tasks.workunit.client.1.smithi088.stdout:9/83: dread d1/f5 [4194304,4194304] 0 2024-03-24T00:48:37.571 INFO:tasks.workunit.client.1.smithi088.stdout:9/84: chown d1/ff 487755547 1 2024-03-24T00:48:37.573 INFO:tasks.workunit.client.1.smithi088.stdout:6/59: dwrite f8 [4194304,4194304] 0 2024-03-24T00:48:37.573 INFO:tasks.workunit.client.1.smithi088.stdout:6/60: write f8 [1916986,117914] 0 2024-03-24T00:48:37.576 INFO:tasks.workunit.client.1.smithi088.stdout:0/48: rename c2 to d9/cd 0 2024-03-24T00:48:37.579 INFO:tasks.workunit.client.0.smithi038.stdout:2/86: dread d4/f1b [0,4194304] 0 2024-03-24T00:48:37.579 INFO:tasks.workunit.client.0.smithi038.stdout:8/72: link d0/ce d0/d1d/c1e 0 2024-03-24T00:48:37.580 INFO:tasks.workunit.client.1.smithi088.stdout:4/90: truncate f1 3333872 0 2024-03-24T00:48:37.580 INFO:tasks.workunit.client.1.smithi088.stdout:4/91: creat d5/d7/f14 x:0 0 0 2024-03-24T00:48:37.581 INFO:tasks.workunit.client.0.smithi038.stdout:1/52: mkdir da/df 0 2024-03-24T00:48:37.583 INFO:tasks.workunit.client.1.smithi088.stdout:5/64: creat d1/d10/f17 x:0 0 0 2024-03-24T00:48:37.583 INFO:tasks.workunit.client.1.smithi088.stdout:3/54: link d3/d6/fb d3/d6/f11 0 2024-03-24T00:48:37.583 INFO:tasks.workunit.client.1.smithi088.stdout:5/65: readlink d1/l4 0 2024-03-24T00:48:37.583 INFO:tasks.workunit.client.1.smithi088.stdout:9/85: creat d1/d10/d11/f1b x:0 0 0 2024-03-24T00:48:37.583 INFO:tasks.workunit.client.1.smithi088.stdout:3/55: truncate d3/d6/fb 1085519 0 2024-03-24T00:48:37.584 INFO:tasks.workunit.client.1.smithi088.stdout:5/66: truncate d1/f14 698361 0 2024-03-24T00:48:37.584 INFO:tasks.workunit.client.1.smithi088.stdout:3/56: write f2 [3106003,122972] 0 2024-03-24T00:48:37.584 INFO:tasks.workunit.client.0.smithi038.stdout:2/87: creat d4/d18/f1f x:0 0 0 2024-03-24T00:48:37.585 INFO:tasks.workunit.client.0.smithi038.stdout:8/73: mknod d0/d10/d19/c1f 0 2024-03-24T00:48:37.585 INFO:tasks.workunit.client.1.smithi088.stdout:7/46: sync 2024-03-24T00:48:37.585 INFO:tasks.workunit.client.1.smithi088.stdout:2/45: sync 2024-03-24T00:48:37.586 INFO:tasks.workunit.client.1.smithi088.stdout:1/51: dread f0 [0,4194304] 0 2024-03-24T00:48:37.587 INFO:tasks.workunit.client.1.smithi088.stdout:0/49: mkdir d9/dc/de 0 2024-03-24T00:48:37.593 INFO:tasks.workunit.client.1.smithi088.stdout:4/92: mkdir d5/d7/d15 0 2024-03-24T00:48:37.594 INFO:tasks.workunit.client.0.smithi038.stdout:8/74: mkdir d0/d1d/d20 0 2024-03-24T00:48:37.597 INFO:tasks.workunit.client.1.smithi088.stdout:3/57: mkdir d3/d12 0 2024-03-24T00:48:37.599 INFO:tasks.workunit.client.1.smithi088.stdout:2/46: dread f6 [4194304,4194304] 0 2024-03-24T00:48:37.600 INFO:tasks.workunit.client.1.smithi088.stdout:7/47: unlink d3/l8 0 2024-03-24T00:48:37.600 INFO:tasks.workunit.client.1.smithi088.stdout:7/48: fsync f1 0 2024-03-24T00:48:37.601 INFO:tasks.workunit.client.1.smithi088.stdout:3/58: fdatasync d3/fa 0 2024-03-24T00:48:37.601 INFO:tasks.workunit.client.0.smithi038.stdout:8/75: dread d0/f16 [0,4194304] 0 2024-03-24T00:48:37.603 INFO:tasks.workunit.client.0.smithi038.stdout:8/76: write d0/f16 [3198088,96403] 0 2024-03-24T00:48:37.606 INFO:tasks.workunit.client.1.smithi088.stdout:0/50: link c6 d9/cf 0 2024-03-24T00:48:37.606 INFO:tasks.workunit.client.1.smithi088.stdout:4/93: mkdir d5/d16 0 2024-03-24T00:48:37.606 INFO:tasks.workunit.client.1.smithi088.stdout:9/86: mknod d1/c1c 0 2024-03-24T00:48:37.606 INFO:tasks.workunit.client.1.smithi088.stdout:0/51: write f3 [4720027,19737] 0 2024-03-24T00:48:37.606 INFO:tasks.workunit.client.1.smithi088.stdout:9/87: truncate d1/d10/d11/f17 1373676 0 2024-03-24T00:48:37.606 INFO:tasks.workunit.client.1.smithi088.stdout:9/88: readlink d1/le 0 2024-03-24T00:48:37.609 INFO:tasks.workunit.client.0.smithi038.stdout:8/77: rename d0/l2 to d0/d10/l21 0 2024-03-24T00:48:37.610 INFO:tasks.workunit.client.0.smithi038.stdout:8/78: readlink d0/ld 0 2024-03-24T00:48:37.616 INFO:tasks.workunit.client.1.smithi088.stdout:2/47: mknod ca 0 2024-03-24T00:48:37.617 INFO:tasks.workunit.client.1.smithi088.stdout:5/67: write d1/ff [4042424,116522] 0 2024-03-24T00:48:37.617 INFO:tasks.workunit.client.1.smithi088.stdout:5/68: fsync d1/fa 0 2024-03-24T00:48:37.618 INFO:tasks.workunit.client.1.smithi088.stdout:7/49: link d3/f7 d3/fd 0 2024-03-24T00:48:37.620 INFO:tasks.workunit.client.1.smithi088.stdout:1/52: dread d6/f8 [0,4194304] 0 2024-03-24T00:48:37.622 INFO:tasks.workunit.client.0.smithi038.stdout:6/82: sync 2024-03-24T00:48:37.625 INFO:tasks.workunit.client.0.smithi038.stdout:3/75: dwrite d3/d9/dc/df/d14/f17 [0,4194304] 0 2024-03-24T00:48:37.625 INFO:tasks.workunit.client.1.smithi088.stdout:4/94: mkdir d5/d17 0 2024-03-24T00:48:37.625 INFO:tasks.workunit.client.1.smithi088.stdout:9/89: creat d1/d10/f1d x:0 0 0 2024-03-24T00:48:37.625 INFO:tasks.workunit.client.1.smithi088.stdout:9/90: fdatasync d1/d10/f18 0 2024-03-24T00:48:37.626 INFO:tasks.workunit.client.1.smithi088.stdout:2/48: mkdir db 0 2024-03-24T00:48:37.635 INFO:tasks.workunit.client.1.smithi088.stdout:1/53: symlink d6/lc 0 2024-03-24T00:48:37.635 INFO:tasks.workunit.client.1.smithi088.stdout:1/54: write f0 [44136,119364] 0 2024-03-24T00:48:37.636 INFO:tasks.workunit.client.0.smithi038.stdout:2/88: rmdir d4 39 2024-03-24T00:48:37.638 INFO:tasks.workunit.client.1.smithi088.stdout:2/49: mkdir db/dc 0 2024-03-24T00:48:37.638 INFO:tasks.workunit.client.1.smithi088.stdout:2/50: chown f5 374888 1 2024-03-24T00:48:37.638 INFO:tasks.workunit.client.1.smithi088.stdout:2/51: chown c3 18399708 1 2024-03-24T00:48:37.643 INFO:tasks.workunit.client.0.smithi038.stdout:3/76: mknod d3/d9/dc/df/c18 0 2024-03-24T00:48:37.643 INFO:tasks.workunit.client.0.smithi038.stdout:3/77: readlink - no filename 2024-03-24T00:48:37.643 INFO:tasks.workunit.client.0.smithi038.stdout:7/48: dwrite d2/d9/fb [0,4194304] 0 2024-03-24T00:48:37.647 INFO:tasks.workunit.client.0.smithi038.stdout:7/49: write d2/d9/da/dc/fd [246495,9411] 0 2024-03-24T00:48:37.647 INFO:tasks.workunit.client.0.smithi038.stdout:7/50: creat d2/f13 x:0 0 0 2024-03-24T00:48:37.647 INFO:tasks.workunit.client.0.smithi038.stdout:9/85: dwrite fa [4194304,4194304] 0 2024-03-24T00:48:37.651 INFO:tasks.workunit.client.0.smithi038.stdout:9/86: write f6 [4567076,46194] 0 2024-03-24T00:48:37.651 INFO:tasks.workunit.client.0.smithi038.stdout:9/87: fsync f8 0 2024-03-24T00:48:37.654 INFO:tasks.workunit.client.0.smithi038.stdout:6/83: dread d1/da/fb [0,4194304] 0 2024-03-24T00:48:37.654 INFO:tasks.workunit.client.0.smithi038.stdout:6/84: fdatasync d1/f8 0 2024-03-24T00:48:37.654 INFO:tasks.workunit.client.0.smithi038.stdout:6/85: chown d1/f10 108 1 2024-03-24T00:48:37.655 INFO:tasks.workunit.client.0.smithi038.stdout:6/86: write d1/f5 [869736,14328] 0 2024-03-24T00:48:37.655 INFO:tasks.workunit.client.0.smithi038.stdout:0/69: dwrite d0/ff [4194304,4194304] 0 2024-03-24T00:48:37.658 INFO:tasks.workunit.client.1.smithi088.stdout:6/61: truncate f8 2012608 0 2024-03-24T00:48:37.662 INFO:tasks.workunit.client.0.smithi038.stdout:8/79: dread d0/f16 [0,4194304] 0 2024-03-24T00:48:37.664 INFO:tasks.workunit.client.0.smithi038.stdout:2/89: mknod d4/d18/c20 0 2024-03-24T00:48:37.664 INFO:tasks.workunit.client.0.smithi038.stdout:2/90: dread - d4/f15 zero size 2024-03-24T00:48:37.666 INFO:tasks.workunit.client.1.smithi088.stdout:2/52: mkdir db/dd 0 2024-03-24T00:48:37.666 INFO:tasks.workunit.client.1.smithi088.stdout:2/53: fsync f1 0 2024-03-24T00:48:37.666 INFO:tasks.workunit.client.1.smithi088.stdout:2/54: write f1 [4506711,5467] 0 2024-03-24T00:48:37.674 INFO:tasks.workunit.client.0.smithi038.stdout:2/91: mknod d4/c21 0 2024-03-24T00:48:37.677 INFO:tasks.workunit.client.0.smithi038.stdout:8/80: getdents d0/d15 0 2024-03-24T00:48:37.680 INFO:tasks.workunit.client.1.smithi088.stdout:2/55: dread f6 [0,4194304] 0 2024-03-24T00:48:37.689 INFO:tasks.workunit.client.0.smithi038.stdout:7/51: dread d2/d9/fb [0,4194304] 0 2024-03-24T00:48:37.689 INFO:tasks.workunit.client.0.smithi038.stdout:7/52: truncate d2/f13 778655 0 2024-03-24T00:48:37.692 INFO:tasks.workunit.client.0.smithi038.stdout:6/87: write d1/f8 [2963969,65669] 0 2024-03-24T00:48:37.695 INFO:tasks.workunit.client.0.smithi038.stdout:7/53: rmdir d2/d9/df 39 2024-03-24T00:48:37.696 INFO:tasks.workunit.client.0.smithi038.stdout:6/88: getdents d1 0 2024-03-24T00:48:37.697 INFO:tasks.workunit.client.0.smithi038.stdout:6/89: write d1/da/f14 [1393747,71663] 0 2024-03-24T00:48:37.697 INFO:tasks.workunit.client.0.smithi038.stdout:6/90: stat d1/da/lf 0 2024-03-24T00:48:37.697 INFO:tasks.workunit.client.0.smithi038.stdout:6/91: readlink d1/ld 0 2024-03-24T00:48:37.697 INFO:tasks.workunit.client.0.smithi038.stdout:6/92: truncate d1/f12 1487898 0 2024-03-24T00:48:37.701 INFO:tasks.workunit.client.1.smithi088.stdout:9/91: getdents d1 0 2024-03-24T00:48:37.701 INFO:tasks.workunit.client.0.smithi038.stdout:7/54: read d2/f13 [292727,123982] 0 2024-03-24T00:48:37.701 INFO:tasks.workunit.client.0.smithi038.stdout:7/55: write d2/d9/da/dc/fd [1129588,50542] 0 2024-03-24T00:48:37.705 INFO:tasks.workunit.client.1.smithi088.stdout:1/55: getdents d6 0 2024-03-24T00:48:37.709 INFO:tasks.workunit.client.1.smithi088.stdout:1/56: rename d6/f7 to d6/fd 0 2024-03-24T00:48:37.713 INFO:tasks.workunit.client.0.smithi038.stdout:2/92: dread d4/f1b [0,4194304] 0 2024-03-24T00:48:37.713 INFO:tasks.workunit.client.0.smithi038.stdout:2/93: write d4/f19 [209845,37362] 0 2024-03-24T00:48:37.713 INFO:tasks.workunit.client.0.smithi038.stdout:2/94: dread - d4/f12 zero size 2024-03-24T00:48:37.718 INFO:tasks.workunit.client.1.smithi088.stdout:9/92: read d1/f7 [7777,97267] 0 2024-03-24T00:48:37.723 INFO:tasks.workunit.client.0.smithi038.stdout:2/95: creat d4/d18/f22 x:0 0 0 2024-03-24T00:48:37.723 INFO:tasks.workunit.client.0.smithi038.stdout:2/96: fsync d4/f1d 0 2024-03-24T00:48:37.730 INFO:tasks.workunit.client.0.smithi038.stdout:2/97: unlink d4/f12 0 2024-03-24T00:48:37.730 INFO:tasks.workunit.client.0.smithi038.stdout:2/98: chown d4/f1b 4864 1 2024-03-24T00:48:37.730 INFO:tasks.workunit.client.0.smithi038.stdout:2/99: readlink d4/d18/l1a 0 2024-03-24T00:48:37.734 INFO:tasks.workunit.client.1.smithi088.stdout:5/69: dwrite d1/fa [0,4194304] 0 2024-03-24T00:48:37.739 INFO:tasks.workunit.client.1.smithi088.stdout:3/59: dwrite d3/fc [0,4194304] 0 2024-03-24T00:48:37.740 INFO:tasks.workunit.client.0.smithi038.stdout:1/53: dwrite f7 [0,4194304] 0 2024-03-24T00:48:37.741 INFO:tasks.workunit.client.1.smithi088.stdout:7/50: dwrite d3/fc [0,4194304] 0 2024-03-24T00:48:37.750 INFO:tasks.workunit.client.1.smithi088.stdout:5/70: symlink d1/d10/l18 0 2024-03-24T00:48:37.754 INFO:tasks.workunit.client.1.smithi088.stdout:6/62: dread f8 [0,4194304] 0 2024-03-24T00:48:37.754 INFO:tasks.workunit.client.1.smithi088.stdout:6/63: stat db 0 2024-03-24T00:48:37.757 INFO:tasks.workunit.client.1.smithi088.stdout:4/95: dwrite f2 [0,4194304] 0 2024-03-24T00:48:37.757 INFO:tasks.workunit.client.0.smithi038.stdout:5/55: sync 2024-03-24T00:48:37.763 INFO:tasks.workunit.client.1.smithi088.stdout:2/56: dwrite f8 [0,4194304] 0 2024-03-24T00:48:37.767 INFO:tasks.workunit.client.1.smithi088.stdout:5/71: mkdir d1/d19 0 2024-03-24T00:48:37.769 INFO:tasks.workunit.client.0.smithi038.stdout:3/78: dwrite d3/f8 [4194304,4194304] 0 2024-03-24T00:48:37.770 INFO:tasks.workunit.client.1.smithi088.stdout:6/64: creat db/dd/f10 x:0 0 0 2024-03-24T00:48:37.770 INFO:tasks.workunit.client.0.smithi038.stdout:6/93: dread d1/f6 [0,4194304] 0 2024-03-24T00:48:37.779 INFO:tasks.workunit.client.0.smithi038.stdout:4/70: sync 2024-03-24T00:48:37.779 INFO:tasks.workunit.client.0.smithi038.stdout:9/88: dwrite d12/f18 [0,4194304] 0 2024-03-24T00:48:37.780 INFO:tasks.workunit.client.1.smithi088.stdout:6/65: dread f8 [0,4194304] 0 2024-03-24T00:48:37.781 INFO:tasks.workunit.client.0.smithi038.stdout:3/79: mkdir d3/d9/dc/df/d14/d19 0 2024-03-24T00:48:37.781 INFO:tasks.workunit.client.0.smithi038.stdout:3/80: chown d3/d9/fb 15807656 1 2024-03-24T00:48:37.785 INFO:tasks.workunit.client.0.smithi038.stdout:6/94: mknod d1/da/c16 0 2024-03-24T00:48:37.787 INFO:tasks.workunit.client.1.smithi088.stdout:1/57: truncate d6/f8 2849827 0 2024-03-24T00:48:37.789 INFO:tasks.workunit.client.1.smithi088.stdout:3/60: dread f2 [4194304,4194304] 0 2024-03-24T00:48:37.789 INFO:tasks.workunit.client.1.smithi088.stdout:3/61: chown d3/fc 198582 1 2024-03-24T00:48:37.789 INFO:tasks.workunit.client.1.smithi088.stdout:2/57: dread f6 [4194304,4194304] 0 2024-03-24T00:48:37.791 INFO:tasks.workunit.client.0.smithi038.stdout:5/56: write d6/fa [28535,30392] 0 2024-03-24T00:48:37.793 INFO:tasks.workunit.client.1.smithi088.stdout:6/66: dread f8 [0,4194304] 0 2024-03-24T00:48:37.796 INFO:tasks.workunit.client.0.smithi038.stdout:9/89: rename fa to d12/d1a/f20 0 2024-03-24T00:48:37.801 INFO:tasks.workunit.client.0.smithi038.stdout:5/57: symlink d6/d10/l12 0 2024-03-24T00:48:37.803 INFO:tasks.workunit.client.1.smithi088.stdout:7/51: write f1 [1794815,26404] 0 2024-03-24T00:48:37.803 INFO:tasks.workunit.client.1.smithi088.stdout:7/52: chown c2 7 1 2024-03-24T00:48:37.803 INFO:tasks.workunit.client.1.smithi088.stdout:7/53: creat d3/fe x:0 0 0 2024-03-24T00:48:37.804 INFO:tasks.workunit.client.1.smithi088.stdout:7/54: stat d3 0 2024-03-24T00:48:37.804 INFO:tasks.workunit.client.1.smithi088.stdout:7/55: creat d3/ff x:0 0 0 2024-03-24T00:48:37.804 INFO:tasks.workunit.client.1.smithi088.stdout:5/72: fdatasync d1/ff 0 2024-03-24T00:48:37.804 INFO:tasks.workunit.client.1.smithi088.stdout:9/93: dwrite d1/f6 [4194304,4194304] 0 2024-03-24T00:48:37.804 INFO:tasks.workunit.client.1.smithi088.stdout:0/52: dwrite f3 [0,4194304] 0 2024-03-24T00:48:37.805 INFO:tasks.workunit.client.0.smithi038.stdout:9/90: rmdir d12/d1a 39 2024-03-24T00:48:37.805 INFO:tasks.workunit.client.0.smithi038.stdout:6/95: dread d1/f6 [0,4194304] 0 2024-03-24T00:48:37.805 INFO:tasks.workunit.client.0.smithi038.stdout:6/96: fsync d1/da/fb 0 2024-03-24T00:48:37.805 INFO:tasks.workunit.client.0.smithi038.stdout:6/97: fdatasync d1/f7 0 2024-03-24T00:48:37.805 INFO:tasks.workunit.client.0.smithi038.stdout:6/98: readlink d1/da/lc 0 2024-03-24T00:48:37.805 INFO:tasks.workunit.client.0.smithi038.stdout:6/99: fdatasync d1/f8 0 2024-03-24T00:48:37.805 INFO:tasks.workunit.client.0.smithi038.stdout:6/100: stat d1/f10 0 2024-03-24T00:48:37.811 INFO:tasks.workunit.client.1.smithi088.stdout:4/96: mkdir d5/d13/d18 0 2024-03-24T00:48:37.811 INFO:tasks.workunit.client.1.smithi088.stdout:4/97: dread - d5/d7/fe zero size 2024-03-24T00:48:37.814 INFO:tasks.workunit.client.0.smithi038.stdout:6/101: creat d1/f17 x:0 0 0 2024-03-24T00:48:37.814 INFO:tasks.workunit.client.0.smithi038.stdout:6/102: chown d1/c11 1560504 1 2024-03-24T00:48:37.816 INFO:tasks.workunit.client.0.smithi038.stdout:4/71: read f9 [2190265,97699] 0 2024-03-24T00:48:37.816 INFO:tasks.workunit.client.0.smithi038.stdout:4/72: read f9 [2767527,73742] 0 2024-03-24T00:48:37.816 INFO:tasks.workunit.client.0.smithi038.stdout:4/73: dread - da/dc/dd/f10 zero size 2024-03-24T00:48:37.817 INFO:tasks.workunit.client.0.smithi038.stdout:7/56: dwrite d2/f4 [0,4194304] 0 2024-03-24T00:48:37.820 INFO:tasks.workunit.client.0.smithi038.stdout:8/81: dwrite d0/f16 [0,4194304] 0 2024-03-24T00:48:37.825 INFO:tasks.workunit.client.1.smithi088.stdout:9/94: write d1/f4 [4593381,27189] 0 2024-03-24T00:48:37.826 INFO:tasks.workunit.client.1.smithi088.stdout:2/58: creat db/fe x:0 0 0 2024-03-24T00:48:37.827 INFO:tasks.workunit.client.0.smithi038.stdout:9/91: getdents d12 0 2024-03-24T00:48:37.828 INFO:tasks.workunit.client.0.smithi038.stdout:9/92: stat d12/d1f 0 2024-03-24T00:48:37.830 INFO:tasks.workunit.client.0.smithi038.stdout:6/103: mknod d1/c18 0 2024-03-24T00:48:37.831 INFO:tasks.workunit.client.1.smithi088.stdout:6/67: symlink db/l11 0 2024-03-24T00:48:37.835 INFO:tasks.workunit.client.1.smithi088.stdout:3/62: dwrite f1 [0,4194304] 0 2024-03-24T00:48:37.835 INFO:tasks.workunit.client.0.smithi038.stdout:4/74: rename da/dc/dd/f10 to da/f16 0 2024-03-24T00:48:37.839 INFO:tasks.workunit.client.0.smithi038.stdout:7/57: creat d2/d9/da/f14 x:0 0 0 2024-03-24T00:48:37.846 INFO:tasks.workunit.client.0.smithi038.stdout:5/58: getdents d6/d10 0 2024-03-24T00:48:37.848 INFO:tasks.workunit.client.1.smithi088.stdout:7/56: mkdir d3/d10 0 2024-03-24T00:48:37.848 INFO:tasks.workunit.client.1.smithi088.stdout:7/57: stat d3/fd 0 2024-03-24T00:48:37.851 INFO:tasks.workunit.client.0.smithi038.stdout:1/54: dwrite f1 [0,4194304] 0 2024-03-24T00:48:37.851 INFO:tasks.workunit.client.0.smithi038.stdout:1/55: truncate f5 5122295 0 2024-03-24T00:48:37.853 INFO:tasks.workunit.client.0.smithi038.stdout:0/70: dwrite d0/f7 [0,4194304] 0 2024-03-24T00:48:37.858 INFO:tasks.workunit.client.1.smithi088.stdout:5/73: rename d1/d10/l18 to d1/d10/l1a 0 2024-03-24T00:48:37.858 INFO:tasks.workunit.client.1.smithi088.stdout:5/74: truncate d1/dd/f15 10987 0 2024-03-24T00:48:37.860 INFO:tasks.workunit.client.0.smithi038.stdout:6/104: symlink d1/l19 0 2024-03-24T00:48:37.860 INFO:tasks.workunit.client.0.smithi038.stdout:6/105: fsync d1/f5 0 2024-03-24T00:48:37.861 INFO:tasks.workunit.client.1.smithi088.stdout:0/53: mkdir d9/dc/de/d10 0 2024-03-24T00:48:37.862 INFO:tasks.workunit.client.1.smithi088.stdout:0/54: fsync f3 0 2024-03-24T00:48:37.862 INFO:tasks.workunit.client.1.smithi088.stdout:0/55: stat d9/dc 0 2024-03-24T00:48:37.862 INFO:tasks.workunit.client.1.smithi088.stdout:4/98: creat d5/d13/d18/f19 x:0 0 0 2024-03-24T00:48:37.863 INFO:tasks.workunit.client.1.smithi088.stdout:9/95: dread d1/f7 [4194304,4194304] 0 2024-03-24T00:48:37.864 INFO:tasks.workunit.client.0.smithi038.stdout:9/93: rename d12/c14 to d12/d1a/c21 0 2024-03-24T00:48:37.865 INFO:tasks.workunit.client.1.smithi088.stdout:1/58: dwrite d6/fd [0,4194304] 0 2024-03-24T00:48:37.866 INFO:tasks.workunit.client.1.smithi088.stdout:2/59: rename l2 to db/dd/lf 0 2024-03-24T00:48:37.868 INFO:tasks.workunit.client.0.smithi038.stdout:7/58: mknod d2/d9/da/c15 0 2024-03-24T00:48:37.871 INFO:tasks.workunit.client.1.smithi088.stdout:6/68: symlink db/dd/l12 0 2024-03-24T00:48:37.871 INFO:tasks.workunit.client.0.smithi038.stdout:3/81: dwrite d3/f6 [4194304,4194304] 0 2024-03-24T00:48:37.872 INFO:tasks.workunit.client.0.smithi038.stdout:3/82: stat d3/d9/dc/df/d14/f17 0 2024-03-24T00:48:37.872 INFO:tasks.workunit.client.0.smithi038.stdout:3/83: dread - d3/d9/f16 zero size 2024-03-24T00:48:37.873 INFO:tasks.workunit.client.0.smithi038.stdout:5/59: rename d6 to d6/df/d13 22 2024-03-24T00:48:37.873 INFO:tasks.workunit.client.0.smithi038.stdout:5/60: chown d6/fd 2 1 2024-03-24T00:48:37.873 INFO:tasks.workunit.client.0.smithi038.stdout:5/61: stat d6/d10/l12 0 2024-03-24T00:48:37.875 INFO:tasks.workunit.client.0.smithi038.stdout:1/56: mkdir da/dd/d10 0 2024-03-24T00:48:37.880 INFO:tasks.workunit.client.1.smithi088.stdout:3/63: link f1 d3/d12/f13 0 2024-03-24T00:48:37.882 INFO:tasks.workunit.client.0.smithi038.stdout:0/71: mkdir d0/d12/d15/d1a 0 2024-03-24T00:48:37.887 INFO:tasks.workunit.client.1.smithi088.stdout:8/70: sync 2024-03-24T00:48:37.887 INFO:tasks.workunit.client.1.smithi088.stdout:8/71: dread - d9/f10 zero size 2024-03-24T00:48:37.887 INFO:tasks.workunit.client.1.smithi088.stdout:8/72: write d9/fb [3553485,37560] 0 2024-03-24T00:48:37.897 INFO:tasks.workunit.client.1.smithi088.stdout:7/58: rmdir d3/d10 0 2024-03-24T00:48:37.898 INFO:tasks.workunit.client.0.smithi038.stdout:9/94: write f4 [1933378,50292] 0 2024-03-24T00:48:37.904 INFO:tasks.workunit.client.1.smithi088.stdout:5/75: mkdir d1/d10/d1b 0 2024-03-24T00:48:37.904 INFO:tasks.workunit.client.1.smithi088.stdout:5/76: write d1/dd/f15 [648795,120757] 0 2024-03-24T00:48:37.905 INFO:tasks.workunit.client.0.smithi038.stdout:7/59: mkdir d2/d9/d16 0 2024-03-24T00:48:37.905 INFO:tasks.workunit.client.0.smithi038.stdout:7/60: write d2/f4 [4595836,75648] 0 2024-03-24T00:48:37.907 INFO:tasks.workunit.client.0.smithi038.stdout:3/84: getdents d3 0 2024-03-24T00:48:37.909 INFO:tasks.workunit.client.1.smithi088.stdout:0/56: creat d9/dc/de/f11 x:0 0 0 2024-03-24T00:48:37.912 INFO:tasks.workunit.client.0.smithi038.stdout:5/62: rename d6/d10 to d6/df/d14 0 2024-03-24T00:48:37.913 INFO:tasks.workunit.client.1.smithi088.stdout:9/96: rmdir d1/d15 39 2024-03-24T00:48:37.916 INFO:tasks.workunit.client.1.smithi088.stdout:4/99: dwrite f4 [0,4194304] 0 2024-03-24T00:48:37.917 INFO:tasks.workunit.client.1.smithi088.stdout:2/60: mknod db/c10 0 2024-03-24T00:48:37.919 INFO:tasks.workunit.client.0.smithi038.stdout:3/85: mkdir d3/d1a 0 2024-03-24T00:48:37.919 INFO:tasks.workunit.client.1.smithi088.stdout:1/59: dwrite d6/fd [0,4194304] 0 2024-03-24T00:48:37.920 INFO:tasks.workunit.client.0.smithi038.stdout:9/95: dread f8 [0,4194304] 0 2024-03-24T00:48:37.921 INFO:tasks.workunit.client.0.smithi038.stdout:9/96: readlink l2 0 2024-03-24T00:48:37.922 INFO:tasks.workunit.client.1.smithi088.stdout:6/69: creat db/f13 x:0 0 0 2024-03-24T00:48:37.922 INFO:tasks.workunit.client.1.smithi088.stdout:6/70: chown f8 5262 1 2024-03-24T00:48:37.925 INFO:tasks.workunit.client.0.smithi038.stdout:0/72: dread d0/f6 [0,4194304] 0 2024-03-24T00:48:37.925 INFO:tasks.workunit.client.0.smithi038.stdout:0/73: fsync d0/f7 0 2024-03-24T00:48:37.928 INFO:tasks.workunit.client.0.smithi038.stdout:3/86: read - d3/d9/f16 zero size 2024-03-24T00:48:37.931 INFO:tasks.workunit.client.1.smithi088.stdout:7/59: symlink d3/l11 0 2024-03-24T00:48:37.935 INFO:tasks.workunit.client.0.smithi038.stdout:4/75: dwrite da/f16 [0,4194304] 0 2024-03-24T00:48:37.935 INFO:tasks.workunit.client.1.smithi088.stdout:0/57: rmdir d9/dc/de/d10 0 2024-03-24T00:48:37.935 INFO:tasks.workunit.client.1.smithi088.stdout:0/58: write d9/dc/de/f11 [94547,64455] 0 2024-03-24T00:48:37.942 INFO:tasks.workunit.client.0.smithi038.stdout:5/63: rename d6/cc to d6/c15 0 2024-03-24T00:48:37.942 INFO:tasks.workunit.client.1.smithi088.stdout:9/97: dread d1/f4 [0,4194304] 0 2024-03-24T00:48:37.943 INFO:tasks.workunit.client.1.smithi088.stdout:3/64: dwrite f2 [0,4194304] 0 2024-03-24T00:48:37.944 INFO:tasks.workunit.client.1.smithi088.stdout:9/98: read d1/d10/d11/f17 [1053552,50135] 0 2024-03-24T00:48:37.944 INFO:tasks.workunit.client.1.smithi088.stdout:0/59: fdatasync d9/dc/de/f11 0 2024-03-24T00:48:37.951 INFO:tasks.workunit.client.1.smithi088.stdout:4/100: mkdir d5/d16/d1a 0 2024-03-24T00:48:37.958 INFO:tasks.workunit.client.1.smithi088.stdout:2/61: unlink f8 0 2024-03-24T00:48:37.958 INFO:tasks.workunit.client.0.smithi038.stdout:4/76: creat da/dc/dd/d15/f17 x:0 0 0 2024-03-24T00:48:37.961 INFO:tasks.workunit.client.0.smithi038.stdout:4/77: write da/dc/dd/d15/f17 [681071,124046] 0 2024-03-24T00:48:37.962 INFO:tasks.workunit.client.1.smithi088.stdout:6/71: rename db/lc to db/dd/l14 0 2024-03-24T00:48:37.965 INFO:tasks.workunit.client.0.smithi038.stdout:5/64: unlink d6/fd 0 2024-03-24T00:48:37.970 INFO:tasks.workunit.client.1.smithi088.stdout:2/62: read f6 [4002909,102782] 0 2024-03-24T00:48:37.970 INFO:tasks.workunit.client.1.smithi088.stdout:2/63: dread - db/fe zero size 2024-03-24T00:48:37.973 INFO:tasks.workunit.client.0.smithi038.stdout:4/78: creat da/f18 x:0 0 0 2024-03-24T00:48:37.973 INFO:tasks.workunit.client.0.smithi038.stdout:4/79: truncate da/f18 670797 0 2024-03-24T00:48:37.973 INFO:tasks.workunit.client.0.smithi038.stdout:6/106: dwrite d1/f6 [0,4194304] 0 2024-03-24T00:48:37.974 INFO:tasks.workunit.client.0.smithi038.stdout:9/97: dwrite fc [0,4194304] 0 2024-03-24T00:48:37.975 INFO:tasks.workunit.client.1.smithi088.stdout:8/73: rmdir d9 39 2024-03-24T00:48:37.979 INFO:tasks.workunit.client.1.smithi088.stdout:0/60: dread f3 [0,4194304] 0 2024-03-24T00:48:37.979 INFO:tasks.workunit.client.1.smithi088.stdout:0/61: chown d9/dc/de 20969 1 2024-03-24T00:48:37.981 INFO:tasks.workunit.client.0.smithi038.stdout:5/65: unlink l1 0 2024-03-24T00:48:37.984 INFO:tasks.workunit.client.1.smithi088.stdout:7/60: symlink d3/l12 0 2024-03-24T00:48:37.986 INFO:tasks.workunit.client.1.smithi088.stdout:9/99: dread d1/f4 [4194304,4194304] 0 2024-03-24T00:48:37.987 INFO:tasks.workunit.client.0.smithi038.stdout:6/107: creat d1/da/f1a x:0 0 0 2024-03-24T00:48:37.989 INFO:tasks.workunit.client.0.smithi038.stdout:5/66: truncate f5 2275642 0 2024-03-24T00:48:37.990 INFO:tasks.workunit.client.1.smithi088.stdout:3/65: symlink d3/l14 0 2024-03-24T00:48:37.996 INFO:tasks.workunit.client.0.smithi038.stdout:3/87: dwrite f1 [4194304,4194304] 0 2024-03-24T00:48:37.996 INFO:tasks.workunit.client.1.smithi088.stdout:4/101: link d5/fc d5/d13/d18/f1b 0 2024-03-24T00:48:37.996 INFO:tasks.workunit.client.1.smithi088.stdout:4/102: fsync d5/f6 0 2024-03-24T00:48:37.996 INFO:tasks.workunit.client.1.smithi088.stdout:7/61: dread d3/f4 [0,4194304] 0 2024-03-24T00:48:37.996 INFO:tasks.workunit.client.1.smithi088.stdout:6/72: rename l5 to db/l15 0 2024-03-24T00:48:37.997 INFO:tasks.workunit.client.1.smithi088.stdout:8/74: mknod d9/c18 0 2024-03-24T00:48:37.997 INFO:tasks.workunit.client.1.smithi088.stdout:8/75: chown d9/c18 451067535 1 2024-03-24T00:48:37.997 INFO:tasks.workunit.client.1.smithi088.stdout:8/76: write d9/f17 [187673,124151] 0 2024-03-24T00:48:37.998 INFO:tasks.workunit.client.0.smithi038.stdout:3/88: symlink d3/d1a/l1b 0 2024-03-24T00:48:37.998 INFO:tasks.workunit.client.0.smithi038.stdout:3/89: creat d3/d9/f1c x:0 0 0 2024-03-24T00:48:37.999 INFO:tasks.workunit.client.0.smithi038.stdout:0/74: dwrite d0/fa [4194304,4194304] 0 2024-03-24T00:48:38.001 INFO:tasks.workunit.client.0.smithi038.stdout:6/108: dread d1/f8 [0,4194304] 0 2024-03-24T00:48:38.001 INFO:tasks.workunit.client.1.smithi088.stdout:2/64: rename f5 to db/dc/f11 0 2024-03-24T00:48:38.001 INFO:tasks.workunit.client.1.smithi088.stdout:6/73: rename db to db/d16 22 2024-03-24T00:48:38.001 INFO:tasks.workunit.client.1.smithi088.stdout:8/77: mknod d9/dc/c19 0 2024-03-24T00:48:38.002 INFO:tasks.workunit.client.1.smithi088.stdout:5/77: dwrite d1/dd/f13 [0,4194304] 0 2024-03-24T00:48:38.003 INFO:tasks.workunit.client.1.smithi088.stdout:5/78: creat d1/f1c x:0 0 0 2024-03-24T00:48:38.006 INFO:tasks.workunit.client.0.smithi038.stdout:3/90: unlink d3/d9/dc/f11 0 2024-03-24T00:48:38.006 INFO:tasks.workunit.client.0.smithi038.stdout:3/91: creat d3/d9/dc/df/f1d x:0 0 0 2024-03-24T00:48:38.010 INFO:tasks.workunit.client.1.smithi088.stdout:6/74: rename db/dd/l12 to db/l17 0 2024-03-24T00:48:38.013 INFO:tasks.workunit.client.0.smithi038.stdout:0/75: mkdir d0/d1b 0 2024-03-24T00:48:38.014 INFO:tasks.workunit.client.1.smithi088.stdout:2/65: symlink db/l12 0 2024-03-24T00:48:38.021 INFO:tasks.workunit.client.0.smithi038.stdout:6/109: mkdir d1/da/d1b 0 2024-03-24T00:48:38.021 INFO:tasks.workunit.client.0.smithi038.stdout:6/110: truncate d1/da/f1a 35646 0 2024-03-24T00:48:38.022 INFO:tasks.workunit.client.1.smithi088.stdout:8/78: write f2 [1739688,48934] 0 2024-03-24T00:48:38.024 INFO:tasks.workunit.client.0.smithi038.stdout:3/92: symlink d3/d9/dc/df/d14/d19/l1e 0 2024-03-24T00:48:38.025 INFO:tasks.workunit.client.1.smithi088.stdout:5/79: creat d1/d19/f1d x:0 0 0 2024-03-24T00:48:38.026 INFO:tasks.workunit.client.1.smithi088.stdout:2/66: symlink db/dd/l13 0 2024-03-24T00:48:38.029 INFO:tasks.workunit.client.1.smithi088.stdout:8/79: symlink d9/dc/d11/l1a 0 2024-03-24T00:48:38.030 INFO:tasks.workunit.client.1.smithi088.stdout:5/80: mkdir d1/d19/d1e 0 2024-03-24T00:48:38.032 INFO:tasks.workunit.client.0.smithi038.stdout:6/111: rename d1/f17 to d1/f1c 0 2024-03-24T00:48:38.033 INFO:tasks.workunit.client.1.smithi088.stdout:2/67: mknod db/c14 0 2024-03-24T00:48:38.039 INFO:tasks.workunit.client.1.smithi088.stdout:6/75: dread f8 [0,4194304] 0 2024-03-24T00:48:38.040 INFO:tasks.workunit.client.1.smithi088.stdout:6/76: fsync db/f13 0 2024-03-24T00:48:38.040 INFO:tasks.workunit.client.1.smithi088.stdout:6/77: dread - db/f13 zero size 2024-03-24T00:48:38.040 INFO:tasks.workunit.client.1.smithi088.stdout:6/78: write db/f13 [1009233,12513] 0 2024-03-24T00:48:38.040 INFO:tasks.workunit.client.1.smithi088.stdout:8/80: mkdir d9/dc/d11/d1b 0 2024-03-24T00:48:38.048 INFO:tasks.workunit.client.1.smithi088.stdout:6/79: symlink db/dd/l18 0 2024-03-24T00:48:38.048 INFO:tasks.workunit.client.1.smithi088.stdout:6/80: chown db/cf 1717719 1 2024-03-24T00:48:38.048 INFO:tasks.workunit.client.1.smithi088.stdout:6/81: chown db/l17 2 1 2024-03-24T00:48:38.048 INFO:tasks.workunit.client.1.smithi088.stdout:6/82: write f8 [2139754,82166] 0 2024-03-24T00:48:38.048 INFO:tasks.workunit.client.1.smithi088.stdout:6/83: write f8 [2713646,46147] 0 2024-03-24T00:48:38.048 INFO:tasks.workunit.client.1.smithi088.stdout:6/84: readlink db/l17 0 2024-03-24T00:48:38.048 INFO:tasks.workunit.client.1.smithi088.stdout:6/85: stat f8 0 2024-03-24T00:48:38.048 INFO:tasks.workunit.client.1.smithi088.stdout:6/86: creat db/f19 x:0 0 0 2024-03-24T00:48:38.049 INFO:tasks.workunit.client.1.smithi088.stdout:0/62: dwrite d9/dc/de/f11 [0,4194304] 0 2024-03-24T00:48:38.051 INFO:tasks.workunit.client.1.smithi088.stdout:6/87: mknod db/c1a 0 2024-03-24T00:48:38.053 INFO:tasks.workunit.client.1.smithi088.stdout:0/63: creat d9/f12 x:0 0 0 2024-03-24T00:48:38.055 INFO:tasks.workunit.client.1.smithi088.stdout:6/88: mknod db/dd/c1b 0 2024-03-24T00:48:38.055 INFO:tasks.workunit.client.1.smithi088.stdout:6/89: write f8 [3296854,46454] 0 2024-03-24T00:48:38.055 INFO:tasks.workunit.client.1.smithi088.stdout:6/90: dread - db/dd/f10 zero size 2024-03-24T00:48:38.055 INFO:tasks.workunit.client.1.smithi088.stdout:6/91: chown db/f19 15 1 2024-03-24T00:48:38.055 INFO:tasks.workunit.client.1.smithi088.stdout:6/92: creat db/f1c x:0 0 0 2024-03-24T00:48:38.055 INFO:tasks.workunit.client.1.smithi088.stdout:6/93: chown db/l11 305475025 1 2024-03-24T00:48:38.056 INFO:tasks.workunit.client.1.smithi088.stdout:0/64: rename c4 to d9/dc/c13 0 2024-03-24T00:48:38.056 INFO:tasks.workunit.client.1.smithi088.stdout:0/65: truncate d9/dc/de/f11 4659874 0 2024-03-24T00:48:38.056 INFO:tasks.workunit.client.1.smithi088.stdout:0/66: creat d9/dc/de/f14 x:0 0 0 2024-03-24T00:48:38.059 INFO:tasks.workunit.client.1.smithi088.stdout:2/68: write f6 [6553815,20689] 0 2024-03-24T00:48:38.061 INFO:tasks.workunit.client.1.smithi088.stdout:6/94: mkdir db/d1d 0 2024-03-24T00:48:38.061 INFO:tasks.workunit.client.1.smithi088.stdout:6/95: read - db/f19 zero size 2024-03-24T00:48:38.062 INFO:tasks.workunit.client.1.smithi088.stdout:6/96: creat db/dd/f1e x:0 0 0 2024-03-24T00:48:38.062 INFO:tasks.workunit.client.1.smithi088.stdout:6/97: chown db/f1c 125 1 2024-03-24T00:48:38.064 INFO:tasks.workunit.client.0.smithi038.stdout:3/93: fsync d3/d9/f1c 0 2024-03-24T00:48:38.065 INFO:tasks.workunit.client.1.smithi088.stdout:7/62: dwrite d3/fe [0,4194304] 0 2024-03-24T00:48:38.065 INFO:tasks.workunit.client.1.smithi088.stdout:7/63: chown d3/f5 112621 1 2024-03-24T00:48:38.066 INFO:tasks.workunit.client.1.smithi088.stdout:0/67: mkdir d9/d15 0 2024-03-24T00:48:38.066 INFO:tasks.workunit.client.1.smithi088.stdout:0/68: stat d9 0 2024-03-24T00:48:38.067 INFO:tasks.workunit.client.0.smithi038.stdout:4/80: dwrite f9 [0,4194304] 0 2024-03-24T00:48:38.068 INFO:tasks.workunit.client.0.smithi038.stdout:4/81: chown da/dc/dd/l14 30680206 1 2024-03-24T00:48:38.071 INFO:tasks.workunit.client.0.smithi038.stdout:4/82: fsync da/f16 0 2024-03-24T00:48:38.071 INFO:tasks.workunit.client.0.smithi038.stdout:4/83: truncate da/f18 753294 0 2024-03-24T00:48:38.071 INFO:tasks.workunit.client.0.smithi038.stdout:9/98: dwrite f4 [0,4194304] 0 2024-03-24T00:48:38.071 INFO:tasks.workunit.client.0.smithi038.stdout:5/67: dwrite d6/f9 [0,4194304] 0 2024-03-24T00:48:38.083 INFO:tasks.workunit.client.1.smithi088.stdout:9/100: dwrite d1/d10/d11/f1b [0,4194304] 0 2024-03-24T00:48:38.083 INFO:tasks.workunit.client.1.smithi088.stdout:9/101: fdatasync d1/d10/f1d 0 2024-03-24T00:48:38.101 INFO:tasks.workunit.client.1.smithi088.stdout:9/102: dread d1/f4 [0,4194304] 0 2024-03-24T00:48:38.102 INFO:tasks.workunit.client.0.smithi038.stdout:9/99: write fb [4002993,6373] 0 2024-03-24T00:48:38.103 INFO:tasks.workunit.client.0.smithi038.stdout:6/112: dwrite d1/da/f1a [0,4194304] 0 2024-03-24T00:48:38.109 INFO:tasks.workunit.client.1.smithi088.stdout:9/103: dread d1/f8 [0,4194304] 0 2024-03-24T00:48:38.116 INFO:tasks.workunit.client.0.smithi038.stdout:6/113: dread d1/f5 [0,4194304] 0 2024-03-24T00:48:38.116 INFO:tasks.workunit.client.0.smithi038.stdout:6/114: fdatasync d1/f10 0 2024-03-24T00:48:38.125 INFO:tasks.workunit.client.1.smithi088.stdout:9/104: dread d1/f4 [0,4194304] 0 2024-03-24T00:48:38.125 INFO:tasks.workunit.client.1.smithi088.stdout:9/105: fsync d1/d10/f18 0 2024-03-24T00:48:38.125 INFO:tasks.workunit.client.1.smithi088.stdout:9/106: creat d1/d10/f1e x:0 0 0 2024-03-24T00:48:38.125 INFO:tasks.workunit.client.1.smithi088.stdout:9/107: readlink d1/le 0 2024-03-24T00:48:38.126 INFO:tasks.workunit.client.1.smithi088.stdout:9/108: readlink d1/le 0 2024-03-24T00:48:38.126 INFO:tasks.workunit.client.1.smithi088.stdout:9/109: chown d1/d10/d11/c13 164859 1 2024-03-24T00:48:38.130 INFO:tasks.workunit.client.0.smithi038.stdout:5/68: dwrite d6/fa [4194304,4194304] 0 2024-03-24T00:48:38.131 INFO:tasks.workunit.client.1.smithi088.stdout:7/64: dwrite d3/fa [0,4194304] 0 2024-03-24T00:48:38.163 INFO:tasks.workunit.client.1.smithi088.stdout:4/103: dwrite d5/d7/f11 [0,4194304] 0 2024-03-24T00:48:38.166 INFO:tasks.workunit.client.1.smithi088.stdout:8/81: dread d9/fb [0,4194304] 0 2024-03-24T00:48:38.166 INFO:tasks.workunit.client.1.smithi088.stdout:8/82: write d9/dc/f16 [655630,24729] 0 2024-03-24T00:48:38.167 INFO:tasks.workunit.client.1.smithi088.stdout:5/81: dwrite d1/f1c [0,4194304] 0 2024-03-24T00:48:38.167 INFO:tasks.workunit.client.1.smithi088.stdout:2/69: dwrite f1 [0,4194304] 0 2024-03-24T00:48:38.167 INFO:tasks.workunit.client.1.smithi088.stdout:2/70: chown db/dc/f11 8 1 2024-03-24T00:48:38.168 INFO:tasks.workunit.client.1.smithi088.stdout:2/71: fdatasync db/fe 0 2024-03-24T00:48:38.175 INFO:tasks.workunit.client.1.smithi088.stdout:8/83: dread d9/f17 [0,4194304] 0 2024-03-24T00:48:38.175 INFO:tasks.workunit.client.1.smithi088.stdout:8/84: chown l6 41806 1 2024-03-24T00:48:38.175 INFO:tasks.workunit.client.1.smithi088.stdout:8/85: write f2 [2450513,50675] 0 2024-03-24T00:48:38.180 INFO:tasks.workunit.client.1.smithi088.stdout:6/98: dwrite db/dd/f1e [0,4194304] 0 2024-03-24T00:48:38.181 INFO:tasks.workunit.client.1.smithi088.stdout:6/99: write f8 [125555,106400] 0 2024-03-24T00:48:38.181 INFO:tasks.workunit.client.1.smithi088.stdout:6/100: read - db/f19 zero size 2024-03-24T00:48:38.196 INFO:tasks.workunit.client.1.smithi088.stdout:5/82: dwrite d1/dd/f15 [0,4194304] 0 2024-03-24T00:48:38.203 INFO:tasks.workunit.client.1.smithi088.stdout:0/69: symlink d9/l16 0 2024-03-24T00:48:38.204 INFO:tasks.workunit.client.1.smithi088.stdout:5/83: dread d1/fb [0,4194304] 0 2024-03-24T00:48:38.204 INFO:tasks.workunit.client.1.smithi088.stdout:5/84: fdatasync d1/fa 0 2024-03-24T00:48:38.221 INFO:tasks.workunit.client.0.smithi038.stdout:3/94: unlink d3/d9/dc/df/c18 0 2024-03-24T00:48:38.222 INFO:tasks.workunit.client.1.smithi088.stdout:9/110: mkdir d1/d10/d1f 0 2024-03-24T00:48:38.233 INFO:tasks.workunit.client.0.smithi038.stdout:4/84: truncate f8 8302103 0 2024-03-24T00:48:38.248 INFO:tasks.workunit.client.1.smithi088.stdout:9/111: dread d1/ff [4194304,4194304] 0 2024-03-24T00:48:38.251 INFO:tasks.workunit.client.1.smithi088.stdout:0/70: dwrite d9/dc/de/f14 [0,4194304] 0 2024-03-24T00:48:38.251 INFO:tasks.workunit.client.1.smithi088.stdout:0/71: write d9/dc/de/f11 [5227706,71635] 0 2024-03-24T00:48:38.252 INFO:tasks.workunit.client.0.smithi038.stdout:9/100: symlink d12/l22 0 2024-03-24T00:48:38.252 INFO:tasks.workunit.client.0.smithi038.stdout:9/101: stat d12/d1a/c1c 0 2024-03-24T00:48:38.252 INFO:tasks.workunit.client.0.smithi038.stdout:9/102: fsync f6 0 2024-03-24T00:48:38.255 INFO:tasks.workunit.client.0.smithi038.stdout:6/115: mkdir d1/da/d1b/d1d 0 2024-03-24T00:48:38.257 INFO:tasks.workunit.client.1.smithi088.stdout:4/104: creat d5/d17/f1c x:0 0 0 2024-03-24T00:48:38.257 INFO:tasks.workunit.client.1.smithi088.stdout:4/105: creat d5/d17/f1d x:0 0 0 2024-03-24T00:48:38.257 INFO:tasks.workunit.client.1.smithi088.stdout:4/106: chown d5/d7/ca 7574074 1 2024-03-24T00:48:38.257 INFO:tasks.workunit.client.1.smithi088.stdout:4/107: dread - d5/d17/f1c zero size 2024-03-24T00:48:38.257 INFO:tasks.workunit.client.1.smithi088.stdout:4/108: chown f1 3 1 2024-03-24T00:48:38.264 INFO:tasks.workunit.client.1.smithi088.stdout:2/72: symlink db/dd/l15 0 2024-03-24T00:48:38.265 INFO:tasks.workunit.client.1.smithi088.stdout:9/112: dread d1/f6 [4194304,4194304] 0 2024-03-24T00:48:38.266 INFO:tasks.workunit.client.1.smithi088.stdout:8/86: symlink d9/dc/d11/l1c 0 2024-03-24T00:48:38.268 INFO:tasks.workunit.client.0.smithi038.stdout:3/95: creat d3/d9/dc/f1f x:0 0 0 2024-03-24T00:48:38.268 INFO:tasks.workunit.client.0.smithi038.stdout:3/96: read - d3/d9/dc/f1f zero size 2024-03-24T00:48:38.268 INFO:tasks.workunit.client.0.smithi038.stdout:3/97: read - d3/d9/f16 zero size 2024-03-24T00:48:38.269 INFO:tasks.workunit.client.1.smithi088.stdout:0/72: dread d9/dc/de/f14 [0,4194304] 0 2024-03-24T00:48:38.269 INFO:tasks.workunit.client.1.smithi088.stdout:0/73: chown c8 242 1 2024-03-24T00:48:38.269 INFO:tasks.workunit.client.1.smithi088.stdout:0/74: creat d9/dc/de/f17 x:0 0 0 2024-03-24T00:48:38.270 INFO:tasks.workunit.client.0.smithi038.stdout:4/85: creat da/dc/f19 x:0 0 0 2024-03-24T00:48:38.274 INFO:tasks.workunit.client.0.smithi038.stdout:9/103: unlink d12/c16 0 2024-03-24T00:48:38.275 INFO:tasks.workunit.client.1.smithi088.stdout:0/75: fsync d9/dc/de/f11 0 2024-03-24T00:48:38.275 INFO:tasks.workunit.client.1.smithi088.stdout:0/76: chown d9/dc 124160 1 2024-03-24T00:48:38.276 INFO:tasks.workunit.client.1.smithi088.stdout:9/113: dread d1/f8 [0,4194304] 0 2024-03-24T00:48:38.276 INFO:tasks.workunit.client.1.smithi088.stdout:9/114: read d1/f4 [4471002,71270] 0 2024-03-24T00:48:38.277 INFO:tasks.workunit.client.0.smithi038.stdout:6/116: symlink d1/da/d1b/d1d/l1e 0 2024-03-24T00:48:38.277 INFO:tasks.workunit.client.0.smithi038.stdout:6/117: chown d1/f6 2046 1 2024-03-24T00:48:38.277 INFO:tasks.workunit.client.0.smithi038.stdout:6/118: creat d1/f1f x:0 0 0 2024-03-24T00:48:38.277 INFO:tasks.workunit.client.0.smithi038.stdout:6/119: write d1/f1f [568724,6221] 0 2024-03-24T00:48:38.279 INFO:tasks.workunit.client.0.smithi038.stdout:5/69: rename d6/df to d6/d16 0 2024-03-24T00:48:38.279 INFO:tasks.workunit.client.0.smithi038.stdout:5/70: chown f5 0 1 2024-03-24T00:48:38.279 INFO:tasks.workunit.client.0.smithi038.stdout:5/71: getdents d6/d16/d14 0 2024-03-24T00:48:38.284 INFO:tasks.workunit.client.1.smithi088.stdout:5/85: rename d1/c5 to d1/d10/c1f 0 2024-03-24T00:48:38.284 INFO:tasks.workunit.client.0.smithi038.stdout:3/98: mknod d3/d9/dc/df/c20 0 2024-03-24T00:48:38.289 INFO:tasks.workunit.client.0.smithi038.stdout:6/120: creat d1/da/d1b/f20 x:0 0 0 2024-03-24T00:48:38.290 INFO:tasks.workunit.client.0.smithi038.stdout:5/72: creat d6/d16/f17 x:0 0 0 2024-03-24T00:48:38.295 INFO:tasks.workunit.client.1.smithi088.stdout:1/60: sync 2024-03-24T00:48:38.298 INFO:tasks.workunit.client.0.smithi038.stdout:6/121: link d1/f8 d1/f21 0 2024-03-24T00:48:38.298 INFO:tasks.workunit.client.0.smithi038.stdout:6/122: write d1/f5 [2451751,72689] 0 2024-03-24T00:48:38.298 INFO:tasks.workunit.client.0.smithi038.stdout:9/104: write d12/d1a/f20 [1467498,67946] 0 2024-03-24T00:48:38.299 INFO:tasks.workunit.client.0.smithi038.stdout:9/105: write f6 [4568915,71511] 0 2024-03-24T00:48:38.299 INFO:tasks.workunit.client.0.smithi038.stdout:5/73: creat d6/f18 x:0 0 0 2024-03-24T00:48:38.300 INFO:tasks.workunit.client.0.smithi038.stdout:9/106: symlink d12/d1f/l23 0 2024-03-24T00:48:38.302 INFO:tasks.workunit.client.0.smithi038.stdout:5/74: rmdir d6/d16/d14 39 2024-03-24T00:48:38.305 INFO:tasks.workunit.client.0.smithi038.stdout:9/107: symlink d12/d1a/l24 0 2024-03-24T00:48:38.305 INFO:tasks.workunit.client.0.smithi038.stdout:9/108: chown d12/d1a/c1d 26 1 2024-03-24T00:48:38.306 INFO:tasks.workunit.client.1.smithi088.stdout:8/87: symlink d9/dc/d11/l1d 0 2024-03-24T00:48:38.311 INFO:tasks.workunit.client.0.smithi038.stdout:9/109: rename d12/c1b to d12/c25 0 2024-03-24T00:48:38.314 INFO:tasks.workunit.client.0.smithi038.stdout:5/75: mkdir d6/d19 0 2024-03-24T00:48:38.314 INFO:tasks.workunit.client.0.smithi038.stdout:3/99: dread f2 [0,4194304] 0 2024-03-24T00:48:38.314 INFO:tasks.workunit.client.0.smithi038.stdout:3/100: creat d3/d9/dc/df/d14/f21 x:0 0 0 2024-03-24T00:48:38.315 INFO:tasks.workunit.client.1.smithi088.stdout:0/77: symlink d9/l18 0 2024-03-24T00:48:38.318 INFO:tasks.workunit.client.0.smithi038.stdout:2/100: sync 2024-03-24T00:48:38.318 INFO:tasks.workunit.client.0.smithi038.stdout:9/110: unlink d12/f15 0 2024-03-24T00:48:38.319 INFO:tasks.workunit.client.1.smithi088.stdout:1/61: dread f0 [0,4194304] 0 2024-03-24T00:48:38.325 INFO:tasks.workunit.client.1.smithi088.stdout:4/109: rename d5/f6 to d5/d16/d1a/f1e 0 2024-03-24T00:48:38.326 INFO:tasks.workunit.client.1.smithi088.stdout:4/110: readlink - no filename 2024-03-24T00:48:38.326 INFO:tasks.workunit.client.1.smithi088.stdout:4/111: stat d5/d7/cb 0 2024-03-24T00:48:38.329 INFO:tasks.workunit.client.1.smithi088.stdout:5/86: symlink d1/l20 0 2024-03-24T00:48:38.332 INFO:tasks.workunit.client.1.smithi088.stdout:6/101: truncate f8 337339 0 2024-03-24T00:48:38.332 INFO:tasks.workunit.client.1.smithi088.stdout:6/102: dread - db/f1c zero size 2024-03-24T00:48:38.334 INFO:tasks.workunit.client.0.smithi038.stdout:3/101: mknod d3/d1a/c22 0 2024-03-24T00:48:38.336 INFO:tasks.workunit.client.1.smithi088.stdout:0/78: mkdir d9/dc/de/d19 0 2024-03-24T00:48:38.338 INFO:tasks.workunit.client.1.smithi088.stdout:1/62: creat d6/fe x:0 0 0 2024-03-24T00:48:38.340 INFO:tasks.workunit.client.0.smithi038.stdout:4/86: dwrite da/f18 [0,4194304] 0 2024-03-24T00:48:38.340 INFO:tasks.workunit.client.0.smithi038.stdout:2/101: creat d4/f23 x:0 0 0 2024-03-24T00:48:38.346 INFO:tasks.workunit.client.1.smithi088.stdout:8/88: rename d9/fb to d9/dc/d11/d1b/f1e 0 2024-03-24T00:48:38.347 INFO:tasks.workunit.client.0.smithi038.stdout:9/111: read fc [4165937,29041] 0 2024-03-24T00:48:38.350 INFO:tasks.workunit.client.1.smithi088.stdout:4/112: rmdir d5/d16 39 2024-03-24T00:48:38.353 INFO:tasks.workunit.client.1.smithi088.stdout:1/63: unlink d6/fe 0 2024-03-24T00:48:38.356 INFO:tasks.workunit.client.1.smithi088.stdout:5/87: dread d1/fb [0,4194304] 0 2024-03-24T00:48:38.358 INFO:tasks.workunit.client.1.smithi088.stdout:8/89: mkdir d9/d1f 0 2024-03-24T00:48:38.358 INFO:tasks.workunit.client.1.smithi088.stdout:8/90: stat d9/dc/d11/l1d 0 2024-03-24T00:48:38.358 INFO:tasks.workunit.client.0.smithi038.stdout:4/87: dread da/f16 [0,4194304] 0 2024-03-24T00:48:38.359 INFO:tasks.workunit.client.1.smithi088.stdout:1/64: truncate d6/f8 2842240 0 2024-03-24T00:48:38.360 INFO:tasks.workunit.client.0.smithi038.stdout:5/76: rmdir d6 39 2024-03-24T00:48:38.362 INFO:tasks.workunit.client.0.smithi038.stdout:3/102: write d3/d9/fb [885166,357] 0 2024-03-24T00:48:38.364 INFO:tasks.workunit.client.1.smithi088.stdout:6/103: dread db/dd/f1e [0,4194304] 0 2024-03-24T00:48:38.373 INFO:tasks.workunit.client.0.smithi038.stdout:2/102: chown d4/l17 194164 1 2024-03-24T00:48:38.375 INFO:tasks.workunit.client.0.smithi038.stdout:9/112: dread f6 [0,4194304] 0 2024-03-24T00:48:38.377 INFO:tasks.workunit.client.0.smithi038.stdout:2/103: stat d4/fb 0 2024-03-24T00:48:38.378 INFO:tasks.workunit.client.1.smithi088.stdout:6/104: read db/dd/f1e [1517183,2417] 0 2024-03-24T00:48:38.378 INFO:tasks.workunit.client.1.smithi088.stdout:8/91: creat d9/f20 x:0 0 0 2024-03-24T00:48:38.379 INFO:tasks.workunit.client.0.smithi038.stdout:3/103: unlink d3/fd 0 2024-03-24T00:48:38.379 INFO:tasks.workunit.client.0.smithi038.stdout:4/88: dread da/f16 [0,4194304] 0 2024-03-24T00:48:38.381 INFO:tasks.workunit.client.0.smithi038.stdout:9/113: mkdir d12/d1f/d26 0 2024-03-24T00:48:38.381 INFO:tasks.workunit.client.0.smithi038.stdout:9/114: truncate d12/f1e 821815 0 2024-03-24T00:48:38.381 INFO:tasks.workunit.client.0.smithi038.stdout:9/115: write f6 [2668064,107872] 0 2024-03-24T00:48:38.381 INFO:tasks.workunit.client.0.smithi038.stdout:6/123: dwrite d1/f10 [4194304,4194304] 0 2024-03-24T00:48:38.382 INFO:tasks.workunit.client.1.smithi088.stdout:7/65: sync 2024-03-24T00:48:38.383 INFO:tasks.workunit.client.1.smithi088.stdout:3/66: sync 2024-03-24T00:48:38.385 INFO:tasks.workunit.client.1.smithi088.stdout:6/105: mkdir db/d1f 0 2024-03-24T00:48:38.385 INFO:tasks.workunit.client.1.smithi088.stdout:6/106: chown db/l11 188 1 2024-03-24T00:48:38.385 INFO:tasks.workunit.client.1.smithi088.stdout:6/107: write db/dd/f10 [317925,118208] 0 2024-03-24T00:48:38.385 INFO:tasks.workunit.client.0.smithi038.stdout:2/104: creat d4/f24 x:0 0 0 2024-03-24T00:48:38.386 INFO:tasks.workunit.client.0.smithi038.stdout:5/77: write f5 [2170127,15815] 0 2024-03-24T00:48:38.386 INFO:tasks.workunit.client.0.smithi038.stdout:5/78: truncate d6/d16/f17 953943 0 2024-03-24T00:48:38.386 INFO:tasks.workunit.client.1.smithi088.stdout:8/92: creat d9/d1f/f21 x:0 0 0 2024-03-24T00:48:38.392 INFO:tasks.workunit.client.1.smithi088.stdout:6/108: link db/c1a db/d1d/c20 0 2024-03-24T00:48:38.393 INFO:tasks.workunit.client.1.smithi088.stdout:5/88: dread d1/dd/f15 [0,4194304] 0 2024-03-24T00:48:38.394 INFO:tasks.workunit.client.1.smithi088.stdout:8/93: symlink d9/dc/d11/d1b/l22 0 2024-03-24T00:48:38.395 INFO:tasks.workunit.client.0.smithi038.stdout:0/76: sync 2024-03-24T00:48:38.395 INFO:tasks.workunit.client.0.smithi038.stdout:0/77: chown d0/fa 2048274 1 2024-03-24T00:48:38.395 INFO:tasks.workunit.client.0.smithi038.stdout:0/78: chown d0/l9 1 1 2024-03-24T00:48:38.395 INFO:tasks.workunit.client.0.smithi038.stdout:4/89: dread da/dc/dd/d15/f17 [0,4194304] 0 2024-03-24T00:48:38.395 INFO:tasks.workunit.client.0.smithi038.stdout:4/90: truncate da/dc/f19 238176 0 2024-03-24T00:48:38.395 INFO:tasks.workunit.client.0.smithi038.stdout:9/116: symlink d12/d1f/d26/l27 0 2024-03-24T00:48:38.395 INFO:tasks.workunit.client.0.smithi038.stdout:1/57: sync 2024-03-24T00:48:38.395 INFO:tasks.workunit.client.0.smithi038.stdout:8/82: sync 2024-03-24T00:48:38.395 INFO:tasks.workunit.client.0.smithi038.stdout:1/58: readlink l0 0 2024-03-24T00:48:38.395 INFO:tasks.workunit.client.0.smithi038.stdout:8/83: truncate d0/f16 5035094 0 2024-03-24T00:48:38.396 INFO:tasks.workunit.client.0.smithi038.stdout:7/61: sync 2024-03-24T00:48:38.396 INFO:tasks.workunit.client.0.smithi038.stdout:7/62: write d2/d9/da/f14 [772886,63961] 0 2024-03-24T00:48:38.403 INFO:tasks.workunit.client.1.smithi088.stdout:0/79: read d9/dc/de/f11 [1075970,49171] 0 2024-03-24T00:48:38.404 INFO:tasks.workunit.client.1.smithi088.stdout:7/66: rmdir d3 39 2024-03-24T00:48:38.408 INFO:tasks.workunit.client.0.smithi038.stdout:6/124: link d1/c18 d1/da/d1b/c22 0 2024-03-24T00:48:38.410 INFO:tasks.workunit.client.0.smithi038.stdout:3/104: rename d3/d1a to d3/d23 0 2024-03-24T00:48:38.411 INFO:tasks.workunit.client.1.smithi088.stdout:6/109: dread db/dd/f1e [0,4194304] 0 2024-03-24T00:48:38.415 INFO:tasks.workunit.client.0.smithi038.stdout:1/59: link f7 da/f11 0 2024-03-24T00:48:38.416 INFO:tasks.workunit.client.1.smithi088.stdout:3/67: write d3/fc [30136,121961] 0 2024-03-24T00:48:38.420 INFO:tasks.workunit.client.1.smithi088.stdout:5/89: symlink d1/d10/l21 0 2024-03-24T00:48:38.424 INFO:tasks.workunit.client.1.smithi088.stdout:0/80: dread d9/dc/de/f11 [4194304,4194304] 0 2024-03-24T00:48:38.425 INFO:tasks.workunit.client.0.smithi038.stdout:7/63: mknod d2/d9/da/dc/c17 0 2024-03-24T00:48:38.427 INFO:tasks.workunit.client.1.smithi088.stdout:8/94: mknod d9/dc/d11/d1b/c23 0 2024-03-24T00:48:38.435 INFO:tasks.workunit.client.1.smithi088.stdout:7/67: symlink d3/l13 0 2024-03-24T00:48:38.435 INFO:tasks.workunit.client.0.smithi038.stdout:2/105: symlink d4/d18/l25 0 2024-03-24T00:48:38.436 INFO:tasks.workunit.client.0.smithi038.stdout:6/125: mknod d1/c23 0 2024-03-24T00:48:38.438 INFO:tasks.workunit.client.1.smithi088.stdout:6/110: creat db/d1d/f21 x:0 0 0 2024-03-24T00:48:38.438 INFO:tasks.workunit.client.1.smithi088.stdout:8/95: dread d9/dc/d11/d1b/f1e [0,4194304] 0 2024-03-24T00:48:38.438 INFO:tasks.workunit.client.1.smithi088.stdout:8/96: chown d9/dc/f16 3387341 1 2024-03-24T00:48:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:7/64: truncate d2/d9/fb 925966 0 2024-03-24T00:48:38.440 INFO:tasks.workunit.client.1.smithi088.stdout:5/90: symlink d1/dd/l22 0 2024-03-24T00:48:38.440 INFO:tasks.workunit.client.1.smithi088.stdout:5/91: write d1/f14 [6197,9326] 0 2024-03-24T00:48:38.440 INFO:tasks.workunit.client.1.smithi088.stdout:5/92: chown d1/d10/d1b 12 1 2024-03-24T00:48:38.442 INFO:tasks.workunit.client.0.smithi038.stdout:9/117: dread f8 [0,4194304] 0 2024-03-24T00:48:38.442 INFO:tasks.workunit.client.0.smithi038.stdout:1/60: dread f4 [0,4194304] 0 2024-03-24T00:48:38.442 INFO:tasks.workunit.client.0.smithi038.stdout:2/106: dread - d4/fa zero size 2024-03-24T00:48:38.445 INFO:tasks.workunit.client.1.smithi088.stdout:7/68: link d3/fa d3/f14 0 2024-03-24T00:48:38.448 INFO:tasks.workunit.client.0.smithi038.stdout:7/65: mkdir d2/d9/da/dc/d18 0 2024-03-24T00:48:38.448 INFO:tasks.workunit.client.0.smithi038.stdout:6/126: creat d1/da/d1b/d1d/f24 x:0 0 0 2024-03-24T00:48:38.448 INFO:tasks.workunit.client.0.smithi038.stdout:2/107: mkdir d4/d18/d26 0 2024-03-24T00:48:38.448 INFO:tasks.workunit.client.0.smithi038.stdout:6/127: mkdir d1/da/d1b/d25 0 2024-03-24T00:48:38.449 INFO:tasks.workunit.client.1.smithi088.stdout:7/69: creat d3/f15 x:0 0 0 2024-03-24T00:48:38.449 INFO:tasks.workunit.client.1.smithi088.stdout:7/70: creat d3/f16 x:0 0 0 2024-03-24T00:48:38.453 INFO:tasks.workunit.client.1.smithi088.stdout:8/97: dread d9/dc/f16 [0,4194304] 0 2024-03-24T00:48:38.455 INFO:tasks.workunit.client.0.smithi038.stdout:8/84: dread d0/f9 [0,4194304] 0 2024-03-24T00:48:38.456 INFO:tasks.workunit.client.1.smithi088.stdout:8/98: mkdir d9/dc/d11/d24 0 2024-03-24T00:48:38.458 INFO:tasks.workunit.client.1.smithi088.stdout:8/99: write d9/dc/f16 [273670,125878] 0 2024-03-24T00:48:38.460 INFO:tasks.workunit.client.0.smithi038.stdout:2/108: dread d4/f10 [0,4194304] 0 2024-03-24T00:48:38.461 INFO:tasks.workunit.client.1.smithi088.stdout:8/100: symlink d9/dc/d11/d24/l25 0 2024-03-24T00:48:38.462 INFO:tasks.workunit.client.1.smithi088.stdout:4/113: dwrite d5/d17/f1c [0,4194304] 0 2024-03-24T00:48:38.463 INFO:tasks.workunit.client.1.smithi088.stdout:8/101: mknod d9/dc/d11/c26 0 2024-03-24T00:48:38.464 INFO:tasks.workunit.client.0.smithi038.stdout:0/79: write d0/ff [815754,67688] 0 2024-03-24T00:48:38.464 INFO:tasks.workunit.client.0.smithi038.stdout:9/118: fsync f4 0 2024-03-24T00:48:38.469 INFO:tasks.workunit.client.1.smithi088.stdout:1/65: dwrite d6/fd [0,4194304] 0 2024-03-24T00:48:38.469 INFO:tasks.workunit.client.1.smithi088.stdout:1/66: fsync f0 0 2024-03-24T00:48:38.469 INFO:tasks.workunit.client.1.smithi088.stdout:1/67: readlink d6/lb 0 2024-03-24T00:48:38.474 INFO:tasks.workunit.client.1.smithi088.stdout:6/111: getdents db/d1d 0 2024-03-24T00:48:38.475 INFO:tasks.workunit.client.1.smithi088.stdout:6/112: write db/f1c [310941,33572] 0 2024-03-24T00:48:38.478 INFO:tasks.workunit.client.0.smithi038.stdout:4/91: dwrite da/dc/f19 [0,4194304] 0 2024-03-24T00:48:38.478 INFO:tasks.workunit.client.0.smithi038.stdout:4/92: write f8 [5308166,116985] 0 2024-03-24T00:48:38.480 INFO:tasks.workunit.client.1.smithi088.stdout:4/114: mknod d5/d7/c1f 0 2024-03-24T00:48:38.487 INFO:tasks.workunit.client.0.smithi038.stdout:8/85: creat d0/d1d/f22 x:0 0 0 2024-03-24T00:48:38.487 INFO:tasks.workunit.client.0.smithi038.stdout:6/128: rmdir d1/da/d1b/d1d 39 2024-03-24T00:48:38.490 INFO:tasks.workunit.client.1.smithi088.stdout:8/102: mknod d9/dc/d11/c27 0 2024-03-24T00:48:38.490 INFO:tasks.workunit.client.1.smithi088.stdout:8/103: dread - d9/d1f/f21 zero size 2024-03-24T00:48:38.490 INFO:tasks.workunit.client.1.smithi088.stdout:8/104: fsync d9/f17 0 2024-03-24T00:48:38.491 INFO:tasks.workunit.client.0.smithi038.stdout:0/80: dread d0/f7 [0,4194304] 0 2024-03-24T00:48:38.501 INFO:tasks.workunit.client.0.smithi038.stdout:4/93: rename da/dc/dd/l14 to da/dc/dd/l1a 0 2024-03-24T00:48:38.502 INFO:tasks.workunit.client.1.smithi088.stdout:4/115: symlink d5/d16/l20 0 2024-03-24T00:48:38.502 INFO:tasks.workunit.client.1.smithi088.stdout:2/73: sync 2024-03-24T00:48:38.502 INFO:tasks.workunit.client.1.smithi088.stdout:4/116: write f1 [4004338,128401] 0 2024-03-24T00:48:38.503 INFO:tasks.workunit.client.0.smithi038.stdout:8/86: creat d0/d1d/d20/f23 x:0 0 0 2024-03-24T00:48:38.503 INFO:tasks.workunit.client.0.smithi038.stdout:8/87: getdents d0/d15 0 2024-03-24T00:48:38.505 INFO:tasks.workunit.client.1.smithi088.stdout:7/71: dwrite d3/f15 [0,4194304] 0 2024-03-24T00:48:38.505 INFO:tasks.workunit.client.1.smithi088.stdout:7/72: creat d3/f17 x:0 0 0 2024-03-24T00:48:38.506 INFO:tasks.workunit.client.0.smithi038.stdout:5/79: dwrite d6/d16/f17 [0,4194304] 0 2024-03-24T00:48:38.506 INFO:tasks.workunit.client.0.smithi038.stdout:5/80: dread - d6/f11 zero size 2024-03-24T00:48:38.506 INFO:tasks.workunit.client.0.smithi038.stdout:2/109: mkdir d4/d27 0 2024-03-24T00:48:38.506 INFO:tasks.workunit.client.0.smithi038.stdout:2/110: creat d4/d18/f28 x:0 0 0 2024-03-24T00:48:38.508 INFO:tasks.workunit.client.1.smithi088.stdout:1/68: getdents d6 0 2024-03-24T00:48:38.509 INFO:tasks.workunit.client.1.smithi088.stdout:6/113: getdents db 0 2024-03-24T00:48:38.509 INFO:tasks.workunit.client.0.smithi038.stdout:0/81: unlink d0/f6 0 2024-03-24T00:48:38.509 INFO:tasks.workunit.client.0.smithi038.stdout:6/129: dread d1/da/f14 [0,4194304] 0 2024-03-24T00:48:38.510 INFO:tasks.workunit.client.0.smithi038.stdout:4/94: mknod da/dc/dd/c1b 0 2024-03-24T00:48:38.510 INFO:tasks.workunit.client.0.smithi038.stdout:4/95: truncate da/dc/dd/d15/f17 1033285 0 2024-03-24T00:48:38.512 INFO:tasks.workunit.client.1.smithi088.stdout:2/74: symlink db/dd/l16 0 2024-03-24T00:48:38.514 INFO:tasks.workunit.client.1.smithi088.stdout:9/115: sync 2024-03-24T00:48:38.515 INFO:tasks.workunit.client.0.smithi038.stdout:8/88: creat d0/f24 x:0 0 0 2024-03-24T00:48:38.520 INFO:tasks.workunit.client.0.smithi038.stdout:6/130: dread d1/f12 [0,4194304] 0 2024-03-24T00:48:38.520 INFO:tasks.workunit.client.0.smithi038.stdout:6/131: creat d1/da/d1b/f26 x:0 0 0 2024-03-24T00:48:38.521 INFO:tasks.workunit.client.0.smithi038.stdout:1/61: write da/fb [1957753,2023] 0 2024-03-24T00:48:38.522 INFO:tasks.workunit.client.0.smithi038.stdout:9/119: dwrite f4 [0,4194304] 0 2024-03-24T00:48:38.523 INFO:tasks.workunit.client.1.smithi088.stdout:4/117: creat d5/d7/dd/f21 x:0 0 0 2024-03-24T00:48:38.523 INFO:tasks.workunit.client.1.smithi088.stdout:7/73: write d3/fe [795088,112048] 0 2024-03-24T00:48:38.526 INFO:tasks.workunit.client.1.smithi088.stdout:8/105: rmdir d9/d1f 39 2024-03-24T00:48:38.526 INFO:tasks.workunit.client.1.smithi088.stdout:8/106: write f2 [453570,117559] 0 2024-03-24T00:48:38.526 INFO:tasks.workunit.client.1.smithi088.stdout:8/107: write f2 [4788661,38509] 0 2024-03-24T00:48:38.526 INFO:tasks.workunit.client.1.smithi088.stdout:8/108: creat d9/f28 x:0 0 0 2024-03-24T00:48:38.527 INFO:tasks.workunit.client.0.smithi038.stdout:5/81: symlink d6/d16/l1a 0 2024-03-24T00:48:38.532 INFO:tasks.workunit.client.0.smithi038.stdout:0/82: symlink d0/d12/d15/d1a/l1c 0 2024-03-24T00:48:38.535 INFO:tasks.workunit.client.0.smithi038.stdout:2/111: read d4/fc [351517,17009] 0 2024-03-24T00:48:38.536 INFO:tasks.workunit.client.0.smithi038.stdout:2/112: dread - d4/d18/f28 zero size 2024-03-24T00:48:38.537 INFO:tasks.workunit.client.1.smithi088.stdout:2/75: creat db/dd/f17 x:0 0 0 2024-03-24T00:48:38.537 INFO:tasks.workunit.client.1.smithi088.stdout:1/69: write f0 [3965080,7570] 0 2024-03-24T00:48:38.537 INFO:tasks.workunit.client.1.smithi088.stdout:6/114: symlink db/d1d/l22 0 2024-03-24T00:48:38.537 INFO:tasks.workunit.client.1.smithi088.stdout:9/116: creat d1/f20 x:0 0 0 2024-03-24T00:48:38.538 INFO:tasks.workunit.client.0.smithi038.stdout:8/89: mknod d0/d10/c25 0 2024-03-24T00:48:38.538 INFO:tasks.workunit.client.0.smithi038.stdout:8/90: fdatasync d0/d1d/d20/f23 0 2024-03-24T00:48:38.540 INFO:tasks.workunit.client.0.smithi038.stdout:7/66: dwrite d2/d9/da/dc/fd [0,4194304] 0 2024-03-24T00:48:38.541 INFO:tasks.workunit.client.1.smithi088.stdout:4/118: mkdir d5/d16/d1a/d22 0 2024-03-24T00:48:38.541 INFO:tasks.workunit.client.1.smithi088.stdout:4/119: creat d5/d7/f23 x:0 0 0 2024-03-24T00:48:38.542 INFO:tasks.workunit.client.1.smithi088.stdout:4/120: stat d5/d7/fe 0 2024-03-24T00:48:38.544 INFO:tasks.workunit.client.1.smithi088.stdout:9/117: dread d1/d10/f18 [0,4194304] 0 2024-03-24T00:48:38.544 INFO:tasks.workunit.client.1.smithi088.stdout:9/118: chown d1/f5 2 1 2024-03-24T00:48:38.545 INFO:tasks.workunit.client.0.smithi038.stdout:5/82: write d6/d16/f17 [1248477,127440] 0 2024-03-24T00:48:38.546 INFO:tasks.workunit.client.0.smithi038.stdout:1/62: getdents da/dd 0 2024-03-24T00:48:38.550 INFO:tasks.workunit.client.0.smithi038.stdout:9/120: mkdir d12/d28 0 2024-03-24T00:48:38.550 INFO:tasks.workunit.client.0.smithi038.stdout:0/83: link d0/f5 d0/d1b/f1d 0 2024-03-24T00:48:38.553 INFO:tasks.workunit.client.0.smithi038.stdout:4/96: unlink da/dc/f19 0 2024-03-24T00:48:38.553 INFO:tasks.workunit.client.0.smithi038.stdout:4/97: fdatasync da/dc/dd/d15/f17 0 2024-03-24T00:48:38.555 INFO:tasks.workunit.client.1.smithi088.stdout:8/109: mkdir d9/dc/d11/d24/d29 0 2024-03-24T00:48:38.555 INFO:tasks.workunit.client.1.smithi088.stdout:2/76: dread f1 [0,4194304] 0 2024-03-24T00:48:38.555 INFO:tasks.workunit.client.1.smithi088.stdout:6/115: mkdir db/d1d/d23 0 2024-03-24T00:48:38.562 INFO:tasks.workunit.client.0.smithi038.stdout:7/67: creat d2/d9/d16/f19 x:0 0 0 2024-03-24T00:48:38.562 INFO:tasks.workunit.client.0.smithi038.stdout:7/68: chown l1 833 1 2024-03-24T00:48:38.565 INFO:tasks.workunit.client.0.smithi038.stdout:4/98: link f8 da/f1c 0 2024-03-24T00:48:38.565 INFO:tasks.workunit.client.0.smithi038.stdout:4/99: creat da/f1d x:0 0 0 2024-03-24T00:48:38.565 INFO:tasks.workunit.client.0.smithi038.stdout:4/100: write da/f16 [5208565,97545] 0 2024-03-24T00:48:38.569 INFO:tasks.workunit.client.0.smithi038.stdout:5/83: write d6/f9 [3151831,63292] 0 2024-03-24T00:48:38.569 INFO:tasks.workunit.client.0.smithi038.stdout:5/84: fdatasync d6/fa 0 2024-03-24T00:48:38.571 INFO:tasks.workunit.client.0.smithi038.stdout:5/85: mknod d6/d16/c1b 0 2024-03-24T00:48:38.576 INFO:tasks.workunit.client.1.smithi088.stdout:6/116: dread db/dd/f10 [0,4194304] 0 2024-03-24T00:48:38.576 INFO:tasks.workunit.client.1.smithi088.stdout:6/117: fsync db/f19 0 2024-03-24T00:48:38.576 INFO:tasks.workunit.client.1.smithi088.stdout:6/118: chown db/f19 1871 1 2024-03-24T00:48:38.592 INFO:tasks.workunit.client.0.smithi038.stdout:2/113: dread d4/f1b [0,4194304] 0 2024-03-24T00:48:38.599 INFO:tasks.workunit.client.0.smithi038.stdout:2/114: stat d4/f13 0 2024-03-24T00:48:38.599 INFO:tasks.workunit.client.0.smithi038.stdout:2/115: getdents d4/d27 0 2024-03-24T00:48:38.603 INFO:tasks.workunit.client.0.smithi038.stdout:2/116: creat d4/d18/d26/f29 x:0 0 0 2024-03-24T00:48:38.604 INFO:tasks.workunit.client.0.smithi038.stdout:2/117: fsync d4/fd 0 2024-03-24T00:48:38.604 INFO:tasks.workunit.client.0.smithi038.stdout:2/118: dread - d4/f24 zero size 2024-03-24T00:48:38.607 INFO:tasks.workunit.client.0.smithi038.stdout:2/119: creat d4/d27/f2a x:0 0 0 2024-03-24T00:48:38.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:38 smithi038 ceph-mon[29271]: pgmap v25: 65 pgs: 65 active+clean; 368 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 449 op/s 2024-03-24T00:48:38.612 INFO:tasks.workunit.client.0.smithi038.stdout:7/69: write d2/d9/da/dc/fd [4441539,2752] 0 2024-03-24T00:48:38.615 INFO:tasks.workunit.client.0.smithi038.stdout:9/121: dwrite f6 [0,4194304] 0 2024-03-24T00:48:38.616 INFO:tasks.workunit.client.0.smithi038.stdout:2/120: link d4/fa d4/f2b 0 2024-03-24T00:48:38.622 INFO:tasks.workunit.client.0.smithi038.stdout:4/101: fsync f8 0 2024-03-24T00:48:38.624 INFO:tasks.workunit.client.0.smithi038.stdout:9/122: symlink d12/d1a/l29 0 2024-03-24T00:48:38.624 INFO:tasks.workunit.client.0.smithi038.stdout:9/123: readlink d12/l13 0 2024-03-24T00:48:38.625 INFO:tasks.workunit.client.0.smithi038.stdout:9/124: fsync fb 0 2024-03-24T00:48:38.627 INFO:tasks.workunit.client.1.smithi088.stdout:4/121: dwrite f2 [4194304,4194304] 0 2024-03-24T00:48:38.629 INFO:tasks.workunit.client.0.smithi038.stdout:2/121: rename d4/fb to d4/d18/d26/f2c 0 2024-03-24T00:48:38.631 INFO:tasks.workunit.client.0.smithi038.stdout:8/91: dwrite d0/f16 [0,4194304] 0 2024-03-24T00:48:38.632 INFO:tasks.workunit.client.0.smithi038.stdout:2/122: mknod d4/d27/c2d 0 2024-03-24T00:48:38.632 INFO:tasks.workunit.client.0.smithi038.stdout:2/123: creat d4/d18/f2e x:0 0 0 2024-03-24T00:48:38.634 INFO:tasks.workunit.client.0.smithi038.stdout:8/92: unlink d0/c8 0 2024-03-24T00:48:38.637 INFO:tasks.workunit.client.0.smithi038.stdout:2/124: mkdir d4/d27/d2f 0 2024-03-24T00:48:38.638 INFO:tasks.workunit.client.0.smithi038.stdout:8/93: symlink d0/d10/d19/l26 0 2024-03-24T00:48:38.643 INFO:tasks.workunit.client.0.smithi038.stdout:2/125: mkdir d4/d27/d2f/d30 0 2024-03-24T00:48:38.643 INFO:tasks.workunit.client.0.smithi038.stdout:2/126: creat d4/d18/f31 x:0 0 0 2024-03-24T00:48:38.643 INFO:tasks.workunit.client.1.smithi088.stdout:4/122: write d5/d7/f11 [4106003,29577] 0 2024-03-24T00:48:38.645 INFO:tasks.workunit.client.1.smithi088.stdout:4/123: truncate f4 982554 0 2024-03-24T00:48:38.645 INFO:tasks.workunit.client.1.smithi088.stdout:4/124: dread - d5/d7/f23 zero size 2024-03-24T00:48:38.647 INFO:tasks.workunit.client.1.smithi088.stdout:8/110: dwrite d9/f17 [0,4194304] 0 2024-03-24T00:48:38.649 INFO:tasks.workunit.client.1.smithi088.stdout:4/125: mkdir d5/d13/d18/d24 0 2024-03-24T00:48:38.654 INFO:tasks.workunit.client.1.smithi088.stdout:2/77: dwrite db/dd/f17 [0,4194304] 0 2024-03-24T00:48:38.654 INFO:tasks.workunit.client.1.smithi088.stdout:2/78: stat ca 0 2024-03-24T00:48:38.655 INFO:tasks.workunit.client.1.smithi088.stdout:6/119: dwrite db/dd/f10 [0,4194304] 0 2024-03-24T00:48:38.658 INFO:tasks.workunit.client.1.smithi088.stdout:1/70: dwrite d6/fd [0,4194304] 0 2024-03-24T00:48:38.664 INFO:tasks.workunit.client.0.smithi038.stdout:0/84: dwrite d0/f2 [0,4194304] 0 2024-03-24T00:48:38.668 INFO:tasks.workunit.client.1.smithi088.stdout:2/79: read f1 [2939372,9434] 0 2024-03-24T00:48:38.671 INFO:tasks.workunit.client.0.smithi038.stdout:5/86: dwrite d6/fa [0,4194304] 0 2024-03-24T00:48:38.676 INFO:tasks.workunit.client.0.smithi038.stdout:9/125: dwrite f8 [0,4194304] 0 2024-03-24T00:48:38.676 INFO:tasks.workunit.client.0.smithi038.stdout:2/127: write d4/f1b [2367430,28456] 0 2024-03-24T00:48:38.679 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:38 smithi088 ceph-mon[36610]: pgmap v25: 65 pgs: 65 active+clean; 368 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 449 op/s 2024-03-24T00:48:38.681 INFO:tasks.workunit.client.1.smithi088.stdout:6/120: mkdir db/d1f/d24 0 2024-03-24T00:48:38.681 INFO:tasks.workunit.client.1.smithi088.stdout:6/121: chown ca 3501 1 2024-03-24T00:48:38.682 INFO:tasks.workunit.client.1.smithi088.stdout:6/122: fsync db/dd/f10 0 2024-03-24T00:48:38.685 INFO:tasks.workunit.client.0.smithi038.stdout:4/102: write da/f18 [562914,89434] 0 2024-03-24T00:48:38.692 INFO:tasks.workunit.client.0.smithi038.stdout:0/85: read d0/f2 [3873912,47588] 0 2024-03-24T00:48:38.698 INFO:tasks.workunit.client.0.smithi038.stdout:0/86: creat d0/d12/d15/d1a/f1e x:0 0 0 2024-03-24T00:48:38.699 INFO:tasks.workunit.client.0.smithi038.stdout:9/126: read d12/f18 [3662312,40496] 0 2024-03-24T00:48:38.705 INFO:tasks.workunit.client.1.smithi088.stdout:1/71: dread d6/fd [0,4194304] 0 2024-03-24T00:48:38.706 INFO:tasks.workunit.client.1.smithi088.stdout:3/68: sync 2024-03-24T00:48:38.706 INFO:tasks.workunit.client.1.smithi088.stdout:0/81: sync 2024-03-24T00:48:38.706 INFO:tasks.workunit.client.1.smithi088.stdout:5/93: sync 2024-03-24T00:48:38.707 INFO:tasks.workunit.client.0.smithi038.stdout:0/87: mknod d0/d12/d13/c1f 0 2024-03-24T00:48:38.709 INFO:tasks.workunit.client.0.smithi038.stdout:9/127: symlink d12/l2a 0 2024-03-24T00:48:38.710 INFO:tasks.workunit.client.1.smithi088.stdout:1/72: link f0 d6/ff 0 2024-03-24T00:48:38.710 INFO:tasks.workunit.client.1.smithi088.stdout:1/73: readlink d6/lb 0 2024-03-24T00:48:38.713 INFO:tasks.workunit.client.1.smithi088.stdout:5/94: unlink d1/d19/f1d 0 2024-03-24T00:48:38.713 INFO:tasks.workunit.client.1.smithi088.stdout:3/69: rmdir d3/de 39 2024-03-24T00:48:38.713 INFO:tasks.workunit.client.1.smithi088.stdout:5/95: chown d1/l8 8 1 2024-03-24T00:48:38.713 INFO:tasks.workunit.client.1.smithi088.stdout:5/96: dread - d1/d10/f17 zero size 2024-03-24T00:48:38.713 INFO:tasks.workunit.client.1.smithi088.stdout:0/82: creat d9/dc/de/d19/f1a x:0 0 0 2024-03-24T00:48:38.716 INFO:tasks.workunit.client.1.smithi088.stdout:1/74: truncate d6/f8 1338679 0 2024-03-24T00:48:38.718 INFO:tasks.workunit.client.1.smithi088.stdout:3/70: truncate f1 1608109 0 2024-03-24T00:48:38.718 INFO:tasks.workunit.client.1.smithi088.stdout:3/71: write d3/fc [2616981,29547] 0 2024-03-24T00:48:38.719 INFO:tasks.workunit.client.1.smithi088.stdout:5/97: unlink d1/d10/l21 0 2024-03-24T00:48:38.721 INFO:tasks.workunit.client.1.smithi088.stdout:1/75: creat d6/f10 x:0 0 0 2024-03-24T00:48:38.722 INFO:tasks.workunit.client.1.smithi088.stdout:1/76: chown d6/f8 1 1 2024-03-24T00:48:38.722 INFO:tasks.workunit.client.0.smithi038.stdout:8/94: dwrite d0/d15/f17 [0,4194304] 0 2024-03-24T00:48:38.738 INFO:tasks.workunit.client.1.smithi088.stdout:1/77: getdents d6 0 2024-03-24T00:48:38.756 INFO:tasks.workunit.client.1.smithi088.stdout:1/78: dread d6/fd [0,4194304] 0 2024-03-24T00:48:38.759 INFO:tasks.workunit.client.1.smithi088.stdout:1/79: link l4 d6/l11 0 2024-03-24T00:48:38.763 INFO:tasks.workunit.client.0.smithi038.stdout:8/95: dread d0/d15/f1a [0,4194304] 0 2024-03-24T00:48:38.769 INFO:tasks.workunit.client.1.smithi088.stdout:1/80: symlink d6/l12 0 2024-03-24T00:48:38.773 INFO:tasks.workunit.client.0.smithi038.stdout:7/70: dwrite d2/d9/da/dc/fd [0,4194304] 0 2024-03-24T00:48:38.773 INFO:tasks.workunit.client.0.smithi038.stdout:8/96: mkdir d0/d1d/d27 0 2024-03-24T00:48:38.773 INFO:tasks.workunit.client.0.smithi038.stdout:8/97: dread - d0/d1d/d20/f23 zero size 2024-03-24T00:48:38.773 INFO:tasks.workunit.client.0.smithi038.stdout:8/98: fsync d0/f9 0 2024-03-24T00:48:38.784 INFO:tasks.workunit.client.0.smithi038.stdout:7/71: rmdir d2/d9/da/dc/d18 0 2024-03-24T00:48:38.791 INFO:tasks.workunit.client.0.smithi038.stdout:8/99: mknod d0/c28 0 2024-03-24T00:48:38.793 INFO:tasks.workunit.client.0.smithi038.stdout:8/100: mkdir d0/d10/d19/d29 0 2024-03-24T00:48:38.794 INFO:tasks.workunit.client.0.smithi038.stdout:8/101: mknod d0/d10/d19/d29/c2a 0 2024-03-24T00:48:38.796 INFO:tasks.workunit.client.0.smithi038.stdout:8/102: link d0/d10/d19/c1f d0/d10/d19/c2b 0 2024-03-24T00:48:38.796 INFO:tasks.workunit.client.0.smithi038.stdout:8/103: write d0/d1d/d20/f23 [342087,6582] 0 2024-03-24T00:48:38.797 INFO:tasks.workunit.client.0.smithi038.stdout:8/104: mkdir d0/d15/d2c 0 2024-03-24T00:48:38.798 INFO:tasks.workunit.client.0.smithi038.stdout:8/105: readlink d0/d10/d19/l1c 0 2024-03-24T00:48:38.800 INFO:tasks.workunit.client.0.smithi038.stdout:8/106: unlink d0/d15/f1a 0 2024-03-24T00:48:38.810 INFO:tasks.workunit.client.1.smithi088.stdout:1/81: write f0 [98470,125722] 0 2024-03-24T00:48:38.811 INFO:tasks.workunit.client.1.smithi088.stdout:1/82: chown l5 108236758 1 2024-03-24T00:48:38.811 INFO:tasks.workunit.client.1.smithi088.stdout:1/83: write d6/f10 [473852,71706] 0 2024-03-24T00:48:38.811 INFO:tasks.workunit.client.1.smithi088.stdout:1/84: stat d6/f10 0 2024-03-24T00:48:38.811 INFO:tasks.workunit.client.1.smithi088.stdout:1/85: truncate d6/f10 1416593 0 2024-03-24T00:48:38.811 INFO:tasks.workunit.client.0.smithi038.stdout:5/87: dwrite d6/fa [4194304,4194304] 0 2024-03-24T00:48:38.811 INFO:tasks.workunit.client.0.smithi038.stdout:5/88: read f5 [223999,122106] 0 2024-03-24T00:48:38.813 INFO:tasks.workunit.client.0.smithi038.stdout:5/89: creat d6/d19/f1c x:0 0 0 2024-03-24T00:48:38.813 INFO:tasks.workunit.client.0.smithi038.stdout:9/128: dwrite f6 [0,4194304] 0 2024-03-24T00:48:38.813 INFO:tasks.workunit.client.0.smithi038.stdout:5/90: stat d6/d19/f1c 0 2024-03-24T00:48:38.814 INFO:tasks.workunit.client.1.smithi088.stdout:3/72: dwrite d3/d6/fb [0,4194304] 0 2024-03-24T00:48:38.814 INFO:tasks.workunit.client.1.smithi088.stdout:3/73: chown d3/fc 49 1 2024-03-24T00:48:38.815 INFO:tasks.workunit.client.1.smithi088.stdout:1/86: creat d6/f13 x:0 0 0 2024-03-24T00:48:38.815 INFO:tasks.workunit.client.1.smithi088.stdout:1/87: chown d6/la 12351861 1 2024-03-24T00:48:38.815 INFO:tasks.workunit.client.1.smithi088.stdout:1/88: read - d6/f13 zero size 2024-03-24T00:48:38.815 INFO:tasks.workunit.client.1.smithi088.stdout:1/89: chown d6/lc 11 1 2024-03-24T00:48:38.817 INFO:tasks.workunit.client.0.smithi038.stdout:5/91: unlink d6/f11 0 2024-03-24T00:48:38.817 INFO:tasks.workunit.client.0.smithi038.stdout:7/72: write d2/d9/da/dc/fd [1503030,22818] 0 2024-03-24T00:48:38.821 INFO:tasks.workunit.client.0.smithi038.stdout:7/73: creat d2/f1a x:0 0 0 2024-03-24T00:48:38.821 INFO:tasks.workunit.client.0.smithi038.stdout:4/103: dwrite da/f1c [0,4194304] 0 2024-03-24T00:48:38.821 INFO:tasks.workunit.client.0.smithi038.stdout:9/129: unlink d12/d1a/l29 0 2024-03-24T00:48:38.821 INFO:tasks.workunit.client.0.smithi038.stdout:9/130: fsync fb 0 2024-03-24T00:48:38.826 INFO:tasks.workunit.client.1.smithi088.stdout:3/74: rename d3/fc to d3/de/df/f15 0 2024-03-24T00:48:38.827 INFO:tasks.workunit.client.1.smithi088.stdout:0/83: dwrite f3 [0,4194304] 0 2024-03-24T00:48:38.827 INFO:tasks.workunit.client.1.smithi088.stdout:0/84: dread - d9/dc/de/d19/f1a zero size 2024-03-24T00:48:38.827 INFO:tasks.workunit.client.1.smithi088.stdout:0/85: dread - d9/dc/de/f17 zero size 2024-03-24T00:48:38.827 INFO:tasks.workunit.client.1.smithi088.stdout:0/86: chown c6 359 1 2024-03-24T00:48:38.829 INFO:tasks.workunit.client.1.smithi088.stdout:1/90: mknod d6/c14 0 2024-03-24T00:48:38.829 INFO:tasks.workunit.client.0.smithi038.stdout:0/88: dwrite d0/f7 [0,4194304] 0 2024-03-24T00:48:38.833 INFO:tasks.workunit.client.0.smithi038.stdout:0/89: rename d0/d1b/f1d to d0/d1b/f20 0 2024-03-24T00:48:38.833 INFO:tasks.workunit.client.0.smithi038.stdout:0/90: stat d0/d12/d13/c1f 0 2024-03-24T00:48:38.833 INFO:tasks.workunit.client.0.smithi038.stdout:0/91: truncate d0/d12/d15/f17 196769 0 2024-03-24T00:48:38.835 INFO:tasks.workunit.client.0.smithi038.stdout:2/128: dwrite d4/d18/d26/f2c [0,4194304] 0 2024-03-24T00:48:38.837 INFO:tasks.workunit.client.0.smithi038.stdout:0/92: rmdir d0/d1b 39 2024-03-24T00:48:38.845 INFO:tasks.workunit.client.0.smithi038.stdout:8/107: rmdir d0 39 2024-03-24T00:48:38.846 INFO:tasks.workunit.client.1.smithi088.stdout:3/75: write f2 [6377981,85682] 0 2024-03-24T00:48:38.846 INFO:tasks.workunit.client.1.smithi088.stdout:3/76: fsync d3/d6/fb 0 2024-03-24T00:48:38.846 INFO:tasks.workunit.client.0.smithi038.stdout:5/92: write d6/fa [3685820,54751] 0 2024-03-24T00:48:38.851 INFO:tasks.workunit.client.0.smithi038.stdout:7/74: write d2/f4 [255023,125161] 0 2024-03-24T00:48:38.852 INFO:tasks.workunit.client.0.smithi038.stdout:7/75: write d2/f1a [887995,13319] 0 2024-03-24T00:48:38.852 INFO:tasks.workunit.client.0.smithi038.stdout:7/76: readlink l1 0 2024-03-24T00:48:38.852 INFO:tasks.workunit.client.0.smithi038.stdout:7/77: chown d2/d9 2767699 1 2024-03-24T00:48:38.852 INFO:tasks.workunit.client.0.smithi038.stdout:7/78: write d2/f13 [1646918,36611] 0 2024-03-24T00:48:38.852 INFO:tasks.workunit.client.0.smithi038.stdout:7/79: chown d2/d9/da/dc/c17 1815 1 2024-03-24T00:48:38.856 INFO:tasks.workunit.client.1.smithi088.stdout:0/87: creat d9/d15/f1b x:0 0 0 2024-03-24T00:48:38.861 INFO:tasks.workunit.client.1.smithi088.stdout:1/91: symlink d6/l15 0 2024-03-24T00:48:38.869 INFO:tasks.workunit.client.1.smithi088.stdout:1/92: link d6/c14 d6/c16 0 2024-03-24T00:48:38.869 INFO:tasks.workunit.client.1.smithi088.stdout:1/93: fsync d6/f8 0 2024-03-24T00:48:38.869 INFO:tasks.workunit.client.1.smithi088.stdout:1/94: chown d6/fd 1 1 2024-03-24T00:48:38.870 INFO:tasks.workunit.client.1.smithi088.stdout:1/95: fsync d6/f10 0 2024-03-24T00:48:38.870 INFO:tasks.workunit.client.1.smithi088.stdout:1/96: fdatasync d6/fd 0 2024-03-24T00:48:38.870 INFO:tasks.workunit.client.1.smithi088.stdout:1/97: fsync f2 0 2024-03-24T00:48:38.872 INFO:tasks.workunit.client.1.smithi088.stdout:4/126: sync 2024-03-24T00:48:38.872 INFO:tasks.workunit.client.1.smithi088.stdout:6/123: sync 2024-03-24T00:48:38.872 INFO:tasks.workunit.client.1.smithi088.stdout:5/98: sync 2024-03-24T00:48:38.872 INFO:tasks.workunit.client.1.smithi088.stdout:8/111: sync 2024-03-24T00:48:38.872 INFO:tasks.workunit.client.1.smithi088.stdout:7/74: sync 2024-03-24T00:48:38.873 INFO:tasks.workunit.client.1.smithi088.stdout:2/80: sync 2024-03-24T00:48:38.873 INFO:tasks.workunit.client.1.smithi088.stdout:9/119: sync 2024-03-24T00:48:38.873 INFO:tasks.workunit.client.1.smithi088.stdout:4/127: stat d5/d16 0 2024-03-24T00:48:38.873 INFO:tasks.workunit.client.1.smithi088.stdout:6/124: readlink db/dd/l14 0 2024-03-24T00:48:38.873 INFO:tasks.workunit.client.1.smithi088.stdout:7/75: fdatasync f1 0 2024-03-24T00:48:38.873 INFO:tasks.workunit.client.1.smithi088.stdout:2/81: fsync db/fe 0 2024-03-24T00:48:38.873 INFO:tasks.workunit.client.1.smithi088.stdout:1/98: rename d6/fd to d6/f17 0 2024-03-24T00:48:38.874 INFO:tasks.workunit.client.1.smithi088.stdout:9/120: mkdir d1/d15/d21 0 2024-03-24T00:48:38.876 INFO:tasks.workunit.client.1.smithi088.stdout:7/76: mknod d3/c18 0 2024-03-24T00:48:38.876 INFO:tasks.workunit.client.0.smithi038.stdout:2/129: dread - d4/f15 zero size 2024-03-24T00:48:38.878 INFO:tasks.workunit.client.0.smithi038.stdout:0/93: symlink d0/d12/d13/l21 0 2024-03-24T00:48:38.880 INFO:tasks.workunit.client.1.smithi088.stdout:1/99: mkdir d6/d18 0 2024-03-24T00:48:38.880 INFO:tasks.workunit.client.1.smithi088.stdout:1/100: creat d6/f19 x:0 0 0 2024-03-24T00:48:38.884 INFO:tasks.workunit.client.0.smithi038.stdout:8/108: symlink d0/d15/l2d 0 2024-03-24T00:48:38.884 INFO:tasks.workunit.client.0.smithi038.stdout:5/93: rename d6/d16/d14/l12 to d6/d16/d14/l1d 0 2024-03-24T00:48:38.884 INFO:tasks.workunit.client.0.smithi038.stdout:2/130: chown d4/f1c 1189599410 1 2024-03-24T00:48:38.884 INFO:tasks.workunit.client.0.smithi038.stdout:5/94: write f5 [1874522,14885] 0 2024-03-24T00:48:38.884 INFO:tasks.workunit.client.0.smithi038.stdout:2/131: write d4/f13 [303241,49726] 0 2024-03-24T00:48:38.884 INFO:tasks.workunit.client.0.smithi038.stdout:2/132: fsync d4/d18/d26/f2c 0 2024-03-24T00:48:38.889 INFO:tasks.workunit.client.1.smithi088.stdout:7/77: symlink d3/l19 0 2024-03-24T00:48:38.890 INFO:tasks.workunit.client.1.smithi088.stdout:0/88: write f3 [3039662,41201] 0 2024-03-24T00:48:38.890 INFO:tasks.workunit.client.1.smithi088.stdout:0/89: dread - d9/dc/de/d19/f1a zero size 2024-03-24T00:48:38.890 INFO:tasks.workunit.client.1.smithi088.stdout:0/90: creat d9/f1c x:0 0 0 2024-03-24T00:48:38.890 INFO:tasks.workunit.client.1.smithi088.stdout:0/91: creat d9/dc/de/f1d x:0 0 0 2024-03-24T00:48:38.892 INFO:tasks.workunit.client.0.smithi038.stdout:2/133: rmdir d4/d18/d26 39 2024-03-24T00:48:38.892 INFO:tasks.workunit.client.0.smithi038.stdout:2/134: dread - d4/d18/d26/f29 zero size 2024-03-24T00:48:38.901 INFO:tasks.workunit.client.0.smithi038.stdout:8/109: write d0/f16 [1226176,103292] 0 2024-03-24T00:48:38.902 INFO:tasks.workunit.client.0.smithi038.stdout:9/131: dwrite fc [0,4194304] 0 2024-03-24T00:48:38.904 INFO:tasks.workunit.client.1.smithi088.stdout:7/78: mkdir d3/d1a 0 2024-03-24T00:48:38.907 INFO:tasks.workunit.client.0.smithi038.stdout:3/105: sync 2024-03-24T00:48:38.907 INFO:tasks.workunit.client.0.smithi038.stdout:6/132: sync 2024-03-24T00:48:38.907 INFO:tasks.workunit.client.0.smithi038.stdout:1/63: sync 2024-03-24T00:48:38.910 INFO:tasks.workunit.client.0.smithi038.stdout:8/110: mknod d0/d1d/d27/c2e 0 2024-03-24T00:48:38.913 INFO:tasks.workunit.client.1.smithi088.stdout:4/128: dread f2 [4194304,4194304] 0 2024-03-24T00:48:38.913 INFO:tasks.workunit.client.1.smithi088.stdout:4/129: creat d5/d13/d18/f25 x:0 0 0 2024-03-24T00:48:38.914 INFO:tasks.workunit.client.1.smithi088.stdout:4/130: chown d5/d17/f1c 78132 1 2024-03-24T00:48:38.916 INFO:tasks.workunit.client.0.smithi038.stdout:2/135: dread f2 [0,4194304] 0 2024-03-24T00:48:38.916 INFO:tasks.workunit.client.0.smithi038.stdout:9/132: creat d12/d1f/f2b x:0 0 0 2024-03-24T00:48:38.917 INFO:tasks.workunit.client.1.smithi088.stdout:0/92: symlink d9/d15/l1e 0 2024-03-24T00:48:38.928 INFO:tasks.workunit.client.0.smithi038.stdout:4/104: dwrite f8 [0,4194304] 0 2024-03-24T00:48:38.928 INFO:tasks.workunit.client.0.smithi038.stdout:4/105: creat da/f1e x:0 0 0 2024-03-24T00:48:38.928 INFO:tasks.workunit.client.0.smithi038.stdout:4/106: fdatasync da/f1d 0 2024-03-24T00:48:38.931 INFO:tasks.workunit.client.0.smithi038.stdout:3/106: symlink d3/d9/dc/df/l24 0 2024-03-24T00:48:38.931 INFO:tasks.workunit.client.0.smithi038.stdout:6/133: mknod d1/da/d1b/d1d/c27 0 2024-03-24T00:48:38.931 INFO:tasks.workunit.client.0.smithi038.stdout:6/134: chown d1/da/d1b/d25 94298 1 2024-03-24T00:48:38.931 INFO:tasks.workunit.client.0.smithi038.stdout:3/107: write d3/d9/fb [505100,114470] 0 2024-03-24T00:48:38.940 INFO:tasks.workunit.client.1.smithi088.stdout:6/125: dwrite db/d1d/f21 [0,4194304] 0 2024-03-24T00:48:38.944 INFO:tasks.workunit.client.1.smithi088.stdout:4/131: mknod d5/d13/d18/d24/c26 0 2024-03-24T00:48:38.944 INFO:tasks.workunit.client.0.smithi038.stdout:2/136: dread d4/f13 [0,4194304] 0 2024-03-24T00:48:38.946 INFO:tasks.workunit.client.1.smithi088.stdout:5/99: dwrite d1/fa [0,4194304] 0 2024-03-24T00:48:38.947 INFO:tasks.workunit.client.1.smithi088.stdout:0/93: rename d9/dc/de/f17 to d9/dc/de/d19/f1f 0 2024-03-24T00:48:38.948 INFO:tasks.workunit.client.1.smithi088.stdout:0/94: chown c7 909771 1 2024-03-24T00:48:38.948 INFO:tasks.workunit.client.0.smithi038.stdout:5/95: truncate f5 1151371 0 2024-03-24T00:48:38.948 INFO:tasks.workunit.client.0.smithi038.stdout:0/94: dwrite d0/f2 [0,4194304] 0 2024-03-24T00:48:38.948 INFO:tasks.workunit.client.0.smithi038.stdout:0/95: fdatasync d0/f2 0 2024-03-24T00:48:38.948 INFO:tasks.workunit.client.0.smithi038.stdout:0/96: creat d0/d12/d15/f22 x:0 0 0 2024-03-24T00:48:38.951 INFO:tasks.workunit.client.1.smithi088.stdout:1/101: getdents d6 0 2024-03-24T00:48:38.953 INFO:tasks.workunit.client.0.smithi038.stdout:4/107: rename c4 to da/dc/c1f 0 2024-03-24T00:48:38.958 INFO:tasks.workunit.client.1.smithi088.stdout:7/79: getdents d3 0 2024-03-24T00:48:38.958 INFO:tasks.workunit.client.1.smithi088.stdout:7/80: chown d3/fa 15 1 2024-03-24T00:48:38.958 INFO:tasks.workunit.client.1.smithi088.stdout:7/81: fsync d3/f7 0 2024-03-24T00:48:38.961 INFO:tasks.workunit.client.0.smithi038.stdout:6/135: rename d1/c15 to d1/c28 0 2024-03-24T00:48:38.961 INFO:tasks.workunit.client.0.smithi038.stdout:6/136: write d1/f1f [1015473,6385] 0 2024-03-24T00:48:38.961 INFO:tasks.workunit.client.0.smithi038.stdout:6/137: stat d1 0 2024-03-24T00:48:38.962 INFO:tasks.workunit.client.0.smithi038.stdout:6/138: write d1/f1f [1286306,94081] 0 2024-03-24T00:48:38.964 INFO:tasks.workunit.client.1.smithi088.stdout:4/132: mknod d5/d17/c27 0 2024-03-24T00:48:38.964 INFO:tasks.workunit.client.1.smithi088.stdout:4/133: stat d5/d13/d18/f25 0 2024-03-24T00:48:38.965 INFO:tasks.workunit.client.1.smithi088.stdout:0/95: fsync d9/dc/de/f1d 0 2024-03-24T00:48:38.965 INFO:tasks.workunit.client.0.smithi038.stdout:8/111: dwrite d0/d1d/d20/f23 [0,4194304] 0 2024-03-24T00:48:38.972 INFO:tasks.workunit.client.0.smithi038.stdout:1/64: dwrite da/f11 [0,4194304] 0 2024-03-24T00:48:38.973 INFO:tasks.workunit.client.1.smithi088.stdout:1/102: creat d6/d18/f1a x:0 0 0 2024-03-24T00:48:38.974 INFO:tasks.workunit.client.0.smithi038.stdout:9/133: dwrite f4 [0,4194304] 0 2024-03-24T00:48:38.977 INFO:tasks.workunit.client.1.smithi088.stdout:7/82: dread d3/f14 [0,4194304] 0 2024-03-24T00:48:38.977 INFO:tasks.workunit.client.0.smithi038.stdout:0/97: mkdir d0/d12/d15/d23 0 2024-03-24T00:48:38.981 INFO:tasks.workunit.client.1.smithi088.stdout:4/134: symlink d5/d16/d1a/d22/l28 0 2024-03-24T00:48:38.986 INFO:tasks.workunit.client.0.smithi038.stdout:4/108: link l5 da/dc/dd/l20 0 2024-03-24T00:48:38.986 INFO:tasks.workunit.client.1.smithi088.stdout:0/96: mkdir d9/d20 0 2024-03-24T00:48:38.988 INFO:tasks.workunit.client.0.smithi038.stdout:6/139: symlink d1/da/l29 0 2024-03-24T00:48:38.989 INFO:tasks.workunit.client.1.smithi088.stdout:1/103: dread f2 [0,4194304] 0 2024-03-24T00:48:38.989 INFO:tasks.workunit.client.1.smithi088.stdout:1/104: chown d6/f13 103232931 1 2024-03-24T00:48:38.991 INFO:tasks.workunit.client.1.smithi088.stdout:6/126: dwrite db/dd/f10 [4194304,4194304] 0 2024-03-24T00:48:38.992 INFO:tasks.workunit.client.1.smithi088.stdout:7/83: rmdir d3/d1a 0 2024-03-24T00:48:38.992 INFO:tasks.workunit.client.1.smithi088.stdout:7/84: chown d3/ff 113 1 2024-03-24T00:48:38.994 INFO:tasks.workunit.client.1.smithi088.stdout:4/135: creat d5/d16/f29 x:0 0 0 2024-03-24T00:48:38.994 INFO:tasks.workunit.client.1.smithi088.stdout:4/136: readlink d5/d16/l20 0 2024-03-24T00:48:38.994 INFO:tasks.workunit.client.1.smithi088.stdout:4/137: creat d5/d17/f2a x:0 0 0 2024-03-24T00:48:38.999 INFO:tasks.workunit.client.0.smithi038.stdout:8/112: rename d0/c28 to d0/d10/d19/d29/c2f 0 2024-03-24T00:48:39.004 INFO:tasks.workunit.client.1.smithi088.stdout:0/97: dread f3 [0,4194304] 0 2024-03-24T00:48:39.005 INFO:tasks.workunit.client.0.smithi038.stdout:2/137: mknod d4/c32 0 2024-03-24T00:48:39.005 INFO:tasks.workunit.client.0.smithi038.stdout:2/138: readlink d4/d18/l25 0 2024-03-24T00:48:39.008 INFO:tasks.workunit.client.1.smithi088.stdout:6/127: rmdir db 39 2024-03-24T00:48:39.009 INFO:tasks.workunit.client.0.smithi038.stdout:9/134: symlink d12/d1f/d26/l2c 0 2024-03-24T00:48:39.018 INFO:tasks.workunit.client.1.smithi088.stdout:7/85: rename f1 to d3/f1b 0 2024-03-24T00:48:39.020 INFO:tasks.workunit.client.0.smithi038.stdout:2/139: rename d4/fd to d4/d27/f33 0 2024-03-24T00:48:39.032 INFO:tasks.workunit.client.0.smithi038.stdout:9/135: truncate f8 2673724 0 2024-03-24T00:48:39.034 INFO:tasks.workunit.client.1.smithi088.stdout:4/138: mknod d5/d13/d18/c2b 0 2024-03-24T00:48:39.034 INFO:tasks.workunit.client.1.smithi088.stdout:4/139: dread - d5/d13/f12 zero size 2024-03-24T00:48:39.034 INFO:tasks.workunit.client.0.smithi038.stdout:3/108: write d3/f8 [4556025,42868] 0 2024-03-24T00:48:39.035 INFO:tasks.workunit.client.0.smithi038.stdout:5/96: dwrite d6/d16/f17 [4194304,4194304] 0 2024-03-24T00:48:39.036 INFO:tasks.workunit.client.1.smithi088.stdout:0/98: stat c7 0 2024-03-24T00:48:39.036 INFO:tasks.workunit.client.1.smithi088.stdout:0/99: write d9/dc/de/d19/f1a [282105,26062] 0 2024-03-24T00:48:39.038 INFO:tasks.workunit.client.1.smithi088.stdout:6/128: symlink db/dd/l25 0 2024-03-24T00:48:39.038 INFO:tasks.workunit.client.0.smithi038.stdout:4/109: dread da/f1c [4194304,4194304] 0 2024-03-24T00:48:39.038 INFO:tasks.workunit.client.0.smithi038.stdout:4/110: write da/f1e [529842,124586] 0 2024-03-24T00:48:39.039 INFO:tasks.workunit.client.0.smithi038.stdout:4/111: creat da/dc/dd/f21 x:0 0 0 2024-03-24T00:48:39.039 INFO:tasks.workunit.client.0.smithi038.stdout:2/140: symlink d4/d27/d2f/d30/l34 0 2024-03-24T00:48:39.039 INFO:tasks.workunit.client.0.smithi038.stdout:2/141: fdatasync d4/f10 0 2024-03-24T00:48:39.041 INFO:tasks.workunit.client.1.smithi088.stdout:7/86: symlink d3/l1c 0 2024-03-24T00:48:39.044 INFO:tasks.workunit.client.1.smithi088.stdout:1/105: dwrite d6/f10 [0,4194304] 0 2024-03-24T00:48:39.044 INFO:tasks.workunit.client.1.smithi088.stdout:0/100: symlink d9/dc/l21 0 2024-03-24T00:48:39.044 INFO:tasks.workunit.client.0.smithi038.stdout:6/140: rmdir d1/da 39 2024-03-24T00:48:39.044 INFO:tasks.workunit.client.0.smithi038.stdout:6/141: chown d1/da/d1b 4 1 2024-03-24T00:48:39.048 INFO:tasks.workunit.client.1.smithi088.stdout:6/129: rmdir db/d1f 39 2024-03-24T00:48:39.049 INFO:tasks.workunit.client.1.smithi088.stdout:6/130: write db/dd/f1e [2297206,53207] 0 2024-03-24T00:48:39.050 INFO:tasks.workunit.client.0.smithi038.stdout:9/136: mknod d12/c2d 0 2024-03-24T00:48:39.051 INFO:tasks.workunit.client.1.smithi088.stdout:8/112: sync 2024-03-24T00:48:39.052 INFO:tasks.workunit.client.1.smithi088.stdout:3/77: sync 2024-03-24T00:48:39.052 INFO:tasks.workunit.client.1.smithi088.stdout:2/82: sync 2024-03-24T00:48:39.052 INFO:tasks.workunit.client.1.smithi088.stdout:3/78: stat d3/c8 0 2024-03-24T00:48:39.052 INFO:tasks.workunit.client.1.smithi088.stdout:8/113: write d9/f10 [581346,42033] 0 2024-03-24T00:48:39.052 INFO:tasks.workunit.client.1.smithi088.stdout:7/87: rmdir d3 39 2024-03-24T00:48:39.058 INFO:tasks.workunit.client.0.smithi038.stdout:3/109: chown d3/ce 38100088 1 2024-03-24T00:48:39.060 INFO:tasks.workunit.client.1.smithi088.stdout:6/131: dread db/f13 [0,4194304] 0 2024-03-24T00:48:39.060 INFO:tasks.workunit.client.1.smithi088.stdout:6/132: write f8 [912880,81011] 0 2024-03-24T00:48:39.061 INFO:tasks.workunit.client.0.smithi038.stdout:1/65: dwrite da/fb [0,4194304] 0 2024-03-24T00:48:39.061 INFO:tasks.workunit.client.0.smithi038.stdout:0/98: dwrite d0/d1b/f20 [0,4194304] 0 2024-03-24T00:48:39.065 INFO:tasks.workunit.client.0.smithi038.stdout:6/142: creat d1/da/d1b/d25/f2a x:0 0 0 2024-03-24T00:48:39.065 INFO:tasks.workunit.client.0.smithi038.stdout:6/143: read d1/f12 [435090,3089] 0 2024-03-24T00:48:39.067 INFO:tasks.workunit.client.1.smithi088.stdout:1/106: link d6/c16 d6/d18/c1b 0 2024-03-24T00:48:39.069 INFO:tasks.workunit.client.1.smithi088.stdout:0/101: mknod d9/d20/c22 0 2024-03-24T00:48:39.072 INFO:tasks.workunit.client.0.smithi038.stdout:5/97: mkdir d6/d1e 0 2024-03-24T00:48:39.073 INFO:tasks.workunit.client.1.smithi088.stdout:3/79: rmdir d3/d12 39 2024-03-24T00:48:39.073 INFO:tasks.workunit.client.1.smithi088.stdout:3/80: chown f2 101 1 2024-03-24T00:48:39.074 INFO:tasks.workunit.client.0.smithi038.stdout:1/66: mknod da/dd/d10/c12 0 2024-03-24T00:48:39.076 INFO:tasks.workunit.client.1.smithi088.stdout:7/88: creat d3/f1d x:0 0 0 2024-03-24T00:48:39.078 INFO:tasks.workunit.client.1.smithi088.stdout:6/133: mknod db/d1d/c26 0 2024-03-24T00:48:39.079 INFO:tasks.workunit.client.1.smithi088.stdout:1/107: mknod d6/c1c 0 2024-03-24T00:48:39.084 INFO:tasks.workunit.client.1.smithi088.stdout:0/102: symlink d9/d15/l23 0 2024-03-24T00:48:39.084 INFO:tasks.workunit.client.1.smithi088.stdout:0/103: fsync d9/dc/de/f14 0 2024-03-24T00:48:39.086 INFO:tasks.workunit.client.1.smithi088.stdout:3/81: creat d3/de/f16 x:0 0 0 2024-03-24T00:48:39.087 INFO:tasks.workunit.client.1.smithi088.stdout:6/134: rename db/dd/l18 to db/l27 0 2024-03-24T00:48:39.087 INFO:tasks.workunit.client.1.smithi088.stdout:6/135: truncate db/f13 1485199 0 2024-03-24T00:48:39.088 INFO:tasks.workunit.client.0.smithi038.stdout:4/112: read da/dc/dd/d15/f17 [963986,120958] 0 2024-03-24T00:48:39.088 INFO:tasks.workunit.client.0.smithi038.stdout:4/113: creat da/dc/f22 x:0 0 0 2024-03-24T00:48:39.088 INFO:tasks.workunit.client.0.smithi038.stdout:4/114: chown da/d12 84982791 1 2024-03-24T00:48:39.088 INFO:tasks.workunit.client.0.smithi038.stdout:4/115: write da/f18 [668923,81002] 0 2024-03-24T00:48:39.090 INFO:tasks.workunit.client.1.smithi088.stdout:0/104: mknod d9/dc/de/d19/c24 0 2024-03-24T00:48:39.091 INFO:tasks.workunit.client.0.smithi038.stdout:3/110: rmdir d3/d9 39 2024-03-24T00:48:39.091 INFO:tasks.workunit.client.1.smithi088.stdout:2/83: dread db/dc/f11 [0,4194304] 0 2024-03-24T00:48:39.095 INFO:tasks.workunit.client.1.smithi088.stdout:2/84: creat db/f18 x:0 0 0 2024-03-24T00:48:39.095 INFO:tasks.workunit.client.1.smithi088.stdout:3/82: rmdir d3/d6 39 2024-03-24T00:48:39.097 INFO:tasks.workunit.client.0.smithi038.stdout:6/144: creat d1/da/d1b/f2b x:0 0 0 2024-03-24T00:48:39.099 INFO:tasks.workunit.client.1.smithi088.stdout:6/136: creat db/d1f/f28 x:0 0 0 2024-03-24T00:48:39.102 INFO:tasks.workunit.client.0.smithi038.stdout:1/67: link f7 da/df/f13 0 2024-03-24T00:48:39.107 INFO:tasks.workunit.client.1.smithi088.stdout:0/105: symlink d9/d20/l25 0 2024-03-24T00:48:39.111 INFO:tasks.workunit.client.1.smithi088.stdout:6/137: link db/cf db/c29 0 2024-03-24T00:48:39.113 INFO:tasks.workunit.client.1.smithi088.stdout:9/121: sync 2024-03-24T00:48:39.113 INFO:tasks.workunit.client.1.smithi088.stdout:5/100: sync 2024-03-24T00:48:39.113 INFO:tasks.workunit.client.1.smithi088.stdout:5/101: creat d1/dd/f23 x:0 0 0 2024-03-24T00:48:39.115 INFO:tasks.workunit.client.1.smithi088.stdout:0/106: unlink c8 0 2024-03-24T00:48:39.115 INFO:tasks.workunit.client.1.smithi088.stdout:0/107: write d9/d15/f1b [536759,33040] 0 2024-03-24T00:48:39.121 INFO:tasks.workunit.client.1.smithi088.stdout:5/102: mknod d1/d10/c24 0 2024-03-24T00:48:39.121 INFO:tasks.workunit.client.1.smithi088.stdout:5/103: fdatasync d1/d10/f17 0 2024-03-24T00:48:39.122 INFO:tasks.workunit.client.0.smithi038.stdout:3/111: creat d3/f25 x:0 0 0 2024-03-24T00:48:39.126 INFO:tasks.workunit.client.1.smithi088.stdout:0/108: mkdir d9/d26 0 2024-03-24T00:48:39.129 INFO:tasks.workunit.client.1.smithi088.stdout:0/109: symlink d9/d26/l27 0 2024-03-24T00:48:39.129 INFO:tasks.workunit.client.1.smithi088.stdout:0/110: truncate d9/dc/de/d19/f1f 338342 0 2024-03-24T00:48:39.129 INFO:tasks.workunit.client.0.smithi038.stdout:1/68: mkdir da/dd/d10/d14 0 2024-03-24T00:48:39.130 INFO:tasks.workunit.client.0.smithi038.stdout:3/112: creat d3/d9/f26 x:0 0 0 2024-03-24T00:48:39.131 INFO:tasks.workunit.client.1.smithi088.stdout:0/111: mknod d9/dc/de/c28 0 2024-03-24T00:48:39.132 INFO:tasks.workunit.client.0.smithi038.stdout:2/142: dwrite f2 [0,4194304] 0 2024-03-24T00:48:39.134 INFO:tasks.workunit.client.0.smithi038.stdout:1/69: link da/df/f13 da/dd/d10/f15 0 2024-03-24T00:48:39.134 INFO:tasks.workunit.client.0.smithi038.stdout:1/70: creat da/f16 x:0 0 0 2024-03-24T00:48:39.137 INFO:tasks.workunit.client.1.smithi088.stdout:0/112: truncate f3 2426895 0 2024-03-24T00:48:39.143 INFO:tasks.workunit.client.0.smithi038.stdout:3/113: rename f2 to d3/d9/dc/df/f27 0 2024-03-24T00:48:39.144 INFO:tasks.workunit.client.0.smithi038.stdout:3/114: chown d3/d23/c22 1 1 2024-03-24T00:48:39.148 INFO:tasks.workunit.client.1.smithi088.stdout:9/122: read d1/d10/f18 [118652,102865] 0 2024-03-24T00:48:39.150 INFO:tasks.workunit.client.1.smithi088.stdout:0/113: write d9/dc/de/f11 [4534026,100615] 0 2024-03-24T00:48:39.150 INFO:tasks.workunit.client.1.smithi088.stdout:0/114: write d9/dc/de/d19/f1f [1008832,3261] 0 2024-03-24T00:48:39.151 INFO:tasks.workunit.client.1.smithi088.stdout:0/115: readlink d9/d20/l25 0 2024-03-24T00:48:39.151 INFO:tasks.workunit.client.1.smithi088.stdout:0/116: stat d9/d15/f1b 0 2024-03-24T00:48:39.151 INFO:tasks.workunit.client.1.smithi088.stdout:7/89: dwrite d3/f15 [0,4194304] 0 2024-03-24T00:48:39.151 INFO:tasks.workunit.client.1.smithi088.stdout:7/90: write d3/f1d [835984,31531] 0 2024-03-24T00:48:39.151 INFO:tasks.workunit.client.0.smithi038.stdout:1/71: creat da/df/f17 x:0 0 0 2024-03-24T00:48:39.151 INFO:tasks.workunit.client.0.smithi038.stdout:1/72: chown f8 417895 1 2024-03-24T00:48:39.151 INFO:tasks.workunit.client.0.smithi038.stdout:1/73: chown l6 199564 1 2024-03-24T00:48:39.154 INFO:tasks.workunit.client.0.smithi038.stdout:3/115: creat d3/d9/dc/df/d14/d19/f28 x:0 0 0 2024-03-24T00:48:39.157 INFO:tasks.workunit.client.0.smithi038.stdout:8/113: dwrite d0/f16 [0,4194304] 0 2024-03-24T00:48:39.157 INFO:tasks.workunit.client.0.smithi038.stdout:8/114: chown d0/c5 15414174 1 2024-03-24T00:48:39.157 INFO:tasks.workunit.client.1.smithi088.stdout:8/114: dwrite d9/d1f/f21 [0,4194304] 0 2024-03-24T00:48:39.158 INFO:tasks.workunit.client.1.smithi088.stdout:1/108: dwrite d6/ff [0,4194304] 0 2024-03-24T00:48:39.159 INFO:tasks.workunit.client.0.smithi038.stdout:6/145: rmdir d1 39 2024-03-24T00:48:39.159 INFO:tasks.workunit.client.0.smithi038.stdout:6/146: chown d1/f10 2234 1 2024-03-24T00:48:39.159 INFO:tasks.workunit.client.0.smithi038.stdout:3/116: creat d3/f29 x:0 0 0 2024-03-24T00:48:39.170 INFO:tasks.workunit.client.0.smithi038.stdout:8/115: rmdir d0/d10 39 2024-03-24T00:48:39.173 INFO:tasks.workunit.client.1.smithi088.stdout:9/123: rmdir d1/d10 39 2024-03-24T00:48:39.173 INFO:tasks.workunit.client.1.smithi088.stdout:9/124: dread - d1/d10/f1e zero size 2024-03-24T00:48:39.175 INFO:tasks.workunit.client.1.smithi088.stdout:7/91: dread d3/f5 [0,4194304] 0 2024-03-24T00:48:39.175 INFO:tasks.workunit.client.1.smithi088.stdout:7/92: chown d3/f14 0 1 2024-03-24T00:48:39.176 INFO:tasks.workunit.client.0.smithi038.stdout:3/117: rename d3/d23 to d3/d2a 0 2024-03-24T00:48:39.177 INFO:tasks.workunit.client.0.smithi038.stdout:3/118: creat d3/d9/f2b x:0 0 0 2024-03-24T00:48:39.183 INFO:tasks.workunit.client.1.smithi088.stdout:8/115: rename d9/d1f/f21 to d9/dc/d11/d24/f2a 0 2024-03-24T00:48:39.183 INFO:tasks.workunit.client.1.smithi088.stdout:2/85: dwrite db/dd/f17 [0,4194304] 0 2024-03-24T00:48:39.184 INFO:tasks.workunit.client.1.smithi088.stdout:2/86: write db/dc/f11 [622995,96614] 0 2024-03-24T00:48:39.184 INFO:tasks.workunit.client.1.smithi088.stdout:3/83: dwrite d3/d6/f11 [0,4194304] 0 2024-03-24T00:48:39.186 INFO:tasks.workunit.client.0.smithi038.stdout:0/99: dwrite d0/d12/d15/d1a/f1e [0,4194304] 0 2024-03-24T00:48:39.190 INFO:tasks.workunit.client.0.smithi038.stdout:4/116: dwrite da/f16 [4194304,4194304] 0 2024-03-24T00:48:39.190 INFO:tasks.workunit.client.0.smithi038.stdout:3/119: mknod d3/d9/dc/c2c 0 2024-03-24T00:48:39.191 INFO:tasks.workunit.client.1.smithi088.stdout:5/104: getdents d1/dd 0 2024-03-24T00:48:39.193 INFO:tasks.workunit.client.1.smithi088.stdout:9/125: truncate d1/f6 209305 0 2024-03-24T00:48:39.193 INFO:tasks.workunit.client.1.smithi088.stdout:9/126: dread - d1/f20 zero size 2024-03-24T00:48:39.195 INFO:tasks.workunit.client.0.smithi038.stdout:8/116: write d0/f16 [1349810,103710] 0 2024-03-24T00:48:39.195 INFO:tasks.workunit.client.0.smithi038.stdout:8/117: write d0/d10/d19/f1b [421612,125161] 0 2024-03-24T00:48:39.197 INFO:tasks.workunit.client.1.smithi088.stdout:6/138: dread f8 [0,4194304] 0 2024-03-24T00:48:39.200 INFO:tasks.workunit.client.0.smithi038.stdout:4/117: rename da/f18 to da/dc/dd/d15/f23 0 2024-03-24T00:48:39.200 INFO:tasks.workunit.client.0.smithi038.stdout:4/118: readlink l5 0 2024-03-24T00:48:39.209 INFO:tasks.workunit.client.0.smithi038.stdout:5/98: dwrite d6/f18 [0,4194304] 0 2024-03-24T00:48:39.210 INFO:tasks.workunit.client.1.smithi088.stdout:8/116: read d9/f17 [3392152,56857] 0 2024-03-24T00:48:39.210 INFO:tasks.workunit.client.1.smithi088.stdout:6/139: dread f8 [0,4194304] 0 2024-03-24T00:48:39.210 INFO:tasks.workunit.client.1.smithi088.stdout:7/93: read d3/f5 [2982430,4885] 0 2024-03-24T00:48:39.210 INFO:tasks.workunit.client.1.smithi088.stdout:3/84: unlink d3/l14 0 2024-03-24T00:48:39.210 INFO:tasks.workunit.client.1.smithi088.stdout:5/105: mknod d1/d10/d1b/c25 0 2024-03-24T00:48:39.210 INFO:tasks.workunit.client.1.smithi088.stdout:9/127: mkdir d1/d22 0 2024-03-24T00:48:39.213 INFO:tasks.workunit.client.0.smithi038.stdout:8/118: mknod d0/d10/d19/d29/c30 0 2024-03-24T00:48:39.215 INFO:tasks.workunit.client.1.smithi088.stdout:6/140: creat db/d1d/d23/f2a x:0 0 0 2024-03-24T00:48:39.215 INFO:tasks.workunit.client.1.smithi088.stdout:6/141: dread - db/d1d/d23/f2a zero size 2024-03-24T00:48:39.217 INFO:tasks.workunit.client.0.smithi038.stdout:4/119: mkdir da/d12/d24 0 2024-03-24T00:48:39.218 INFO:tasks.workunit.client.1.smithi088.stdout:7/94: symlink d3/l1e 0 2024-03-24T00:48:39.224 INFO:tasks.workunit.client.1.smithi088.stdout:3/85: rename d3/fa to d3/de/f17 0 2024-03-24T00:48:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:6/147: dwrite d1/f12 [0,4194304] 0 2024-03-24T00:48:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:5/99: rmdir d6/d16 39 2024-03-24T00:48:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:3/120: rmdir d3/d2a 39 2024-03-24T00:48:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:0/100: write d0/ff [8023702,6628] 0 2024-03-24T00:48:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:3/121: write d3/d9/dc/df/f12 [744942,12671] 0 2024-03-24T00:48:39.235 INFO:tasks.workunit.client.1.smithi088.stdout:0/117: dwrite d9/dc/de/f1d [0,4194304] 0 2024-03-24T00:48:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:2/143: dwrite d4/d18/f22 [0,4194304] 0 2024-03-24T00:48:39.245 INFO:tasks.workunit.client.0.smithi038.stdout:6/148: creat d1/da/d1b/d1d/f2c x:0 0 0 2024-03-24T00:48:39.246 INFO:tasks.workunit.client.0.smithi038.stdout:6/149: stat d1/da/d1b/f2b 0 2024-03-24T00:48:39.248 INFO:tasks.workunit.client.0.smithi038.stdout:5/100: creat d6/d1e/f1f x:0 0 0 2024-03-24T00:48:39.249 INFO:tasks.workunit.client.1.smithi088.stdout:1/109: dwrite d6/f8 [0,4194304] 0 2024-03-24T00:48:39.253 INFO:tasks.workunit.client.0.smithi038.stdout:3/122: rmdir d3/d2a 39 2024-03-24T00:48:39.255 INFO:tasks.workunit.client.0.smithi038.stdout:1/74: dwrite da/dd/d10/f15 [0,4194304] 0 2024-03-24T00:48:39.255 INFO:tasks.workunit.client.0.smithi038.stdout:1/75: chown da/dd 4 1 2024-03-24T00:48:39.255 INFO:tasks.workunit.client.0.smithi038.stdout:1/76: rename da/dd/d10 to da/dd/d10/d18 22 2024-03-24T00:48:39.260 INFO:tasks.workunit.client.0.smithi038.stdout:5/101: mkdir d6/d19/d20 0 2024-03-24T00:48:39.263 INFO:tasks.workunit.client.1.smithi088.stdout:8/117: dwrite d9/f28 [0,4194304] 0 2024-03-24T00:48:39.266 INFO:tasks.workunit.client.1.smithi088.stdout:7/95: link d3/fc d3/f1f 0 2024-03-24T00:48:39.267 INFO:tasks.workunit.client.1.smithi088.stdout:2/87: dwrite db/f18 [0,4194304] 0 2024-03-24T00:48:39.267 INFO:tasks.workunit.client.1.smithi088.stdout:2/88: truncate db/fe 589942 0 2024-03-24T00:48:39.267 INFO:tasks.workunit.client.1.smithi088.stdout:2/89: readlink db/dd/l15 0 2024-03-24T00:48:39.268 INFO:tasks.workunit.client.1.smithi088.stdout:3/86: mknod d3/de/df/c18 0 2024-03-24T00:48:39.280 INFO:tasks.workunit.client.1.smithi088.stdout:8/118: dread d9/f17 [0,4194304] 0 2024-03-24T00:48:39.281 INFO:tasks.workunit.client.1.smithi088.stdout:0/118: mkdir d9/d26/d29 0 2024-03-24T00:48:39.285 INFO:tasks.workunit.client.0.smithi038.stdout:3/123: mknod d3/d9/dc/df/d14/c2d 0 2024-03-24T00:48:39.286 INFO:tasks.workunit.client.0.smithi038.stdout:3/124: truncate d3/d9/dc/df/f27 1363381 0 2024-03-24T00:48:39.286 INFO:tasks.workunit.client.0.smithi038.stdout:3/125: fdatasync d3/d9/f1c 0 2024-03-24T00:48:39.287 INFO:tasks.workunit.client.0.smithi038.stdout:1/77: link da/dd/d10/c12 da/df/c19 0 2024-03-24T00:48:39.290 INFO:tasks.workunit.client.1.smithi088.stdout:7/96: write d3/fc [1631018,39095] 0 2024-03-24T00:48:39.290 INFO:tasks.workunit.client.1.smithi088.stdout:1/110: truncate f2 3554077 0 2024-03-24T00:48:39.290 INFO:tasks.workunit.client.1.smithi088.stdout:1/111: creat d6/d18/f1d x:0 0 0 2024-03-24T00:48:39.295 INFO:tasks.workunit.client.0.smithi038.stdout:3/126: mknod d3/d9/dc/c2e 0 2024-03-24T00:48:39.296 INFO:tasks.workunit.client.1.smithi088.stdout:6/142: rename db/d1d/d23/f2a to db/dd/f2b 0 2024-03-24T00:48:39.299 INFO:tasks.workunit.client.0.smithi038.stdout:8/119: dwrite d0/f9 [0,4194304] 0 2024-03-24T00:48:39.299 INFO:tasks.workunit.client.0.smithi038.stdout:8/120: chown d0/cf 3712 1 2024-03-24T00:48:39.299 INFO:tasks.workunit.client.0.smithi038.stdout:8/121: dread - d0/d1d/f22 zero size 2024-03-24T00:48:39.300 INFO:tasks.workunit.client.0.smithi038.stdout:3/127: mknod d3/d2a/c2f 0 2024-03-24T00:48:39.300 INFO:tasks.workunit.client.0.smithi038.stdout:3/128: dread - d3/d9/f26 zero size 2024-03-24T00:48:39.301 INFO:tasks.workunit.client.1.smithi088.stdout:2/90: mknod db/dc/c19 0 2024-03-24T00:48:39.302 INFO:tasks.workunit.client.1.smithi088.stdout:3/87: mkdir d3/d6/d19 0 2024-03-24T00:48:39.302 INFO:tasks.workunit.client.0.smithi038.stdout:8/122: symlink d0/d1d/l31 0 2024-03-24T00:48:39.308 INFO:tasks.workunit.client.1.smithi088.stdout:5/106: dwrite d1/d10/f17 [0,4194304] 0 2024-03-24T00:48:39.310 INFO:tasks.workunit.client.0.smithi038.stdout:1/78: write f1 [3712349,102164] 0 2024-03-24T00:48:39.311 INFO:tasks.workunit.client.0.smithi038.stdout:3/129: chown d3/f6 26232425 1 2024-03-24T00:48:39.311 INFO:tasks.workunit.client.0.smithi038.stdout:3/130: stat d3/d9/dc/df/f1d 0 2024-03-24T00:48:39.313 INFO:tasks.workunit.client.1.smithi088.stdout:8/119: getdents d9/dc/d11 0 2024-03-24T00:48:39.321 INFO:tasks.workunit.client.1.smithi088.stdout:1/112: mkdir d6/d18/d1e 0 2024-03-24T00:48:39.323 INFO:tasks.workunit.client.1.smithi088.stdout:2/91: link c7 db/dc/c1a 0 2024-03-24T00:48:39.329 INFO:tasks.workunit.client.0.smithi038.stdout:3/131: mkdir d3/d30 0 2024-03-24T00:48:39.330 INFO:tasks.workunit.client.0.smithi038.stdout:8/123: read d0/d1d/d20/f23 [470763,32873] 0 2024-03-24T00:48:39.333 INFO:tasks.workunit.client.1.smithi088.stdout:3/88: mkdir d3/d1a 0 2024-03-24T00:48:39.333 INFO:tasks.workunit.client.1.smithi088.stdout:3/89: write f1 [1803708,121383] 0 2024-03-24T00:48:39.337 INFO:tasks.workunit.client.0.smithi038.stdout:5/102: dwrite d6/d16/f17 [0,4194304] 0 2024-03-24T00:48:39.338 INFO:tasks.workunit.client.0.smithi038.stdout:1/79: dread f1 [0,4194304] 0 2024-03-24T00:48:39.338 INFO:tasks.workunit.client.1.smithi088.stdout:2/92: unlink db/dc/c1a 0 2024-03-24T00:48:39.338 INFO:tasks.workunit.client.1.smithi088.stdout:2/93: chown f6 492277306 1 2024-03-24T00:48:39.339 INFO:tasks.workunit.client.1.smithi088.stdout:6/143: dread db/dd/f10 [0,4194304] 0 2024-03-24T00:48:39.342 INFO:tasks.workunit.client.0.smithi038.stdout:1/80: rename l0 to da/df/l1a 0 2024-03-24T00:48:39.343 INFO:tasks.workunit.client.1.smithi088.stdout:6/144: stat db/d1d/f21 0 2024-03-24T00:48:39.343 INFO:tasks.workunit.client.1.smithi088.stdout:6/145: chown db/d1d/d23 36862 1 2024-03-24T00:48:39.343 INFO:tasks.workunit.client.1.smithi088.stdout:6/146: write db/dd/f1e [4304398,35354] 0 2024-03-24T00:48:39.343 INFO:tasks.workunit.client.1.smithi088.stdout:0/119: dwrite d9/dc/de/f14 [0,4194304] 0 2024-03-24T00:48:39.345 INFO:tasks.workunit.client.1.smithi088.stdout:1/113: read d6/f8 [208794,93754] 0 2024-03-24T00:48:39.345 INFO:tasks.workunit.client.0.smithi038.stdout:4/120: dwrite f8 [8388608,4194304] 0 2024-03-24T00:48:39.345 INFO:tasks.workunit.client.0.smithi038.stdout:4/121: readlink da/l13 0 2024-03-24T00:48:39.345 INFO:tasks.workunit.client.0.smithi038.stdout:4/122: dread - da/dc/f22 zero size 2024-03-24T00:48:39.345 INFO:tasks.workunit.client.0.smithi038.stdout:0/101: dread d0/d12/d15/f17 [0,4194304] 0 2024-03-24T00:48:39.346 INFO:tasks.workunit.client.0.smithi038.stdout:4/123: write da/f1d [995917,117106] 0 2024-03-24T00:48:39.347 INFO:tasks.workunit.client.1.smithi088.stdout:7/97: dwrite d3/f5 [0,4194304] 0 2024-03-24T00:48:39.347 INFO:tasks.workunit.client.1.smithi088.stdout:2/94: creat db/dc/f1b x:0 0 0 2024-03-24T00:48:39.349 INFO:tasks.workunit.client.1.smithi088.stdout:6/147: dread db/f1c [0,4194304] 0 2024-03-24T00:48:39.350 INFO:tasks.workunit.client.1.smithi088.stdout:6/148: write db/d1f/f28 [1006326,96616] 0 2024-03-24T00:48:39.350 INFO:tasks.workunit.client.1.smithi088.stdout:6/149: dread - db/f19 zero size 2024-03-24T00:48:39.350 INFO:tasks.workunit.client.1.smithi088.stdout:6/150: chown db/l17 107 1 2024-03-24T00:48:39.350 INFO:tasks.workunit.client.0.smithi038.stdout:0/102: mknod d0/d12/d15/c24 0 2024-03-24T00:48:39.354 INFO:tasks.workunit.client.0.smithi038.stdout:4/124: mknod da/d12/c25 0 2024-03-24T00:48:39.355 INFO:tasks.workunit.client.0.smithi038.stdout:7/80: sync 2024-03-24T00:48:39.355 INFO:tasks.workunit.client.0.smithi038.stdout:7/81: chown d2/f13 26 1 2024-03-24T00:48:39.355 INFO:tasks.workunit.client.0.smithi038.stdout:7/82: dread - d2/d9/d16/f19 zero size 2024-03-24T00:48:39.363 INFO:tasks.workunit.client.1.smithi088.stdout:0/120: rename d9/d26/d29 to d9/d20/d2a 0 2024-03-24T00:48:39.365 INFO:tasks.workunit.client.1.smithi088.stdout:4/140: sync 2024-03-24T00:48:39.369 INFO:tasks.workunit.client.1.smithi088.stdout:3/90: dread f1 [0,4194304] 0 2024-03-24T00:48:39.372 INFO:tasks.workunit.client.1.smithi088.stdout:1/114: creat d6/d18/d1e/f1f x:0 0 0 2024-03-24T00:48:39.372 INFO:tasks.workunit.client.1.smithi088.stdout:1/115: write d6/f19 [426727,85644] 0 2024-03-24T00:48:39.374 INFO:tasks.workunit.client.0.smithi038.stdout:3/132: getdents d3/d9/dc 0 2024-03-24T00:48:39.374 INFO:tasks.workunit.client.0.smithi038.stdout:1/81: write f4 [1714453,81940] 0 2024-03-24T00:48:39.374 INFO:tasks.workunit.client.0.smithi038.stdout:0/103: mknod d0/d12/d15/c25 0 2024-03-24T00:48:39.376 INFO:tasks.workunit.client.0.smithi038.stdout:7/83: mknod d2/d9/df/c1b 0 2024-03-24T00:48:39.394 INFO:tasks.workunit.client.0.smithi038.stdout:1/82: rmdir da/dd 39 2024-03-24T00:48:39.399 INFO:tasks.workunit.client.0.smithi038.stdout:3/133: link d3/d9/fb d3/d9/dc/df/f31 0 2024-03-24T00:48:39.399 INFO:tasks.workunit.client.0.smithi038.stdout:3/134: write d3/d9/fb [3860608,115327] 0 2024-03-24T00:48:39.399 INFO:tasks.workunit.client.1.smithi088.stdout:5/107: dwrite d1/dd/f23 [0,4194304] 0 2024-03-24T00:48:39.402 INFO:tasks.workunit.client.0.smithi038.stdout:2/144: dwrite d4/d18/f2e [0,4194304] 0 2024-03-24T00:48:39.402 INFO:tasks.workunit.client.1.smithi088.stdout:2/95: rename db/dd/l15 to db/dc/l1c 0 2024-03-24T00:48:39.403 INFO:tasks.workunit.client.1.smithi088.stdout:3/91: symlink d3/d6/l1b 0 2024-03-24T00:48:39.409 INFO:tasks.workunit.client.0.smithi038.stdout:0/104: dread d0/f2 [0,4194304] 0 2024-03-24T00:48:39.409 INFO:tasks.workunit.client.0.smithi038.stdout:0/105: chown d0/fc 900 1 2024-03-24T00:48:39.409 INFO:tasks.workunit.client.1.smithi088.stdout:6/151: getdents db/dd 0 2024-03-24T00:48:39.412 INFO:tasks.workunit.client.0.smithi038.stdout:2/145: mknod d4/d18/d26/c35 0 2024-03-24T00:48:39.413 INFO:tasks.workunit.client.0.smithi038.stdout:5/103: dwrite d6/d19/f1c [0,4194304] 0 2024-03-24T00:48:39.417 INFO:tasks.workunit.client.0.smithi038.stdout:7/84: rmdir d2/d9/da 39 2024-03-24T00:48:39.418 INFO:tasks.workunit.client.1.smithi088.stdout:5/108: mkdir d1/d19/d1e/d26 0 2024-03-24T00:48:39.418 INFO:tasks.workunit.client.1.smithi088.stdout:5/109: read d1/ff [2475246,96194] 0 2024-03-24T00:48:39.420 INFO:tasks.workunit.client.1.smithi088.stdout:3/92: mkdir d3/d1a/d1c 0 2024-03-24T00:48:39.420 INFO:tasks.workunit.client.1.smithi088.stdout:2/96: read db/dd/f17 [594855,118204] 0 2024-03-24T00:48:39.420 INFO:tasks.workunit.client.1.smithi088.stdout:2/97: write db/dc/f1b [720670,64310] 0 2024-03-24T00:48:39.422 INFO:tasks.workunit.client.0.smithi038.stdout:0/106: rmdir d0/d12/d15/d23 0 2024-03-24T00:48:39.422 INFO:tasks.workunit.client.0.smithi038.stdout:0/107: chown d0/le 1 1 2024-03-24T00:48:39.422 INFO:tasks.workunit.client.1.smithi088.stdout:7/98: dwrite d3/f5 [0,4194304] 0 2024-03-24T00:48:39.422 INFO:tasks.workunit.client.1.smithi088.stdout:7/99: read d3/fc [2321172,90272] 0 2024-03-24T00:48:39.423 INFO:tasks.workunit.client.1.smithi088.stdout:1/116: rmdir d6/d18/d1e 39 2024-03-24T00:48:39.431 INFO:tasks.workunit.client.1.smithi088.stdout:6/152: rename db/cf to db/c2c 0 2024-03-24T00:48:39.432 INFO:tasks.workunit.client.1.smithi088.stdout:6/153: fsync db/f19 0 2024-03-24T00:48:39.432 INFO:tasks.workunit.client.0.smithi038.stdout:1/83: dread f4 [0,4194304] 0 2024-03-24T00:48:39.432 INFO:tasks.workunit.client.0.smithi038.stdout:1/84: fsync f5 0 2024-03-24T00:48:39.433 INFO:tasks.workunit.client.0.smithi038.stdout:1/85: write f4 [3493066,111787] 0 2024-03-24T00:48:39.433 INFO:tasks.workunit.client.0.smithi038.stdout:3/135: dread d3/f5 [0,4194304] 0 2024-03-24T00:48:39.436 INFO:tasks.workunit.client.1.smithi088.stdout:3/93: truncate f1 614009 0 2024-03-24T00:48:39.441 INFO:tasks.workunit.client.1.smithi088.stdout:9/128: sync 2024-03-24T00:48:39.443 INFO:tasks.workunit.client.1.smithi088.stdout:2/98: rename db/dc/c19 to db/c1d 0 2024-03-24T00:48:39.446 INFO:tasks.workunit.client.0.smithi038.stdout:5/104: getdents d6/d16/d14 0 2024-03-24T00:48:39.449 INFO:tasks.workunit.client.0.smithi038.stdout:7/85: creat d2/d9/da/dc/f1c x:0 0 0 2024-03-24T00:48:39.450 INFO:tasks.workunit.client.0.smithi038.stdout:2/146: dread d4/f7 [0,4194304] 0 2024-03-24T00:48:39.450 INFO:tasks.workunit.client.0.smithi038.stdout:1/86: write da/fb [2480056,14386] 0 2024-03-24T00:48:39.451 INFO:tasks.workunit.client.1.smithi088.stdout:7/100: link d3/f1b d3/f20 0 2024-03-24T00:48:39.452 INFO:tasks.workunit.client.0.smithi038.stdout:4/125: dwrite da/f16 [4194304,4194304] 0 2024-03-24T00:48:39.453 INFO:tasks.workunit.client.0.smithi038.stdout:0/108: mknod d0/d12/d15/d1a/c26 0 2024-03-24T00:48:39.456 INFO:tasks.workunit.client.0.smithi038.stdout:3/136: creat d3/f32 x:0 0 0 2024-03-24T00:48:39.456 INFO:tasks.workunit.client.1.smithi088.stdout:1/117: symlink d6/d18/l20 0 2024-03-24T00:48:39.457 INFO:tasks.workunit.client.1.smithi088.stdout:1/118: write d6/d18/d1e/f1f [964846,119908] 0 2024-03-24T00:48:39.457 INFO:tasks.workunit.client.1.smithi088.stdout:1/119: chown d6/d18/c1b 301094205 1 2024-03-24T00:48:39.460 INFO:tasks.workunit.client.0.smithi038.stdout:7/86: creat d2/d9/da/f1d x:0 0 0 2024-03-24T00:48:39.460 INFO:tasks.workunit.client.0.smithi038.stdout:7/87: stat d2/d9/da/f14 0 2024-03-24T00:48:39.462 INFO:tasks.workunit.client.1.smithi088.stdout:6/154: truncate db/dd/f10 4226272 0 2024-03-24T00:48:39.463 INFO:tasks.workunit.client.0.smithi038.stdout:2/147: mknod d4/d18/d26/c36 0 2024-03-24T00:48:39.464 INFO:tasks.workunit.client.0.smithi038.stdout:5/105: read d6/f18 [3775381,37384] 0 2024-03-24T00:48:39.469 INFO:tasks.workunit.client.1.smithi088.stdout:5/110: dwrite d1/d10/f17 [0,4194304] 0 2024-03-24T00:48:39.471 INFO:tasks.workunit.client.0.smithi038.stdout:6/150: dwrite d1/da/d1b/d1d/f2c [0,4194304] 0 2024-03-24T00:48:39.471 INFO:tasks.workunit.client.0.smithi038.stdout:7/88: dread d2/d9/da/dc/fd [4194304,4194304] 0 2024-03-24T00:48:39.471 INFO:tasks.workunit.client.0.smithi038.stdout:7/89: stat d2/d9/df 0 2024-03-24T00:48:39.477 INFO:tasks.workunit.client.1.smithi088.stdout:9/129: mkdir d1/d10/d11/d23 0 2024-03-24T00:48:39.477 INFO:tasks.workunit.client.1.smithi088.stdout:9/130: chown d1/d10/d1f 5547 1 2024-03-24T00:48:39.478 INFO:tasks.workunit.client.1.smithi088.stdout:9/131: fsync d1/f4 0 2024-03-24T00:48:39.478 INFO:tasks.workunit.client.1.smithi088.stdout:9/132: read d1/d10/f18 [684416,123655] 0 2024-03-24T00:48:39.478 INFO:tasks.workunit.client.1.smithi088.stdout:9/133: write d1/f8 [1496124,24435] 0 2024-03-24T00:48:39.486 INFO:tasks.workunit.client.1.smithi088.stdout:7/101: link d3/f5 d3/f21 0 2024-03-24T00:48:39.486 INFO:tasks.workunit.client.1.smithi088.stdout:7/102: truncate d3/f17 663556 0 2024-03-24T00:48:39.488 INFO:tasks.workunit.client.0.smithi038.stdout:4/126: creat da/d12/d24/f26 x:0 0 0 2024-03-24T00:48:39.490 INFO:tasks.workunit.client.0.smithi038.stdout:0/109: mknod d0/d12/d15/c27 0 2024-03-24T00:48:39.493 INFO:tasks.workunit.client.1.smithi088.stdout:7/103: dread d3/fd [0,4194304] 0 2024-03-24T00:48:39.493 INFO:tasks.workunit.client.1.smithi088.stdout:7/104: stat d3/f1b 0 2024-03-24T00:48:39.495 INFO:tasks.workunit.client.0.smithi038.stdout:2/148: mkdir d4/d18/d26/d37 0 2024-03-24T00:48:39.498 INFO:tasks.workunit.client.0.smithi038.stdout:8/124: dwrite d0/d10/d19/f1b [0,4194304] 0 2024-03-24T00:48:39.499 INFO:tasks.workunit.client.1.smithi088.stdout:2/99: dwrite f6 [0,4194304] 0 2024-03-24T00:48:39.500 INFO:tasks.workunit.client.0.smithi038.stdout:5/106: mkdir d6/d16/d21 0 2024-03-24T00:48:39.500 INFO:tasks.workunit.client.0.smithi038.stdout:5/107: stat d6/d1e 0 2024-03-24T00:48:39.500 INFO:tasks.workunit.client.1.smithi088.stdout:5/111: link d1/l9 d1/d10/d1b/l27 0 2024-03-24T00:48:39.512 INFO:tasks.workunit.client.1.smithi088.stdout:9/134: write d1/f5 [3375500,90068] 0 2024-03-24T00:48:39.512 INFO:tasks.workunit.client.1.smithi088.stdout:3/94: write f1 [324899,49266] 0 2024-03-24T00:48:39.514 INFO:tasks.workunit.client.1.smithi088.stdout:6/155: dwrite db/f13 [0,4194304] 0 2024-03-24T00:48:39.517 INFO:tasks.workunit.client.1.smithi088.stdout:1/120: dwrite d6/f10 [0,4194304] 0 2024-03-24T00:48:39.517 INFO:tasks.workunit.client.1.smithi088.stdout:1/121: dread - d6/f13 zero size 2024-03-24T00:48:39.517 INFO:tasks.workunit.client.1.smithi088.stdout:1/122: write d6/d18/f1a [528448,57998] 0 2024-03-24T00:48:39.517 INFO:tasks.workunit.client.1.smithi088.stdout:7/105: unlink d3/f4 0 2024-03-24T00:48:39.518 INFO:tasks.workunit.client.1.smithi088.stdout:2/100: symlink db/dd/l1e 0 2024-03-24T00:48:39.518 INFO:tasks.workunit.client.1.smithi088.stdout:2/101: read db/dd/f17 [3287837,87401] 0 2024-03-24T00:48:39.518 INFO:tasks.workunit.client.1.smithi088.stdout:8/120: sync 2024-03-24T00:48:39.519 INFO:tasks.workunit.client.1.smithi088.stdout:5/112: unlink d1/f14 0 2024-03-24T00:48:39.525 INFO:tasks.workunit.client.1.smithi088.stdout:7/106: dread d3/f7 [0,4194304] 0 2024-03-24T00:48:39.528 INFO:tasks.workunit.client.1.smithi088.stdout:6/156: mknod db/d1f/c2d 0 2024-03-24T00:48:39.528 INFO:tasks.workunit.client.1.smithi088.stdout:0/121: sync 2024-03-24T00:48:39.536 INFO:tasks.workunit.client.1.smithi088.stdout:1/123: creat d6/d18/d1e/f21 x:0 0 0 2024-03-24T00:48:39.536 INFO:tasks.workunit.client.1.smithi088.stdout:2/102: dread f1 [0,4194304] 0 2024-03-24T00:48:39.536 INFO:tasks.workunit.client.1.smithi088.stdout:1/124: creat d6/d18/d1e/f22 x:0 0 0 2024-03-24T00:48:39.536 INFO:tasks.workunit.client.1.smithi088.stdout:1/125: fsync d6/f10 0 2024-03-24T00:48:39.537 INFO:tasks.workunit.client.1.smithi088.stdout:1/126: chown d6/d18/f1d 51829404 1 2024-03-24T00:48:39.537 INFO:tasks.workunit.client.1.smithi088.stdout:1/127: truncate d6/d18/d1e/f21 972076 0 2024-03-24T00:48:39.537 INFO:tasks.workunit.client.0.smithi038.stdout:7/90: symlink d2/d9/d16/l1e 0 2024-03-24T00:48:39.537 INFO:tasks.workunit.client.0.smithi038.stdout:7/91: creat d2/f1f x:0 0 0 2024-03-24T00:48:39.538 INFO:tasks.workunit.client.0.smithi038.stdout:1/87: getdents da 0 2024-03-24T00:48:39.538 INFO:tasks.workunit.client.1.smithi088.stdout:5/113: write d1/d10/f17 [1202282,42472] 0 2024-03-24T00:48:39.540 INFO:tasks.workunit.client.0.smithi038.stdout:4/127: mkdir da/d27 0 2024-03-24T00:48:39.540 INFO:tasks.workunit.client.0.smithi038.stdout:4/128: creat da/f28 x:0 0 0 2024-03-24T00:48:39.541 INFO:tasks.workunit.client.0.smithi038.stdout:4/129: fsync f8 0 2024-03-24T00:48:39.541 INFO:tasks.workunit.client.0.smithi038.stdout:9/137: sync 2024-03-24T00:48:39.543 INFO:tasks.workunit.client.0.smithi038.stdout:0/110: stat d0/le 0 2024-03-24T00:48:39.544 INFO:tasks.workunit.client.1.smithi088.stdout:7/107: symlink d3/l22 0 2024-03-24T00:48:39.544 INFO:tasks.workunit.client.1.smithi088.stdout:7/108: creat d3/f23 x:0 0 0 2024-03-24T00:48:39.545 INFO:tasks.workunit.client.0.smithi038.stdout:2/149: mkdir d4/d18/d38 0 2024-03-24T00:48:39.546 INFO:tasks.workunit.client.0.smithi038.stdout:2/150: creat d4/d27/f39 x:0 0 0 2024-03-24T00:48:39.547 INFO:tasks.workunit.client.1.smithi088.stdout:4/141: sync 2024-03-24T00:48:39.552 INFO:tasks.workunit.client.0.smithi038.stdout:8/125: creat d0/f32 x:0 0 0 2024-03-24T00:48:39.552 INFO:tasks.workunit.client.0.smithi038.stdout:8/126: write d0/f32 [232762,88439] 0 2024-03-24T00:48:39.553 INFO:tasks.workunit.client.1.smithi088.stdout:6/157: unlink db/l17 0 2024-03-24T00:48:39.553 INFO:tasks.workunit.client.1.smithi088.stdout:0/122: mknod d9/c2b 0 2024-03-24T00:48:39.553 INFO:tasks.workunit.client.1.smithi088.stdout:2/103: truncate db/f18 2378200 0 2024-03-24T00:48:39.553 INFO:tasks.workunit.client.1.smithi088.stdout:1/128: dread d6/ff [0,4194304] 0 2024-03-24T00:48:39.554 INFO:tasks.workunit.client.1.smithi088.stdout:3/95: dwrite d3/de/f16 [0,4194304] 0 2024-03-24T00:48:39.555 INFO:tasks.workunit.client.0.smithi038.stdout:5/108: mknod d6/d16/d14/c22 0 2024-03-24T00:48:39.557 INFO:tasks.workunit.client.0.smithi038.stdout:6/151: rmdir d1/da/d1b/d1d 39 2024-03-24T00:48:39.559 INFO:tasks.workunit.client.1.smithi088.stdout:5/114: truncate d1/ff 4726783 0 2024-03-24T00:48:39.559 INFO:tasks.workunit.client.1.smithi088.stdout:5/115: write d1/dd/f15 [4315025,57515] 0 2024-03-24T00:48:39.561 INFO:tasks.workunit.client.1.smithi088.stdout:7/109: dread d3/f1f [0,4194304] 0 2024-03-24T00:48:39.561 INFO:tasks.workunit.client.1.smithi088.stdout:7/110: dread - d3/f23 zero size 2024-03-24T00:48:39.561 INFO:tasks.workunit.client.0.smithi038.stdout:7/92: creat d2/d9/f20 x:0 0 0 2024-03-24T00:48:39.563 INFO:tasks.workunit.client.0.smithi038.stdout:1/88: symlink da/dd/l1b 0 2024-03-24T00:48:39.563 INFO:tasks.workunit.client.0.smithi038.stdout:1/89: write da/df/f13 [4533504,23359] 0 2024-03-24T00:48:39.564 INFO:tasks.workunit.client.1.smithi088.stdout:4/142: truncate d5/d16/d1a/f1e 1317578 0 2024-03-24T00:48:39.564 INFO:tasks.workunit.client.1.smithi088.stdout:4/143: write d5/d13/f12 [280533,59993] 0 2024-03-24T00:48:39.567 INFO:tasks.workunit.client.1.smithi088.stdout:6/158: mknod db/d1f/c2e 0 2024-03-24T00:48:39.567 INFO:tasks.workunit.client.1.smithi088.stdout:6/159: stat db/dd/f2b 0 2024-03-24T00:48:39.568 INFO:tasks.workunit.client.1.smithi088.stdout:6/160: fdatasync db/d1f/f28 0 2024-03-24T00:48:39.568 INFO:tasks.workunit.client.1.smithi088.stdout:6/161: creat db/d1f/f2f x:0 0 0 2024-03-24T00:48:39.568 INFO:tasks.workunit.client.0.smithi038.stdout:4/130: mkdir da/d12/d29 0 2024-03-24T00:48:39.570 INFO:tasks.workunit.client.1.smithi088.stdout:0/123: symlink d9/l2c 0 2024-03-24T00:48:39.570 INFO:tasks.workunit.client.1.smithi088.stdout:0/124: fsync f3 0 2024-03-24T00:48:39.571 INFO:tasks.workunit.client.0.smithi038.stdout:9/138: mknod d12/d1f/d26/c2e 0 2024-03-24T00:48:39.581 INFO:tasks.workunit.client.0.smithi038.stdout:8/127: mkdir d0/d10/d33 0 2024-03-24T00:48:39.583 INFO:tasks.workunit.client.0.smithi038.stdout:6/152: truncate d1/da/f14 2202320 0 2024-03-24T00:48:39.584 INFO:tasks.workunit.client.0.smithi038.stdout:6/153: read - d1/da/d1b/f26 zero size 2024-03-24T00:48:39.586 INFO:tasks.workunit.client.1.smithi088.stdout:3/96: getdents d3/d1a 0 2024-03-24T00:48:39.590 INFO:tasks.workunit.client.1.smithi088.stdout:5/116: symlink d1/d19/d1e/l28 0 2024-03-24T00:48:39.591 INFO:tasks.workunit.client.0.smithi038.stdout:7/93: mkdir d2/d9/da/d21 0 2024-03-24T00:48:39.591 INFO:tasks.workunit.client.0.smithi038.stdout:7/94: truncate d2/d9/d16/f19 152976 0 2024-03-24T00:48:39.591 INFO:tasks.workunit.client.0.smithi038.stdout:7/95: dread - d2/f1f zero size 2024-03-24T00:48:39.592 INFO:tasks.workunit.client.1.smithi088.stdout:8/121: rmdir d9/d1f 0 2024-03-24T00:48:39.592 INFO:tasks.workunit.client.1.smithi088.stdout:8/122: creat d9/f2b x:0 0 0 2024-03-24T00:48:39.597 INFO:tasks.workunit.client.0.smithi038.stdout:9/139: dread d12/f18 [0,4194304] 0 2024-03-24T00:48:39.597 INFO:tasks.workunit.client.0.smithi038.stdout:2/151: dread d4/ff [0,4194304] 0 2024-03-24T00:48:39.598 INFO:tasks.workunit.client.0.smithi038.stdout:1/90: creat da/dd/f1c x:0 0 0 2024-03-24T00:48:39.604 INFO:tasks.workunit.client.0.smithi038.stdout:4/131: link da/dc/dd/d15/f17 da/dc/dd/f2a 0 2024-03-24T00:48:39.605 INFO:tasks.workunit.client.0.smithi038.stdout:4/132: creat da/dc/dd/f2b x:0 0 0 2024-03-24T00:48:39.606 INFO:tasks.workunit.client.0.smithi038.stdout:2/152: dread d4/f10 [0,4194304] 0 2024-03-24T00:48:39.609 INFO:tasks.workunit.client.1.smithi088.stdout:6/162: creat db/d1f/d24/f30 x:0 0 0 2024-03-24T00:48:39.609 INFO:tasks.workunit.client.0.smithi038.stdout:8/128: creat d0/d15/d2c/f34 x:0 0 0 2024-03-24T00:48:39.610 INFO:tasks.workunit.client.0.smithi038.stdout:6/154: creat d1/da/d1b/d1d/f2d x:0 0 0 2024-03-24T00:48:39.612 INFO:tasks.workunit.client.0.smithi038.stdout:7/96: truncate d2/f4 3547050 0 2024-03-24T00:48:39.612 INFO:tasks.workunit.client.0.smithi038.stdout:7/97: write d2/d9/da/f14 [849863,28759] 0 2024-03-24T00:48:39.616 INFO:tasks.workunit.client.0.smithi038.stdout:0/111: dwrite d0/d12/d15/f16 [0,4194304] 0 2024-03-24T00:48:39.619 INFO:tasks.workunit.client.1.smithi088.stdout:8/123: dread d9/dc/d11/d24/f2a [0,4194304] 0 2024-03-24T00:48:39.619 INFO:tasks.workunit.client.1.smithi088.stdout:8/124: truncate d9/f2b 933821 0 2024-03-24T00:48:39.621 INFO:tasks.workunit.client.1.smithi088.stdout:1/129: rmdir d6/d18 39 2024-03-24T00:48:39.624 INFO:tasks.workunit.client.0.smithi038.stdout:9/140: rename cf to d12/d28/c2f 0 2024-03-24T00:48:39.624 INFO:tasks.workunit.client.0.smithi038.stdout:1/91: symlink da/df/l1d 0 2024-03-24T00:48:39.624 INFO:tasks.workunit.client.0.smithi038.stdout:9/141: creat d12/d1f/f30 x:0 0 0 2024-03-24T00:48:39.624 INFO:tasks.workunit.client.0.smithi038.stdout:9/142: stat d12/d1f/d26/l27 0 2024-03-24T00:48:39.624 INFO:tasks.workunit.client.0.smithi038.stdout:9/143: stat d12/d1f/f2b 0 2024-03-24T00:48:39.624 INFO:tasks.workunit.client.0.smithi038.stdout:9/144: fdatasync f4 0 2024-03-24T00:48:39.624 INFO:tasks.workunit.client.0.smithi038.stdout:9/145: rename d12 to d12/d1a/d31 22 2024-03-24T00:48:39.625 INFO:tasks.workunit.client.1.smithi088.stdout:2/104: dwrite db/dc/f11 [8388608,4194304] 0 2024-03-24T00:48:39.625 INFO:tasks.workunit.client.1.smithi088.stdout:3/97: link d3/d6/fb d3/d1a/d1c/f1d 0 2024-03-24T00:48:39.625 INFO:tasks.workunit.client.1.smithi088.stdout:2/105: creat db/dd/f1f x:0 0 0 2024-03-24T00:48:39.630 INFO:tasks.workunit.client.1.smithi088.stdout:7/111: dwrite d3/f20 [0,4194304] 0 2024-03-24T00:48:39.630 INFO:tasks.workunit.client.1.smithi088.stdout:7/112: creat d3/f24 x:0 0 0 2024-03-24T00:48:39.632 INFO:tasks.workunit.client.1.smithi088.stdout:7/113: write d3/f17 [447365,48456] 0 2024-03-24T00:48:39.634 INFO:tasks.workunit.client.1.smithi088.stdout:8/125: dread d9/dc/d11/d1b/f1e [0,4194304] 0 2024-03-24T00:48:39.646 INFO:tasks.workunit.client.1.smithi088.stdout:5/117: mkdir d1/d29 0 2024-03-24T00:48:39.647 INFO:tasks.workunit.client.1.smithi088.stdout:5/118: fdatasync d1/d10/f17 0 2024-03-24T00:48:39.648 INFO:tasks.workunit.client.0.smithi038.stdout:8/129: getdents d0/d1d/d27 0 2024-03-24T00:48:39.649 INFO:tasks.workunit.client.0.smithi038.stdout:5/109: dwrite d6/fa [0,4194304] 0 2024-03-24T00:48:39.653 INFO:tasks.workunit.client.0.smithi038.stdout:6/155: unlink d1/f12 0 2024-03-24T00:48:39.653 INFO:tasks.workunit.client.0.smithi038.stdout:6/156: fsync d1/da/f1a 0 2024-03-24T00:48:39.653 INFO:tasks.workunit.client.0.smithi038.stdout:6/157: fsync d1/f13 0 2024-03-24T00:48:39.653 INFO:tasks.workunit.client.0.smithi038.stdout:6/158: readlink d1/da/lc 0 2024-03-24T00:48:39.654 INFO:tasks.workunit.client.0.smithi038.stdout:6/159: write d1/f13 [994879,26949] 0 2024-03-24T00:48:39.657 INFO:tasks.workunit.client.1.smithi088.stdout:7/114: write d3/f20 [912558,41697] 0 2024-03-24T00:48:39.662 INFO:tasks.workunit.client.1.smithi088.stdout:4/144: rmdir d5/d7 39 2024-03-24T00:48:39.662 INFO:tasks.workunit.client.1.smithi088.stdout:4/145: creat d5/d13/d18/f2c x:0 0 0 2024-03-24T00:48:39.664 INFO:tasks.workunit.client.1.smithi088.stdout:6/163: unlink db/l11 0 2024-03-24T00:48:39.664 INFO:tasks.workunit.client.1.smithi088.stdout:6/164: creat db/d1f/d24/f31 x:0 0 0 2024-03-24T00:48:39.666 INFO:tasks.workunit.client.1.smithi088.stdout:6/165: truncate f8 1514510 0 2024-03-24T00:48:39.667 INFO:tasks.workunit.client.1.smithi088.stdout:0/125: rmdir d9/dc/de/d19 39 2024-03-24T00:48:39.669 INFO:tasks.workunit.client.0.smithi038.stdout:0/112: mkdir d0/d12/d13/d28 0 2024-03-24T00:48:39.670 INFO:tasks.workunit.client.1.smithi088.stdout:1/130: unlink d6/c16 0 2024-03-24T00:48:39.675 INFO:tasks.workunit.client.0.smithi038.stdout:1/92: dread f8 [0,4194304] 0 2024-03-24T00:48:39.675 INFO:tasks.workunit.client.0.smithi038.stdout:1/93: fdatasync da/df/f13 0 2024-03-24T00:48:39.679 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:39 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:48:39.682 INFO:tasks.workunit.client.0.smithi038.stdout:8/130: rename d0/d10/d33 to d0/d1d/d35 0 2024-03-24T00:48:39.682 INFO:tasks.workunit.client.0.smithi038.stdout:8/131: readlink d0/d10/l11 0 2024-03-24T00:48:39.682 INFO:tasks.workunit.client.0.smithi038.stdout:8/132: chown d0/d10 914012156 1 2024-03-24T00:48:39.685 INFO:tasks.workunit.client.0.smithi038.stdout:0/113: getdents d0/d12 0 2024-03-24T00:48:39.694 INFO:tasks.workunit.client.0.smithi038.stdout:1/94: link l6 da/dd/l1e 0 2024-03-24T00:48:39.697 INFO:tasks.workunit.client.0.smithi038.stdout:1/95: write da/dd/d10/f15 [4976580,113054] 0 2024-03-24T00:48:39.701 INFO:tasks.workunit.client.0.smithi038.stdout:5/110: mkdir d6/d23 0 2024-03-24T00:48:39.701 INFO:tasks.workunit.client.0.smithi038.stdout:4/133: dwrite da/f1d [0,4194304] 0 2024-03-24T00:48:39.708 INFO:tasks.workunit.client.1.smithi088.stdout:5/119: rename d1/l4 to d1/dd/l2a 0 2024-03-24T00:48:39.709 INFO:tasks.workunit.client.0.smithi038.stdout:0/114: symlink d0/l29 0 2024-03-24T00:48:39.712 INFO:tasks.workunit.client.0.smithi038.stdout:1/96: creat da/dd/d10/f1f x:0 0 0 2024-03-24T00:48:39.719 INFO:tasks.workunit.client.1.smithi088.stdout:2/106: dwrite f1 [0,4194304] 0 2024-03-24T00:48:39.720 INFO:tasks.workunit.client.0.smithi038.stdout:4/134: unlink da/f16 0 2024-03-24T00:48:39.722 INFO:tasks.workunit.client.0.smithi038.stdout:1/97: mknod da/df/c20 0 2024-03-24T00:48:39.722 INFO:tasks.workunit.client.0.smithi038.stdout:1/98: truncate da/f16 204142 0 2024-03-24T00:48:39.724 INFO:tasks.workunit.client.1.smithi088.stdout:8/126: dwrite d9/f20 [0,4194304] 0 2024-03-24T00:48:39.724 INFO:tasks.workunit.client.1.smithi088.stdout:8/127: fdatasync d9/f10 0 2024-03-24T00:48:39.724 INFO:tasks.workunit.client.1.smithi088.stdout:4/146: symlink d5/l2d 0 2024-03-24T00:48:39.725 INFO:tasks.workunit.client.1.smithi088.stdout:4/147: dread - d5/d17/f1d zero size 2024-03-24T00:48:39.725 INFO:tasks.workunit.client.1.smithi088.stdout:4/148: creat d5/d16/f2e x:0 0 0 2024-03-24T00:48:39.725 INFO:tasks.workunit.client.1.smithi088.stdout:0/126: symlink d9/dc/de/l2d 0 2024-03-24T00:48:39.726 INFO:tasks.workunit.client.0.smithi038.stdout:1/99: creat da/f21 x:0 0 0 2024-03-24T00:48:39.726 INFO:tasks.workunit.client.0.smithi038.stdout:1/100: stat f4 0 2024-03-24T00:48:39.728 INFO:tasks.workunit.client.1.smithi088.stdout:1/131: mkdir d6/d23 0 2024-03-24T00:48:39.731 INFO:tasks.workunit.client.0.smithi038.stdout:9/146: dwrite d12/d1f/f2b [0,4194304] 0 2024-03-24T00:48:39.734 INFO:tasks.workunit.client.1.smithi088.stdout:5/120: mknod d1/d10/d1b/c2b 0 2024-03-24T00:48:39.734 INFO:tasks.workunit.client.0.smithi038.stdout:7/98: dread d2/f4 [0,4194304] 0 2024-03-24T00:48:39.735 INFO:tasks.workunit.client.0.smithi038.stdout:7/99: chown d2/d9/da/c15 1312387 1 2024-03-24T00:48:39.738 INFO:tasks.workunit.client.0.smithi038.stdout:5/111: dread d6/d19/f1c [0,4194304] 0 2024-03-24T00:48:39.743 INFO:tasks.workunit.client.0.smithi038.stdout:0/115: write d0/f5 [1929372,46735] 0 2024-03-24T00:48:39.743 INFO:tasks.workunit.client.1.smithi088.stdout:6/166: dwrite db/f13 [0,4194304] 0 2024-03-24T00:48:39.743 INFO:tasks.workunit.client.1.smithi088.stdout:6/167: chown db/dd/f10 11167353 1 2024-03-24T00:48:39.745 INFO:tasks.workunit.client.0.smithi038.stdout:7/100: symlink d2/d9/l22 0 2024-03-24T00:48:39.745 INFO:tasks.workunit.client.0.smithi038.stdout:7/101: read - d2/d9/da/dc/f1c zero size 2024-03-24T00:48:39.746 INFO:tasks.workunit.client.1.smithi088.stdout:2/107: link db/f18 db/f20 0 2024-03-24T00:48:39.748 INFO:tasks.workunit.client.0.smithi038.stdout:6/160: dwrite d1/da/fb [0,4194304] 0 2024-03-24T00:48:39.748 INFO:tasks.workunit.client.0.smithi038.stdout:6/161: readlink d1/da/lf 0 2024-03-24T00:48:39.748 INFO:tasks.workunit.client.0.smithi038.stdout:6/162: fdatasync d1/da/d1b/d25/f2a 0 2024-03-24T00:48:39.748 INFO:tasks.workunit.client.0.smithi038.stdout:5/112: unlink d6/d16/f17 0 2024-03-24T00:48:39.751 INFO:tasks.workunit.client.0.smithi038.stdout:1/101: dread f8 [0,4194304] 0 2024-03-24T00:48:39.751 INFO:tasks.workunit.client.0.smithi038.stdout:1/102: chown da/dd/d10/d14 25828600 1 2024-03-24T00:48:39.751 INFO:tasks.workunit.client.0.smithi038.stdout:8/133: dwrite d0/d15/f17 [0,4194304] 0 2024-03-24T00:48:39.756 INFO:tasks.workunit.client.1.smithi088.stdout:8/128: unlink d9/dc/d11/l1a 0 2024-03-24T00:48:39.761 INFO:tasks.workunit.client.0.smithi038.stdout:0/116: rmdir d0/d12 39 2024-03-24T00:48:39.761 INFO:tasks.workunit.client.0.smithi038.stdout:0/117: chown d0/l29 23 1 2024-03-24T00:48:39.762 INFO:tasks.workunit.client.0.smithi038.stdout:9/147: write f6 [3429799,90159] 0 2024-03-24T00:48:39.768 INFO:tasks.workunit.client.0.smithi038.stdout:7/102: mkdir d2/d9/da/d23 0 2024-03-24T00:48:39.768 INFO:tasks.workunit.client.0.smithi038.stdout:7/103: write d2/d9/da/f1d [362537,42916] 0 2024-03-24T00:48:39.772 INFO:tasks.workunit.client.1.smithi088.stdout:4/149: mkdir d5/d16/d1a/d22/d2f 0 2024-03-24T00:48:39.772 INFO:tasks.workunit.client.1.smithi088.stdout:4/150: stat d5/fc 0 2024-03-24T00:48:39.775 INFO:tasks.workunit.client.1.smithi088.stdout:0/127: mkdir d9/d20/d2a/d2e 0 2024-03-24T00:48:39.775 INFO:tasks.workunit.client.1.smithi088.stdout:0/128: stat d9/dc/de/f11 0 2024-03-24T00:48:39.775 INFO:tasks.workunit.client.1.smithi088.stdout:0/129: truncate d9/dc/de/d19/f1a 347077 0 2024-03-24T00:48:39.775 INFO:tasks.workunit.client.1.smithi088.stdout:0/130: write f3 [2802818,93670] 0 2024-03-24T00:48:39.775 INFO:tasks.workunit.client.1.smithi088.stdout:0/131: fdatasync d9/dc/de/f1d 0 2024-03-24T00:48:39.775 INFO:tasks.workunit.client.1.smithi088.stdout:0/132: write f3 [1403441,99944] 0 2024-03-24T00:48:39.775 INFO:tasks.workunit.client.1.smithi088.stdout:7/115: dwrite d3/f5 [0,4194304] 0 2024-03-24T00:48:39.777 INFO:tasks.workunit.client.0.smithi038.stdout:5/113: creat d6/d16/d14/f24 x:0 0 0 2024-03-24T00:48:39.777 INFO:tasks.workunit.client.0.smithi038.stdout:5/114: getdents d6/d23 0 2024-03-24T00:48:39.777 INFO:tasks.workunit.client.0.smithi038.stdout:5/115: chown f5 9 1 2024-03-24T00:48:39.777 INFO:tasks.workunit.client.0.smithi038.stdout:5/116: chown d6/fa 44588 1 2024-03-24T00:48:39.778 INFO:tasks.workunit.client.0.smithi038.stdout:5/117: chown d6/d23 4 1 2024-03-24T00:48:39.778 INFO:tasks.workunit.client.1.smithi088.stdout:6/168: dread db/f13 [0,4194304] 0 2024-03-24T00:48:39.778 INFO:tasks.workunit.client.1.smithi088.stdout:6/169: chown db/dd/f2b 7655889 1 2024-03-24T00:48:39.778 INFO:tasks.workunit.client.1.smithi088.stdout:6/170: chown db/dd/ce 44 1 2024-03-24T00:48:39.779 INFO:tasks.workunit.client.1.smithi088.stdout:6/171: chown db/f19 0 1 2024-03-24T00:48:39.779 INFO:tasks.workunit.client.1.smithi088.stdout:6/172: fdatasync db/f1c 0 2024-03-24T00:48:39.785 INFO:tasks.workunit.client.1.smithi088.stdout:1/132: mkdir d6/d24 0 2024-03-24T00:48:39.789 INFO:tasks.workunit.client.0.smithi038.stdout:8/134: mknod d0/d15/c36 0 2024-03-24T00:48:39.789 INFO:tasks.workunit.client.0.smithi038.stdout:8/135: chown d0/d15/c36 83 1 2024-03-24T00:48:39.796 INFO:tasks.workunit.client.1.smithi088.stdout:8/129: creat d9/dc/d11/d24/d29/f2c x:0 0 0 2024-03-24T00:48:39.803 INFO:tasks.workunit.client.1.smithi088.stdout:4/151: rename d5/d7/f11 to d5/d13/d18/d24/f30 0 2024-03-24T00:48:39.804 INFO:tasks.workunit.client.1.smithi088.stdout:4/152: getdents d5/d7/d15 0 2024-03-24T00:48:39.807 INFO:tasks.workunit.client.0.smithi038.stdout:9/148: unlink fb 0 2024-03-24T00:48:39.816 INFO:tasks.workunit.client.0.smithi038.stdout:7/104: mknod d2/d9/d16/c24 0 2024-03-24T00:48:39.819 INFO:tasks.workunit.client.0.smithi038.stdout:4/135: dwrite da/dc/dd/f2a [0,4194304] 0 2024-03-24T00:48:39.819 INFO:tasks.workunit.client.0.smithi038.stdout:6/163: dwrite d1/da/f1a [0,4194304] 0 2024-03-24T00:48:39.820 INFO:tasks.workunit.client.1.smithi088.stdout:2/108: dwrite f6 [4194304,4194304] 0 2024-03-24T00:48:39.821 INFO:tasks.workunit.client.0.smithi038.stdout:8/136: unlink d0/f9 0 2024-03-24T00:48:39.822 INFO:tasks.workunit.client.0.smithi038.stdout:8/137: chown d0/ce 399 1 2024-03-24T00:48:39.822 INFO:tasks.workunit.client.0.smithi038.stdout:5/118: dread f5 [0,4194304] 0 2024-03-24T00:48:39.822 INFO:tasks.workunit.client.0.smithi038.stdout:8/138: chown d0/d1d/d35 3 1 2024-03-24T00:48:39.822 INFO:tasks.workunit.client.0.smithi038.stdout:5/119: read d6/f9 [433021,12077] 0 2024-03-24T00:48:39.822 INFO:tasks.workunit.client.0.smithi038.stdout:8/139: write d0/d1d/d20/f23 [3995120,78394] 0 2024-03-24T00:48:39.822 INFO:tasks.workunit.client.0.smithi038.stdout:8/140: fsync d0/f32 0 2024-03-24T00:48:39.823 INFO:tasks.workunit.client.1.smithi088.stdout:5/121: dwrite d1/fb [0,4194304] 0 2024-03-24T00:48:39.824 INFO:tasks.workunit.client.1.smithi088.stdout:5/122: chown d1/l8 5 1 2024-03-24T00:48:39.824 INFO:tasks.workunit.client.1.smithi088.stdout:5/123: fdatasync d1/dd/f13 0 2024-03-24T00:48:39.824 INFO:tasks.workunit.client.1.smithi088.stdout:5/124: creat d1/d19/f2c x:0 0 0 2024-03-24T00:48:39.824 INFO:tasks.workunit.client.1.smithi088.stdout:5/125: dread - d1/d19/f2c zero size 2024-03-24T00:48:39.824 INFO:tasks.workunit.client.1.smithi088.stdout:5/126: truncate d1/dd/f15 4398910 0 2024-03-24T00:48:39.824 INFO:tasks.workunit.client.1.smithi088.stdout:0/133: mknod d9/dc/c2f 0 2024-03-24T00:48:39.824 INFO:tasks.workunit.client.1.smithi088.stdout:0/134: readlink d9/la 0 2024-03-24T00:48:39.824 INFO:tasks.workunit.client.1.smithi088.stdout:5/127: write d1/fa [4596821,120154] 0 2024-03-24T00:48:39.824 INFO:tasks.workunit.client.1.smithi088.stdout:7/116: mknod d3/c25 0 2024-03-24T00:48:39.824 INFO:tasks.workunit.client.1.smithi088.stdout:7/117: chown d3/l1c 2375584 1 2024-03-24T00:48:39.824 INFO:tasks.workunit.client.1.smithi088.stdout:7/118: chown d3/l1e 1586498 1 2024-03-24T00:48:39.825 INFO:tasks.workunit.client.0.smithi038.stdout:1/103: dwrite f9 [0,4194304] 0 2024-03-24T00:48:39.825 INFO:tasks.workunit.client.1.smithi088.stdout:6/173: symlink db/d1f/l32 0 2024-03-24T00:48:39.825 INFO:tasks.workunit.client.1.smithi088.stdout:6/174: truncate db/f19 673912 0 2024-03-24T00:48:39.825 INFO:tasks.workunit.client.1.smithi088.stdout:9/135: sync 2024-03-24T00:48:39.829 INFO:tasks.workunit.client.1.smithi088.stdout:1/133: symlink d6/d23/l25 0 2024-03-24T00:48:39.830 INFO:tasks.workunit.client.1.smithi088.stdout:8/130: mknod d9/dc/d11/d24/c2d 0 2024-03-24T00:48:39.834 INFO:tasks.workunit.client.0.smithi038.stdout:9/149: dread fc [0,4194304] 0 2024-03-24T00:48:39.836 INFO:tasks.workunit.client.1.smithi088.stdout:2/109: mkdir db/d21 0 2024-03-24T00:48:39.837 INFO:tasks.workunit.client.0.smithi038.stdout:0/118: dwrite d0/f2 [4194304,4194304] 0 2024-03-24T00:48:39.837 INFO:tasks.workunit.client.0.smithi038.stdout:0/119: stat d0/d12/d15/d1a 0 2024-03-24T00:48:39.837 INFO:tasks.workunit.client.0.smithi038.stdout:0/120: chown d0/d12/d13/l21 0 1 2024-03-24T00:48:39.837 INFO:tasks.workunit.client.0.smithi038.stdout:0/121: chown d0/d12/d13/c1f 2994459 1 2024-03-24T00:48:39.837 INFO:tasks.workunit.client.0.smithi038.stdout:0/122: creat d0/d12/d15/f2a x:0 0 0 2024-03-24T00:48:39.840 INFO:tasks.workunit.client.0.smithi038.stdout:7/105: symlink d2/d9/df/l25 0 2024-03-24T00:48:39.842 INFO:tasks.workunit.client.1.smithi088.stdout:4/153: dwrite d5/d13/d18/f1b [0,4194304] 0 2024-03-24T00:48:39.856 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:39 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:48:39.860 INFO:tasks.workunit.client.1.smithi088.stdout:6/175: write db/d1d/f21 [1756222,43527] 0 2024-03-24T00:48:39.866 INFO:tasks.workunit.client.0.smithi038.stdout:4/136: rename da/f28 to da/d12/f2c 0 2024-03-24T00:48:39.866 INFO:tasks.workunit.client.0.smithi038.stdout:4/137: write da/d12/d24/f26 [807562,6289] 0 2024-03-24T00:48:39.871 INFO:tasks.workunit.client.1.smithi088.stdout:5/128: symlink d1/d19/l2d 0 2024-03-24T00:48:39.872 INFO:tasks.workunit.client.0.smithi038.stdout:5/120: mknod d6/d19/d20/c25 0 2024-03-24T00:48:39.879 INFO:tasks.workunit.client.1.smithi088.stdout:8/131: mknod d9/c2e 0 2024-03-24T00:48:39.884 INFO:tasks.workunit.client.1.smithi088.stdout:3/98: sync 2024-03-24T00:48:39.884 INFO:tasks.workunit.client.0.smithi038.stdout:8/141: mkdir d0/d1d/d35/d37 0 2024-03-24T00:48:39.885 INFO:tasks.workunit.client.0.smithi038.stdout:8/142: chown d0/d15/c36 54223556 1 2024-03-24T00:48:39.887 INFO:tasks.workunit.client.1.smithi088.stdout:2/110: rename c3 to db/dd/c22 0 2024-03-24T00:48:39.887 INFO:tasks.workunit.client.1.smithi088.stdout:2/111: fdatasync db/dc/f1b 0 2024-03-24T00:48:39.887 INFO:tasks.workunit.client.1.smithi088.stdout:2/112: dread - db/dd/f1f zero size 2024-03-24T00:48:39.894 INFO:tasks.workunit.client.0.smithi038.stdout:1/104: mkdir da/dd/d22 0 2024-03-24T00:48:39.894 INFO:tasks.workunit.client.0.smithi038.stdout:1/105: creat da/f23 x:0 0 0 2024-03-24T00:48:39.896 INFO:tasks.workunit.client.1.smithi088.stdout:4/154: symlink d5/d16/d1a/d22/d2f/l31 0 2024-03-24T00:48:39.897 INFO:tasks.workunit.client.0.smithi038.stdout:9/150: symlink d12/l32 0 2024-03-24T00:48:39.902 INFO:tasks.workunit.client.1.smithi088.stdout:6/176: creat db/dd/f33 x:0 0 0 2024-03-24T00:48:39.906 INFO:tasks.workunit.client.0.smithi038.stdout:7/106: symlink d2/l26 0 2024-03-24T00:48:39.908 INFO:tasks.workunit.client.1.smithi088.stdout:5/129: mkdir d1/d10/d1b/d2e 0 2024-03-24T00:48:39.909 INFO:tasks.workunit.client.0.smithi038.stdout:4/138: rename da/dc/dd/l1a to da/d12/l2d 0 2024-03-24T00:48:39.911 INFO:tasks.workunit.client.0.smithi038.stdout:5/121: unlink d6/d16/d14/c22 0 2024-03-24T00:48:39.911 INFO:tasks.workunit.client.0.smithi038.stdout:5/122: truncate d6/d16/d14/f24 855931 0 2024-03-24T00:48:39.911 INFO:tasks.workunit.client.1.smithi088.stdout:7/119: rmdir d3 39 2024-03-24T00:48:39.911 INFO:tasks.workunit.client.1.smithi088.stdout:7/120: truncate d3/fd 807949 0 2024-03-24T00:48:39.914 INFO:tasks.workunit.client.1.smithi088.stdout:8/132: creat d9/dc/d11/f2f x:0 0 0 2024-03-24T00:48:39.916 INFO:tasks.workunit.client.1.smithi088.stdout:3/99: mkdir d3/d1a/d1e 0 2024-03-24T00:48:39.917 INFO:tasks.workunit.client.0.smithi038.stdout:1/106: symlink da/dd/l24 0 2024-03-24T00:48:39.917 INFO:tasks.workunit.client.0.smithi038.stdout:1/107: chown f4 207 1 2024-03-24T00:48:39.917 INFO:tasks.workunit.client.0.smithi038.stdout:1/108: truncate da/dd/d10/f1f 440072 0 2024-03-24T00:48:39.917 INFO:tasks.workunit.client.1.smithi088.stdout:1/134: dwrite d6/d18/d1e/f1f [0,4194304] 0 2024-03-24T00:48:39.921 INFO:tasks.workunit.client.1.smithi088.stdout:2/113: unlink f6 0 2024-03-24T00:48:39.923 INFO:tasks.workunit.client.0.smithi038.stdout:7/107: unlink d2/d9/df/c1b 0 2024-03-24T00:48:39.924 INFO:tasks.workunit.client.0.smithi038.stdout:7/108: truncate d2/f1f 50596 0 2024-03-24T00:48:39.924 INFO:tasks.workunit.client.1.smithi088.stdout:9/136: dwrite d1/f20 [0,4194304] 0 2024-03-24T00:48:39.925 INFO:tasks.workunit.client.1.smithi088.stdout:1/135: dread d6/d18/d1e/f21 [0,4194304] 0 2024-03-24T00:48:39.925 INFO:tasks.workunit.client.1.smithi088.stdout:1/136: chown d6/l11 0 1 2024-03-24T00:48:39.926 INFO:tasks.workunit.client.0.smithi038.stdout:5/123: dread d6/d19/f1c [0,4194304] 0 2024-03-24T00:48:39.926 INFO:tasks.workunit.client.0.smithi038.stdout:0/123: dwrite d0/d12/d15/f2a [0,4194304] 0 2024-03-24T00:48:39.926 INFO:tasks.workunit.client.0.smithi038.stdout:0/124: write d0/ff [4699595,111778] 0 2024-03-24T00:48:39.927 INFO:tasks.workunit.client.1.smithi088.stdout:4/155: rename d5/d7/f23 to d5/d13/d18/f32 0 2024-03-24T00:48:39.927 INFO:tasks.workunit.client.1.smithi088.stdout:4/156: creat d5/d17/f33 x:0 0 0 2024-03-24T00:48:39.929 INFO:tasks.workunit.client.1.smithi088.stdout:5/130: mkdir d1/d10/d1b/d2f 0 2024-03-24T00:48:39.929 INFO:tasks.workunit.client.1.smithi088.stdout:5/131: chown d1/dd/l2a 2 1 2024-03-24T00:48:39.932 INFO:tasks.workunit.client.0.smithi038.stdout:4/139: mknod da/d12/d29/c2e 0 2024-03-24T00:48:39.936 INFO:tasks.workunit.client.1.smithi088.stdout:2/114: write db/f18 [1778139,15211] 0 2024-03-24T00:48:39.942 INFO:tasks.workunit.client.1.smithi088.stdout:1/137: unlink d6/la 0 2024-03-24T00:48:39.942 INFO:tasks.workunit.client.1.smithi088.stdout:1/138: dread - d6/d18/f1d zero size 2024-03-24T00:48:39.943 INFO:tasks.workunit.client.0.smithi038.stdout:0/125: write d0/f2 [1329788,130893] 0 2024-03-24T00:48:39.947 INFO:tasks.workunit.client.1.smithi088.stdout:6/177: rename db/d1d/f21 to db/d1f/f34 0 2024-03-24T00:48:39.956 INFO:tasks.workunit.client.1.smithi088.stdout:5/132: link d1/ff d1/d19/f30 0 2024-03-24T00:48:39.956 INFO:tasks.workunit.client.1.smithi088.stdout:5/133: creat d1/f31 x:0 0 0 2024-03-24T00:48:39.956 INFO:tasks.workunit.client.1.smithi088.stdout:5/134: chown d1/l16 1 1 2024-03-24T00:48:39.959 INFO:tasks.workunit.client.0.smithi038.stdout:8/143: rename d0/d1d/d35 to d0/d15/d38 0 2024-03-24T00:48:39.960 INFO:tasks.workunit.client.0.smithi038.stdout:5/124: creat d6/f26 x:0 0 0 2024-03-24T00:48:39.960 INFO:tasks.workunit.client.0.smithi038.stdout:5/125: chown d6/d16/d14 26054 1 2024-03-24T00:48:39.961 INFO:tasks.workunit.client.1.smithi088.stdout:1/139: mknod d6/c26 0 2024-03-24T00:48:39.961 INFO:tasks.workunit.client.0.smithi038.stdout:4/140: stat da/dc/c1f 0 2024-03-24T00:48:39.963 INFO:tasks.workunit.client.1.smithi088.stdout:8/133: rename d9/dc/d11/l1c to d9/dc/d11/d24/d29/l30 0 2024-03-24T00:48:39.965 INFO:tasks.workunit.client.1.smithi088.stdout:7/121: dwrite d3/fc [0,4194304] 0 2024-03-24T00:48:39.967 INFO:tasks.workunit.client.1.smithi088.stdout:9/137: dwrite d1/f8 [0,4194304] 0 2024-03-24T00:48:39.967 INFO:tasks.workunit.client.1.smithi088.stdout:3/100: dwrite f2 [0,4194304] 0 2024-03-24T00:48:39.967 INFO:tasks.workunit.client.1.smithi088.stdout:9/138: truncate d1/d10/f1d 350503 0 2024-03-24T00:48:39.969 INFO:tasks.workunit.client.1.smithi088.stdout:5/135: mknod d1/d19/d1e/c32 0 2024-03-24T00:48:39.971 INFO:tasks.workunit.client.1.smithi088.stdout:1/140: rename d6/d18 to d6/d27 0 2024-03-24T00:48:39.971 INFO:tasks.workunit.client.1.smithi088.stdout:1/141: readlink l5 0 2024-03-24T00:48:39.972 INFO:tasks.workunit.client.1.smithi088.stdout:8/134: truncate d9/f10 7563 0 2024-03-24T00:48:39.974 INFO:tasks.workunit.client.0.smithi038.stdout:5/126: dread d6/d19/f1c [0,4194304] 0 2024-03-24T00:48:39.976 INFO:tasks.workunit.client.1.smithi088.stdout:2/115: rename db/dd/lf to db/dc/l23 0 2024-03-24T00:48:39.977 INFO:tasks.workunit.client.0.smithi038.stdout:7/109: getdents d2/d9/da/dc 0 2024-03-24T00:48:39.977 INFO:tasks.workunit.client.0.smithi038.stdout:7/110: fsync d2/f13 0 2024-03-24T00:48:39.980 INFO:tasks.workunit.client.0.smithi038.stdout:8/144: unlink d0/c3 0 2024-03-24T00:48:39.980 INFO:tasks.workunit.client.0.smithi038.stdout:8/145: write d0/f32 [908373,39444] 0 2024-03-24T00:48:39.980 INFO:tasks.workunit.client.0.smithi038.stdout:8/146: write d0/d15/d2c/f34 [593682,61269] 0 2024-03-24T00:48:39.980 INFO:tasks.workunit.client.1.smithi088.stdout:0/135: truncate d9/dc/de/d19/f1a 40763 0 2024-03-24T00:48:39.982 INFO:tasks.workunit.client.0.smithi038.stdout:9/151: rename d12/l2a to d12/d28/l33 0 2024-03-24T00:48:39.984 INFO:tasks.workunit.client.1.smithi088.stdout:7/122: link d3/l12 d3/l26 0 2024-03-24T00:48:39.984 INFO:tasks.workunit.client.0.smithi038.stdout:8/147: fsync d0/f32 0 2024-03-24T00:48:39.985 INFO:tasks.workunit.client.0.smithi038.stdout:4/141: truncate da/f1d 827496 0 2024-03-24T00:48:39.985 INFO:tasks.workunit.client.1.smithi088.stdout:4/157: dwrite d5/d7/dd/ff [0,4194304] 0 2024-03-24T00:48:39.989 INFO:tasks.workunit.client.0.smithi038.stdout:5/127: dread d6/d19/f1c [0,4194304] 0 2024-03-24T00:48:39.993 INFO:tasks.workunit.client.0.smithi038.stdout:7/111: creat d2/d9/da/d21/f27 x:0 0 0 2024-03-24T00:48:39.993 INFO:tasks.workunit.client.0.smithi038.stdout:7/112: stat d2/d9/f20 0 2024-03-24T00:48:39.996 INFO:tasks.workunit.client.0.smithi038.stdout:1/109: getdents da/dd 0 2024-03-24T00:48:39.996 INFO:tasks.workunit.client.0.smithi038.stdout:1/110: readlink da/dd/l24 0 2024-03-24T00:48:39.997 INFO:tasks.workunit.client.0.smithi038.stdout:1/111: stat da/dd/d10/f1f 0 2024-03-24T00:48:40.003 INFO:tasks.workunit.client.1.smithi088.stdout:9/139: mknod d1/d10/d1f/c24 0 2024-03-24T00:48:40.003 INFO:tasks.workunit.client.1.smithi088.stdout:3/101: creat d3/d6/f1f x:0 0 0 2024-03-24T00:48:40.007 INFO:tasks.workunit.client.0.smithi038.stdout:8/148: mknod d0/d1d/d20/c39 0 2024-03-24T00:48:40.011 INFO:tasks.workunit.client.1.smithi088.stdout:2/116: mknod db/dd/c24 0 2024-03-24T00:48:40.013 INFO:tasks.workunit.client.0.smithi038.stdout:4/142: dread f9 [0,4194304] 0 2024-03-24T00:48:40.019 INFO:tasks.workunit.client.1.smithi088.stdout:5/136: rename d1/d10/c1f to d1/d29/c33 0 2024-03-24T00:48:40.022 INFO:tasks.workunit.client.1.smithi088.stdout:7/123: creat d3/f27 x:0 0 0 2024-03-24T00:48:40.024 INFO:tasks.workunit.client.0.smithi038.stdout:7/113: symlink d2/d9/df/l28 0 2024-03-24T00:48:40.024 INFO:tasks.workunit.client.0.smithi038.stdout:7/114: write d2/f1f [1079130,65607] 0 2024-03-24T00:48:40.025 INFO:tasks.workunit.client.1.smithi088.stdout:4/158: symlink d5/d7/dd/l34 0 2024-03-24T00:48:40.029 INFO:tasks.workunit.client.1.smithi088.stdout:6/178: rmdir db/d1f 39 2024-03-24T00:48:40.029 INFO:tasks.workunit.client.0.smithi038.stdout:4/143: mknod da/d12/d24/c2f 0 2024-03-24T00:48:40.029 INFO:tasks.workunit.client.0.smithi038.stdout:4/144: truncate da/dc/dd/d15/f23 4955863 0 2024-03-24T00:48:40.029 INFO:tasks.workunit.client.0.smithi038.stdout:4/145: write da/dc/dd/f2b [53721,45806] 0 2024-03-24T00:48:40.029 INFO:tasks.workunit.client.1.smithi088.stdout:6/179: write db/dd/f33 [237651,83812] 0 2024-03-24T00:48:40.030 INFO:tasks.workunit.client.1.smithi088.stdout:0/136: dwrite d9/dc/de/f1d [0,4194304] 0 2024-03-24T00:48:40.030 INFO:tasks.workunit.client.1.smithi088.stdout:9/140: symlink d1/d15/l25 0 2024-03-24T00:48:40.030 INFO:tasks.workunit.client.1.smithi088.stdout:9/141: fsync d1/f20 0 2024-03-24T00:48:40.036 INFO:tasks.workunit.client.1.smithi088.stdout:2/117: truncate db/dd/f17 1481850 0 2024-03-24T00:48:40.036 INFO:tasks.workunit.client.1.smithi088.stdout:2/118: readlink db/dd/l16 0 2024-03-24T00:48:40.039 INFO:tasks.workunit.client.0.smithi038.stdout:7/115: creat d2/d9/df/f29 x:0 0 0 2024-03-24T00:48:40.051 INFO:tasks.workunit.client.1.smithi088.stdout:8/135: write d9/f10 [34793,84537] 0 2024-03-24T00:48:40.052 INFO:tasks.workunit.client.0.smithi038.stdout:7/116: mknod d2/d9/df/c2a 0 2024-03-24T00:48:40.056 INFO:tasks.workunit.client.0.smithi038.stdout:5/128: dwrite d6/fa [0,4194304] 0 2024-03-24T00:48:40.056 INFO:tasks.workunit.client.0.smithi038.stdout:5/129: write d6/f26 [944382,75699] 0 2024-03-24T00:48:40.057 INFO:tasks.workunit.client.1.smithi088.stdout:3/102: dwrite d3/de/f16 [0,4194304] 0 2024-03-24T00:48:40.063 INFO:tasks.workunit.client.1.smithi088.stdout:1/142: rename d6/l15 to d6/d23/l28 0 2024-03-24T00:48:40.063 INFO:tasks.workunit.client.1.smithi088.stdout:4/159: rename d5/d7/dd to d5/d7/dd/d35 22 2024-03-24T00:48:40.064 INFO:tasks.workunit.client.1.smithi088.stdout:5/137: symlink d1/d29/l34 0 2024-03-24T00:48:40.065 INFO:tasks.workunit.client.0.smithi038.stdout:1/112: dwrite da/df/f13 [0,4194304] 0 2024-03-24T00:48:40.065 INFO:tasks.workunit.client.0.smithi038.stdout:1/113: fsync da/f11 0 2024-03-24T00:48:40.071 INFO:tasks.workunit.client.0.smithi038.stdout:7/117: mkdir d2/d9/da/d23/d2b 0 2024-03-24T00:48:40.071 INFO:tasks.workunit.client.0.smithi038.stdout:9/152: rename d12/d28/l33 to d12/d28/l34 0 2024-03-24T00:48:40.071 INFO:tasks.workunit.client.0.smithi038.stdout:9/153: chown d12/d1f/d26/c2e 143009653 1 2024-03-24T00:48:40.083 INFO:tasks.workunit.client.1.smithi088.stdout:0/137: dread d9/dc/de/d19/f1a [0,4194304] 0 2024-03-24T00:48:40.084 INFO:tasks.workunit.client.1.smithi088.stdout:0/138: chown d9/dc/de/c28 467 1 2024-03-24T00:48:40.091 INFO:tasks.workunit.client.0.smithi038.stdout:8/149: dwrite d0/f16 [0,4194304] 0 2024-03-24T00:48:40.091 INFO:tasks.workunit.client.0.smithi038.stdout:8/150: readlink d0/d10/d19/l1c 0 2024-03-24T00:48:40.092 INFO:tasks.workunit.client.0.smithi038.stdout:9/154: creat d12/f35 x:0 0 0 2024-03-24T00:48:40.092 INFO:tasks.workunit.client.0.smithi038.stdout:1/114: creat da/dd/d22/f25 x:0 0 0 2024-03-24T00:48:40.092 INFO:tasks.workunit.client.0.smithi038.stdout:1/115: readlink da/df/l1d 0 2024-03-24T00:48:40.097 INFO:tasks.workunit.client.0.smithi038.stdout:8/151: symlink d0/d10/d19/d29/l3a 0 2024-03-24T00:48:40.101 INFO:tasks.workunit.client.0.smithi038.stdout:2/153: sync 2024-03-24T00:48:40.101 INFO:tasks.workunit.client.0.smithi038.stdout:3/137: sync 2024-03-24T00:48:40.101 INFO:tasks.workunit.client.0.smithi038.stdout:3/138: chown d3/d9 4 1 2024-03-24T00:48:40.101 INFO:tasks.workunit.client.0.smithi038.stdout:2/154: dread - d4/f24 zero size 2024-03-24T00:48:40.102 INFO:tasks.workunit.client.0.smithi038.stdout:1/116: symlink da/dd/d22/l26 0 2024-03-24T00:48:40.102 INFO:tasks.workunit.client.0.smithi038.stdout:3/139: write d3/d9/dc/df/f31 [4436995,27311] 0 2024-03-24T00:48:40.102 INFO:tasks.workunit.client.0.smithi038.stdout:3/140: truncate d3/d9/f16 1029884 0 2024-03-24T00:48:40.107 INFO:tasks.workunit.client.0.smithi038.stdout:9/155: truncate d12/d1a/f20 2552434 0 2024-03-24T00:48:40.109 INFO:tasks.workunit.client.0.smithi038.stdout:8/152: mknod d0/c3b 0 2024-03-24T00:48:40.114 INFO:tasks.workunit.client.1.smithi088.stdout:3/103: mknod d3/de/c20 0 2024-03-24T00:48:40.116 INFO:tasks.workunit.client.0.smithi038.stdout:2/155: mkdir d4/d27/d2f/d3a 0 2024-03-24T00:48:40.116 INFO:tasks.workunit.client.0.smithi038.stdout:2/156: chown d4/d27/d2f 14551 1 2024-03-24T00:48:40.117 INFO:tasks.workunit.client.1.smithi088.stdout:6/180: dwrite db/d1f/f2f [0,4194304] 0 2024-03-24T00:48:40.119 INFO:tasks.workunit.client.0.smithi038.stdout:1/117: mknod da/dd/d22/c27 0 2024-03-24T00:48:40.121 INFO:tasks.workunit.client.1.smithi088.stdout:1/143: creat d6/d23/f29 x:0 0 0 2024-03-24T00:48:40.121 INFO:tasks.workunit.client.1.smithi088.stdout:1/144: chown d6/f10 23218713 1 2024-03-24T00:48:40.121 INFO:tasks.workunit.client.1.smithi088.stdout:1/145: readlink l5 0 2024-03-24T00:48:40.122 INFO:tasks.workunit.client.0.smithi038.stdout:3/141: creat d3/d30/f33 x:0 0 0 2024-03-24T00:48:40.123 INFO:tasks.workunit.client.1.smithi088.stdout:2/119: dwrite db/f18 [0,4194304] 0 2024-03-24T00:48:40.125 INFO:tasks.workunit.client.1.smithi088.stdout:4/160: creat d5/d13/f36 x:0 0 0 2024-03-24T00:48:40.125 INFO:tasks.workunit.client.1.smithi088.stdout:4/161: readlink d5/d16/d1a/d22/l28 0 2024-03-24T00:48:40.126 INFO:tasks.workunit.client.0.smithi038.stdout:9/156: creat d12/d28/f36 x:0 0 0 2024-03-24T00:48:40.127 INFO:tasks.workunit.client.1.smithi088.stdout:5/138: creat d1/d19/d1e/d26/f35 x:0 0 0 2024-03-24T00:48:40.127 INFO:tasks.workunit.client.1.smithi088.stdout:5/139: write d1/f1c [4324254,10342] 0 2024-03-24T00:48:40.127 INFO:tasks.workunit.client.1.smithi088.stdout:5/140: dread - d1/f31 zero size 2024-03-24T00:48:40.127 INFO:tasks.workunit.client.1.smithi088.stdout:5/141: dread - d1/d19/f2c zero size 2024-03-24T00:48:40.128 INFO:tasks.workunit.client.0.smithi038.stdout:6/164: sync 2024-03-24T00:48:40.128 INFO:tasks.workunit.client.0.smithi038.stdout:8/153: mkdir d0/d1d/d20/d3c 0 2024-03-24T00:48:40.129 INFO:tasks.workunit.client.0.smithi038.stdout:4/146: dwrite da/dc/dd/f2a [0,4194304] 0 2024-03-24T00:48:40.129 INFO:tasks.workunit.client.0.smithi038.stdout:4/147: write da/dc/dd/f2b [202430,68945] 0 2024-03-24T00:48:40.129 INFO:tasks.workunit.client.0.smithi038.stdout:4/148: chown da/d12/d24/c2f 525 1 2024-03-24T00:48:40.129 INFO:tasks.workunit.client.0.smithi038.stdout:5/130: dwrite d6/d1e/f1f [0,4194304] 0 2024-03-24T00:48:40.130 INFO:tasks.workunit.client.0.smithi038.stdout:5/131: creat d6/d1e/f27 x:0 0 0 2024-03-24T00:48:40.130 INFO:tasks.workunit.client.0.smithi038.stdout:5/132: write f5 [173929,113928] 0 2024-03-24T00:48:40.130 INFO:tasks.workunit.client.0.smithi038.stdout:5/133: write f5 [599514,60642] 0 2024-03-24T00:48:40.132 INFO:tasks.workunit.client.1.smithi088.stdout:7/124: getdents d3 0 2024-03-24T00:48:40.132 INFO:tasks.workunit.client.1.smithi088.stdout:7/125: dread - d3/f24 zero size 2024-03-24T00:48:40.134 INFO:tasks.workunit.client.0.smithi038.stdout:2/157: rmdir d4/d27/d2f 39 2024-03-24T00:48:40.134 INFO:tasks.workunit.client.0.smithi038.stdout:2/158: fsync d4/d18/f2e 0 2024-03-24T00:48:40.134 INFO:tasks.workunit.client.1.smithi088.stdout:3/104: dread f2 [0,4194304] 0 2024-03-24T00:48:40.135 INFO:tasks.workunit.client.0.smithi038.stdout:7/118: dwrite d2/d9/d16/f19 [0,4194304] 0 2024-03-24T00:48:40.135 INFO:tasks.workunit.client.1.smithi088.stdout:0/139: link d9/d20/l25 d9/dc/l30 0 2024-03-24T00:48:40.140 INFO:tasks.workunit.client.0.smithi038.stdout:6/165: mknod d1/c2e 0 2024-03-24T00:48:40.141 INFO:tasks.workunit.client.0.smithi038.stdout:6/166: fsync d1/f8 0 2024-03-24T00:48:40.141 INFO:tasks.workunit.client.0.smithi038.stdout:6/167: write d1/f13 [1412445,98981] 0 2024-03-24T00:48:40.142 INFO:tasks.workunit.client.1.smithi088.stdout:6/181: creat db/f35 x:0 0 0 2024-03-24T00:48:40.145 INFO:tasks.workunit.client.1.smithi088.stdout:6/182: write db/f1c [170752,89981] 0 2024-03-24T00:48:40.146 INFO:tasks.workunit.client.0.smithi038.stdout:1/118: dread f7 [0,4194304] 0 2024-03-24T00:48:40.163 INFO:tasks.workunit.client.1.smithi088.stdout:5/142: dwrite d1/dd/f13 [4194304,4194304] 0 2024-03-24T00:48:40.189 INFO:tasks.workunit.client.0.smithi038.stdout:7/119: dwrite d2/d9/da/f14 [0,4194304] 0 2024-03-24T00:48:40.208 INFO:tasks.workunit.client.1.smithi088.stdout:5/143: dwrite d1/fa [4194304,4194304] 0 2024-03-24T00:48:40.220 INFO:tasks.workunit.client.0.smithi038.stdout:9/157: fsync d12/f35 0 2024-03-24T00:48:40.221 INFO:tasks.workunit.client.1.smithi088.stdout:1/146: creat d6/f2a x:0 0 0 2024-03-24T00:48:40.223 INFO:tasks.workunit.client.1.smithi088.stdout:2/120: link db/dd/c24 db/d21/c25 0 2024-03-24T00:48:40.225 INFO:tasks.workunit.client.1.smithi088.stdout:4/162: mkdir d5/d16/d37 0 2024-03-24T00:48:40.226 INFO:tasks.workunit.client.0.smithi038.stdout:8/154: symlink d0/d1d/d20/l3d 0 2024-03-24T00:48:40.226 INFO:tasks.workunit.client.0.smithi038.stdout:8/155: fsync d0/f16 0 2024-03-24T00:48:40.230 INFO:tasks.workunit.client.1.smithi088.stdout:7/126: rename d3/f7 to d3/f28 0 2024-03-24T00:48:40.233 INFO:tasks.workunit.client.1.smithi088.stdout:3/105: symlink d3/d1a/l21 0 2024-03-24T00:48:40.235 INFO:tasks.workunit.client.1.smithi088.stdout:3/106: write d3/d12/f13 [1225071,57172] 0 2024-03-24T00:48:40.235 INFO:tasks.workunit.client.1.smithi088.stdout:3/107: chown f1 20449 1 2024-03-24T00:48:40.235 INFO:tasks.workunit.client.1.smithi088.stdout:3/108: truncate d3/de/df/f15 4610040 0 2024-03-24T00:48:40.235 INFO:tasks.workunit.client.1.smithi088.stdout:3/109: fdatasync d3/d6/f11 0 2024-03-24T00:48:40.235 INFO:tasks.workunit.client.1.smithi088.stdout:3/110: write d3/d6/fb [4367187,70569] 0 2024-03-24T00:48:40.235 INFO:tasks.workunit.client.1.smithi088.stdout:3/111: chown d3/de/df/f15 18238 1 2024-03-24T00:48:40.237 INFO:tasks.workunit.client.1.smithi088.stdout:4/163: write d5/d16/d1a/f1e [1266428,79898] 0 2024-03-24T00:48:40.237 INFO:tasks.workunit.client.1.smithi088.stdout:4/164: truncate d5/d7/dd/ff 5076008 0 2024-03-24T00:48:40.237 INFO:tasks.workunit.client.1.smithi088.stdout:0/140: unlink d9/dc/de/d19/f1f 0 2024-03-24T00:48:40.242 INFO:tasks.workunit.client.1.smithi088.stdout:6/183: rename db/d1f/c2d to db/d1f/c36 0 2024-03-24T00:48:40.244 INFO:tasks.workunit.client.0.smithi038.stdout:3/142: truncate d3/d9/fb 2180422 0 2024-03-24T00:48:40.244 INFO:tasks.workunit.client.0.smithi038.stdout:3/143: chown d3/ce 12 1 2024-03-24T00:48:40.244 INFO:tasks.workunit.client.0.smithi038.stdout:3/144: stat d3/d9/dc/f1f 0 2024-03-24T00:48:40.245 INFO:tasks.workunit.client.0.smithi038.stdout:3/145: creat d3/d30/f34 x:0 0 0 2024-03-24T00:48:40.245 INFO:tasks.workunit.client.0.smithi038.stdout:3/146: write d3/d30/f33 [106017,3977] 0 2024-03-24T00:48:40.245 INFO:tasks.workunit.client.0.smithi038.stdout:3/147: stat d3/d2a/l1b 0 2024-03-24T00:48:40.248 INFO:tasks.workunit.client.0.smithi038.stdout:6/168: rmdir d1/da 39 2024-03-24T00:48:40.248 INFO:tasks.workunit.client.0.smithi038.stdout:6/169: write d1/da/d1b/f2b [914529,68977] 0 2024-03-24T00:48:40.249 INFO:tasks.workunit.client.0.smithi038.stdout:6/170: chown d1/da/d1b/c22 9729 1 2024-03-24T00:48:40.252 INFO:tasks.workunit.client.0.smithi038.stdout:1/119: mknod da/dd/d10/d14/c28 0 2024-03-24T00:48:40.252 INFO:tasks.workunit.client.0.smithi038.stdout:1/120: write da/df/f17 [564933,117261] 0 2024-03-24T00:48:40.255 INFO:tasks.workunit.client.1.smithi088.stdout:2/121: write db/f20 [2229060,2854] 0 2024-03-24T00:48:40.257 INFO:tasks.workunit.client.1.smithi088.stdout:5/144: mknod d1/d10/d1b/d2e/c36 0 2024-03-24T00:48:40.259 INFO:tasks.workunit.client.1.smithi088.stdout:1/147: rename l5 to d6/d27/l2b 0 2024-03-24T00:48:40.265 INFO:tasks.workunit.client.1.smithi088.stdout:7/127: getdents d3 0 2024-03-24T00:48:40.265 INFO:tasks.workunit.client.1.smithi088.stdout:7/128: chown d3/l11 1163 1 2024-03-24T00:48:40.265 INFO:tasks.workunit.client.1.smithi088.stdout:7/129: getdents d3 0 2024-03-24T00:48:40.267 INFO:tasks.workunit.client.1.smithi088.stdout:3/112: write d3/d6/fb [3211716,71633] 0 2024-03-24T00:48:40.267 INFO:tasks.workunit.client.0.smithi038.stdout:8/156: creat d0/d1d/f3e x:0 0 0 2024-03-24T00:48:40.269 INFO:tasks.workunit.client.0.smithi038.stdout:4/149: getdents da 0 2024-03-24T00:48:40.269 INFO:tasks.workunit.client.0.smithi038.stdout:4/150: dread - da/dc/f22 zero size 2024-03-24T00:48:40.271 INFO:tasks.workunit.client.1.smithi088.stdout:0/141: getdents d9/dc 0 2024-03-24T00:48:40.275 INFO:tasks.workunit.client.1.smithi088.stdout:4/165: symlink d5/d13/d18/l38 0 2024-03-24T00:48:40.275 INFO:tasks.workunit.client.0.smithi038.stdout:3/148: mknod d3/d9/dc/df/d14/d19/c35 0 2024-03-24T00:48:40.275 INFO:tasks.workunit.client.0.smithi038.stdout:3/149: chown d3/ce 0 1 2024-03-24T00:48:40.275 INFO:tasks.workunit.client.0.smithi038.stdout:3/150: write d3/d9/f26 [631466,79709] 0 2024-03-24T00:48:40.277 INFO:tasks.workunit.client.0.smithi038.stdout:6/171: mkdir d1/da/d1b/d2f 0 2024-03-24T00:48:40.277 INFO:tasks.workunit.client.0.smithi038.stdout:6/172: write d1/da/d1b/d1d/f24 [289401,73663] 0 2024-03-24T00:48:40.278 INFO:tasks.workunit.client.0.smithi038.stdout:1/121: truncate f8 2223549 0 2024-03-24T00:48:40.278 INFO:tasks.workunit.client.0.smithi038.stdout:1/122: stat da/dd/d22/c27 0 2024-03-24T00:48:40.281 INFO:tasks.workunit.client.0.smithi038.stdout:7/120: getdents d2/d9/d16 0 2024-03-24T00:48:40.281 INFO:tasks.workunit.client.1.smithi088.stdout:1/148: stat l4 0 2024-03-24T00:48:40.281 INFO:tasks.workunit.client.1.smithi088.stdout:9/142: sync 2024-03-24T00:48:40.282 INFO:tasks.workunit.client.1.smithi088.stdout:8/136: sync 2024-03-24T00:48:40.282 INFO:tasks.workunit.client.1.smithi088.stdout:8/137: chown d9/dc/d11/d24/c2d 1 1 2024-03-24T00:48:40.282 INFO:tasks.workunit.client.1.smithi088.stdout:6/184: rename db/d1f/f2f to db/dd/f37 0 2024-03-24T00:48:40.282 INFO:tasks.workunit.client.1.smithi088.stdout:6/185: chown db/d1f/c2e 534879464 1 2024-03-24T00:48:40.283 INFO:tasks.workunit.client.1.smithi088.stdout:6/186: creat db/f38 x:0 0 0 2024-03-24T00:48:40.286 INFO:tasks.workunit.client.0.smithi038.stdout:9/158: rename cd to d12/d1f/c37 0 2024-03-24T00:48:40.286 INFO:tasks.workunit.client.0.smithi038.stdout:9/159: readlink d12/d28/l34 0 2024-03-24T00:48:40.287 INFO:tasks.workunit.client.1.smithi088.stdout:7/130: creat d3/f29 x:0 0 0 2024-03-24T00:48:40.287 INFO:tasks.workunit.client.1.smithi088.stdout:7/131: chown d3/l1c 0 1 2024-03-24T00:48:40.287 INFO:tasks.workunit.client.1.smithi088.stdout:7/132: creat d3/f2a x:0 0 0 2024-03-24T00:48:40.287 INFO:tasks.workunit.client.1.smithi088.stdout:7/133: truncate d3/f5 4693523 0 2024-03-24T00:48:40.287 INFO:tasks.workunit.client.1.smithi088.stdout:7/134: stat d3/f16 0 2024-03-24T00:48:40.287 INFO:tasks.workunit.client.1.smithi088.stdout:7/135: creat d3/f2b x:0 0 0 2024-03-24T00:48:40.287 INFO:tasks.workunit.client.1.smithi088.stdout:7/136: chown d3/f5 7392 1 2024-03-24T00:48:40.288 INFO:tasks.workunit.client.0.smithi038.stdout:8/157: symlink d0/d1d/d27/l3f 0 2024-03-24T00:48:40.290 INFO:tasks.workunit.client.1.smithi088.stdout:3/113: symlink d3/l22 0 2024-03-24T00:48:40.290 INFO:tasks.workunit.client.1.smithi088.stdout:4/166: truncate d5/d13/d18/d24/f30 3484183 0 2024-03-24T00:48:40.291 INFO:tasks.workunit.client.1.smithi088.stdout:9/143: mknod d1/d10/c26 0 2024-03-24T00:48:40.292 INFO:tasks.workunit.client.0.smithi038.stdout:4/151: unlink da/dc/dd/f2a 0 2024-03-24T00:48:40.292 INFO:tasks.workunit.client.0.smithi038.stdout:2/159: dwrite d4/f1d [0,4194304] 0 2024-03-24T00:48:40.294 INFO:tasks.workunit.client.0.smithi038.stdout:6/173: symlink d1/da/d1b/l30 0 2024-03-24T00:48:40.297 INFO:tasks.workunit.client.1.smithi088.stdout:0/142: rename d9/cf to d9/dc/c31 0 2024-03-24T00:48:40.297 INFO:tasks.workunit.client.1.smithi088.stdout:0/143: creat d9/dc/de/f32 x:0 0 0 2024-03-24T00:48:40.299 INFO:tasks.workunit.client.1.smithi088.stdout:6/187: creat db/d1f/f39 x:0 0 0 2024-03-24T00:48:40.300 INFO:tasks.workunit.client.1.smithi088.stdout:6/188: write db/f19 [1186548,1847] 0 2024-03-24T00:48:40.304 INFO:tasks.workunit.client.1.smithi088.stdout:2/122: write db/dc/f11 [2703395,21889] 0 2024-03-24T00:48:40.304 INFO:tasks.workunit.client.0.smithi038.stdout:7/121: mkdir d2/d2c 0 2024-03-24T00:48:40.305 INFO:tasks.workunit.client.0.smithi038.stdout:8/158: dread d0/f32 [0,4194304] 0 2024-03-24T00:48:40.313 INFO:tasks.workunit.client.1.smithi088.stdout:7/137: rename d3/f23 to d3/f2c 0 2024-03-24T00:48:40.313 INFO:tasks.workunit.client.1.smithi088.stdout:7/138: creat d3/f2d x:0 0 0 2024-03-24T00:48:40.316 INFO:tasks.workunit.client.1.smithi088.stdout:3/114: symlink d3/d12/l23 0 2024-03-24T00:48:40.322 INFO:tasks.workunit.client.1.smithi088.stdout:9/144: mkdir d1/d22/d27 0 2024-03-24T00:48:40.328 INFO:tasks.workunit.client.1.smithi088.stdout:0/144: truncate d9/dc/de/f1d 2185702 0 2024-03-24T00:48:40.330 INFO:tasks.workunit.client.0.smithi038.stdout:3/151: symlink d3/l36 0 2024-03-24T00:48:40.330 INFO:tasks.workunit.client.0.smithi038.stdout:3/152: dread - d3/d9/dc/f1f zero size 2024-03-24T00:48:40.331 INFO:tasks.workunit.client.0.smithi038.stdout:3/153: fsync d3/f5 0 2024-03-24T00:48:40.335 INFO:tasks.workunit.client.1.smithi088.stdout:9/145: dread d1/f6 [0,4194304] 0 2024-03-24T00:48:40.336 INFO:tasks.workunit.client.1.smithi088.stdout:7/139: rename d3/l9 to d3/l2e 0 2024-03-24T00:48:40.336 INFO:tasks.workunit.client.1.smithi088.stdout:7/140: chown d3/l19 56278 1 2024-03-24T00:48:40.337 INFO:tasks.workunit.client.0.smithi038.stdout:2/160: rmdir d4/d27/d2f/d3a 0 2024-03-24T00:48:40.339 INFO:tasks.workunit.client.0.smithi038.stdout:2/161: read d4/f13 [126705,24129] 0 2024-03-24T00:48:40.340 INFO:tasks.workunit.client.1.smithi088.stdout:3/115: link f2 d3/d1a/d1c/f24 0 2024-03-24T00:48:40.342 INFO:tasks.workunit.client.1.smithi088.stdout:4/167: rmdir d5/d13/d18 39 2024-03-24T00:48:40.344 INFO:tasks.workunit.client.0.smithi038.stdout:6/174: mknod d1/da/d1b/d25/c31 0 2024-03-24T00:48:40.348 INFO:tasks.workunit.client.1.smithi088.stdout:0/145: symlink d9/d20/d2a/l33 0 2024-03-24T00:48:40.351 INFO:tasks.workunit.client.1.smithi088.stdout:3/116: rename d3/d12/f13 to d3/de/df/f25 0 2024-03-24T00:48:40.351 INFO:tasks.workunit.client.1.smithi088.stdout:3/117: readlink d3/l22 0 2024-03-24T00:48:40.351 INFO:tasks.workunit.client.1.smithi088.stdout:3/118: chown d3 6 1 2024-03-24T00:48:40.352 INFO:tasks.workunit.client.0.smithi038.stdout:7/122: mknod d2/d9/df/c2d 0 2024-03-24T00:48:40.353 INFO:tasks.workunit.client.0.smithi038.stdout:9/160: rmdir d12/d28 39 2024-03-24T00:48:40.353 INFO:tasks.workunit.client.0.smithi038.stdout:9/161: chown d12/d28/f36 17912137 1 2024-03-24T00:48:40.353 INFO:tasks.workunit.client.0.smithi038.stdout:9/162: truncate d12/d1f/f30 349288 0 2024-03-24T00:48:40.356 INFO:tasks.workunit.client.1.smithi088.stdout:3/119: link d3/l22 d3/d1a/d1c/l26 0 2024-03-24T00:48:40.356 INFO:tasks.workunit.client.0.smithi038.stdout:3/154: mknod d3/d2a/c37 0 2024-03-24T00:48:40.357 INFO:tasks.workunit.client.0.smithi038.stdout:6/175: symlink d1/da/d1b/d2f/l32 0 2024-03-24T00:48:40.357 INFO:tasks.workunit.client.0.smithi038.stdout:6/176: fsync d1/da/f1a 0 2024-03-24T00:48:40.360 INFO:tasks.workunit.client.0.smithi038.stdout:7/123: symlink d2/d9/l2e 0 2024-03-24T00:48:40.360 INFO:tasks.workunit.client.0.smithi038.stdout:9/163: truncate d12/f1e 9245 0 2024-03-24T00:48:40.360 INFO:tasks.workunit.client.1.smithi088.stdout:1/149: write d6/d27/d1e/f1f [3949167,26236] 0 2024-03-24T00:48:40.360 INFO:tasks.workunit.client.0.smithi038.stdout:9/164: write d12/d1a/f20 [3294178,87262] 0 2024-03-24T00:48:40.363 INFO:tasks.workunit.client.0.smithi038.stdout:9/165: symlink d12/d1a/l38 0 2024-03-24T00:48:40.365 INFO:tasks.workunit.client.1.smithi088.stdout:1/150: symlink d6/d27/l2c 0 2024-03-24T00:48:40.365 INFO:tasks.workunit.client.1.smithi088.stdout:1/151: creat d6/d23/f2d x:0 0 0 2024-03-24T00:48:40.365 INFO:tasks.workunit.client.1.smithi088.stdout:1/152: truncate d6/f13 652538 0 2024-03-24T00:48:40.365 INFO:tasks.workunit.client.1.smithi088.stdout:1/153: readlink d6/d27/l2c 0 2024-03-24T00:48:40.365 INFO:tasks.workunit.client.0.smithi038.stdout:3/155: symlink d3/l38 0 2024-03-24T00:48:40.366 INFO:tasks.workunit.client.0.smithi038.stdout:3/156: stat d3/d9/dc/df/d14/f17 0 2024-03-24T00:48:40.366 INFO:tasks.workunit.client.0.smithi038.stdout:3/157: read - d3/d9/dc/df/f1d zero size 2024-03-24T00:48:40.366 INFO:tasks.workunit.client.0.smithi038.stdout:3/158: getdents d3/d9/dc 0 2024-03-24T00:48:40.368 INFO:tasks.workunit.client.1.smithi088.stdout:4/168: read f2 [3760808,55466] 0 2024-03-24T00:48:40.369 INFO:tasks.workunit.client.0.smithi038.stdout:7/124: getdents d2/d9/da 0 2024-03-24T00:48:40.369 INFO:tasks.workunit.client.0.smithi038.stdout:7/125: fdatasync d2/d9/da/dc/fd 0 2024-03-24T00:48:40.369 INFO:tasks.workunit.client.0.smithi038.stdout:9/166: rename d12/c25 to d12/d1f/d26/c39 0 2024-03-24T00:48:40.370 INFO:tasks.workunit.client.1.smithi088.stdout:4/169: symlink d5/d13/d18/d24/l39 0 2024-03-24T00:48:40.371 INFO:tasks.workunit.client.0.smithi038.stdout:3/159: rename d3/d9/dc/df/d14/d19/l1e to d3/d30/l39 0 2024-03-24T00:48:40.371 INFO:tasks.workunit.client.0.smithi038.stdout:3/160: chown d3/d9/dc/df/f12 1 1 2024-03-24T00:48:40.372 INFO:tasks.workunit.client.0.smithi038.stdout:9/167: write f4 [824004,40013] 0 2024-03-24T00:48:40.372 INFO:tasks.workunit.client.1.smithi088.stdout:0/146: dread f3 [0,4194304] 0 2024-03-24T00:48:40.373 INFO:tasks.workunit.client.1.smithi088.stdout:0/147: readlink d9/dc/de/l2d 0 2024-03-24T00:48:40.373 INFO:tasks.workunit.client.1.smithi088.stdout:4/170: mkdir d5/d13/d18/d24/d3a 0 2024-03-24T00:48:40.374 INFO:tasks.workunit.client.0.smithi038.stdout:3/161: rename d3/f32 to d3/d9/dc/df/f3a 0 2024-03-24T00:48:40.374 INFO:tasks.workunit.client.1.smithi088.stdout:8/138: dwrite d9/dc/d11/f2f [0,4194304] 0 2024-03-24T00:48:40.374 INFO:tasks.workunit.client.1.smithi088.stdout:8/139: fdatasync d9/f28 0 2024-03-24T00:48:40.374 INFO:tasks.workunit.client.1.smithi088.stdout:8/140: creat d9/dc/d11/f31 x:0 0 0 2024-03-24T00:48:40.375 INFO:tasks.workunit.client.1.smithi088.stdout:8/141: creat d9/f32 x:0 0 0 2024-03-24T00:48:40.377 INFO:tasks.workunit.client.1.smithi088.stdout:0/148: link d9/cb d9/dc/de/c34 0 2024-03-24T00:48:40.377 INFO:tasks.workunit.client.1.smithi088.stdout:0/149: read - d9/f1c zero size 2024-03-24T00:48:40.378 INFO:tasks.workunit.client.0.smithi038.stdout:2/162: write f2 [3576858,82918] 0 2024-03-24T00:48:40.378 INFO:tasks.workunit.client.0.smithi038.stdout:2/163: chown d4/d27 1362 1 2024-03-24T00:48:40.379 INFO:tasks.workunit.client.1.smithi088.stdout:4/171: rmdir d5/d13/d18/d24/d3a 0 2024-03-24T00:48:40.384 INFO:tasks.workunit.client.0.smithi038.stdout:3/162: rename d3/d9/dc/df/d14/d19/c35 to d3/d30/c3b 0 2024-03-24T00:48:40.384 INFO:tasks.workunit.client.0.smithi038.stdout:3/163: creat d3/d9/f3c x:0 0 0 2024-03-24T00:48:40.388 INFO:tasks.workunit.client.0.smithi038.stdout:8/159: dwrite d0/f12 [0,4194304] 0 2024-03-24T00:48:40.393 INFO:tasks.workunit.client.0.smithi038.stdout:3/164: creat d3/f3d x:0 0 0 2024-03-24T00:48:40.393 INFO:tasks.workunit.client.0.smithi038.stdout:3/165: fsync d3/f5 0 2024-03-24T00:48:40.394 INFO:tasks.workunit.client.0.smithi038.stdout:3/166: creat d3/d9/f3e x:0 0 0 2024-03-24T00:48:40.401 INFO:tasks.workunit.client.1.smithi088.stdout:5/145: dwrite d1/f31 [0,4194304] 0 2024-03-24T00:48:40.401 INFO:tasks.workunit.client.1.smithi088.stdout:2/123: dwrite db/dc/f1b [0,4194304] 0 2024-03-24T00:48:40.402 INFO:tasks.workunit.client.1.smithi088.stdout:9/146: dwrite d1/f7 [0,4194304] 0 2024-03-24T00:48:40.402 INFO:tasks.workunit.client.1.smithi088.stdout:9/147: chown d1/d15/d21 9215059 1 2024-03-24T00:48:40.404 INFO:tasks.workunit.client.1.smithi088.stdout:8/142: rename l4 to d9/dc/d11/d24/l33 0 2024-03-24T00:48:40.407 INFO:tasks.workunit.client.1.smithi088.stdout:0/150: creat d9/d20/f35 x:0 0 0 2024-03-24T00:48:40.408 INFO:tasks.workunit.client.1.smithi088.stdout:3/120: dread d3/d6/fb [0,4194304] 0 2024-03-24T00:48:40.408 INFO:tasks.workunit.client.1.smithi088.stdout:3/121: write f1 [2055397,22179] 0 2024-03-24T00:48:40.408 INFO:tasks.workunit.client.1.smithi088.stdout:3/122: write d3/de/df/f15 [821254,127762] 0 2024-03-24T00:48:40.409 INFO:tasks.workunit.client.0.smithi038.stdout:4/152: dwrite da/dc/dd/f21 [0,4194304] 0 2024-03-24T00:48:40.409 INFO:tasks.workunit.client.0.smithi038.stdout:4/153: read da/dc/dd/f2b [33365,87336] 0 2024-03-24T00:48:40.409 INFO:tasks.workunit.client.0.smithi038.stdout:8/160: unlink d0/f12 0 2024-03-24T00:48:40.409 INFO:tasks.workunit.client.0.smithi038.stdout:7/126: dread d2/f4 [0,4194304] 0 2024-03-24T00:48:40.409 INFO:tasks.workunit.client.0.smithi038.stdout:7/127: fsync d2/d9/da/d21/f27 0 2024-03-24T00:48:40.436 INFO:tasks.workunit.client.1.smithi088.stdout:7/141: dwrite d3/f2c [0,4194304] 0 2024-03-24T00:48:40.436 INFO:tasks.workunit.client.1.smithi088.stdout:7/142: creat d3/f2f x:0 0 0 2024-03-24T00:48:40.437 INFO:tasks.workunit.client.1.smithi088.stdout:7/143: creat d3/f30 x:0 0 0 2024-03-24T00:48:40.439 INFO:tasks.workunit.client.0.smithi038.stdout:7/128: mknod d2/d9/da/d23/d2b/c2f 0 2024-03-24T00:48:40.439 INFO:tasks.workunit.client.0.smithi038.stdout:9/168: getdents d12 0 2024-03-24T00:48:40.440 INFO:tasks.workunit.client.0.smithi038.stdout:7/129: creat d2/d9/da/d21/f30 x:0 0 0 2024-03-24T00:48:40.440 INFO:tasks.workunit.client.0.smithi038.stdout:8/161: dread d0/d15/f17 [0,4194304] 0 2024-03-24T00:48:40.446 INFO:tasks.workunit.client.1.smithi088.stdout:8/143: creat d9/dc/d11/d1b/f34 x:0 0 0 2024-03-24T00:48:40.448 INFO:tasks.workunit.client.1.smithi088.stdout:4/172: dwrite d5/d16/f2e [0,4194304] 0 2024-03-24T00:48:40.456 INFO:tasks.workunit.client.1.smithi088.stdout:9/148: rename d1/d10/d1f to d1/d10/d11/d28 0 2024-03-24T00:48:40.456 INFO:tasks.workunit.client.1.smithi088.stdout:9/149: write d1/d10/f18 [2169672,20851] 0 2024-03-24T00:48:40.469 INFO:tasks.workunit.client.1.smithi088.stdout:3/123: mkdir d3/d6/d27 0 2024-03-24T00:48:40.472 INFO:tasks.workunit.client.1.smithi088.stdout:5/146: dwrite d1/dd/f23 [0,4194304] 0 2024-03-24T00:48:40.472 INFO:tasks.workunit.client.1.smithi088.stdout:5/147: write d1/ff [1374882,49705] 0 2024-03-24T00:48:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:7/130: stat d2/d9/df/c10 0 2024-03-24T00:48:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:7/131: getdents d2/d2c 0 2024-03-24T00:48:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:7/132: chown d2/d9/da/dc 147108791 1 2024-03-24T00:48:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:7/133: creat d2/d9/da/d21/f31 x:0 0 0 2024-03-24T00:48:40.482 INFO:tasks.workunit.client.1.smithi088.stdout:2/124: dwrite db/dd/f17 [0,4194304] 0 2024-03-24T00:48:40.483 INFO:tasks.workunit.client.0.smithi038.stdout:9/169: truncate d12/d1f/f2b 538536 0 2024-03-24T00:48:40.489 INFO:tasks.workunit.client.0.smithi038.stdout:6/177: dwrite d1/f8 [0,4194304] 0 2024-03-24T00:48:40.503 INFO:tasks.workunit.client.0.smithi038.stdout:7/134: creat d2/d2c/f32 x:0 0 0 2024-03-24T00:48:40.503 INFO:tasks.workunit.client.0.smithi038.stdout:7/135: dread - d2/d9/da/d21/f30 zero size 2024-03-24T00:48:40.503 INFO:tasks.workunit.client.0.smithi038.stdout:2/164: dwrite d4/d18/d26/f29 [0,4194304] 0 2024-03-24T00:48:40.510 INFO:tasks.workunit.client.1.smithi088.stdout:7/144: mkdir d3/d31 0 2024-03-24T00:48:40.511 INFO:tasks.workunit.client.0.smithi038.stdout:2/165: dread d4/f10 [0,4194304] 0 2024-03-24T00:48:40.512 INFO:tasks.workunit.client.1.smithi088.stdout:4/173: dwrite d5/d17/f33 [0,4194304] 0 2024-03-24T00:48:40.515 INFO:tasks.workunit.client.1.smithi088.stdout:1/154: dwrite d6/d27/d1e/f1f [0,4194304] 0 2024-03-24T00:48:40.516 INFO:tasks.workunit.client.1.smithi088.stdout:8/144: symlink d9/l35 0 2024-03-24T00:48:40.516 INFO:tasks.workunit.client.1.smithi088.stdout:8/145: chown l8 442553296 1 2024-03-24T00:48:40.517 INFO:tasks.workunit.client.0.smithi038.stdout:3/167: dwrite d3/d9/f3c [0,4194304] 0 2024-03-24T00:48:40.517 INFO:tasks.workunit.client.0.smithi038.stdout:3/168: write d3/f3d [383216,127114] 0 2024-03-24T00:48:40.517 INFO:tasks.workunit.client.0.smithi038.stdout:6/178: creat d1/da/f33 x:0 0 0 2024-03-24T00:48:40.527 INFO:tasks.workunit.client.0.smithi038.stdout:2/166: creat d4/d27/d2f/d30/f3b x:0 0 0 2024-03-24T00:48:40.527 INFO:tasks.workunit.client.0.smithi038.stdout:2/167: fdatasync d4/d18/f31 0 2024-03-24T00:48:40.527 INFO:tasks.workunit.client.0.smithi038.stdout:2/168: fsync d4/d27/f39 0 2024-03-24T00:48:40.529 INFO:tasks.workunit.client.0.smithi038.stdout:2/169: write d4/f7 [1151339,125327] 0 2024-03-24T00:48:40.529 INFO:tasks.workunit.client.0.smithi038.stdout:4/154: dwrite da/d12/f2c [0,4194304] 0 2024-03-24T00:48:40.534 INFO:tasks.workunit.client.1.smithi088.stdout:9/150: truncate d1/d10/d11/f1b 202510 0 2024-03-24T00:48:40.534 INFO:tasks.workunit.client.0.smithi038.stdout:0/126: sync 2024-03-24T00:48:40.534 INFO:tasks.workunit.client.0.smithi038.stdout:0/127: write d0/f2 [1215899,113332] 0 2024-03-24T00:48:40.534 INFO:tasks.workunit.client.0.smithi038.stdout:0/128: chown d0/d12/d15/c18 25364 1 2024-03-24T00:48:40.534 INFO:tasks.workunit.client.0.smithi038.stdout:0/129: fsync d0/d12/d15/d1a/f1e 0 2024-03-24T00:48:40.534 INFO:tasks.workunit.client.0.smithi038.stdout:0/130: stat d0/cb 0 2024-03-24T00:48:40.536 INFO:tasks.workunit.client.1.smithi088.stdout:3/124: creat d3/d1a/d1e/f28 x:0 0 0 2024-03-24T00:48:40.538 INFO:tasks.workunit.client.0.smithi038.stdout:3/169: unlink d3/d9/dc/f1f 0 2024-03-24T00:48:40.541 INFO:tasks.workunit.client.1.smithi088.stdout:2/125: dwrite db/dd/f17 [0,4194304] 0 2024-03-24T00:48:40.541 INFO:tasks.workunit.client.1.smithi088.stdout:5/148: dwrite d1/dd/f15 [0,4194304] 0 2024-03-24T00:48:40.549 INFO:tasks.workunit.client.0.smithi038.stdout:6/179: dread d1/da/d1b/d1d/f2c [0,4194304] 0 2024-03-24T00:48:40.554 INFO:tasks.workunit.client.1.smithi088.stdout:3/125: dread f2 [0,4194304] 0 2024-03-24T00:48:40.554 INFO:tasks.workunit.client.0.smithi038.stdout:8/162: dwrite d0/f32 [0,4194304] 0 2024-03-24T00:48:40.563 INFO:tasks.workunit.client.0.smithi038.stdout:3/170: getdents d3/d2a 0 2024-03-24T00:48:40.564 INFO:tasks.workunit.client.1.smithi088.stdout:4/174: rmdir d5/d16/d1a/d22 39 2024-03-24T00:48:40.567 INFO:tasks.workunit.client.1.smithi088.stdout:1/155: rmdir d6/d24 0 2024-03-24T00:48:40.570 INFO:tasks.workunit.client.1.smithi088.stdout:7/145: dwrite d3/f29 [0,4194304] 0 2024-03-24T00:48:40.570 INFO:tasks.workunit.client.1.smithi088.stdout:7/146: write d3/f30 [1045509,51871] 0 2024-03-24T00:48:40.572 INFO:tasks.workunit.client.0.smithi038.stdout:9/170: rename d12/d1f to d12/d28/d3a 0 2024-03-24T00:48:40.572 INFO:tasks.workunit.client.0.smithi038.stdout:9/171: write d12/d1a/f20 [4208540,28052] 0 2024-03-24T00:48:40.572 INFO:tasks.workunit.client.0.smithi038.stdout:9/172: fsync d12/f35 0 2024-03-24T00:48:40.573 INFO:tasks.workunit.client.1.smithi088.stdout:8/146: mkdir d9/dc/d11/d24/d29/d36 0 2024-03-24T00:48:40.573 INFO:tasks.workunit.client.1.smithi088.stdout:8/147: chown d9/f28 988853 1 2024-03-24T00:48:40.573 INFO:tasks.workunit.client.1.smithi088.stdout:8/148: dread - d9/dc/d11/f31 zero size 2024-03-24T00:48:40.573 INFO:tasks.workunit.client.1.smithi088.stdout:8/149: creat d9/f37 x:0 0 0 2024-03-24T00:48:40.574 INFO:tasks.workunit.client.0.smithi038.stdout:9/173: creat d12/d28/f3b x:0 0 0 2024-03-24T00:48:40.574 INFO:tasks.workunit.client.0.smithi038.stdout:9/174: truncate d12/f35 372953 0 2024-03-24T00:48:40.576 INFO:tasks.workunit.client.0.smithi038.stdout:3/171: read d3/d9/f16 [17145,74897] 0 2024-03-24T00:48:40.576 INFO:tasks.workunit.client.1.smithi088.stdout:2/126: symlink db/dc/l26 0 2024-03-24T00:48:40.576 INFO:tasks.workunit.client.1.smithi088.stdout:2/127: write db/f20 [2529256,65277] 0 2024-03-24T00:48:40.578 INFO:tasks.workunit.client.0.smithi038.stdout:9/175: creat d12/d1a/f3c x:0 0 0 2024-03-24T00:48:40.580 INFO:tasks.workunit.client.1.smithi088.stdout:6/189: sync 2024-03-24T00:48:40.582 INFO:tasks.workunit.client.1.smithi088.stdout:5/149: truncate d1/fb 1538507 0 2024-03-24T00:48:40.583 INFO:tasks.workunit.client.0.smithi038.stdout:8/163: dread d0/d10/d19/f1b [0,4194304] 0 2024-03-24T00:48:40.583 INFO:tasks.workunit.client.0.smithi038.stdout:8/164: write d0/d1d/f22 [403793,81602] 0 2024-03-24T00:48:40.583 INFO:tasks.workunit.client.0.smithi038.stdout:9/176: stat d12/d28/d3a/c37 0 2024-03-24T00:48:40.584 INFO:tasks.workunit.client.1.smithi088.stdout:4/175: write d5/d17/f1c [1472746,128765] 0 2024-03-24T00:48:40.584 INFO:tasks.workunit.client.1.smithi088.stdout:4/176: truncate d5/d13/d18/f32 386678 0 2024-03-24T00:48:40.584 INFO:tasks.workunit.client.0.smithi038.stdout:2/170: dwrite d4/fa [0,4194304] 0 2024-03-24T00:48:40.586 INFO:tasks.workunit.client.0.smithi038.stdout:0/131: dwrite d0/d12/d15/f16 [0,4194304] 0 2024-03-24T00:48:40.588 INFO:tasks.workunit.client.1.smithi088.stdout:3/126: truncate f2 3542169 0 2024-03-24T00:48:40.590 INFO:tasks.workunit.client.0.smithi038.stdout:7/136: rename d2/d9/da/d21 to d2/d9/df/d33 0 2024-03-24T00:48:40.592 INFO:tasks.workunit.client.1.smithi088.stdout:7/147: dread d3/f2c [0,4194304] 0 2024-03-24T00:48:40.594 INFO:tasks.workunit.client.1.smithi088.stdout:1/156: mknod d6/d27/c2e 0 2024-03-24T00:48:40.601 INFO:tasks.workunit.client.0.smithi038.stdout:9/177: unlink d12/d1a/c1c 0 2024-03-24T00:48:40.603 INFO:tasks.workunit.client.1.smithi088.stdout:5/150: write d1/f31 [3972225,47662] 0 2024-03-24T00:48:40.603 INFO:tasks.workunit.client.0.smithi038.stdout:8/165: write d0/f16 [2465377,30449] 0 2024-03-24T00:48:40.604 INFO:tasks.workunit.client.1.smithi088.stdout:7/148: write d3/f1d [728440,26826] 0 2024-03-24T00:48:40.604 INFO:tasks.workunit.client.1.smithi088.stdout:2/128: write db/dd/f17 [1985235,20328] 0 2024-03-24T00:48:40.604 INFO:tasks.workunit.client.1.smithi088.stdout:7/149: write d3/f28 [940826,71381] 0 2024-03-24T00:48:40.604 INFO:tasks.workunit.client.1.smithi088.stdout:2/129: write db/dd/f1f [282684,67206] 0 2024-03-24T00:48:40.604 INFO:tasks.workunit.client.1.smithi088.stdout:2/130: creat db/dd/f27 x:0 0 0 2024-03-24T00:48:40.605 INFO:tasks.workunit.client.1.smithi088.stdout:8/150: mknod d9/c38 0 2024-03-24T00:48:40.606 INFO:tasks.workunit.client.1.smithi088.stdout:6/190: dread db/d1f/f34 [0,4194304] 0 2024-03-24T00:48:40.606 INFO:tasks.workunit.client.1.smithi088.stdout:6/191: write db/f35 [330074,69110] 0 2024-03-24T00:48:40.606 INFO:tasks.workunit.client.1.smithi088.stdout:6/192: dread - db/f38 zero size 2024-03-24T00:48:40.609 INFO:tasks.workunit.client.1.smithi088.stdout:4/177: mknod d5/d16/d1a/d22/d2f/c3b 0 2024-03-24T00:48:40.610 INFO:tasks.workunit.client.1.smithi088.stdout:4/178: read d5/d7/dd/ff [4266457,70195] 0 2024-03-24T00:48:40.612 INFO:tasks.workunit.client.1.smithi088.stdout:3/127: truncate d3/de/df/f15 3332392 0 2024-03-24T00:48:40.612 INFO:tasks.workunit.client.1.smithi088.stdout:3/128: chown d3/d1a/d1c 133675436 1 2024-03-24T00:48:40.614 INFO:tasks.workunit.client.0.smithi038.stdout:5/134: sync 2024-03-24T00:48:40.614 INFO:tasks.workunit.client.0.smithi038.stdout:5/135: chown d6/d19/d20/c25 911 1 2024-03-24T00:48:40.614 INFO:tasks.workunit.client.0.smithi038.stdout:5/136: readlink d6/d16/d14/l1d 0 2024-03-24T00:48:40.614 INFO:tasks.workunit.client.0.smithi038.stdout:5/137: readlink d6/d16/l1a 0 2024-03-24T00:48:40.616 INFO:tasks.workunit.client.0.smithi038.stdout:1/123: sync 2024-03-24T00:48:40.617 INFO:tasks.workunit.client.1.smithi088.stdout:1/157: mkdir d6/d2f 0 2024-03-24T00:48:40.621 INFO:tasks.workunit.client.0.smithi038.stdout:6/180: dwrite d1/da/d1b/f20 [0,4194304] 0 2024-03-24T00:48:40.621 INFO:tasks.workunit.client.0.smithi038.stdout:3/172: dwrite d3/f25 [0,4194304] 0 2024-03-24T00:48:40.623 INFO:tasks.workunit.client.1.smithi088.stdout:7/150: mkdir d3/d32 0 2024-03-24T00:48:40.625 INFO:tasks.workunit.client.1.smithi088.stdout:0/151: sync 2024-03-24T00:48:40.625 INFO:tasks.workunit.client.1.smithi088.stdout:0/152: fdatasync d9/dc/de/f32 0 2024-03-24T00:48:40.626 INFO:tasks.workunit.client.0.smithi038.stdout:8/166: rename d0/d15/d2c/f34 to d0/d10/d19/d29/f40 0 2024-03-24T00:48:40.626 INFO:tasks.workunit.client.0.smithi038.stdout:8/167: chown d0/cc 2925 1 2024-03-24T00:48:40.627 INFO:tasks.workunit.client.0.smithi038.stdout:8/168: write d0/d1d/d20/f23 [3098105,86307] 0 2024-03-24T00:48:40.627 INFO:tasks.workunit.client.0.smithi038.stdout:8/169: write d0/f16 [6005589,100966] 0 2024-03-24T00:48:40.630 INFO:tasks.workunit.client.0.smithi038.stdout:2/171: rmdir d4 39 2024-03-24T00:48:40.634 INFO:tasks.workunit.client.1.smithi088.stdout:8/151: truncate d9/f28 2615557 0 2024-03-24T00:48:40.634 INFO:tasks.workunit.client.1.smithi088.stdout:8/152: dread - d9/dc/d11/d24/d29/f2c zero size 2024-03-24T00:48:40.635 INFO:tasks.workunit.client.1.smithi088.stdout:8/153: write d9/dc/d11/d1b/f34 [46102,79104] 0 2024-03-24T00:48:40.635 INFO:tasks.workunit.client.1.smithi088.stdout:4/179: dread d5/d13/d18/f32 [0,4194304] 0 2024-03-24T00:48:40.636 INFO:tasks.workunit.client.1.smithi088.stdout:6/193: truncate db/f19 665866 0 2024-03-24T00:48:40.642 INFO:tasks.workunit.client.0.smithi038.stdout:9/178: link fc d12/d28/d3a/d26/f3d 0 2024-03-24T00:48:40.642 INFO:tasks.workunit.client.0.smithi038.stdout:9/179: readlink d12/l13 0 2024-03-24T00:48:40.642 INFO:tasks.workunit.client.1.smithi088.stdout:3/129: getdents d3/d1a 0 2024-03-24T00:48:40.645 INFO:tasks.workunit.client.1.smithi088.stdout:1/158: symlink d6/d27/d1e/l30 0 2024-03-24T00:48:40.645 INFO:tasks.workunit.client.0.smithi038.stdout:5/138: truncate f5 133897 0 2024-03-24T00:48:40.650 INFO:tasks.workunit.client.1.smithi088.stdout:5/151: dwrite d1/d19/f30 [4194304,4194304] 0 2024-03-24T00:48:40.653 INFO:tasks.workunit.client.0.smithi038.stdout:1/124: mknod da/dd/d22/c29 0 2024-03-24T00:48:40.657 INFO:tasks.workunit.client.1.smithi088.stdout:7/151: unlink d3/fa 0 2024-03-24T00:48:40.657 INFO:tasks.workunit.client.1.smithi088.stdout:7/152: write d3/f2a [748100,123070] 0 2024-03-24T00:48:40.657 INFO:tasks.workunit.client.1.smithi088.stdout:7/153: chown d3/l1c 194776 1 2024-03-24T00:48:40.657 INFO:tasks.workunit.client.1.smithi088.stdout:7/154: creat d3/f33 x:0 0 0 2024-03-24T00:48:40.662 INFO:tasks.workunit.client.1.smithi088.stdout:3/130: dread f1 [0,4194304] 0 2024-03-24T00:48:40.662 INFO:tasks.workunit.client.1.smithi088.stdout:3/131: creat d3/de/f29 x:0 0 0 2024-03-24T00:48:40.665 INFO:tasks.workunit.client.1.smithi088.stdout:0/153: truncate f3 10142 0 2024-03-24T00:48:40.665 INFO:tasks.workunit.client.1.smithi088.stdout:0/154: readlink d9/d20/d2a/l33 0 2024-03-24T00:48:40.665 INFO:tasks.workunit.client.1.smithi088.stdout:0/155: stat d9/dc/de/d19/c24 0 2024-03-24T00:48:40.668 INFO:tasks.workunit.client.1.smithi088.stdout:2/131: dwrite db/dd/f1f [0,4194304] 0 2024-03-24T00:48:40.668 INFO:tasks.workunit.client.1.smithi088.stdout:2/132: creat db/dd/f28 x:0 0 0 2024-03-24T00:48:40.668 INFO:tasks.workunit.client.1.smithi088.stdout:2/133: write db/dc/f11 [108430,42644] 0 2024-03-24T00:48:40.670 INFO:tasks.workunit.client.1.smithi088.stdout:8/154: mkdir d9/dc/d11/d24/d29/d39 0 2024-03-24T00:48:40.675 INFO:tasks.workunit.client.1.smithi088.stdout:4/180: unlink d5/d13/f36 0 2024-03-24T00:48:40.678 INFO:tasks.workunit.client.1.smithi088.stdout:6/194: rename db/f1c to db/f3a 0 2024-03-24T00:48:40.678 INFO:tasks.workunit.client.1.smithi088.stdout:6/195: creat db/dd/f3b x:0 0 0 2024-03-24T00:48:40.681 INFO:tasks.workunit.client.0.smithi038.stdout:7/137: dwrite d2/d9/df/d33/f30 [0,4194304] 0 2024-03-24T00:48:40.681 INFO:tasks.workunit.client.0.smithi038.stdout:6/181: unlink d1/da/lf 0 2024-03-24T00:48:40.681 INFO:tasks.workunit.client.0.smithi038.stdout:6/182: fsync d1/da/d1b/f26 0 2024-03-24T00:48:40.681 INFO:tasks.workunit.client.0.smithi038.stdout:6/183: creat d1/da/f34 x:0 0 0 2024-03-24T00:48:40.681 INFO:tasks.workunit.client.0.smithi038.stdout:6/184: write d1/da/f33 [638223,105612] 0 2024-03-24T00:48:40.683 INFO:tasks.workunit.client.1.smithi088.stdout:1/159: rename d6/d27/l2b to d6/d27/d1e/l31 0 2024-03-24T00:48:40.689 INFO:tasks.workunit.client.0.smithi038.stdout:3/173: unlink d3/d2a/c22 0 2024-03-24T00:48:40.689 INFO:tasks.workunit.client.0.smithi038.stdout:3/174: readlink d3/d9/dc/df/l24 0 2024-03-24T00:48:40.689 INFO:tasks.workunit.client.0.smithi038.stdout:7/138: write d2/d9/fb [527238,56757] 0 2024-03-24T00:48:40.689 INFO:tasks.workunit.client.0.smithi038.stdout:7/139: truncate d2/d9/da/f14 4707340 0 2024-03-24T00:48:40.690 INFO:tasks.workunit.client.0.smithi038.stdout:9/180: dwrite d12/d1a/f20 [0,4194304] 0 2024-03-24T00:48:40.693 INFO:tasks.workunit.client.1.smithi088.stdout:6/196: dread db/dd/f33 [0,4194304] 0 2024-03-24T00:48:40.694 INFO:tasks.workunit.client.1.smithi088.stdout:7/155: dread d3/f30 [0,4194304] 0 2024-03-24T00:48:40.696 INFO:tasks.workunit.client.0.smithi038.stdout:8/170: symlink d0/d1d/d20/l41 0 2024-03-24T00:48:40.696 INFO:tasks.workunit.client.0.smithi038.stdout:8/171: chown d0/d1d/d20/l41 227517527 1 2024-03-24T00:48:40.696 INFO:tasks.workunit.client.0.smithi038.stdout:8/172: stat d0/f24 0 2024-03-24T00:48:40.696 INFO:tasks.workunit.client.0.smithi038.stdout:8/173: write d0/d1d/f3e [9991,121314] 0 2024-03-24T00:48:40.704 INFO:tasks.workunit.client.0.smithi038.stdout:2/172: creat d4/d27/d2f/f3c x:0 0 0 2024-03-24T00:48:40.713 INFO:tasks.workunit.client.0.smithi038.stdout:7/140: write d2/d9/d16/f19 [142607,83917] 0 2024-03-24T00:48:40.713 INFO:tasks.workunit.client.0.smithi038.stdout:7/141: dread - d2/d9/da/dc/f1c zero size 2024-03-24T00:48:40.713 INFO:tasks.workunit.client.0.smithi038.stdout:7/142: fdatasync d2/d9/da/f14 0 2024-03-24T00:48:40.717 INFO:tasks.workunit.client.0.smithi038.stdout:1/125: dwrite da/dd/d10/f1f [0,4194304] 0 2024-03-24T00:48:40.717 INFO:tasks.workunit.client.0.smithi038.stdout:1/126: write da/dd/d22/f25 [560907,10648] 0 2024-03-24T00:48:40.717 INFO:tasks.workunit.client.0.smithi038.stdout:1/127: creat da/df/f2a x:0 0 0 2024-03-24T00:48:40.721 INFO:tasks.workunit.client.0.smithi038.stdout:6/185: symlink d1/l35 0 2024-03-24T00:48:40.722 INFO:tasks.workunit.client.0.smithi038.stdout:7/143: read d2/f1a [93026,16884] 0 2024-03-24T00:48:40.722 INFO:tasks.workunit.client.0.smithi038.stdout:7/144: stat d2/d9/da/dc/c17 0 2024-03-24T00:48:40.724 INFO:tasks.workunit.client.0.smithi038.stdout:9/181: rename d12/d28/f3b to d12/f3e 0 2024-03-24T00:48:40.725 INFO:tasks.workunit.client.1.smithi088.stdout:5/152: dwrite d1/dd/f23 [0,4194304] 0 2024-03-24T00:48:40.725 INFO:tasks.workunit.client.0.smithi038.stdout:9/182: write d12/f18 [3549969,106853] 0 2024-03-24T00:48:40.727 INFO:tasks.workunit.client.0.smithi038.stdout:2/173: link d4/f24 d4/d18/d26/f3d 0 2024-03-24T00:48:40.735 INFO:tasks.workunit.client.1.smithi088.stdout:0/156: symlink d9/d15/l36 0 2024-03-24T00:48:40.738 INFO:tasks.workunit.client.1.smithi088.stdout:2/134: symlink db/d21/l29 0 2024-03-24T00:48:40.741 INFO:tasks.workunit.client.1.smithi088.stdout:6/197: dwrite f8 [0,4194304] 0 2024-03-24T00:48:40.742 INFO:tasks.workunit.client.1.smithi088.stdout:8/155: link d9/dc/d11/l1d d9/dc/l3a 0 2024-03-24T00:48:40.742 INFO:tasks.workunit.client.1.smithi088.stdout:6/198: dread - db/d1f/f39 zero size 2024-03-24T00:48:40.742 INFO:tasks.workunit.client.1.smithi088.stdout:8/156: truncate d9/dc/d11/f31 808969 0 2024-03-24T00:48:40.742 INFO:tasks.workunit.client.1.smithi088.stdout:8/157: write f2 [115251,97818] 0 2024-03-24T00:48:40.743 INFO:tasks.workunit.client.0.smithi038.stdout:3/175: dwrite d3/f8 [0,4194304] 0 2024-03-24T00:48:40.743 INFO:tasks.workunit.client.1.smithi088.stdout:4/181: unlink c0 0 2024-03-24T00:48:40.745 INFO:tasks.workunit.client.0.smithi038.stdout:6/186: symlink d1/l36 0 2024-03-24T00:48:40.748 INFO:tasks.workunit.client.0.smithi038.stdout:7/145: unlink d2/d9/l2e 0 2024-03-24T00:48:40.748 INFO:tasks.workunit.client.0.smithi038.stdout:7/146: write d2/d9/f20 [565407,75326] 0 2024-03-24T00:48:40.748 INFO:tasks.workunit.client.0.smithi038.stdout:7/147: dread - d2/d9/df/d33/f27 zero size 2024-03-24T00:48:40.748 INFO:tasks.workunit.client.0.smithi038.stdout:7/148: getdents d2/d9/d16 0 2024-03-24T00:48:40.749 INFO:tasks.workunit.client.1.smithi088.stdout:1/160: creat d6/d2f/f32 x:0 0 0 2024-03-24T00:48:40.749 INFO:tasks.workunit.client.1.smithi088.stdout:1/161: creat d6/d23/f33 x:0 0 0 2024-03-24T00:48:40.749 INFO:tasks.workunit.client.1.smithi088.stdout:1/162: creat d6/d23/f34 x:0 0 0 2024-03-24T00:48:40.749 INFO:tasks.workunit.client.1.smithi088.stdout:1/163: stat d6/ff 0 2024-03-24T00:48:40.749 INFO:tasks.workunit.client.1.smithi088.stdout:1/164: stat d6/d27/d1e/f22 0 2024-03-24T00:48:40.749 INFO:tasks.workunit.client.1.smithi088.stdout:1/165: chown d6/l12 41 1 2024-03-24T00:48:40.750 INFO:tasks.workunit.client.0.smithi038.stdout:9/183: creat d12/d28/d3a/f3f x:0 0 0 2024-03-24T00:48:40.752 INFO:tasks.workunit.client.0.smithi038.stdout:2/174: mkdir d4/d27/d2f/d30/d3e 0 2024-03-24T00:48:40.752 INFO:tasks.workunit.client.1.smithi088.stdout:7/156: mkdir d3/d32/d34 0 2024-03-24T00:48:40.754 INFO:tasks.workunit.client.0.smithi038.stdout:1/128: getdents da 0 2024-03-24T00:48:40.755 INFO:tasks.workunit.client.1.smithi088.stdout:6/199: dread f8 [0,4194304] 0 2024-03-24T00:48:40.756 INFO:tasks.workunit.client.0.smithi038.stdout:3/176: rename d3/d9/dc/df/d14/c2d to d3/d9/dc/df/d14/d19/c3f 0 2024-03-24T00:48:40.758 INFO:tasks.workunit.client.0.smithi038.stdout:6/187: mkdir d1/d37 0 2024-03-24T00:48:40.762 INFO:tasks.workunit.client.1.smithi088.stdout:5/153: mknod d1/c37 0 2024-03-24T00:48:40.764 INFO:tasks.workunit.client.0.smithi038.stdout:7/149: symlink d2/d2c/l34 0 2024-03-24T00:48:40.765 INFO:tasks.workunit.client.0.smithi038.stdout:7/150: fsync d2/d9/df/f29 0 2024-03-24T00:48:40.765 INFO:tasks.workunit.client.0.smithi038.stdout:9/184: stat f6 0 2024-03-24T00:48:40.765 INFO:tasks.workunit.client.0.smithi038.stdout:9/185: write f4 [5348707,75034] 0 2024-03-24T00:48:40.765 INFO:tasks.workunit.client.0.smithi038.stdout:9/186: creat d12/d28/d3a/f40 x:0 0 0 2024-03-24T00:48:40.766 INFO:tasks.workunit.client.1.smithi088.stdout:9/151: sync 2024-03-24T00:48:40.767 INFO:tasks.workunit.client.0.smithi038.stdout:1/129: rename da/df/c19 to da/dd/c2b 0 2024-03-24T00:48:40.771 INFO:tasks.workunit.client.1.smithi088.stdout:0/157: symlink d9/d20/l37 0 2024-03-24T00:48:40.773 INFO:tasks.workunit.client.0.smithi038.stdout:2/175: dread d4/f2b [0,4194304] 0 2024-03-24T00:48:40.773 INFO:tasks.workunit.client.0.smithi038.stdout:2/176: chown d4/ff 6793912 1 2024-03-24T00:48:40.773 INFO:tasks.workunit.client.0.smithi038.stdout:2/177: fsync d4/d18/f2e 0 2024-03-24T00:48:40.773 INFO:tasks.workunit.client.0.smithi038.stdout:2/178: write d4/d27/f2a [631954,16508] 0 2024-03-24T00:48:40.774 INFO:tasks.workunit.client.1.smithi088.stdout:5/154: dread d1/dd/f23 [0,4194304] 0 2024-03-24T00:48:40.775 INFO:tasks.workunit.client.0.smithi038.stdout:3/177: mknod d3/d9/dc/df/c40 0 2024-03-24T00:48:40.784 INFO:tasks.workunit.client.1.smithi088.stdout:2/135: mknod db/d21/c2a 0 2024-03-24T00:48:40.786 INFO:tasks.workunit.client.1.smithi088.stdout:2/136: write db/fe [543617,116866] 0 2024-03-24T00:48:40.786 INFO:tasks.workunit.client.1.smithi088.stdout:2/137: creat db/dc/f2b x:0 0 0 2024-03-24T00:48:40.788 INFO:tasks.workunit.client.1.smithi088.stdout:8/158: unlink d9/dc/d11/d1b/l22 0 2024-03-24T00:48:40.788 INFO:tasks.workunit.client.1.smithi088.stdout:8/159: chown d9/dc/d11/f2f 1601674568 1 2024-03-24T00:48:40.790 INFO:tasks.workunit.client.0.smithi038.stdout:9/187: mkdir d12/d41 0 2024-03-24T00:48:40.792 INFO:tasks.workunit.client.0.smithi038.stdout:2/179: symlink d4/d27/l3f 0 2024-03-24T00:48:40.794 INFO:tasks.workunit.client.0.smithi038.stdout:6/188: write d1/f9 [983964,94644] 0 2024-03-24T00:48:40.797 INFO:tasks.workunit.client.0.smithi038.stdout:3/178: symlink d3/d9/dc/df/d14/l41 0 2024-03-24T00:48:40.804 INFO:tasks.workunit.client.0.smithi038.stdout:2/180: rename d4/f1c to d4/d18/d26/d37/f40 0 2024-03-24T00:48:40.804 INFO:tasks.workunit.client.0.smithi038.stdout:2/181: chown d4/d27/d2f 23 1 2024-03-24T00:48:40.812 INFO:tasks.workunit.client.0.smithi038.stdout:3/179: rename d3/d9/f3e to d3/d9/dc/df/d14/d19/f42 0 2024-03-24T00:48:40.814 INFO:tasks.workunit.client.1.smithi088.stdout:0/158: creat d9/d26/f38 x:0 0 0 2024-03-24T00:48:40.819 INFO:tasks.workunit.client.0.smithi038.stdout:6/189: dread d1/da/d1b/d1d/f2c [0,4194304] 0 2024-03-24T00:48:40.819 INFO:tasks.workunit.client.0.smithi038.stdout:6/190: chown d1/f10 9385 1 2024-03-24T00:48:40.819 INFO:tasks.workunit.client.0.smithi038.stdout:6/191: write d1/f9 [740040,97494] 0 2024-03-24T00:48:40.820 INFO:tasks.workunit.client.1.smithi088.stdout:2/138: rename db/dd/f27 to db/dc/f2c 0 2024-03-24T00:48:40.822 INFO:tasks.workunit.client.1.smithi088.stdout:1/166: dwrite d6/d23/f33 [0,4194304] 0 2024-03-24T00:48:40.822 INFO:tasks.workunit.client.1.smithi088.stdout:1/167: chown d6/d2f 195517 1 2024-03-24T00:48:40.826 INFO:tasks.workunit.client.1.smithi088.stdout:8/160: symlink d9/l3b 0 2024-03-24T00:48:40.826 INFO:tasks.workunit.client.1.smithi088.stdout:8/161: creat d9/f3c x:0 0 0 2024-03-24T00:48:40.826 INFO:tasks.workunit.client.1.smithi088.stdout:8/162: stat d9/dc/d11/d24/d29/d36 0 2024-03-24T00:48:40.826 INFO:tasks.workunit.client.1.smithi088.stdout:8/163: write d9/dc/d11/d1b/f34 [472222,5739] 0 2024-03-24T00:48:40.826 INFO:tasks.workunit.client.1.smithi088.stdout:8/164: fdatasync d9/f32 0 2024-03-24T00:48:40.827 INFO:tasks.workunit.client.1.smithi088.stdout:6/200: dwrite db/f35 [0,4194304] 0 2024-03-24T00:48:40.827 INFO:tasks.workunit.client.1.smithi088.stdout:2/139: rename db/dd/l16 to db/d21/l2d 0 2024-03-24T00:48:40.831 INFO:tasks.workunit.client.0.smithi038.stdout:7/151: dwrite d2/d9/da/f14 [0,4194304] 0 2024-03-24T00:48:40.831 INFO:tasks.workunit.client.0.smithi038.stdout:1/130: dwrite da/df/f2a [0,4194304] 0 2024-03-24T00:48:40.832 INFO:tasks.workunit.client.1.smithi088.stdout:1/168: mkdir d6/d27/d1e/d35 0 2024-03-24T00:48:40.832 INFO:tasks.workunit.client.1.smithi088.stdout:1/169: dread - d6/d23/f34 zero size 2024-03-24T00:48:40.833 INFO:tasks.workunit.client.1.smithi088.stdout:7/157: dread d3/f5 [0,4194304] 0 2024-03-24T00:48:40.835 INFO:tasks.workunit.client.1.smithi088.stdout:0/159: dread d9/dc/de/f14 [0,4194304] 0 2024-03-24T00:48:40.835 INFO:tasks.workunit.client.1.smithi088.stdout:0/160: readlink d9/d20/d2a/l33 0 2024-03-24T00:48:40.836 INFO:tasks.workunit.client.0.smithi038.stdout:7/152: dread d2/d9/da/f1d [0,4194304] 0 2024-03-24T00:48:40.845 INFO:tasks.workunit.client.1.smithi088.stdout:8/165: symlink d9/l3d 0 2024-03-24T00:48:40.846 INFO:tasks.workunit.client.0.smithi038.stdout:9/188: getdents d12/d28/d3a 0 2024-03-24T00:48:40.848 INFO:tasks.workunit.client.0.smithi038.stdout:1/131: mkdir da/dd/d2c 0 2024-03-24T00:48:40.856 INFO:tasks.workunit.client.0.smithi038.stdout:9/189: fdatasync f8 0 2024-03-24T00:48:40.858 INFO:tasks.workunit.client.1.smithi088.stdout:6/201: mkdir db/d1d/d23/d3c 0 2024-03-24T00:48:40.858 INFO:tasks.workunit.client.1.smithi088.stdout:6/202: chown db/d1f/d24/f31 6076 1 2024-03-24T00:48:40.858 INFO:tasks.workunit.client.1.smithi088.stdout:3/132: sync 2024-03-24T00:48:40.858 INFO:tasks.workunit.client.1.smithi088.stdout:3/133: readlink d3/d6/l1b 0 2024-03-24T00:48:40.859 INFO:tasks.workunit.client.0.smithi038.stdout:1/132: mkdir da/df/d2d 0 2024-03-24T00:48:40.862 INFO:tasks.workunit.client.1.smithi088.stdout:2/140: creat db/d21/f2e x:0 0 0 2024-03-24T00:48:40.864 INFO:tasks.workunit.client.0.smithi038.stdout:3/180: dread d3/f5 [0,4194304] 0 2024-03-24T00:48:40.864 INFO:tasks.workunit.client.0.smithi038.stdout:2/182: dwrite d4/d27/f39 [0,4194304] 0 2024-03-24T00:48:40.866 INFO:tasks.workunit.client.1.smithi088.stdout:1/170: rmdir d6/d27/d1e/d35 0 2024-03-24T00:48:40.867 INFO:tasks.workunit.client.1.smithi088.stdout:1/171: write d6/f19 [447933,72111] 0 2024-03-24T00:48:40.867 INFO:tasks.workunit.client.1.smithi088.stdout:1/172: fdatasync d6/ff 0 2024-03-24T00:48:40.867 INFO:tasks.workunit.client.1.smithi088.stdout:1/173: write f0 [4096838,107657] 0 2024-03-24T00:48:40.867 INFO:tasks.workunit.client.1.smithi088.stdout:1/174: fsync d6/d23/f29 0 2024-03-24T00:48:40.868 INFO:tasks.workunit.client.0.smithi038.stdout:9/190: creat d12/f42 x:0 0 0 2024-03-24T00:48:40.872 INFO:tasks.workunit.client.1.smithi088.stdout:2/141: dread db/dd/f17 [0,4194304] 0 2024-03-24T00:48:40.879 INFO:tasks.workunit.client.1.smithi088.stdout:8/166: dread d9/f20 [0,4194304] 0 2024-03-24T00:48:40.880 INFO:tasks.workunit.client.1.smithi088.stdout:6/203: mkdir db/d3d 0 2024-03-24T00:48:40.880 INFO:tasks.workunit.client.1.smithi088.stdout:6/204: fdatasync db/dd/f3b 0 2024-03-24T00:48:40.881 INFO:tasks.workunit.client.1.smithi088.stdout:3/134: rename d3/d12 to d3/d6/d19/d2a 0 2024-03-24T00:48:40.881 INFO:tasks.workunit.client.1.smithi088.stdout:3/135: creat d3/d6/f2b x:0 0 0 2024-03-24T00:48:40.881 INFO:tasks.workunit.client.1.smithi088.stdout:3/136: chown d3/de 43139 1 2024-03-24T00:48:40.881 INFO:tasks.workunit.client.1.smithi088.stdout:1/175: truncate f2 1322537 0 2024-03-24T00:48:40.881 INFO:tasks.workunit.client.1.smithi088.stdout:1/176: dread - d6/f2a zero size 2024-03-24T00:48:40.885 INFO:tasks.workunit.client.0.smithi038.stdout:2/183: read d4/f1d [1267693,71355] 0 2024-03-24T00:48:40.885 INFO:tasks.workunit.client.0.smithi038.stdout:1/133: unlink da/dd/l1e 0 2024-03-24T00:48:40.885 INFO:tasks.workunit.client.0.smithi038.stdout:1/134: dread - da/f23 zero size 2024-03-24T00:48:40.885 INFO:tasks.workunit.client.0.smithi038.stdout:2/184: write d4/d18/f1f [430868,95702] 0 2024-03-24T00:48:40.885 INFO:tasks.workunit.client.0.smithi038.stdout:1/135: write da/f23 [1012412,38615] 0 2024-03-24T00:48:40.885 INFO:tasks.workunit.client.0.smithi038.stdout:2/185: truncate d4/f1b 4707517 0 2024-03-24T00:48:40.885 INFO:tasks.workunit.client.0.smithi038.stdout:2/186: chown d4/d18/l1e 131 1 2024-03-24T00:48:40.885 INFO:tasks.workunit.client.0.smithi038.stdout:1/136: fdatasync da/df/f17 0 2024-03-24T00:48:40.885 INFO:tasks.workunit.client.0.smithi038.stdout:2/187: read d4/f7 [3005748,88602] 0 2024-03-24T00:48:40.891 INFO:tasks.workunit.client.1.smithi088.stdout:3/137: mknod d3/d1a/d1e/c2c 0 2024-03-24T00:48:40.895 INFO:tasks.workunit.client.1.smithi088.stdout:1/177: link d6/d27/l2c d6/d27/l36 0 2024-03-24T00:48:40.895 INFO:tasks.workunit.client.1.smithi088.stdout:1/178: read - d6/d27/d1e/f22 zero size 2024-03-24T00:48:40.895 INFO:tasks.workunit.client.1.smithi088.stdout:1/179: dread - d6/d23/f34 zero size 2024-03-24T00:48:40.895 INFO:tasks.workunit.client.1.smithi088.stdout:1/180: fsync d6/d27/d1e/f22 0 2024-03-24T00:48:40.896 INFO:tasks.workunit.client.1.smithi088.stdout:0/161: dwrite f3 [0,4194304] 0 2024-03-24T00:48:40.896 INFO:tasks.workunit.client.1.smithi088.stdout:0/162: write d9/f12 [489031,76785] 0 2024-03-24T00:48:40.896 INFO:tasks.workunit.client.1.smithi088.stdout:0/163: stat d9/dc 0 2024-03-24T00:48:40.896 INFO:tasks.workunit.client.1.smithi088.stdout:0/164: dread - d9/f1c zero size 2024-03-24T00:48:40.896 INFO:tasks.workunit.client.1.smithi088.stdout:0/165: creat d9/d20/f39 x:0 0 0 2024-03-24T00:48:40.899 INFO:tasks.workunit.client.0.smithi038.stdout:3/181: mknod d3/d9/dc/df/c43 0 2024-03-24T00:48:40.899 INFO:tasks.workunit.client.0.smithi038.stdout:3/182: write d3/d9/f2b [672435,15252] 0 2024-03-24T00:48:40.899 INFO:tasks.workunit.client.0.smithi038.stdout:3/183: chown d3/d9/dc/df/c43 93 1 2024-03-24T00:48:40.899 INFO:tasks.workunit.client.0.smithi038.stdout:3/184: dread - d3/d9/f1c zero size 2024-03-24T00:48:40.900 INFO:tasks.workunit.client.1.smithi088.stdout:3/138: creat d3/de/df/f2d x:0 0 0 2024-03-24T00:48:40.907 INFO:tasks.workunit.client.1.smithi088.stdout:3/139: unlink d3/c8 0 2024-03-24T00:48:40.908 INFO:tasks.workunit.client.0.smithi038.stdout:9/191: mknod d12/d28/d3a/c43 0 2024-03-24T00:48:40.912 INFO:tasks.workunit.client.0.smithi038.stdout:7/153: truncate d2/d9/d16/f19 1487057 0 2024-03-24T00:48:40.912 INFO:tasks.workunit.client.0.smithi038.stdout:7/154: creat d2/d9/df/f35 x:0 0 0 2024-03-24T00:48:40.920 INFO:tasks.workunit.client.0.smithi038.stdout:1/137: creat da/dd/d10/d14/f2e x:0 0 0 2024-03-24T00:48:40.921 INFO:tasks.workunit.client.1.smithi088.stdout:3/140: dread d3/de/df/f15 [0,4194304] 0 2024-03-24T00:48:40.921 INFO:tasks.workunit.client.1.smithi088.stdout:3/141: fsync d3/de/df/f15 0 2024-03-24T00:48:40.923 INFO:tasks.workunit.client.0.smithi038.stdout:2/188: link d4/d27/d2f/f3c d4/f41 0 2024-03-24T00:48:40.930 INFO:tasks.workunit.client.0.smithi038.stdout:2/189: mkdir d4/d27/d42 0 2024-03-24T00:48:40.931 INFO:tasks.workunit.client.1.smithi088.stdout:2/142: dwrite db/dc/f11 [12582912,4194304] 0 2024-03-24T00:48:40.936 INFO:tasks.workunit.client.1.smithi088.stdout:3/142: dread d3/d6/f11 [0,4194304] 0 2024-03-24T00:48:40.941 INFO:tasks.workunit.client.1.smithi088.stdout:2/143: dread db/fe [0,4194304] 0 2024-03-24T00:48:40.941 INFO:tasks.workunit.client.1.smithi088.stdout:2/144: write db/f18 [4614339,20292] 0 2024-03-24T00:48:40.942 INFO:tasks.workunit.client.1.smithi088.stdout:0/166: dread f3 [0,4194304] 0 2024-03-24T00:48:40.942 INFO:tasks.workunit.client.1.smithi088.stdout:7/158: dwrite d3/f33 [0,4194304] 0 2024-03-24T00:48:40.942 INFO:tasks.workunit.client.1.smithi088.stdout:0/167: dread - d9/d20/f35 zero size 2024-03-24T00:48:40.944 INFO:tasks.workunit.client.0.smithi038.stdout:9/192: dread f4 [0,4194304] 0 2024-03-24T00:48:40.958 INFO:tasks.workunit.client.1.smithi088.stdout:2/145: unlink db/dd/l13 0 2024-03-24T00:48:40.958 INFO:tasks.workunit.client.1.smithi088.stdout:2/146: creat db/dd/f2f x:0 0 0 2024-03-24T00:48:40.958 INFO:tasks.workunit.client.1.smithi088.stdout:2/147: creat db/dc/f30 x:0 0 0 2024-03-24T00:48:40.958 INFO:tasks.workunit.client.1.smithi088.stdout:2/148: write db/dd/f2f [765009,73005] 0 2024-03-24T00:48:40.962 INFO:tasks.workunit.client.1.smithi088.stdout:0/168: rename d9/d15/f1b to d9/f3a 0 2024-03-24T00:48:40.963 INFO:tasks.workunit.client.1.smithi088.stdout:0/169: write d9/f12 [1454041,130308] 0 2024-03-24T00:48:40.963 INFO:tasks.workunit.client.1.smithi088.stdout:8/167: dwrite d9/dc/d11/d1b/f34 [0,4194304] 0 2024-03-24T00:48:40.963 INFO:tasks.workunit.client.1.smithi088.stdout:8/168: creat d9/f3e x:0 0 0 2024-03-24T00:48:40.970 INFO:tasks.workunit.client.0.smithi038.stdout:1/138: dread da/dd/d10/f15 [0,4194304] 0 2024-03-24T00:48:40.972 INFO:tasks.workunit.client.1.smithi088.stdout:2/149: link c7 db/d21/c31 0 2024-03-24T00:48:40.976 INFO:tasks.workunit.client.1.smithi088.stdout:7/159: rename d3/f5 to d3/f35 0 2024-03-24T00:48:40.978 INFO:tasks.workunit.client.1.smithi088.stdout:1/181: dwrite f0 [4194304,4194304] 0 2024-03-24T00:48:40.982 INFO:tasks.workunit.client.1.smithi088.stdout:0/170: rename d9/f12 to d9/dc/de/d19/f3b 0 2024-03-24T00:48:40.982 INFO:tasks.workunit.client.1.smithi088.stdout:2/150: rename db/dc to db/dc/d32 22 2024-03-24T00:48:40.982 INFO:tasks.workunit.client.1.smithi088.stdout:0/171: creat d9/d26/f3c x:0 0 0 2024-03-24T00:48:40.982 INFO:tasks.workunit.client.1.smithi088.stdout:0/172: creat d9/dc/de/f3d x:0 0 0 2024-03-24T00:48:40.989 INFO:tasks.workunit.client.1.smithi088.stdout:1/182: dread d6/f19 [0,4194304] 0 2024-03-24T00:48:40.991 INFO:tasks.workunit.client.1.smithi088.stdout:8/169: mknod d9/dc/d11/d24/d29/d36/c3f 0 2024-03-24T00:48:40.991 INFO:tasks.workunit.client.1.smithi088.stdout:8/170: read f2 [1852484,102346] 0 2024-03-24T00:48:40.991 INFO:tasks.workunit.client.1.smithi088.stdout:8/171: write d9/f17 [5234294,113982] 0 2024-03-24T00:48:40.994 INFO:tasks.workunit.client.0.smithi038.stdout:1/139: dread da/dd/d10/f15 [0,4194304] 0 2024-03-24T00:48:40.994 INFO:tasks.workunit.client.1.smithi088.stdout:4/182: sync 2024-03-24T00:48:40.994 INFO:tasks.workunit.client.0.smithi038.stdout:1/140: write f7 [5888538,33485] 0 2024-03-24T00:48:41.002 INFO:tasks.workunit.client.0.smithi038.stdout:1/141: unlink f9 0 2024-03-24T00:48:41.009 INFO:tasks.workunit.client.1.smithi088.stdout:8/172: dread d9/dc/d11/d24/f2a [0,4194304] 0 2024-03-24T00:48:41.009 INFO:tasks.workunit.client.0.smithi038.stdout:9/193: dwrite d12/d28/d3a/f30 [0,4194304] 0 2024-03-24T00:48:41.015 INFO:tasks.workunit.client.0.smithi038.stdout:1/142: truncate da/fb 781968 0 2024-03-24T00:48:41.015 INFO:tasks.workunit.client.0.smithi038.stdout:1/143: write da/f23 [1265243,41212] 0 2024-03-24T00:48:41.018 INFO:tasks.workunit.client.1.smithi088.stdout:0/173: truncate d9/dc/de/f14 1304834 0 2024-03-24T00:48:41.018 INFO:tasks.workunit.client.1.smithi088.stdout:0/174: chown d9/dc/de/l2d 78 1 2024-03-24T00:48:41.018 INFO:tasks.workunit.client.1.smithi088.stdout:0/175: fdatasync d9/dc/de/f32 0 2024-03-24T00:48:41.019 INFO:tasks.workunit.client.0.smithi038.stdout:3/185: dwrite d3/d9/dc/df/d14/d19/f28 [0,4194304] 0 2024-03-24T00:48:41.019 INFO:tasks.workunit.client.0.smithi038.stdout:3/186: chown d3/d9/dc/df/d14/d19/c3f 539 1 2024-03-24T00:48:41.024 INFO:tasks.workunit.client.1.smithi088.stdout:1/183: symlink d6/d23/l37 0 2024-03-24T00:48:41.025 INFO:tasks.workunit.client.1.smithi088.stdout:1/184: truncate d6/f2a 320576 0 2024-03-24T00:48:41.028 INFO:tasks.workunit.client.1.smithi088.stdout:4/183: dread d5/d16/f2e [0,4194304] 0 2024-03-24T00:48:41.029 INFO:tasks.workunit.client.0.smithi038.stdout:9/194: symlink d12/d28/d3a/l44 0 2024-03-24T00:48:41.029 INFO:tasks.workunit.client.0.smithi038.stdout:9/195: fdatasync f4 0 2024-03-24T00:48:41.029 INFO:tasks.workunit.client.0.smithi038.stdout:9/196: chown d12/d28/d3a/c43 6104208 1 2024-03-24T00:48:41.029 INFO:tasks.workunit.client.0.smithi038.stdout:1/144: creat da/dd/f2f x:0 0 0 2024-03-24T00:48:41.034 INFO:tasks.workunit.client.1.smithi088.stdout:2/151: truncate db/dc/f11 4093665 0 2024-03-24T00:48:41.040 INFO:tasks.workunit.client.1.smithi088.stdout:8/173: creat d9/dc/d11/d24/f40 x:0 0 0 2024-03-24T00:48:41.040 INFO:tasks.workunit.client.1.smithi088.stdout:8/174: fdatasync f1 0 2024-03-24T00:48:41.040 INFO:tasks.workunit.client.1.smithi088.stdout:8/175: write d9/f2b [1372841,23759] 0 2024-03-24T00:48:41.040 INFO:tasks.workunit.client.1.smithi088.stdout:0/176: rename d9/dc/de/f32 to d9/dc/de/f3e 0 2024-03-24T00:48:41.041 INFO:tasks.workunit.client.0.smithi038.stdout:1/145: rename da/ce to da/dd/d22/c30 0 2024-03-24T00:48:41.042 INFO:tasks.workunit.client.0.smithi038.stdout:1/146: write da/dd/f1c [374325,40262] 0 2024-03-24T00:48:41.042 INFO:tasks.workunit.client.0.smithi038.stdout:1/147: chown f4 1374230212 1 2024-03-24T00:48:41.043 INFO:tasks.workunit.client.0.smithi038.stdout:3/187: getdents d3/d2a 0 2024-03-24T00:48:41.043 INFO:tasks.workunit.client.0.smithi038.stdout:3/188: fsync d3/f5 0 2024-03-24T00:48:41.047 INFO:tasks.workunit.client.0.smithi038.stdout:7/155: dwrite d2/d9/f20 [0,4194304] 0 2024-03-24T00:48:41.048 INFO:tasks.workunit.client.0.smithi038.stdout:2/190: dwrite d4/ff [0,4194304] 0 2024-03-24T00:48:41.048 INFO:tasks.workunit.client.0.smithi038.stdout:2/191: dread - d4/d18/f28 zero size 2024-03-24T00:48:41.049 INFO:tasks.workunit.client.1.smithi088.stdout:7/160: rename d3/fe to d3/d32/d34/f36 0 2024-03-24T00:48:41.049 INFO:tasks.workunit.client.1.smithi088.stdout:0/177: rename d9 to d9/d3f 22 2024-03-24T00:48:41.049 INFO:tasks.workunit.client.1.smithi088.stdout:0/178: chown d9/c2b 41 1 2024-03-24T00:48:41.049 INFO:tasks.workunit.client.1.smithi088.stdout:4/184: mkdir d5/d16/d1a/d22/d3c 0 2024-03-24T00:48:41.052 INFO:tasks.workunit.client.0.smithi038.stdout:9/197: rmdir d12/d28 39 2024-03-24T00:48:41.055 INFO:tasks.workunit.client.1.smithi088.stdout:0/179: stat d9/cb 0 2024-03-24T00:48:41.055 INFO:tasks.workunit.client.1.smithi088.stdout:0/180: dread - d9/dc/de/f3e zero size 2024-03-24T00:48:41.056 INFO:tasks.workunit.client.1.smithi088.stdout:0/181: write d9/dc/de/f3d [51140,61393] 0 2024-03-24T00:48:41.056 INFO:tasks.workunit.client.1.smithi088.stdout:0/182: stat d9 0 2024-03-24T00:48:41.056 INFO:tasks.workunit.client.1.smithi088.stdout:0/183: truncate d9/f3a 1157882 0 2024-03-24T00:48:41.056 INFO:tasks.workunit.client.1.smithi088.stdout:0/184: creat d9/dc/de/f40 x:0 0 0 2024-03-24T00:48:41.056 INFO:tasks.workunit.client.1.smithi088.stdout:2/152: dread db/fe [0,4194304] 0 2024-03-24T00:48:41.058 INFO:tasks.workunit.client.0.smithi038.stdout:3/189: rmdir d3 39 2024-03-24T00:48:41.059 INFO:tasks.workunit.client.1.smithi088.stdout:7/161: link d3/f35 d3/d32/d34/f37 0 2024-03-24T00:48:41.059 INFO:tasks.workunit.client.0.smithi038.stdout:1/148: write da/f11 [26024,3829] 0 2024-03-24T00:48:41.059 INFO:tasks.workunit.client.1.smithi088.stdout:9/152: sync 2024-03-24T00:48:41.065 INFO:tasks.workunit.client.1.smithi088.stdout:4/185: mkdir d5/d13/d18/d24/d3d 0 2024-03-24T00:48:41.065 INFO:tasks.workunit.client.1.smithi088.stdout:4/186: fsync d5/d17/f33 0 2024-03-24T00:48:41.065 INFO:tasks.workunit.client.1.smithi088.stdout:4/187: read - d5/d13/d18/f2c zero size 2024-03-24T00:48:41.065 INFO:tasks.workunit.client.1.smithi088.stdout:8/176: getdents d9/dc/d11 0 2024-03-24T00:48:41.066 INFO:tasks.workunit.client.1.smithi088.stdout:8/177: chown d9/dc/c19 1082571 1 2024-03-24T00:48:41.066 INFO:tasks.workunit.client.1.smithi088.stdout:8/178: readlink d9/l35 0 2024-03-24T00:48:41.066 INFO:tasks.workunit.client.1.smithi088.stdout:8/179: dread - d9/dc/d11/d24/d29/f2c zero size 2024-03-24T00:48:41.068 INFO:tasks.workunit.client.1.smithi088.stdout:3/143: sync 2024-03-24T00:48:41.068 INFO:tasks.workunit.client.1.smithi088.stdout:3/144: rename d3/d1a/d1e to d3/d1a/d1e/d2e 22 2024-03-24T00:48:41.068 INFO:tasks.workunit.client.1.smithi088.stdout:6/205: sync 2024-03-24T00:48:41.069 INFO:tasks.workunit.client.1.smithi088.stdout:5/155: sync 2024-03-24T00:48:41.071 INFO:tasks.workunit.client.1.smithi088.stdout:0/185: mknod d9/dc/de/d19/c41 0 2024-03-24T00:48:41.071 INFO:tasks.workunit.client.1.smithi088.stdout:0/186: readlink d9/d26/l27 0 2024-03-24T00:48:41.072 INFO:tasks.workunit.client.1.smithi088.stdout:7/162: write d3/f1d [1038390,69102] 0 2024-03-24T00:48:41.077 INFO:tasks.workunit.client.1.smithi088.stdout:9/153: mkdir d1/d10/d29 0 2024-03-24T00:48:41.077 INFO:tasks.workunit.client.1.smithi088.stdout:9/154: readlink d1/l16 0 2024-03-24T00:48:41.079 INFO:tasks.workunit.client.1.smithi088.stdout:4/188: creat d5/d16/d1a/f3e x:0 0 0 2024-03-24T00:48:41.083 INFO:tasks.workunit.client.1.smithi088.stdout:8/180: mkdir d9/dc/d11/d41 0 2024-03-24T00:48:41.084 INFO:tasks.workunit.client.0.smithi038.stdout:7/156: creat d2/d9/da/f36 x:0 0 0 2024-03-24T00:48:41.087 INFO:tasks.workunit.client.1.smithi088.stdout:3/145: mkdir d3/d1a/d1e/d2f 0 2024-03-24T00:48:41.088 INFO:tasks.workunit.client.1.smithi088.stdout:6/206: mknod db/d1f/d24/c3e 0 2024-03-24T00:48:41.088 INFO:tasks.workunit.client.1.smithi088.stdout:6/207: chown db/l15 4 1 2024-03-24T00:48:41.091 INFO:tasks.workunit.client.1.smithi088.stdout:5/156: creat d1/d10/d1b/d2f/f38 x:0 0 0 2024-03-24T00:48:41.092 INFO:tasks.workunit.client.0.smithi038.stdout:7/157: write d2/f13 [1510743,72962] 0 2024-03-24T00:48:41.096 INFO:tasks.workunit.client.1.smithi088.stdout:7/163: chown d3/l11 933450 1 2024-03-24T00:48:41.098 INFO:tasks.workunit.client.1.smithi088.stdout:9/155: symlink d1/d10/d11/d23/l2a 0 2024-03-24T00:48:41.100 INFO:tasks.workunit.client.0.smithi038.stdout:2/192: rename d4/d27/d2f/d30/f3b to d4/d18/d26/d37/f43 0 2024-03-24T00:48:41.102 INFO:tasks.workunit.client.0.smithi038.stdout:4/155: sync 2024-03-24T00:48:41.103 INFO:tasks.workunit.client.1.smithi088.stdout:4/189: symlink d5/d13/d18/d24/d3d/l3f 0 2024-03-24T00:48:41.103 INFO:tasks.workunit.client.1.smithi088.stdout:4/190: chown d5/d16/d1a/d22/d2f/l31 51495 1 2024-03-24T00:48:41.103 INFO:tasks.workunit.client.1.smithi088.stdout:4/191: write d5/d13/d18/d24/f30 [3010435,14131] 0 2024-03-24T00:48:41.105 INFO:tasks.workunit.client.1.smithi088.stdout:8/181: mknod d9/dc/c42 0 2024-03-24T00:48:41.106 INFO:tasks.workunit.client.1.smithi088.stdout:8/182: dread - d9/f32 zero size 2024-03-24T00:48:41.106 INFO:tasks.workunit.client.0.smithi038.stdout:8/174: sync 2024-03-24T00:48:41.106 INFO:tasks.workunit.client.0.smithi038.stdout:5/139: sync 2024-03-24T00:48:41.106 INFO:tasks.workunit.client.0.smithi038.stdout:0/132: sync 2024-03-24T00:48:41.107 INFO:tasks.workunit.client.0.smithi038.stdout:0/133: truncate d0/d12/d15/d1a/f1e 5228705 0 2024-03-24T00:48:41.107 INFO:tasks.workunit.client.0.smithi038.stdout:0/134: fdatasync d0/fc 0 2024-03-24T00:48:41.107 INFO:tasks.workunit.client.0.smithi038.stdout:9/198: creat d12/d28/d3a/f45 x:0 0 0 2024-03-24T00:48:41.107 INFO:tasks.workunit.client.0.smithi038.stdout:7/158: mkdir d2/d9/df/d33/d37 0 2024-03-24T00:48:41.107 INFO:tasks.workunit.client.0.smithi038.stdout:7/159: write d2/f1a [902775,51304] 0 2024-03-24T00:48:41.107 INFO:tasks.workunit.client.0.smithi038.stdout:3/190: dread d3/d9/f2b [0,4194304] 0 2024-03-24T00:48:41.107 INFO:tasks.workunit.client.0.smithi038.stdout:3/191: truncate d3/d9/f2b 1386420 0 2024-03-24T00:48:41.107 INFO:tasks.workunit.client.0.smithi038.stdout:6/192: sync 2024-03-24T00:48:41.108 INFO:tasks.workunit.client.1.smithi088.stdout:3/146: unlink d3/d6/f1f 0 2024-03-24T00:48:41.109 INFO:tasks.workunit.client.0.smithi038.stdout:2/193: symlink d4/d27/d42/l44 0 2024-03-24T00:48:41.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:40 smithi038 ceph-mon[29271]: pgmap v26: 65 pgs: 65 active+clean; 430 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 27 MiB/s wr, 460 op/s 2024-03-24T00:48:41.110 INFO:tasks.workunit.client.0.smithi038.stdout:2/194: readlink d4/d18/l25 0 2024-03-24T00:48:41.112 INFO:tasks.workunit.client.1.smithi088.stdout:6/208: rename db/d1d/c26 to db/d1d/d23/d3c/c3f 0 2024-03-24T00:48:41.112 INFO:tasks.workunit.client.1.smithi088.stdout:0/187: getdents d9/d26 0 2024-03-24T00:48:41.113 INFO:tasks.workunit.client.0.smithi038.stdout:9/199: rename d12/d1a/c1d to d12/d41/c46 0 2024-03-24T00:48:41.113 INFO:tasks.workunit.client.0.smithi038.stdout:7/160: mknod d2/d9/da/c38 0 2024-03-24T00:48:41.113 INFO:tasks.workunit.client.1.smithi088.stdout:7/164: chown d3/fd 3462 1 2024-03-24T00:48:41.115 INFO:tasks.workunit.client.0.smithi038.stdout:6/193: mknod d1/da/d1b/c38 0 2024-03-24T00:48:41.115 INFO:tasks.workunit.client.0.smithi038.stdout:6/194: write d1/da/d1b/d1d/f2d [953963,48145] 0 2024-03-24T00:48:41.115 INFO:tasks.workunit.client.0.smithi038.stdout:6/195: readlink d1/da/lc 0 2024-03-24T00:48:41.116 INFO:tasks.workunit.client.1.smithi088.stdout:9/156: unlink d1/d10/d11/d23/l2a 0 2024-03-24T00:48:41.116 INFO:tasks.workunit.client.1.smithi088.stdout:9/157: readlink d1/d15/l25 0 2024-03-24T00:48:41.116 INFO:tasks.workunit.client.1.smithi088.stdout:9/158: fdatasync d1/f7 0 2024-03-24T00:48:41.117 INFO:tasks.workunit.client.0.smithi038.stdout:2/195: truncate d4/d27/f33 148833 0 2024-03-24T00:48:41.120 INFO:tasks.workunit.client.1.smithi088.stdout:4/192: mkdir d5/d7/d15/d40 0 2024-03-24T00:48:41.120 INFO:tasks.workunit.client.1.smithi088.stdout:4/193: chown d5/d16/d1a/d22/d2f/c3b 6 1 2024-03-24T00:48:41.120 INFO:tasks.workunit.client.1.smithi088.stdout:4/194: read - d5/d17/f1d zero size 2024-03-24T00:48:41.122 INFO:tasks.workunit.client.0.smithi038.stdout:9/200: symlink d12/l47 0 2024-03-24T00:48:41.122 INFO:tasks.workunit.client.0.smithi038.stdout:9/201: chown d12/l47 51 1 2024-03-24T00:48:41.122 INFO:tasks.workunit.client.0.smithi038.stdout:9/202: creat d12/d28/d3a/f48 x:0 0 0 2024-03-24T00:48:41.122 INFO:tasks.workunit.client.0.smithi038.stdout:9/203: write d12/f42 [700703,39858] 0 2024-03-24T00:48:41.127 INFO:tasks.workunit.client.0.smithi038.stdout:8/175: rename d0/d10/d19/d29 to d0/d15/d42 0 2024-03-24T00:48:41.127 INFO:tasks.workunit.client.0.smithi038.stdout:7/161: creat d2/d9/da/d23/f39 x:0 0 0 2024-03-24T00:48:41.128 INFO:tasks.workunit.client.1.smithi088.stdout:8/183: write d9/dc/d11/d24/d29/f2c [251,123027] 0 2024-03-24T00:48:41.129 INFO:tasks.workunit.client.1.smithi088.stdout:1/185: dwrite d6/d27/d1e/f1f [0,4194304] 0 2024-03-24T00:48:41.129 INFO:tasks.workunit.client.1.smithi088.stdout:3/147: dread d3/d1a/d1c/f1d [0,4194304] 0 2024-03-24T00:48:41.132 INFO:tasks.workunit.client.0.smithi038.stdout:6/196: link d1/f6 d1/da/d1b/d1d/f39 0 2024-03-24T00:48:41.135 INFO:tasks.workunit.client.0.smithi038.stdout:3/192: dread d3/f6 [0,4194304] 0 2024-03-24T00:48:41.138 INFO:tasks.workunit.client.1.smithi088.stdout:6/209: unlink db/d1d/c20 0 2024-03-24T00:48:41.139 INFO:tasks.workunit.client.0.smithi038.stdout:0/135: dread d0/ff [0,4194304] 0 2024-03-24T00:48:41.142 INFO:tasks.workunit.client.1.smithi088.stdout:0/188: symlink d9/dc/de/d19/l42 0 2024-03-24T00:48:41.142 INFO:tasks.workunit.client.0.smithi038.stdout:7/162: mkdir d2/d9/df/d33/d37/d3a 0 2024-03-24T00:48:41.142 INFO:tasks.workunit.client.0.smithi038.stdout:1/149: dwrite da/df/f13 [0,4194304] 0 2024-03-24T00:48:41.143 INFO:tasks.workunit.client.0.smithi038.stdout:1/150: readlink da/df/l1a 0 2024-03-24T00:48:41.143 INFO:tasks.workunit.client.0.smithi038.stdout:1/151: stat da/dd/d10/d14/c28 0 2024-03-24T00:48:41.144 INFO:tasks.workunit.client.0.smithi038.stdout:3/193: creat d3/d9/f44 x:0 0 0 2024-03-24T00:48:41.151 INFO:tasks.workunit.client.0.smithi038.stdout:9/204: dread f8 [0,4194304] 0 2024-03-24T00:48:41.151 INFO:tasks.workunit.client.0.smithi038.stdout:4/156: dread da/d12/d24/f26 [0,4194304] 0 2024-03-24T00:48:41.151 INFO:tasks.workunit.client.0.smithi038.stdout:4/157: read - da/dc/f22 zero size 2024-03-24T00:48:41.153 INFO:tasks.workunit.client.0.smithi038.stdout:2/196: dread d4/fc [0,4194304] 0 2024-03-24T00:48:41.154 INFO:tasks.workunit.client.0.smithi038.stdout:2/197: chown d4/d18/f2e 158786 1 2024-03-24T00:48:41.154 INFO:tasks.workunit.client.0.smithi038.stdout:8/176: dread d0/d1d/d20/f23 [0,4194304] 0 2024-03-24T00:48:41.157 INFO:tasks.workunit.client.1.smithi088.stdout:7/165: chown d3/d31 111 1 2024-03-24T00:48:41.157 INFO:tasks.workunit.client.1.smithi088.stdout:4/195: mknod d5/d7/d15/c41 0 2024-03-24T00:48:41.158 INFO:tasks.workunit.client.1.smithi088.stdout:4/196: fdatasync f1 0 2024-03-24T00:48:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:9/205: dread d12/f35 [0,4194304] 0 2024-03-24T00:48:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:1/152: creat da/dd/d2c/f31 x:0 0 0 2024-03-24T00:48:41.165 INFO:tasks.workunit.client.1.smithi088.stdout:8/184: mkdir d9/dc/d11/d24/d43 0 2024-03-24T00:48:41.169 INFO:tasks.workunit.client.1.smithi088.stdout:1/186: getdents d6/d27/d1e 0 2024-03-24T00:48:41.174 INFO:tasks.workunit.client.1.smithi088.stdout:3/148: creat d3/d1a/d1c/f30 x:0 0 0 2024-03-24T00:48:41.176 INFO:tasks.workunit.client.0.smithi038.stdout:3/194: mknod d3/d9/c45 0 2024-03-24T00:48:41.177 INFO:tasks.workunit.client.1.smithi088.stdout:6/210: mkdir db/d40 0 2024-03-24T00:48:41.179 INFO:tasks.workunit.client.0.smithi038.stdout:2/198: rmdir d4/d27 39 2024-03-24T00:48:41.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:40 smithi088 ceph-mon[36610]: pgmap v26: 65 pgs: 65 active+clean; 430 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 27 MiB/s wr, 460 op/s 2024-03-24T00:48:41.180 INFO:tasks.workunit.client.1.smithi088.stdout:9/159: rmdir d1 39 2024-03-24T00:48:41.180 INFO:tasks.workunit.client.1.smithi088.stdout:9/160: truncate d1/d10/f18 3169072 0 2024-03-24T00:48:41.180 INFO:tasks.workunit.client.1.smithi088.stdout:9/161: fdatasync d1/d10/d11/f17 0 2024-03-24T00:48:41.181 INFO:tasks.workunit.client.0.smithi038.stdout:6/197: rmdir d1 39 2024-03-24T00:48:41.183 INFO:tasks.workunit.client.0.smithi038.stdout:8/177: truncate d0/d15/f17 413279 0 2024-03-24T00:48:41.183 INFO:tasks.workunit.client.0.smithi038.stdout:8/178: chown d0/d1d/d20/c39 121028 1 2024-03-24T00:48:41.184 INFO:tasks.workunit.client.1.smithi088.stdout:7/166: chown d3/l13 73 1 2024-03-24T00:48:41.184 INFO:tasks.workunit.client.1.smithi088.stdout:7/167: fdatasync d3/d32/d34/f37 0 2024-03-24T00:48:41.188 INFO:tasks.workunit.client.0.smithi038.stdout:9/206: rmdir d12/d41 39 2024-03-24T00:48:41.188 INFO:tasks.workunit.client.0.smithi038.stdout:9/207: read d12/f35 [97257,66750] 0 2024-03-24T00:48:41.192 INFO:tasks.workunit.client.0.smithi038.stdout:1/153: mknod da/c32 0 2024-03-24T00:48:41.194 INFO:tasks.workunit.client.1.smithi088.stdout:1/187: symlink d6/d2f/l38 0 2024-03-24T00:48:41.194 INFO:tasks.workunit.client.1.smithi088.stdout:1/188: chown d6/f19 1711523 1 2024-03-24T00:48:41.194 INFO:tasks.workunit.client.1.smithi088.stdout:1/189: stat d6/f10 0 2024-03-24T00:48:41.195 INFO:tasks.workunit.client.1.smithi088.stdout:1/190: chown d6 33 1 2024-03-24T00:48:41.197 INFO:tasks.workunit.client.0.smithi038.stdout:9/208: unlink d12/f18 0 2024-03-24T00:48:41.198 INFO:tasks.workunit.client.1.smithi088.stdout:5/157: dwrite d1/d19/d1e/d26/f35 [0,4194304] 0 2024-03-24T00:48:41.208 INFO:tasks.workunit.client.1.smithi088.stdout:3/149: mkdir d3/de/df/d31 0 2024-03-24T00:48:41.208 INFO:tasks.workunit.client.1.smithi088.stdout:3/150: write d3/de/df/f15 [3797566,75677] 0 2024-03-24T00:48:41.213 INFO:tasks.workunit.client.0.smithi038.stdout:1/154: mknod da/dd/d10/d14/c33 0 2024-03-24T00:48:41.215 INFO:tasks.workunit.client.0.smithi038.stdout:5/140: dwrite f5 [0,4194304] 0 2024-03-24T00:48:41.221 INFO:tasks.workunit.client.0.smithi038.stdout:5/141: truncate d6/d19/f1c 1973414 0 2024-03-24T00:48:41.225 INFO:tasks.workunit.client.1.smithi088.stdout:9/162: creat d1/d10/d29/f2b x:0 0 0 2024-03-24T00:48:41.225 INFO:tasks.workunit.client.1.smithi088.stdout:9/163: write d1/d10/d29/f2b [793117,42645] 0 2024-03-24T00:48:41.229 INFO:tasks.workunit.client.1.smithi088.stdout:9/164: rmdir d1 39 2024-03-24T00:48:41.229 INFO:tasks.workunit.client.0.smithi038.stdout:5/142: creat d6/d16/d21/f28 x:0 0 0 2024-03-24T00:48:41.230 INFO:tasks.workunit.client.0.smithi038.stdout:5/143: chown d6/f18 18 1 2024-03-24T00:48:41.233 INFO:tasks.workunit.client.0.smithi038.stdout:3/195: read d3/d30/f33 [62348,10119] 0 2024-03-24T00:48:41.233 INFO:tasks.workunit.client.0.smithi038.stdout:3/196: chown d3/d9/dc/df/c20 43 1 2024-03-24T00:48:41.236 INFO:tasks.workunit.client.1.smithi088.stdout:9/165: mkdir d1/d10/d2c 0 2024-03-24T00:48:41.243 INFO:tasks.workunit.client.0.smithi038.stdout:3/197: creat d3/d9/dc/df/d14/d19/f46 x:0 0 0 2024-03-24T00:48:41.243 INFO:tasks.workunit.client.0.smithi038.stdout:3/198: creat d3/d9/dc/df/d14/d19/f47 x:0 0 0 2024-03-24T00:48:41.243 INFO:tasks.workunit.client.0.smithi038.stdout:3/199: stat d3/d2a 0 2024-03-24T00:48:41.255 INFO:tasks.workunit.client.1.smithi088.stdout:9/166: dread d1/ff [0,4194304] 0 2024-03-24T00:48:41.257 INFO:tasks.workunit.client.0.smithi038.stdout:2/199: dread d4/fa [0,4194304] 0 2024-03-24T00:48:41.257 INFO:tasks.workunit.client.0.smithi038.stdout:0/136: dread d0/f2 [4194304,4194304] 0 2024-03-24T00:48:41.258 INFO:tasks.workunit.client.1.smithi088.stdout:5/158: write d1/ff [8068043,78089] 0 2024-03-24T00:48:41.258 INFO:tasks.workunit.client.1.smithi088.stdout:5/159: fdatasync d1/dd/f15 0 2024-03-24T00:48:41.259 INFO:tasks.workunit.client.0.smithi038.stdout:1/155: truncate da/dd/d10/f15 659899 0 2024-03-24T00:48:41.263 INFO:tasks.workunit.client.1.smithi088.stdout:2/153: sync 2024-03-24T00:48:41.263 INFO:tasks.workunit.client.1.smithi088.stdout:2/154: creat db/d21/f33 x:0 0 0 2024-03-24T00:48:41.263 INFO:tasks.workunit.client.0.smithi038.stdout:2/200: creat d4/d18/d38/f45 x:0 0 0 2024-03-24T00:48:41.265 INFO:tasks.workunit.client.1.smithi088.stdout:4/197: dwrite d5/d16/d1a/f1e [0,4194304] 0 2024-03-24T00:48:41.265 INFO:tasks.workunit.client.1.smithi088.stdout:4/198: fsync d5/d17/f33 0 2024-03-24T00:48:41.273 INFO:tasks.workunit.client.0.smithi038.stdout:7/163: dwrite d2/d9/df/f29 [0,4194304] 0 2024-03-24T00:48:41.273 INFO:tasks.workunit.client.0.smithi038.stdout:7/164: truncate d2/f1a 1839211 0 2024-03-24T00:48:41.273 INFO:tasks.workunit.client.0.smithi038.stdout:5/144: read d6/fa [5114444,127641] 0 2024-03-24T00:48:41.274 INFO:tasks.workunit.client.1.smithi088.stdout:2/155: rename db/c1d to db/dd/c34 0 2024-03-24T00:48:41.276 INFO:tasks.workunit.client.1.smithi088.stdout:4/199: link d5/f9 d5/d13/d18/d24/d3d/f42 0 2024-03-24T00:48:41.277 INFO:tasks.workunit.client.0.smithi038.stdout:2/201: write d4/d18/f1f [361156,94277] 0 2024-03-24T00:48:41.279 INFO:tasks.workunit.client.0.smithi038.stdout:0/137: getdents d0/d12/d15 0 2024-03-24T00:48:41.279 INFO:tasks.workunit.client.0.smithi038.stdout:0/138: creat d0/d12/d15/d1a/f2b x:0 0 0 2024-03-24T00:48:41.288 INFO:tasks.workunit.client.1.smithi088.stdout:2/156: mkdir db/d21/d35 0 2024-03-24T00:48:41.294 INFO:tasks.workunit.client.1.smithi088.stdout:3/151: dwrite d3/d6/f2b [0,4194304] 0 2024-03-24T00:48:41.297 INFO:tasks.workunit.client.1.smithi088.stdout:7/168: dwrite d3/fc [4194304,4194304] 0 2024-03-24T00:48:41.297 INFO:tasks.workunit.client.0.smithi038.stdout:6/198: dwrite d1/da/f1a [0,4194304] 0 2024-03-24T00:48:41.297 INFO:tasks.workunit.client.0.smithi038.stdout:1/156: dread f5 [0,4194304] 0 2024-03-24T00:48:41.298 INFO:tasks.workunit.client.1.smithi088.stdout:4/200: link d5/d7/d15/c41 d5/d16/c43 0 2024-03-24T00:48:41.300 INFO:tasks.workunit.client.0.smithi038.stdout:7/165: getdents d2/d9/da/d23/d2b 0 2024-03-24T00:48:41.306 INFO:tasks.workunit.client.0.smithi038.stdout:2/202: unlink d4/d18/d26/c36 0 2024-03-24T00:48:41.309 INFO:tasks.workunit.client.0.smithi038.stdout:5/145: creat d6/f29 x:0 0 0 2024-03-24T00:48:41.312 INFO:tasks.workunit.client.1.smithi088.stdout:3/152: mkdir d3/d32 0 2024-03-24T00:48:41.312 INFO:tasks.workunit.client.1.smithi088.stdout:7/169: dread - d3/f2d zero size 2024-03-24T00:48:41.313 INFO:tasks.workunit.client.0.smithi038.stdout:4/158: dwrite da/dc/dd/d15/f17 [0,4194304] 0 2024-03-24T00:48:41.314 INFO:tasks.workunit.client.1.smithi088.stdout:1/191: dwrite d6/d27/d1e/f21 [0,4194304] 0 2024-03-24T00:48:41.314 INFO:tasks.workunit.client.1.smithi088.stdout:1/192: truncate d6/f17 9328633 0 2024-03-24T00:48:41.314 INFO:tasks.workunit.client.1.smithi088.stdout:1/193: truncate d6/f8 9862417 0 2024-03-24T00:48:41.314 INFO:tasks.workunit.client.1.smithi088.stdout:1/194: dread - d6/d27/f1d zero size 2024-03-24T00:48:41.314 INFO:tasks.workunit.client.1.smithi088.stdout:4/201: creat d5/d7/f44 x:0 0 0 2024-03-24T00:48:41.315 INFO:tasks.workunit.client.1.smithi088.stdout:4/202: chown d5/d17/f33 927216742 1 2024-03-24T00:48:41.316 INFO:tasks.workunit.client.0.smithi038.stdout:2/203: unlink d4/d18/l25 0 2024-03-24T00:48:41.317 INFO:tasks.workunit.client.1.smithi088.stdout:8/185: dwrite d9/dc/d11/d1b/f1e [0,4194304] 0 2024-03-24T00:48:41.326 INFO:tasks.workunit.client.0.smithi038.stdout:9/209: dwrite d12/d1a/f20 [0,4194304] 0 2024-03-24T00:48:41.326 INFO:tasks.workunit.client.0.smithi038.stdout:9/210: fdatasync d12/d28/d3a/f40 0 2024-03-24T00:48:41.327 INFO:tasks.workunit.client.0.smithi038.stdout:9/211: chown d12/d28/d3a/c37 68861992 1 2024-03-24T00:48:41.327 INFO:tasks.workunit.client.0.smithi038.stdout:9/212: creat d12/d1a/f49 x:0 0 0 2024-03-24T00:48:41.327 INFO:tasks.workunit.client.0.smithi038.stdout:4/159: unlink l5 0 2024-03-24T00:48:41.328 INFO:tasks.workunit.client.0.smithi038.stdout:4/160: write da/dc/f22 [561473,93427] 0 2024-03-24T00:48:41.328 INFO:tasks.workunit.client.0.smithi038.stdout:4/161: chown da/d12/d29/c2e 24289059 1 2024-03-24T00:48:41.331 INFO:tasks.workunit.client.1.smithi088.stdout:1/195: creat d6/d27/d1e/f39 x:0 0 0 2024-03-24T00:48:41.333 INFO:tasks.workunit.client.1.smithi088.stdout:9/167: dwrite d1/f20 [0,4194304] 0 2024-03-24T00:48:41.333 INFO:tasks.workunit.client.1.smithi088.stdout:9/168: truncate d1/f8 4409638 0 2024-03-24T00:48:41.333 INFO:tasks.workunit.client.1.smithi088.stdout:5/160: dwrite d1/d19/d1e/d26/f35 [0,4194304] 0 2024-03-24T00:48:41.334 INFO:tasks.workunit.client.0.smithi038.stdout:2/204: symlink d4/d27/d2f/d30/d3e/l46 0 2024-03-24T00:48:41.334 INFO:tasks.workunit.client.1.smithi088.stdout:7/170: dread d3/fc [0,4194304] 0 2024-03-24T00:48:41.335 INFO:tasks.workunit.client.0.smithi038.stdout:6/199: write d1/f21 [1337850,108477] 0 2024-03-24T00:48:41.335 INFO:tasks.workunit.client.0.smithi038.stdout:6/200: truncate d1/da/f14 2374876 0 2024-03-24T00:48:41.340 INFO:tasks.workunit.client.0.smithi038.stdout:5/146: rmdir d6/d16/d21 39 2024-03-24T00:48:41.351 INFO:tasks.workunit.client.1.smithi088.stdout:4/203: write d5/d16/d1a/f1e [489391,48694] 0 2024-03-24T00:48:41.356 INFO:tasks.workunit.client.1.smithi088.stdout:2/157: dwrite db/dc/f1b [0,4194304] 0 2024-03-24T00:48:41.356 INFO:tasks.workunit.client.1.smithi088.stdout:2/158: creat db/f36 x:0 0 0 2024-03-24T00:48:41.356 INFO:tasks.workunit.client.1.smithi088.stdout:2/159: readlink db/d21/l2d 0 2024-03-24T00:48:41.358 INFO:tasks.workunit.client.0.smithi038.stdout:0/139: dwrite d0/f5 [0,4194304] 0 2024-03-24T00:48:41.359 INFO:tasks.workunit.client.0.smithi038.stdout:4/162: rename da/dc/dd/d15/f23 to da/d12/d24/f30 0 2024-03-24T00:48:41.359 INFO:tasks.workunit.client.0.smithi038.stdout:4/163: chown da/d12/d24/f30 1 1 2024-03-24T00:48:41.361 INFO:tasks.workunit.client.1.smithi088.stdout:9/169: write d1/f5 [6486102,70649] 0 2024-03-24T00:48:41.361 INFO:tasks.workunit.client.1.smithi088.stdout:9/170: truncate d1/d10/f1e 800020 0 2024-03-24T00:48:41.366 INFO:tasks.workunit.client.1.smithi088.stdout:3/153: dwrite f1 [0,4194304] 0 2024-03-24T00:48:41.392 INFO:tasks.workunit.client.1.smithi088.stdout:2/160: read f1 [4424856,111493] 0 2024-03-24T00:48:41.400 INFO:tasks.workunit.client.0.smithi038.stdout:1/157: dwrite da/f16 [0,4194304] 0 2024-03-24T00:48:41.400 INFO:tasks.workunit.client.0.smithi038.stdout:1/158: write da/dd/f2f [190228,8209] 0 2024-03-24T00:48:41.409 INFO:tasks.workunit.client.0.smithi038.stdout:7/166: dwrite d2/d9/df/d33/f30 [0,4194304] 0 2024-03-24T00:48:41.427 INFO:tasks.workunit.client.1.smithi088.stdout:9/171: dwrite d1/f8 [4194304,4194304] 0 2024-03-24T00:48:41.427 INFO:tasks.workunit.client.1.smithi088.stdout:9/172: chown d1/f6 5042645 1 2024-03-24T00:48:41.440 INFO:tasks.workunit.client.1.smithi088.stdout:9/173: dread d1/f6 [0,4194304] 0 2024-03-24T00:48:41.441 INFO:tasks.workunit.client.0.smithi038.stdout:4/164: dwrite da/f1d [0,4194304] 0 2024-03-24T00:48:41.441 INFO:tasks.workunit.client.1.smithi088.stdout:1/196: rename d6/d27/d1e/f21 to d6/d27/f3a 0 2024-03-24T00:48:41.441 INFO:tasks.workunit.client.1.smithi088.stdout:1/197: write d6/d23/f34 [183529,129415] 0 2024-03-24T00:48:41.442 INFO:tasks.workunit.client.1.smithi088.stdout:4/204: dwrite d5/d13/f12 [0,4194304] 0 2024-03-24T00:48:41.442 INFO:tasks.workunit.client.1.smithi088.stdout:4/205: write f2 [2749899,123798] 0 2024-03-24T00:48:41.446 INFO:tasks.workunit.client.0.smithi038.stdout:2/205: creat d4/d27/f47 x:0 0 0 2024-03-24T00:48:41.446 INFO:tasks.workunit.client.0.smithi038.stdout:2/206: write d4/d18/d26/d37/f40 [138317,122980] 0 2024-03-24T00:48:41.447 INFO:tasks.workunit.client.1.smithi088.stdout:5/161: unlink d1/f1c 0 2024-03-24T00:48:41.448 INFO:tasks.workunit.client.1.smithi088.stdout:5/162: write d1/d10/f17 [298404,80971] 0 2024-03-24T00:48:41.450 INFO:tasks.workunit.client.1.smithi088.stdout:7/171: creat d3/d31/f38 x:0 0 0 2024-03-24T00:48:41.450 INFO:tasks.workunit.client.0.smithi038.stdout:6/201: mknod d1/da/d1b/d2f/c3a 0 2024-03-24T00:48:41.453 INFO:tasks.workunit.client.1.smithi088.stdout:2/161: dwrite db/f36 [0,4194304] 0 2024-03-24T00:48:41.453 INFO:tasks.workunit.client.1.smithi088.stdout:2/162: readlink db/d21/l2d 0 2024-03-24T00:48:41.459 INFO:tasks.workunit.client.0.smithi038.stdout:5/147: creat d6/d16/f2a x:0 0 0 2024-03-24T00:48:41.460 INFO:tasks.workunit.client.0.smithi038.stdout:9/213: rmdir d12/d28/d3a 39 2024-03-24T00:48:41.474 INFO:tasks.workunit.client.1.smithi088.stdout:6/211: sync 2024-03-24T00:48:41.474 INFO:tasks.workunit.client.1.smithi088.stdout:0/189: sync 2024-03-24T00:48:41.474 INFO:tasks.workunit.client.1.smithi088.stdout:6/212: write db/dd/f1e [4990258,41212] 0 2024-03-24T00:48:41.474 INFO:tasks.workunit.client.1.smithi088.stdout:6/213: creat db/dd/f41 x:0 0 0 2024-03-24T00:48:41.474 INFO:tasks.workunit.client.1.smithi088.stdout:6/214: stat db/f38 0 2024-03-24T00:48:41.476 INFO:tasks.workunit.client.0.smithi038.stdout:0/140: mkdir d0/d1b/d2c 0 2024-03-24T00:48:41.476 INFO:tasks.workunit.client.0.smithi038.stdout:1/159: mkdir da/dd/d10/d34 0 2024-03-24T00:48:41.482 INFO:tasks.workunit.client.1.smithi088.stdout:3/154: mknod d3/d1a/d1e/d2f/c33 0 2024-03-24T00:48:41.484 INFO:tasks.workunit.client.0.smithi038.stdout:5/148: write d6/d16/d14/f24 [573583,72803] 0 2024-03-24T00:48:41.492 INFO:tasks.workunit.client.1.smithi088.stdout:9/174: dwrite d1/f7 [0,4194304] 0 2024-03-24T00:48:41.497 INFO:tasks.workunit.client.0.smithi038.stdout:7/167: mknod d2/d9/df/d33/d37/c3b 0 2024-03-24T00:48:41.529 INFO:tasks.workunit.client.1.smithi088.stdout:1/198: symlink d6/d23/l3b 0 2024-03-24T00:48:41.530 INFO:tasks.workunit.client.0.smithi038.stdout:2/207: creat d4/d18/f48 x:0 0 0 2024-03-24T00:48:41.530 INFO:tasks.workunit.client.0.smithi038.stdout:4/165: dwrite da/f1d [0,4194304] 0 2024-03-24T00:48:41.534 INFO:tasks.workunit.client.1.smithi088.stdout:4/206: link d5/d7/dd/l34 d5/d16/d1a/d22/d2f/l45 0 2024-03-24T00:48:41.536 INFO:tasks.workunit.client.0.smithi038.stdout:6/202: creat d1/da/f3b x:0 0 0 2024-03-24T00:48:41.537 INFO:tasks.workunit.client.1.smithi088.stdout:5/163: mknod d1/d10/c39 0 2024-03-24T00:48:41.537 INFO:tasks.workunit.client.1.smithi088.stdout:5/164: fsync d1/d19/f30 0 2024-03-24T00:48:41.546 INFO:tasks.workunit.client.1.smithi088.stdout:1/199: dread f2 [0,4194304] 0 2024-03-24T00:48:41.547 INFO:tasks.workunit.client.0.smithi038.stdout:5/149: dwrite d6/d16/f2a [0,4194304] 0 2024-03-24T00:48:41.548 INFO:tasks.workunit.client.1.smithi088.stdout:2/163: symlink db/dd/l37 0 2024-03-24T00:48:41.548 INFO:tasks.workunit.client.1.smithi088.stdout:2/164: write db/d21/f2e [1001008,45402] 0 2024-03-24T00:48:41.552 INFO:tasks.workunit.client.1.smithi088.stdout:0/190: symlink d9/d20/l43 0 2024-03-24T00:48:41.554 INFO:tasks.workunit.client.0.smithi038.stdout:1/160: creat da/f35 x:0 0 0 2024-03-24T00:48:41.560 INFO:tasks.workunit.client.0.smithi038.stdout:5/150: write d6/d19/f1c [1349333,95180] 0 2024-03-24T00:48:41.560 INFO:tasks.workunit.client.0.smithi038.stdout:5/151: stat d6 0 2024-03-24T00:48:41.564 INFO:tasks.workunit.client.0.smithi038.stdout:9/214: dread fe [0,4194304] 0 2024-03-24T00:48:41.565 INFO:tasks.workunit.client.1.smithi088.stdout:6/215: mknod db/c42 0 2024-03-24T00:48:41.575 INFO:tasks.workunit.client.0.smithi038.stdout:7/168: creat d2/d9/df/d33/f3c x:0 0 0 2024-03-24T00:48:41.578 INFO:tasks.workunit.client.1.smithi088.stdout:3/155: link d3/d1a/l21 d3/l34 0 2024-03-24T00:48:41.578 INFO:tasks.workunit.client.1.smithi088.stdout:3/156: stat d3 0 2024-03-24T00:48:41.582 INFO:tasks.workunit.client.0.smithi038.stdout:5/152: write d6/d16/f2a [589457,84341] 0 2024-03-24T00:48:41.583 INFO:tasks.workunit.client.1.smithi088.stdout:9/175: link d1/c9 d1/d10/d29/c2d 0 2024-03-24T00:48:41.583 INFO:tasks.workunit.client.1.smithi088.stdout:9/176: creat d1/d10/d29/f2e x:0 0 0 2024-03-24T00:48:41.586 INFO:tasks.workunit.client.0.smithi038.stdout:5/153: fsync d6/d16/d14/f24 0 2024-03-24T00:48:41.587 INFO:tasks.workunit.client.1.smithi088.stdout:2/165: unlink c7 0 2024-03-24T00:48:41.587 INFO:tasks.workunit.client.1.smithi088.stdout:2/166: fdatasync db/dd/f28 0 2024-03-24T00:48:41.587 INFO:tasks.workunit.client.1.smithi088.stdout:2/167: chown db/dc/l1c 1090280217 1 2024-03-24T00:48:41.588 INFO:tasks.workunit.client.1.smithi088.stdout:7/172: dwrite d3/f1f [0,4194304] 0 2024-03-24T00:48:41.591 INFO:tasks.workunit.client.1.smithi088.stdout:0/191: mkdir d9/dc/de/d19/d44 0 2024-03-24T00:48:41.591 INFO:tasks.workunit.client.1.smithi088.stdout:0/192: fsync d9/d26/f3c 0 2024-03-24T00:48:41.591 INFO:tasks.workunit.client.1.smithi088.stdout:0/193: creat d9/f45 x:0 0 0 2024-03-24T00:48:41.591 INFO:tasks.workunit.client.1.smithi088.stdout:0/194: chown d9/d15/l23 4456 1 2024-03-24T00:48:41.596 INFO:tasks.workunit.client.1.smithi088.stdout:6/216: truncate db/d1f/f28 810319 0 2024-03-24T00:48:41.599 INFO:tasks.workunit.client.1.smithi088.stdout:5/165: dread d1/fb [0,4194304] 0 2024-03-24T00:48:41.607 INFO:tasks.workunit.client.0.smithi038.stdout:2/208: mknod d4/d18/d26/c49 0 2024-03-24T00:48:41.607 INFO:tasks.workunit.client.0.smithi038.stdout:2/209: write d4/f2b [475961,88041] 0 2024-03-24T00:48:41.607 INFO:tasks.workunit.client.1.smithi088.stdout:9/177: read d1/d10/f1e [763684,63958] 0 2024-03-24T00:48:41.607 INFO:tasks.workunit.client.1.smithi088.stdout:9/178: chown d1/cd 496 1 2024-03-24T00:48:41.609 INFO:tasks.workunit.client.1.smithi088.stdout:3/157: symlink d3/de/l35 0 2024-03-24T00:48:41.616 INFO:tasks.workunit.client.0.smithi038.stdout:4/166: unlink da/f1c 0 2024-03-24T00:48:41.619 INFO:tasks.workunit.client.1.smithi088.stdout:2/168: link db/dd/f28 db/f38 0 2024-03-24T00:48:41.619 INFO:tasks.workunit.client.1.smithi088.stdout:2/169: chown db/dc/f1b 266965 1 2024-03-24T00:48:41.621 INFO:tasks.workunit.client.0.smithi038.stdout:2/210: read d4/d18/d26/f29 [1757313,8966] 0 2024-03-24T00:48:41.622 INFO:tasks.workunit.client.1.smithi088.stdout:5/166: write d1/dd/f15 [3066618,108025] 0 2024-03-24T00:48:41.623 INFO:tasks.workunit.client.1.smithi088.stdout:4/207: dwrite d5/d16/d1a/f1e [0,4194304] 0 2024-03-24T00:48:41.623 INFO:tasks.workunit.client.1.smithi088.stdout:4/208: creat d5/f46 x:0 0 0 2024-03-24T00:48:41.623 INFO:tasks.workunit.client.1.smithi088.stdout:4/209: stat d5/d13/d18/d24/f30 0 2024-03-24T00:48:41.623 INFO:tasks.workunit.client.0.smithi038.stdout:6/203: rmdir d1/da/d1b/d25 39 2024-03-24T00:48:41.625 INFO:tasks.workunit.client.1.smithi088.stdout:0/195: unlink d9/dc/c31 0 2024-03-24T00:48:41.625 INFO:tasks.workunit.client.1.smithi088.stdout:0/196: truncate d9/d26/f38 391185 0 2024-03-24T00:48:41.625 INFO:tasks.workunit.client.1.smithi088.stdout:0/197: stat d9/d20/l37 0 2024-03-24T00:48:41.629 INFO:tasks.workunit.client.0.smithi038.stdout:1/161: unlink da/dd/d22/c27 0 2024-03-24T00:48:41.632 INFO:tasks.workunit.client.1.smithi088.stdout:9/179: dread d1/f5 [0,4194304] 0 2024-03-24T00:48:41.635 INFO:tasks.workunit.client.0.smithi038.stdout:9/215: rename d12/f3e to d12/d28/d3a/d26/f4a 0 2024-03-24T00:48:41.635 INFO:tasks.workunit.client.1.smithi088.stdout:7/173: dread d3/f28 [0,4194304] 0 2024-03-24T00:48:41.638 INFO:tasks.workunit.client.1.smithi088.stdout:3/158: unlink d3/d1a/d1c/f1d 0 2024-03-24T00:48:41.639 INFO:tasks.workunit.client.0.smithi038.stdout:7/169: truncate d2/f1a 1376724 0 2024-03-24T00:48:41.646 INFO:tasks.workunit.client.0.smithi038.stdout:5/154: mknod d6/d16/c2b 0 2024-03-24T00:48:41.647 INFO:tasks.workunit.client.1.smithi088.stdout:4/210: mknod d5/d17/c47 0 2024-03-24T00:48:41.650 INFO:tasks.workunit.client.1.smithi088.stdout:5/167: dread d1/ff [4194304,4194304] 0 2024-03-24T00:48:41.653 INFO:tasks.workunit.client.1.smithi088.stdout:0/198: mkdir d9/dc/de/d19/d44/d46 0 2024-03-24T00:48:41.656 INFO:tasks.workunit.client.0.smithi038.stdout:1/162: creat da/df/d2d/f36 x:0 0 0 2024-03-24T00:48:41.656 INFO:tasks.workunit.client.0.smithi038.stdout:1/163: fsync f1 0 2024-03-24T00:48:41.657 INFO:tasks.workunit.client.1.smithi088.stdout:9/180: getdents d1/d22 0 2024-03-24T00:48:41.660 INFO:tasks.workunit.client.0.smithi038.stdout:9/216: creat d12/d28/d3a/f4b x:0 0 0 2024-03-24T00:48:41.661 INFO:tasks.workunit.client.0.smithi038.stdout:9/217: readlink d12/l47 0 2024-03-24T00:48:41.661 INFO:tasks.workunit.client.0.smithi038.stdout:9/218: dread - d12/d28/d3a/f48 zero size 2024-03-24T00:48:41.665 INFO:tasks.workunit.client.1.smithi088.stdout:3/159: creat d3/d1a/d1e/d2f/f36 x:0 0 0 2024-03-24T00:48:41.672 INFO:tasks.workunit.client.0.smithi038.stdout:7/170: link d2/d9/l22 d2/d2c/l3d 0 2024-03-24T00:48:41.672 INFO:tasks.workunit.client.0.smithi038.stdout:6/204: write d1/da/fb [1509207,102731] 0 2024-03-24T00:48:41.679 INFO:tasks.workunit.client.0.smithi038.stdout:9/219: creat d12/d28/f4c x:0 0 0 2024-03-24T00:48:41.680 INFO:tasks.workunit.client.0.smithi038.stdout:7/171: dread d2/d9/da/f1d [0,4194304] 0 2024-03-24T00:48:41.689 INFO:tasks.workunit.client.0.smithi038.stdout:2/211: dwrite d4/d27/f39 [0,4194304] 0 2024-03-24T00:48:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:1/164: getdents da/df 0 2024-03-24T00:48:41.710 INFO:tasks.workunit.client.1.smithi088.stdout:4/211: creat d5/d16/d37/f48 x:0 0 0 2024-03-24T00:48:41.710 INFO:tasks.workunit.client.1.smithi088.stdout:4/212: dread - d5/d7/dd/f21 zero size 2024-03-24T00:48:41.714 INFO:tasks.workunit.client.1.smithi088.stdout:0/199: rename d9/dc/de/c34 to d9/dc/de/c47 0 2024-03-24T00:48:41.728 INFO:tasks.workunit.client.1.smithi088.stdout:5/168: dwrite d1/f31 [4194304,4194304] 0 2024-03-24T00:48:41.729 INFO:tasks.workunit.client.1.smithi088.stdout:5/169: fsync d1/d19/f2c 0 2024-03-24T00:48:41.730 INFO:tasks.workunit.client.1.smithi088.stdout:7/174: symlink d3/l39 0 2024-03-24T00:48:41.733 INFO:tasks.workunit.client.0.smithi038.stdout:9/220: link d12/d28/d3a/f30 d12/d28/f4d 0 2024-03-24T00:48:41.736 INFO:tasks.workunit.client.1.smithi088.stdout:3/160: creat d3/d1a/f37 x:0 0 0 2024-03-24T00:48:41.742 INFO:tasks.workunit.client.0.smithi038.stdout:4/167: dread da/dc/dd/f21 [0,4194304] 0 2024-03-24T00:48:41.745 INFO:tasks.workunit.client.1.smithi088.stdout:0/200: creat d9/dc/de/d19/f48 x:0 0 0 2024-03-24T00:48:41.746 INFO:tasks.workunit.client.0.smithi038.stdout:1/165: mknod da/df/c37 0 2024-03-24T00:48:41.746 INFO:tasks.workunit.client.0.smithi038.stdout:1/166: read f1 [3664016,127256] 0 2024-03-24T00:48:41.753 INFO:tasks.workunit.client.1.smithi088.stdout:9/181: dwrite d1/d10/d11/f1b [0,4194304] 0 2024-03-24T00:48:41.757 INFO:tasks.workunit.client.0.smithi038.stdout:9/221: truncate d12/d28/f4d 458135 0 2024-03-24T00:48:41.758 INFO:tasks.workunit.client.0.smithi038.stdout:9/222: write d12/f35 [710180,128616] 0 2024-03-24T00:48:41.758 INFO:tasks.workunit.client.0.smithi038.stdout:9/223: creat d12/d28/d3a/f4e x:0 0 0 2024-03-24T00:48:41.759 INFO:tasks.workunit.client.1.smithi088.stdout:5/170: dread d1/dd/f23 [0,4194304] 0 2024-03-24T00:48:41.759 INFO:tasks.workunit.client.1.smithi088.stdout:5/171: fsync d1/dd/f23 0 2024-03-24T00:48:41.772 INFO:tasks.workunit.client.1.smithi088.stdout:7/175: symlink d3/d31/l3a 0 2024-03-24T00:48:41.777 INFO:tasks.workunit.client.0.smithi038.stdout:5/155: dwrite f5 [0,4194304] 0 2024-03-24T00:48:41.787 INFO:tasks.workunit.client.0.smithi038.stdout:4/168: dread da/dc/dd/f2b [0,4194304] 0 2024-03-24T00:48:41.790 INFO:tasks.workunit.client.1.smithi088.stdout:3/161: unlink d3/d1a/d1e/d2f/c33 0 2024-03-24T00:48:41.802 INFO:tasks.workunit.client.1.smithi088.stdout:4/213: dwrite d5/d16/d1a/f3e [0,4194304] 0 2024-03-24T00:48:41.803 INFO:tasks.workunit.client.1.smithi088.stdout:0/201: creat d9/dc/f49 x:0 0 0 2024-03-24T00:48:41.803 INFO:tasks.workunit.client.1.smithi088.stdout:0/202: write d9/dc/de/f1d [2799362,113848] 0 2024-03-24T00:48:41.803 INFO:tasks.workunit.client.1.smithi088.stdout:0/203: truncate d9/d20/f39 945275 0 2024-03-24T00:48:41.803 INFO:tasks.workunit.client.1.smithi088.stdout:0/204: write d9/dc/de/f11 [660796,7229] 0 2024-03-24T00:48:41.813 INFO:tasks.workunit.client.0.smithi038.stdout:5/156: rename d6/f29 to d6/d16/d21/f2c 0 2024-03-24T00:48:41.813 INFO:tasks.workunit.client.0.smithi038.stdout:5/157: fsync d6/d1e/f27 0 2024-03-24T00:48:41.814 INFO:tasks.workunit.client.0.smithi038.stdout:6/205: dwrite d1/da/d1b/d25/f2a [0,4194304] 0 2024-03-24T00:48:41.818 INFO:tasks.workunit.client.1.smithi088.stdout:7/176: creat d3/d32/d34/f3b x:0 0 0 2024-03-24T00:48:41.820 INFO:tasks.workunit.client.1.smithi088.stdout:5/172: write d1/d19/d1e/d26/f35 [1653810,34786] 0 2024-03-24T00:48:41.820 INFO:tasks.workunit.client.1.smithi088.stdout:9/182: dwrite d1/d10/f1e [0,4194304] 0 2024-03-24T00:48:41.821 INFO:tasks.workunit.client.1.smithi088.stdout:3/162: unlink d3/d1a/d1c/f24 0 2024-03-24T00:48:41.822 INFO:tasks.workunit.client.1.smithi088.stdout:3/163: fdatasync d3/de/f16 0 2024-03-24T00:48:41.822 INFO:tasks.workunit.client.1.smithi088.stdout:3/164: dread - d3/d1a/f37 zero size 2024-03-24T00:48:41.823 INFO:tasks.workunit.client.0.smithi038.stdout:1/167: truncate da/df/f17 491251 0 2024-03-24T00:48:41.828 INFO:tasks.workunit.client.0.smithi038.stdout:5/158: mkdir d6/d16/d21/d2d 0 2024-03-24T00:48:41.828 INFO:tasks.workunit.client.0.smithi038.stdout:5/159: write d6/d16/f2a [1928518,52492] 0 2024-03-24T00:48:41.829 INFO:tasks.workunit.client.0.smithi038.stdout:5/160: write d6/d1e/f27 [280081,109985] 0 2024-03-24T00:48:41.831 INFO:tasks.workunit.client.0.smithi038.stdout:6/206: mkdir d1/da/d1b/d1d/d3c 0 2024-03-24T00:48:41.832 INFO:tasks.workunit.client.1.smithi088.stdout:4/214: link d5/d13/d18/d24/l39 d5/d16/l49 0 2024-03-24T00:48:41.838 INFO:tasks.workunit.client.0.smithi038.stdout:2/212: dwrite d4/f2b [0,4194304] 0 2024-03-24T00:48:41.840 INFO:tasks.workunit.client.1.smithi088.stdout:0/205: rmdir d9/d26 39 2024-03-24T00:48:41.840 INFO:tasks.workunit.client.1.smithi088.stdout:0/206: dread - d9/dc/de/f40 zero size 2024-03-24T00:48:41.846 INFO:tasks.workunit.client.1.smithi088.stdout:8/186: sync 2024-03-24T00:48:41.849 INFO:tasks.workunit.client.1.smithi088.stdout:9/183: getdents d1/d15 0 2024-03-24T00:48:41.852 INFO:tasks.workunit.client.1.smithi088.stdout:8/187: rmdir d9/dc 39 2024-03-24T00:48:41.855 INFO:tasks.workunit.client.1.smithi088.stdout:9/184: link d1/c1c d1/d10/d2c/c2f 0 2024-03-24T00:48:41.857 INFO:tasks.workunit.client.0.smithi038.stdout:1/168: rename da/fb to da/dd/d10/d14/f38 0 2024-03-24T00:48:41.857 INFO:tasks.workunit.client.0.smithi038.stdout:5/161: symlink d6/d16/l2e 0 2024-03-24T00:48:41.857 INFO:tasks.workunit.client.0.smithi038.stdout:5/162: chown f5 2016406 1 2024-03-24T00:48:41.857 INFO:tasks.workunit.client.0.smithi038.stdout:5/163: truncate d6/d1e/f27 716307 0 2024-03-24T00:48:41.857 INFO:tasks.workunit.client.1.smithi088.stdout:4/215: getdents d5/d16/d1a/d22/d2f 0 2024-03-24T00:48:41.859 INFO:tasks.workunit.client.1.smithi088.stdout:8/188: link d9/f28 d9/dc/f44 0 2024-03-24T00:48:41.859 INFO:tasks.workunit.client.1.smithi088.stdout:8/189: dread - d9/f3e zero size 2024-03-24T00:48:41.860 INFO:tasks.workunit.client.1.smithi088.stdout:8/190: write d9/f37 [62877,14808] 0 2024-03-24T00:48:41.860 INFO:tasks.workunit.client.1.smithi088.stdout:8/191: read d9/dc/d11/d24/d29/f2c [57678,86575] 0 2024-03-24T00:48:41.860 INFO:tasks.workunit.client.1.smithi088.stdout:8/192: write f2 [3494354,2508] 0 2024-03-24T00:48:41.861 INFO:tasks.workunit.client.1.smithi088.stdout:8/193: creat d9/dc/d11/d24/f45 x:0 0 0 2024-03-24T00:48:41.861 INFO:tasks.workunit.client.1.smithi088.stdout:8/194: truncate d9/f3e 115124 0 2024-03-24T00:48:41.861 INFO:tasks.workunit.client.1.smithi088.stdout:8/195: dread - d9/dc/d11/d24/f45 zero size 2024-03-24T00:48:41.861 INFO:tasks.workunit.client.1.smithi088.stdout:8/196: fsync d9/dc/d11/d24/f2a 0 2024-03-24T00:48:41.862 INFO:tasks.workunit.client.0.smithi038.stdout:2/213: mknod d4/d27/d2f/d30/d3e/c4a 0 2024-03-24T00:48:41.868 INFO:tasks.workunit.client.1.smithi088.stdout:9/185: symlink d1/d10/d11/d28/l30 0 2024-03-24T00:48:41.868 INFO:tasks.workunit.client.1.smithi088.stdout:9/186: write d1/d10/d29/f2e [34004,2320] 0 2024-03-24T00:48:41.880 INFO:tasks.workunit.client.0.smithi038.stdout:4/169: rmdir da/dc/dd/d15 39 2024-03-24T00:48:41.882 INFO:tasks.workunit.client.0.smithi038.stdout:5/164: write d6/d1e/f1f [1933357,106727] 0 2024-03-24T00:48:41.884 INFO:tasks.workunit.client.1.smithi088.stdout:8/197: creat d9/dc/d11/d24/d29/d36/f46 x:0 0 0 2024-03-24T00:48:41.888 INFO:tasks.workunit.client.1.smithi088.stdout:9/187: mknod d1/d22/d27/c31 0 2024-03-24T00:48:41.888 INFO:tasks.workunit.client.1.smithi088.stdout:9/188: chown d1/c2 22 1 2024-03-24T00:48:41.891 INFO:tasks.workunit.client.1.smithi088.stdout:8/198: fsync d9/dc/d11/f2f 0 2024-03-24T00:48:41.891 INFO:tasks.workunit.client.0.smithi038.stdout:4/170: symlink da/d12/d29/l31 0 2024-03-24T00:48:41.899 INFO:tasks.workunit.client.1.smithi088.stdout:9/189: symlink d1/d10/d11/l32 0 2024-03-24T00:48:41.899 INFO:tasks.workunit.client.1.smithi088.stdout:9/190: readlink d1/d10/d11/l32 0 2024-03-24T00:48:41.899 INFO:tasks.workunit.client.1.smithi088.stdout:9/191: chown d1/d10/d11/d23 6602 1 2024-03-24T00:48:41.899 INFO:tasks.workunit.client.1.smithi088.stdout:7/177: dwrite d3/f1f [0,4194304] 0 2024-03-24T00:48:41.904 INFO:tasks.workunit.client.0.smithi038.stdout:4/171: mknod da/d27/c32 0 2024-03-24T00:48:41.905 INFO:tasks.workunit.client.1.smithi088.stdout:8/199: unlink d9/f10 0 2024-03-24T00:48:41.908 INFO:tasks.workunit.client.0.smithi038.stdout:6/207: dwrite d1/da/d1b/f26 [0,4194304] 0 2024-03-24T00:48:41.924 INFO:tasks.workunit.client.1.smithi088.stdout:3/165: dwrite d3/de/f16 [0,4194304] 0 2024-03-24T00:48:41.925 INFO:tasks.workunit.client.1.smithi088.stdout:3/166: creat d3/d1a/f38 x:0 0 0 2024-03-24T00:48:41.925 INFO:tasks.workunit.client.1.smithi088.stdout:9/192: write d1/f7 [1328715,24844] 0 2024-03-24T00:48:41.925 INFO:tasks.workunit.client.1.smithi088.stdout:4/216: dwrite d5/d17/f1c [0,4194304] 0 2024-03-24T00:48:41.938 INFO:tasks.workunit.client.1.smithi088.stdout:9/193: unlink d1/d10/d11/c14 0 2024-03-24T00:48:41.939 INFO:tasks.workunit.client.0.smithi038.stdout:6/208: write d1/f1f [370400,89097] 0 2024-03-24T00:48:41.942 INFO:tasks.workunit.client.1.smithi088.stdout:4/217: mknod d5/d16/d1a/c4a 0 2024-03-24T00:48:41.949 INFO:tasks.workunit.client.1.smithi088.stdout:6/217: rename db/d1f to db/d1d/d23/d3c/d43 0 2024-03-24T00:48:41.949 INFO:tasks.workunit.client.1.smithi088.stdout:6/218: fsync db/dd/f37 0 2024-03-24T00:48:41.954 INFO:tasks.workunit.client.0.smithi038.stdout:2/214: dwrite d4/d18/d26/f2c [0,4194304] 0 2024-03-24T00:48:41.954 INFO:tasks.workunit.client.0.smithi038.stdout:2/215: fdatasync d4/f23 0 2024-03-24T00:48:41.968 INFO:tasks.workunit.client.0.smithi038.stdout:6/209: creat d1/da/d1b/d2f/f3d x:0 0 0 2024-03-24T00:48:41.970 INFO:tasks.workunit.client.1.smithi088.stdout:5/173: rename c0 to d1/d19/c3a 0 2024-03-24T00:48:41.970 INFO:tasks.workunit.client.1.smithi088.stdout:5/174: creat d1/f3b x:0 0 0 2024-03-24T00:48:41.971 INFO:tasks.workunit.client.0.smithi038.stdout:1/169: dwrite f7 [0,4194304] 0 2024-03-24T00:48:41.976 INFO:tasks.workunit.client.0.smithi038.stdout:6/210: getdents d1/da 0 2024-03-24T00:48:41.977 INFO:tasks.workunit.client.1.smithi088.stdout:8/200: dwrite d9/dc/f44 [0,4194304] 0 2024-03-24T00:48:41.980 INFO:tasks.workunit.client.1.smithi088.stdout:9/194: read d1/f8 [3621707,118029] 0 2024-03-24T00:48:41.986 INFO:tasks.workunit.client.0.smithi038.stdout:6/211: mknod d1/da/d1b/d2f/c3e 0 2024-03-24T00:48:41.991 INFO:tasks.workunit.client.1.smithi088.stdout:0/207: rename d9/dc/de/f3d to d9/dc/de/d19/d44/f4a 0 2024-03-24T00:48:41.991 INFO:tasks.workunit.client.0.smithi038.stdout:6/212: mknod d1/da/c3f 0 2024-03-24T00:48:41.991 INFO:tasks.workunit.client.0.smithi038.stdout:6/213: write d1/da/f14 [2847828,101901] 0 2024-03-24T00:48:41.994 INFO:tasks.workunit.client.0.smithi038.stdout:5/165: dwrite d6/f9 [0,4194304] 0 2024-03-24T00:48:41.998 INFO:tasks.workunit.client.0.smithi038.stdout:6/214: symlink d1/d37/l40 0 2024-03-24T00:48:41.999 INFO:tasks.workunit.client.0.smithi038.stdout:2/216: dread d4/ff [0,4194304] 0 2024-03-24T00:48:42.001 INFO:tasks.workunit.client.0.smithi038.stdout:2/217: chown d4/f10 6 1 2024-03-24T00:48:42.002 INFO:tasks.workunit.client.0.smithi038.stdout:5/166: symlink d6/d19/l2f 0 2024-03-24T00:48:42.002 INFO:tasks.workunit.client.1.smithi088.stdout:6/219: rename db/d1d/d23/d3c/c3f to db/d1d/d23/d3c/d43/d24/c44 0 2024-03-24T00:48:42.003 INFO:tasks.workunit.client.1.smithi088.stdout:6/220: chown db/d40 733899749 1 2024-03-24T00:48:42.004 INFO:tasks.workunit.client.0.smithi038.stdout:5/167: chown d6/d16/d14/f24 13975935 1 2024-03-24T00:48:42.021 INFO:tasks.workunit.client.1.smithi088.stdout:7/178: getdents d3/d31 0 2024-03-24T00:48:42.021 INFO:tasks.workunit.client.1.smithi088.stdout:7/179: write d3/d32/d34/f3b [894306,12815] 0 2024-03-24T00:48:42.023 INFO:tasks.workunit.client.1.smithi088.stdout:3/167: dwrite d3/d1a/d1c/f30 [0,4194304] 0 2024-03-24T00:48:42.023 INFO:tasks.workunit.client.1.smithi088.stdout:3/168: creat d3/d1a/d1c/f39 x:0 0 0 2024-03-24T00:48:42.024 INFO:tasks.workunit.client.0.smithi038.stdout:6/215: write d1/da/d1b/f26 [3772808,70891] 0 2024-03-24T00:48:42.025 INFO:tasks.workunit.client.1.smithi088.stdout:7/180: creat d3/d31/f3c x:0 0 0 2024-03-24T00:48:42.034 INFO:tasks.workunit.client.0.smithi038.stdout:4/172: dwrite da/d12/d24/f26 [0,4194304] 0 2024-03-24T00:48:42.036 INFO:tasks.workunit.client.1.smithi088.stdout:5/175: dwrite d1/d10/f17 [0,4194304] 0 2024-03-24T00:48:42.039 INFO:tasks.workunit.client.0.smithi038.stdout:6/216: symlink d1/l41 0 2024-03-24T00:48:42.043 INFO:tasks.workunit.client.1.smithi088.stdout:4/218: dwrite d5/d13/d18/f32 [0,4194304] 0 2024-03-24T00:48:42.043 INFO:tasks.workunit.client.1.smithi088.stdout:4/219: creat d5/f4b x:0 0 0 2024-03-24T00:48:42.049 INFO:tasks.workunit.client.1.smithi088.stdout:3/169: mknod d3/d1a/d1c/c3a 0 2024-03-24T00:48:42.049 INFO:tasks.workunit.client.1.smithi088.stdout:3/170: chown d3/d1a/d1e/f28 51 1 2024-03-24T00:48:42.052 INFO:tasks.workunit.client.1.smithi088.stdout:0/208: dwrite d9/dc/de/d19/f1a [0,4194304] 0 2024-03-24T00:48:42.052 INFO:tasks.workunit.client.1.smithi088.stdout:0/209: write d9/d20/f35 [999219,89055] 0 2024-03-24T00:48:42.055 INFO:tasks.workunit.client.0.smithi038.stdout:2/218: dwrite d4/f1d [0,4194304] 0 2024-03-24T00:48:42.055 INFO:tasks.workunit.client.1.smithi088.stdout:5/176: mknod d1/d10/d1b/c3c 0 2024-03-24T00:48:42.057 INFO:tasks.workunit.client.0.smithi038.stdout:6/217: symlink d1/da/d1b/l42 0 2024-03-24T00:48:42.058 INFO:tasks.workunit.client.1.smithi088.stdout:8/201: dwrite d9/dc/d11/d24/d29/d36/f46 [0,4194304] 0 2024-03-24T00:48:42.058 INFO:tasks.workunit.client.1.smithi088.stdout:8/202: write d9/dc/d11/d24/f45 [591770,24108] 0 2024-03-24T00:48:42.058 INFO:tasks.workunit.client.1.smithi088.stdout:8/203: rename d9/dc/d11/d24/d29/d39 to d9/dc/d11/d24/d29/d39/d47 22 2024-03-24T00:48:42.059 INFO:tasks.workunit.client.1.smithi088.stdout:6/221: dwrite db/dd/f41 [0,4194304] 0 2024-03-24T00:48:42.059 INFO:tasks.workunit.client.1.smithi088.stdout:6/222: dread - db/dd/f2b zero size 2024-03-24T00:48:42.060 INFO:tasks.workunit.client.0.smithi038.stdout:4/173: mkdir da/dc/dd/d15/d33 0 2024-03-24T00:48:42.061 INFO:tasks.workunit.client.1.smithi088.stdout:7/181: dread d3/f17 [0,4194304] 0 2024-03-24T00:48:42.061 INFO:tasks.workunit.client.1.smithi088.stdout:7/182: chown c2 25169757 1 2024-03-24T00:48:42.061 INFO:tasks.workunit.client.1.smithi088.stdout:7/183: truncate d3/d32/d34/f3b 1761665 0 2024-03-24T00:48:42.068 INFO:tasks.workunit.client.0.smithi038.stdout:6/218: rmdir d1/d37 39 2024-03-24T00:48:42.072 INFO:tasks.workunit.client.1.smithi088.stdout:3/171: link d3/de/df/f15 d3/d6/f3b 0 2024-03-24T00:48:42.073 INFO:tasks.workunit.client.0.smithi038.stdout:7/172: mknod d2/c3e 0 2024-03-24T00:48:42.074 INFO:tasks.workunit.client.0.smithi038.stdout:7/173: creat d2/d9/da/f3f x:0 0 0 2024-03-24T00:48:42.079 INFO:tasks.workunit.client.0.smithi038.stdout:6/219: unlink d1/ld 0 2024-03-24T00:48:42.080 INFO:tasks.workunit.client.0.smithi038.stdout:0/141: mkdir d0/d2d 0 2024-03-24T00:48:42.085 INFO:tasks.workunit.client.0.smithi038.stdout:2/219: rmdir d4 39 2024-03-24T00:48:42.085 INFO:tasks.workunit.client.1.smithi088.stdout:0/210: symlink d9/d20/l4b 0 2024-03-24T00:48:42.086 INFO:tasks.workunit.client.1.smithi088.stdout:0/211: write d9/f45 [257388,92938] 0 2024-03-24T00:48:42.087 INFO:tasks.workunit.client.1.smithi088.stdout:5/177: mknod d1/d10/d1b/d2e/c3d 0 2024-03-24T00:48:42.087 INFO:tasks.workunit.client.1.smithi088.stdout:5/178: chown d1/d19 138884784 1 2024-03-24T00:48:42.090 INFO:tasks.workunit.client.0.smithi038.stdout:6/220: creat d1/da/d1b/d1d/f43 x:0 0 0 2024-03-24T00:48:42.091 INFO:tasks.workunit.client.0.smithi038.stdout:9/224: dwrite d12/d28/f4d [0,4194304] 0 2024-03-24T00:48:42.091 INFO:tasks.workunit.client.0.smithi038.stdout:9/225: chown fc 26596272 1 2024-03-24T00:48:42.091 INFO:tasks.workunit.client.0.smithi038.stdout:9/226: chown f8 55 1 2024-03-24T00:48:42.091 INFO:tasks.workunit.client.0.smithi038.stdout:9/227: truncate d12/d28/f36 763378 0 2024-03-24T00:48:42.092 INFO:tasks.workunit.client.1.smithi088.stdout:9/195: dwrite d1/ff [0,4194304] 0 2024-03-24T00:48:42.092 INFO:tasks.workunit.client.1.smithi088.stdout:9/196: write d1/d10/f1d [1218683,40739] 0 2024-03-24T00:48:42.092 INFO:tasks.workunit.client.0.smithi038.stdout:0/142: mknod d0/d2d/c2e 0 2024-03-24T00:48:42.106 INFO:tasks.workunit.client.0.smithi038.stdout:2/220: getdents d4 0 2024-03-24T00:48:42.109 INFO:tasks.workunit.client.1.smithi088.stdout:8/204: creat d9/dc/d11/d24/d29/d39/f48 x:0 0 0 2024-03-24T00:48:42.112 INFO:tasks.workunit.client.0.smithi038.stdout:9/228: dread f4 [0,4194304] 0 2024-03-24T00:48:42.112 INFO:tasks.workunit.client.1.smithi088.stdout:7/184: stat d3/l1e 0 2024-03-24T00:48:42.123 INFO:tasks.workunit.client.0.smithi038.stdout:0/143: dread d0/ff [4194304,4194304] 0 2024-03-24T00:48:42.129 INFO:tasks.workunit.client.1.smithi088.stdout:0/212: mknod d9/c4c 0 2024-03-24T00:48:42.129 INFO:tasks.workunit.client.1.smithi088.stdout:0/213: dread - d9/dc/f49 zero size 2024-03-24T00:48:42.130 INFO:tasks.workunit.client.1.smithi088.stdout:0/214: chown d9/dc/de/d19/d44/d46 22762066 1 2024-03-24T00:48:42.130 INFO:tasks.workunit.client.1.smithi088.stdout:0/215: write d9/dc/de/f40 [324859,85345] 0 2024-03-24T00:48:42.133 INFO:tasks.workunit.client.1.smithi088.stdout:8/205: symlink d9/dc/l49 0 2024-03-24T00:48:42.133 INFO:tasks.workunit.client.0.smithi038.stdout:2/221: dread d4/d18/f1f [0,4194304] 0 2024-03-24T00:48:42.135 INFO:tasks.workunit.client.0.smithi038.stdout:9/229: dread fc [0,4194304] 0 2024-03-24T00:48:42.138 INFO:tasks.workunit.client.0.smithi038.stdout:9/230: fdatasync d12/d1a/f20 0 2024-03-24T00:48:42.141 INFO:tasks.workunit.client.0.smithi038.stdout:0/144: unlink d0/ff 0 2024-03-24T00:48:42.141 INFO:tasks.workunit.client.0.smithi038.stdout:0/145: write d0/d12/d15/f16 [4448588,44785] 0 2024-03-24T00:48:42.142 INFO:tasks.workunit.client.0.smithi038.stdout:0/146: fdatasync d0/d12/d15/d1a/f1e 0 2024-03-24T00:48:42.148 INFO:tasks.workunit.client.1.smithi088.stdout:5/179: write d1/d10/f17 [1523191,126813] 0 2024-03-24T00:48:42.149 INFO:tasks.workunit.client.1.smithi088.stdout:7/185: fsync d3/f1b 0 2024-03-24T00:48:42.149 INFO:tasks.workunit.client.1.smithi088.stdout:7/186: chown d3/l39 29 1 2024-03-24T00:48:42.151 INFO:tasks.workunit.client.0.smithi038.stdout:2/222: mkdir d4/d18/d26/d4b 0 2024-03-24T00:48:42.153 INFO:tasks.workunit.client.0.smithi038.stdout:9/231: write d12/d28/d3a/d26/f3d [891810,11623] 0 2024-03-24T00:48:42.156 INFO:tasks.workunit.client.1.smithi088.stdout:2/170: sync 2024-03-24T00:48:42.156 INFO:tasks.workunit.client.1.smithi088.stdout:1/200: sync 2024-03-24T00:48:42.157 INFO:tasks.workunit.client.1.smithi088.stdout:5/180: mknod d1/d10/c3e 0 2024-03-24T00:48:42.158 INFO:tasks.workunit.client.1.smithi088.stdout:3/172: dwrite d3/d1a/d1e/f28 [0,4194304] 0 2024-03-24T00:48:42.160 INFO:tasks.workunit.client.0.smithi038.stdout:2/223: unlink d4/d18/d26/f29 0 2024-03-24T00:48:42.160 INFO:tasks.workunit.client.0.smithi038.stdout:2/224: chown d4/d27/f39 0 1 2024-03-24T00:48:42.161 INFO:tasks.workunit.client.1.smithi088.stdout:8/206: dread d9/dc/d11/f31 [0,4194304] 0 2024-03-24T00:48:42.162 INFO:tasks.workunit.client.1.smithi088.stdout:7/187: chown d3/l22 114 1 2024-03-24T00:48:42.165 INFO:tasks.workunit.client.0.smithi038.stdout:9/232: creat d12/d41/f4f x:0 0 0 2024-03-24T00:48:42.166 INFO:tasks.workunit.client.0.smithi038.stdout:9/233: dread - d12/d28/d3a/f48 zero size 2024-03-24T00:48:42.167 INFO:tasks.workunit.client.0.smithi038.stdout:2/225: symlink d4/d27/d2f/d30/l4c 0 2024-03-24T00:48:42.167 INFO:tasks.workunit.client.0.smithi038.stdout:2/226: dread - d4/d18/f48 zero size 2024-03-24T00:48:42.170 INFO:tasks.workunit.client.1.smithi088.stdout:1/201: creat d6/d27/f3c x:0 0 0 2024-03-24T00:48:42.170 INFO:tasks.workunit.client.1.smithi088.stdout:1/202: readlink d6/d27/l36 0 2024-03-24T00:48:42.175 INFO:tasks.workunit.client.1.smithi088.stdout:6/223: dwrite db/d1d/d23/d3c/d43/f34 [4194304,4194304] 0 2024-03-24T00:48:42.175 INFO:tasks.workunit.client.0.smithi038.stdout:2/227: symlink d4/d27/d2f/d30/d3e/l4d 0 2024-03-24T00:48:42.176 INFO:tasks.workunit.client.1.smithi088.stdout:9/197: dwrite d1/d10/d11/f17 [0,4194304] 0 2024-03-24T00:48:42.177 INFO:tasks.workunit.client.0.smithi038.stdout:0/147: write d0/d12/d15/f16 [178128,50665] 0 2024-03-24T00:48:42.182 INFO:tasks.workunit.client.1.smithi088.stdout:6/224: dread db/f19 [0,4194304] 0 2024-03-24T00:48:42.183 INFO:tasks.workunit.client.0.smithi038.stdout:0/148: read d0/d12/d15/f17 [8567,57350] 0 2024-03-24T00:48:42.184 INFO:tasks.workunit.client.0.smithi038.stdout:7/174: dwrite d2/d9/da/f1d [0,4194304] 0 2024-03-24T00:48:42.195 INFO:tasks.workunit.client.0.smithi038.stdout:9/234: rmdir d12/d28 39 2024-03-24T00:48:42.196 INFO:tasks.workunit.client.1.smithi088.stdout:5/181: mkdir d1/d29/d3f 0 2024-03-24T00:48:42.200 INFO:tasks.workunit.client.1.smithi088.stdout:8/207: mkdir d9/dc/d11/d24/d43/d4a 0 2024-03-24T00:48:42.203 INFO:tasks.workunit.client.1.smithi088.stdout:7/188: write d3/ff [452485,64290] 0 2024-03-24T00:48:42.204 INFO:tasks.workunit.client.0.smithi038.stdout:0/149: creat d0/d2d/f2f x:0 0 0 2024-03-24T00:48:42.204 INFO:tasks.workunit.client.0.smithi038.stdout:0/150: dread - d0/d12/d15/f22 zero size 2024-03-24T00:48:42.208 INFO:tasks.workunit.client.1.smithi088.stdout:2/171: symlink db/d21/d35/l39 0 2024-03-24T00:48:42.211 INFO:tasks.workunit.client.0.smithi038.stdout:4/174: truncate da/d12/d24/f26 2316179 0 2024-03-24T00:48:42.211 INFO:tasks.workunit.client.0.smithi038.stdout:4/175: stat da 0 2024-03-24T00:48:42.211 INFO:tasks.workunit.client.0.smithi038.stdout:4/176: write da/dc/dd/f2b [1160903,7092] 0 2024-03-24T00:48:42.215 INFO:tasks.workunit.client.1.smithi088.stdout:4/220: rmdir d5 39 2024-03-24T00:48:42.215 INFO:tasks.workunit.client.1.smithi088.stdout:4/221: truncate d5/d7/fe 462774 0 2024-03-24T00:48:42.215 INFO:tasks.workunit.client.1.smithi088.stdout:4/222: chown d5/d16/l49 4030821 1 2024-03-24T00:48:42.215 INFO:tasks.workunit.client.1.smithi088.stdout:4/223: read - d5/d7/f14 zero size 2024-03-24T00:48:42.219 INFO:tasks.workunit.client.0.smithi038.stdout:3/200: sync 2024-03-24T00:48:42.219 INFO:tasks.workunit.client.0.smithi038.stdout:3/201: readlink d3/d2a/l1b 0 2024-03-24T00:48:42.219 INFO:tasks.workunit.client.0.smithi038.stdout:3/202: write d3/d9/f16 [1334201,27691] 0 2024-03-24T00:48:42.219 INFO:tasks.workunit.client.0.smithi038.stdout:3/203: dread - d3/f29 zero size 2024-03-24T00:48:42.219 INFO:tasks.workunit.client.0.smithi038.stdout:8/179: sync 2024-03-24T00:48:42.222 INFO:tasks.workunit.client.0.smithi038.stdout:6/221: dwrite d1/da/d1b/d1d/f2c [0,4194304] 0 2024-03-24T00:48:42.222 INFO:tasks.workunit.client.0.smithi038.stdout:6/222: readlink d1/da/d1b/d1d/l1e 0 2024-03-24T00:48:42.229 INFO:tasks.workunit.client.1.smithi088.stdout:0/216: dwrite d9/dc/de/f40 [0,4194304] 0 2024-03-24T00:48:42.229 INFO:tasks.workunit.client.1.smithi088.stdout:0/217: stat d9/d26 0 2024-03-24T00:48:42.230 INFO:tasks.workunit.client.0.smithi038.stdout:7/175: creat d2/d9/df/d33/d37/f40 x:0 0 0 2024-03-24T00:48:42.231 INFO:tasks.workunit.client.0.smithi038.stdout:7/176: creat d2/d9/da/f41 x:0 0 0 2024-03-24T00:48:42.234 INFO:tasks.workunit.client.1.smithi088.stdout:1/203: dwrite d6/ff [4194304,4194304] 0 2024-03-24T00:48:42.234 INFO:tasks.workunit.client.1.smithi088.stdout:1/204: creat d6/d2f/f3d x:0 0 0 2024-03-24T00:48:42.235 INFO:tasks.workunit.client.0.smithi038.stdout:9/235: creat d12/d28/d3a/f50 x:0 0 0 2024-03-24T00:48:42.239 INFO:tasks.workunit.client.0.smithi038.stdout:3/204: mknod d3/d30/c48 0 2024-03-24T00:48:42.241 INFO:tasks.workunit.client.1.smithi088.stdout:6/225: symlink db/d1d/d23/d3c/l45 0 2024-03-24T00:48:42.244 INFO:tasks.workunit.client.1.smithi088.stdout:0/218: dread d9/dc/de/d19/f3b [0,4194304] 0 2024-03-24T00:48:42.251 INFO:tasks.workunit.client.0.smithi038.stdout:8/180: mknod d0/d15/c43 0 2024-03-24T00:48:42.253 INFO:tasks.workunit.client.1.smithi088.stdout:5/182: getdents d1/d10/d1b 0 2024-03-24T00:48:42.258 INFO:tasks.workunit.client.0.smithi038.stdout:6/223: rename d1/f9 to d1/da/d1b/d25/f44 0 2024-03-24T00:48:42.258 INFO:tasks.workunit.client.0.smithi038.stdout:6/224: chown d1/f7 23083722 1 2024-03-24T00:48:42.259 INFO:tasks.workunit.client.0.smithi038.stdout:0/151: dread d0/fa [4194304,4194304] 0 2024-03-24T00:48:42.259 INFO:tasks.workunit.client.0.smithi038.stdout:0/152: readlink d0/l29 0 2024-03-24T00:48:42.260 INFO:tasks.workunit.client.1.smithi088.stdout:7/189: symlink d3/d31/l3d 0 2024-03-24T00:48:42.260 INFO:tasks.workunit.client.1.smithi088.stdout:4/224: symlink d5/d16/d1a/d22/l4c 0 2024-03-24T00:48:42.260 INFO:tasks.workunit.client.1.smithi088.stdout:4/225: dread - d5/d13/d18/f25 zero size 2024-03-24T00:48:42.260 INFO:tasks.workunit.client.0.smithi038.stdout:0/153: write d0/d12/d15/f16 [5070408,42927] 0 2024-03-24T00:48:42.260 INFO:tasks.workunit.client.0.smithi038.stdout:0/154: dread - d0/d2d/f2f zero size 2024-03-24T00:48:42.260 INFO:tasks.workunit.client.0.smithi038.stdout:0/155: write d0/d2d/f2f [60858,128453] 0 2024-03-24T00:48:42.267 INFO:tasks.workunit.client.1.smithi088.stdout:6/226: creat db/d40/f46 x:0 0 0 2024-03-24T00:48:42.267 INFO:tasks.workunit.client.1.smithi088.stdout:6/227: truncate db/dd/f2b 160158 0 2024-03-24T00:48:42.271 INFO:tasks.workunit.client.1.smithi088.stdout:3/173: rename d3/d1a/d1c to d3/d6/d27/d3c 0 2024-03-24T00:48:42.272 INFO:tasks.workunit.client.1.smithi088.stdout:5/183: symlink d1/d10/l40 0 2024-03-24T00:48:42.273 INFO:tasks.workunit.client.1.smithi088.stdout:7/190: truncate d3/f2b 993521 0 2024-03-24T00:48:42.275 INFO:tasks.workunit.client.0.smithi038.stdout:9/236: creat d12/d28/f51 x:0 0 0 2024-03-24T00:48:42.275 INFO:tasks.workunit.client.0.smithi038.stdout:9/237: dread - d12/d28/f4c zero size 2024-03-24T00:48:42.276 INFO:tasks.workunit.client.1.smithi088.stdout:2/172: mknod db/d21/c3a 0 2024-03-24T00:48:42.279 INFO:tasks.workunit.client.1.smithi088.stdout:6/228: truncate db/dd/f33 150783 0 2024-03-24T00:48:42.280 INFO:tasks.workunit.client.1.smithi088.stdout:6/229: truncate db/d1d/d23/d3c/d43/f39 857745 0 2024-03-24T00:48:42.280 INFO:tasks.workunit.client.1.smithi088.stdout:6/230: write db/d1d/d23/d3c/d43/f28 [1744223,84752] 0 2024-03-24T00:48:42.280 INFO:tasks.workunit.client.1.smithi088.stdout:6/231: chown db/d1d/d23 6 1 2024-03-24T00:48:42.280 INFO:tasks.workunit.client.1.smithi088.stdout:6/232: fdatasync db/f35 0 2024-03-24T00:48:42.281 INFO:tasks.workunit.client.0.smithi038.stdout:6/225: write d1/da/d1b/f20 [186930,86672] 0 2024-03-24T00:48:42.282 INFO:tasks.workunit.client.1.smithi088.stdout:3/174: rename d3/de/df/c18 to d3/d1a/d1e/d2f/c3d 0 2024-03-24T00:48:42.284 INFO:tasks.workunit.client.1.smithi088.stdout:5/184: unlink d1/d19/d1e/d26/f35 0 2024-03-24T00:48:42.285 INFO:tasks.workunit.client.1.smithi088.stdout:7/191: symlink d3/d31/l3e 0 2024-03-24T00:48:42.285 INFO:tasks.workunit.client.0.smithi038.stdout:4/177: getdents da/dc 0 2024-03-24T00:48:42.286 INFO:tasks.workunit.client.1.smithi088.stdout:3/175: write d3/d6/fb [5089672,122546] 0 2024-03-24T00:48:42.286 INFO:tasks.workunit.client.0.smithi038.stdout:0/156: link d0/d12/d15/c25 d0/d1b/c30 0 2024-03-24T00:48:42.290 INFO:tasks.workunit.client.0.smithi038.stdout:2/228: dwrite d4/d18/d26/d37/f43 [0,4194304] 0 2024-03-24T00:48:42.292 INFO:tasks.workunit.client.0.smithi038.stdout:7/177: dwrite d2/d9/da/d23/f39 [0,4194304] 0 2024-03-24T00:48:42.293 INFO:tasks.workunit.client.1.smithi088.stdout:5/185: creat d1/d29/d3f/f41 x:0 0 0 2024-03-24T00:48:42.299 INFO:tasks.workunit.client.1.smithi088.stdout:3/176: mknod d3/d1a/d1e/c3e 0 2024-03-24T00:48:42.300 INFO:tasks.workunit.client.0.smithi038.stdout:4/178: creat da/dc/dd/f34 x:0 0 0 2024-03-24T00:48:42.301 INFO:tasks.workunit.client.1.smithi088.stdout:3/177: write d3/d1a/f37 [295484,94940] 0 2024-03-24T00:48:42.301 INFO:tasks.workunit.client.1.smithi088.stdout:8/208: dwrite d9/f37 [0,4194304] 0 2024-03-24T00:48:42.301 INFO:tasks.workunit.client.1.smithi088.stdout:8/209: readlink d9/dc/d11/d24/l25 0 2024-03-24T00:48:42.308 INFO:tasks.workunit.client.0.smithi038.stdout:3/205: link d3/d9/dc/df/f12 d3/f49 0 2024-03-24T00:48:42.309 INFO:tasks.workunit.client.0.smithi038.stdout:0/157: creat d0/d1b/d2c/f31 x:0 0 0 2024-03-24T00:48:42.313 INFO:tasks.workunit.client.1.smithi088.stdout:5/186: creat d1/d19/d1e/f42 x:0 0 0 2024-03-24T00:48:42.313 INFO:tasks.workunit.client.1.smithi088.stdout:5/187: fsync d1/f31 0 2024-03-24T00:48:42.313 INFO:tasks.workunit.client.1.smithi088.stdout:5/188: write d1/dd/f15 [917586,87094] 0 2024-03-24T00:48:42.314 INFO:tasks.workunit.client.1.smithi088.stdout:5/189: read d1/d19/f30 [7620191,54464] 0 2024-03-24T00:48:42.314 INFO:tasks.workunit.client.1.smithi088.stdout:5/190: chown d1/d29/l34 2364086 1 2024-03-24T00:48:42.318 INFO:tasks.workunit.client.0.smithi038.stdout:6/226: write d1/f7 [3070803,115726] 0 2024-03-24T00:48:42.318 INFO:tasks.workunit.client.0.smithi038.stdout:6/227: chown d1/da/d1b/d1d/d3c 23055275 1 2024-03-24T00:48:42.318 INFO:tasks.workunit.client.0.smithi038.stdout:6/228: chown d1/l19 864248 1 2024-03-24T00:48:42.318 INFO:tasks.workunit.client.1.smithi088.stdout:0/219: dwrite d9/f3a [0,4194304] 0 2024-03-24T00:48:42.318 INFO:tasks.workunit.client.1.smithi088.stdout:0/220: chown d9/dc/de/d19/l42 21373229 1 2024-03-24T00:48:42.320 INFO:tasks.workunit.client.0.smithi038.stdout:2/229: rename d4/d18/f48 to d4/d18/d26/f4e 0 2024-03-24T00:48:42.337 INFO:tasks.workunit.client.1.smithi088.stdout:8/210: write d9/dc/f44 [3622857,72180] 0 2024-03-24T00:48:42.337 INFO:tasks.workunit.client.1.smithi088.stdout:1/205: rmdir d6/d2f 39 2024-03-24T00:48:42.339 INFO:tasks.workunit.client.1.smithi088.stdout:0/221: mknod d9/d20/d2a/c4d 0 2024-03-24T00:48:42.341 INFO:tasks.workunit.client.1.smithi088.stdout:9/198: truncate d1/f6 3975773 0 2024-03-24T00:48:42.342 INFO:tasks.workunit.client.0.smithi038.stdout:3/206: symlink d3/d30/l4a 0 2024-03-24T00:48:42.342 INFO:tasks.workunit.client.1.smithi088.stdout:0/222: write d9/dc/de/d19/f3b [432279,77296] 0 2024-03-24T00:48:42.342 INFO:tasks.workunit.client.1.smithi088.stdout:0/223: fsync d9/dc/de/f3e 0 2024-03-24T00:48:42.342 INFO:tasks.workunit.client.1.smithi088.stdout:0/224: readlink d9/l18 0 2024-03-24T00:48:42.342 INFO:tasks.workunit.client.1.smithi088.stdout:0/225: chown d9/dc/c2f 118376 1 2024-03-24T00:48:42.343 INFO:tasks.workunit.client.1.smithi088.stdout:0/226: truncate d9/d20/f35 2048469 0 2024-03-24T00:48:42.343 INFO:tasks.workunit.client.1.smithi088.stdout:5/191: rmdir d1/d10/d1b/d2f 39 2024-03-24T00:48:42.346 INFO:tasks.workunit.client.0.smithi038.stdout:0/158: mknod d0/d12/c32 0 2024-03-24T00:48:42.347 INFO:tasks.workunit.client.1.smithi088.stdout:8/211: symlink d9/dc/d11/d41/l4b 0 2024-03-24T00:48:42.349 INFO:tasks.workunit.client.1.smithi088.stdout:3/178: dread d3/de/df/f25 [0,4194304] 0 2024-03-24T00:48:42.350 INFO:tasks.workunit.client.1.smithi088.stdout:2/173: dwrite db/f20 [0,4194304] 0 2024-03-24T00:48:42.350 INFO:tasks.workunit.client.1.smithi088.stdout:4/226: dwrite d5/d16/f29 [0,4194304] 0 2024-03-24T00:48:42.354 INFO:tasks.workunit.client.1.smithi088.stdout:9/199: mknod d1/c33 0 2024-03-24T00:48:42.354 INFO:tasks.workunit.client.0.smithi038.stdout:9/238: dwrite d12/d28/d3a/f50 [0,4194304] 0 2024-03-24T00:48:42.355 INFO:tasks.workunit.client.1.smithi088.stdout:0/227: creat d9/d20/d2a/d2e/f4e x:0 0 0 2024-03-24T00:48:42.357 INFO:tasks.workunit.client.0.smithi038.stdout:6/229: link d1/da/l29 d1/da/d1b/l45 0 2024-03-24T00:48:42.361 INFO:tasks.workunit.client.1.smithi088.stdout:5/192: mkdir d1/d43 0 2024-03-24T00:48:42.362 INFO:tasks.workunit.client.0.smithi038.stdout:7/178: getdents d2/d2c 0 2024-03-24T00:48:42.362 INFO:tasks.workunit.client.0.smithi038.stdout:7/179: read - d2/d9/df/f35 zero size 2024-03-24T00:48:42.362 INFO:tasks.workunit.client.0.smithi038.stdout:4/179: unlink da/dc/dd/c11 0 2024-03-24T00:48:42.363 INFO:tasks.workunit.client.0.smithi038.stdout:4/180: chown f9 122150618 1 2024-03-24T00:48:42.367 INFO:tasks.workunit.client.1.smithi088.stdout:1/206: read d6/f17 [7430126,124473] 0 2024-03-24T00:48:42.367 INFO:tasks.workunit.client.1.smithi088.stdout:6/233: dwrite db/dd/f3b [0,4194304] 0 2024-03-24T00:48:42.367 INFO:tasks.workunit.client.1.smithi088.stdout:6/234: readlink db/dd/l14 0 2024-03-24T00:48:42.367 INFO:tasks.workunit.client.1.smithi088.stdout:1/207: creat d6/f3e x:0 0 0 2024-03-24T00:48:42.367 INFO:tasks.workunit.client.1.smithi088.stdout:1/208: write d6/d27/f3c [860863,121640] 0 2024-03-24T00:48:42.368 INFO:tasks.workunit.client.0.smithi038.stdout:9/239: dread d12/d28/d3a/f2b [0,4194304] 0 2024-03-24T00:48:42.368 INFO:tasks.workunit.client.0.smithi038.stdout:9/240: readlink d12/l22 0 2024-03-24T00:48:42.379 INFO:tasks.workunit.client.0.smithi038.stdout:3/207: creat d3/d2a/f4b x:0 0 0 2024-03-24T00:48:42.379 INFO:tasks.workunit.client.0.smithi038.stdout:3/208: dread - d3/d30/f34 zero size 2024-03-24T00:48:42.379 INFO:tasks.workunit.client.0.smithi038.stdout:3/209: chown d3/d9/dc/df 612071 1 2024-03-24T00:48:42.390 INFO:tasks.workunit.client.1.smithi088.stdout:3/179: mknod d3/d1a/c3f 0 2024-03-24T00:48:42.390 INFO:tasks.workunit.client.1.smithi088.stdout:5/193: dread d1/ff [4194304,4194304] 0 2024-03-24T00:48:42.394 INFO:tasks.workunit.client.1.smithi088.stdout:2/174: unlink db/dd/c22 0 2024-03-24T00:48:42.399 INFO:tasks.workunit.client.0.smithi038.stdout:7/180: dread d2/d9/da/f14 [0,4194304] 0 2024-03-24T00:48:42.401 INFO:tasks.workunit.client.1.smithi088.stdout:4/227: rename d5/d13/d18/f25 to d5/f4d 0 2024-03-24T00:48:42.403 INFO:tasks.workunit.client.1.smithi088.stdout:7/192: dwrite d3/fc [0,4194304] 0 2024-03-24T00:48:42.404 INFO:tasks.workunit.client.0.smithi038.stdout:6/230: mknod d1/da/d1b/d25/c46 0 2024-03-24T00:48:42.405 INFO:tasks.workunit.client.1.smithi088.stdout:0/228: mkdir d9/dc/d4f 0 2024-03-24T00:48:42.407 INFO:tasks.workunit.client.0.smithi038.stdout:4/181: readlink da/dc/dd/l20 0 2024-03-24T00:48:42.407 INFO:tasks.workunit.client.0.smithi038.stdout:9/241: dread d12/d28/d3a/f50 [0,4194304] 0 2024-03-24T00:48:42.410 INFO:tasks.workunit.client.0.smithi038.stdout:4/182: write da/f1e [445284,14735] 0 2024-03-24T00:48:42.412 INFO:tasks.workunit.client.0.smithi038.stdout:0/159: dwrite d0/f2 [4194304,4194304] 0 2024-03-24T00:48:42.412 INFO:tasks.workunit.client.0.smithi038.stdout:3/210: symlink d3/d9/dc/df/d14/d19/l4c 0 2024-03-24T00:48:42.412 INFO:tasks.workunit.client.0.smithi038.stdout:3/211: write d3/d9/f1c [1034281,47118] 0 2024-03-24T00:48:42.415 INFO:tasks.workunit.client.0.smithi038.stdout:4/183: write da/dc/f22 [485942,115463] 0 2024-03-24T00:48:42.417 INFO:tasks.workunit.client.0.smithi038.stdout:6/231: symlink d1/da/d1b/d1d/d3c/l47 0 2024-03-24T00:48:42.424 INFO:tasks.workunit.client.1.smithi088.stdout:8/212: dwrite d9/dc/d11/d24/d29/f2c [0,4194304] 0 2024-03-24T00:48:42.428 INFO:tasks.workunit.client.1.smithi088.stdout:6/235: rename db/f35 to db/f47 0 2024-03-24T00:48:42.429 INFO:tasks.workunit.client.1.smithi088.stdout:6/236: dread - db/f38 zero size 2024-03-24T00:48:42.429 INFO:tasks.workunit.client.1.smithi088.stdout:6/237: write f8 [4244032,97990] 0 2024-03-24T00:48:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:0/160: link d0/d12/d15/d1a/c26 d0/d2d/c33 0 2024-03-24T00:48:42.430 INFO:tasks.workunit.client.1.smithi088.stdout:1/209: mkdir d6/d27/d1e/d3f 0 2024-03-24T00:48:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:3/212: truncate d3/d9/f26 107562 0 2024-03-24T00:48:42.432 INFO:tasks.workunit.client.1.smithi088.stdout:3/180: link d3/de/f29 d3/d6/d27/f40 0 2024-03-24T00:48:42.434 INFO:tasks.workunit.client.1.smithi088.stdout:4/228: creat d5/d16/d37/f4e x:0 0 0 2024-03-24T00:48:42.434 INFO:tasks.workunit.client.1.smithi088.stdout:4/229: chown d5/d16/d1a/d22/d2f 1195790575 1 2024-03-24T00:48:42.434 INFO:tasks.workunit.client.1.smithi088.stdout:4/230: getdents d5/d7/d15/d40 0 2024-03-24T00:48:42.434 INFO:tasks.workunit.client.1.smithi088.stdout:4/231: write d5/d16/d37/f48 [602859,126518] 0 2024-03-24T00:48:42.434 INFO:tasks.workunit.client.0.smithi038.stdout:9/242: getdents d12/d28 0 2024-03-24T00:48:42.436 INFO:tasks.workunit.client.0.smithi038.stdout:0/161: mknod d0/d12/d15/c34 0 2024-03-24T00:48:42.436 INFO:tasks.workunit.client.0.smithi038.stdout:0/162: read - d0/d1b/d2c/f31 zero size 2024-03-24T00:48:42.441 INFO:tasks.workunit.client.0.smithi038.stdout:6/232: getdents d1/da/d1b/d2f 0 2024-03-24T00:48:42.441 INFO:tasks.workunit.client.0.smithi038.stdout:6/233: truncate d1/da/d1b/d1d/f2d 1254427 0 2024-03-24T00:48:42.441 INFO:tasks.workunit.client.0.smithi038.stdout:6/234: dread - d1/da/d1b/d2f/f3d zero size 2024-03-24T00:48:42.443 INFO:tasks.workunit.client.1.smithi088.stdout:8/213: mknod d9/dc/d11/d24/d29/c4c 0 2024-03-24T00:48:42.443 INFO:tasks.workunit.client.1.smithi088.stdout:4/232: dread d5/d7/fe [0,4194304] 0 2024-03-24T00:48:42.443 INFO:tasks.workunit.client.1.smithi088.stdout:4/233: fsync d5/f9 0 2024-03-24T00:48:42.445 INFO:tasks.workunit.client.1.smithi088.stdout:6/238: creat db/d1d/f48 x:0 0 0 2024-03-24T00:48:42.445 INFO:tasks.workunit.client.1.smithi088.stdout:6/239: stat db/d1d/d23/d3c/d43 0 2024-03-24T00:48:42.445 INFO:tasks.workunit.client.1.smithi088.stdout:6/240: chown db/dd/f37 39385071 1 2024-03-24T00:48:42.446 INFO:tasks.workunit.client.0.smithi038.stdout:9/243: symlink d12/d28/l52 0 2024-03-24T00:48:42.446 INFO:tasks.workunit.client.0.smithi038.stdout:0/163: mknod d0/d12/d13/d28/c35 0 2024-03-24T00:48:42.446 INFO:tasks.workunit.client.1.smithi088.stdout:6/241: truncate db/f19 798750 0 2024-03-24T00:48:42.446 INFO:tasks.workunit.client.1.smithi088.stdout:6/242: dread - db/d1d/d23/d3c/d43/d24/f30 zero size 2024-03-24T00:48:42.447 INFO:tasks.workunit.client.0.smithi038.stdout:3/213: rmdir d3/d2a 39 2024-03-24T00:48:42.448 INFO:tasks.workunit.client.1.smithi088.stdout:1/210: truncate f0 5220488 0 2024-03-24T00:48:42.448 INFO:tasks.workunit.client.1.smithi088.stdout:1/211: truncate d6/d27/f1d 515527 0 2024-03-24T00:48:42.451 INFO:tasks.workunit.client.1.smithi088.stdout:3/181: symlink d3/d6/l41 0 2024-03-24T00:48:42.452 INFO:tasks.workunit.client.0.smithi038.stdout:9/244: symlink d12/d41/l53 0 2024-03-24T00:48:42.452 INFO:tasks.workunit.client.1.smithi088.stdout:7/193: unlink d3/c6 0 2024-03-24T00:48:42.452 INFO:tasks.workunit.client.1.smithi088.stdout:7/194: dread - d3/f2d zero size 2024-03-24T00:48:42.452 INFO:tasks.workunit.client.1.smithi088.stdout:7/195: chown d3/l13 20 1 2024-03-24T00:48:42.457 INFO:tasks.workunit.client.1.smithi088.stdout:4/234: rmdir d5/d16/d1a/d22/d2f 39 2024-03-24T00:48:42.457 INFO:tasks.workunit.client.1.smithi088.stdout:6/243: symlink db/d1d/l49 0 2024-03-24T00:48:42.466 INFO:tasks.workunit.client.1.smithi088.stdout:1/212: rename d6/f19 to d6/d27/d1e/f40 0 2024-03-24T00:48:42.466 INFO:tasks.workunit.client.1.smithi088.stdout:1/213: chown d6/f3e 1343 1 2024-03-24T00:48:42.468 INFO:tasks.workunit.client.1.smithi088.stdout:3/182: creat d3/d6/d27/d3c/f42 x:0 0 0 2024-03-24T00:48:42.468 INFO:tasks.workunit.client.1.smithi088.stdout:3/183: readlink d3/l34 0 2024-03-24T00:48:42.469 INFO:tasks.workunit.client.1.smithi088.stdout:9/200: dwrite d1/d10/d11/f17 [0,4194304] 0 2024-03-24T00:48:42.470 INFO:tasks.workunit.client.1.smithi088.stdout:7/196: mkdir d3/d32/d3f 0 2024-03-24T00:48:42.471 INFO:tasks.workunit.client.0.smithi038.stdout:9/245: mknod d12/d28/d3a/c54 0 2024-03-24T00:48:42.471 INFO:tasks.workunit.client.0.smithi038.stdout:9/246: write d12/d28/d3a/d26/f3d [4599447,25576] 0 2024-03-24T00:48:42.471 INFO:tasks.workunit.client.0.smithi038.stdout:9/247: chown d12/d28 11 1 2024-03-24T00:48:42.472 INFO:tasks.workunit.client.1.smithi088.stdout:4/235: truncate d5/d13/d18/f32 1398331 0 2024-03-24T00:48:42.472 INFO:tasks.workunit.client.0.smithi038.stdout:0/164: dread d0/d12/d15/f16 [0,4194304] 0 2024-03-24T00:48:42.476 INFO:tasks.workunit.client.1.smithi088.stdout:6/244: mknod db/d1d/d23/d3c/d43/d24/c4a 0 2024-03-24T00:48:42.476 INFO:tasks.workunit.client.1.smithi088.stdout:6/245: write db/dd/f10 [5116470,117568] 0 2024-03-24T00:48:42.477 INFO:tasks.workunit.client.0.smithi038.stdout:9/248: truncate d12/d1a/f20 3416028 0 2024-03-24T00:48:42.477 INFO:tasks.workunit.client.0.smithi038.stdout:6/235: dread d1/da/f33 [0,4194304] 0 2024-03-24T00:48:42.477 INFO:tasks.workunit.client.0.smithi038.stdout:6/236: chown d1/da/d1b/d1d/c27 347456302 1 2024-03-24T00:48:42.477 INFO:tasks.workunit.client.0.smithi038.stdout:6/237: dread - d1/da/f3b zero size 2024-03-24T00:48:42.477 INFO:tasks.workunit.client.0.smithi038.stdout:6/238: write d1/da/fb [277494,56359] 0 2024-03-24T00:48:42.478 INFO:tasks.workunit.client.1.smithi088.stdout:2/175: dwrite db/dc/f2c [0,4194304] 0 2024-03-24T00:48:42.478 INFO:tasks.workunit.client.0.smithi038.stdout:0/165: mknod d0/d14/c36 0 2024-03-24T00:48:42.485 INFO:tasks.workunit.client.0.smithi038.stdout:6/239: mknod d1/c48 0 2024-03-24T00:48:42.487 INFO:tasks.workunit.client.1.smithi088.stdout:1/214: unlink d6/c14 0 2024-03-24T00:48:42.488 INFO:tasks.workunit.client.1.smithi088.stdout:3/184: getdents d3/d6/d19/d2a 0 2024-03-24T00:48:42.489 INFO:tasks.workunit.client.0.smithi038.stdout:4/184: dwrite da/d12/f2c [0,4194304] 0 2024-03-24T00:48:42.489 INFO:tasks.workunit.client.0.smithi038.stdout:0/166: symlink d0/d2d/l37 0 2024-03-24T00:48:42.490 INFO:tasks.workunit.client.1.smithi088.stdout:5/194: dwrite d1/f3b [0,4194304] 0 2024-03-24T00:48:42.492 INFO:tasks.workunit.client.1.smithi088.stdout:9/201: unlink d1/d10/f18 0 2024-03-24T00:48:42.495 INFO:tasks.workunit.client.1.smithi088.stdout:7/197: creat d3/d32/d3f/f40 x:0 0 0 2024-03-24T00:48:42.495 INFO:tasks.workunit.client.0.smithi038.stdout:7/181: dwrite d2/d9/fb [0,4194304] 0 2024-03-24T00:48:42.496 INFO:tasks.workunit.client.1.smithi088.stdout:0/229: dwrite d9/f1c [0,4194304] 0 2024-03-24T00:48:42.497 INFO:tasks.workunit.client.1.smithi088.stdout:0/230: write d9/dc/de/f40 [4382094,10298] 0 2024-03-24T00:48:42.499 INFO:tasks.workunit.client.1.smithi088.stdout:2/176: read f1 [2620594,42779] 0 2024-03-24T00:48:42.509 INFO:tasks.workunit.client.1.smithi088.stdout:1/215: dread d6/f2a [0,4194304] 0 2024-03-24T00:48:42.509 INFO:tasks.workunit.client.1.smithi088.stdout:6/246: write db/f13 [1244490,93631] 0 2024-03-24T00:48:42.511 INFO:tasks.workunit.client.1.smithi088.stdout:6/247: chown db/c2c 769236791 1 2024-03-24T00:48:42.511 INFO:tasks.workunit.client.1.smithi088.stdout:6/248: write db/f19 [878464,55423] 0 2024-03-24T00:48:42.511 INFO:tasks.workunit.client.1.smithi088.stdout:6/249: write db/f3a [638021,6480] 0 2024-03-24T00:48:42.512 INFO:tasks.workunit.client.0.smithi038.stdout:4/185: getdents da/d12/d24 0 2024-03-24T00:48:42.518 INFO:tasks.workunit.client.1.smithi088.stdout:5/195: getdents d1/d29 0 2024-03-24T00:48:42.526 INFO:tasks.workunit.client.0.smithi038.stdout:0/167: dread d0/d12/d15/f2a [0,4194304] 0 2024-03-24T00:48:42.526 INFO:tasks.workunit.client.0.smithi038.stdout:0/168: dread - d0/d12/d15/d1a/f2b zero size 2024-03-24T00:48:42.533 INFO:tasks.workunit.client.1.smithi088.stdout:0/231: creat d9/dc/de/d19/d44/f50 x:0 0 0 2024-03-24T00:48:42.534 INFO:tasks.workunit.client.0.smithi038.stdout:4/186: dread da/d12/d24/f26 [0,4194304] 0 2024-03-24T00:48:42.536 INFO:tasks.workunit.client.1.smithi088.stdout:2/177: unlink db/f36 0 2024-03-24T00:48:42.536 INFO:tasks.workunit.client.1.smithi088.stdout:2/178: fdatasync db/dd/f28 0 2024-03-24T00:48:42.536 INFO:tasks.workunit.client.1.smithi088.stdout:2/179: read db/fe [237352,59457] 0 2024-03-24T00:48:42.537 INFO:tasks.workunit.client.0.smithi038.stdout:4/187: symlink da/l35 0 2024-03-24T00:48:42.539 INFO:tasks.workunit.client.0.smithi038.stdout:9/249: dwrite d12/d28/d3a/f30 [0,4194304] 0 2024-03-24T00:48:42.539 INFO:tasks.workunit.client.0.smithi038.stdout:0/169: dread d0/d12/d15/f16 [4194304,4194304] 0 2024-03-24T00:48:42.539 INFO:tasks.workunit.client.0.smithi038.stdout:0/170: readlink d0/d12/d13/l21 0 2024-03-24T00:48:42.539 INFO:tasks.workunit.client.1.smithi088.stdout:4/236: dwrite d5/d7/f44 [0,4194304] 0 2024-03-24T00:48:42.541 INFO:tasks.workunit.client.1.smithi088.stdout:9/202: dwrite d1/ff [0,4194304] 0 2024-03-24T00:48:42.545 INFO:tasks.workunit.client.0.smithi038.stdout:4/188: symlink da/d12/l36 0 2024-03-24T00:48:42.547 INFO:tasks.workunit.client.0.smithi038.stdout:9/250: mkdir d12/d55 0 2024-03-24T00:48:42.553 INFO:tasks.workunit.client.0.smithi038.stdout:4/189: truncate da/f1d 272046 0 2024-03-24T00:48:42.555 INFO:tasks.workunit.client.0.smithi038.stdout:9/251: mkdir d12/d55/d56 0 2024-03-24T00:48:42.555 INFO:tasks.workunit.client.0.smithi038.stdout:9/252: chown d12/d1a/c21 1070923735 1 2024-03-24T00:48:42.558 INFO:tasks.workunit.client.0.smithi038.stdout:0/171: dread d0/d12/d15/f16 [0,4194304] 0 2024-03-24T00:48:42.562 INFO:tasks.workunit.client.0.smithi038.stdout:3/214: dread d3/d9/f26 [0,4194304] 0 2024-03-24T00:48:42.569 INFO:tasks.workunit.client.0.smithi038.stdout:9/253: mkdir d12/d41/d57 0 2024-03-24T00:48:42.569 INFO:tasks.workunit.client.0.smithi038.stdout:9/254: creat d12/d41/f58 x:0 0 0 2024-03-24T00:48:42.576 INFO:tasks.workunit.client.0.smithi038.stdout:7/182: dread d2/d9/da/f1d [0,4194304] 0 2024-03-24T00:48:42.576 INFO:tasks.workunit.client.0.smithi038.stdout:7/183: getdents d2/d9/da/d23/d2b 0 2024-03-24T00:48:42.582 INFO:tasks.workunit.client.0.smithi038.stdout:0/172: dread d0/fa [4194304,4194304] 0 2024-03-24T00:48:42.582 INFO:tasks.workunit.client.0.smithi038.stdout:0/173: chown d0/d1b/d2c 31 1 2024-03-24T00:48:42.586 INFO:tasks.workunit.client.1.smithi088.stdout:1/216: dwrite d6/d27/d1e/f1f [0,4194304] 0 2024-03-24T00:48:42.587 INFO:tasks.workunit.client.0.smithi038.stdout:3/215: link d3/l36 d3/d30/l4d 0 2024-03-24T00:48:42.591 INFO:tasks.workunit.client.0.smithi038.stdout:6/240: dwrite d1/f10 [0,4194304] 0 2024-03-24T00:48:42.591 INFO:tasks.workunit.client.0.smithi038.stdout:6/241: dread - d1/da/f3b zero size 2024-03-24T00:48:42.592 INFO:tasks.workunit.client.0.smithi038.stdout:4/190: rename da/dc to da/d12/d37 0 2024-03-24T00:48:42.593 INFO:tasks.workunit.client.0.smithi038.stdout:4/191: write da/d12/d37/f22 [173198,71088] 0 2024-03-24T00:48:42.593 INFO:tasks.workunit.client.0.smithi038.stdout:3/216: rename d3/d9/dc/df/d14 to d3/d9/dc/df/d14/d19/d4e 22 2024-03-24T00:48:42.593 INFO:tasks.workunit.client.1.smithi088.stdout:6/250: rmdir db/d1d/d23/d3c 39 2024-03-24T00:48:42.594 INFO:tasks.workunit.client.1.smithi088.stdout:9/203: dwrite d1/f6 [0,4194304] 0 2024-03-24T00:48:42.595 INFO:tasks.workunit.client.0.smithi038.stdout:9/255: stat d12/d28/d3a/d26/c39 0 2024-03-24T00:48:42.597 INFO:tasks.workunit.client.0.smithi038.stdout:7/184: link d2/d9/d16/l1e d2/d2c/l42 0 2024-03-24T00:48:42.602 INFO:tasks.workunit.client.0.smithi038.stdout:0/174: symlink d0/d12/d13/l38 0 2024-03-24T00:48:42.602 INFO:tasks.workunit.client.1.smithi088.stdout:5/196: rename d1/d10/c39 to d1/d19/d1e/c44 0 2024-03-24T00:48:42.603 INFO:tasks.workunit.client.1.smithi088.stdout:7/198: symlink d3/l41 0 2024-03-24T00:48:42.605 INFO:tasks.workunit.client.1.smithi088.stdout:7/199: write d3/f35 [4279994,1907] 0 2024-03-24T00:48:42.607 INFO:tasks.workunit.client.0.smithi038.stdout:6/242: mkdir d1/da/d1b/d25/d49 0 2024-03-24T00:48:42.610 INFO:tasks.workunit.client.1.smithi088.stdout:2/180: mknod db/d21/d35/c3b 0 2024-03-24T00:48:42.611 INFO:tasks.workunit.client.0.smithi038.stdout:3/217: unlink d3/d9/dc/df/d14/f17 0 2024-03-24T00:48:42.614 INFO:tasks.workunit.client.1.smithi088.stdout:9/204: write d1/f8 [7750069,60784] 0 2024-03-24T00:48:42.618 INFO:tasks.workunit.client.1.smithi088.stdout:4/237: rmdir d5/d16/d1a/d22/d2f 39 2024-03-24T00:48:42.618 INFO:tasks.workunit.client.1.smithi088.stdout:4/238: dread - d5/f46 zero size 2024-03-24T00:48:42.618 INFO:tasks.workunit.client.1.smithi088.stdout:4/239: chown d5/d13/d18/d24/l39 4396409 1 2024-03-24T00:48:42.619 INFO:tasks.workunit.client.1.smithi088.stdout:5/197: dread d1/dd/f13 [4194304,4194304] 0 2024-03-24T00:48:42.619 INFO:tasks.workunit.client.1.smithi088.stdout:5/198: chown d1/d10/d1b/l27 13364102 1 2024-03-24T00:48:42.620 INFO:tasks.workunit.client.0.smithi038.stdout:7/185: mknod d2/d9/da/dc/c43 0 2024-03-24T00:48:42.620 INFO:tasks.workunit.client.0.smithi038.stdout:0/175: stat d0/cd 0 2024-03-24T00:48:42.620 INFO:tasks.workunit.client.0.smithi038.stdout:7/186: write d2/f1f [2025176,96073] 0 2024-03-24T00:48:42.630 INFO:tasks.workunit.client.1.smithi088.stdout:6/251: mkdir db/d1d/d23/d3c/d4b 0 2024-03-24T00:48:42.631 INFO:tasks.workunit.client.0.smithi038.stdout:9/256: link d12/d28/d3a/c54 d12/d28/c59 0 2024-03-24T00:48:42.631 INFO:tasks.workunit.client.0.smithi038.stdout:9/257: dread - d12/d28/d3a/f3f zero size 2024-03-24T00:48:42.634 INFO:tasks.workunit.client.1.smithi088.stdout:1/217: dwrite d6/f3e [0,4194304] 0 2024-03-24T00:48:42.635 INFO:tasks.workunit.client.0.smithi038.stdout:0/176: rename d0/d12/d15/f16 to d0/d12/d13/d28/f39 0 2024-03-24T00:48:42.635 INFO:tasks.workunit.client.0.smithi038.stdout:0/177: read - d0/d1b/d2c/f31 zero size 2024-03-24T00:48:42.635 INFO:tasks.workunit.client.1.smithi088.stdout:0/232: getdents d9/dc/de/d19 0 2024-03-24T00:48:42.635 INFO:tasks.workunit.client.1.smithi088.stdout:0/233: chown d9/dc/de/c47 133576715 1 2024-03-24T00:48:42.637 INFO:tasks.workunit.client.1.smithi088.stdout:2/181: truncate db/f20 4523146 0 2024-03-24T00:48:42.637 INFO:tasks.workunit.client.1.smithi088.stdout:2/182: creat db/f3c x:0 0 0 2024-03-24T00:48:42.641 INFO:tasks.workunit.client.0.smithi038.stdout:6/243: rmdir d1/da/d1b/d1d 39 2024-03-24T00:48:42.642 INFO:tasks.workunit.client.0.smithi038.stdout:5/168: sync 2024-03-24T00:48:42.642 INFO:tasks.workunit.client.0.smithi038.stdout:1/170: sync 2024-03-24T00:48:42.642 INFO:tasks.workunit.client.0.smithi038.stdout:5/169: truncate f5 4380704 0 2024-03-24T00:48:42.646 INFO:tasks.workunit.client.0.smithi038.stdout:3/218: symlink d3/l4f 0 2024-03-24T00:48:42.652 INFO:tasks.workunit.client.1.smithi088.stdout:9/205: creat d1/d10/f34 x:0 0 0 2024-03-24T00:48:42.652 INFO:tasks.workunit.client.1.smithi088.stdout:4/240: mknod d5/d16/d1a/c4f 0 2024-03-24T00:48:42.654 INFO:tasks.workunit.client.1.smithi088.stdout:5/199: getdents d1/d10/d1b/d2e 0 2024-03-24T00:48:42.654 INFO:tasks.workunit.client.1.smithi088.stdout:5/200: creat d1/d19/f45 x:0 0 0 2024-03-24T00:48:42.655 INFO:tasks.workunit.client.0.smithi038.stdout:9/258: creat d12/d28/d3a/d26/f5a x:0 0 0 2024-03-24T00:48:42.655 INFO:tasks.workunit.client.0.smithi038.stdout:9/259: creat d12/d41/f5b x:0 0 0 2024-03-24T00:48:42.655 INFO:tasks.workunit.client.0.smithi038.stdout:9/260: write d12/d28/d3a/f45 [224298,30915] 0 2024-03-24T00:48:42.655 INFO:tasks.workunit.client.0.smithi038.stdout:9/261: fdatasync d12/f35 0 2024-03-24T00:48:42.656 INFO:tasks.workunit.client.0.smithi038.stdout:7/187: getdents d2/d9/da/d23 0 2024-03-24T00:48:42.658 INFO:tasks.workunit.client.1.smithi088.stdout:6/252: dread db/d1d/d23/d3c/d43/f34 [0,4194304] 0 2024-03-24T00:48:42.658 INFO:tasks.workunit.client.1.smithi088.stdout:6/253: creat db/d1d/d23/f4c x:0 0 0 2024-03-24T00:48:42.658 INFO:tasks.workunit.client.0.smithi038.stdout:0/178: dread d0/d12/d15/f2a [0,4194304] 0 2024-03-24T00:48:42.666 INFO:tasks.workunit.client.0.smithi038.stdout:2/230: sync 2024-03-24T00:48:42.666 INFO:tasks.workunit.client.0.smithi038.stdout:8/181: sync 2024-03-24T00:48:42.667 INFO:tasks.workunit.client.0.smithi038.stdout:8/182: chown d0/d15/d38 100834 1 2024-03-24T00:48:42.667 INFO:tasks.workunit.client.0.smithi038.stdout:6/244: rmdir d1/da/d1b/d1d 39 2024-03-24T00:48:42.671 INFO:tasks.workunit.client.0.smithi038.stdout:1/171: creat da/dd/d10/d34/f39 x:0 0 0 2024-03-24T00:48:42.672 INFO:tasks.workunit.client.0.smithi038.stdout:1/172: write da/dd/d10/d14/f38 [1302748,89365] 0 2024-03-24T00:48:42.673 INFO:tasks.workunit.client.1.smithi088.stdout:3/185: sync 2024-03-24T00:48:42.674 INFO:tasks.workunit.client.1.smithi088.stdout:8/214: sync 2024-03-24T00:48:42.674 INFO:tasks.workunit.client.1.smithi088.stdout:3/186: fdatasync d3/d6/f3b 0 2024-03-24T00:48:42.674 INFO:tasks.workunit.client.1.smithi088.stdout:8/215: creat d9/dc/d11/d24/d29/d36/f4d x:0 0 0 2024-03-24T00:48:42.674 INFO:tasks.workunit.client.1.smithi088.stdout:8/216: creat d9/dc/d11/d24/d29/d36/f4e x:0 0 0 2024-03-24T00:48:42.676 INFO:tasks.workunit.client.1.smithi088.stdout:1/218: write d6/d27/d1e/f22 [742550,76120] 0 2024-03-24T00:48:42.676 INFO:tasks.workunit.client.1.smithi088.stdout:1/219: stat d6/c1c 0 2024-03-24T00:48:42.679 INFO:tasks.workunit.client.0.smithi038.stdout:9/262: getdents d12/d55 0 2024-03-24T00:48:42.679 INFO:tasks.workunit.client.0.smithi038.stdout:9/263: chown d12/d41/f4f 28545 1 2024-03-24T00:48:42.681 INFO:tasks.workunit.client.1.smithi088.stdout:2/183: creat db/d21/d35/f3d x:0 0 0 2024-03-24T00:48:42.686 INFO:tasks.workunit.client.0.smithi038.stdout:7/188: symlink d2/d9/df/l44 0 2024-03-24T00:48:42.691 INFO:tasks.workunit.client.1.smithi088.stdout:4/241: creat d5/d16/d1a/f50 x:0 0 0 2024-03-24T00:48:42.691 INFO:tasks.workunit.client.1.smithi088.stdout:4/242: dread - d5/f4d zero size 2024-03-24T00:48:42.693 INFO:tasks.workunit.client.0.smithi038.stdout:8/183: creat d0/d15/d38/d37/f44 x:0 0 0 2024-03-24T00:48:42.699 INFO:tasks.workunit.client.1.smithi088.stdout:6/254: dread db/dd/f10 [0,4194304] 0 2024-03-24T00:48:42.700 INFO:tasks.workunit.client.0.smithi038.stdout:0/179: write d0/f5 [3601342,25859] 0 2024-03-24T00:48:42.701 INFO:tasks.workunit.client.1.smithi088.stdout:0/234: dwrite d9/d20/f35 [0,4194304] 0 2024-03-24T00:48:42.701 INFO:tasks.workunit.client.1.smithi088.stdout:0/235: fdatasync d9/d26/f38 0 2024-03-24T00:48:42.705 INFO:tasks.workunit.client.0.smithi038.stdout:6/245: dread d1/da/d1b/d1d/f39 [0,4194304] 0 2024-03-24T00:48:42.705 INFO:tasks.workunit.client.0.smithi038.stdout:6/246: fdatasync d1/da/d1b/d1d/f39 0 2024-03-24T00:48:42.705 INFO:tasks.workunit.client.0.smithi038.stdout:6/247: getdents d1/da/d1b/d2f 0 2024-03-24T00:48:42.705 INFO:tasks.workunit.client.0.smithi038.stdout:6/248: fsync d1/f21 0 2024-03-24T00:48:42.706 INFO:tasks.workunit.client.0.smithi038.stdout:6/249: write d1/f1c [745997,104408] 0 2024-03-24T00:48:42.706 INFO:tasks.workunit.client.0.smithi038.stdout:1/173: mknod da/dd/d22/c3a 0 2024-03-24T00:48:42.706 INFO:tasks.workunit.client.0.smithi038.stdout:1/174: chown da/df/c37 48660 1 2024-03-24T00:48:42.708 INFO:tasks.workunit.client.0.smithi038.stdout:9/264: fdatasync d12/d28/f4d 0 2024-03-24T00:48:42.709 INFO:tasks.workunit.client.1.smithi088.stdout:3/187: truncate f2 3969450 0 2024-03-24T00:48:42.709 INFO:tasks.workunit.client.1.smithi088.stdout:3/188: chown d3/d1a/c3f 22 1 2024-03-24T00:48:42.713 INFO:tasks.workunit.client.0.smithi038.stdout:7/189: mknod d2/d9/df/c45 0 2024-03-24T00:48:42.717 INFO:tasks.workunit.client.1.smithi088.stdout:1/220: stat d6/d27/l20 0 2024-03-24T00:48:42.721 INFO:tasks.workunit.client.0.smithi038.stdout:4/192: dwrite da/d12/d37/f22 [0,4194304] 0 2024-03-24T00:48:42.722 INFO:tasks.workunit.client.1.smithi088.stdout:4/243: mknod d5/d13/d18/d24/c51 0 2024-03-24T00:48:42.726 INFO:tasks.workunit.client.0.smithi038.stdout:6/250: mknod d1/da/d1b/d1d/c4a 0 2024-03-24T00:48:42.726 INFO:tasks.workunit.client.0.smithi038.stdout:6/251: write d1/da/d1b/f26 [3132068,122568] 0 2024-03-24T00:48:42.727 INFO:tasks.workunit.client.0.smithi038.stdout:7/190: link d2/d9/d16/c24 d2/d9/da/d23/d2b/c46 0 2024-03-24T00:48:42.727 INFO:tasks.workunit.client.0.smithi038.stdout:7/191: stat d2/d9/da/dc 0 2024-03-24T00:48:42.727 INFO:tasks.workunit.client.0.smithi038.stdout:7/192: dread - d2/d9/da/dc/f1c zero size 2024-03-24T00:48:42.727 INFO:tasks.workunit.client.0.smithi038.stdout:7/193: write d2/f13 [598704,106426] 0 2024-03-24T00:48:42.728 INFO:tasks.workunit.client.0.smithi038.stdout:7/194: creat d2/d9/da/f47 x:0 0 0 2024-03-24T00:48:42.729 INFO:tasks.workunit.client.1.smithi088.stdout:0/236: truncate d9/dc/de/f14 782639 0 2024-03-24T00:48:42.730 INFO:tasks.workunit.client.0.smithi038.stdout:3/219: dwrite d3/d9/f44 [0,4194304] 0 2024-03-24T00:48:42.732 INFO:tasks.workunit.client.1.smithi088.stdout:5/201: dwrite d1/d10/f17 [0,4194304] 0 2024-03-24T00:48:42.733 INFO:tasks.workunit.client.0.smithi038.stdout:1/175: symlink da/df/l3b 0 2024-03-24T00:48:42.734 INFO:tasks.workunit.client.1.smithi088.stdout:3/189: mknod d3/d1a/d1e/c43 0 2024-03-24T00:48:42.736 INFO:tasks.workunit.client.1.smithi088.stdout:2/184: dread db/dd/f2f [0,4194304] 0 2024-03-24T00:48:42.736 INFO:tasks.workunit.client.1.smithi088.stdout:4/244: rmdir d5/d16/d1a/d22/d3c 0 2024-03-24T00:48:42.738 INFO:tasks.workunit.client.1.smithi088.stdout:0/237: rmdir d9/dc/de/d19 39 2024-03-24T00:48:42.738 INFO:tasks.workunit.client.1.smithi088.stdout:0/238: write d9/dc/f49 [984132,38381] 0 2024-03-24T00:48:42.740 INFO:tasks.workunit.client.0.smithi038.stdout:4/193: truncate f9 2299214 0 2024-03-24T00:48:42.740 INFO:tasks.workunit.client.1.smithi088.stdout:7/200: dwrite d3/f29 [0,4194304] 0 2024-03-24T00:48:42.741 INFO:tasks.workunit.client.1.smithi088.stdout:3/190: symlink d3/d32/l44 0 2024-03-24T00:48:42.742 INFO:tasks.workunit.client.1.smithi088.stdout:3/191: chown d3/d1a/d1e 237100 1 2024-03-24T00:48:42.744 INFO:tasks.workunit.client.1.smithi088.stdout:2/185: write db/d21/f33 [376205,111130] 0 2024-03-24T00:48:42.746 INFO:tasks.workunit.client.1.smithi088.stdout:4/245: dread d5/f9 [0,4194304] 0 2024-03-24T00:48:42.746 INFO:tasks.workunit.client.1.smithi088.stdout:7/201: creat d3/d32/f42 x:0 0 0 2024-03-24T00:48:42.747 INFO:tasks.workunit.client.1.smithi088.stdout:1/221: dread d6/d27/d1e/f22 [0,4194304] 0 2024-03-24T00:48:42.750 INFO:tasks.workunit.client.1.smithi088.stdout:8/217: dwrite d9/dc/d11/f2f [0,4194304] 0 2024-03-24T00:48:42.750 INFO:tasks.workunit.client.1.smithi088.stdout:8/218: dread - d9/dc/d11/d24/d29/d36/f4d zero size 2024-03-24T00:48:42.751 INFO:tasks.workunit.client.0.smithi038.stdout:3/220: mknod d3/d9/dc/c50 0 2024-03-24T00:48:42.754 INFO:tasks.workunit.client.0.smithi038.stdout:7/195: getdents d2/d9/df/d33/d37 0 2024-03-24T00:48:42.761 INFO:tasks.workunit.client.0.smithi038.stdout:7/196: rmdir d2/d2c 39 2024-03-24T00:48:42.761 INFO:tasks.workunit.client.0.smithi038.stdout:7/197: getdents d2/d9/df/d33/d37/d3a 0 2024-03-24T00:48:42.761 INFO:tasks.workunit.client.0.smithi038.stdout:7/198: dread - d2/d9/da/f41 zero size 2024-03-24T00:48:42.761 INFO:tasks.workunit.client.0.smithi038.stdout:7/199: readlink l1 0 2024-03-24T00:48:42.764 INFO:tasks.workunit.client.1.smithi088.stdout:3/192: mkdir d3/d1a/d1e/d2f/d45 0 2024-03-24T00:48:42.769 INFO:tasks.workunit.client.0.smithi038.stdout:2/231: dwrite d4/f13 [0,4194304] 0 2024-03-24T00:48:42.769 INFO:tasks.workunit.client.0.smithi038.stdout:2/232: dread - d4/f41 zero size 2024-03-24T00:48:42.769 INFO:tasks.workunit.client.0.smithi038.stdout:2/233: getdents d4 0 2024-03-24T00:48:42.769 INFO:tasks.workunit.client.0.smithi038.stdout:2/234: fdatasync d4/f7 0 2024-03-24T00:48:42.769 INFO:tasks.workunit.client.0.smithi038.stdout:5/170: dwrite d6/d1e/f1f [0,4194304] 0 2024-03-24T00:48:42.769 INFO:tasks.workunit.client.1.smithi088.stdout:5/202: write d1/fb [673297,36904] 0 2024-03-24T00:48:42.770 INFO:tasks.workunit.client.1.smithi088.stdout:9/206: dwrite d1/f8 [0,4194304] 0 2024-03-24T00:48:42.774 INFO:tasks.workunit.client.0.smithi038.stdout:6/252: write d1/f10 [3555649,120011] 0 2024-03-24T00:48:42.774 INFO:tasks.workunit.client.0.smithi038.stdout:6/253: chown d1/da/d1b/d1d/d3c 32074 1 2024-03-24T00:48:42.774 INFO:tasks.workunit.client.0.smithi038.stdout:6/254: fdatasync d1/f1f 0 2024-03-24T00:48:42.776 INFO:tasks.workunit.client.1.smithi088.stdout:2/186: rename db/c10 to db/dc/c3e 0 2024-03-24T00:48:42.776 INFO:tasks.workunit.client.1.smithi088.stdout:2/187: readlink db/dd/l1e 0 2024-03-24T00:48:42.776 INFO:tasks.workunit.client.1.smithi088.stdout:7/202: chown d3/f21 690179245 1 2024-03-24T00:48:42.776 INFO:tasks.workunit.client.1.smithi088.stdout:4/246: creat d5/d16/d1a/d22/f52 x:0 0 0 2024-03-24T00:48:42.778 INFO:tasks.workunit.client.0.smithi038.stdout:3/221: write d3/f8 [611905,35573] 0 2024-03-24T00:48:42.785 INFO:tasks.workunit.client.0.smithi038.stdout:8/184: rename d0/d15/d38 to d0/d45 0 2024-03-24T00:48:42.785 INFO:tasks.workunit.client.0.smithi038.stdout:8/185: chown d0/f16 6395503 1 2024-03-24T00:48:42.793 INFO:tasks.workunit.client.0.smithi038.stdout:7/200: dread d2/d9/da/f1d [0,4194304] 0 2024-03-24T00:48:42.793 INFO:tasks.workunit.client.0.smithi038.stdout:7/201: getdents d2/d9/d16 0 2024-03-24T00:48:42.794 INFO:tasks.workunit.client.1.smithi088.stdout:8/219: mknod d9/c4f 0 2024-03-24T00:48:42.796 INFO:tasks.workunit.client.1.smithi088.stdout:0/239: dwrite d9/f3a [0,4194304] 0 2024-03-24T00:48:42.800 INFO:tasks.workunit.client.0.smithi038.stdout:2/235: unlink d4/d27/d42/l44 0 2024-03-24T00:48:42.801 INFO:tasks.workunit.client.1.smithi088.stdout:3/193: symlink d3/de/df/l46 0 2024-03-24T00:48:42.801 INFO:tasks.workunit.client.1.smithi088.stdout:3/194: creat d3/d6/d27/d3c/f47 x:0 0 0 2024-03-24T00:48:42.801 INFO:tasks.workunit.client.1.smithi088.stdout:3/195: write d3/de/df/f2d [831035,95685] 0 2024-03-24T00:48:42.803 INFO:tasks.workunit.client.0.smithi038.stdout:5/171: unlink d6/d16/d14/f24 0 2024-03-24T00:48:42.805 INFO:tasks.workunit.client.1.smithi088.stdout:9/207: creat d1/d10/d2c/f35 x:0 0 0 2024-03-24T00:48:42.807 INFO:tasks.workunit.client.0.smithi038.stdout:1/176: rename da/df to da/d3c 0 2024-03-24T00:48:42.809 INFO:tasks.workunit.client.0.smithi038.stdout:3/222: rmdir d3/d9/dc/df 39 2024-03-24T00:48:42.809 INFO:tasks.workunit.client.0.smithi038.stdout:3/223: fsync d3/f6 0 2024-03-24T00:48:42.809 INFO:tasks.workunit.client.0.smithi038.stdout:3/224: fdatasync d3/f29 0 2024-03-24T00:48:42.809 INFO:tasks.workunit.client.0.smithi038.stdout:3/225: chown d3/d9/dc/df/f1d 119531926 1 2024-03-24T00:48:42.810 INFO:tasks.workunit.client.1.smithi088.stdout:2/188: chown db/d21/l2d 116 1 2024-03-24T00:48:42.812 INFO:tasks.workunit.client.1.smithi088.stdout:1/222: dwrite d6/d2f/f3d [0,4194304] 0 2024-03-24T00:48:42.813 INFO:tasks.workunit.client.0.smithi038.stdout:0/180: rename d0/d12/d15/f2a to d0/d12/f3a 0 2024-03-24T00:48:42.813 INFO:tasks.workunit.client.0.smithi038.stdout:0/181: chown d0/d14/c36 449 1 2024-03-24T00:48:42.816 INFO:tasks.workunit.client.1.smithi088.stdout:7/203: dread - d3/f16 zero size 2024-03-24T00:48:42.816 INFO:tasks.workunit.client.1.smithi088.stdout:7/204: chown d3/d31/l3e 14483 1 2024-03-24T00:48:42.819 INFO:tasks.workunit.client.1.smithi088.stdout:8/220: write d9/f37 [1709421,124257] 0 2024-03-24T00:48:42.820 INFO:tasks.workunit.client.1.smithi088.stdout:0/240: unlink f3 0 2024-03-24T00:48:42.820 INFO:tasks.workunit.client.1.smithi088.stdout:0/241: dread - d9/dc/de/f3e zero size 2024-03-24T00:48:42.824 INFO:tasks.workunit.client.0.smithi038.stdout:7/202: creat d2/d9/da/dc/f48 x:0 0 0 2024-03-24T00:48:42.830 INFO:tasks.workunit.client.0.smithi038.stdout:2/236: link d4/d27/d2f/d30/d3e/c4a d4/d18/d38/c4f 0 2024-03-24T00:48:42.830 INFO:tasks.workunit.client.0.smithi038.stdout:2/237: stat d4/d18/d26/f2c 0 2024-03-24T00:48:42.830 INFO:tasks.workunit.client.0.smithi038.stdout:5/172: symlink d6/d19/d20/l30 0 2024-03-24T00:48:42.834 INFO:tasks.workunit.client.1.smithi088.stdout:9/208: unlink d1/d10/d29/f2b 0 2024-03-24T00:48:42.835 INFO:tasks.workunit.client.1.smithi088.stdout:1/223: write d6/f3e [1282801,127283] 0 2024-03-24T00:48:42.835 INFO:tasks.workunit.client.1.smithi088.stdout:1/224: chown d6/c1c 424160 1 2024-03-24T00:48:42.835 INFO:tasks.workunit.client.1.smithi088.stdout:1/225: write f0 [1058429,20513] 0 2024-03-24T00:48:42.836 INFO:tasks.workunit.client.0.smithi038.stdout:3/226: mkdir d3/d9/dc/d51 0 2024-03-24T00:48:42.836 INFO:tasks.workunit.client.0.smithi038.stdout:3/227: dread - d3/d9/dc/df/f3a zero size 2024-03-24T00:48:42.841 INFO:tasks.workunit.client.0.smithi038.stdout:9/265: rename d12/d41/l53 to d12/d1a/l5c 0 2024-03-24T00:48:42.843 INFO:tasks.workunit.client.0.smithi038.stdout:5/173: mknod d6/d16/d14/c31 0 2024-03-24T00:48:42.843 INFO:tasks.workunit.client.0.smithi038.stdout:5/174: write d6/fa [9149104,58646] 0 2024-03-24T00:48:42.845 INFO:tasks.workunit.client.1.smithi088.stdout:7/205: link d3/f2d d3/d32/f43 0 2024-03-24T00:48:42.845 INFO:tasks.workunit.client.1.smithi088.stdout:7/206: creat d3/d32/f44 x:0 0 0 2024-03-24T00:48:42.845 INFO:tasks.workunit.client.1.smithi088.stdout:7/207: chown d3/f2d 219 1 2024-03-24T00:48:42.845 INFO:tasks.workunit.client.1.smithi088.stdout:7/208: read - d3/d31/f3c zero size 2024-03-24T00:48:42.847 INFO:tasks.workunit.client.0.smithi038.stdout:3/228: creat d3/d9/dc/d51/f52 x:0 0 0 2024-03-24T00:48:42.847 INFO:tasks.workunit.client.0.smithi038.stdout:3/229: chown d3/d9/dc/df/f1d 45 1 2024-03-24T00:48:42.851 INFO:tasks.workunit.client.1.smithi088.stdout:8/221: creat d9/dc/d11/d24/d43/f50 x:0 0 0 2024-03-24T00:48:42.851 INFO:tasks.workunit.client.1.smithi088.stdout:8/222: dread - d9/dc/d11/d24/d43/f50 zero size 2024-03-24T00:48:42.851 INFO:tasks.workunit.client.1.smithi088.stdout:8/223: dread - d9/dc/d11/d24/d29/d39/f48 zero size 2024-03-24T00:48:42.853 INFO:tasks.workunit.client.0.smithi038.stdout:1/177: rename f1 to da/dd/d10/d34/f3d 0 2024-03-24T00:48:42.856 INFO:tasks.workunit.client.1.smithi088.stdout:0/242: creat d9/dc/de/d19/f51 x:0 0 0 2024-03-24T00:48:42.856 INFO:tasks.workunit.client.1.smithi088.stdout:0/243: truncate d9/dc/de/f3e 584707 0 2024-03-24T00:48:42.856 INFO:tasks.workunit.client.1.smithi088.stdout:0/244: creat d9/dc/f52 x:0 0 0 2024-03-24T00:48:42.858 INFO:tasks.workunit.client.1.smithi088.stdout:5/203: dwrite d1/d10/d1b/d2f/f38 [0,4194304] 0 2024-03-24T00:48:42.862 INFO:tasks.workunit.client.1.smithi088.stdout:9/209: link d1/f5 d1/d10/d11/d28/f36 0 2024-03-24T00:48:42.862 INFO:tasks.workunit.client.1.smithi088.stdout:9/210: chown d1/d10/d11/c13 0 1 2024-03-24T00:48:42.863 INFO:tasks.workunit.client.0.smithi038.stdout:4/194: dread f9 [0,4194304] 0 2024-03-24T00:48:42.864 INFO:tasks.workunit.client.0.smithi038.stdout:4/195: truncate da/f1d 1192482 0 2024-03-24T00:48:42.866 INFO:tasks.workunit.client.0.smithi038.stdout:3/230: creat d3/d9/dc/df/d14/f53 x:0 0 0 2024-03-24T00:48:42.871 INFO:tasks.workunit.client.0.smithi038.stdout:9/266: link c11 d12/d41/d57/c5d 0 2024-03-24T00:48:42.874 INFO:tasks.workunit.client.0.smithi038.stdout:6/255: dwrite d1/da/d1b/d1d/f39 [0,4194304] 0 2024-03-24T00:48:42.874 INFO:tasks.workunit.client.0.smithi038.stdout:6/256: chown d1/da/f33 85675 1 2024-03-24T00:48:42.876 INFO:tasks.workunit.client.0.smithi038.stdout:8/186: dwrite d0/d15/f17 [0,4194304] 0 2024-03-24T00:48:42.876 INFO:tasks.workunit.client.1.smithi088.stdout:7/209: dread d3/f1f [0,4194304] 0 2024-03-24T00:48:42.880 INFO:tasks.workunit.client.0.smithi038.stdout:4/196: mkdir da/d38 0 2024-03-24T00:48:42.880 INFO:tasks.workunit.client.0.smithi038.stdout:2/238: dread d4/fc [0,4194304] 0 2024-03-24T00:48:42.880 INFO:tasks.workunit.client.0.smithi038.stdout:2/239: fdatasync d4/f16 0 2024-03-24T00:48:42.882 INFO:tasks.workunit.client.1.smithi088.stdout:4/247: dwrite d5/d7/dd/ff [0,4194304] 0 2024-03-24T00:48:42.885 INFO:tasks.workunit.client.1.smithi088.stdout:0/245: dread d9/dc/de/f1d [0,4194304] 0 2024-03-24T00:48:42.885 INFO:tasks.workunit.client.1.smithi088.stdout:8/224: unlink l5 0 2024-03-24T00:48:42.891 INFO:tasks.workunit.client.1.smithi088.stdout:3/196: dwrite d3/de/df/f2d [0,4194304] 0 2024-03-24T00:48:42.891 INFO:tasks.workunit.client.1.smithi088.stdout:3/197: dread - d3/d6/d27/d3c/f39 zero size 2024-03-24T00:48:42.892 INFO:tasks.workunit.client.0.smithi038.stdout:1/178: mknod da/d3c/d2d/c3e 0 2024-03-24T00:48:42.906 INFO:tasks.workunit.client.0.smithi038.stdout:9/267: link c11 d12/d55/d56/c5e 0 2024-03-24T00:48:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:1/179: dread da/dd/d22/f25 [0,4194304] 0 2024-03-24T00:48:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:1/180: write da/dd/d10/d14/f2e [219854,105320] 0 2024-03-24T00:48:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:1/181: stat da/dd/l1b 0 2024-03-24T00:48:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:1/182: truncate da/f35 589541 0 2024-03-24T00:48:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:1/183: write da/dd/d10/d34/f39 [375271,109700] 0 2024-03-24T00:48:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:1/184: stat da/f35 0 2024-03-24T00:48:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:5/175: dwrite d6/d19/f1c [0,4194304] 0 2024-03-24T00:48:42.911 INFO:tasks.workunit.client.1.smithi088.stdout:2/189: dwrite db/f20 [4194304,4194304] 0 2024-03-24T00:48:42.911 INFO:tasks.workunit.client.1.smithi088.stdout:5/204: mkdir d1/d10/d1b/d2e/d46 0 2024-03-24T00:48:42.912 INFO:tasks.workunit.client.1.smithi088.stdout:5/205: fdatasync d1/d19/f30 0 2024-03-24T00:48:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:8/187: mknod d0/d15/c46 0 2024-03-24T00:48:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:0/182: dwrite d0/d1b/d2c/f31 [0,4194304] 0 2024-03-24T00:48:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:0/183: read - d0/d12/d15/f22 zero size 2024-03-24T00:48:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:0/184: creat d0/d1b/d2c/f3b x:0 0 0 2024-03-24T00:48:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:0/185: fdatasync d0/d12/d15/d1a/f2b 0 2024-03-24T00:48:42.918 INFO:tasks.workunit.client.1.smithi088.stdout:9/211: rename d1/d10/f1d to d1/d15/f37 0 2024-03-24T00:48:42.918 INFO:tasks.workunit.client.1.smithi088.stdout:1/226: rmdir d6/d23 39 2024-03-24T00:48:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:2/240: creat d4/f50 x:0 0 0 2024-03-24T00:48:42.920 INFO:tasks.workunit.client.1.smithi088.stdout:4/248: truncate d5/d16/f29 2025378 0 2024-03-24T00:48:42.921 INFO:tasks.workunit.client.1.smithi088.stdout:0/246: truncate d9/dc/de/d19/f1a 4173161 0 2024-03-24T00:48:42.922 INFO:tasks.workunit.client.0.smithi038.stdout:1/185: truncate da/dd/d10/d14/f38 694234 0 2024-03-24T00:48:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:9/268: rmdir d12/d55/d56 39 2024-03-24T00:48:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:9/269: chown d12/l32 371 1 2024-03-24T00:48:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:7/203: dwrite d2/d9/da/dc/f48 [0,4194304] 0 2024-03-24T00:48:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:7/204: fsync d2/f1a 0 2024-03-24T00:48:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:7/205: fsync d2/d9/df/d33/f3c 0 2024-03-24T00:48:42.928 INFO:tasks.workunit.client.1.smithi088.stdout:3/198: mkdir d3/d1a/d1e/d48 0 2024-03-24T00:48:42.929 INFO:tasks.workunit.client.1.smithi088.stdout:3/199: stat d3/de/df 0 2024-03-24T00:48:42.929 INFO:tasks.workunit.client.1.smithi088.stdout:3/200: stat d3/de/f17 0 2024-03-24T00:48:42.929 INFO:tasks.workunit.client.1.smithi088.stdout:3/201: write d3/d6/d27/d3c/f42 [585410,120622] 0 2024-03-24T00:48:42.929 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:42 smithi088 ceph-mon[36610]: pgmap v27: 65 pgs: 65 active+clean; 774 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 72 MiB/s rd, 84 MiB/s wr, 529 op/s 2024-03-24T00:48:42.934 INFO:tasks.workunit.client.0.smithi038.stdout:5/176: creat d6/d23/f32 x:0 0 0 2024-03-24T00:48:42.938 INFO:tasks.workunit.client.0.smithi038.stdout:0/186: creat d0/d12/f3c x:0 0 0 2024-03-24T00:48:42.938 INFO:tasks.workunit.client.0.smithi038.stdout:3/231: dwrite d3/d2a/f4b [0,4194304] 0 2024-03-24T00:48:42.938 INFO:tasks.workunit.client.0.smithi038.stdout:2/241: symlink d4/d27/d2f/l51 0 2024-03-24T00:48:42.939 INFO:tasks.workunit.client.0.smithi038.stdout:4/197: dwrite da/f1d [0,4194304] 0 2024-03-24T00:48:42.940 INFO:tasks.workunit.client.0.smithi038.stdout:9/270: mkdir d12/d1a/d5f 0 2024-03-24T00:48:42.941 INFO:tasks.workunit.client.0.smithi038.stdout:9/271: fsync d12/d28/f4d 0 2024-03-24T00:48:42.945 INFO:tasks.workunit.client.0.smithi038.stdout:6/257: dwrite d1/da/d1b/f26 [0,4194304] 0 2024-03-24T00:48:42.946 INFO:tasks.workunit.client.0.smithi038.stdout:7/206: dread d2/d9/d16/f19 [0,4194304] 0 2024-03-24T00:48:42.948 INFO:tasks.workunit.client.0.smithi038.stdout:5/177: mkdir d6/d19/d20/d33 0 2024-03-24T00:48:42.957 INFO:tasks.workunit.client.0.smithi038.stdout:0/187: symlink d0/d12/l3d 0 2024-03-24T00:48:42.957 INFO:tasks.workunit.client.0.smithi038.stdout:0/188: chown d0/d12/l3d 241895 1 2024-03-24T00:48:42.957 INFO:tasks.workunit.client.0.smithi038.stdout:0/189: stat d0/f5 0 2024-03-24T00:48:42.958 INFO:tasks.workunit.client.1.smithi088.stdout:5/206: unlink d1/l6 0 2024-03-24T00:48:42.958 INFO:tasks.workunit.client.1.smithi088.stdout:5/207: chown d1/d10/d1b/c25 7447 1 2024-03-24T00:48:42.960 INFO:tasks.workunit.client.0.smithi038.stdout:1/186: link da/d3c/l1a da/l3f 0 2024-03-24T00:48:42.963 INFO:tasks.workunit.client.1.smithi088.stdout:8/225: rename d9/dc/d11/f31 to d9/dc/d11/d24/d29/f51 0 2024-03-24T00:48:42.965 INFO:tasks.workunit.client.1.smithi088.stdout:9/212: rmdir d1 39 2024-03-24T00:48:42.966 INFO:tasks.workunit.client.0.smithi038.stdout:4/198: unlink f8 0 2024-03-24T00:48:42.968 INFO:tasks.workunit.client.0.smithi038.stdout:9/272: mknod d12/d28/d3a/c60 0 2024-03-24T00:48:42.968 INFO:tasks.workunit.client.0.smithi038.stdout:9/273: write d12/d1a/f3c [572673,4414] 0 2024-03-24T00:48:42.969 INFO:tasks.workunit.client.1.smithi088.stdout:1/227: creat d6/d27/d1e/d3f/f41 x:0 0 0 2024-03-24T00:48:42.970 INFO:tasks.workunit.client.0.smithi038.stdout:6/258: getdents d1/da/d1b/d25 0 2024-03-24T00:48:42.971 INFO:tasks.workunit.client.1.smithi088.stdout:0/247: getdents d9/d20/d2a 0 2024-03-24T00:48:42.972 INFO:tasks.workunit.client.1.smithi088.stdout:0/248: write d9/dc/de/f3e [585655,8462] 0 2024-03-24T00:48:42.972 INFO:tasks.workunit.client.1.smithi088.stdout:0/249: chown d9/d20/c22 0 1 2024-03-24T00:48:42.973 INFO:tasks.workunit.client.0.smithi038.stdout:7/207: symlink d2/d2c/l49 0 2024-03-24T00:48:42.973 INFO:tasks.workunit.client.0.smithi038.stdout:7/208: dread - d2/d9/da/f47 zero size 2024-03-24T00:48:42.986 INFO:tasks.workunit.client.1.smithi088.stdout:2/190: rename db/d21/l2d to db/d21/l3f 0 2024-03-24T00:48:42.987 INFO:tasks.workunit.client.1.smithi088.stdout:0/250: dread d9/dc/de/f40 [0,4194304] 0 2024-03-24T00:48:42.987 INFO:tasks.workunit.client.1.smithi088.stdout:5/208: truncate d1/d10/f17 2753956 0 2024-03-24T00:48:42.989 INFO:tasks.workunit.client.0.smithi038.stdout:0/190: truncate d0/d12/f3a 1682758 0 2024-03-24T00:48:42.989 INFO:tasks.workunit.client.0.smithi038.stdout:9/274: mknod d12/d55/c61 0 2024-03-24T00:48:42.989 INFO:tasks.workunit.client.0.smithi038.stdout:6/259: link d1/f1c d1/da/d1b/d25/d49/f4b 0 2024-03-24T00:48:42.992 INFO:tasks.workunit.client.1.smithi088.stdout:8/226: creat d9/dc/f52 x:0 0 0 2024-03-24T00:48:42.993 INFO:tasks.workunit.client.0.smithi038.stdout:5/178: symlink d6/l34 0 2024-03-24T00:48:42.997 INFO:tasks.workunit.client.1.smithi088.stdout:9/213: creat d1/d10/d11/f38 x:0 0 0 2024-03-24T00:48:43.005 INFO:tasks.workunit.client.1.smithi088.stdout:1/228: unlink d6/d27/d1e/f1f 0 2024-03-24T00:48:43.007 INFO:tasks.workunit.client.1.smithi088.stdout:4/249: getdents d5/d16/d37 0 2024-03-24T00:48:43.011 INFO:tasks.workunit.client.1.smithi088.stdout:3/202: rmdir d3/d1a/d1e 39 2024-03-24T00:48:43.015 INFO:tasks.workunit.client.1.smithi088.stdout:6/255: sync 2024-03-24T00:48:43.016 INFO:tasks.workunit.client.1.smithi088.stdout:2/191: mknod db/dc/c40 0 2024-03-24T00:48:43.021 INFO:tasks.workunit.client.0.smithi038.stdout:5/179: dread d6/f9 [0,4194304] 0 2024-03-24T00:48:43.021 INFO:tasks.workunit.client.1.smithi088.stdout:0/251: symlink d9/dc/de/d19/d44/d46/l53 0 2024-03-24T00:48:43.022 INFO:tasks.workunit.client.1.smithi088.stdout:5/209: unlink d1/d10/d1b/c2b 0 2024-03-24T00:48:43.024 INFO:tasks.workunit.client.1.smithi088.stdout:1/229: getdents d6/d23 0 2024-03-24T00:48:43.031 INFO:tasks.workunit.client.1.smithi088.stdout:4/250: rmdir d5/d13/d18 39 2024-03-24T00:48:43.032 INFO:tasks.workunit.client.1.smithi088.stdout:3/203: mkdir d3/de/df/d49 0 2024-03-24T00:48:43.036 INFO:tasks.workunit.client.0.smithi038.stdout:5/180: write d6/d1e/f1f [1020679,63053] 0 2024-03-24T00:48:43.036 INFO:tasks.workunit.client.0.smithi038.stdout:5/181: truncate d6/d23/f32 558814 0 2024-03-24T00:48:43.045 INFO:tasks.workunit.client.1.smithi088.stdout:2/192: truncate db/dd/f1f 3015438 0 2024-03-24T00:48:43.045 INFO:tasks.workunit.client.1.smithi088.stdout:2/193: write db/dd/f2f [1713368,67103] 0 2024-03-24T00:48:43.053 INFO:tasks.workunit.client.1.smithi088.stdout:5/210: read d1/f3b [978864,42238] 0 2024-03-24T00:48:43.056 INFO:tasks.workunit.client.1.smithi088.stdout:0/252: mknod d9/dc/c54 0 2024-03-24T00:48:43.060 INFO:tasks.workunit.client.1.smithi088.stdout:9/214: getdents d1/d10/d29 0 2024-03-24T00:48:43.062 INFO:tasks.workunit.client.1.smithi088.stdout:4/251: creat d5/d13/d18/d24/f53 x:0 0 0 2024-03-24T00:48:43.066 INFO:tasks.workunit.client.1.smithi088.stdout:3/204: link d3/d6/d27/d3c/c3a d3/de/df/d49/c4a 0 2024-03-24T00:48:43.069 INFO:tasks.workunit.client.1.smithi088.stdout:9/215: mkdir d1/d22/d27/d39 0 2024-03-24T00:48:43.074 INFO:tasks.workunit.client.1.smithi088.stdout:4/252: mknod d5/d7/dd/c54 0 2024-03-24T00:48:43.076 INFO:tasks.workunit.client.1.smithi088.stdout:3/205: getdents d3/d6/d19 0 2024-03-24T00:48:43.080 INFO:tasks.workunit.client.0.smithi038.stdout:1/187: dwrite f4 [4194304,4194304] 0 2024-03-24T00:48:43.086 INFO:tasks.workunit.client.0.smithi038.stdout:3/232: dread d3/f49 [0,4194304] 0 2024-03-24T00:48:43.086 INFO:tasks.workunit.client.0.smithi038.stdout:1/188: mkdir da/dd/d2c/d40 0 2024-03-24T00:48:43.086 INFO:tasks.workunit.client.0.smithi038.stdout:8/188: dwrite d0/d10/d19/f1b [0,4194304] 0 2024-03-24T00:48:43.086 INFO:tasks.workunit.client.0.smithi038.stdout:8/189: fdatasync d0/d45/d37/f44 0 2024-03-24T00:48:43.087 INFO:tasks.workunit.client.1.smithi088.stdout:4/253: creat d5/d13/d18/d24/d3d/f55 x:0 0 0 2024-03-24T00:48:43.087 INFO:tasks.workunit.client.1.smithi088.stdout:3/206: mkdir d3/d1a/d1e/d48/d4b 0 2024-03-24T00:48:43.089 INFO:tasks.workunit.client.0.smithi038.stdout:3/233: link d3/f25 d3/d9/dc/df/d14/f54 0 2024-03-24T00:48:43.092 INFO:tasks.workunit.client.0.smithi038.stdout:8/190: creat d0/d1d/d20/d3c/f47 x:0 0 0 2024-03-24T00:48:43.094 INFO:tasks.workunit.client.0.smithi038.stdout:3/234: mknod d3/d2a/c55 0 2024-03-24T00:48:43.095 INFO:tasks.workunit.client.1.smithi088.stdout:4/254: creat d5/f56 x:0 0 0 2024-03-24T00:48:43.095 INFO:tasks.workunit.client.1.smithi088.stdout:4/255: chown d5/d16/d1a/f1e 2372 1 2024-03-24T00:48:43.096 INFO:tasks.workunit.client.0.smithi038.stdout:2/242: dwrite d4/d27/f47 [0,4194304] 0 2024-03-24T00:48:43.097 INFO:tasks.workunit.client.1.smithi088.stdout:4/256: creat d5/d16/f57 x:0 0 0 2024-03-24T00:48:43.097 INFO:tasks.workunit.client.1.smithi088.stdout:4/257: readlink d5/l2d 0 2024-03-24T00:48:43.099 INFO:tasks.workunit.client.0.smithi038.stdout:6/260: dread d1/da/d1b/d25/d49/f4b [0,4194304] 0 2024-03-24T00:48:43.099 INFO:tasks.workunit.client.1.smithi088.stdout:4/258: symlink d5/d13/d18/d24/l58 0 2024-03-24T00:48:43.099 INFO:tasks.workunit.client.1.smithi088.stdout:4/259: write d5/d16/d37/f4e [680030,18472] 0 2024-03-24T00:48:43.101 INFO:tasks.workunit.client.1.smithi088.stdout:4/260: symlink d5/d13/l59 0 2024-03-24T00:48:43.102 INFO:tasks.workunit.client.1.smithi088.stdout:4/261: dread - d5/d16/d1a/f50 zero size 2024-03-24T00:48:43.102 INFO:tasks.workunit.client.0.smithi038.stdout:1/189: write da/dd/f1c [134810,116260] 0 2024-03-24T00:48:43.103 INFO:tasks.workunit.client.0.smithi038.stdout:2/243: symlink d4/d18/d26/d37/l52 0 2024-03-24T00:48:43.104 INFO:tasks.workunit.client.0.smithi038.stdout:2/244: dread - d4/d18/d26/f3d zero size 2024-03-24T00:48:43.104 INFO:tasks.workunit.client.1.smithi088.stdout:5/211: dread d1/f3b [0,4194304] 0 2024-03-24T00:48:43.104 INFO:tasks.workunit.client.1.smithi088.stdout:4/262: creat d5/d16/d37/f5a x:0 0 0 2024-03-24T00:48:43.105 INFO:tasks.workunit.client.0.smithi038.stdout:8/191: creat d0/d45/d37/f48 x:0 0 0 2024-03-24T00:48:43.105 INFO:tasks.workunit.client.1.smithi088.stdout:5/212: creat d1/d19/d1e/f47 x:0 0 0 2024-03-24T00:48:43.105 INFO:tasks.workunit.client.1.smithi088.stdout:5/213: truncate d1/dd/f23 4496176 0 2024-03-24T00:48:43.106 INFO:tasks.workunit.client.1.smithi088.stdout:6/256: dwrite db/f47 [0,4194304] 0 2024-03-24T00:48:43.106 INFO:tasks.workunit.client.1.smithi088.stdout:6/257: write db/dd/f10 [5384319,67898] 0 2024-03-24T00:48:43.106 INFO:tasks.workunit.client.1.smithi088.stdout:6/258: truncate db/f47 4273207 0 2024-03-24T00:48:43.106 INFO:tasks.workunit.client.1.smithi088.stdout:6/259: creat db/d40/f4d x:0 0 0 2024-03-24T00:48:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:42 smithi038 ceph-mon[29271]: pgmap v27: 65 pgs: 65 active+clean; 774 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 72 MiB/s rd, 84 MiB/s wr, 529 op/s 2024-03-24T00:48:43.111 INFO:tasks.workunit.client.0.smithi038.stdout:1/190: mkdir da/dd/d2c/d41 0 2024-03-24T00:48:43.111 INFO:tasks.workunit.client.0.smithi038.stdout:2/245: mkdir d4/d18/d26/d4b/d53 0 2024-03-24T00:48:43.112 INFO:tasks.workunit.client.0.smithi038.stdout:0/191: dwrite d0/d2d/f2f [0,4194304] 0 2024-03-24T00:48:43.114 INFO:tasks.workunit.client.1.smithi088.stdout:5/214: truncate d1/d10/f17 1726926 0 2024-03-24T00:48:43.114 INFO:tasks.workunit.client.1.smithi088.stdout:6/260: mknod db/d1d/d23/d3c/d43/d24/c4e 0 2024-03-24T00:48:43.114 INFO:tasks.workunit.client.1.smithi088.stdout:6/261: fdatasync db/f38 0 2024-03-24T00:48:43.119 INFO:tasks.workunit.client.0.smithi038.stdout:8/192: symlink d0/d15/l49 0 2024-03-24T00:48:43.120 INFO:tasks.workunit.client.0.smithi038.stdout:8/193: write d0/d10/d19/f1b [4892164,104146] 0 2024-03-24T00:48:43.120 INFO:tasks.workunit.client.0.smithi038.stdout:8/194: write d0/d45/d37/f44 [314300,86864] 0 2024-03-24T00:48:43.120 INFO:tasks.workunit.client.0.smithi038.stdout:8/195: read - d0/d45/d37/f48 zero size 2024-03-24T00:48:43.120 INFO:tasks.workunit.client.0.smithi038.stdout:8/196: truncate d0/d15/d42/f40 1661537 0 2024-03-24T00:48:43.120 INFO:tasks.workunit.client.0.smithi038.stdout:8/197: chown d0/d1d/c1e 2 1 2024-03-24T00:48:43.125 INFO:tasks.workunit.client.1.smithi088.stdout:0/253: write d9/f3a [1088859,61014] 0 2024-03-24T00:48:43.125 INFO:tasks.workunit.client.1.smithi088.stdout:8/227: dwrite d9/f17 [0,4194304] 0 2024-03-24T00:48:43.136 INFO:tasks.workunit.client.1.smithi088.stdout:1/230: dwrite d6/d27/d1e/f39 [0,4194304] 0 2024-03-24T00:48:43.139 INFO:tasks.workunit.client.0.smithi038.stdout:0/192: dread d0/d12/d13/d28/f39 [4194304,4194304] 0 2024-03-24T00:48:43.149 INFO:tasks.workunit.client.1.smithi088.stdout:1/231: unlink d6/f13 0 2024-03-24T00:48:43.151 INFO:tasks.workunit.client.1.smithi088.stdout:3/207: rmdir d3/de/df 39 2024-03-24T00:48:43.153 INFO:tasks.workunit.client.0.smithi038.stdout:8/198: dread d0/f32 [0,4194304] 0 2024-03-24T00:48:43.153 INFO:tasks.workunit.client.0.smithi038.stdout:8/199: fsync d0/d1d/f3e 0 2024-03-24T00:48:43.155 INFO:tasks.workunit.client.0.smithi038.stdout:0/193: mknod d0/d2d/c3e 0 2024-03-24T00:48:43.156 INFO:tasks.workunit.client.0.smithi038.stdout:0/194: write d0/d12/d15/f17 [141347,105983] 0 2024-03-24T00:48:43.159 INFO:tasks.workunit.client.1.smithi088.stdout:1/232: mknod d6/d23/c42 0 2024-03-24T00:48:43.159 INFO:tasks.workunit.client.1.smithi088.stdout:1/233: fsync d6/d23/f2d 0 2024-03-24T00:48:43.159 INFO:tasks.workunit.client.1.smithi088.stdout:4/263: dread d5/d16/d1a/f1e [0,4194304] 0 2024-03-24T00:48:43.165 INFO:tasks.workunit.client.1.smithi088.stdout:1/234: chown d6/d27/c2e 1423 1 2024-03-24T00:48:43.168 INFO:tasks.workunit.client.0.smithi038.stdout:4/199: dwrite da/d12/d37/dd/d15/f17 [0,4194304] 0 2024-03-24T00:48:43.169 INFO:tasks.workunit.client.1.smithi088.stdout:4/264: rename d5/d13/l59 to d5/d7/dd/l5b 0 2024-03-24T00:48:43.170 INFO:tasks.workunit.client.1.smithi088.stdout:3/208: getdents d3/d6 0 2024-03-24T00:48:43.175 INFO:tasks.workunit.client.1.smithi088.stdout:0/254: dread d9/dc/de/f3e [0,4194304] 0 2024-03-24T00:48:43.175 INFO:tasks.workunit.client.1.smithi088.stdout:0/255: write d9/dc/de/d19/f3b [280037,101096] 0 2024-03-24T00:48:43.175 INFO:tasks.workunit.client.1.smithi088.stdout:1/235: mkdir d6/d43 0 2024-03-24T00:48:43.178 INFO:tasks.workunit.client.1.smithi088.stdout:8/228: write d9/dc/d11/d24/f2a [1773390,96530] 0 2024-03-24T00:48:43.180 INFO:tasks.workunit.client.1.smithi088.stdout:4/265: rename d5/d16/d1a/d22/l4c to d5/d13/d18/l5c 0 2024-03-24T00:48:43.184 INFO:tasks.workunit.client.1.smithi088.stdout:9/216: dwrite d1/f7 [0,4194304] 0 2024-03-24T00:48:43.186 INFO:tasks.workunit.client.0.smithi038.stdout:5/182: dwrite d6/f26 [0,4194304] 0 2024-03-24T00:48:43.186 INFO:tasks.workunit.client.0.smithi038.stdout:5/183: chown d6/d16/d21/f2c 11 1 2024-03-24T00:48:43.187 INFO:tasks.workunit.client.0.smithi038.stdout:6/261: dwrite d1/da/fb [0,4194304] 0 2024-03-24T00:48:43.191 INFO:tasks.workunit.client.1.smithi088.stdout:5/215: dwrite d1/dd/f13 [4194304,4194304] 0 2024-03-24T00:48:43.191 INFO:tasks.workunit.client.1.smithi088.stdout:5/216: write d1/dd/f15 [3162128,85720] 0 2024-03-24T00:48:43.192 INFO:tasks.workunit.client.0.smithi038.stdout:5/184: dread d6/d23/f32 [0,4194304] 0 2024-03-24T00:48:43.193 INFO:tasks.workunit.client.1.smithi088.stdout:8/229: stat d9/dc/d11/d24/d29/l30 0 2024-03-24T00:48:43.193 INFO:tasks.workunit.client.1.smithi088.stdout:8/230: chown d9/dc/d11/d24/d29/d36/f4d 14 1 2024-03-24T00:48:43.193 INFO:tasks.workunit.client.1.smithi088.stdout:8/231: stat d9/f17 0 2024-03-24T00:48:43.193 INFO:tasks.workunit.client.1.smithi088.stdout:8/232: fsync d9/f28 0 2024-03-24T00:48:43.194 INFO:tasks.workunit.client.0.smithi038.stdout:3/235: dwrite d3/d2a/f4b [4194304,4194304] 0 2024-03-24T00:48:43.194 INFO:tasks.workunit.client.0.smithi038.stdout:3/236: chown d3/l4f 184258 1 2024-03-24T00:48:43.194 INFO:tasks.workunit.client.0.smithi038.stdout:3/237: chown c0 38 1 2024-03-24T00:48:43.194 INFO:tasks.workunit.client.0.smithi038.stdout:4/200: getdents da/d12 0 2024-03-24T00:48:43.196 INFO:tasks.workunit.client.0.smithi038.stdout:1/191: dwrite da/f16 [0,4194304] 0 2024-03-24T00:48:43.196 INFO:tasks.workunit.client.0.smithi038.stdout:1/192: chown da/dd/d10/c12 361624129 1 2024-03-24T00:48:43.197 INFO:tasks.workunit.client.1.smithi088.stdout:8/233: truncate f1 4214283 0 2024-03-24T00:48:43.197 INFO:tasks.workunit.client.1.smithi088.stdout:0/256: dread d9/dc/de/f1d [0,4194304] 0 2024-03-24T00:48:43.197 INFO:tasks.workunit.client.1.smithi088.stdout:2/194: dwrite db/f38 [0,4194304] 0 2024-03-24T00:48:43.197 INFO:tasks.workunit.client.1.smithi088.stdout:2/195: stat db/dc/l26 0 2024-03-24T00:48:43.197 INFO:tasks.workunit.client.1.smithi088.stdout:6/262: dwrite db/d1d/f48 [0,4194304] 0 2024-03-24T00:48:43.197 INFO:tasks.workunit.client.1.smithi088.stdout:4/266: rename d5/l2d to d5/d7/d15/l5d 0 2024-03-24T00:48:43.206 INFO:tasks.workunit.client.1.smithi088.stdout:9/217: creat d1/d15/d21/f3a x:0 0 0 2024-03-24T00:48:43.206 INFO:tasks.workunit.client.0.smithi038.stdout:2/246: rename d4/d18/d26/d4b to d4/d27/d54 0 2024-03-24T00:48:43.207 INFO:tasks.workunit.client.0.smithi038.stdout:2/247: chown d4/d27/c2d 7046 1 2024-03-24T00:48:43.207 INFO:tasks.workunit.client.0.smithi038.stdout:2/248: chown d4/fc 46 1 2024-03-24T00:48:43.207 INFO:tasks.workunit.client.0.smithi038.stdout:2/249: chown d4/d18/d26 489717 1 2024-03-24T00:48:43.207 INFO:tasks.workunit.client.0.smithi038.stdout:2/250: truncate d4/f41 568205 0 2024-03-24T00:48:43.209 INFO:tasks.workunit.client.1.smithi088.stdout:5/217: dread d1/dd/f13 [4194304,4194304] 0 2024-03-24T00:48:43.209 INFO:tasks.workunit.client.1.smithi088.stdout:5/218: chown d1/l8 5039689 1 2024-03-24T00:48:43.210 INFO:tasks.workunit.client.1.smithi088.stdout:1/236: mknod d6/d27/c44 0 2024-03-24T00:48:43.210 INFO:tasks.workunit.client.1.smithi088.stdout:8/234: dread d9/f3e [0,4194304] 0 2024-03-24T00:48:43.215 INFO:tasks.workunit.client.1.smithi088.stdout:2/196: dread db/dc/f11 [0,4194304] 0 2024-03-24T00:48:43.215 INFO:tasks.workunit.client.1.smithi088.stdout:2/197: dread - db/f3c zero size 2024-03-24T00:48:43.217 INFO:tasks.workunit.client.0.smithi038.stdout:6/262: symlink d1/d37/l4c 0 2024-03-24T00:48:43.218 INFO:tasks.workunit.client.1.smithi088.stdout:6/263: dread db/d1d/f48 [0,4194304] 0 2024-03-24T00:48:43.218 INFO:tasks.workunit.client.1.smithi088.stdout:6/264: truncate db/f47 4573457 0 2024-03-24T00:48:43.221 INFO:tasks.workunit.client.0.smithi038.stdout:8/200: dread d0/d10/d19/f1b [0,4194304] 0 2024-03-24T00:48:43.221 INFO:tasks.workunit.client.0.smithi038.stdout:8/201: write d0/f16 [420985,39853] 0 2024-03-24T00:48:43.227 INFO:tasks.workunit.client.1.smithi088.stdout:4/267: dread d5/d13/f12 [0,4194304] 0 2024-03-24T00:48:43.231 INFO:tasks.workunit.client.1.smithi088.stdout:2/198: dread db/dd/f1f [0,4194304] 0 2024-03-24T00:48:43.234 INFO:tasks.workunit.client.0.smithi038.stdout:3/238: symlink d3/d9/dc/df/d14/l56 0 2024-03-24T00:48:43.234 INFO:tasks.workunit.client.0.smithi038.stdout:3/239: chown d3/l4f 3984667 1 2024-03-24T00:48:43.238 INFO:tasks.workunit.client.1.smithi088.stdout:0/257: link d9/dc/de/d19/d44/f4a d9/f55 0 2024-03-24T00:48:43.238 INFO:tasks.workunit.client.1.smithi088.stdout:0/258: fdatasync d9/f55 0 2024-03-24T00:48:43.242 INFO:tasks.workunit.client.0.smithi038.stdout:0/195: rmdir d0 39 2024-03-24T00:48:43.242 INFO:tasks.workunit.client.0.smithi038.stdout:0/196: dread - d0/d1b/d2c/f3b zero size 2024-03-24T00:48:43.248 INFO:tasks.workunit.client.0.smithi038.stdout:6/263: write d1/da/fb [3010820,124208] 0 2024-03-24T00:48:43.248 INFO:tasks.workunit.client.0.smithi038.stdout:6/264: readlink d1/da/l29 0 2024-03-24T00:48:43.258 INFO:tasks.workunit.client.1.smithi088.stdout:5/219: mkdir d1/d19/d48 0 2024-03-24T00:48:43.258 INFO:tasks.workunit.client.1.smithi088.stdout:4/268: mknod d5/d17/c5e 0 2024-03-24T00:48:43.259 INFO:tasks.workunit.client.1.smithi088.stdout:9/218: dwrite d1/f20 [0,4194304] 0 2024-03-24T00:48:43.259 INFO:tasks.workunit.client.1.smithi088.stdout:1/237: chown d6/d27/f3a 438 1 2024-03-24T00:48:43.262 INFO:tasks.workunit.client.1.smithi088.stdout:2/199: truncate db/dd/f17 3824691 0 2024-03-24T00:48:43.270 INFO:tasks.workunit.client.1.smithi088.stdout:7/210: sync 2024-03-24T00:48:43.270 INFO:tasks.workunit.client.1.smithi088.stdout:4/269: rmdir d5/d13 39 2024-03-24T00:48:43.272 INFO:tasks.workunit.client.0.smithi038.stdout:2/251: mkdir d4/d55 0 2024-03-24T00:48:43.276 INFO:tasks.workunit.client.1.smithi088.stdout:9/219: mknod d1/d22/d27/d39/c3b 0 2024-03-24T00:48:43.277 INFO:tasks.workunit.client.1.smithi088.stdout:5/220: unlink d1/d19/d1e/c44 0 2024-03-24T00:48:43.277 INFO:tasks.workunit.client.1.smithi088.stdout:5/221: dread - d1/d19/d1e/f47 zero size 2024-03-24T00:48:43.277 INFO:tasks.workunit.client.1.smithi088.stdout:5/222: write d1/dd/f23 [5485757,14688] 0 2024-03-24T00:48:43.280 INFO:tasks.workunit.client.0.smithi038.stdout:5/185: dwrite d6/fa [0,4194304] 0 2024-03-24T00:48:43.280 INFO:tasks.workunit.client.0.smithi038.stdout:5/186: stat d6/d16/d21/d2d 0 2024-03-24T00:48:43.283 INFO:tasks.workunit.client.1.smithi088.stdout:1/238: rename d6/ff to d6/d43/f45 0 2024-03-24T00:48:43.284 INFO:tasks.workunit.client.0.smithi038.stdout:8/202: mknod d0/d10/d19/c4a 0 2024-03-24T00:48:43.285 INFO:tasks.workunit.client.0.smithi038.stdout:8/203: chown d0/c4 104053212 1 2024-03-24T00:48:43.290 INFO:tasks.workunit.client.1.smithi088.stdout:9/220: symlink d1/d10/d11/d23/l3c 0 2024-03-24T00:48:43.291 INFO:tasks.workunit.client.0.smithi038.stdout:6/265: write d1/f10 [7030114,26619] 0 2024-03-24T00:48:43.291 INFO:tasks.workunit.client.0.smithi038.stdout:6/266: read - d1/da/d1b/d1d/f43 zero size 2024-03-24T00:48:43.294 INFO:tasks.workunit.client.1.smithi088.stdout:2/200: dread db/dc/f1b [0,4194304] 0 2024-03-24T00:48:43.299 INFO:tasks.workunit.client.1.smithi088.stdout:8/235: dwrite d9/dc/d11/d1b/f1e [0,4194304] 0 2024-03-24T00:48:43.299 INFO:tasks.workunit.client.1.smithi088.stdout:8/236: creat d9/dc/d11/d1b/f53 x:0 0 0 2024-03-24T00:48:43.299 INFO:tasks.workunit.client.1.smithi088.stdout:8/237: chown d9/dc/d11/d24/d43/f50 5 1 2024-03-24T00:48:43.299 INFO:tasks.workunit.client.1.smithi088.stdout:8/238: truncate d9/dc/d11/d24/d29/d36/f4e 118009 0 2024-03-24T00:48:43.300 INFO:tasks.workunit.client.0.smithi038.stdout:5/187: rename d6/d16/d21/f28 to d6/d16/d21/f35 0 2024-03-24T00:48:43.303 INFO:tasks.workunit.client.1.smithi088.stdout:7/211: dread d3/f33 [0,4194304] 0 2024-03-24T00:48:43.303 INFO:tasks.workunit.client.1.smithi088.stdout:5/223: symlink d1/d19/d1e/d26/l49 0 2024-03-24T00:48:43.303 INFO:tasks.workunit.client.1.smithi088.stdout:5/224: truncate d1/d19/d1e/f42 828698 0 2024-03-24T00:48:43.303 INFO:tasks.workunit.client.1.smithi088.stdout:5/225: fdatasync d1/d29/d3f/f41 0 2024-03-24T00:48:43.305 INFO:tasks.workunit.client.1.smithi088.stdout:4/270: dread d5/d13/d18/d24/f30 [0,4194304] 0 2024-03-24T00:48:43.305 INFO:tasks.workunit.client.1.smithi088.stdout:8/239: write d9/dc/d11/d24/d29/f51 [44014,112039] 0 2024-03-24T00:48:43.305 INFO:tasks.workunit.client.1.smithi088.stdout:4/271: write d5/f56 [1038264,50389] 0 2024-03-24T00:48:43.306 INFO:tasks.workunit.client.1.smithi088.stdout:2/201: dread db/d21/f33 [0,4194304] 0 2024-03-24T00:48:43.306 INFO:tasks.workunit.client.1.smithi088.stdout:0/259: dwrite d9/dc/f49 [0,4194304] 0 2024-03-24T00:48:43.307 INFO:tasks.workunit.client.1.smithi088.stdout:2/202: dread - db/dc/f30 zero size 2024-03-24T00:48:43.307 INFO:tasks.workunit.client.1.smithi088.stdout:0/260: write d9/dc/de/f3e [215230,42741] 0 2024-03-24T00:48:43.309 INFO:tasks.workunit.client.0.smithi038.stdout:3/240: rename d3/c15 to d3/d2a/c57 0 2024-03-24T00:48:43.309 INFO:tasks.workunit.client.0.smithi038.stdout:3/241: chown d3/d9/dc/c13 1 1 2024-03-24T00:48:43.311 INFO:tasks.workunit.client.1.smithi088.stdout:6/265: dwrite db/dd/f37 [4194304,4194304] 0 2024-03-24T00:48:43.311 INFO:tasks.workunit.client.1.smithi088.stdout:6/266: fdatasync db/f47 0 2024-03-24T00:48:43.313 INFO:tasks.workunit.client.1.smithi088.stdout:1/239: symlink d6/l46 0 2024-03-24T00:48:43.324 INFO:tasks.workunit.client.0.smithi038.stdout:8/204: getdents d0/d15/d2c 0 2024-03-24T00:48:43.328 INFO:tasks.workunit.client.0.smithi038.stdout:1/193: dwrite da/dd/d10/f1f [4194304,4194304] 0 2024-03-24T00:48:43.328 INFO:tasks.workunit.client.0.smithi038.stdout:1/194: write da/dd/d10/d34/f3d [1538302,103832] 0 2024-03-24T00:48:43.332 INFO:tasks.workunit.client.0.smithi038.stdout:3/242: rename d3/d9/dc/df/d14/f21 to d3/d9/f58 0 2024-03-24T00:48:43.337 INFO:tasks.workunit.client.1.smithi088.stdout:6/267: dread db/d1d/d23/d3c/d43/f34 [0,4194304] 0 2024-03-24T00:48:43.337 INFO:tasks.workunit.client.1.smithi088.stdout:7/212: creat d3/f45 x:0 0 0 2024-03-24T00:48:43.345 INFO:tasks.workunit.client.1.smithi088.stdout:5/226: creat d1/d10/d1b/d2f/f4a x:0 0 0 2024-03-24T00:48:43.345 INFO:tasks.workunit.client.0.smithi038.stdout:5/188: rmdir d6/d16/d14 39 2024-03-24T00:48:43.353 INFO:tasks.workunit.client.0.smithi038.stdout:2/252: dwrite d4/f23 [0,4194304] 0 2024-03-24T00:48:43.353 INFO:tasks.workunit.client.0.smithi038.stdout:4/201: dwrite da/d12/d24/f30 [0,4194304] 0 2024-03-24T00:48:43.353 INFO:tasks.workunit.client.0.smithi038.stdout:8/205: truncate d0/f32 3811242 0 2024-03-24T00:48:43.353 INFO:tasks.workunit.client.0.smithi038.stdout:8/206: chown d0/d15/d42 19689 1 2024-03-24T00:48:43.353 INFO:tasks.workunit.client.0.smithi038.stdout:8/207: stat d0/d10/d19/f1b 0 2024-03-24T00:48:43.354 INFO:tasks.workunit.client.0.smithi038.stdout:6/267: dwrite d1/da/d1b/f2b [0,4194304] 0 2024-03-24T00:48:43.356 INFO:tasks.workunit.client.1.smithi088.stdout:6/268: write f8 [3259399,48289] 0 2024-03-24T00:48:43.364 INFO:tasks.workunit.client.1.smithi088.stdout:5/227: dread d1/f31 [0,4194304] 0 2024-03-24T00:48:43.370 INFO:tasks.workunit.client.1.smithi088.stdout:2/203: unlink db/dd/l37 0 2024-03-24T00:48:43.374 INFO:tasks.workunit.client.1.smithi088.stdout:0/261: rename d9/d20/d2a/d2e/f4e to d9/f56 0 2024-03-24T00:48:43.374 INFO:tasks.workunit.client.1.smithi088.stdout:0/262: dread - d9/d26/f3c zero size 2024-03-24T00:48:43.375 INFO:tasks.workunit.client.0.smithi038.stdout:0/197: rename d0/d12/d15/c27 to d0/d14/c3f 0 2024-03-24T00:48:43.375 INFO:tasks.workunit.client.0.smithi038.stdout:2/253: rename d4/d27/d2f to d4/d27/d2f/d56 22 2024-03-24T00:48:43.375 INFO:tasks.workunit.client.0.smithi038.stdout:2/254: creat d4/d18/d38/f57 x:0 0 0 2024-03-24T00:48:43.375 INFO:tasks.workunit.client.0.smithi038.stdout:3/243: rmdir d3/d9 39 2024-03-24T00:48:43.375 INFO:tasks.workunit.client.0.smithi038.stdout:3/244: readlink d3/l38 0 2024-03-24T00:48:43.379 INFO:tasks.workunit.client.1.smithi088.stdout:8/240: dwrite d9/dc/f44 [0,4194304] 0 2024-03-24T00:48:43.405 INFO:tasks.workunit.client.1.smithi088.stdout:1/240: write d6/d27/f1d [719924,73004] 0 2024-03-24T00:48:43.410 INFO:tasks.workunit.client.1.smithi088.stdout:7/213: unlink d3/f16 0 2024-03-24T00:48:43.410 INFO:tasks.workunit.client.1.smithi088.stdout:9/221: truncate d1/f6 2515744 0 2024-03-24T00:48:43.410 INFO:tasks.workunit.client.1.smithi088.stdout:9/222: readlink d1/l16 0 2024-03-24T00:48:43.411 INFO:tasks.workunit.client.1.smithi088.stdout:9/223: write d1/d10/d11/d28/f36 [3273410,65236] 0 2024-03-24T00:48:43.411 INFO:tasks.workunit.client.1.smithi088.stdout:9/224: fsync d1/d15/d21/f3a 0 2024-03-24T00:48:43.415 INFO:tasks.workunit.client.0.smithi038.stdout:1/195: write da/d3c/f13 [1381762,123325] 0 2024-03-24T00:48:43.415 INFO:tasks.workunit.client.0.smithi038.stdout:1/196: creat da/dd/d10/d34/f42 x:0 0 0 2024-03-24T00:48:43.415 INFO:tasks.workunit.client.0.smithi038.stdout:1/197: truncate da/f21 1047591 0 2024-03-24T00:48:43.418 INFO:tasks.workunit.client.1.smithi088.stdout:6/269: unlink db/l15 0 2024-03-24T00:48:43.418 INFO:tasks.workunit.client.1.smithi088.stdout:4/272: getdents d5 0 2024-03-24T00:48:43.419 INFO:tasks.workunit.client.0.smithi038.stdout:6/268: unlink d1/da/d1b/f20 0 2024-03-24T00:48:43.419 INFO:tasks.workunit.client.0.smithi038.stdout:6/269: chown d1/da/d1b/d1d 21252 1 2024-03-24T00:48:43.421 INFO:tasks.workunit.client.1.smithi088.stdout:0/263: creat d9/dc/de/d19/d44/d46/f57 x:0 0 0 2024-03-24T00:48:43.421 INFO:tasks.workunit.client.1.smithi088.stdout:2/204: write db/f20 [7046250,28341] 0 2024-03-24T00:48:43.423 INFO:tasks.workunit.client.1.smithi088.stdout:8/241: truncate d9/f37 2896154 0 2024-03-24T00:48:43.425 INFO:tasks.workunit.client.0.smithi038.stdout:2/255: creat d4/d27/d2f/d30/d3e/f58 x:0 0 0 2024-03-24T00:48:43.426 INFO:tasks.workunit.client.0.smithi038.stdout:5/189: mknod d6/c36 0 2024-03-24T00:48:43.427 INFO:tasks.workunit.client.0.smithi038.stdout:4/202: dwrite da/f1d [0,4194304] 0 2024-03-24T00:48:43.428 INFO:tasks.workunit.client.1.smithi088.stdout:9/225: mknod d1/d10/d2c/c3d 0 2024-03-24T00:48:43.430 INFO:tasks.workunit.client.0.smithi038.stdout:1/198: mknod da/dd/d10/d14/c43 0 2024-03-24T00:48:43.431 INFO:tasks.workunit.client.1.smithi088.stdout:6/270: mknod db/d1d/d23/d3c/d43/d24/c4f 0 2024-03-24T00:48:43.431 INFO:tasks.workunit.client.1.smithi088.stdout:6/271: fdatasync db/dd/f10 0 2024-03-24T00:48:43.431 INFO:tasks.workunit.client.0.smithi038.stdout:6/270: symlink d1/da/d1b/d1d/l4d 0 2024-03-24T00:48:43.432 INFO:tasks.workunit.client.0.smithi038.stdout:2/256: mknod d4/d18/d38/c59 0 2024-03-24T00:48:43.433 INFO:tasks.workunit.client.0.smithi038.stdout:2/257: fdatasync d4/d27/f47 0 2024-03-24T00:48:43.434 INFO:tasks.workunit.client.1.smithi088.stdout:2/205: link db/dc/l26 db/d21/l41 0 2024-03-24T00:48:43.438 INFO:tasks.workunit.client.0.smithi038.stdout:5/190: mknod d6/d19/d20/d33/c37 0 2024-03-24T00:48:43.438 INFO:tasks.workunit.client.0.smithi038.stdout:5/191: write d6/d1e/f1f [2266346,94303] 0 2024-03-24T00:48:43.438 INFO:tasks.workunit.client.0.smithi038.stdout:4/203: symlink da/d12/d29/l39 0 2024-03-24T00:48:43.438 INFO:tasks.workunit.client.0.smithi038.stdout:4/204: getdents da/d12/d37/dd/d15/d33 0 2024-03-24T00:48:43.438 INFO:tasks.workunit.client.0.smithi038.stdout:4/205: chown da/d12/d29/l31 40 1 2024-03-24T00:48:43.439 INFO:tasks.workunit.client.1.smithi088.stdout:6/272: mkdir db/d1d/d50 0 2024-03-24T00:48:43.440 INFO:tasks.workunit.client.0.smithi038.stdout:6/271: mkdir d1/da/d1b/d25/d49/d4e 0 2024-03-24T00:48:43.440 INFO:tasks.workunit.client.0.smithi038.stdout:6/272: getdents d1/da/d1b/d2f 0 2024-03-24T00:48:43.448 INFO:tasks.workunit.client.0.smithi038.stdout:2/258: rename d4/d18/f1f to d4/d18/d38/f5a 0 2024-03-24T00:48:43.448 INFO:tasks.workunit.client.0.smithi038.stdout:2/259: readlink d4/d27/d2f/l51 0 2024-03-24T00:48:43.448 INFO:tasks.workunit.client.0.smithi038.stdout:4/206: creat da/d12/d37/dd/f3a x:0 0 0 2024-03-24T00:48:43.449 INFO:tasks.workunit.client.0.smithi038.stdout:4/207: rmdir da/d12/d24 39 2024-03-24T00:48:43.453 INFO:tasks.workunit.client.1.smithi088.stdout:3/209: sync 2024-03-24T00:48:43.453 INFO:tasks.workunit.client.0.smithi038.stdout:9/275: sync 2024-03-24T00:48:43.458 INFO:tasks.workunit.client.0.smithi038.stdout:7/209: sync 2024-03-24T00:48:43.458 INFO:tasks.workunit.client.0.smithi038.stdout:9/276: mknod d12/d28/d3a/d26/c62 0 2024-03-24T00:48:43.462 INFO:tasks.workunit.client.1.smithi088.stdout:7/214: dread d3/f33 [0,4194304] 0 2024-03-24T00:48:43.463 INFO:tasks.workunit.client.0.smithi038.stdout:3/245: dwrite d3/f25 [4194304,4194304] 0 2024-03-24T00:48:43.467 INFO:tasks.workunit.client.0.smithi038.stdout:9/277: mknod d12/d28/d3a/c63 0 2024-03-24T00:48:43.468 INFO:tasks.workunit.client.0.smithi038.stdout:7/210: mkdir d2/d9/da/dc/d4a 0 2024-03-24T00:48:43.472 INFO:tasks.workunit.client.0.smithi038.stdout:5/192: dread d6/d19/f1c [0,4194304] 0 2024-03-24T00:48:43.472 INFO:tasks.workunit.client.1.smithi088.stdout:7/215: symlink d3/d32/d3f/l46 0 2024-03-24T00:48:43.473 INFO:tasks.workunit.client.0.smithi038.stdout:9/278: rename d12/d1a/l24 to d12/d41/d57/l64 0 2024-03-24T00:48:43.473 INFO:tasks.workunit.client.0.smithi038.stdout:9/279: truncate d12/d41/f5b 870240 0 2024-03-24T00:48:43.473 INFO:tasks.workunit.client.0.smithi038.stdout:9/280: read - d12/d1a/f49 zero size 2024-03-24T00:48:43.473 INFO:tasks.workunit.client.0.smithi038.stdout:9/281: chown d12/d1a/d5f 154974 1 2024-03-24T00:48:43.473 INFO:tasks.workunit.client.0.smithi038.stdout:9/282: fsync d12/d28/d3a/d26/f5a 0 2024-03-24T00:48:43.475 INFO:tasks.workunit.client.0.smithi038.stdout:7/211: link d2/f13 d2/d2c/f4b 0 2024-03-24T00:48:43.475 INFO:tasks.workunit.client.0.smithi038.stdout:2/260: write d4/f13 [211987,76245] 0 2024-03-24T00:48:43.477 INFO:tasks.workunit.client.0.smithi038.stdout:5/193: unlink d6/d16/l1a 0 2024-03-24T00:48:43.477 INFO:tasks.workunit.client.1.smithi088.stdout:5/228: dwrite d1/f31 [0,4194304] 0 2024-03-24T00:48:43.480 INFO:tasks.workunit.client.0.smithi038.stdout:9/283: truncate d12/d28/d3a/f30 3214860 0 2024-03-24T00:48:43.480 INFO:tasks.workunit.client.0.smithi038.stdout:9/284: fdatasync f4 0 2024-03-24T00:48:43.480 INFO:tasks.workunit.client.0.smithi038.stdout:9/285: chown d12/d28/d3a/d26/f3d 25698 1 2024-03-24T00:48:43.487 INFO:tasks.workunit.client.1.smithi088.stdout:7/216: dread d3/d32/d34/f3b [0,4194304] 0 2024-03-24T00:48:43.487 INFO:tasks.workunit.client.0.smithi038.stdout:7/212: creat d2/d9/df/d33/d37/f4c x:0 0 0 2024-03-24T00:48:43.489 INFO:tasks.workunit.client.0.smithi038.stdout:2/261: mkdir d4/d18/d5b 0 2024-03-24T00:48:43.491 INFO:tasks.workunit.client.0.smithi038.stdout:7/213: creat d2/d2c/f4d x:0 0 0 2024-03-24T00:48:43.491 INFO:tasks.workunit.client.1.smithi088.stdout:8/242: write d9/dc/d11/d1b/f1e [3773061,40644] 0 2024-03-24T00:48:43.491 INFO:tasks.workunit.client.1.smithi088.stdout:8/243: stat d9/dc/d11/d24/d29/d39/f48 0 2024-03-24T00:48:43.491 INFO:tasks.workunit.client.0.smithi038.stdout:3/246: dread d3/d9/dc/df/f31 [0,4194304] 0 2024-03-24T00:48:43.495 INFO:tasks.workunit.client.0.smithi038.stdout:5/194: mknod d6/d19/d20/c38 0 2024-03-24T00:48:43.498 INFO:tasks.workunit.client.0.smithi038.stdout:2/262: unlink d4/c32 0 2024-03-24T00:48:43.498 INFO:tasks.workunit.client.0.smithi038.stdout:2/263: truncate d4/d27/f2a 1550649 0 2024-03-24T00:48:43.499 INFO:tasks.workunit.client.0.smithi038.stdout:5/195: link d6/d19/d20/d33/c37 d6/d1e/c39 0 2024-03-24T00:48:43.502 INFO:tasks.workunit.client.0.smithi038.stdout:5/196: unlink d6/d16/d14/l1d 0 2024-03-24T00:48:43.502 INFO:tasks.workunit.client.0.smithi038.stdout:5/197: chown d6/d19 15 1 2024-03-24T00:48:43.504 INFO:tasks.workunit.client.0.smithi038.stdout:6/273: dread d1/f21 [0,4194304] 0 2024-03-24T00:48:43.505 INFO:tasks.workunit.client.0.smithi038.stdout:6/274: creat d1/da/d1b/d1d/f4f x:0 0 0 2024-03-24T00:48:43.505 INFO:tasks.workunit.client.0.smithi038.stdout:9/286: dread d12/d28/d3a/f2b [0,4194304] 0 2024-03-24T00:48:43.505 INFO:tasks.workunit.client.0.smithi038.stdout:9/287: chown d12/d28/f4d 0 1 2024-03-24T00:48:43.510 INFO:tasks.workunit.client.0.smithi038.stdout:9/288: mknod d12/d1a/d5f/c65 0 2024-03-24T00:48:43.514 INFO:tasks.workunit.client.1.smithi088.stdout:2/206: dwrite db/d21/d35/f3d [0,4194304] 0 2024-03-24T00:48:43.514 INFO:tasks.workunit.client.1.smithi088.stdout:9/226: dwrite d1/f20 [0,4194304] 0 2024-03-24T00:48:43.514 INFO:tasks.workunit.client.1.smithi088.stdout:9/227: readlink d1/le 0 2024-03-24T00:48:43.514 INFO:tasks.workunit.client.1.smithi088.stdout:9/228: write d1/d15/d21/f3a [802921,117862] 0 2024-03-24T00:48:43.514 INFO:tasks.workunit.client.1.smithi088.stdout:9/229: creat d1/d10/f3e x:0 0 0 2024-03-24T00:48:43.514 INFO:tasks.workunit.client.0.smithi038.stdout:4/208: write da/d12/f2c [4067144,24686] 0 2024-03-24T00:48:43.518 INFO:tasks.workunit.client.0.smithi038.stdout:6/275: rename d1/da/d1b/d1d/l1e to d1/da/l50 0 2024-03-24T00:48:43.518 INFO:tasks.workunit.client.0.smithi038.stdout:6/276: chown d1/da/d1b/d1d/f2c 8287 1 2024-03-24T00:48:43.521 INFO:tasks.workunit.client.1.smithi088.stdout:0/264: dwrite d9/d20/f39 [0,4194304] 0 2024-03-24T00:48:43.531 INFO:tasks.workunit.client.1.smithi088.stdout:1/241: dwrite d6/d23/f33 [0,4194304] 0 2024-03-24T00:48:43.531 INFO:tasks.workunit.client.1.smithi088.stdout:1/242: chown d6/d27/c2e 677 1 2024-03-24T00:48:43.531 INFO:tasks.workunit.client.1.smithi088.stdout:4/273: dread d5/d16/f29 [0,4194304] 0 2024-03-24T00:48:43.531 INFO:tasks.workunit.client.1.smithi088.stdout:4/274: write d5/d7/dd/f21 [247398,13249] 0 2024-03-24T00:48:43.534 INFO:tasks.workunit.client.0.smithi038.stdout:0/198: dwrite d0/f5 [0,4194304] 0 2024-03-24T00:48:43.540 INFO:tasks.workunit.client.1.smithi088.stdout:9/230: creat d1/d22/f3f x:0 0 0 2024-03-24T00:48:43.542 INFO:tasks.workunit.client.0.smithi038.stdout:2/264: write d4/ff [3983968,86544] 0 2024-03-24T00:48:43.546 INFO:tasks.workunit.client.0.smithi038.stdout:9/289: rename d12/d28/d3a/c54 to d12/d55/d56/c66 0 2024-03-24T00:48:43.546 INFO:tasks.workunit.client.0.smithi038.stdout:9/290: chown d12/d28/l52 0 1 2024-03-24T00:48:43.549 INFO:tasks.workunit.client.0.smithi038.stdout:0/199: write d0/d12/f3a [520871,87747] 0 2024-03-24T00:48:43.549 INFO:tasks.workunit.client.0.smithi038.stdout:0/200: dread - d0/d12/d15/d1a/f2b zero size 2024-03-24T00:48:43.550 INFO:tasks.workunit.client.0.smithi038.stdout:6/277: mkdir d1/da/d1b/d25/d49/d4e/d51 0 2024-03-24T00:48:43.553 INFO:tasks.workunit.client.0.smithi038.stdout:6/278: read d1/da/f33 [209761,58441] 0 2024-03-24T00:48:43.554 INFO:tasks.workunit.client.0.smithi038.stdout:6/279: chown d1/d37/l40 82936 1 2024-03-24T00:48:43.554 INFO:tasks.workunit.client.0.smithi038.stdout:6/280: chown d1/da/d1b/d25/d49 32145265 1 2024-03-24T00:48:43.554 INFO:tasks.workunit.client.0.smithi038.stdout:6/281: dread - d1/da/d1b/d1d/f4f zero size 2024-03-24T00:48:43.554 INFO:tasks.workunit.client.0.smithi038.stdout:6/282: write d1/f21 [827936,35632] 0 2024-03-24T00:48:43.554 INFO:tasks.workunit.client.1.smithi088.stdout:3/210: dwrite d3/d6/f2b [0,4194304] 0 2024-03-24T00:48:43.555 INFO:tasks.workunit.client.0.smithi038.stdout:1/199: dwrite da/dd/d22/f25 [0,4194304] 0 2024-03-24T00:48:43.555 INFO:tasks.workunit.client.1.smithi088.stdout:9/231: write d1/f6 [2235567,30565] 0 2024-03-24T00:48:43.559 INFO:tasks.workunit.client.1.smithi088.stdout:0/265: link d9/d20/f35 d9/d20/f58 0 2024-03-24T00:48:43.560 INFO:tasks.workunit.client.1.smithi088.stdout:1/243: getdents d6/d43 0 2024-03-24T00:48:43.560 INFO:tasks.workunit.client.1.smithi088.stdout:5/229: dread d1/dd/f23 [0,4194304] 0 2024-03-24T00:48:43.561 INFO:tasks.workunit.client.1.smithi088.stdout:7/217: dwrite d3/d32/d34/f37 [0,4194304] 0 2024-03-24T00:48:43.564 INFO:tasks.workunit.client.0.smithi038.stdout:9/291: creat d12/d55/f67 x:0 0 0 2024-03-24T00:48:43.570 INFO:tasks.workunit.client.1.smithi088.stdout:4/275: rename d5/d13/d18/d24/d3d/l3f to d5/d16/d1a/d22/l5f 0 2024-03-24T00:48:43.577 INFO:tasks.workunit.client.0.smithi038.stdout:7/214: dwrite d2/d9/da/d23/f39 [0,4194304] 0 2024-03-24T00:48:43.578 INFO:tasks.workunit.client.0.smithi038.stdout:0/201: link d0/d12/d15/d1a/c26 d0/d12/d13/d28/c40 0 2024-03-24T00:48:43.578 INFO:tasks.workunit.client.0.smithi038.stdout:0/202: chown d0/d2d/f2f 403854885 1 2024-03-24T00:48:43.580 INFO:tasks.workunit.client.1.smithi088.stdout:3/211: dread d3/de/f16 [0,4194304] 0 2024-03-24T00:48:43.583 INFO:tasks.workunit.client.1.smithi088.stdout:2/207: rename db/d21/c31 to db/d21/c42 0 2024-03-24T00:48:43.583 INFO:tasks.workunit.client.1.smithi088.stdout:2/208: chown db/f38 62 1 2024-03-24T00:48:43.583 INFO:tasks.workunit.client.1.smithi088.stdout:2/209: creat db/f43 x:0 0 0 2024-03-24T00:48:43.589 INFO:tasks.workunit.client.1.smithi088.stdout:8/244: dwrite f1 [0,4194304] 0 2024-03-24T00:48:43.590 INFO:tasks.workunit.client.1.smithi088.stdout:0/266: mkdir d9/dc/de/d19/d44/d46/d59 0 2024-03-24T00:48:43.592 INFO:tasks.workunit.client.1.smithi088.stdout:1/244: symlink d6/d43/l47 0 2024-03-24T00:48:43.592 INFO:tasks.workunit.client.1.smithi088.stdout:1/245: chown d6/d23/f2d 397 1 2024-03-24T00:48:43.594 INFO:tasks.workunit.client.0.smithi038.stdout:4/209: dwrite da/d12/d37/dd/f3a [0,4194304] 0 2024-03-24T00:48:43.597 INFO:tasks.workunit.client.0.smithi038.stdout:6/283: creat d1/da/f52 x:0 0 0 2024-03-24T00:48:43.598 INFO:tasks.workunit.client.1.smithi088.stdout:8/245: dread d9/dc/f16 [0,4194304] 0 2024-03-24T00:48:43.599 INFO:tasks.workunit.client.1.smithi088.stdout:8/246: dread - d9/dc/d11/d24/f40 zero size 2024-03-24T00:48:43.599 INFO:tasks.workunit.client.1.smithi088.stdout:8/247: dread - d9/dc/d11/d24/d29/d39/f48 zero size 2024-03-24T00:48:43.600 INFO:tasks.workunit.client.0.smithi038.stdout:1/200: chown da/d3c/c20 0 1 2024-03-24T00:48:43.600 INFO:tasks.workunit.client.0.smithi038.stdout:1/201: getdents da/dd/d2c/d41 0 2024-03-24T00:48:43.609 INFO:tasks.workunit.client.1.smithi088.stdout:0/267: write d9/dc/f49 [1699266,3065] 0 2024-03-24T00:48:43.610 INFO:tasks.workunit.client.1.smithi088.stdout:0/268: fsync d9/dc/de/d19/d44/f4a 0 2024-03-24T00:48:43.610 INFO:tasks.workunit.client.1.smithi088.stdout:1/246: dread d6/d2f/f3d [0,4194304] 0 2024-03-24T00:48:43.613 INFO:tasks.workunit.client.0.smithi038.stdout:0/203: truncate d0/d12/d15/d1a/f1e 393040 0 2024-03-24T00:48:43.615 INFO:tasks.workunit.client.1.smithi088.stdout:7/218: chown d3/c18 288766 1 2024-03-24T00:48:43.615 INFO:tasks.workunit.client.1.smithi088.stdout:7/219: creat d3/d31/f47 x:0 0 0 2024-03-24T00:48:43.617 INFO:tasks.workunit.client.1.smithi088.stdout:4/276: truncate d5/d16/d1a/f3e 969506 0 2024-03-24T00:48:43.617 INFO:tasks.workunit.client.0.smithi038.stdout:5/198: rmdir d6/d16/d14 39 2024-03-24T00:48:43.622 INFO:tasks.workunit.client.1.smithi088.stdout:3/212: creat d3/d1a/d1e/d48/d4b/f4c x:0 0 0 2024-03-24T00:48:43.628 INFO:tasks.workunit.client.0.smithi038.stdout:9/292: dwrite d12/d55/f67 [0,4194304] 0 2024-03-24T00:48:43.631 INFO:tasks.workunit.client.0.smithi038.stdout:4/210: creat da/f3b x:0 0 0 2024-03-24T00:48:43.632 INFO:tasks.workunit.client.1.smithi088.stdout:8/248: chown d9/dc/d11/l1d 1 1 2024-03-24T00:48:43.635 INFO:tasks.workunit.client.0.smithi038.stdout:6/284: symlink d1/da/d1b/l53 0 2024-03-24T00:48:43.637 INFO:tasks.workunit.client.1.smithi088.stdout:0/269: unlink d9/dc/de/d19/c24 0 2024-03-24T00:48:43.638 INFO:tasks.workunit.client.0.smithi038.stdout:1/202: chown da/d3c/f2a 104592888 1 2024-03-24T00:48:43.638 INFO:tasks.workunit.client.0.smithi038.stdout:1/203: readlink l2 0 2024-03-24T00:48:43.638 INFO:tasks.workunit.client.0.smithi038.stdout:1/204: chown da/dd/d2c/d41 16978327 1 2024-03-24T00:48:43.639 INFO:tasks.workunit.client.0.smithi038.stdout:0/204: read d0/f5 [4063409,5201] 0 2024-03-24T00:48:43.640 INFO:tasks.workunit.client.1.smithi088.stdout:1/247: creat d6/d2f/f48 x:0 0 0 2024-03-24T00:48:43.644 INFO:tasks.workunit.client.0.smithi038.stdout:7/215: getdents d2/d9 0 2024-03-24T00:48:43.644 INFO:tasks.workunit.client.0.smithi038.stdout:7/216: write d2/d9/df/f35 [1033853,103939] 0 2024-03-24T00:48:43.644 INFO:tasks.workunit.client.0.smithi038.stdout:7/217: fsync d2/d9/da/f41 0 2024-03-24T00:48:43.646 INFO:tasks.workunit.client.0.smithi038.stdout:5/199: truncate d6/d16/f2a 4108284 0 2024-03-24T00:48:43.652 INFO:tasks.workunit.client.1.smithi088.stdout:4/277: unlink f1 0 2024-03-24T00:48:43.654 INFO:tasks.workunit.client.0.smithi038.stdout:4/211: truncate da/d12/d37/f22 3134009 0 2024-03-24T00:48:43.655 INFO:tasks.workunit.client.1.smithi088.stdout:3/213: mknod d3/d1a/d1e/d2f/d45/c4d 0 2024-03-24T00:48:43.657 INFO:tasks.workunit.client.0.smithi038.stdout:6/285: symlink d1/da/l54 0 2024-03-24T00:48:43.659 INFO:tasks.workunit.client.1.smithi088.stdout:2/210: link db/dd/f2f db/d21/f44 0 2024-03-24T00:48:43.659 INFO:tasks.workunit.client.1.smithi088.stdout:2/211: fdatasync f1 0 2024-03-24T00:48:43.659 INFO:tasks.workunit.client.1.smithi088.stdout:2/212: fsync db/dc/f1b 0 2024-03-24T00:48:43.660 INFO:tasks.workunit.client.0.smithi038.stdout:1/205: mknod da/c44 0 2024-03-24T00:48:43.662 INFO:tasks.workunit.client.1.smithi088.stdout:8/249: unlink d9/dc/l14 0 2024-03-24T00:48:43.662 INFO:tasks.workunit.client.1.smithi088.stdout:8/250: write f2 [1309776,47713] 0 2024-03-24T00:48:43.662 INFO:tasks.workunit.client.1.smithi088.stdout:5/230: truncate d1/f31 5179979 0 2024-03-24T00:48:43.666 INFO:tasks.workunit.client.1.smithi088.stdout:0/270: creat d9/d26/f5a x:0 0 0 2024-03-24T00:48:43.666 INFO:tasks.workunit.client.1.smithi088.stdout:0/271: creat d9/dc/de/d19/d44/f5b x:0 0 0 2024-03-24T00:48:43.666 INFO:tasks.workunit.client.1.smithi088.stdout:0/272: chown c6 40 1 2024-03-24T00:48:43.666 INFO:tasks.workunit.client.1.smithi088.stdout:0/273: write d9/dc/de/f40 [4379728,4182] 0 2024-03-24T00:48:43.667 INFO:tasks.workunit.client.0.smithi038.stdout:2/265: rmdir d4/d27/d42 0 2024-03-24T00:48:43.667 INFO:tasks.workunit.client.0.smithi038.stdout:2/266: write d4/f50 [171372,74337] 0 2024-03-24T00:48:43.668 INFO:tasks.workunit.client.0.smithi038.stdout:7/218: link d2/d9/da/dc/c17 d2/d9/c4e 0 2024-03-24T00:48:43.669 INFO:tasks.workunit.client.1.smithi088.stdout:1/248: fsync d6/d27/f3c 0 2024-03-24T00:48:43.672 INFO:tasks.workunit.client.0.smithi038.stdout:5/200: symlink d6/d16/d21/d2d/l3a 0 2024-03-24T00:48:43.672 INFO:tasks.workunit.client.0.smithi038.stdout:5/201: fdatasync d6/d16/d21/f2c 0 2024-03-24T00:48:43.679 INFO:tasks.workunit.client.1.smithi088.stdout:7/220: getdents d3 0 2024-03-24T00:48:43.679 INFO:tasks.workunit.client.1.smithi088.stdout:7/221: creat d3/d32/f48 x:0 0 0 2024-03-24T00:48:43.684 INFO:tasks.workunit.client.0.smithi038.stdout:4/212: creat da/d38/f3c x:0 0 0 2024-03-24T00:48:43.686 INFO:tasks.workunit.client.1.smithi088.stdout:4/278: rmdir d5 39 2024-03-24T00:48:43.688 INFO:tasks.workunit.client.0.smithi038.stdout:6/286: truncate d1/da/d1b/f2b 2339351 0 2024-03-24T00:48:43.688 INFO:tasks.workunit.client.0.smithi038.stdout:6/287: readlink d1/da/d1b/l42 0 2024-03-24T00:48:43.688 INFO:tasks.workunit.client.0.smithi038.stdout:6/288: fsync d1/da/f33 0 2024-03-24T00:48:43.690 INFO:tasks.workunit.client.1.smithi088.stdout:3/214: creat d3/de/df/d49/f4e x:0 0 0 2024-03-24T00:48:43.691 INFO:tasks.workunit.client.0.smithi038.stdout:7/219: mknod d2/d9/da/c4f 0 2024-03-24T00:48:43.691 INFO:tasks.workunit.client.0.smithi038.stdout:7/220: creat d2/d9/da/f50 x:0 0 0 2024-03-24T00:48:43.694 INFO:tasks.workunit.client.0.smithi038.stdout:9/293: dwrite d12/d28/d3a/f50 [4194304,4194304] 0 2024-03-24T00:48:43.695 INFO:tasks.workunit.client.1.smithi088.stdout:8/251: mkdir d9/dc/d11/d41/d54 0 2024-03-24T00:48:43.701 INFO:tasks.workunit.client.1.smithi088.stdout:5/231: stat d1/d29/c33 0 2024-03-24T00:48:43.702 INFO:tasks.workunit.client.1.smithi088.stdout:5/232: fsync d1/d19/f30 0 2024-03-24T00:48:43.702 INFO:tasks.workunit.client.1.smithi088.stdout:5/233: read d1/dd/f15 [2312105,103302] 0 2024-03-24T00:48:43.702 INFO:tasks.workunit.client.1.smithi088.stdout:5/234: chown d1/d10/d1b/c25 184287 1 2024-03-24T00:48:43.702 INFO:tasks.workunit.client.0.smithi038.stdout:1/206: dread da/dd/d10/f15 [4194304,4194304] 0 2024-03-24T00:48:43.702 INFO:tasks.workunit.client.0.smithi038.stdout:5/202: unlink d6/d1e/f27 0 2024-03-24T00:48:43.708 INFO:tasks.workunit.client.0.smithi038.stdout:0/205: dwrite d0/d12/d15/d1a/f1e [0,4194304] 0 2024-03-24T00:48:43.708 INFO:tasks.workunit.client.0.smithi038.stdout:0/206: fsync d0/fc 0 2024-03-24T00:48:43.709 INFO:tasks.workunit.client.1.smithi088.stdout:0/274: unlink d9/dc/de/c28 0 2024-03-24T00:48:43.714 INFO:tasks.workunit.client.0.smithi038.stdout:9/294: link d12/d55/c61 d12/d55/d56/c68 0 2024-03-24T00:48:43.717 INFO:tasks.workunit.client.1.smithi088.stdout:1/249: rename d6/d43/l47 to d6/d23/l49 0 2024-03-24T00:48:43.717 INFO:tasks.workunit.client.1.smithi088.stdout:1/250: truncate d6/d27/d1e/f40 837418 0 2024-03-24T00:48:43.717 INFO:tasks.workunit.client.1.smithi088.stdout:1/251: chown d6/d43/f45 13 1 2024-03-24T00:48:43.717 INFO:tasks.workunit.client.1.smithi088.stdout:1/252: write d6/d23/f34 [889007,51137] 0 2024-03-24T00:48:43.717 INFO:tasks.workunit.client.1.smithi088.stdout:1/253: truncate d6/d23/f2d 284700 0 2024-03-24T00:48:43.717 INFO:tasks.workunit.client.0.smithi038.stdout:6/289: write d1/da/d1b/d25/f2a [537976,19788] 0 2024-03-24T00:48:43.719 INFO:tasks.workunit.client.0.smithi038.stdout:1/207: symlink da/dd/d2c/d40/l45 0 2024-03-24T00:48:43.723 INFO:tasks.workunit.client.0.smithi038.stdout:5/203: rename d6/d16/d21/f35 to d6/d23/f3b 0 2024-03-24T00:48:43.725 INFO:tasks.workunit.client.1.smithi088.stdout:4/279: mkdir d5/d7/d15/d60 0 2024-03-24T00:48:43.730 INFO:tasks.workunit.client.0.smithi038.stdout:0/207: link d0/d2d/c3e d0/d14/c41 0 2024-03-24T00:48:43.731 INFO:tasks.workunit.client.1.smithi088.stdout:6/273: sync 2024-03-24T00:48:43.732 INFO:tasks.workunit.client.1.smithi088.stdout:9/232: sync 2024-03-24T00:48:43.732 INFO:tasks.workunit.client.1.smithi088.stdout:9/233: truncate d1/d22/f3f 200913 0 2024-03-24T00:48:43.734 INFO:tasks.workunit.client.1.smithi088.stdout:3/215: unlink d3/d6/fb 0 2024-03-24T00:48:43.735 INFO:tasks.workunit.client.1.smithi088.stdout:0/275: dread d9/d20/f39 [0,4194304] 0 2024-03-24T00:48:43.736 INFO:tasks.workunit.client.1.smithi088.stdout:5/235: unlink d1/d10/d1b/c3c 0 2024-03-24T00:48:43.736 INFO:tasks.workunit.client.0.smithi038.stdout:1/208: truncate f4 7998659 0 2024-03-24T00:48:43.737 INFO:tasks.workunit.client.1.smithi088.stdout:1/254: truncate d6/d27/f1d 583606 0 2024-03-24T00:48:43.737 INFO:tasks.workunit.client.1.smithi088.stdout:1/255: chown f2 1 1 2024-03-24T00:48:43.739 INFO:tasks.workunit.client.0.smithi038.stdout:6/290: rename d1/da/d1b/d1d/f39 to d1/da/d1b/d25/d49/d4e/f55 0 2024-03-24T00:48:43.742 INFO:tasks.workunit.client.0.smithi038.stdout:5/204: mkdir d6/d16/d3c 0 2024-03-24T00:48:43.743 INFO:tasks.workunit.client.1.smithi088.stdout:3/216: mknod d3/de/df/d49/c4f 0 2024-03-24T00:48:43.744 INFO:tasks.workunit.client.1.smithi088.stdout:3/217: write d3/d1a/f38 [65773,27184] 0 2024-03-24T00:48:43.744 INFO:tasks.workunit.client.0.smithi038.stdout:0/208: mkdir d0/d12/d42 0 2024-03-24T00:48:43.745 INFO:tasks.workunit.client.1.smithi088.stdout:5/236: symlink d1/d10/l4b 0 2024-03-24T00:48:43.747 INFO:tasks.workunit.client.0.smithi038.stdout:6/291: mknod d1/da/d1b/d25/d49/d4e/c56 0 2024-03-24T00:48:43.752 INFO:tasks.workunit.client.1.smithi088.stdout:1/256: unlink d6/d27/c2e 0 2024-03-24T00:48:43.752 INFO:tasks.workunit.client.1.smithi088.stdout:1/257: chown f2 0 1 2024-03-24T00:48:43.754 INFO:tasks.workunit.client.1.smithi088.stdout:3/218: creat d3/d1a/d1e/d48/f50 x:0 0 0 2024-03-24T00:48:43.756 INFO:tasks.workunit.client.0.smithi038.stdout:5/205: truncate d6/f18 1845024 0 2024-03-24T00:48:43.756 INFO:tasks.workunit.client.0.smithi038.stdout:6/292: mknod d1/da/d1b/d25/d49/c57 0 2024-03-24T00:48:43.757 INFO:tasks.workunit.client.0.smithi038.stdout:6/293: write d1/f6 [4689267,712] 0 2024-03-24T00:48:43.763 INFO:tasks.workunit.client.1.smithi088.stdout:1/258: creat d6/d23/f4a x:0 0 0 2024-03-24T00:48:43.763 INFO:tasks.workunit.client.1.smithi088.stdout:1/259: chown d6/d23/f29 0 1 2024-03-24T00:48:43.763 INFO:tasks.workunit.client.1.smithi088.stdout:1/260: chown d6/d27/d1e/f39 81374 1 2024-03-24T00:48:43.763 INFO:tasks.workunit.client.1.smithi088.stdout:1/261: dread - d6/d23/f29 zero size 2024-03-24T00:48:43.764 INFO:tasks.workunit.client.1.smithi088.stdout:3/219: dread d3/d1a/f37 [0,4194304] 0 2024-03-24T00:48:43.764 INFO:tasks.workunit.client.1.smithi088.stdout:5/237: getdents d1/d19/d1e 0 2024-03-24T00:48:43.776 INFO:tasks.workunit.client.1.smithi088.stdout:3/220: symlink d3/d32/l51 0 2024-03-24T00:48:43.778 INFO:tasks.workunit.client.1.smithi088.stdout:7/222: dwrite d3/d31/f38 [0,4194304] 0 2024-03-24T00:48:43.778 INFO:tasks.workunit.client.1.smithi088.stdout:1/262: rmdir d6/d2f 39 2024-03-24T00:48:43.790 INFO:tasks.workunit.client.1.smithi088.stdout:7/223: symlink d3/d32/l49 0 2024-03-24T00:48:43.794 INFO:tasks.workunit.client.0.smithi038.stdout:6/294: write d1/da/d1b/d25/d49/f4b [422633,45998] 0 2024-03-24T00:48:43.794 INFO:tasks.workunit.client.0.smithi038.stdout:5/206: write d6/f9 [3050550,29216] 0 2024-03-24T00:48:43.795 INFO:tasks.workunit.client.0.smithi038.stdout:6/295: symlink d1/da/d1b/d25/l58 0 2024-03-24T00:48:43.796 INFO:tasks.workunit.client.0.smithi038.stdout:9/295: dwrite d12/d1a/f3c [0,4194304] 0 2024-03-24T00:48:43.799 INFO:tasks.workunit.client.0.smithi038.stdout:6/296: write d1/da/f14 [1240512,69316] 0 2024-03-24T00:48:43.799 INFO:tasks.workunit.client.0.smithi038.stdout:6/297: dread - d1/da/f34 zero size 2024-03-24T00:48:43.802 INFO:tasks.workunit.client.0.smithi038.stdout:6/298: rename d1/da/d1b/d25/d49/f4b to d1/da/d1b/d2f/f59 0 2024-03-24T00:48:43.807 INFO:tasks.workunit.client.1.smithi088.stdout:9/234: dwrite d1/d10/f1e [0,4194304] 0 2024-03-24T00:48:43.807 INFO:tasks.workunit.client.1.smithi088.stdout:9/235: chown d1/f8 38846 1 2024-03-24T00:48:43.808 INFO:tasks.workunit.client.1.smithi088.stdout:7/224: dread d3/f1d [0,4194304] 0 2024-03-24T00:48:43.808 INFO:tasks.workunit.client.1.smithi088.stdout:4/280: dwrite d5/d13/d18/d24/f30 [0,4194304] 0 2024-03-24T00:48:43.808 INFO:tasks.workunit.client.1.smithi088.stdout:4/281: chown d5/d13 47903596 1 2024-03-24T00:48:43.808 INFO:tasks.workunit.client.1.smithi088.stdout:7/225: truncate d3/d32/d3f/f40 421227 0 2024-03-24T00:48:43.810 INFO:tasks.workunit.client.1.smithi088.stdout:7/226: link d3/c25 d3/d32/d3f/c4a 0 2024-03-24T00:48:43.810 INFO:tasks.workunit.client.1.smithi088.stdout:7/227: fsync d3/d31/f38 0 2024-03-24T00:48:43.811 INFO:tasks.workunit.client.1.smithi088.stdout:7/228: mkdir d3/d32/d3f/d4b 0 2024-03-24T00:48:43.819 INFO:tasks.workunit.client.0.smithi038.stdout:0/209: dwrite d0/d12/f3a [0,4194304] 0 2024-03-24T00:48:43.821 INFO:tasks.workunit.client.0.smithi038.stdout:0/210: creat d0/f43 x:0 0 0 2024-03-24T00:48:43.821 INFO:tasks.workunit.client.0.smithi038.stdout:0/211: readlink d0/l29 0 2024-03-24T00:48:43.824 INFO:tasks.workunit.client.1.smithi088.stdout:9/236: dread d1/d10/d11/f17 [0,4194304] 0 2024-03-24T00:48:43.853 INFO:tasks.workunit.client.0.smithi038.stdout:7/221: dread d2/d9/f20 [0,4194304] 0 2024-03-24T00:48:43.854 INFO:tasks.workunit.client.0.smithi038.stdout:9/296: write f8 [1140309,110785] 0 2024-03-24T00:48:43.854 INFO:tasks.workunit.client.0.smithi038.stdout:9/297: truncate d12/d28/d3a/f4b 332811 0 2024-03-24T00:48:43.857 INFO:tasks.workunit.client.0.smithi038.stdout:9/298: rmdir d12/d28/d3a/d26 39 2024-03-24T00:48:43.862 INFO:tasks.workunit.client.0.smithi038.stdout:9/299: read f4 [4934224,85293] 0 2024-03-24T00:48:43.862 INFO:tasks.workunit.client.0.smithi038.stdout:2/267: dwrite d4/f1b [0,4194304] 0 2024-03-24T00:48:43.864 INFO:tasks.workunit.client.1.smithi088.stdout:6/274: dwrite db/d1d/d23/d3c/d43/f39 [0,4194304] 0 2024-03-24T00:48:43.864 INFO:tasks.workunit.client.1.smithi088.stdout:5/238: write d1/d10/d1b/d2f/f38 [2927196,125430] 0 2024-03-24T00:48:43.865 INFO:tasks.workunit.client.1.smithi088.stdout:5/239: write d1/dd/f15 [2195729,127036] 0 2024-03-24T00:48:43.865 INFO:tasks.workunit.client.1.smithi088.stdout:2/213: dwrite db/f3c [0,4194304] 0 2024-03-24T00:48:43.865 INFO:tasks.workunit.client.1.smithi088.stdout:2/214: chown db/dc/l26 246174 1 2024-03-24T00:48:43.878 INFO:tasks.workunit.client.0.smithi038.stdout:7/222: write d2/d9/da/dc/f48 [2798227,20070] 0 2024-03-24T00:48:43.882 INFO:tasks.workunit.client.0.smithi038.stdout:1/209: dwrite da/d3c/f17 [0,4194304] 0 2024-03-24T00:48:43.883 INFO:tasks.workunit.client.1.smithi088.stdout:5/240: dread d1/dd/f23 [0,4194304] 0 2024-03-24T00:48:43.885 INFO:tasks.workunit.client.0.smithi038.stdout:5/207: dwrite d6/d23/f3b [0,4194304] 0 2024-03-24T00:48:43.887 INFO:tasks.workunit.client.1.smithi088.stdout:0/276: dwrite d9/f55 [0,4194304] 0 2024-03-24T00:48:43.903 INFO:tasks.workunit.client.1.smithi088.stdout:9/237: write d1/f20 [1722824,87889] 0 2024-03-24T00:48:43.904 INFO:tasks.workunit.client.1.smithi088.stdout:7/229: dwrite d3/d31/f47 [0,4194304] 0 2024-03-24T00:48:43.908 INFO:tasks.workunit.client.0.smithi038.stdout:2/268: symlink d4/d18/d38/l5c 0 2024-03-24T00:48:43.911 INFO:tasks.workunit.client.0.smithi038.stdout:7/223: getdents d2 0 2024-03-24T00:48:43.922 INFO:tasks.workunit.client.1.smithi088.stdout:9/238: write d1/d10/f1e [2197618,77127] 0 2024-03-24T00:48:43.926 INFO:tasks.workunit.client.1.smithi088.stdout:5/241: symlink d1/d19/d1e/l4c 0 2024-03-24T00:48:43.926 INFO:tasks.workunit.client.1.smithi088.stdout:5/242: dread - d1/d29/d3f/f41 zero size 2024-03-24T00:48:43.926 INFO:tasks.workunit.client.1.smithi088.stdout:5/243: readlink d1/d10/l4b 0 2024-03-24T00:48:43.932 INFO:tasks.workunit.client.0.smithi038.stdout:2/269: symlink d4/d27/d54/l5d 0 2024-03-24T00:48:43.934 INFO:tasks.workunit.client.1.smithi088.stdout:3/221: rmdir d3 39 2024-03-24T00:48:43.934 INFO:tasks.workunit.client.1.smithi088.stdout:3/222: write d3/d6/d27/d3c/f42 [1098052,21366] 0 2024-03-24T00:48:43.934 INFO:tasks.workunit.client.1.smithi088.stdout:3/223: fsync d3/de/f17 0 2024-03-24T00:48:43.934 INFO:tasks.workunit.client.1.smithi088.stdout:3/224: truncate d3/d6/d27/d3c/f39 373089 0 2024-03-24T00:48:43.935 INFO:tasks.workunit.client.0.smithi038.stdout:2/270: write d4/d27/d2f/d30/d3e/f58 [1041346,96562] 0 2024-03-24T00:48:43.937 INFO:tasks.workunit.client.1.smithi088.stdout:6/275: rmdir db 39 2024-03-24T00:48:43.937 INFO:tasks.workunit.client.1.smithi088.stdout:6/276: stat db/d1d/d23/d3c/d43/d24/c4a 0 2024-03-24T00:48:43.937 INFO:tasks.workunit.client.1.smithi088.stdout:6/277: chown db/l27 2881 1 2024-03-24T00:48:43.938 INFO:tasks.workunit.client.0.smithi038.stdout:2/271: mknod d4/d18/d26/d37/c5e 0 2024-03-24T00:48:43.941 INFO:tasks.workunit.client.1.smithi088.stdout:2/215: dread db/d21/f2e [0,4194304] 0 2024-03-24T00:48:43.945 INFO:tasks.workunit.client.0.smithi038.stdout:2/272: rename d4/f1d to d4/d27/d54/d53/f5f 0 2024-03-24T00:48:43.947 INFO:tasks.workunit.client.0.smithi038.stdout:0/212: rmdir d0 39 2024-03-24T00:48:43.949 INFO:tasks.workunit.client.0.smithi038.stdout:2/273: link d4/d27/d2f/d30/d3e/l4d d4/d27/d2f/d30/l60 0 2024-03-24T00:48:43.950 INFO:tasks.workunit.client.0.smithi038.stdout:2/274: creat d4/d27/d2f/d30/d3e/f61 x:0 0 0 2024-03-24T00:48:43.950 INFO:tasks.workunit.client.0.smithi038.stdout:5/208: write d6/d23/f3b [4080322,66336] 0 2024-03-24T00:48:43.952 INFO:tasks.workunit.client.0.smithi038.stdout:0/213: mkdir d0/d12/d42/d44 0 2024-03-24T00:48:43.959 INFO:tasks.workunit.client.1.smithi088.stdout:7/230: creat d3/d32/d3f/d4b/f4c x:0 0 0 2024-03-24T00:48:43.959 INFO:tasks.workunit.client.1.smithi088.stdout:7/231: readlink d3/l13 0 2024-03-24T00:48:43.959 INFO:tasks.workunit.client.0.smithi038.stdout:2/275: creat d4/d18/d26/f62 x:0 0 0 2024-03-24T00:48:43.963 INFO:tasks.workunit.client.1.smithi088.stdout:4/282: unlink d5/d7/dd/ff 0 2024-03-24T00:48:43.967 INFO:tasks.workunit.client.0.smithi038.stdout:5/209: mknod d6/d19/d20/c3d 0 2024-03-24T00:48:43.969 INFO:tasks.workunit.client.0.smithi038.stdout:0/214: mknod d0/d1b/c45 0 2024-03-24T00:48:43.970 INFO:tasks.workunit.client.1.smithi088.stdout:5/244: mknod d1/d29/d3f/c4d 0 2024-03-24T00:48:43.970 INFO:tasks.workunit.client.1.smithi088.stdout:5/245: write d1/d19/d1e/f47 [102372,20868] 0 2024-03-24T00:48:43.972 INFO:tasks.workunit.client.0.smithi038.stdout:5/210: mknod d6/d16/d21/c3e 0 2024-03-24T00:48:43.976 INFO:tasks.workunit.client.1.smithi088.stdout:6/278: creat db/d1d/d23/d3c/d43/d24/f51 x:0 0 0 2024-03-24T00:48:43.978 INFO:tasks.workunit.client.0.smithi038.stdout:8/208: sync 2024-03-24T00:48:43.981 INFO:tasks.workunit.client.1.smithi088.stdout:3/225: dread d3/d1a/f38 [0,4194304] 0 2024-03-24T00:48:43.982 INFO:tasks.workunit.client.0.smithi038.stdout:3/247: sync 2024-03-24T00:48:43.983 INFO:tasks.workunit.client.1.smithi088.stdout:5/246: mkdir d1/d43/d4e 0 2024-03-24T00:48:43.987 INFO:tasks.workunit.client.0.smithi038.stdout:5/211: rename d6/d16/d21/f2c to d6/f3f 0 2024-03-24T00:48:43.987 INFO:tasks.workunit.client.0.smithi038.stdout:5/212: chown d6/f3f 6031 1 2024-03-24T00:48:43.989 INFO:tasks.workunit.client.1.smithi088.stdout:4/283: dread d5/d17/f33 [0,4194304] 0 2024-03-24T00:48:43.993 INFO:tasks.workunit.client.0.smithi038.stdout:8/209: symlink d0/d1d/d27/l4b 0 2024-03-24T00:48:43.993 INFO:tasks.workunit.client.1.smithi088.stdout:6/279: dread db/f19 [0,4194304] 0 2024-03-24T00:48:43.995 INFO:tasks.workunit.client.0.smithi038.stdout:7/224: dwrite d2/d9/da/dc/f1c [0,4194304] 0 2024-03-24T00:48:43.996 INFO:tasks.workunit.client.0.smithi038.stdout:8/210: truncate d0/d15/f17 185728 0 2024-03-24T00:48:43.997 INFO:tasks.workunit.client.1.smithi088.stdout:2/216: rename db/dc/f11 to db/d21/f45 0 2024-03-24T00:48:43.998 INFO:tasks.workunit.client.0.smithi038.stdout:2/276: dread d4/d27/d54/d53/f5f [0,4194304] 0 2024-03-24T00:48:44.000 INFO:tasks.workunit.client.1.smithi088.stdout:4/284: symlink d5/d13/d18/l61 0 2024-03-24T00:48:44.003 INFO:tasks.workunit.client.0.smithi038.stdout:7/225: rename d2/d9/da/dc/c43 to d2/d9/df/c51 0 2024-03-24T00:48:44.003 INFO:tasks.workunit.client.0.smithi038.stdout:7/226: read d2/f13 [160989,23035] 0 2024-03-24T00:48:44.005 INFO:tasks.workunit.client.0.smithi038.stdout:8/211: mknod d0/d15/d2c/c4c 0 2024-03-24T00:48:44.005 INFO:tasks.workunit.client.1.smithi088.stdout:6/280: symlink db/d1d/d50/l52 0 2024-03-24T00:48:44.005 INFO:tasks.workunit.client.1.smithi088.stdout:3/226: creat d3/d6/d19/f52 x:0 0 0 2024-03-24T00:48:44.007 INFO:tasks.workunit.client.0.smithi038.stdout:2/277: mkdir d4/d18/d38/d63 0 2024-03-24T00:48:44.010 INFO:tasks.workunit.client.0.smithi038.stdout:9/300: dwrite f8 [0,4194304] 0 2024-03-24T00:48:44.010 INFO:tasks.workunit.client.0.smithi038.stdout:9/301: creat d12/f69 x:0 0 0 2024-03-24T00:48:44.010 INFO:tasks.workunit.client.0.smithi038.stdout:9/302: creat d12/d41/f6a x:0 0 0 2024-03-24T00:48:44.010 INFO:tasks.workunit.client.0.smithi038.stdout:7/227: symlink d2/d9/da/dc/d4a/l52 0 2024-03-24T00:48:44.012 INFO:tasks.workunit.client.1.smithi088.stdout:3/227: fdatasync d3/d6/d27/f40 0 2024-03-24T00:48:44.012 INFO:tasks.workunit.client.1.smithi088.stdout:3/228: write d3/de/f17 [5037639,14462] 0 2024-03-24T00:48:44.013 INFO:tasks.workunit.client.0.smithi038.stdout:7/228: truncate d2/d9/da/f47 74724 0 2024-03-24T00:48:44.016 INFO:tasks.workunit.client.1.smithi088.stdout:7/232: dwrite d3/d32/f43 [0,4194304] 0 2024-03-24T00:48:44.019 INFO:tasks.workunit.client.0.smithi038.stdout:5/213: dread d6/f26 [0,4194304] 0 2024-03-24T00:48:44.020 INFO:tasks.workunit.client.0.smithi038.stdout:8/212: mkdir d0/d1d/d27/d4d 0 2024-03-24T00:48:44.021 INFO:tasks.workunit.client.1.smithi088.stdout:8/252: sync 2024-03-24T00:48:44.021 INFO:tasks.workunit.client.1.smithi088.stdout:3/229: rmdir d3/d1a/d1e/d2f/d45 39 2024-03-24T00:48:44.024 INFO:tasks.workunit.client.0.smithi038.stdout:2/278: creat d4/d27/d54/d53/f64 x:0 0 0 2024-03-24T00:48:44.025 INFO:tasks.workunit.client.1.smithi088.stdout:1/263: sync 2024-03-24T00:48:44.025 INFO:tasks.workunit.client.1.smithi088.stdout:1/264: write d6/d27/f1d [1178533,68536] 0 2024-03-24T00:48:44.025 INFO:tasks.workunit.client.1.smithi088.stdout:1/265: chown d6/d23/l3b 2084 1 2024-03-24T00:48:44.028 INFO:tasks.workunit.client.0.smithi038.stdout:0/215: dwrite d0/f2 [0,4194304] 0 2024-03-24T00:48:44.035 INFO:tasks.workunit.client.1.smithi088.stdout:9/239: dwrite d1/f7 [0,4194304] 0 2024-03-24T00:48:44.038 INFO:tasks.workunit.client.1.smithi088.stdout:8/253: creat d9/dc/d11/d24/d43/d4a/f55 x:0 0 0 2024-03-24T00:48:44.038 INFO:tasks.workunit.client.1.smithi088.stdout:8/254: chown d9/dc/d11/d24/c2d 42883 1 2024-03-24T00:48:44.043 INFO:tasks.workunit.client.0.smithi038.stdout:9/303: truncate fc 750156 0 2024-03-24T00:48:44.045 INFO:tasks.workunit.client.0.smithi038.stdout:7/229: link d2/d9/da/f41 d2/d9/d16/f53 0 2024-03-24T00:48:44.045 INFO:tasks.workunit.client.0.smithi038.stdout:7/230: write d2/f13 [1553044,28627] 0 2024-03-24T00:48:44.046 INFO:tasks.workunit.client.0.smithi038.stdout:7/231: dread - d2/d9/df/d33/f31 zero size 2024-03-24T00:48:44.046 INFO:tasks.workunit.client.0.smithi038.stdout:7/232: fsync d2/d2c/f4d 0 2024-03-24T00:48:44.049 INFO:tasks.workunit.client.1.smithi088.stdout:9/240: rename d1/cc to d1/d10/d11/c40 0 2024-03-24T00:48:44.050 INFO:tasks.workunit.client.0.smithi038.stdout:5/214: symlink d6/d16/d21/d2d/l40 0 2024-03-24T00:48:44.050 INFO:tasks.workunit.client.0.smithi038.stdout:5/215: chown d6/f9 9 1 2024-03-24T00:48:44.052 INFO:tasks.workunit.client.1.smithi088.stdout:8/255: mkdir d9/dc/d11/d41/d56 0 2024-03-24T00:48:44.052 INFO:tasks.workunit.client.1.smithi088.stdout:8/256: write d9/dc/d11/d24/f45 [992475,53002] 0 2024-03-24T00:48:44.055 INFO:tasks.workunit.client.0.smithi038.stdout:0/216: mknod d0/c46 0 2024-03-24T00:48:44.055 INFO:tasks.workunit.client.1.smithi088.stdout:8/257: creat d9/f57 x:0 0 0 2024-03-24T00:48:44.055 INFO:tasks.workunit.client.1.smithi088.stdout:8/258: write d9/dc/d11/d24/f40 [326337,30156] 0 2024-03-24T00:48:44.055 INFO:tasks.workunit.client.1.smithi088.stdout:8/259: creat d9/dc/d11/d24/f58 x:0 0 0 2024-03-24T00:48:44.056 INFO:tasks.workunit.client.1.smithi088.stdout:8/260: write d9/f3e [1067713,62689] 0 2024-03-24T00:48:44.056 INFO:tasks.workunit.client.0.smithi038.stdout:0/217: write d0/f7 [4287169,98423] 0 2024-03-24T00:48:44.067 INFO:tasks.workunit.client.0.smithi038.stdout:7/233: link d2/d9/da/dc/fd d2/d9/f54 0 2024-03-24T00:48:44.067 INFO:tasks.workunit.client.0.smithi038.stdout:7/234: dread - d2/d9/da/f41 zero size 2024-03-24T00:48:44.072 INFO:tasks.workunit.client.1.smithi088.stdout:8/261: creat d9/dc/d11/d41/d56/f59 x:0 0 0 2024-03-24T00:48:44.073 INFO:tasks.workunit.client.0.smithi038.stdout:9/304: dread d12/d1a/f20 [0,4194304] 0 2024-03-24T00:48:44.076 INFO:tasks.workunit.client.1.smithi088.stdout:8/262: symlink d9/dc/d11/d41/l5a 0 2024-03-24T00:48:44.076 INFO:tasks.workunit.client.1.smithi088.stdout:8/263: truncate d9/dc/d11/d24/d29/d39/f48 510899 0 2024-03-24T00:48:44.085 INFO:tasks.workunit.client.0.smithi038.stdout:0/218: mkdir d0/d1b/d2c/d47 0 2024-03-24T00:48:44.086 INFO:tasks.workunit.client.1.smithi088.stdout:4/285: dwrite d5/d16/d1a/f1e [0,4194304] 0 2024-03-24T00:48:44.086 INFO:tasks.workunit.client.1.smithi088.stdout:4/286: getdents d5/d16/d1a/d22/d2f 0 2024-03-24T00:48:44.086 INFO:tasks.workunit.client.1.smithi088.stdout:4/287: fsync d5/d13/d18/d24/d3d/f42 0 2024-03-24T00:48:44.086 INFO:tasks.workunit.client.1.smithi088.stdout:4/288: fsync d5/fc 0 2024-03-24T00:48:44.086 INFO:tasks.workunit.client.1.smithi088.stdout:4/289: dread - d5/d17/f2a zero size 2024-03-24T00:48:44.088 INFO:tasks.workunit.client.0.smithi038.stdout:7/235: link d2/d9/f20 d2/d2c/f55 0 2024-03-24T00:48:44.088 INFO:tasks.workunit.client.0.smithi038.stdout:7/236: chown d2/d9/df/l25 7806 1 2024-03-24T00:48:44.088 INFO:tasks.workunit.client.0.smithi038.stdout:7/237: creat d2/d2c/f56 x:0 0 0 2024-03-24T00:48:44.088 INFO:tasks.workunit.client.0.smithi038.stdout:7/238: fsync d2/d9/da/d23/f39 0 2024-03-24T00:48:44.088 INFO:tasks.workunit.client.0.smithi038.stdout:7/239: readlink d2/d9/d16/l1e 0 2024-03-24T00:48:44.089 INFO:tasks.workunit.client.0.smithi038.stdout:5/216: mknod d6/c41 0 2024-03-24T00:48:44.089 INFO:tasks.workunit.client.0.smithi038.stdout:5/217: dread - d6/f3f zero size 2024-03-24T00:48:44.091 INFO:tasks.workunit.client.0.smithi038.stdout:8/213: dwrite d0/f24 [0,4194304] 0 2024-03-24T00:48:44.092 INFO:tasks.workunit.client.0.smithi038.stdout:0/219: rename d0/d1b/d2c/f31 to d0/d12/d13/d28/f48 0 2024-03-24T00:48:44.092 INFO:tasks.workunit.client.0.smithi038.stdout:0/220: chown d0/l10 12278251 1 2024-03-24T00:48:44.092 INFO:tasks.workunit.client.0.smithi038.stdout:0/221: chown d0/l10 3769 1 2024-03-24T00:48:44.092 INFO:tasks.workunit.client.1.smithi088.stdout:4/290: getdents d5/d7 0 2024-03-24T00:48:44.095 INFO:tasks.workunit.client.1.smithi088.stdout:8/264: rename d9/dc/d11/d1b to d9/dc/d11/d24/d29/d5b 0 2024-03-24T00:48:44.096 INFO:tasks.workunit.client.1.smithi088.stdout:3/230: rmdir d3/d6/d19 39 2024-03-24T00:48:44.099 INFO:tasks.workunit.client.1.smithi088.stdout:6/281: dwrite db/d1d/d23/d3c/d43/f34 [4194304,4194304] 0 2024-03-24T00:48:44.099 INFO:tasks.workunit.client.1.smithi088.stdout:6/282: write db/f13 [3415541,58004] 0 2024-03-24T00:48:44.099 INFO:tasks.workunit.client.1.smithi088.stdout:6/283: chown db/c1a 124448 1 2024-03-24T00:48:44.099 INFO:tasks.workunit.client.1.smithi088.stdout:5/247: dwrite d1/d19/f2c [0,4194304] 0 2024-03-24T00:48:44.099 INFO:tasks.workunit.client.1.smithi088.stdout:5/248: write d1/d10/f17 [2490873,72898] 0 2024-03-24T00:48:44.099 INFO:tasks.workunit.client.1.smithi088.stdout:5/249: write d1/f3b [2347201,105820] 0 2024-03-24T00:48:44.099 INFO:tasks.workunit.client.1.smithi088.stdout:5/250: chown d1/d10/d1b/d2e/c36 3952 1 2024-03-24T00:48:44.100 INFO:tasks.workunit.client.0.smithi038.stdout:2/279: rmdir d4/d18/d38 39 2024-03-24T00:48:44.100 INFO:tasks.workunit.client.0.smithi038.stdout:2/280: stat d4/d18/l1a 0 2024-03-24T00:48:44.100 INFO:tasks.workunit.client.1.smithi088.stdout:2/217: dwrite db/dd/f28 [0,4194304] 0 2024-03-24T00:48:44.100 INFO:tasks.workunit.client.1.smithi088.stdout:4/291: creat d5/f62 x:0 0 0 2024-03-24T00:48:44.102 INFO:tasks.workunit.client.1.smithi088.stdout:9/241: dwrite d1/d15/f37 [0,4194304] 0 2024-03-24T00:48:44.102 INFO:tasks.workunit.client.1.smithi088.stdout:7/233: dwrite d3/ff [0,4194304] 0 2024-03-24T00:48:44.103 INFO:tasks.workunit.client.0.smithi038.stdout:5/218: mkdir d6/d19/d20/d42 0 2024-03-24T00:48:44.107 INFO:tasks.workunit.client.1.smithi088.stdout:1/266: dwrite d6/f17 [0,4194304] 0 2024-03-24T00:48:44.107 INFO:tasks.workunit.client.1.smithi088.stdout:1/267: fdatasync d6/d23/f4a 0 2024-03-24T00:48:44.107 INFO:tasks.workunit.client.1.smithi088.stdout:1/268: creat d6/d27/d1e/d3f/f4b x:0 0 0 2024-03-24T00:48:44.107 INFO:tasks.workunit.client.1.smithi088.stdout:1/269: creat d6/d27/d1e/d3f/f4c x:0 0 0 2024-03-24T00:48:44.107 INFO:tasks.workunit.client.1.smithi088.stdout:1/270: stat l4 0 2024-03-24T00:48:44.107 INFO:tasks.workunit.client.1.smithi088.stdout:1/271: chown d6/d27/d1e/f40 732 1 2024-03-24T00:48:44.107 INFO:tasks.workunit.client.1.smithi088.stdout:1/272: chown d6/d2f/l38 80 1 2024-03-24T00:48:44.112 INFO:tasks.workunit.client.1.smithi088.stdout:6/284: symlink db/d1d/d23/l53 0 2024-03-24T00:48:44.112 INFO:tasks.workunit.client.1.smithi088.stdout:6/285: truncate db/d1d/d23/d3c/d43/d24/f30 109546 0 2024-03-24T00:48:44.112 INFO:tasks.workunit.client.1.smithi088.stdout:6/286: chown db/dd/f41 12311 1 2024-03-24T00:48:44.117 INFO:tasks.workunit.client.0.smithi038.stdout:0/222: rmdir d0/d1b/d2c/d47 0 2024-03-24T00:48:44.117 INFO:tasks.workunit.client.0.smithi038.stdout:0/223: dread - d0/d12/d15/f22 zero size 2024-03-24T00:48:44.121 INFO:tasks.workunit.client.1.smithi088.stdout:5/251: mkdir d1/d10/d4f 0 2024-03-24T00:48:44.121 INFO:tasks.workunit.client.1.smithi088.stdout:5/252: chown d1/d19/d1e/c32 27570164 1 2024-03-24T00:48:44.132 INFO:tasks.workunit.client.0.smithi038.stdout:9/305: dread d12/d28/d3a/d26/f3d [0,4194304] 0 2024-03-24T00:48:44.132 INFO:tasks.workunit.client.0.smithi038.stdout:9/306: write d12/d28/d3a/d26/f3d [1019620,102705] 0 2024-03-24T00:48:44.142 INFO:tasks.workunit.client.0.smithi038.stdout:7/240: rmdir d2/d9/da/dc 39 2024-03-24T00:48:44.143 INFO:tasks.workunit.client.0.smithi038.stdout:7/241: creat d2/d2c/f57 x:0 0 0 2024-03-24T00:48:44.143 INFO:tasks.workunit.client.0.smithi038.stdout:7/242: chown d2/d2c/f57 12 1 2024-03-24T00:48:44.143 INFO:tasks.workunit.client.0.smithi038.stdout:0/224: dread d0/d12/f3a [0,4194304] 0 2024-03-24T00:48:44.145 INFO:tasks.workunit.client.0.smithi038.stdout:8/214: rmdir d0 39 2024-03-24T00:48:44.145 INFO:tasks.workunit.client.0.smithi038.stdout:8/215: fsync d0/d1d/d20/d3c/f47 0 2024-03-24T00:48:44.156 INFO:tasks.workunit.client.0.smithi038.stdout:2/281: getdents d4/d27 0 2024-03-24T00:48:44.159 INFO:tasks.workunit.client.1.smithi088.stdout:9/242: dwrite d1/d10/d11/f1b [4194304,4194304] 0 2024-03-24T00:48:44.166 INFO:tasks.workunit.client.0.smithi038.stdout:5/219: dwrite d6/d16/f2a [0,4194304] 0 2024-03-24T00:48:44.166 INFO:tasks.workunit.client.0.smithi038.stdout:5/220: dread - d6/f3f zero size 2024-03-24T00:48:44.166 INFO:tasks.workunit.client.0.smithi038.stdout:5/221: chown d6/l34 51302841 1 2024-03-24T00:48:44.169 INFO:tasks.workunit.client.0.smithi038.stdout:7/243: link d2/d9/da/f50 d2/d9/df/f58 0 2024-03-24T00:48:44.169 INFO:tasks.workunit.client.0.smithi038.stdout:7/244: read d2/d9/f54 [3423369,95853] 0 2024-03-24T00:48:44.169 INFO:tasks.workunit.client.0.smithi038.stdout:7/245: write d2/d9/da/f50 [1002460,130189] 0 2024-03-24T00:48:44.169 INFO:tasks.workunit.client.0.smithi038.stdout:7/246: dread - d2/d9/df/d33/f3c zero size 2024-03-24T00:48:44.169 INFO:tasks.workunit.client.0.smithi038.stdout:0/225: symlink d0/d12/d15/l49 0 2024-03-24T00:48:44.170 INFO:tasks.workunit.client.1.smithi088.stdout:8/265: dwrite d9/f3e [0,4194304] 0 2024-03-24T00:48:44.170 INFO:tasks.workunit.client.1.smithi088.stdout:8/266: creat d9/dc/f5c x:0 0 0 2024-03-24T00:48:44.171 INFO:tasks.workunit.client.0.smithi038.stdout:8/216: unlink d0/d15/d42/c2f 0 2024-03-24T00:48:44.173 INFO:tasks.workunit.client.0.smithi038.stdout:9/307: symlink d12/d28/l6b 0 2024-03-24T00:48:44.182 INFO:tasks.workunit.client.0.smithi038.stdout:5/222: mkdir d6/d19/d20/d42/d43 0 2024-03-24T00:48:44.185 INFO:tasks.workunit.client.0.smithi038.stdout:8/217: dread d0/d1d/f3e [0,4194304] 0 2024-03-24T00:48:44.185 INFO:tasks.workunit.client.0.smithi038.stdout:8/218: truncate d0/d1d/d20/d3c/f47 1020627 0 2024-03-24T00:48:44.186 INFO:tasks.workunit.client.0.smithi038.stdout:8/219: chown d0/ce 64 1 2024-03-24T00:48:44.186 INFO:tasks.workunit.client.0.smithi038.stdout:8/220: chown d0/d10/l11 2183 1 2024-03-24T00:48:44.186 INFO:tasks.workunit.client.0.smithi038.stdout:8/221: write d0/d15/d42/f40 [2170710,4890] 0 2024-03-24T00:48:44.186 INFO:tasks.workunit.client.0.smithi038.stdout:8/222: write d0/d1d/f22 [1169391,58809] 0 2024-03-24T00:48:44.193 INFO:tasks.workunit.client.1.smithi088.stdout:3/231: creat d3/d6/d19/d2a/f53 x:0 0 0 2024-03-24T00:48:44.195 INFO:tasks.workunit.client.0.smithi038.stdout:7/247: rename d2/d9/da/f50 to d2/d9/f59 0 2024-03-24T00:48:44.204 INFO:tasks.workunit.client.0.smithi038.stdout:0/226: creat d0/d1b/f4a x:0 0 0 2024-03-24T00:48:44.207 INFO:tasks.workunit.client.0.smithi038.stdout:7/248: creat d2/d9/da/d23/f5a x:0 0 0 2024-03-24T00:48:44.208 INFO:tasks.workunit.client.1.smithi088.stdout:0/277: sync 2024-03-24T00:48:44.208 INFO:tasks.workunit.client.1.smithi088.stdout:0/278: creat d9/dc/de/f5c x:0 0 0 2024-03-24T00:48:44.209 INFO:tasks.workunit.client.1.smithi088.stdout:1/273: getdents d6/d2f 0 2024-03-24T00:48:44.209 INFO:tasks.workunit.client.1.smithi088.stdout:1/274: write d6/d27/f3c [1082238,74610] 0 2024-03-24T00:48:44.210 INFO:tasks.workunit.client.0.smithi038.stdout:4/213: sync 2024-03-24T00:48:44.210 INFO:tasks.workunit.client.0.smithi038.stdout:4/214: write da/f3b [335827,120989] 0 2024-03-24T00:48:44.210 INFO:tasks.workunit.client.0.smithi038.stdout:4/215: dread - da/d12/d37/dd/f34 zero size 2024-03-24T00:48:44.210 INFO:tasks.workunit.client.0.smithi038.stdout:6/299: sync 2024-03-24T00:48:44.211 INFO:tasks.workunit.client.0.smithi038.stdout:6/300: fsync d1/da/f34 0 2024-03-24T00:48:44.211 INFO:tasks.workunit.client.0.smithi038.stdout:6/301: chown d1/da/d1b/d2f/f3d 7 1 2024-03-24T00:48:44.211 INFO:tasks.workunit.client.0.smithi038.stdout:0/227: truncate d0/d2d/f2f 1442442 0 2024-03-24T00:48:44.213 INFO:tasks.workunit.client.1.smithi088.stdout:3/232: write d3/de/df/f2d [1905796,68162] 0 2024-03-24T00:48:44.213 INFO:tasks.workunit.client.1.smithi088.stdout:3/233: readlink d3/d32/l44 0 2024-03-24T00:48:44.214 INFO:tasks.workunit.client.1.smithi088.stdout:6/287: symlink db/d1d/d23/d3c/l54 0 2024-03-24T00:48:44.218 INFO:tasks.workunit.client.1.smithi088.stdout:5/253: creat d1/d29/f50 x:0 0 0 2024-03-24T00:48:44.218 INFO:tasks.workunit.client.1.smithi088.stdout:5/254: stat d1/d19/d1e 0 2024-03-24T00:48:44.220 INFO:tasks.workunit.client.0.smithi038.stdout:3/248: sync 2024-03-24T00:48:44.220 INFO:tasks.workunit.client.0.smithi038.stdout:1/210: sync 2024-03-24T00:48:44.221 INFO:tasks.workunit.client.0.smithi038.stdout:4/216: symlink da/d27/l3d 0 2024-03-24T00:48:44.222 INFO:tasks.workunit.client.0.smithi038.stdout:0/228: symlink d0/d12/d15/l4b 0 2024-03-24T00:48:44.222 INFO:tasks.workunit.client.0.smithi038.stdout:0/229: dread - d0/d1b/d2c/f3b zero size 2024-03-24T00:48:44.223 INFO:tasks.workunit.client.1.smithi088.stdout:9/243: mkdir d1/d10/d41 0 2024-03-24T00:48:44.228 INFO:tasks.workunit.client.1.smithi088.stdout:2/218: getdents db/dd 0 2024-03-24T00:48:44.230 INFO:tasks.workunit.client.0.smithi038.stdout:3/249: creat d3/d2a/f59 x:0 0 0 2024-03-24T00:48:44.230 INFO:tasks.workunit.client.0.smithi038.stdout:3/250: chown d3/d2a/c2f 97112362 1 2024-03-24T00:48:44.234 INFO:tasks.workunit.client.0.smithi038.stdout:1/211: symlink da/dd/d2c/l46 0 2024-03-24T00:48:44.234 INFO:tasks.workunit.client.0.smithi038.stdout:1/212: fsync da/dd/d10/d34/f3d 0 2024-03-24T00:48:44.236 INFO:tasks.workunit.client.1.smithi088.stdout:9/244: write d1/f6 [394916,43521] 0 2024-03-24T00:48:44.236 INFO:tasks.workunit.client.1.smithi088.stdout:9/245: chown d1/f6 1127415 1 2024-03-24T00:48:44.237 INFO:tasks.workunit.client.1.smithi088.stdout:2/219: dread db/fe [0,4194304] 0 2024-03-24T00:48:44.239 INFO:tasks.workunit.client.0.smithi038.stdout:2/282: dwrite d4/d27/f2a [0,4194304] 0 2024-03-24T00:48:44.240 INFO:tasks.workunit.client.0.smithi038.stdout:2/283: chown d4/d18/d26/d37 20163432 1 2024-03-24T00:48:44.241 INFO:tasks.workunit.client.0.smithi038.stdout:4/217: link da/d27/c32 da/d27/c3e 0 2024-03-24T00:48:44.246 INFO:tasks.workunit.client.0.smithi038.stdout:0/230: link d0/d2d/c33 d0/d12/d13/c4c 0 2024-03-24T00:48:44.246 INFO:tasks.workunit.client.0.smithi038.stdout:0/231: chown d0/d14/c36 1942 1 2024-03-24T00:48:44.247 INFO:tasks.workunit.client.1.smithi088.stdout:7/234: rmdir d3/d32/d3f 39 2024-03-24T00:48:44.250 INFO:tasks.workunit.client.0.smithi038.stdout:3/251: truncate d3/d9/f1c 735650 0 2024-03-24T00:48:44.255 INFO:tasks.workunit.client.1.smithi088.stdout:2/220: write db/dd/f1f [2933248,1147] 0 2024-03-24T00:48:44.255 INFO:tasks.workunit.client.1.smithi088.stdout:2/221: write db/f20 [6850836,128973] 0 2024-03-24T00:48:44.261 INFO:tasks.workunit.client.0.smithi038.stdout:1/213: rename da/f11 to da/dd/d10/d14/f47 0 2024-03-24T00:48:44.265 INFO:tasks.workunit.client.1.smithi088.stdout:0/279: rename d9/l2c to d9/d20/d2a/l5d 0 2024-03-24T00:48:44.267 INFO:tasks.workunit.client.1.smithi088.stdout:6/288: mkdir db/d40/d55 0 2024-03-24T00:48:44.267 INFO:tasks.workunit.client.1.smithi088.stdout:9/246: symlink d1/d10/l42 0 2024-03-24T00:48:44.267 INFO:tasks.workunit.client.1.smithi088.stdout:9/247: creat d1/d22/f43 x:0 0 0 2024-03-24T00:48:44.268 INFO:tasks.workunit.client.1.smithi088.stdout:9/248: dread - d1/d22/f43 zero size 2024-03-24T00:48:44.268 INFO:tasks.workunit.client.0.smithi038.stdout:2/284: symlink d4/d18/d38/d63/l65 0 2024-03-24T00:48:44.270 INFO:tasks.workunit.client.0.smithi038.stdout:4/218: creat da/d12/f3f x:0 0 0 2024-03-24T00:48:44.271 INFO:tasks.workunit.client.1.smithi088.stdout:4/292: rmdir d5/d7/d15/d40 0 2024-03-24T00:48:44.271 INFO:tasks.workunit.client.1.smithi088.stdout:4/293: fsync d5/d13/d18/f32 0 2024-03-24T00:48:44.272 INFO:tasks.workunit.client.1.smithi088.stdout:2/222: rename db/fe to db/f46 0 2024-03-24T00:48:44.273 INFO:tasks.workunit.client.0.smithi038.stdout:0/232: mknod d0/d12/d15/d1a/c4d 0 2024-03-24T00:48:44.273 INFO:tasks.workunit.client.1.smithi088.stdout:5/255: dread d1/d19/f30 [4194304,4194304] 0 2024-03-24T00:48:44.273 INFO:tasks.workunit.client.1.smithi088.stdout:0/280: creat d9/d20/d2a/f5e x:0 0 0 2024-03-24T00:48:44.274 INFO:tasks.workunit.client.1.smithi088.stdout:0/281: creat d9/d20/d2a/d2e/f5f x:0 0 0 2024-03-24T00:48:44.275 INFO:tasks.workunit.client.1.smithi088.stdout:6/289: unlink db/dd/f3b 0 2024-03-24T00:48:44.279 INFO:tasks.workunit.client.0.smithi038.stdout:4/219: unlink da/d12/d37/f22 0 2024-03-24T00:48:44.279 INFO:tasks.workunit.client.0.smithi038.stdout:4/220: chown da/f1e 61 1 2024-03-24T00:48:44.280 INFO:tasks.workunit.client.0.smithi038.stdout:4/221: fdatasync f9 0 2024-03-24T00:48:44.283 INFO:tasks.workunit.client.0.smithi038.stdout:8/223: dwrite d0/d1d/f3e [0,4194304] 0 2024-03-24T00:48:44.283 INFO:tasks.workunit.client.1.smithi088.stdout:9/249: mkdir d1/d15/d44 0 2024-03-24T00:48:44.283 INFO:tasks.workunit.client.1.smithi088.stdout:9/250: write d1/d10/f3e [726329,84600] 0 2024-03-24T00:48:44.285 INFO:tasks.workunit.client.0.smithi038.stdout:0/233: unlink d0/d12/d15/f17 0 2024-03-24T00:48:44.287 INFO:tasks.workunit.client.1.smithi088.stdout:4/294: mkdir d5/d13/d63 0 2024-03-24T00:48:44.289 INFO:tasks.workunit.client.0.smithi038.stdout:3/252: mkdir d3/d5a 0 2024-03-24T00:48:44.289 INFO:tasks.workunit.client.1.smithi088.stdout:2/223: mknod db/dd/c47 0 2024-03-24T00:48:44.292 INFO:tasks.workunit.client.0.smithi038.stdout:5/223: dwrite d6/d23/f3b [0,4194304] 0 2024-03-24T00:48:44.292 INFO:tasks.workunit.client.1.smithi088.stdout:8/267: dwrite d9/dc/d11/d24/d29/f51 [0,4194304] 0 2024-03-24T00:48:44.294 INFO:tasks.workunit.client.1.smithi088.stdout:1/275: dwrite d6/d23/f33 [0,4194304] 0 2024-03-24T00:48:44.294 INFO:tasks.workunit.client.1.smithi088.stdout:1/276: write d6/d2f/f48 [1032851,91732] 0 2024-03-24T00:48:44.294 INFO:tasks.workunit.client.1.smithi088.stdout:1/277: write d6/f17 [4497814,100618] 0 2024-03-24T00:48:44.294 INFO:tasks.workunit.client.1.smithi088.stdout:1/278: readlink l4 0 2024-03-24T00:48:44.295 INFO:tasks.workunit.client.0.smithi038.stdout:4/222: unlink da/d27/l3d 0 2024-03-24T00:48:44.295 INFO:tasks.workunit.client.0.smithi038.stdout:2/285: write d4/f16 [1298589,94264] 0 2024-03-24T00:48:44.296 INFO:tasks.workunit.client.0.smithi038.stdout:2/286: chown d4/d27/d2f/f3c 13 1 2024-03-24T00:48:44.296 INFO:tasks.workunit.client.0.smithi038.stdout:6/302: dwrite d1/da/f34 [0,4194304] 0 2024-03-24T00:48:44.296 INFO:tasks.workunit.client.0.smithi038.stdout:6/303: truncate d1/da/d1b/d1d/f4f 840529 0 2024-03-24T00:48:44.311 INFO:tasks.workunit.client.0.smithi038.stdout:8/224: rename d0/ce to d0/d15/c4e 0 2024-03-24T00:48:44.311 INFO:tasks.workunit.client.0.smithi038.stdout:8/225: read d0/d1d/d20/f23 [200690,110879] 0 2024-03-24T00:48:44.323 INFO:tasks.workunit.client.0.smithi038.stdout:3/253: mknod d3/d2a/c5b 0 2024-03-24T00:48:44.323 INFO:tasks.workunit.client.0.smithi038.stdout:7/249: dwrite d2/d9/f54 [0,4194304] 0 2024-03-24T00:48:44.326 INFO:tasks.workunit.client.1.smithi088.stdout:0/282: rename d9/dc/c13 to d9/d26/c60 0 2024-03-24T00:48:44.326 INFO:tasks.workunit.client.1.smithi088.stdout:9/251: creat d1/d10/d29/f45 x:0 0 0 2024-03-24T00:48:44.326 INFO:tasks.workunit.client.1.smithi088.stdout:8/268: mkdir d9/dc/d11/d41/d54/d5d 0 2024-03-24T00:48:44.326 INFO:tasks.workunit.client.1.smithi088.stdout:9/252: write d1/d10/f1e [3123442,32173] 0 2024-03-24T00:48:44.326 INFO:tasks.workunit.client.1.smithi088.stdout:8/269: truncate d9/dc/d11/d24/d43/d4a/f55 943452 0 2024-03-24T00:48:44.327 INFO:tasks.workunit.client.1.smithi088.stdout:8/270: creat d9/dc/d11/d24/f5e x:0 0 0 2024-03-24T00:48:44.331 INFO:tasks.workunit.client.1.smithi088.stdout:1/279: mknod d6/d43/c4d 0 2024-03-24T00:48:44.334 INFO:tasks.workunit.client.0.smithi038.stdout:0/234: rename d0/d12/d15/c25 to d0/d1b/d2c/c4e 0 2024-03-24T00:48:44.334 INFO:tasks.workunit.client.0.smithi038.stdout:1/214: rmdir da/dd/d10/d14 39 2024-03-24T00:48:44.344 INFO:tasks.workunit.client.0.smithi038.stdout:3/254: symlink d3/d9/dc/d51/l5c 0 2024-03-24T00:48:44.344 INFO:tasks.workunit.client.0.smithi038.stdout:3/255: chown d3/d2a/c37 46 1 2024-03-24T00:48:44.350 INFO:tasks.workunit.client.1.smithi088.stdout:3/234: dread d3/de/f17 [0,4194304] 0 2024-03-24T00:48:44.351 INFO:tasks.workunit.client.0.smithi038.stdout:7/250: mkdir d2/d9/da/d23/d2b/d5b 0 2024-03-24T00:48:44.356 INFO:tasks.workunit.client.0.smithi038.stdout:0/235: write d0/fa [5398525,129325] 0 2024-03-24T00:48:44.357 INFO:tasks.workunit.client.0.smithi038.stdout:3/256: unlink d3/d9/c45 0 2024-03-24T00:48:44.368 INFO:tasks.workunit.client.0.smithi038.stdout:0/236: mknod d0/d12/d13/d28/c4f 0 2024-03-24T00:48:44.368 INFO:tasks.workunit.client.0.smithi038.stdout:0/237: stat d0/d12/d13/d28/c4f 0 2024-03-24T00:48:44.374 INFO:tasks.workunit.client.1.smithi088.stdout:6/290: rename db/d1d/d23/d3c/d43/d24 to db/d40/d56 0 2024-03-24T00:48:44.377 INFO:tasks.workunit.client.0.smithi038.stdout:1/215: dread da/dd/d10/d34/f39 [0,4194304] 0 2024-03-24T00:48:44.377 INFO:tasks.workunit.client.0.smithi038.stdout:1/216: chown da/dd/d22/c30 10994 1 2024-03-24T00:48:44.382 INFO:tasks.workunit.client.1.smithi088.stdout:5/256: dwrite d1/d19/d1e/f42 [0,4194304] 0 2024-03-24T00:48:44.382 INFO:tasks.workunit.client.0.smithi038.stdout:6/304: dread d1/da/d1b/d25/f44 [0,4194304] 0 2024-03-24T00:48:44.382 INFO:tasks.workunit.client.0.smithi038.stdout:6/305: chown d1/da/d1b/f26 142 1 2024-03-24T00:48:44.387 INFO:tasks.workunit.client.0.smithi038.stdout:0/238: symlink d0/d12/d13/l50 0 2024-03-24T00:48:44.394 INFO:tasks.workunit.client.0.smithi038.stdout:6/306: rename d1/d37/l40 to d1/da/d1b/d25/d49/d4e/d51/l5a 0 2024-03-24T00:48:44.397 INFO:tasks.workunit.client.0.smithi038.stdout:7/251: read d2/d9/df/f29 [2392503,6338] 0 2024-03-24T00:48:44.398 INFO:tasks.workunit.client.0.smithi038.stdout:1/217: mknod da/d3c/c48 0 2024-03-24T00:48:44.398 INFO:tasks.workunit.client.0.smithi038.stdout:2/287: dwrite d4/d27/d54/d53/f5f [0,4194304] 0 2024-03-24T00:48:44.399 INFO:tasks.workunit.client.0.smithi038.stdout:5/224: dwrite d6/d19/f1c [4194304,4194304] 0 2024-03-24T00:48:44.401 INFO:tasks.workunit.client.1.smithi088.stdout:5/257: truncate d1/dd/f15 2102807 0 2024-03-24T00:48:44.404 INFO:tasks.workunit.client.1.smithi088.stdout:7/235: dwrite d3/f1d [0,4194304] 0 2024-03-24T00:48:44.406 INFO:tasks.workunit.client.0.smithi038.stdout:6/307: symlink d1/da/d1b/l5b 0 2024-03-24T00:48:44.406 INFO:tasks.workunit.client.0.smithi038.stdout:6/308: write d1/da/fb [893073,7336] 0 2024-03-24T00:48:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:7/252: rmdir d2/d9/d16 39 2024-03-24T00:48:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:2/288: link d4/f10 d4/f66 0 2024-03-24T00:48:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:2/289: write d4/d27/d2f/d30/d3e/f58 [1902576,922] 0 2024-03-24T00:48:44.416 INFO:tasks.workunit.client.0.smithi038.stdout:0/239: dread d0/f7 [0,4194304] 0 2024-03-24T00:48:44.416 INFO:tasks.workunit.client.0.smithi038.stdout:5/225: mknod d6/d19/d20/d42/d43/c44 0 2024-03-24T00:48:44.416 INFO:tasks.workunit.client.0.smithi038.stdout:0/240: stat d0/d2d/c2e 0 2024-03-24T00:48:44.420 INFO:tasks.workunit.client.1.smithi088.stdout:9/253: dwrite d1/d10/d11/d28/f36 [4194304,4194304] 0 2024-03-24T00:48:44.421 INFO:tasks.workunit.client.1.smithi088.stdout:1/280: dwrite d6/d27/d1e/d3f/f4c [0,4194304] 0 2024-03-24T00:48:44.421 INFO:tasks.workunit.client.1.smithi088.stdout:6/291: dread db/dd/f37 [4194304,4194304] 0 2024-03-24T00:48:44.422 INFO:tasks.workunit.client.1.smithi088.stdout:2/224: dwrite db/f38 [0,4194304] 0 2024-03-24T00:48:44.425 INFO:tasks.workunit.client.1.smithi088.stdout:7/236: mknod d3/d32/d3f/c4d 0 2024-03-24T00:48:44.427 INFO:tasks.workunit.client.0.smithi038.stdout:4/223: dwrite da/d12/f3f [0,4194304] 0 2024-03-24T00:48:44.428 INFO:tasks.workunit.client.0.smithi038.stdout:1/218: dread da/dd/d10/f15 [0,4194304] 0 2024-03-24T00:48:44.428 INFO:tasks.workunit.client.0.smithi038.stdout:2/290: mkdir d4/d27/d54/d67 0 2024-03-24T00:48:44.430 INFO:tasks.workunit.client.1.smithi088.stdout:5/258: dread d1/fa [0,4194304] 0 2024-03-24T00:48:44.430 INFO:tasks.workunit.client.1.smithi088.stdout:2/225: mkdir db/dc/d48 0 2024-03-24T00:48:44.430 INFO:tasks.workunit.client.1.smithi088.stdout:2/226: dread - db/f43 zero size 2024-03-24T00:48:44.431 INFO:tasks.workunit.client.0.smithi038.stdout:5/226: rmdir d6/d16/d14 39 2024-03-24T00:48:44.432 INFO:tasks.workunit.client.1.smithi088.stdout:7/237: symlink d3/d31/l4e 0 2024-03-24T00:48:44.434 INFO:tasks.workunit.client.1.smithi088.stdout:5/259: mknod d1/d10/d1b/d2e/d46/c51 0 2024-03-24T00:48:44.434 INFO:tasks.workunit.client.0.smithi038.stdout:6/309: read d1/da/d1b/f26 [3365185,53058] 0 2024-03-24T00:48:44.434 INFO:tasks.workunit.client.1.smithi088.stdout:5/260: creat d1/d10/f52 x:0 0 0 2024-03-24T00:48:44.435 INFO:tasks.workunit.client.0.smithi038.stdout:8/226: dwrite d0/d1d/f3e [0,4194304] 0 2024-03-24T00:48:44.435 INFO:tasks.workunit.client.0.smithi038.stdout:8/227: fsync d0/d45/d37/f48 0 2024-03-24T00:48:44.436 INFO:tasks.workunit.client.1.smithi088.stdout:7/238: symlink d3/l4f 0 2024-03-24T00:48:44.442 INFO:tasks.workunit.client.0.smithi038.stdout:6/310: symlink d1/d37/l5c 0 2024-03-24T00:48:44.442 INFO:tasks.workunit.client.0.smithi038.stdout:3/257: dwrite d3/d9/dc/df/f1d [0,4194304] 0 2024-03-24T00:48:44.442 INFO:tasks.workunit.client.0.smithi038.stdout:0/241: dread d0/fc [0,4194304] 0 2024-03-24T00:48:44.442 INFO:tasks.workunit.client.0.smithi038.stdout:0/242: fdatasync d0/f43 0 2024-03-24T00:48:44.444 INFO:tasks.workunit.client.1.smithi088.stdout:2/227: dread db/dd/f1f [0,4194304] 0 2024-03-24T00:48:44.448 INFO:tasks.workunit.client.1.smithi088.stdout:2/228: rename db/dc/d48 to db/d21/d35/d49 0 2024-03-24T00:48:44.448 INFO:tasks.workunit.client.1.smithi088.stdout:7/239: dread d3/f2a [0,4194304] 0 2024-03-24T00:48:44.450 INFO:tasks.workunit.client.0.smithi038.stdout:8/228: creat d0/d10/d19/f4f x:0 0 0 2024-03-24T00:48:44.450 INFO:tasks.workunit.client.0.smithi038.stdout:8/229: chown d0/d1d/d27/c2e 775327 1 2024-03-24T00:48:44.450 INFO:tasks.workunit.client.0.smithi038.stdout:4/224: rename da/d12/d37/dd/c1b to da/d12/d37/dd/d15/d33/c40 0 2024-03-24T00:48:44.451 INFO:tasks.workunit.client.0.smithi038.stdout:4/225: creat da/d38/f41 x:0 0 0 2024-03-24T00:48:44.457 INFO:tasks.workunit.client.0.smithi038.stdout:6/311: link d1/da/c3f d1/da/d1b/d25/d49/d4e/c5d 0 2024-03-24T00:48:44.457 INFO:tasks.workunit.client.0.smithi038.stdout:6/312: stat d1/da/d1b/d1d/f2d 0 2024-03-24T00:48:44.457 INFO:tasks.workunit.client.0.smithi038.stdout:6/313: readlink d1/l36 0 2024-03-24T00:48:44.457 INFO:tasks.workunit.client.0.smithi038.stdout:3/258: mkdir d3/d30/d5d 0 2024-03-24T00:48:44.457 INFO:tasks.workunit.client.0.smithi038.stdout:3/259: chown d3/d30/f34 27864395 1 2024-03-24T00:48:44.457 INFO:tasks.workunit.client.0.smithi038.stdout:6/314: fsync d1/da/d1b/d25/d49/d4e/f55 0 2024-03-24T00:48:44.457 INFO:tasks.workunit.client.0.smithi038.stdout:3/260: fsync d3/d9/f3c 0 2024-03-24T00:48:44.457 INFO:tasks.workunit.client.0.smithi038.stdout:3/261: write d3/d9/dc/df/d14/d19/f46 [927420,3897] 0 2024-03-24T00:48:44.457 INFO:tasks.workunit.client.0.smithi038.stdout:3/262: getdents d3/d30/d5d 0 2024-03-24T00:48:44.459 INFO:tasks.workunit.client.0.smithi038.stdout:0/243: unlink d0/d12/f3a 0 2024-03-24T00:48:44.464 INFO:tasks.workunit.client.1.smithi088.stdout:5/261: write d1/d19/d1e/f42 [4217396,129920] 0 2024-03-24T00:48:44.466 INFO:tasks.workunit.client.0.smithi038.stdout:3/263: write d3/d9/f2b [687807,41312] 0 2024-03-24T00:48:44.470 INFO:tasks.workunit.client.0.smithi038.stdout:4/226: creat da/d12/d37/dd/d15/f42 x:0 0 0 2024-03-24T00:48:44.472 INFO:tasks.workunit.client.1.smithi088.stdout:2/229: dread db/f3c [0,4194304] 0 2024-03-24T00:48:44.475 INFO:tasks.workunit.client.0.smithi038.stdout:5/227: write d6/fa [2160019,106169] 0 2024-03-24T00:48:44.482 INFO:tasks.workunit.client.0.smithi038.stdout:0/244: dread d0/d2d/f2f [0,4194304] 0 2024-03-24T00:48:44.486 INFO:tasks.workunit.client.1.smithi088.stdout:5/262: unlink d1/c2 0 2024-03-24T00:48:44.486 INFO:tasks.workunit.client.1.smithi088.stdout:5/263: write d1/d10/d1b/d2f/f38 [2994355,29734] 0 2024-03-24T00:48:44.486 INFO:tasks.workunit.client.1.smithi088.stdout:5/264: write d1/dd/f23 [5991689,32174] 0 2024-03-24T00:48:44.486 INFO:tasks.workunit.client.1.smithi088.stdout:5/265: write d1/ff [3484730,56025] 0 2024-03-24T00:48:44.486 INFO:tasks.workunit.client.1.smithi088.stdout:5/266: stat d1/ff 0 2024-03-24T00:48:44.486 INFO:tasks.workunit.client.1.smithi088.stdout:5/267: stat d1/l20 0 2024-03-24T00:48:44.487 INFO:tasks.workunit.client.1.smithi088.stdout:5/268: write d1/d19/f45 [720835,87631] 0 2024-03-24T00:48:44.487 INFO:tasks.workunit.client.1.smithi088.stdout:3/235: dwrite d3/de/f17 [0,4194304] 0 2024-03-24T00:48:44.489 INFO:tasks.workunit.client.0.smithi038.stdout:8/230: truncate d0/d15/d42/f40 1087007 0 2024-03-24T00:48:44.489 INFO:tasks.workunit.client.0.smithi038.stdout:6/315: creat d1/da/d1b/d25/d49/d4e/d51/f5e x:0 0 0 2024-03-24T00:48:44.489 INFO:tasks.workunit.client.1.smithi088.stdout:1/281: dread d6/d27/d1e/f39 [0,4194304] 0 2024-03-24T00:48:44.490 INFO:tasks.workunit.client.1.smithi088.stdout:1/282: truncate d6/d23/f33 4222849 0 2024-03-24T00:48:44.490 INFO:tasks.workunit.client.1.smithi088.stdout:1/283: stat d6/d2f 0 2024-03-24T00:48:44.490 INFO:tasks.workunit.client.1.smithi088.stdout:0/283: dwrite d9/dc/de/d19/f48 [0,4194304] 0 2024-03-24T00:48:44.494 INFO:tasks.workunit.client.1.smithi088.stdout:5/269: rename d1/d19/d1e/d26/l49 to d1/d19/d1e/l53 0 2024-03-24T00:48:44.495 INFO:tasks.workunit.client.0.smithi038.stdout:7/253: dwrite d2/d9/df/f29 [0,4194304] 0 2024-03-24T00:48:44.498 INFO:tasks.workunit.client.1.smithi088.stdout:2/230: read db/f38 [2295802,58886] 0 2024-03-24T00:48:44.501 INFO:tasks.workunit.client.1.smithi088.stdout:1/284: unlink d6/d2f/l38 0 2024-03-24T00:48:44.506 INFO:tasks.workunit.client.0.smithi038.stdout:0/245: creat d0/d12/d42/f51 x:0 0 0 2024-03-24T00:48:44.510 INFO:tasks.workunit.client.1.smithi088.stdout:5/270: creat d1/dd/f54 x:0 0 0 2024-03-24T00:48:44.510 INFO:tasks.workunit.client.1.smithi088.stdout:5/271: chown d1/d10/d1b/c25 0 1 2024-03-24T00:48:44.514 INFO:tasks.workunit.client.0.smithi038.stdout:6/316: symlink d1/d37/l5f 0 2024-03-24T00:48:44.514 INFO:tasks.workunit.client.0.smithi038.stdout:7/254: symlink d2/d9/l5c 0 2024-03-24T00:48:44.514 INFO:tasks.workunit.client.0.smithi038.stdout:7/255: stat l1 0 2024-03-24T00:48:44.514 INFO:tasks.workunit.client.0.smithi038.stdout:8/231: symlink d0/d45/d37/l50 0 2024-03-24T00:48:44.519 INFO:tasks.workunit.client.0.smithi038.stdout:4/227: rename da/cf to da/d12/d37/dd/c43 0 2024-03-24T00:48:44.523 INFO:tasks.workunit.client.0.smithi038.stdout:8/232: mknod d0/d15/c51 0 2024-03-24T00:48:44.523 INFO:tasks.workunit.client.0.smithi038.stdout:8/233: dread - d0/d45/d37/f48 zero size 2024-03-24T00:48:44.523 INFO:tasks.workunit.client.0.smithi038.stdout:8/234: stat d0/d1d/f3e 0 2024-03-24T00:48:44.529 INFO:tasks.workunit.client.0.smithi038.stdout:7/256: dread d2/d9/d16/f19 [0,4194304] 0 2024-03-24T00:48:44.535 INFO:tasks.workunit.client.1.smithi088.stdout:1/285: mknod d6/d23/c4e 0 2024-03-24T00:48:44.538 INFO:tasks.workunit.client.1.smithi088.stdout:5/272: symlink d1/d10/d1b/d2e/l55 0 2024-03-24T00:48:44.541 INFO:tasks.workunit.client.0.smithi038.stdout:4/228: symlink da/d38/l44 0 2024-03-24T00:48:44.542 INFO:tasks.workunit.client.0.smithi038.stdout:4/229: readlink da/d12/l2d 0 2024-03-24T00:48:44.549 INFO:tasks.workunit.client.1.smithi088.stdout:9/254: dwrite d1/d10/f1e [0,4194304] 0 2024-03-24T00:48:44.550 INFO:tasks.workunit.client.0.smithi038.stdout:8/235: mknod d0/d45/c52 0 2024-03-24T00:48:44.551 INFO:tasks.workunit.client.0.smithi038.stdout:8/236: write d0/d1d/d20/f23 [1120345,71909] 0 2024-03-24T00:48:44.552 INFO:tasks.workunit.client.0.smithi038.stdout:7/257: mkdir d2/d9/df/d5d 0 2024-03-24T00:48:44.552 INFO:tasks.workunit.client.0.smithi038.stdout:7/258: truncate d2/d9/d16/f53 196344 0 2024-03-24T00:48:44.552 INFO:tasks.workunit.client.0.smithi038.stdout:2/291: dwrite d4/d18/d26/f4e [0,4194304] 0 2024-03-24T00:48:44.555 INFO:tasks.workunit.client.1.smithi088.stdout:2/231: mkdir db/d21/d4a 0 2024-03-24T00:48:44.557 INFO:tasks.workunit.client.1.smithi088.stdout:3/236: dwrite d3/de/df/f25 [0,4194304] 0 2024-03-24T00:48:44.557 INFO:tasks.workunit.client.1.smithi088.stdout:3/237: readlink d3/l22 0 2024-03-24T00:48:44.557 INFO:tasks.workunit.client.1.smithi088.stdout:3/238: fsync d3/d6/d27/d3c/f30 0 2024-03-24T00:48:44.559 INFO:tasks.workunit.client.0.smithi038.stdout:8/237: unlink d0/d1d/d27/l4b 0 2024-03-24T00:48:44.559 INFO:tasks.workunit.client.0.smithi038.stdout:8/238: write d0/d10/d19/f4f [536439,66974] 0 2024-03-24T00:48:44.562 INFO:tasks.workunit.client.0.smithi038.stdout:3/264: dwrite f1 [0,4194304] 0 2024-03-24T00:48:44.569 INFO:tasks.workunit.client.0.smithi038.stdout:5/228: dwrite d6/f3f [0,4194304] 0 2024-03-24T00:48:44.573 INFO:tasks.workunit.client.1.smithi088.stdout:9/255: symlink d1/d10/l46 0 2024-03-24T00:48:44.573 INFO:tasks.workunit.client.1.smithi088.stdout:9/256: truncate d1/f20 5088718 0 2024-03-24T00:48:44.574 INFO:tasks.workunit.client.0.smithi038.stdout:0/246: dwrite d0/d12/d15/f22 [0,4194304] 0 2024-03-24T00:48:44.574 INFO:tasks.workunit.client.0.smithi038.stdout:0/247: dread - d0/d12/d42/f51 zero size 2024-03-24T00:48:44.574 INFO:tasks.workunit.client.1.smithi088.stdout:2/232: creat db/d21/d4a/f4b x:0 0 0 2024-03-24T00:48:44.575 INFO:tasks.workunit.client.1.smithi088.stdout:2/233: dread - db/dc/f2b zero size 2024-03-24T00:48:44.578 INFO:tasks.workunit.client.1.smithi088.stdout:6/292: dwrite db/d1d/d23/d3c/d43/f39 [0,4194304] 0 2024-03-24T00:48:44.578 INFO:tasks.workunit.client.1.smithi088.stdout:0/284: dwrite d9/d20/d2a/f5e [0,4194304] 0 2024-03-24T00:48:44.578 INFO:tasks.workunit.client.0.smithi038.stdout:7/259: unlink d2/d9/da/d23/d2b/c46 0 2024-03-24T00:48:44.579 INFO:tasks.workunit.client.0.smithi038.stdout:2/292: dread d4/d18/d26/d37/f43 [0,4194304] 0 2024-03-24T00:48:44.585 INFO:tasks.workunit.client.0.smithi038.stdout:8/239: creat d0/d1d/d27/d4d/f53 x:0 0 0 2024-03-24T00:48:44.585 INFO:tasks.workunit.client.1.smithi088.stdout:9/257: dread d1/f7 [0,4194304] 0 2024-03-24T00:48:44.585 INFO:tasks.workunit.client.1.smithi088.stdout:6/293: dread db/dd/f33 [0,4194304] 0 2024-03-24T00:48:44.586 INFO:tasks.workunit.client.1.smithi088.stdout:5/273: getdents d1/d19/d1e 0 2024-03-24T00:48:44.590 INFO:tasks.workunit.client.0.smithi038.stdout:3/265: creat d3/d30/d5d/f5e x:0 0 0 2024-03-24T00:48:44.590 INFO:tasks.workunit.client.0.smithi038.stdout:3/266: fdatasync d3/d9/f16 0 2024-03-24T00:48:44.591 INFO:tasks.workunit.client.0.smithi038.stdout:3/267: readlink d3/d9/dc/d51/l5c 0 2024-03-24T00:48:44.591 INFO:tasks.workunit.client.1.smithi088.stdout:2/234: dread db/dc/f1b [0,4194304] 0 2024-03-24T00:48:44.600 INFO:tasks.workunit.client.1.smithi088.stdout:4/295: sync 2024-03-24T00:48:44.600 INFO:tasks.workunit.client.1.smithi088.stdout:1/286: dwrite d6/d27/f3c [0,4194304] 0 2024-03-24T00:48:44.600 INFO:tasks.workunit.client.1.smithi088.stdout:4/296: write d5/d16/d37/f48 [1720168,111068] 0 2024-03-24T00:48:44.601 INFO:tasks.workunit.client.0.smithi038.stdout:5/229: truncate d6/d16/f2a 2694468 0 2024-03-24T00:48:44.602 INFO:tasks.workunit.client.0.smithi038.stdout:3/268: dread d3/d30/f33 [0,4194304] 0 2024-03-24T00:48:44.607 INFO:tasks.workunit.client.1.smithi088.stdout:3/239: dwrite d3/de/f16 [0,4194304] 0 2024-03-24T00:48:44.608 INFO:tasks.workunit.client.1.smithi088.stdout:3/240: chown d3/d6/d27/d3c/f47 4691 1 2024-03-24T00:48:44.608 INFO:tasks.workunit.client.1.smithi088.stdout:3/241: write d3/d6/d27/d3c/f47 [302737,65467] 0 2024-03-24T00:48:44.608 INFO:tasks.workunit.client.1.smithi088.stdout:3/242: stat d3/d1a/d1e/c3e 0 2024-03-24T00:48:44.608 INFO:tasks.workunit.client.1.smithi088.stdout:3/243: fdatasync f2 0 2024-03-24T00:48:44.610 INFO:tasks.workunit.client.1.smithi088.stdout:9/258: mkdir d1/d15/d47 0 2024-03-24T00:48:44.613 INFO:tasks.workunit.client.0.smithi038.stdout:7/260: symlink d2/d9/da/d23/d2b/d5b/l5e 0 2024-03-24T00:48:44.613 INFO:tasks.workunit.client.0.smithi038.stdout:7/261: truncate d2/d9/fb 4890030 0 2024-03-24T00:48:44.617 INFO:tasks.workunit.client.1.smithi088.stdout:6/294: unlink db/f3a 0 2024-03-24T00:48:44.617 INFO:tasks.workunit.client.1.smithi088.stdout:6/295: readlink l7 0 2024-03-24T00:48:44.618 INFO:tasks.workunit.client.1.smithi088.stdout:6/296: read db/d40/d56/f30 [22602,19273] 0 2024-03-24T00:48:44.620 INFO:tasks.workunit.client.0.smithi038.stdout:2/293: mknod d4/d27/c68 0 2024-03-24T00:48:44.622 INFO:tasks.workunit.client.1.smithi088.stdout:5/274: link d1/d29/f50 d1/d10/f56 0 2024-03-24T00:48:44.623 INFO:tasks.workunit.client.0.smithi038.stdout:8/240: creat d0/d15/f54 x:0 0 0 2024-03-24T00:48:44.626 INFO:tasks.workunit.client.0.smithi038.stdout:8/241: dread - d0/d45/d37/f48 zero size 2024-03-24T00:48:44.626 INFO:tasks.workunit.client.0.smithi038.stdout:3/269: write d3/d9/dc/df/d14/d19/f28 [4055705,111677] 0 2024-03-24T00:48:44.626 INFO:tasks.workunit.client.0.smithi038.stdout:5/230: symlink d6/d16/d3c/l45 0 2024-03-24T00:48:44.626 INFO:tasks.workunit.client.0.smithi038.stdout:5/231: chown d6/f18 632 1 2024-03-24T00:48:44.633 INFO:tasks.workunit.client.0.smithi038.stdout:3/270: creat d3/d9/dc/df/f5f x:0 0 0 2024-03-24T00:48:44.633 INFO:tasks.workunit.client.0.smithi038.stdout:3/271: readlink d3/l36 0 2024-03-24T00:48:44.635 INFO:tasks.workunit.client.1.smithi088.stdout:4/297: link d5/d7/c1f d5/d16/d1a/d22/d2f/c64 0 2024-03-24T00:48:44.639 INFO:tasks.workunit.client.0.smithi038.stdout:5/232: mknod d6/d16/d21/d2d/c46 0 2024-03-24T00:48:44.640 INFO:tasks.workunit.client.1.smithi088.stdout:8/271: sync 2024-03-24T00:48:44.642 INFO:tasks.workunit.client.0.smithi038.stdout:2/294: getdents d4/d27/d54/d53 0 2024-03-24T00:48:44.642 INFO:tasks.workunit.client.0.smithi038.stdout:2/295: truncate d4/d27/d2f/d30/d3e/f58 2654185 0 2024-03-24T00:48:44.644 INFO:tasks.workunit.client.0.smithi038.stdout:8/242: dread d0/f24 [0,4194304] 0 2024-03-24T00:48:44.648 INFO:tasks.workunit.client.0.smithi038.stdout:7/262: rmdir d2/d9/da 39 2024-03-24T00:48:44.648 INFO:tasks.workunit.client.0.smithi038.stdout:7/263: dread - d2/d9/da/d23/f5a zero size 2024-03-24T00:48:44.651 INFO:tasks.workunit.client.0.smithi038.stdout:2/296: mkdir d4/d18/d26/d37/d69 0 2024-03-24T00:48:44.651 INFO:tasks.workunit.client.0.smithi038.stdout:8/243: creat d0/d45/f55 x:0 0 0 2024-03-24T00:48:44.651 INFO:tasks.workunit.client.0.smithi038.stdout:8/244: chown d0/d10/l21 1 1 2024-03-24T00:48:44.652 INFO:tasks.workunit.client.1.smithi088.stdout:3/244: link f1 d3/de/df/d49/f54 0 2024-03-24T00:48:44.660 INFO:tasks.workunit.client.0.smithi038.stdout:7/264: creat d2/d9/da/d23/d2b/f5f x:0 0 0 2024-03-24T00:48:44.665 INFO:tasks.workunit.client.0.smithi038.stdout:4/230: dwrite da/d38/f3c [0,4194304] 0 2024-03-24T00:48:44.667 INFO:tasks.workunit.client.1.smithi088.stdout:0/285: rmdir d9 39 2024-03-24T00:48:44.667 INFO:tasks.workunit.client.1.smithi088.stdout:9/259: mknod d1/d22/d27/c48 0 2024-03-24T00:48:44.667 INFO:tasks.workunit.client.1.smithi088.stdout:9/260: truncate d1/d10/f3e 1436555 0 2024-03-24T00:48:44.667 INFO:tasks.workunit.client.1.smithi088.stdout:9/261: dread - d1/d10/d11/f38 zero size 2024-03-24T00:48:44.667 INFO:tasks.workunit.client.1.smithi088.stdout:9/262: write d1/d10/d11/f38 [1041063,45837] 0 2024-03-24T00:48:44.667 INFO:tasks.workunit.client.1.smithi088.stdout:9/263: read d1/ff [3710766,21461] 0 2024-03-24T00:48:44.668 INFO:tasks.workunit.client.1.smithi088.stdout:6/297: creat db/d40/f57 x:0 0 0 2024-03-24T00:48:44.674 INFO:tasks.workunit.client.1.smithi088.stdout:3/245: dread d3/de/df/f15 [0,4194304] 0 2024-03-24T00:48:44.674 INFO:tasks.workunit.client.1.smithi088.stdout:5/275: getdents d1/d10/d1b/d2e/d46 0 2024-03-24T00:48:44.678 INFO:tasks.workunit.client.1.smithi088.stdout:2/235: rename db/d21/c3a to db/d21/c4c 0 2024-03-24T00:48:44.678 INFO:tasks.workunit.client.1.smithi088.stdout:2/236: write db/d21/d4a/f4b [968739,36590] 0 2024-03-24T00:48:44.681 INFO:tasks.workunit.client.1.smithi088.stdout:9/264: dread d1/d10/f3e [0,4194304] 0 2024-03-24T00:48:44.684 INFO:tasks.workunit.client.1.smithi088.stdout:1/287: rmdir d6 39 2024-03-24T00:48:44.684 INFO:tasks.workunit.client.1.smithi088.stdout:7/240: sync 2024-03-24T00:48:44.687 INFO:tasks.workunit.client.0.smithi038.stdout:2/297: write d4/d27/d2f/f3c [446958,21551] 0 2024-03-24T00:48:44.687 INFO:tasks.workunit.client.1.smithi088.stdout:4/298: creat d5/d7/dd/f65 x:0 0 0 2024-03-24T00:48:44.693 INFO:tasks.workunit.client.1.smithi088.stdout:8/272: rename d9/dc/d11/d24/d29/d36/c3f to d9/dc/d11/d24/d43/d4a/c5f 0 2024-03-24T00:48:44.695 INFO:tasks.workunit.client.1.smithi088.stdout:0/286: creat d9/d26/f61 x:0 0 0 2024-03-24T00:48:44.695 INFO:tasks.workunit.client.1.smithi088.stdout:0/287: truncate d9/d20/d2a/d2e/f5f 638606 0 2024-03-24T00:48:44.698 INFO:tasks.workunit.client.0.smithi038.stdout:0/248: dwrite d0/f7 [0,4194304] 0 2024-03-24T00:48:44.701 INFO:tasks.workunit.client.1.smithi088.stdout:2/237: truncate f1 4048222 0 2024-03-24T00:48:44.701 INFO:tasks.workunit.client.0.smithi038.stdout:4/231: link da/d12/c25 da/c45 0 2024-03-24T00:48:44.704 INFO:tasks.workunit.client.1.smithi088.stdout:5/276: write d1/fa [591501,127665] 0 2024-03-24T00:48:44.704 INFO:tasks.workunit.client.1.smithi088.stdout:6/298: unlink db/d40/f57 0 2024-03-24T00:48:44.704 INFO:tasks.workunit.client.1.smithi088.stdout:6/299: fdatasync db/f38 0 2024-03-24T00:48:44.704 INFO:tasks.workunit.client.1.smithi088.stdout:6/300: stat db/d40/d56/c4e 0 2024-03-24T00:48:44.705 INFO:tasks.workunit.client.1.smithi088.stdout:0/288: rmdir d9/dc/de/d19 39 2024-03-24T00:48:44.705 INFO:tasks.workunit.client.1.smithi088.stdout:9/265: truncate d1/d10/d11/f17 640630 0 2024-03-24T00:48:44.705 INFO:tasks.workunit.client.1.smithi088.stdout:0/289: fsync d9/d26/f38 0 2024-03-24T00:48:44.705 INFO:tasks.workunit.client.1.smithi088.stdout:4/299: truncate d5/d13/d18/f1b 1181884 0 2024-03-24T00:48:44.705 INFO:tasks.workunit.client.1.smithi088.stdout:5/277: mkdir d1/d10/d1b/d2f/d57 0 2024-03-24T00:48:44.706 INFO:tasks.workunit.client.0.smithi038.stdout:2/298: link d4/c11 d4/d27/d54/d53/c6a 0 2024-03-24T00:48:44.706 INFO:tasks.workunit.client.1.smithi088.stdout:5/278: chown d1/d29/d3f/f41 4954194 1 2024-03-24T00:48:44.706 INFO:tasks.workunit.client.1.smithi088.stdout:5/279: creat d1/d10/d1b/d2f/f58 x:0 0 0 2024-03-24T00:48:44.707 INFO:tasks.workunit.client.1.smithi088.stdout:0/290: mkdir d9/dc/d4f/d62 0 2024-03-24T00:48:44.707 INFO:tasks.workunit.client.0.smithi038.stdout:2/299: chown d4/d18/c20 63 1 2024-03-24T00:48:44.707 INFO:tasks.workunit.client.0.smithi038.stdout:2/300: write d4/ff [4373055,122962] 0 2024-03-24T00:48:44.709 INFO:tasks.workunit.client.1.smithi088.stdout:9/266: rename d1/d10/d11/f1b to d1/d22/f49 0 2024-03-24T00:48:44.709 INFO:tasks.workunit.client.0.smithi038.stdout:4/232: symlink da/d12/d29/l46 0 2024-03-24T00:48:44.714 INFO:tasks.workunit.client.0.smithi038.stdout:2/301: creat d4/d27/d54/d53/f6b x:0 0 0 2024-03-24T00:48:44.714 INFO:tasks.workunit.client.1.smithi088.stdout:1/288: dread f2 [0,4194304] 0 2024-03-24T00:48:44.715 INFO:tasks.workunit.client.1.smithi088.stdout:4/300: creat d5/d7/f66 x:0 0 0 2024-03-24T00:48:44.715 INFO:tasks.workunit.client.1.smithi088.stdout:4/301: readlink d5/d13/d18/l61 0 2024-03-24T00:48:44.715 INFO:tasks.workunit.client.0.smithi038.stdout:4/233: symlink da/d12/d29/l47 0 2024-03-24T00:48:44.719 INFO:tasks.workunit.client.1.smithi088.stdout:9/267: mknod d1/d10/d29/c4a 0 2024-03-24T00:48:44.719 INFO:tasks.workunit.client.1.smithi088.stdout:9/268: fsync d1/d15/d21/f3a 0 2024-03-24T00:48:44.719 INFO:tasks.workunit.client.1.smithi088.stdout:9/269: readlink d1/d10/d11/l32 0 2024-03-24T00:48:44.719 INFO:tasks.workunit.client.1.smithi088.stdout:9/270: fdatasync d1/d10/f1e 0 2024-03-24T00:48:44.719 INFO:tasks.workunit.client.1.smithi088.stdout:1/289: creat d6/d43/f4f x:0 0 0 2024-03-24T00:48:44.719 INFO:tasks.workunit.client.1.smithi088.stdout:7/241: read d3/f30 [631828,101999] 0 2024-03-24T00:48:44.719 INFO:tasks.workunit.client.1.smithi088.stdout:3/246: dread d3/de/df/f25 [0,4194304] 0 2024-03-24T00:48:44.720 INFO:tasks.workunit.client.1.smithi088.stdout:3/247: fdatasync d3/de/df/f15 0 2024-03-24T00:48:44.720 INFO:tasks.workunit.client.1.smithi088.stdout:3/248: fdatasync d3/de/f17 0 2024-03-24T00:48:44.720 INFO:tasks.workunit.client.1.smithi088.stdout:1/290: creat d6/d43/f50 x:0 0 0 2024-03-24T00:48:44.724 INFO:tasks.workunit.client.0.smithi038.stdout:5/233: dwrite d6/d23/f3b [4194304,4194304] 0 2024-03-24T00:48:44.724 INFO:tasks.workunit.client.1.smithi088.stdout:7/242: mknod d3/d32/d3f/c50 0 2024-03-24T00:48:44.734 INFO:tasks.workunit.client.1.smithi088.stdout:1/291: rename d6/d43/f45 to d6/d2f/f51 0 2024-03-24T00:48:44.735 INFO:tasks.workunit.client.1.smithi088.stdout:7/243: creat d3/f51 x:0 0 0 2024-03-24T00:48:44.737 INFO:tasks.workunit.client.1.smithi088.stdout:3/249: mknod d3/d6/c55 0 2024-03-24T00:48:44.737 INFO:tasks.workunit.client.1.smithi088.stdout:3/250: truncate d3/d1a/d1e/d48/d4b/f4c 473090 0 2024-03-24T00:48:44.737 INFO:tasks.workunit.client.0.smithi038.stdout:4/234: dread da/d12/d24/f26 [0,4194304] 0 2024-03-24T00:48:44.739 INFO:tasks.workunit.client.1.smithi088.stdout:1/292: creat d6/d23/f52 x:0 0 0 2024-03-24T00:48:44.745 INFO:tasks.workunit.client.1.smithi088.stdout:7/244: link d3/d32/d3f/c4d d3/c52 0 2024-03-24T00:48:44.756 INFO:tasks.workunit.client.1.smithi088.stdout:0/291: dread d9/dc/de/f40 [0,4194304] 0 2024-03-24T00:48:44.756 INFO:tasks.workunit.client.1.smithi088.stdout:0/292: truncate d9/dc/de/f5c 511512 0 2024-03-24T00:48:44.756 INFO:tasks.workunit.client.1.smithi088.stdout:0/293: read - d9/d26/f3c zero size 2024-03-24T00:48:44.758 INFO:tasks.workunit.client.0.smithi038.stdout:7/265: dwrite d2/f13 [0,4194304] 0 2024-03-24T00:48:44.760 INFO:tasks.workunit.client.1.smithi088.stdout:3/251: unlink d3/d1a/f38 0 2024-03-24T00:48:44.760 INFO:tasks.workunit.client.1.smithi088.stdout:3/252: chown d3/d6/d19/f52 11608949 1 2024-03-24T00:48:44.763 INFO:tasks.workunit.client.0.smithi038.stdout:3/272: dwrite d3/d9/dc/df/f5f [0,4194304] 0 2024-03-24T00:48:44.763 INFO:tasks.workunit.client.0.smithi038.stdout:3/273: truncate d3/d9/f58 661741 0 2024-03-24T00:48:44.763 INFO:tasks.workunit.client.0.smithi038.stdout:3/274: chown d3/d9/dc/d51/f52 12 1 2024-03-24T00:48:44.763 INFO:tasks.workunit.client.0.smithi038.stdout:3/275: write d3/f8 [1653810,16589] 0 2024-03-24T00:48:44.763 INFO:tasks.workunit.client.1.smithi088.stdout:1/293: unlink d6/c26 0 2024-03-24T00:48:44.773 INFO:tasks.workunit.client.1.smithi088.stdout:3/253: mkdir d3/d6/d27/d56 0 2024-03-24T00:48:44.773 INFO:tasks.workunit.client.1.smithi088.stdout:3/254: stat d3/cd 0 2024-03-24T00:48:44.773 INFO:tasks.workunit.client.1.smithi088.stdout:3/255: dread - d3/d6/d19/f52 zero size 2024-03-24T00:48:44.774 INFO:tasks.workunit.client.1.smithi088.stdout:1/294: write d6/d27/f1a [613683,67693] 0 2024-03-24T00:48:44.775 INFO:tasks.workunit.client.1.smithi088.stdout:1/295: write d6/d27/d1e/f40 [1337868,113549] 0 2024-03-24T00:48:44.780 INFO:tasks.workunit.client.0.smithi038.stdout:7/266: truncate d2/d9/df/f58 942914 0 2024-03-24T00:48:44.780 INFO:tasks.workunit.client.0.smithi038.stdout:8/245: dwrite d0/d1d/f22 [0,4194304] 0 2024-03-24T00:48:44.780 INFO:tasks.workunit.client.1.smithi088.stdout:0/294: dread d9/f1c [0,4194304] 0 2024-03-24T00:48:44.783 INFO:tasks.workunit.client.0.smithi038.stdout:5/234: write d6/f26 [2224060,35136] 0 2024-03-24T00:48:44.784 INFO:tasks.workunit.client.0.smithi038.stdout:4/235: dread da/f1d [0,4194304] 0 2024-03-24T00:48:44.784 INFO:tasks.workunit.client.0.smithi038.stdout:4/236: chown da/d12/c25 66076435 1 2024-03-24T00:48:44.784 INFO:tasks.workunit.client.0.smithi038.stdout:4/237: chown da/f3b 62 1 2024-03-24T00:48:44.785 INFO:tasks.workunit.client.0.smithi038.stdout:0/249: dwrite d0/fa [0,4194304] 0 2024-03-24T00:48:44.789 INFO:tasks.workunit.client.1.smithi088.stdout:3/256: mknod d3/c57 0 2024-03-24T00:48:44.791 INFO:tasks.workunit.client.0.smithi038.stdout:4/238: read da/d12/d37/dd/f2b [502418,100933] 0 2024-03-24T00:48:44.797 INFO:tasks.workunit.client.1.smithi088.stdout:2/238: dwrite db/dd/f1f [0,4194304] 0 2024-03-24T00:48:44.797 INFO:tasks.workunit.client.1.smithi088.stdout:6/301: dwrite db/dd/f41 [0,4194304] 0 2024-03-24T00:48:44.797 INFO:tasks.workunit.client.1.smithi088.stdout:6/302: write db/d40/d56/f31 [838004,65225] 0 2024-03-24T00:48:44.799 INFO:tasks.workunit.client.1.smithi088.stdout:9/271: dwrite d1/f20 [0,4194304] 0 2024-03-24T00:48:44.803 INFO:tasks.workunit.client.1.smithi088.stdout:2/239: write db/d21/d4a/f4b [153226,100139] 0 2024-03-24T00:48:44.803 INFO:tasks.workunit.client.1.smithi088.stdout:2/240: stat db/f20 0 2024-03-24T00:48:44.803 INFO:tasks.workunit.client.1.smithi088.stdout:1/296: mkdir d6/d27/d1e/d3f/d53 0 2024-03-24T00:48:44.809 INFO:tasks.workunit.client.0.smithi038.stdout:3/276: rename d3/d9/dc/c2c to d3/d9/dc/c60 0 2024-03-24T00:48:44.810 INFO:tasks.workunit.client.1.smithi088.stdout:4/302: dread d5/fc [0,4194304] 0 2024-03-24T00:48:44.813 INFO:tasks.workunit.client.0.smithi038.stdout:2/302: dwrite d4/d27/f39 [0,4194304] 0 2024-03-24T00:48:44.813 INFO:tasks.workunit.client.1.smithi088.stdout:8/273: dwrite d9/dc/d11/d24/d29/d5b/f34 [4194304,4194304] 0 2024-03-24T00:48:44.814 INFO:tasks.workunit.client.1.smithi088.stdout:0/295: mkdir d9/dc/d4f/d63 0 2024-03-24T00:48:44.821 INFO:tasks.workunit.client.1.smithi088.stdout:3/257: unlink d3/d1a/f37 0 2024-03-24T00:48:44.821 INFO:tasks.workunit.client.1.smithi088.stdout:3/258: fdatasync d3/d6/d27/d3c/f30 0 2024-03-24T00:48:44.821 INFO:tasks.workunit.client.1.smithi088.stdout:3/259: fdatasync d3/d6/d27/d3c/f30 0 2024-03-24T00:48:44.822 INFO:tasks.workunit.client.1.smithi088.stdout:2/241: dread f1 [0,4194304] 0 2024-03-24T00:48:44.823 INFO:tasks.workunit.client.1.smithi088.stdout:2/242: creat db/f4d x:0 0 0 2024-03-24T00:48:44.823 INFO:tasks.workunit.client.1.smithi088.stdout:7/245: dwrite d3/f1b [0,4194304] 0 2024-03-24T00:48:44.825 INFO:tasks.workunit.client.1.smithi088.stdout:6/303: creat db/d1d/f58 x:0 0 0 2024-03-24T00:48:44.825 INFO:tasks.workunit.client.1.smithi088.stdout:6/304: chown db/d1d/d23/d3c/d4b 37 1 2024-03-24T00:48:44.825 INFO:tasks.workunit.client.1.smithi088.stdout:6/305: fdatasync db/d1d/d23/f4c 0 2024-03-24T00:48:44.829 INFO:tasks.workunit.client.0.smithi038.stdout:8/246: dwrite d0/d15/d42/f40 [0,4194304] 0 2024-03-24T00:48:44.829 INFO:tasks.workunit.client.0.smithi038.stdout:8/247: rename d0/d45/d37 to d0/d45/d37/d56 22 2024-03-24T00:48:44.835 INFO:tasks.workunit.client.1.smithi088.stdout:9/272: symlink d1/d15/d47/l4b 0 2024-03-24T00:48:44.839 INFO:tasks.workunit.client.1.smithi088.stdout:4/303: symlink d5/d16/d1a/l67 0 2024-03-24T00:48:44.846 INFO:tasks.workunit.client.0.smithi038.stdout:0/250: link d0/d14/c36 d0/d12/d15/d1a/c52 0 2024-03-24T00:48:44.848 INFO:tasks.workunit.client.0.smithi038.stdout:3/277: truncate d3/d9/dc/df/f1d 2889568 0 2024-03-24T00:48:44.850 INFO:tasks.workunit.client.1.smithi088.stdout:9/273: dread d1/f4 [0,4194304] 0 2024-03-24T00:48:44.850 INFO:tasks.workunit.client.1.smithi088.stdout:9/274: chown d1/ff 778240980 1 2024-03-24T00:48:44.851 INFO:tasks.workunit.client.1.smithi088.stdout:9/275: write d1/d22/f43 [435756,3475] 0 2024-03-24T00:48:44.851 INFO:tasks.workunit.client.1.smithi088.stdout:3/260: dread d3/de/df/d49/f54 [0,4194304] 0 2024-03-24T00:48:44.853 INFO:tasks.workunit.client.0.smithi038.stdout:2/303: creat d4/d18/d26/d37/d69/f6c x:0 0 0 2024-03-24T00:48:44.861 INFO:tasks.workunit.client.0.smithi038.stdout:8/248: symlink d0/d15/d2c/l57 0 2024-03-24T00:48:44.861 INFO:tasks.workunit.client.0.smithi038.stdout:0/251: mknod d0/c53 0 2024-03-24T00:48:44.861 INFO:tasks.workunit.client.0.smithi038.stdout:0/252: fsync d0/d12/d15/f22 0 2024-03-24T00:48:44.861 INFO:tasks.workunit.client.0.smithi038.stdout:2/304: link d4/f50 d4/d18/d26/f6d 0 2024-03-24T00:48:44.863 INFO:tasks.workunit.client.0.smithi038.stdout:8/249: rename d0/d10/d19/f1b to d0/d1d/d20/f58 0 2024-03-24T00:48:44.863 INFO:tasks.workunit.client.0.smithi038.stdout:8/250: fsync d0/d45/d37/f44 0 2024-03-24T00:48:44.864 INFO:tasks.workunit.client.0.smithi038.stdout:8/251: fdatasync d0/d45/f55 0 2024-03-24T00:48:44.866 INFO:tasks.workunit.client.0.smithi038.stdout:0/253: symlink d0/d12/d42/d44/l54 0 2024-03-24T00:48:44.873 INFO:tasks.workunit.client.0.smithi038.stdout:9/308: sync 2024-03-24T00:48:44.876 INFO:tasks.workunit.client.0.smithi038.stdout:8/252: unlink d0/f24 0 2024-03-24T00:48:44.882 INFO:tasks.workunit.client.1.smithi088.stdout:1/297: dwrite d6/d27/f1a [0,4194304] 0 2024-03-24T00:48:44.882 INFO:tasks.workunit.client.1.smithi088.stdout:1/298: chown d6/d27/d1e/d3f/f4c 1 1 2024-03-24T00:48:44.883 INFO:tasks.workunit.client.0.smithi038.stdout:8/253: symlink d0/d1d/d27/l59 0 2024-03-24T00:48:44.886 INFO:tasks.workunit.client.1.smithi088.stdout:8/274: creat d9/dc/d11/d41/d54/d5d/f60 x:0 0 0 2024-03-24T00:48:44.886 INFO:tasks.workunit.client.1.smithi088.stdout:8/275: write d9/dc/f5c [752977,94175] 0 2024-03-24T00:48:44.886 INFO:tasks.workunit.client.1.smithi088.stdout:8/276: fsync d9/dc/d11/d24/d29/d5b/f53 0 2024-03-24T00:48:44.887 INFO:tasks.workunit.client.1.smithi088.stdout:0/296: symlink d9/d15/l64 0 2024-03-24T00:48:44.887 INFO:tasks.workunit.client.0.smithi038.stdout:4/239: write da/d12/f3f [3910423,63234] 0 2024-03-24T00:48:44.888 INFO:tasks.workunit.client.0.smithi038.stdout:8/254: mknod d0/d45/c5a 0 2024-03-24T00:48:44.891 INFO:tasks.workunit.client.1.smithi088.stdout:2/243: mkdir db/d21/d4a/d4e 0 2024-03-24T00:48:44.891 INFO:tasks.workunit.client.1.smithi088.stdout:2/244: fsync db/d21/f44 0 2024-03-24T00:48:44.891 INFO:tasks.workunit.client.1.smithi088.stdout:2/245: dread - db/f4d zero size 2024-03-24T00:48:44.892 INFO:tasks.workunit.client.1.smithi088.stdout:2/246: creat db/f4f x:0 0 0 2024-03-24T00:48:44.892 INFO:tasks.workunit.client.1.smithi088.stdout:2/247: fdatasync db/d21/f44 0 2024-03-24T00:48:44.894 INFO:tasks.workunit.client.0.smithi038.stdout:7/267: dwrite d2/d9/df/d33/d37/f4c [0,4194304] 0 2024-03-24T00:48:44.894 INFO:tasks.workunit.client.0.smithi038.stdout:4/240: rename da/c45 to da/d12/d37/dd/d15/d33/c48 0 2024-03-24T00:48:44.896 INFO:tasks.workunit.client.1.smithi088.stdout:7/246: rename d3/l26 to d3/d32/l53 0 2024-03-24T00:48:44.897 INFO:tasks.workunit.client.1.smithi088.stdout:6/306: symlink db/d40/d56/l59 0 2024-03-24T00:48:44.899 INFO:tasks.workunit.client.1.smithi088.stdout:9/276: symlink d1/d22/d27/l4c 0 2024-03-24T00:48:44.900 INFO:tasks.workunit.client.1.smithi088.stdout:9/277: write d1/f4 [5167520,29268] 0 2024-03-24T00:48:44.900 INFO:tasks.workunit.client.1.smithi088.stdout:9/278: chown d1/d10/c26 21143293 1 2024-03-24T00:48:44.900 INFO:tasks.workunit.client.1.smithi088.stdout:9/279: read d1/d10/d11/d28/f36 [366808,24493] 0 2024-03-24T00:48:44.902 INFO:tasks.workunit.client.0.smithi038.stdout:7/268: symlink d2/d9/d16/l60 0 2024-03-24T00:48:44.903 INFO:tasks.workunit.client.0.smithi038.stdout:4/241: mknod da/d12/d29/c49 0 2024-03-24T00:48:44.904 INFO:tasks.workunit.client.0.smithi038.stdout:4/242: truncate da/d38/f41 991208 0 2024-03-24T00:48:44.904 INFO:tasks.workunit.client.0.smithi038.stdout:4/243: stat da/d12/d29 0 2024-03-24T00:48:44.904 INFO:tasks.workunit.client.0.smithi038.stdout:4/244: creat da/d38/f4a x:0 0 0 2024-03-24T00:48:44.905 INFO:tasks.workunit.client.0.smithi038.stdout:4/245: read da/d12/d37/dd/f2b [86909,80034] 0 2024-03-24T00:48:44.906 INFO:tasks.workunit.client.0.smithi038.stdout:7/269: mknod d2/d9/df/d33/d37/c61 0 2024-03-24T00:48:44.906 INFO:tasks.workunit.client.0.smithi038.stdout:7/270: read - d2/d2c/f56 zero size 2024-03-24T00:48:44.907 INFO:tasks.workunit.client.1.smithi088.stdout:9/280: dread d1/d22/f3f [0,4194304] 0 2024-03-24T00:48:44.907 INFO:tasks.workunit.client.1.smithi088.stdout:9/281: dread - d1/d10/d2c/f35 zero size 2024-03-24T00:48:44.908 INFO:tasks.workunit.client.1.smithi088.stdout:8/277: creat d9/dc/d11/d41/f61 x:0 0 0 2024-03-24T00:48:44.909 INFO:tasks.workunit.client.0.smithi038.stdout:4/246: creat da/d12/d37/dd/d15/d33/f4b x:0 0 0 2024-03-24T00:48:44.911 INFO:tasks.workunit.client.1.smithi088.stdout:6/307: stat db/d40/d56/c44 0 2024-03-24T00:48:44.911 INFO:tasks.workunit.client.1.smithi088.stdout:6/308: dread - db/d1d/d23/f4c zero size 2024-03-24T00:48:44.911 INFO:tasks.workunit.client.1.smithi088.stdout:6/309: dread - db/d40/f4d zero size 2024-03-24T00:48:44.911 INFO:tasks.workunit.client.1.smithi088.stdout:6/310: truncate db/d40/f4d 422224 0 2024-03-24T00:48:44.911 INFO:tasks.workunit.client.1.smithi088.stdout:6/311: creat db/d40/f5a x:0 0 0 2024-03-24T00:48:44.913 INFO:tasks.workunit.client.0.smithi038.stdout:4/247: unlink da/d12/d37/dd/d15/f42 0 2024-03-24T00:48:44.914 INFO:tasks.workunit.client.1.smithi088.stdout:3/261: mkdir d3/d6/d58 0 2024-03-24T00:48:44.916 INFO:tasks.workunit.client.0.smithi038.stdout:4/248: mkdir da/d12/d37/dd/d15/d4c 0 2024-03-24T00:48:44.917 INFO:tasks.workunit.client.0.smithi038.stdout:4/249: creat da/d38/f4d x:0 0 0 2024-03-24T00:48:44.917 INFO:tasks.workunit.client.0.smithi038.stdout:4/250: creat da/d12/f4e x:0 0 0 2024-03-24T00:48:44.918 INFO:tasks.workunit.client.1.smithi088.stdout:6/312: mkdir db/d1d/d23/d3c/d43/d5b 0 2024-03-24T00:48:44.918 INFO:tasks.workunit.client.1.smithi088.stdout:6/313: fsync db/f13 0 2024-03-24T00:48:44.918 INFO:tasks.workunit.client.1.smithi088.stdout:3/262: creat d3/d6/d27/d3c/f59 x:0 0 0 2024-03-24T00:48:44.920 INFO:tasks.workunit.client.1.smithi088.stdout:6/314: mkdir db/d1d/d23/d5c 0 2024-03-24T00:48:44.922 INFO:tasks.workunit.client.1.smithi088.stdout:1/299: dwrite d6/d27/d1e/d3f/f41 [0,4194304] 0 2024-03-24T00:48:44.923 INFO:tasks.workunit.client.1.smithi088.stdout:2/248: dread db/d21/d35/f3d [0,4194304] 0 2024-03-24T00:48:44.923 INFO:tasks.workunit.client.1.smithi088.stdout:2/249: stat db/l12 0 2024-03-24T00:48:44.923 INFO:tasks.workunit.client.1.smithi088.stdout:2/250: write db/f38 [1231653,124445] 0 2024-03-24T00:48:44.924 INFO:tasks.workunit.client.1.smithi088.stdout:3/263: creat d3/d1a/f5a x:0 0 0 2024-03-24T00:48:44.924 INFO:tasks.workunit.client.1.smithi088.stdout:6/315: mknod db/d1d/d23/d5c/c5d 0 2024-03-24T00:48:44.924 INFO:tasks.workunit.client.1.smithi088.stdout:3/264: write d3/d1a/d1e/d2f/f36 [1012255,98056] 0 2024-03-24T00:48:44.925 INFO:tasks.workunit.client.1.smithi088.stdout:3/265: write d3/d6/d27/d3c/f59 [971775,91118] 0 2024-03-24T00:48:44.925 INFO:tasks.workunit.client.1.smithi088.stdout:1/300: getdents d6/d27 0 2024-03-24T00:48:44.927 INFO:tasks.workunit.client.1.smithi088.stdout:3/266: write f1 [4302808,51041] 0 2024-03-24T00:48:44.927 INFO:tasks.workunit.client.1.smithi088.stdout:2/251: link db/dc/l23 db/dc/l50 0 2024-03-24T00:48:44.927 INFO:tasks.workunit.client.1.smithi088.stdout:2/252: dread - db/f4f zero size 2024-03-24T00:48:44.927 INFO:tasks.workunit.client.1.smithi088.stdout:2/253: readlink db/dd/l1e 0 2024-03-24T00:48:44.927 INFO:tasks.workunit.client.1.smithi088.stdout:1/301: rename d6/d23/f52 to d6/d27/d1e/d3f/f54 0 2024-03-24T00:48:44.929 INFO:tasks.workunit.client.1.smithi088.stdout:1/302: mknod d6/d27/d1e/d3f/c55 0 2024-03-24T00:48:44.953 INFO:tasks.workunit.client.0.smithi038.stdout:9/309: dread d12/d28/f36 [0,4194304] 0 2024-03-24T00:48:44.956 INFO:tasks.workunit.client.0.smithi038.stdout:9/310: mkdir d12/d28/d6c 0 2024-03-24T00:48:44.956 INFO:tasks.workunit.client.0.smithi038.stdout:9/311: creat d12/d28/d3a/f6d x:0 0 0 2024-03-24T00:48:44.956 INFO:tasks.workunit.client.0.smithi038.stdout:9/312: creat d12/d55/f6e x:0 0 0 2024-03-24T00:48:44.956 INFO:tasks.workunit.client.0.smithi038.stdout:9/313: truncate d12/d28/d3a/f48 254181 0 2024-03-24T00:48:44.959 INFO:tasks.workunit.client.0.smithi038.stdout:9/314: rmdir d12/d41/d57 39 2024-03-24T00:48:44.967 INFO:tasks.workunit.client.1.smithi088.stdout:0/297: dwrite d9/d20/f58 [0,4194304] 0 2024-03-24T00:48:44.967 INFO:tasks.workunit.client.1.smithi088.stdout:0/298: chown d9/d20/d2a/d2e/f5f 15124051 1 2024-03-24T00:48:44.967 INFO:tasks.workunit.client.1.smithi088.stdout:0/299: getdents d9/dc/de/d19/d44/d46/d59 0 2024-03-24T00:48:44.973 INFO:tasks.workunit.client.0.smithi038.stdout:9/315: dread d12/d28/d3a/f45 [0,4194304] 0 2024-03-24T00:48:44.973 INFO:tasks.workunit.client.0.smithi038.stdout:9/316: chown d12/d41/f5b 4056 1 2024-03-24T00:48:44.973 INFO:tasks.workunit.client.0.smithi038.stdout:9/317: fsync d12/d28/d3a/f2b 0 2024-03-24T00:48:44.975 INFO:tasks.workunit.client.0.smithi038.stdout:9/318: link d12/d55/f67 d12/f6f 0 2024-03-24T00:48:44.978 INFO:tasks.workunit.client.0.smithi038.stdout:0/254: dwrite d0/fa [8388608,4194304] 0 2024-03-24T00:48:44.982 INFO:tasks.workunit.client.0.smithi038.stdout:0/255: rename d0/d12/c32 to d0/d14/c55 0 2024-03-24T00:48:44.987 INFO:tasks.workunit.client.0.smithi038.stdout:0/256: unlink d0/d12/d13/d28/c35 0 2024-03-24T00:48:44.987 INFO:tasks.workunit.client.0.smithi038.stdout:0/257: chown d0/d12/d42/d44/l54 31025 1 2024-03-24T00:48:44.987 INFO:tasks.workunit.client.0.smithi038.stdout:0/258: truncate d0/d1b/f4a 433558 0 2024-03-24T00:48:44.987 INFO:tasks.workunit.client.0.smithi038.stdout:0/259: chown d0/d12/d15/d1a/f2b 83686413 1 2024-03-24T00:48:44.989 INFO:tasks.workunit.client.0.smithi038.stdout:0/260: symlink d0/d1b/l56 0 2024-03-24T00:48:44.989 INFO:tasks.workunit.client.0.smithi038.stdout:0/261: chown d0/d12/d15/d1a/c4d 46293 1 2024-03-24T00:48:44.996 INFO:tasks.workunit.client.0.smithi038.stdout:8/255: dwrite d0/d1d/f3e [0,4194304] 0 2024-03-24T00:48:44.996 INFO:tasks.workunit.client.0.smithi038.stdout:8/256: dread - d0/d45/d37/f48 zero size 2024-03-24T00:48:44.997 INFO:tasks.workunit.client.0.smithi038.stdout:8/257: fdatasync d0/f16 0 2024-03-24T00:48:44.997 INFO:tasks.workunit.client.0.smithi038.stdout:8/258: chown d0/d1d/d20/l3d 1 1 2024-03-24T00:48:44.997 INFO:tasks.workunit.client.0.smithi038.stdout:8/259: creat d0/d15/f5b x:0 0 0 2024-03-24T00:48:44.998 INFO:tasks.workunit.client.1.smithi088.stdout:1/303: dwrite d6/d2f/f32 [0,4194304] 0 2024-03-24T00:48:45.001 INFO:tasks.workunit.client.0.smithi038.stdout:8/260: rename d0/c3b to d0/d10/c5c 0 2024-03-24T00:48:45.002 INFO:tasks.workunit.client.1.smithi088.stdout:6/316: dwrite db/f13 [4194304,4194304] 0 2024-03-24T00:48:45.002 INFO:tasks.workunit.client.1.smithi088.stdout:6/317: write db/f13 [3374459,56443] 0 2024-03-24T00:48:45.004 INFO:tasks.workunit.client.0.smithi038.stdout:8/261: rename d0/d15/d42/l3a to d0/d1d/d27/l5d 0 2024-03-24T00:48:45.007 INFO:tasks.workunit.client.0.smithi038.stdout:8/262: rename d0/d45/c5a to d0/d1d/c5e 0 2024-03-24T00:48:45.009 INFO:tasks.workunit.client.1.smithi088.stdout:6/318: symlink db/d1d/d23/l5e 0 2024-03-24T00:48:45.009 INFO:tasks.workunit.client.1.smithi088.stdout:6/319: chown db/d40/d56/f31 2573633 1 2024-03-24T00:48:45.011 INFO:tasks.workunit.client.0.smithi038.stdout:8/263: mkdir d0/d45/d37/d5f 0 2024-03-24T00:48:45.011 INFO:tasks.workunit.client.0.smithi038.stdout:8/264: write d0/d1d/d20/f23 [3077509,7192] 0 2024-03-24T00:48:45.013 INFO:tasks.workunit.client.0.smithi038.stdout:8/265: rename d0/d15/f5b to d0/d1d/d20/f60 0 2024-03-24T00:48:45.014 INFO:tasks.workunit.client.1.smithi088.stdout:9/282: dwrite d1/d10/d11/f17 [0,4194304] 0 2024-03-24T00:48:45.014 INFO:tasks.workunit.client.0.smithi038.stdout:8/266: fdatasync d0/d45/f55 0 2024-03-24T00:48:45.015 INFO:tasks.workunit.client.1.smithi088.stdout:6/320: unlink db/d1d/l49 0 2024-03-24T00:48:45.016 INFO:tasks.workunit.client.0.smithi038.stdout:5/235: dwrite f5 [0,4194304] 0 2024-03-24T00:48:45.016 INFO:tasks.workunit.client.1.smithi088.stdout:8/278: dread d9/dc/d11/d24/d29/d36/f46 [0,4194304] 0 2024-03-24T00:48:45.016 INFO:tasks.workunit.client.1.smithi088.stdout:8/279: creat d9/dc/d11/d41/d54/d5d/f62 x:0 0 0 2024-03-24T00:48:45.020 INFO:tasks.workunit.client.1.smithi088.stdout:4/304: dwrite f2 [0,4194304] 0 2024-03-24T00:48:45.020 INFO:tasks.workunit.client.1.smithi088.stdout:0/300: dwrite d9/dc/de/f1d [0,4194304] 0 2024-03-24T00:48:45.024 INFO:tasks.workunit.client.1.smithi088.stdout:2/254: dread db/f3c [0,4194304] 0 2024-03-24T00:48:45.024 INFO:tasks.workunit.client.1.smithi088.stdout:3/267: dwrite d3/d6/d27/d3c/f47 [0,4194304] 0 2024-03-24T00:48:45.027 INFO:tasks.workunit.client.1.smithi088.stdout:9/283: creat d1/d22/d27/d39/f4d x:0 0 0 2024-03-24T00:48:45.031 INFO:tasks.workunit.client.1.smithi088.stdout:6/321: rename db/d1d/d23/d3c/d43/c2e to db/c5f 0 2024-03-24T00:48:45.031 INFO:tasks.workunit.client.0.smithi038.stdout:2/305: rmdir d4/d18/d26/d37/d69 39 2024-03-24T00:48:45.037 INFO:tasks.workunit.client.0.smithi038.stdout:5/236: rename d6/d19/d20/d42/d43/c44 to d6/d16/d21/c47 0 2024-03-24T00:48:45.040 INFO:tasks.workunit.client.0.smithi038.stdout:5/237: mkdir d6/d19/d20/d42/d43/d48 0 2024-03-24T00:48:45.043 INFO:tasks.workunit.client.0.smithi038.stdout:9/319: dwrite d12/d28/f4c [0,4194304] 0 2024-03-24T00:48:45.045 INFO:tasks.workunit.client.1.smithi088.stdout:1/304: dwrite d6/d23/f2d [0,4194304] 0 2024-03-24T00:48:45.049 INFO:tasks.workunit.client.0.smithi038.stdout:9/320: getdents d12/d41/d57 0 2024-03-24T00:48:45.049 INFO:tasks.workunit.client.0.smithi038.stdout:9/321: truncate d12/d28/d3a/f3f 996727 0 2024-03-24T00:48:45.055 INFO:tasks.workunit.client.1.smithi088.stdout:2/255: symlink db/d21/d35/d49/l51 0 2024-03-24T00:48:45.056 INFO:tasks.workunit.client.1.smithi088.stdout:2/256: truncate db/f4d 879281 0 2024-03-24T00:48:45.057 INFO:tasks.workunit.client.1.smithi088.stdout:6/322: dread db/dd/f10 [0,4194304] 0 2024-03-24T00:48:45.059 INFO:tasks.workunit.client.1.smithi088.stdout:9/284: mknod d1/d10/d11/c4e 0 2024-03-24T00:48:45.059 INFO:tasks.workunit.client.1.smithi088.stdout:9/285: read - d1/d10/d2c/f35 zero size 2024-03-24T00:48:45.061 INFO:tasks.workunit.client.0.smithi038.stdout:9/322: rename d12/d28/d3a/f48 to d12/d28/f70 0 2024-03-24T00:48:45.063 INFO:tasks.workunit.client.1.smithi088.stdout:4/305: rename d5/d13/d18/f2c to d5/d16/d1a/d22/f68 0 2024-03-24T00:48:45.064 INFO:tasks.workunit.client.1.smithi088.stdout:0/301: dread d9/dc/de/f11 [0,4194304] 0 2024-03-24T00:48:45.064 INFO:tasks.workunit.client.1.smithi088.stdout:8/280: dwrite d9/dc/d11/d41/f61 [0,4194304] 0 2024-03-24T00:48:45.070 INFO:tasks.workunit.client.0.smithi038.stdout:9/323: mkdir d12/d28/d71 0 2024-03-24T00:48:45.077 INFO:tasks.workunit.client.0.smithi038.stdout:9/324: truncate fe 2749215 0 2024-03-24T00:48:45.077 INFO:tasks.workunit.client.0.smithi038.stdout:9/325: chown d12/d28/f70 27 1 2024-03-24T00:48:45.079 INFO:tasks.workunit.client.0.smithi038.stdout:9/326: symlink d12/d55/d56/l72 0 2024-03-24T00:48:45.080 INFO:tasks.workunit.client.1.smithi088.stdout:9/286: getdents d1/d15 0 2024-03-24T00:48:45.080 INFO:tasks.workunit.client.1.smithi088.stdout:9/287: chown d1/c2 8570598 1 2024-03-24T00:48:45.080 INFO:tasks.workunit.client.1.smithi088.stdout:9/288: chown d1/d10/f1e 120 1 2024-03-24T00:48:45.080 INFO:tasks.workunit.client.1.smithi088.stdout:9/289: fsync d1/d22/f49 0 2024-03-24T00:48:45.080 INFO:tasks.workunit.client.1.smithi088.stdout:1/305: rename d6/d27/d1e/l30 to d6/d27/d1e/d3f/d53/l56 0 2024-03-24T00:48:45.080 INFO:tasks.workunit.client.1.smithi088.stdout:4/306: creat d5/d7/d15/d60/f69 x:0 0 0 2024-03-24T00:48:45.081 INFO:tasks.workunit.client.1.smithi088.stdout:3/268: dread d3/d6/f11 [0,4194304] 0 2024-03-24T00:48:45.081 INFO:tasks.workunit.client.1.smithi088.stdout:3/269: fsync d3/d1a/d1e/f28 0 2024-03-24T00:48:45.082 INFO:tasks.workunit.client.1.smithi088.stdout:5/280: sync 2024-03-24T00:48:45.083 INFO:tasks.workunit.client.1.smithi088.stdout:2/257: write db/f3c [3196956,107711] 0 2024-03-24T00:48:45.083 INFO:tasks.workunit.client.1.smithi088.stdout:2/258: dread - db/f43 zero size 2024-03-24T00:48:45.085 INFO:tasks.workunit.client.1.smithi088.stdout:0/302: rename d9/d20 to d9/dc/de/d19/d44/d46/d65 0 2024-03-24T00:48:45.085 INFO:tasks.workunit.client.1.smithi088.stdout:9/290: creat d1/d10/d11/d28/f4f x:0 0 0 2024-03-24T00:48:45.086 INFO:tasks.workunit.client.1.smithi088.stdout:4/307: unlink d5/d7/ca 0 2024-03-24T00:48:45.089 INFO:tasks.workunit.client.1.smithi088.stdout:5/281: mknod d1/d29/d3f/c59 0 2024-03-24T00:48:45.092 INFO:tasks.workunit.client.0.smithi038.stdout:8/267: dwrite d0/f32 [0,4194304] 0 2024-03-24T00:48:45.092 INFO:tasks.workunit.client.0.smithi038.stdout:9/327: dread d12/d28/d3a/d26/f3d [0,4194304] 0 2024-03-24T00:48:45.093 INFO:tasks.workunit.client.1.smithi088.stdout:8/281: rename d9/dc/d11/d41/f61 to d9/f63 0 2024-03-24T00:48:45.098 INFO:tasks.workunit.client.1.smithi088.stdout:8/282: write d9/f3c [395302,15722] 0 2024-03-24T00:48:45.098 INFO:tasks.workunit.client.1.smithi088.stdout:0/303: rmdir d9/d15 39 2024-03-24T00:48:45.099 INFO:tasks.workunit.client.1.smithi088.stdout:2/259: unlink db/d21/c2a 0 2024-03-24T00:48:45.099 INFO:tasks.workunit.client.1.smithi088.stdout:1/306: write d6/d27/f3a [29406,972] 0 2024-03-24T00:48:45.101 INFO:tasks.workunit.client.0.smithi038.stdout:9/328: mknod d12/d28/d3a/c73 0 2024-03-24T00:48:45.108 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:44 smithi038 ceph-mon[29271]: pgmap v28: 65 pgs: 65 active+clean; 774 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 71 MiB/s rd, 82 MiB/s wr, 419 op/s 2024-03-24T00:48:45.110 INFO:tasks.workunit.client.0.smithi038.stdout:8/268: dread d0/d1d/d20/f58 [0,4194304] 0 2024-03-24T00:48:45.111 INFO:tasks.workunit.client.1.smithi088.stdout:6/323: dwrite db/f19 [0,4194304] 0 2024-03-24T00:48:45.114 INFO:tasks.workunit.client.0.smithi038.stdout:5/238: dwrite d6/d19/f1c [4194304,4194304] 0 2024-03-24T00:48:45.152 INFO:tasks.workunit.client.1.smithi088.stdout:3/270: dwrite d3/d6/d27/d3c/f47 [0,4194304] 0 2024-03-24T00:48:45.154 INFO:tasks.workunit.client.0.smithi038.stdout:8/269: dwrite d0/d15/f17 [0,4194304] 0 2024-03-24T00:48:45.165 INFO:tasks.workunit.client.1.smithi088.stdout:8/283: dread d9/f28 [0,4194304] 0 2024-03-24T00:48:45.167 INFO:tasks.workunit.client.1.smithi088.stdout:6/324: dwrite db/d1d/f48 [0,4194304] 0 2024-03-24T00:48:45.170 INFO:tasks.workunit.client.1.smithi088.stdout:2/260: dread db/dd/f28 [0,4194304] 0 2024-03-24T00:48:45.170 INFO:tasks.workunit.client.1.smithi088.stdout:2/261: readlink db/d21/d35/d49/l51 0 2024-03-24T00:48:45.177 INFO:tasks.workunit.client.1.smithi088.stdout:4/308: dwrite d5/f4d [0,4194304] 0 2024-03-24T00:48:45.177 INFO:tasks.workunit.client.1.smithi088.stdout:5/282: dwrite d1/d10/f52 [0,4194304] 0 2024-03-24T00:48:45.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:44 smithi088 ceph-mon[36610]: pgmap v28: 65 pgs: 65 active+clean; 774 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 71 MiB/s rd, 82 MiB/s wr, 419 op/s 2024-03-24T00:48:45.184 INFO:tasks.workunit.client.0.smithi038.stdout:9/329: rename d12/f69 to d12/d28/d6c/f74 0 2024-03-24T00:48:45.187 INFO:tasks.workunit.client.1.smithi088.stdout:1/307: creat d6/d27/d1e/d3f/f57 x:0 0 0 2024-03-24T00:48:45.188 INFO:tasks.workunit.client.0.smithi038.stdout:5/239: mkdir d6/d19/d20/d33/d49 0 2024-03-24T00:48:45.190 INFO:tasks.workunit.client.1.smithi088.stdout:4/309: write d5/d16/f2e [2556432,95736] 0 2024-03-24T00:48:45.191 INFO:tasks.workunit.client.0.smithi038.stdout:8/270: creat d0/d45/d37/f61 x:0 0 0 2024-03-24T00:48:45.195 INFO:tasks.workunit.client.1.smithi088.stdout:8/284: mknod d9/dc/d11/d41/d54/c64 0 2024-03-24T00:48:45.197 INFO:tasks.workunit.client.1.smithi088.stdout:6/325: truncate db/d40/d56/f31 234429 0 2024-03-24T00:48:45.198 INFO:tasks.workunit.client.1.smithi088.stdout:6/326: readlink db/d1d/d23/l53 0 2024-03-24T00:48:45.201 INFO:tasks.workunit.client.0.smithi038.stdout:9/330: mkdir d12/d28/d6c/d75 0 2024-03-24T00:48:45.202 INFO:tasks.workunit.client.1.smithi088.stdout:2/262: truncate db/f20 2526809 0 2024-03-24T00:48:45.202 INFO:tasks.workunit.client.1.smithi088.stdout:2/263: read db/d21/f45 [1100628,1206] 0 2024-03-24T00:48:45.202 INFO:tasks.workunit.client.1.smithi088.stdout:2/264: stat db/dd/f28 0 2024-03-24T00:48:45.206 INFO:tasks.workunit.client.0.smithi038.stdout:8/271: mkdir d0/d10/d19/d62 0 2024-03-24T00:48:45.209 INFO:tasks.workunit.client.0.smithi038.stdout:5/240: dread d6/d23/f3b [0,4194304] 0 2024-03-24T00:48:45.210 INFO:tasks.workunit.client.0.smithi038.stdout:9/331: symlink d12/d41/d57/l76 0 2024-03-24T00:48:45.210 INFO:tasks.workunit.client.0.smithi038.stdout:9/332: fdatasync d12/d41/f6a 0 2024-03-24T00:48:45.213 INFO:tasks.workunit.client.0.smithi038.stdout:5/241: rename d6/l34 to d6/d16/d21/d2d/l4a 0 2024-03-24T00:48:45.214 INFO:tasks.workunit.client.1.smithi088.stdout:4/310: write d5/d17/f33 [1091345,90302] 0 2024-03-24T00:48:45.214 INFO:tasks.workunit.client.1.smithi088.stdout:4/311: dread - d5/d16/f57 zero size 2024-03-24T00:48:45.215 INFO:tasks.workunit.client.1.smithi088.stdout:5/283: link d1/d19/c3a d1/d10/d1b/d2f/d57/c5a 0 2024-03-24T00:48:45.218 INFO:tasks.workunit.client.1.smithi088.stdout:1/308: unlink d6/d27/l20 0 2024-03-24T00:48:45.218 INFO:tasks.workunit.client.1.smithi088.stdout:1/309: write d6/f8 [6023480,839] 0 2024-03-24T00:48:45.218 INFO:tasks.workunit.client.1.smithi088.stdout:1/310: chown d6/d27/l36 131 1 2024-03-24T00:48:45.221 INFO:tasks.workunit.client.1.smithi088.stdout:3/271: link d3/d6/f3b d3/d1a/d1e/d2f/f5b 0 2024-03-24T00:48:45.222 INFO:tasks.workunit.client.0.smithi038.stdout:8/272: dread d0/d1d/d20/f58 [0,4194304] 0 2024-03-24T00:48:45.222 INFO:tasks.workunit.client.0.smithi038.stdout:8/273: dread - d0/d45/d37/f61 zero size 2024-03-24T00:48:45.224 INFO:tasks.workunit.client.1.smithi088.stdout:8/285: mkdir d9/dc/d11/d41/d65 0 2024-03-24T00:48:45.224 INFO:tasks.workunit.client.1.smithi088.stdout:8/286: readlink d9/dc/lf 0 2024-03-24T00:48:45.225 INFO:tasks.workunit.client.1.smithi088.stdout:8/287: dread - d9/dc/f52 zero size 2024-03-24T00:48:45.225 INFO:tasks.workunit.client.1.smithi088.stdout:8/288: write d9/dc/d11/d41/d54/d5d/f62 [205471,80002] 0 2024-03-24T00:48:45.225 INFO:tasks.workunit.client.1.smithi088.stdout:6/327: mkdir db/d40/d60 0 2024-03-24T00:48:45.226 INFO:tasks.workunit.client.1.smithi088.stdout:2/265: symlink db/d21/d4a/l52 0 2024-03-24T00:48:45.227 INFO:tasks.workunit.client.1.smithi088.stdout:4/312: mknod d5/c6a 0 2024-03-24T00:48:45.229 INFO:tasks.workunit.client.1.smithi088.stdout:5/284: mkdir d1/d43/d4e/d5b 0 2024-03-24T00:48:45.231 INFO:tasks.workunit.client.1.smithi088.stdout:1/311: rename d6/d2f to d6/d27/d1e/d58 0 2024-03-24T00:48:45.231 INFO:tasks.workunit.client.1.smithi088.stdout:1/312: stat d6/d27/d1e/d58/f32 0 2024-03-24T00:48:45.233 INFO:tasks.workunit.client.1.smithi088.stdout:3/272: link d3/d6/d19/d2a/l23 d3/d1a/d1e/d48/l5c 0 2024-03-24T00:48:45.233 INFO:tasks.workunit.client.1.smithi088.stdout:8/289: write d9/f2b [13861,83100] 0 2024-03-24T00:48:45.234 INFO:tasks.workunit.client.0.smithi038.stdout:6/317: sync 2024-03-24T00:48:45.234 INFO:tasks.workunit.client.0.smithi038.stdout:1/219: sync 2024-03-24T00:48:45.234 INFO:tasks.workunit.client.0.smithi038.stdout:6/318: chown d1/da/d1b/d1d/f2c 0 1 2024-03-24T00:48:45.234 INFO:tasks.workunit.client.1.smithi088.stdout:2/266: stat db/dc/l23 0 2024-03-24T00:48:45.235 INFO:tasks.workunit.client.1.smithi088.stdout:4/313: mknod d5/d13/d18/d24/c6b 0 2024-03-24T00:48:45.236 INFO:tasks.workunit.client.1.smithi088.stdout:4/314: write d5/fc [1656494,72028] 0 2024-03-24T00:48:45.240 INFO:tasks.workunit.client.1.smithi088.stdout:1/313: symlink d6/d27/d1e/l59 0 2024-03-24T00:48:45.240 INFO:tasks.workunit.client.1.smithi088.stdout:5/285: creat d1/d43/d4e/f5c x:0 0 0 2024-03-24T00:48:45.243 INFO:tasks.workunit.client.1.smithi088.stdout:3/273: creat d3/d1a/d1e/d2f/d45/f5d x:0 0 0 2024-03-24T00:48:45.243 INFO:tasks.workunit.client.1.smithi088.stdout:2/267: truncate db/d21/f2e 344581 0 2024-03-24T00:48:45.243 INFO:tasks.workunit.client.1.smithi088.stdout:2/268: fsync db/f4d 0 2024-03-24T00:48:45.243 INFO:tasks.workunit.client.0.smithi038.stdout:5/242: write f5 [2010350,1719] 0 2024-03-24T00:48:45.246 INFO:tasks.workunit.client.1.smithi088.stdout:4/315: mknod d5/d16/c6c 0 2024-03-24T00:48:45.249 INFO:tasks.workunit.client.0.smithi038.stdout:1/220: dread da/f23 [0,4194304] 0 2024-03-24T00:48:45.251 INFO:tasks.workunit.client.1.smithi088.stdout:5/286: mknod d1/d10/d4f/c5d 0 2024-03-24T00:48:45.251 INFO:tasks.workunit.client.1.smithi088.stdout:5/287: write d1/f3b [2597276,80246] 0 2024-03-24T00:48:45.252 INFO:tasks.workunit.client.0.smithi038.stdout:5/243: mkdir d6/d19/d20/d42/d4b 0 2024-03-24T00:48:45.256 INFO:tasks.workunit.client.0.smithi038.stdout:1/221: rename da/dd/d10/d34/f3d to da/f49 0 2024-03-24T00:48:45.256 INFO:tasks.workunit.client.0.smithi038.stdout:1/222: chown da/dd/d22/f25 2 1 2024-03-24T00:48:45.256 INFO:tasks.workunit.client.0.smithi038.stdout:1/223: chown f7 148413661 1 2024-03-24T00:48:45.261 INFO:tasks.workunit.client.0.smithi038.stdout:1/224: write da/f21 [652167,34721] 0 2024-03-24T00:48:45.263 INFO:tasks.workunit.client.0.smithi038.stdout:5/244: mkdir d6/d19/d20/d42/d43/d4c 0 2024-03-24T00:48:45.263 INFO:tasks.workunit.client.0.smithi038.stdout:5/245: stat d6/d23/f3b 0 2024-03-24T00:48:45.265 INFO:tasks.workunit.client.0.smithi038.stdout:1/225: mkdir da/dd/d4a 0 2024-03-24T00:48:45.265 INFO:tasks.workunit.client.1.smithi088.stdout:9/291: getdents d1/d10/d11 0 2024-03-24T00:48:45.265 INFO:tasks.workunit.client.1.smithi088.stdout:9/292: rename d1/d10/d11 to d1/d10/d11/d28/d50 22 2024-03-24T00:48:45.265 INFO:tasks.workunit.client.1.smithi088.stdout:9/293: truncate d1/d10/f34 753367 0 2024-03-24T00:48:45.268 INFO:tasks.workunit.client.1.smithi088.stdout:3/274: read d3/d6/d27/d3c/f30 [2624325,26885] 0 2024-03-24T00:48:45.268 INFO:tasks.workunit.client.1.smithi088.stdout:3/275: chown f1 2 1 2024-03-24T00:48:45.270 INFO:tasks.workunit.client.0.smithi038.stdout:5/246: mkdir d6/d19/d4d 0 2024-03-24T00:48:45.270 INFO:tasks.workunit.client.0.smithi038.stdout:5/247: readlink d6/d16/d21/d2d/l40 0 2024-03-24T00:48:45.272 INFO:tasks.workunit.client.0.smithi038.stdout:1/226: truncate da/dd/d10/f15 7933079 0 2024-03-24T00:48:45.275 INFO:tasks.workunit.client.1.smithi088.stdout:9/294: rmdir d1/d15/d44 0 2024-03-24T00:48:45.275 INFO:tasks.workunit.client.1.smithi088.stdout:9/295: truncate d1/d10/d29/f45 626084 0 2024-03-24T00:48:45.275 INFO:tasks.workunit.client.1.smithi088.stdout:9/296: write d1/d22/f43 [807323,52335] 0 2024-03-24T00:48:45.277 INFO:tasks.workunit.client.0.smithi038.stdout:5/248: rename d6/d16/d21/c3e to d6/d19/d20/d42/d43/d48/c4e 0 2024-03-24T00:48:45.283 INFO:tasks.workunit.client.0.smithi038.stdout:1/227: symlink da/l4b 0 2024-03-24T00:48:45.283 INFO:tasks.workunit.client.0.smithi038.stdout:1/228: truncate da/dd/f2f 221818 0 2024-03-24T00:48:45.283 INFO:tasks.workunit.client.0.smithi038.stdout:9/333: dwrite d12/d28/d3a/d26/f3d [0,4194304] 0 2024-03-24T00:48:45.283 INFO:tasks.workunit.client.0.smithi038.stdout:9/334: read - d12/d28/d3a/f4e zero size 2024-03-24T00:48:45.283 INFO:tasks.workunit.client.0.smithi038.stdout:9/335: creat d12/d28/d3a/f77 x:0 0 0 2024-03-24T00:48:45.285 INFO:tasks.workunit.client.1.smithi088.stdout:9/297: rmdir d1 39 2024-03-24T00:48:45.286 INFO:tasks.workunit.client.1.smithi088.stdout:3/276: dread d3/d6/d27/d3c/f39 [0,4194304] 0 2024-03-24T00:48:45.288 INFO:tasks.workunit.client.0.smithi038.stdout:5/249: creat d6/d19/d20/d42/d4b/f4f x:0 0 0 2024-03-24T00:48:45.290 INFO:tasks.workunit.client.1.smithi088.stdout:6/328: dwrite db/d40/d56/f30 [0,4194304] 0 2024-03-24T00:48:45.290 INFO:tasks.workunit.client.1.smithi088.stdout:7/247: sync 2024-03-24T00:48:45.290 INFO:tasks.workunit.client.1.smithi088.stdout:7/248: chown d3/f2f 722725284 1 2024-03-24T00:48:45.292 INFO:tasks.workunit.client.0.smithi038.stdout:5/250: mkdir d6/d19/d20/d42/d43/d50 0 2024-03-24T00:48:45.297 INFO:tasks.workunit.client.1.smithi088.stdout:1/314: dwrite d6/d23/f34 [0,4194304] 0 2024-03-24T00:48:45.298 INFO:tasks.workunit.client.0.smithi038.stdout:5/251: link d6/d19/d20/l30 d6/d19/d20/d42/d43/d50/l51 0 2024-03-24T00:48:45.301 INFO:tasks.workunit.client.0.smithi038.stdout:5/252: creat d6/d19/d20/d42/f52 x:0 0 0 2024-03-24T00:48:45.301 INFO:tasks.workunit.client.0.smithi038.stdout:5/253: chown d6/d1e 140 1 2024-03-24T00:48:45.302 INFO:tasks.workunit.client.0.smithi038.stdout:5/254: stat d6/d19/d20/d42/d43/d50/l51 0 2024-03-24T00:48:45.302 INFO:tasks.workunit.client.0.smithi038.stdout:5/255: chown d6/d19/d20/c25 535099 1 2024-03-24T00:48:45.306 INFO:tasks.workunit.client.0.smithi038.stdout:5/256: unlink d6/d16/d3c/l45 0 2024-03-24T00:48:45.306 INFO:tasks.workunit.client.0.smithi038.stdout:5/257: fsync d6/f3f 0 2024-03-24T00:48:45.306 INFO:tasks.workunit.client.0.smithi038.stdout:5/258: stat d6 0 2024-03-24T00:48:45.307 INFO:tasks.workunit.client.1.smithi088.stdout:0/304: rmdir d9/dc/de/d19/d44/d46/d65 39 2024-03-24T00:48:45.307 INFO:tasks.workunit.client.1.smithi088.stdout:0/305: chown d9/dc 114699 1 2024-03-24T00:48:45.320 INFO:tasks.workunit.client.0.smithi038.stdout:8/274: dwrite d0/d1d/d20/d3c/f47 [0,4194304] 0 2024-03-24T00:48:45.320 INFO:tasks.workunit.client.0.smithi038.stdout:8/275: write d0/d45/f55 [690244,70766] 0 2024-03-24T00:48:45.320 INFO:tasks.workunit.client.0.smithi038.stdout:8/276: creat d0/d15/f63 x:0 0 0 2024-03-24T00:48:45.320 INFO:tasks.workunit.client.0.smithi038.stdout:8/277: stat d0/d1d 0 2024-03-24T00:48:45.320 INFO:tasks.workunit.client.0.smithi038.stdout:8/278: chown d0/d1d/d20/f23 1 1 2024-03-24T00:48:45.320 INFO:tasks.workunit.client.0.smithi038.stdout:8/279: fsync d0/f32 0 2024-03-24T00:48:45.320 INFO:tasks.workunit.client.0.smithi038.stdout:8/280: read d0/f16 [1157502,98544] 0 2024-03-24T00:48:45.320 INFO:tasks.workunit.client.0.smithi038.stdout:9/336: write d12/d28/d3a/d26/f3d [1312162,24649] 0 2024-03-24T00:48:45.321 INFO:tasks.workunit.client.0.smithi038.stdout:9/337: getdents d12/d28/d6c/d75 0 2024-03-24T00:48:45.323 INFO:tasks.workunit.client.1.smithi088.stdout:7/249: write d3/d32/f43 [1244627,8992] 0 2024-03-24T00:48:45.329 INFO:tasks.workunit.client.0.smithi038.stdout:8/281: rename d0/d15/c36 to d0/d15/d42/c64 0 2024-03-24T00:48:45.329 INFO:tasks.workunit.client.0.smithi038.stdout:8/282: chown d0/d1d/d20/f60 0 1 2024-03-24T00:48:45.333 INFO:tasks.workunit.client.0.smithi038.stdout:9/338: truncate d12/d1a/f20 2394942 0 2024-03-24T00:48:45.337 INFO:tasks.workunit.client.1.smithi088.stdout:8/290: dwrite d9/f2b [0,4194304] 0 2024-03-24T00:48:45.337 INFO:tasks.workunit.client.1.smithi088.stdout:8/291: creat d9/dc/d11/d41/d56/f66 x:0 0 0 2024-03-24T00:48:45.337 INFO:tasks.workunit.client.1.smithi088.stdout:8/292: fsync f1 0 2024-03-24T00:48:45.337 INFO:tasks.workunit.client.0.smithi038.stdout:8/283: symlink d0/d1d/d27/l65 0 2024-03-24T00:48:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:1/229: dread f8 [0,4194304] 0 2024-03-24T00:48:45.351 INFO:tasks.workunit.client.1.smithi088.stdout:1/315: read d6/d27/d1e/d3f/f4c [3746091,87189] 0 2024-03-24T00:48:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:1/230: rename da/d3c/f17 to da/dd/d22/f4c 0 2024-03-24T00:48:45.363 INFO:tasks.workunit.client.0.smithi038.stdout:1/231: creat da/dd/d10/d14/f4d x:0 0 0 2024-03-24T00:48:45.366 INFO:tasks.workunit.client.0.smithi038.stdout:6/319: dwrite d1/da/d1b/d25/d49/d4e/f55 [0,4194304] 0 2024-03-24T00:48:45.367 INFO:tasks.workunit.client.1.smithi088.stdout:1/316: mknod d6/c5a 0 2024-03-24T00:48:45.368 INFO:tasks.workunit.client.1.smithi088.stdout:8/293: rename d9/l35 to d9/dc/d11/d24/d29/d39/l67 0 2024-03-24T00:48:45.368 INFO:tasks.workunit.client.1.smithi088.stdout:8/294: chown d9/f17 3 1 2024-03-24T00:48:45.368 INFO:tasks.workunit.client.1.smithi088.stdout:8/295: write d9/dc/d11/d24/d43/f50 [133947,38273] 0 2024-03-24T00:48:45.376 INFO:tasks.workunit.client.0.smithi038.stdout:6/320: symlink d1/da/d1b/d1d/d3c/l60 0 2024-03-24T00:48:45.376 INFO:tasks.workunit.client.1.smithi088.stdout:1/317: mkdir d6/d27/d1e/d5b 0 2024-03-24T00:48:45.377 INFO:tasks.workunit.client.1.smithi088.stdout:2/269: dwrite db/d21/d35/f3d [0,4194304] 0 2024-03-24T00:48:45.377 INFO:tasks.workunit.client.1.smithi088.stdout:2/270: creat db/d21/d4a/f53 x:0 0 0 2024-03-24T00:48:45.378 INFO:tasks.workunit.client.1.smithi088.stdout:8/296: mknod d9/c68 0 2024-03-24T00:48:45.378 INFO:tasks.workunit.client.1.smithi088.stdout:5/288: dwrite d1/dd/f54 [0,4194304] 0 2024-03-24T00:48:45.378 INFO:tasks.workunit.client.1.smithi088.stdout:5/289: creat d1/d19/d1e/d26/f5e x:0 0 0 2024-03-24T00:48:45.379 INFO:tasks.workunit.client.1.smithi088.stdout:5/290: readlink d1/l9 0 2024-03-24T00:48:45.380 INFO:tasks.workunit.client.1.smithi088.stdout:3/277: dwrite d3/d1a/d1e/f28 [0,4194304] 0 2024-03-24T00:48:45.382 INFO:tasks.workunit.client.0.smithi038.stdout:6/321: mkdir d1/da/d61 0 2024-03-24T00:48:45.382 INFO:tasks.workunit.client.1.smithi088.stdout:9/298: dwrite d1/f5 [0,4194304] 0 2024-03-24T00:48:45.382 INFO:tasks.workunit.client.1.smithi088.stdout:9/299: chown d1/d15 479229 1 2024-03-24T00:48:45.384 INFO:tasks.workunit.client.1.smithi088.stdout:4/316: dwrite d5/fc [0,4194304] 0 2024-03-24T00:48:45.384 INFO:tasks.workunit.client.1.smithi088.stdout:4/317: readlink d5/d16/l20 0 2024-03-24T00:48:45.385 INFO:tasks.workunit.client.1.smithi088.stdout:1/318: unlink d6/l11 0 2024-03-24T00:48:45.387 INFO:tasks.workunit.client.1.smithi088.stdout:2/271: mkdir db/d54 0 2024-03-24T00:48:45.387 INFO:tasks.workunit.client.1.smithi088.stdout:2/272: chown db/d54 3022395 1 2024-03-24T00:48:45.387 INFO:tasks.workunit.client.1.smithi088.stdout:2/273: chown db/dc/l1c 1968209 1 2024-03-24T00:48:45.393 INFO:tasks.workunit.client.1.smithi088.stdout:8/297: symlink d9/dc/d11/d24/d29/l69 0 2024-03-24T00:48:45.394 INFO:tasks.workunit.client.1.smithi088.stdout:3/278: unlink d3/d1a/d1e/c3e 0 2024-03-24T00:48:45.394 INFO:tasks.workunit.client.1.smithi088.stdout:3/279: write d3/d1a/d1e/d48/d4b/f4c [488419,52746] 0 2024-03-24T00:48:45.398 INFO:tasks.workunit.client.1.smithi088.stdout:1/319: mknod d6/d43/c5c 0 2024-03-24T00:48:45.399 INFO:tasks.workunit.client.1.smithi088.stdout:1/320: chown d6/d27/l36 4 1 2024-03-24T00:48:45.399 INFO:tasks.workunit.client.1.smithi088.stdout:1/321: write d6/d23/f29 [266661,119235] 0 2024-03-24T00:48:45.403 INFO:tasks.workunit.client.1.smithi088.stdout:5/291: write d1/f31 [1508334,68116] 0 2024-03-24T00:48:45.403 INFO:tasks.workunit.client.1.smithi088.stdout:3/280: truncate f1 3955169 0 2024-03-24T00:48:45.407 INFO:tasks.workunit.client.1.smithi088.stdout:1/322: mkdir d6/d27/d1e/d5b/d5d 0 2024-03-24T00:48:45.407 INFO:tasks.workunit.client.1.smithi088.stdout:1/323: write d6/f3e [2021335,24405] 0 2024-03-24T00:48:45.416 INFO:tasks.workunit.client.1.smithi088.stdout:5/292: creat d1/d10/d1b/d2f/d57/f5f x:0 0 0 2024-03-24T00:48:45.416 INFO:tasks.workunit.client.1.smithi088.stdout:6/329: dwrite db/d1d/d23/d3c/d43/f34 [0,4194304] 0 2024-03-24T00:48:45.422 INFO:tasks.workunit.client.0.smithi038.stdout:6/322: dread d1/da/d1b/d1d/f2c [0,4194304] 0 2024-03-24T00:48:45.422 INFO:tasks.workunit.client.0.smithi038.stdout:6/323: write d1/da/d1b/d25/f2a [577270,3375] 0 2024-03-24T00:48:45.422 INFO:tasks.workunit.client.0.smithi038.stdout:6/324: write d1/da/d1b/f26 [2358070,19796] 0 2024-03-24T00:48:45.422 INFO:tasks.workunit.client.0.smithi038.stdout:6/325: chown d1/da/d1b/d25/f44 281 1 2024-03-24T00:48:45.422 INFO:tasks.workunit.client.0.smithi038.stdout:1/232: getdents da/d3c 0 2024-03-24T00:48:45.428 INFO:tasks.workunit.client.0.smithi038.stdout:3/278: sync 2024-03-24T00:48:45.428 INFO:tasks.workunit.client.0.smithi038.stdout:3/279: dread - d3/d9/dc/df/f3a zero size 2024-03-24T00:48:45.428 INFO:tasks.workunit.client.0.smithi038.stdout:3/280: readlink d3/d9/dc/df/d14/d19/l4c 0 2024-03-24T00:48:45.428 INFO:tasks.workunit.client.0.smithi038.stdout:3/281: truncate d3/d9/f16 1367693 0 2024-03-24T00:48:45.429 INFO:tasks.workunit.client.1.smithi088.stdout:5/293: fsync d1/f31 0 2024-03-24T00:48:45.430 INFO:tasks.workunit.client.1.smithi088.stdout:0/306: dwrite d9/dc/de/d19/d44/d46/d65/d2a/f5e [0,4194304] 0 2024-03-24T00:48:45.430 INFO:tasks.workunit.client.1.smithi088.stdout:3/281: symlink d3/d6/d27/l5e 0 2024-03-24T00:48:45.430 INFO:tasks.workunit.client.0.smithi038.stdout:3/282: unlink d3/d2a/c2f 0 2024-03-24T00:48:45.431 INFO:tasks.workunit.client.0.smithi038.stdout:5/259: dwrite f5 [0,4194304] 0 2024-03-24T00:48:45.441 INFO:tasks.workunit.client.1.smithi088.stdout:0/307: unlink d9/f3a 0 2024-03-24T00:48:45.441 INFO:tasks.workunit.client.1.smithi088.stdout:5/294: getdents d1/d43/d4e 0 2024-03-24T00:48:45.443 INFO:tasks.workunit.client.1.smithi088.stdout:5/295: mkdir d1/d10/d1b/d2e/d60 0 2024-03-24T00:48:45.443 INFO:tasks.workunit.client.1.smithi088.stdout:7/250: dwrite d3/f1f [0,4194304] 0 2024-03-24T00:48:45.444 INFO:tasks.workunit.client.1.smithi088.stdout:7/251: chown d3/f14 98440446 1 2024-03-24T00:48:45.444 INFO:tasks.workunit.client.0.smithi038.stdout:6/326: dread d1/f21 [0,4194304] 0 2024-03-24T00:48:45.444 INFO:tasks.workunit.client.1.smithi088.stdout:2/274: rmdir db 39 2024-03-24T00:48:45.450 INFO:tasks.workunit.client.1.smithi088.stdout:2/275: creat db/f55 x:0 0 0 2024-03-24T00:48:45.458 INFO:tasks.workunit.client.1.smithi088.stdout:3/282: read d3/de/f17 [668178,15315] 0 2024-03-24T00:48:45.459 INFO:tasks.workunit.client.1.smithi088.stdout:2/276: mkdir db/d56 0 2024-03-24T00:48:45.459 INFO:tasks.workunit.client.1.smithi088.stdout:3/283: mkdir d3/d6/d19/d2a/d5f 0 2024-03-24T00:48:45.464 INFO:tasks.workunit.client.1.smithi088.stdout:6/330: dread db/d1d/d23/d3c/d43/f39 [0,4194304] 0 2024-03-24T00:48:45.478 INFO:tasks.workunit.client.1.smithi088.stdout:4/318: dwrite d5/d16/d1a/f3e [0,4194304] 0 2024-03-24T00:48:45.479 INFO:tasks.workunit.client.1.smithi088.stdout:4/319: creat d5/d16/d37/f6d x:0 0 0 2024-03-24T00:48:45.485 INFO:tasks.workunit.client.1.smithi088.stdout:8/298: dwrite d9/f32 [0,4194304] 0 2024-03-24T00:48:45.486 INFO:tasks.workunit.client.1.smithi088.stdout:2/277: write db/f20 [41795,5036] 0 2024-03-24T00:48:45.489 INFO:tasks.workunit.client.0.smithi038.stdout:8/284: dwrite d0/d45/d37/f48 [0,4194304] 0 2024-03-24T00:48:45.495 INFO:tasks.workunit.client.0.smithi038.stdout:1/233: dread da/f16 [0,4194304] 0 2024-03-24T00:48:45.495 INFO:tasks.workunit.client.0.smithi038.stdout:3/283: dread d3/d9/f2b [0,4194304] 0 2024-03-24T00:48:45.498 INFO:tasks.workunit.client.0.smithi038.stdout:1/234: mknod da/dd/d4a/c4e 0 2024-03-24T00:48:45.499 INFO:tasks.workunit.client.0.smithi038.stdout:1/235: truncate da/d3c/d2d/f36 931384 0 2024-03-24T00:48:45.499 INFO:tasks.workunit.client.0.smithi038.stdout:1/236: creat da/dd/d22/f4f x:0 0 0 2024-03-24T00:48:45.500 INFO:tasks.workunit.client.0.smithi038.stdout:3/284: symlink d3/d9/dc/df/d14/d19/l61 0 2024-03-24T00:48:45.500 INFO:tasks.workunit.client.1.smithi088.stdout:2/278: symlink db/dc/l57 0 2024-03-24T00:48:45.501 INFO:tasks.workunit.client.1.smithi088.stdout:8/299: mkdir d9/dc/d11/d24/d29/d36/d6a 0 2024-03-24T00:48:45.501 INFO:tasks.workunit.client.1.smithi088.stdout:8/300: stat d9/dc/ce 0 2024-03-24T00:48:45.501 INFO:tasks.workunit.client.1.smithi088.stdout:8/301: readlink d9/dc/d11/d24/d29/l69 0 2024-03-24T00:48:45.504 INFO:tasks.workunit.client.1.smithi088.stdout:9/300: dwrite d1/f20 [0,4194304] 0 2024-03-24T00:48:45.508 INFO:tasks.workunit.client.1.smithi088.stdout:5/296: dwrite d1/d10/d1b/d2f/f58 [0,4194304] 0 2024-03-24T00:48:45.509 INFO:tasks.workunit.client.0.smithi038.stdout:8/285: rename d0/d15/d42 to d0/d45/d66 0 2024-03-24T00:48:45.509 INFO:tasks.workunit.client.1.smithi088.stdout:2/279: symlink db/d21/d4a/l58 0 2024-03-24T00:48:45.511 INFO:tasks.workunit.client.1.smithi088.stdout:8/302: mkdir d9/d6b 0 2024-03-24T00:48:45.512 INFO:tasks.workunit.client.1.smithi088.stdout:1/324: dwrite d6/d23/f4a [0,4194304] 0 2024-03-24T00:48:45.517 INFO:tasks.workunit.client.0.smithi038.stdout:5/260: dwrite d6/f26 [0,4194304] 0 2024-03-24T00:48:45.517 INFO:tasks.workunit.client.0.smithi038.stdout:5/261: stat d6/f9 0 2024-03-24T00:48:45.519 INFO:tasks.workunit.client.0.smithi038.stdout:5/262: rmdir d6/d19/d20/d33/d49 0 2024-03-24T00:48:45.521 INFO:tasks.workunit.client.0.smithi038.stdout:5/263: link d6/fa d6/d23/f53 0 2024-03-24T00:48:45.522 INFO:tasks.workunit.client.1.smithi088.stdout:5/297: symlink d1/d10/d1b/d2f/d57/l61 0 2024-03-24T00:48:45.522 INFO:tasks.workunit.client.1.smithi088.stdout:5/298: readlink d1/l9 0 2024-03-24T00:48:45.522 INFO:tasks.workunit.client.1.smithi088.stdout:5/299: read d1/d19/f30 [8253568,104312] 0 2024-03-24T00:48:45.524 INFO:tasks.workunit.client.1.smithi088.stdout:8/303: rename d9/dc/d11/d41/d54/d5d/f62 to d9/d6b/f6c 0 2024-03-24T00:48:45.524 INFO:tasks.workunit.client.1.smithi088.stdout:8/304: chown d9/dc/c15 132304625 1 2024-03-24T00:48:45.524 INFO:tasks.workunit.client.1.smithi088.stdout:8/305: dread - d9/dc/d11/d24/f58 zero size 2024-03-24T00:48:45.526 INFO:tasks.workunit.client.0.smithi038.stdout:6/327: dwrite d1/da/d1b/d1d/f2d [0,4194304] 0 2024-03-24T00:48:45.532 INFO:tasks.workunit.client.0.smithi038.stdout:6/328: rmdir d1/d37 39 2024-03-24T00:48:45.532 INFO:tasks.workunit.client.0.smithi038.stdout:6/329: fsync d1/f21 0 2024-03-24T00:48:45.535 INFO:tasks.workunit.client.0.smithi038.stdout:5/264: rename d6/d16/c1b to d6/c54 0 2024-03-24T00:48:45.537 INFO:tasks.workunit.client.0.smithi038.stdout:5/265: read d6/d23/f32 [256509,22396] 0 2024-03-24T00:48:45.540 INFO:tasks.workunit.client.1.smithi088.stdout:0/308: dwrite d9/dc/de/d19/d44/d46/d65/d2a/d2e/f5f [0,4194304] 0 2024-03-24T00:48:45.543 INFO:tasks.workunit.client.0.smithi038.stdout:6/330: read d1/da/d1b/d1d/f4f [207408,108741] 0 2024-03-24T00:48:45.543 INFO:tasks.workunit.client.1.smithi088.stdout:2/280: dread db/f20 [0,4194304] 0 2024-03-24T00:48:45.547 INFO:tasks.workunit.client.0.smithi038.stdout:6/331: creat d1/da/d1b/d2f/f62 x:0 0 0 2024-03-24T00:48:45.549 INFO:tasks.workunit.client.1.smithi088.stdout:3/284: dwrite f1 [4194304,4194304] 0 2024-03-24T00:48:45.549 INFO:tasks.workunit.client.1.smithi088.stdout:3/285: write d3/de/df/f2d [51405,110065] 0 2024-03-24T00:48:45.549 INFO:tasks.workunit.client.1.smithi088.stdout:3/286: dread - d3/d1a/d1e/d2f/d45/f5d zero size 2024-03-24T00:48:45.549 INFO:tasks.workunit.client.1.smithi088.stdout:3/287: dread - d3/d6/d27/f40 zero size 2024-03-24T00:48:45.551 INFO:tasks.workunit.client.0.smithi038.stdout:3/285: write d3/d9/f58 [182785,18036] 0 2024-03-24T00:48:45.552 INFO:tasks.workunit.client.0.smithi038.stdout:3/286: chown d3/d30/d5d 174097377 1 2024-03-24T00:48:45.552 INFO:tasks.workunit.client.0.smithi038.stdout:6/332: link d1/da/d1b/d25/c31 d1/da/d1b/c63 0 2024-03-24T00:48:45.557 INFO:tasks.workunit.client.1.smithi088.stdout:4/320: dwrite d5/f56 [0,4194304] 0 2024-03-24T00:48:45.559 INFO:tasks.workunit.client.1.smithi088.stdout:0/309: write d9/dc/de/d19/d44/d46/d65/f35 [3205809,23058] 0 2024-03-24T00:48:45.563 INFO:tasks.workunit.client.0.smithi038.stdout:6/333: rmdir d1/da/d1b/d25/d49/d4e/d51 39 2024-03-24T00:48:45.564 INFO:tasks.workunit.client.1.smithi088.stdout:7/252: dwrite d3/d32/f43 [4194304,4194304] 0 2024-03-24T00:48:45.564 INFO:tasks.workunit.client.1.smithi088.stdout:7/253: creat d3/d32/f54 x:0 0 0 2024-03-24T00:48:45.564 INFO:tasks.workunit.client.1.smithi088.stdout:9/301: dwrite d1/d10/f3e [0,4194304] 0 2024-03-24T00:48:45.565 INFO:tasks.workunit.client.1.smithi088.stdout:9/302: creat d1/d22/f51 x:0 0 0 2024-03-24T00:48:45.566 INFO:tasks.workunit.client.1.smithi088.stdout:1/325: dwrite d6/d23/f2d [0,4194304] 0 2024-03-24T00:48:45.570 INFO:tasks.workunit.client.1.smithi088.stdout:8/306: mkdir d9/dc/d11/d41/d6d 0 2024-03-24T00:48:45.570 INFO:tasks.workunit.client.1.smithi088.stdout:9/303: dread d1/d10/d29/f45 [0,4194304] 0 2024-03-24T00:48:45.571 INFO:tasks.workunit.client.1.smithi088.stdout:5/300: rename d1/c37 to d1/d10/d1b/d2e/d60/c62 0 2024-03-24T00:48:45.574 INFO:tasks.workunit.client.0.smithi038.stdout:1/237: dwrite da/dd/f1c [0,4194304] 0 2024-03-24T00:48:45.576 INFO:tasks.workunit.client.0.smithi038.stdout:6/334: dread d1/da/d1b/d1d/f4f [0,4194304] 0 2024-03-24T00:48:45.576 INFO:tasks.workunit.client.0.smithi038.stdout:3/287: dread d3/d9/f58 [0,4194304] 0 2024-03-24T00:48:45.578 INFO:tasks.workunit.client.1.smithi088.stdout:2/281: symlink db/d21/d4a/d4e/l59 0 2024-03-24T00:48:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:1/238: mknod da/dd/d10/d14/c50 0 2024-03-24T00:48:45.582 INFO:tasks.workunit.client.1.smithi088.stdout:3/288: mknod d3/de/c60 0 2024-03-24T00:48:45.585 INFO:tasks.workunit.client.0.smithi038.stdout:6/335: link d1/da/d1b/d1d/c4a d1/d37/c64 0 2024-03-24T00:48:45.588 INFO:tasks.workunit.client.0.smithi038.stdout:1/239: rename da/dd/c2b to da/dd/d2c/d40/c51 0 2024-03-24T00:48:45.590 INFO:tasks.workunit.client.1.smithi088.stdout:4/321: mkdir d5/d13/d18/d6e 0 2024-03-24T00:48:45.590 INFO:tasks.workunit.client.1.smithi088.stdout:4/322: write d5/d17/f33 [1084589,32984] 0 2024-03-24T00:48:45.590 INFO:tasks.workunit.client.1.smithi088.stdout:4/323: creat d5/d16/d37/f6f x:0 0 0 2024-03-24T00:48:45.592 INFO:tasks.workunit.client.1.smithi088.stdout:0/310: symlink d9/dc/de/d19/d44/d46/d59/l66 0 2024-03-24T00:48:45.592 INFO:tasks.workunit.client.1.smithi088.stdout:0/311: fdatasync d9/f1c 0 2024-03-24T00:48:45.595 INFO:tasks.workunit.client.0.smithi038.stdout:8/286: dwrite d0/d1d/f3e [0,4194304] 0 2024-03-24T00:48:45.595 INFO:tasks.workunit.client.0.smithi038.stdout:5/266: dwrite d6/d23/f3b [4194304,4194304] 0 2024-03-24T00:48:45.598 INFO:tasks.workunit.client.1.smithi088.stdout:7/254: symlink d3/d32/l55 0 2024-03-24T00:48:45.599 INFO:tasks.workunit.client.1.smithi088.stdout:7/255: fdatasync d3/f24 0 2024-03-24T00:48:45.600 INFO:tasks.workunit.client.0.smithi038.stdout:3/288: dread d3/d9/dc/df/d14/f54 [4194304,4194304] 0 2024-03-24T00:48:45.601 INFO:tasks.workunit.client.0.smithi038.stdout:3/289: fdatasync d3/d9/dc/d51/f52 0 2024-03-24T00:48:45.601 INFO:tasks.workunit.client.0.smithi038.stdout:1/240: mknod da/d3c/d2d/c52 0 2024-03-24T00:48:45.616 INFO:tasks.workunit.client.0.smithi038.stdout:6/336: dread d1/da/d1b/d1d/f2d [0,4194304] 0 2024-03-24T00:48:45.618 INFO:tasks.workunit.client.0.smithi038.stdout:8/287: symlink d0/d45/l67 0 2024-03-24T00:48:45.624 INFO:tasks.workunit.client.1.smithi088.stdout:8/307: dwrite d9/f28 [0,4194304] 0 2024-03-24T00:48:45.625 INFO:tasks.workunit.client.0.smithi038.stdout:5/267: rename f5 to d6/d1e/f55 0 2024-03-24T00:48:45.626 INFO:tasks.workunit.client.1.smithi088.stdout:1/326: mkdir d6/d27/d1e/d5b/d5e 0 2024-03-24T00:48:45.631 INFO:tasks.workunit.client.0.smithi038.stdout:3/290: creat d3/d9/f62 x:0 0 0 2024-03-24T00:48:45.631 INFO:tasks.workunit.client.0.smithi038.stdout:3/291: fsync d3/f3d 0 2024-03-24T00:48:45.631 INFO:tasks.workunit.client.0.smithi038.stdout:6/337: write d1/da/d1b/d25/d49/d4e/f55 [253280,80969] 0 2024-03-24T00:48:45.633 INFO:tasks.workunit.client.1.smithi088.stdout:9/304: mknod d1/d22/c52 0 2024-03-24T00:48:45.633 INFO:tasks.workunit.client.1.smithi088.stdout:9/305: fsync d1/f8 0 2024-03-24T00:48:45.633 INFO:tasks.workunit.client.1.smithi088.stdout:9/306: truncate d1/d10/d2c/f35 373637 0 2024-03-24T00:48:45.633 INFO:tasks.workunit.client.1.smithi088.stdout:9/307: chown d1/d10/d11/f17 410320 1 2024-03-24T00:48:45.634 INFO:tasks.workunit.client.0.smithi038.stdout:1/241: creat da/dd/d2c/d40/f53 x:0 0 0 2024-03-24T00:48:45.635 INFO:tasks.workunit.client.0.smithi038.stdout:5/268: rename d6/d19/d20/d42/f52 to d6/d16/d3c/f56 0 2024-03-24T00:48:45.635 INFO:tasks.workunit.client.0.smithi038.stdout:5/269: write d6/d19/d20/d42/d4b/f4f [897011,27274] 0 2024-03-24T00:48:45.637 INFO:tasks.workunit.client.1.smithi088.stdout:2/282: link f1 db/d21/d35/f5a 0 2024-03-24T00:48:45.640 INFO:tasks.workunit.client.1.smithi088.stdout:3/289: rename d3/de/df/f2d to d3/de/f61 0 2024-03-24T00:48:45.640 INFO:tasks.workunit.client.1.smithi088.stdout:3/290: write d3/de/df/f15 [2491606,62169] 0 2024-03-24T00:48:45.641 INFO:tasks.workunit.client.1.smithi088.stdout:0/312: dwrite d9/d26/f3c [0,4194304] 0 2024-03-24T00:48:45.641 INFO:tasks.workunit.client.1.smithi088.stdout:0/313: readlink d9/l18 0 2024-03-24T00:48:45.641 INFO:tasks.workunit.client.1.smithi088.stdout:0/314: readlink d9/d15/l64 0 2024-03-24T00:48:45.642 INFO:tasks.workunit.client.1.smithi088.stdout:5/301: dwrite d1/d10/f17 [0,4194304] 0 2024-03-24T00:48:45.643 INFO:tasks.workunit.client.0.smithi038.stdout:1/242: rename da/dd/d2c/f31 to da/dd/d10/f54 0 2024-03-24T00:48:45.643 INFO:tasks.workunit.client.0.smithi038.stdout:1/243: chown da/dd/d22/f4c 2924096 1 2024-03-24T00:48:45.643 INFO:tasks.workunit.client.0.smithi038.stdout:1/244: write da/dd/d10/d14/f47 [578726,65329] 0 2024-03-24T00:48:45.643 INFO:tasks.workunit.client.0.smithi038.stdout:1/245: stat da/dd/d4a 0 2024-03-24T00:48:45.646 INFO:tasks.workunit.client.1.smithi088.stdout:4/324: symlink d5/d13/d18/l70 0 2024-03-24T00:48:45.649 INFO:tasks.workunit.client.0.smithi038.stdout:6/338: read d1/da/d1b/f2b [1581163,112908] 0 2024-03-24T00:48:45.650 INFO:tasks.workunit.client.1.smithi088.stdout:7/256: mkdir d3/d31/d56 0 2024-03-24T00:48:45.650 INFO:tasks.workunit.client.1.smithi088.stdout:7/257: write d3/f2b [1372626,80558] 0 2024-03-24T00:48:45.650 INFO:tasks.workunit.client.1.smithi088.stdout:8/308: creat d9/dc/d11/d41/d65/f6e x:0 0 0 2024-03-24T00:48:45.651 INFO:tasks.workunit.client.0.smithi038.stdout:3/292: dread d3/d9/fb [0,4194304] 0 2024-03-24T00:48:45.651 INFO:tasks.workunit.client.0.smithi038.stdout:3/293: chown d3/f29 4 1 2024-03-24T00:48:45.651 INFO:tasks.workunit.client.0.smithi038.stdout:3/294: write d3/d9/dc/df/f31 [579874,92849] 0 2024-03-24T00:48:45.651 INFO:tasks.workunit.client.0.smithi038.stdout:3/295: readlink d3/d9/dc/d51/l5c 0 2024-03-24T00:48:45.652 INFO:tasks.workunit.client.0.smithi038.stdout:7/271: sync 2024-03-24T00:48:45.652 INFO:tasks.workunit.client.0.smithi038.stdout:7/272: write d2/f4 [493671,78510] 0 2024-03-24T00:48:45.652 INFO:tasks.workunit.client.0.smithi038.stdout:7/273: chown d2/d9/df/c10 665 1 2024-03-24T00:48:45.652 INFO:tasks.workunit.client.0.smithi038.stdout:7/274: creat d2/d2c/f62 x:0 0 0 2024-03-24T00:48:45.652 INFO:tasks.workunit.client.0.smithi038.stdout:7/275: truncate d2/d2c/f62 585295 0 2024-03-24T00:48:45.653 INFO:tasks.workunit.client.1.smithi088.stdout:9/308: creat d1/d10/d11/d23/f53 x:0 0 0 2024-03-24T00:48:45.653 INFO:tasks.workunit.client.1.smithi088.stdout:1/327: write d6/d27/d1e/d58/f51 [3339115,19075] 0 2024-03-24T00:48:45.653 INFO:tasks.workunit.client.1.smithi088.stdout:1/328: truncate d6/d23/f34 5040397 0 2024-03-24T00:48:45.653 INFO:tasks.workunit.client.1.smithi088.stdout:1/329: chown d6/d27/d1e/d3f/f41 47824 1 2024-03-24T00:48:45.661 INFO:tasks.workunit.client.0.smithi038.stdout:2/306: sync 2024-03-24T00:48:45.661 INFO:tasks.workunit.client.0.smithi038.stdout:0/262: sync 2024-03-24T00:48:45.661 INFO:tasks.workunit.client.0.smithi038.stdout:5/270: dread d6/d23/f3b [0,4194304] 0 2024-03-24T00:48:45.661 INFO:tasks.workunit.client.0.smithi038.stdout:5/271: chown d6/d16/d21/d2d/l4a 4885 1 2024-03-24T00:48:45.661 INFO:tasks.workunit.client.0.smithi038.stdout:4/251: sync 2024-03-24T00:48:45.665 INFO:tasks.workunit.client.1.smithi088.stdout:5/302: dread d1/ff [0,4194304] 0 2024-03-24T00:48:45.666 INFO:tasks.workunit.client.1.smithi088.stdout:6/331: sync 2024-03-24T00:48:45.666 INFO:tasks.workunit.client.1.smithi088.stdout:9/309: dread d1/d10/d29/f45 [0,4194304] 0 2024-03-24T00:48:45.666 INFO:tasks.workunit.client.1.smithi088.stdout:6/332: write db/d1d/f58 [42105,87863] 0 2024-03-24T00:48:45.675 INFO:tasks.workunit.client.0.smithi038.stdout:6/339: dread d1/da/f14 [0,4194304] 0 2024-03-24T00:48:45.682 INFO:tasks.workunit.client.0.smithi038.stdout:7/276: creat d2/f63 x:0 0 0 2024-03-24T00:48:45.692 INFO:tasks.workunit.client.0.smithi038.stdout:3/296: fsync d3/d9/dc/df/f31 0 2024-03-24T00:48:45.702 INFO:tasks.workunit.client.1.smithi088.stdout:8/309: dwrite d9/dc/d11/d24/d29/d5b/f34 [8388608,4194304] 0 2024-03-24T00:48:45.706 INFO:tasks.workunit.client.1.smithi088.stdout:0/315: symlink d9/dc/d4f/l67 0 2024-03-24T00:48:45.709 INFO:tasks.workunit.client.0.smithi038.stdout:5/272: symlink d6/d19/l57 0 2024-03-24T00:48:45.709 INFO:tasks.workunit.client.0.smithi038.stdout:9/339: sync 2024-03-24T00:48:45.709 INFO:tasks.workunit.client.0.smithi038.stdout:9/340: write d12/d28/d3a/f4b [983363,4886] 0 2024-03-24T00:48:45.709 INFO:tasks.workunit.client.0.smithi038.stdout:4/252: rmdir da/d12/d24 39 2024-03-24T00:48:45.709 INFO:tasks.workunit.client.0.smithi038.stdout:4/253: read da/d12/d37/dd/f2b [351276,84323] 0 2024-03-24T00:48:45.710 INFO:tasks.workunit.client.1.smithi088.stdout:4/325: creat d5/d16/d1a/d22/d2f/f71 x:0 0 0 2024-03-24T00:48:45.711 INFO:tasks.workunit.client.1.smithi088.stdout:7/258: mknod d3/d32/d34/c57 0 2024-03-24T00:48:45.712 INFO:tasks.workunit.client.1.smithi088.stdout:7/259: write d3/f20 [5232304,111040] 0 2024-03-24T00:48:45.713 INFO:tasks.workunit.client.1.smithi088.stdout:1/330: symlink d6/d43/l5f 0 2024-03-24T00:48:45.715 INFO:tasks.workunit.client.1.smithi088.stdout:2/283: mknod db/d21/c5b 0 2024-03-24T00:48:45.715 INFO:tasks.workunit.client.1.smithi088.stdout:2/284: read - db/d21/d4a/f53 zero size 2024-03-24T00:48:45.715 INFO:tasks.workunit.client.1.smithi088.stdout:2/285: write f1 [1463536,69491] 0 2024-03-24T00:48:45.718 INFO:tasks.workunit.client.1.smithi088.stdout:5/303: creat d1/d43/d4e/d5b/f63 x:0 0 0 2024-03-24T00:48:45.719 INFO:tasks.workunit.client.0.smithi038.stdout:2/307: getdents d4/d18 0 2024-03-24T00:48:45.719 INFO:tasks.workunit.client.0.smithi038.stdout:3/297: unlink d3/l38 0 2024-03-24T00:48:45.721 INFO:tasks.workunit.client.1.smithi088.stdout:6/333: mkdir db/dd/d61 0 2024-03-24T00:48:45.722 INFO:tasks.workunit.client.1.smithi088.stdout:3/291: rmdir d3/d6/d19/d2a 39 2024-03-24T00:48:45.723 INFO:tasks.workunit.client.0.smithi038.stdout:9/341: symlink d12/d41/l78 0 2024-03-24T00:48:45.728 INFO:tasks.workunit.client.1.smithi088.stdout:0/316: creat d9/dc/de/d19/d44/d46/d65/f68 x:0 0 0 2024-03-24T00:48:45.729 INFO:tasks.workunit.client.1.smithi088.stdout:9/310: dwrite d1/d22/d27/d39/f4d [0,4194304] 0 2024-03-24T00:48:45.729 INFO:tasks.workunit.client.1.smithi088.stdout:9/311: creat d1/d10/d11/d23/f54 x:0 0 0 2024-03-24T00:48:45.729 INFO:tasks.workunit.client.1.smithi088.stdout:9/312: write d1/ff [1769997,64509] 0 2024-03-24T00:48:45.730 INFO:tasks.workunit.client.1.smithi088.stdout:9/313: readlink d1/d10/d11/d23/l3c 0 2024-03-24T00:48:45.730 INFO:tasks.workunit.client.0.smithi038.stdout:4/254: unlink da/d12/c25 0 2024-03-24T00:48:45.733 INFO:tasks.workunit.client.0.smithi038.stdout:2/308: link d4/d27/f47 d4/d27/d2f/d30/d3e/f6e 0 2024-03-24T00:48:45.737 INFO:tasks.workunit.client.1.smithi088.stdout:5/304: truncate d1/d19/f2c 2889532 0 2024-03-24T00:48:45.738 INFO:tasks.workunit.client.0.smithi038.stdout:4/255: symlink da/d12/d37/dd/d15/l4f 0 2024-03-24T00:48:45.739 INFO:tasks.workunit.client.1.smithi088.stdout:6/334: write db/dd/f41 [443259,21896] 0 2024-03-24T00:48:45.741 INFO:tasks.workunit.client.1.smithi088.stdout:2/286: symlink db/d21/l5c 0 2024-03-24T00:48:45.742 INFO:tasks.workunit.client.1.smithi088.stdout:2/287: read - db/dc/f2b zero size 2024-03-24T00:48:45.742 INFO:tasks.workunit.client.1.smithi088.stdout:2/288: write db/f46 [646218,50482] 0 2024-03-24T00:48:45.742 INFO:tasks.workunit.client.0.smithi038.stdout:3/298: mknod d3/c63 0 2024-03-24T00:48:45.745 INFO:tasks.workunit.client.0.smithi038.stdout:3/299: mknod d3/d5a/c64 0 2024-03-24T00:48:45.745 INFO:tasks.workunit.client.1.smithi088.stdout:2/289: symlink db/d21/d35/l5d 0 2024-03-24T00:48:45.745 INFO:tasks.workunit.client.1.smithi088.stdout:2/290: stat db/d21/d35/l5d 0 2024-03-24T00:48:45.746 INFO:tasks.workunit.client.1.smithi088.stdout:2/291: rmdir db/d21/d4a/d4e 39 2024-03-24T00:48:45.747 INFO:tasks.workunit.client.0.smithi038.stdout:3/300: rename d3/d9/dc/c2e to d3/d9/dc/df/c65 0 2024-03-24T00:48:45.748 INFO:tasks.workunit.client.1.smithi088.stdout:2/292: rmdir db/d56 0 2024-03-24T00:48:45.749 INFO:tasks.workunit.client.0.smithi038.stdout:3/301: symlink d3/d5a/l66 0 2024-03-24T00:48:45.751 INFO:tasks.workunit.client.1.smithi088.stdout:8/310: dread d9/dc/d11/d24/d29/d36/f46 [0,4194304] 0 2024-03-24T00:48:45.751 INFO:tasks.workunit.client.1.smithi088.stdout:8/311: dread - d9/f57 zero size 2024-03-24T00:48:45.752 INFO:tasks.workunit.client.1.smithi088.stdout:8/312: dread - d9/dc/d11/d24/d29/d5b/f53 zero size 2024-03-24T00:48:45.752 INFO:tasks.workunit.client.1.smithi088.stdout:8/313: chown d9/dc/d11/d41/d54 348339164 1 2024-03-24T00:48:45.755 INFO:tasks.workunit.client.0.smithi038.stdout:3/302: mknod d3/d30/c67 0 2024-03-24T00:48:45.755 INFO:tasks.workunit.client.0.smithi038.stdout:3/303: chown d3/d9/f62 532 1 2024-03-24T00:48:45.776 INFO:tasks.workunit.client.0.smithi038.stdout:3/304: write d3/d2a/f4b [7629493,22290] 0 2024-03-24T00:48:45.776 INFO:tasks.workunit.client.0.smithi038.stdout:3/305: write d3/d2a/f59 [335115,49310] 0 2024-03-24T00:48:45.785 INFO:tasks.workunit.client.1.smithi088.stdout:4/326: rename d5/d16/d1a/d22/d2f to d5/d7/d72 0 2024-03-24T00:48:45.798 INFO:tasks.workunit.client.1.smithi088.stdout:1/331: rename d6/d27/d1e/f40 to d6/d27/d1e/d5b/d5e/f60 0 2024-03-24T00:48:45.799 INFO:tasks.workunit.client.1.smithi088.stdout:1/332: chown l4 104477052 1 2024-03-24T00:48:45.799 INFO:tasks.workunit.client.0.smithi038.stdout:4/256: write da/d38/f41 [525467,35183] 0 2024-03-24T00:48:45.800 INFO:tasks.workunit.client.0.smithi038.stdout:1/246: dwrite f7 [4194304,4194304] 0 2024-03-24T00:48:45.805 INFO:tasks.workunit.client.0.smithi038.stdout:9/342: getdents d12/d41 0 2024-03-24T00:48:45.806 INFO:tasks.workunit.client.0.smithi038.stdout:8/288: dwrite d0/d45/d66/f40 [0,4194304] 0 2024-03-24T00:48:45.807 INFO:tasks.workunit.client.0.smithi038.stdout:8/289: chown d0/d45/d66/c30 258 1 2024-03-24T00:48:45.809 INFO:tasks.workunit.client.1.smithi088.stdout:5/305: dwrite d1/d19/d1e/d26/f5e [0,4194304] 0 2024-03-24T00:48:45.813 INFO:tasks.workunit.client.0.smithi038.stdout:5/273: dwrite d6/d16/d3c/f56 [0,4194304] 0 2024-03-24T00:48:45.814 INFO:tasks.workunit.client.0.smithi038.stdout:4/257: unlink da/d12/d29/c2e 0 2024-03-24T00:48:45.816 INFO:tasks.workunit.client.0.smithi038.stdout:9/343: dread d12/d41/f5b [0,4194304] 0 2024-03-24T00:48:45.819 INFO:tasks.workunit.client.0.smithi038.stdout:9/344: read d12/d28/d3a/f3f [335840,68279] 0 2024-03-24T00:48:45.819 INFO:tasks.workunit.client.0.smithi038.stdout:9/345: dread - d12/d41/f4f zero size 2024-03-24T00:48:45.822 INFO:tasks.workunit.client.0.smithi038.stdout:3/306: write d3/f25 [5488765,21923] 0 2024-03-24T00:48:45.822 INFO:tasks.workunit.client.0.smithi038.stdout:3/307: write d3/d9/f62 [364352,103119] 0 2024-03-24T00:48:45.822 INFO:tasks.workunit.client.0.smithi038.stdout:3/308: chown d3/d2a/l1b 521972 1 2024-03-24T00:48:45.822 INFO:tasks.workunit.client.0.smithi038.stdout:3/309: write d3/d9/dc/df/f31 [2814261,24557] 0 2024-03-24T00:48:45.823 INFO:tasks.workunit.client.1.smithi088.stdout:6/335: rename db/dd/f33 to db/d1d/f62 0 2024-03-24T00:48:45.823 INFO:tasks.workunit.client.0.smithi038.stdout:8/290: rmdir d0/d1d/d20/d3c 39 2024-03-24T00:48:45.823 INFO:tasks.workunit.client.0.smithi038.stdout:8/291: write d0/d15/f17 [4691925,78037] 0 2024-03-24T00:48:45.826 INFO:tasks.workunit.client.0.smithi038.stdout:1/247: write da/dd/d10/f1f [2174209,9834] 0 2024-03-24T00:48:45.827 INFO:tasks.workunit.client.0.smithi038.stdout:1/248: creat da/dd/d22/f55 x:0 0 0 2024-03-24T00:48:45.827 INFO:tasks.workunit.client.0.smithi038.stdout:1/249: write da/f49 [1166613,22354] 0 2024-03-24T00:48:45.827 INFO:tasks.workunit.client.0.smithi038.stdout:1/250: creat da/f56 x:0 0 0 2024-03-24T00:48:45.829 INFO:tasks.workunit.client.1.smithi088.stdout:0/317: dwrite d9/f56 [0,4194304] 0 2024-03-24T00:48:45.830 INFO:tasks.workunit.client.0.smithi038.stdout:7/277: dwrite d2/d9/da/dc/f48 [0,4194304] 0 2024-03-24T00:48:45.830 INFO:tasks.workunit.client.1.smithi088.stdout:0/318: chown d9/dc/de/d19/d44/d46/d65/f58 4 1 2024-03-24T00:48:45.835 INFO:tasks.workunit.client.0.smithi038.stdout:2/309: dwrite d4/f2b [0,4194304] 0 2024-03-24T00:48:45.846 INFO:tasks.workunit.client.1.smithi088.stdout:4/327: dwrite d5/d13/d18/d24/d3d/f55 [0,4194304] 0 2024-03-24T00:48:45.847 INFO:tasks.workunit.client.0.smithi038.stdout:4/258: truncate da/f3b 277369 0 2024-03-24T00:48:45.848 INFO:tasks.workunit.client.1.smithi088.stdout:5/306: symlink d1/d19/d48/l64 0 2024-03-24T00:48:45.848 INFO:tasks.workunit.client.1.smithi088.stdout:3/292: dwrite d3/de/df/d49/f54 [0,4194304] 0 2024-03-24T00:48:45.848 INFO:tasks.workunit.client.1.smithi088.stdout:7/260: dread d3/f14 [0,4194304] 0 2024-03-24T00:48:45.848 INFO:tasks.workunit.client.1.smithi088.stdout:5/307: write d1/f31 [2943813,96961] 0 2024-03-24T00:48:45.854 INFO:tasks.workunit.client.0.smithi038.stdout:3/310: symlink d3/d9/dc/l68 0 2024-03-24T00:48:45.864 INFO:tasks.workunit.client.1.smithi088.stdout:2/293: rename db/d21 to db/d21/d5e 22 2024-03-24T00:48:45.864 INFO:tasks.workunit.client.1.smithi088.stdout:2/294: write db/f38 [792878,43993] 0 2024-03-24T00:48:45.864 INFO:tasks.workunit.client.1.smithi088.stdout:2/295: readlink db/d21/d35/l5d 0 2024-03-24T00:48:45.866 INFO:tasks.workunit.client.1.smithi088.stdout:6/336: mknod db/d1d/d23/d3c/d4b/c63 0 2024-03-24T00:48:45.866 INFO:tasks.workunit.client.1.smithi088.stdout:6/337: chown db/dd 194738 1 2024-03-24T00:48:45.868 INFO:tasks.workunit.client.1.smithi088.stdout:7/261: write d3/d32/d34/f37 [1256120,7900] 0 2024-03-24T00:48:45.869 INFO:tasks.workunit.client.0.smithi038.stdout:1/251: symlink da/dd/l57 0 2024-03-24T00:48:45.872 INFO:tasks.workunit.client.1.smithi088.stdout:1/333: getdents d6/d23 0 2024-03-24T00:48:45.872 INFO:tasks.workunit.client.1.smithi088.stdout:1/334: fdatasync d6/d27/d1e/f39 0 2024-03-24T00:48:45.885 INFO:tasks.workunit.client.1.smithi088.stdout:2/296: dread db/dd/f17 [0,4194304] 0 2024-03-24T00:48:45.886 INFO:tasks.workunit.client.0.smithi038.stdout:7/278: rename d2/d9/df/l28 to d2/d9/da/d23/d2b/d5b/l64 0 2024-03-24T00:48:45.890 INFO:tasks.workunit.client.1.smithi088.stdout:0/319: dwrite d9/dc/de/d19/d44/d46/f57 [0,4194304] 0 2024-03-24T00:48:45.890 INFO:tasks.workunit.client.1.smithi088.stdout:0/320: fdatasync d9/dc/de/f3e 0 2024-03-24T00:48:45.892 INFO:tasks.workunit.client.0.smithi038.stdout:8/292: dwrite d0/d45/d66/f40 [0,4194304] 0 2024-03-24T00:48:45.892 INFO:tasks.workunit.client.0.smithi038.stdout:8/293: chown d0/d15/d2c/l57 4162502 1 2024-03-24T00:48:45.894 INFO:tasks.workunit.client.0.smithi038.stdout:2/310: symlink d4/d18/d26/d37/l6f 0 2024-03-24T00:48:45.894 INFO:tasks.workunit.client.0.smithi038.stdout:2/311: stat d4/d18/d38 0 2024-03-24T00:48:45.907 INFO:tasks.workunit.client.0.smithi038.stdout:3/311: mknod d3/d5a/c69 0 2024-03-24T00:48:45.907 INFO:tasks.workunit.client.0.smithi038.stdout:3/312: write d3/d9/f44 [4246416,123672] 0 2024-03-24T00:48:45.907 INFO:tasks.workunit.client.0.smithi038.stdout:9/346: dwrite d12/d28/d3a/f50 [0,4194304] 0 2024-03-24T00:48:45.908 INFO:tasks.workunit.client.0.smithi038.stdout:8/294: mkdir d0/d45/d37/d5f/d68 0 2024-03-24T00:48:45.914 INFO:tasks.workunit.client.0.smithi038.stdout:8/295: mknod d0/d10/d19/c69 0 2024-03-24T00:48:45.919 INFO:tasks.workunit.client.0.smithi038.stdout:8/296: mknod d0/d1d/d27/d4d/c6a 0 2024-03-24T00:48:45.922 INFO:tasks.workunit.client.0.smithi038.stdout:7/279: write d2/d2c/f62 [46878,23064] 0 2024-03-24T00:48:45.922 INFO:tasks.workunit.client.0.smithi038.stdout:7/280: fsync d2/f1a 0 2024-03-24T00:48:45.926 INFO:tasks.workunit.client.1.smithi088.stdout:4/328: creat d5/d7/d72/f73 x:0 0 0 2024-03-24T00:48:45.936 INFO:tasks.workunit.client.0.smithi038.stdout:7/281: mkdir d2/d9/df/d65 0 2024-03-24T00:48:45.939 INFO:tasks.workunit.client.1.smithi088.stdout:5/308: creat d1/d10/d1b/d2e/f65 x:0 0 0 2024-03-24T00:48:45.945 INFO:tasks.workunit.client.1.smithi088.stdout:8/314: rename d9/dc/d11/d41/d6d to d9/dc/d11/d24/d29/d5b/d6f 0 2024-03-24T00:48:45.945 INFO:tasks.workunit.client.1.smithi088.stdout:3/293: rename d3 to d3/d1a/d1e/d48/d62 22 2024-03-24T00:48:45.945 INFO:tasks.workunit.client.1.smithi088.stdout:8/315: truncate d9/dc/d11/d24/d29/d36/f4e 796177 0 2024-03-24T00:48:45.946 INFO:tasks.workunit.client.0.smithi038.stdout:7/282: creat d2/d9/df/d33/d37/d3a/f66 x:0 0 0 2024-03-24T00:48:45.949 INFO:tasks.workunit.client.0.smithi038.stdout:4/259: dwrite da/d38/f41 [0,4194304] 0 2024-03-24T00:48:45.949 INFO:tasks.workunit.client.1.smithi088.stdout:6/338: truncate db/f13 8098828 0 2024-03-24T00:48:45.949 INFO:tasks.workunit.client.1.smithi088.stdout:7/262: symlink d3/d32/d3f/d4b/l58 0 2024-03-24T00:48:45.955 INFO:tasks.workunit.client.1.smithi088.stdout:1/335: mkdir d6/d27/d1e/d5b/d5d/d61 0 2024-03-24T00:48:45.957 INFO:tasks.workunit.client.0.smithi038.stdout:3/313: write d3/d9/dc/df/f1d [1715069,36636] 0 2024-03-24T00:48:45.958 INFO:tasks.workunit.client.1.smithi088.stdout:2/297: getdents db/dc 0 2024-03-24T00:48:45.958 INFO:tasks.workunit.client.1.smithi088.stdout:2/298: readlink db/dc/l23 0 2024-03-24T00:48:45.974 INFO:tasks.workunit.client.1.smithi088.stdout:6/339: dread db/dd/f10 [0,4194304] 0 2024-03-24T00:48:45.978 INFO:tasks.workunit.client.1.smithi088.stdout:0/321: creat d9/f69 x:0 0 0 2024-03-24T00:48:45.978 INFO:tasks.workunit.client.1.smithi088.stdout:0/322: write d9/dc/f49 [1597402,67003] 0 2024-03-24T00:48:45.979 INFO:tasks.workunit.client.1.smithi088.stdout:0/323: dread - d9/dc/de/d19/d44/d46/d65/f68 zero size 2024-03-24T00:48:45.979 INFO:tasks.workunit.client.1.smithi088.stdout:0/324: chown d9/dc/de/d19/f3b 286745 1 2024-03-24T00:48:45.979 INFO:tasks.workunit.client.1.smithi088.stdout:0/325: truncate d9/dc/de/d19/d44/f50 729136 0 2024-03-24T00:48:45.979 INFO:tasks.workunit.client.1.smithi088.stdout:0/326: chown d9/cd 10 1 2024-03-24T00:48:45.979 INFO:tasks.workunit.client.0.smithi038.stdout:3/314: mkdir d3/d30/d5d/d6a 0 2024-03-24T00:48:45.980 INFO:tasks.workunit.client.0.smithi038.stdout:3/315: chown d3/d9/dc/df/d14/d19 15575 1 2024-03-24T00:48:45.980 INFO:tasks.workunit.client.1.smithi088.stdout:4/329: mkdir d5/d7/d15/d74 0 2024-03-24T00:48:45.981 INFO:tasks.workunit.client.0.smithi038.stdout:1/252: dwrite da/dd/d10/d14/f38 [0,4194304] 0 2024-03-24T00:48:45.984 INFO:tasks.workunit.client.0.smithi038.stdout:9/347: dwrite d12/d28/d3a/f30 [0,4194304] 0 2024-03-24T00:48:45.984 INFO:tasks.workunit.client.0.smithi038.stdout:9/348: chown d12/d28/d3a/f6d 3 1 2024-03-24T00:48:45.990 INFO:tasks.workunit.client.1.smithi088.stdout:3/294: creat d3/d6/d58/f63 x:0 0 0 2024-03-24T00:48:45.990 INFO:tasks.workunit.client.1.smithi088.stdout:3/295: read d3/de/f61 [467351,19130] 0 2024-03-24T00:48:45.997 INFO:tasks.workunit.client.1.smithi088.stdout:6/340: mkdir db/d1d/d64 0 2024-03-24T00:48:46.003 INFO:tasks.workunit.client.1.smithi088.stdout:0/327: creat d9/dc/d4f/d62/f6a x:0 0 0 2024-03-24T00:48:46.005 INFO:tasks.workunit.client.0.smithi038.stdout:7/283: rmdir d2/d9/df 39 2024-03-24T00:48:46.006 INFO:tasks.workunit.client.1.smithi088.stdout:4/330: readlink d5/d7/dd/l5b 0 2024-03-24T00:48:46.008 INFO:tasks.workunit.client.1.smithi088.stdout:2/299: rename db/f4d to db/d21/f5f 0 2024-03-24T00:48:46.011 INFO:tasks.workunit.client.1.smithi088.stdout:6/341: mknod db/d1d/d23/d3c/d4b/c65 0 2024-03-24T00:48:46.012 INFO:tasks.workunit.client.1.smithi088.stdout:7/263: dread d3/f2c [0,4194304] 0 2024-03-24T00:48:46.013 INFO:tasks.workunit.client.0.smithi038.stdout:8/297: dwrite d0/d1d/d20/f58 [4194304,4194304] 0 2024-03-24T00:48:46.014 INFO:tasks.workunit.client.1.smithi088.stdout:0/328: rmdir d9/dc 39 2024-03-24T00:48:46.017 INFO:tasks.workunit.client.0.smithi038.stdout:1/253: rmdir da 39 2024-03-24T00:48:46.020 INFO:tasks.workunit.client.1.smithi088.stdout:4/331: creat d5/d7/dd/f75 x:0 0 0 2024-03-24T00:48:46.021 INFO:tasks.workunit.client.1.smithi088.stdout:6/342: dread db/d1d/f62 [0,4194304] 0 2024-03-24T00:48:46.021 INFO:tasks.workunit.client.1.smithi088.stdout:6/343: write db/d1d/d23/d3c/d43/f39 [4469667,93122] 0 2024-03-24T00:48:46.021 INFO:tasks.workunit.client.1.smithi088.stdout:6/344: fdatasync db/d40/d56/f30 0 2024-03-24T00:48:46.021 INFO:tasks.workunit.client.1.smithi088.stdout:6/345: truncate db/f47 4665431 0 2024-03-24T00:48:46.021 INFO:tasks.workunit.client.1.smithi088.stdout:6/346: fdatasync db/f38 0 2024-03-24T00:48:46.021 INFO:tasks.workunit.client.1.smithi088.stdout:6/347: fsync db/d1d/d23/f4c 0 2024-03-24T00:48:46.021 INFO:tasks.workunit.client.1.smithi088.stdout:2/300: creat db/d21/d35/f60 x:0 0 0 2024-03-24T00:48:46.021 INFO:tasks.workunit.client.1.smithi088.stdout:2/301: stat db/d21/d35/l39 0 2024-03-24T00:48:46.022 INFO:tasks.workunit.client.0.smithi038.stdout:9/349: write d12/d28/f4d [3264913,84336] 0 2024-03-24T00:48:46.022 INFO:tasks.workunit.client.1.smithi088.stdout:7/264: truncate d3/f1d 2229244 0 2024-03-24T00:48:46.022 INFO:tasks.workunit.client.1.smithi088.stdout:3/296: dread d3/de/df/f15 [0,4194304] 0 2024-03-24T00:48:46.024 INFO:tasks.workunit.client.1.smithi088.stdout:6/348: creat db/dd/f66 x:0 0 0 2024-03-24T00:48:46.026 INFO:tasks.workunit.client.0.smithi038.stdout:7/284: creat d2/d9/da/dc/d4a/f67 x:0 0 0 2024-03-24T00:48:46.027 INFO:tasks.workunit.client.1.smithi088.stdout:7/265: mknod d3/d32/c59 0 2024-03-24T00:48:46.028 INFO:tasks.workunit.client.0.smithi038.stdout:2/312: dwrite d4/d27/d2f/d30/d3e/f6e [4194304,4194304] 0 2024-03-24T00:48:46.028 INFO:tasks.workunit.client.0.smithi038.stdout:2/313: fsync d4/f66 0 2024-03-24T00:48:46.028 INFO:tasks.workunit.client.0.smithi038.stdout:2/314: creat d4/d18/d26/f70 x:0 0 0 2024-03-24T00:48:46.029 INFO:tasks.workunit.client.1.smithi088.stdout:3/297: symlink d3/d6/d27/l64 0 2024-03-24T00:48:46.031 INFO:tasks.workunit.client.0.smithi038.stdout:8/298: mkdir d0/d1d/d27/d6b 0 2024-03-24T00:48:46.032 INFO:tasks.workunit.client.0.smithi038.stdout:1/254: creat da/f58 x:0 0 0 2024-03-24T00:48:46.034 INFO:tasks.workunit.client.1.smithi088.stdout:3/298: creat d3/d6/d19/d2a/d5f/f65 x:0 0 0 2024-03-24T00:48:46.036 INFO:tasks.workunit.client.0.smithi038.stdout:9/350: mkdir d12/d28/d3a/d26/d79 0 2024-03-24T00:48:46.036 INFO:tasks.workunit.client.0.smithi038.stdout:9/351: getdents d12/d28/d3a/d26/d79 0 2024-03-24T00:48:46.039 INFO:tasks.workunit.client.0.smithi038.stdout:7/285: unlink d2/d9/fb 0 2024-03-24T00:48:46.043 INFO:tasks.workunit.client.1.smithi088.stdout:4/332: dread d5/d13/d18/d24/f30 [0,4194304] 0 2024-03-24T00:48:46.045 INFO:tasks.workunit.client.1.smithi088.stdout:0/329: write d9/dc/de/d19/d44/d46/d65/d2a/d2e/f5f [2757055,24797] 0 2024-03-24T00:48:46.048 INFO:tasks.workunit.client.1.smithi088.stdout:9/314: sync 2024-03-24T00:48:46.055 INFO:tasks.workunit.client.1.smithi088.stdout:6/349: write db/d1d/f48 [2889726,119011] 0 2024-03-24T00:48:46.055 INFO:tasks.workunit.client.1.smithi088.stdout:6/350: truncate db/dd/f2b 1167459 0 2024-03-24T00:48:46.055 INFO:tasks.workunit.client.1.smithi088.stdout:7/266: write d3/f1f [7382542,11654] 0 2024-03-24T00:48:46.055 INFO:tasks.workunit.client.1.smithi088.stdout:7/267: creat d3/d32/f5a x:0 0 0 2024-03-24T00:48:46.055 INFO:tasks.workunit.client.1.smithi088.stdout:6/351: write db/dd/f2b [1247801,84688] 0 2024-03-24T00:48:46.055 INFO:tasks.workunit.client.1.smithi088.stdout:7/268: fsync d3/f2f 0 2024-03-24T00:48:46.055 INFO:tasks.workunit.client.1.smithi088.stdout:6/352: readlink db/l27 0 2024-03-24T00:48:46.056 INFO:tasks.workunit.client.0.smithi038.stdout:4/260: dwrite da/d12/f4e [0,4194304] 0 2024-03-24T00:48:46.056 INFO:tasks.workunit.client.0.smithi038.stdout:4/261: readlink da/d12/l2d 0 2024-03-24T00:48:46.056 INFO:tasks.workunit.client.0.smithi038.stdout:2/315: getdents d4/d27/d54 0 2024-03-24T00:48:46.057 INFO:tasks.workunit.client.1.smithi088.stdout:1/336: dwrite d6/d27/d1e/d3f/f4b [0,4194304] 0 2024-03-24T00:48:46.057 INFO:tasks.workunit.client.1.smithi088.stdout:2/302: write db/d21/d35/f3d [2983653,101749] 0 2024-03-24T00:48:46.057 INFO:tasks.workunit.client.1.smithi088.stdout:2/303: dread - db/d21/d4a/f53 zero size 2024-03-24T00:48:46.057 INFO:tasks.workunit.client.1.smithi088.stdout:8/316: dwrite d9/f63 [0,4194304] 0 2024-03-24T00:48:46.058 INFO:tasks.workunit.client.0.smithi038.stdout:1/255: dread da/f49 [0,4194304] 0 2024-03-24T00:48:46.059 INFO:tasks.workunit.client.1.smithi088.stdout:5/309: dwrite d1/d10/d1b/d2f/f4a [0,4194304] 0 2024-03-24T00:48:46.061 INFO:tasks.workunit.client.0.smithi038.stdout:9/352: rename c11 to d12/d41/c7a 0 2024-03-24T00:48:46.090 INFO:tasks.workunit.client.1.smithi088.stdout:4/333: write d5/d13/f12 [1406452,19785] 0 2024-03-24T00:48:46.090 INFO:tasks.workunit.client.1.smithi088.stdout:4/334: chown d5/f56 897824 1 2024-03-24T00:48:46.090 INFO:tasks.workunit.client.1.smithi088.stdout:4/335: fsync d5/f4d 0 2024-03-24T00:48:46.093 INFO:tasks.workunit.client.0.smithi038.stdout:4/262: dread da/d12/d37/dd/f2b [0,4194304] 0 2024-03-24T00:48:46.157 INFO:tasks.workunit.client.1.smithi088.stdout:9/315: symlink d1/d10/d41/l55 0 2024-03-24T00:48:46.159 INFO:tasks.workunit.client.0.smithi038.stdout:3/316: dwrite d3/f29 [0,4194304] 0 2024-03-24T00:48:46.159 INFO:tasks.workunit.client.0.smithi038.stdout:3/317: creat d3/d9/dc/df/f6b x:0 0 0 2024-03-24T00:48:46.159 INFO:tasks.workunit.client.0.smithi038.stdout:3/318: creat d3/d9/dc/df/d14/d19/f6c x:0 0 0 2024-03-24T00:48:46.159 INFO:tasks.workunit.client.0.smithi038.stdout:3/319: dread - d3/d9/dc/d51/f52 zero size 2024-03-24T00:48:46.159 INFO:tasks.workunit.client.0.smithi038.stdout:3/320: chown d3/d9/dc/df/f6b 4740386 1 2024-03-24T00:48:46.161 INFO:tasks.workunit.client.0.smithi038.stdout:7/286: dwrite d2/d9/df/d33/f31 [0,4194304] 0 2024-03-24T00:48:46.166 INFO:tasks.workunit.client.1.smithi088.stdout:6/353: creat db/d1d/d23/d5c/f67 x:0 0 0 2024-03-24T00:48:46.172 INFO:tasks.workunit.client.1.smithi088.stdout:1/337: symlink d6/l62 0 2024-03-24T00:48:46.174 INFO:tasks.workunit.client.1.smithi088.stdout:2/304: rename db/f43 to db/d21/d4a/d4e/f61 0 2024-03-24T00:48:46.175 INFO:tasks.workunit.client.1.smithi088.stdout:1/338: write d6/d27/d1e/d5b/d5e/f60 [1013413,118157] 0 2024-03-24T00:48:46.180 INFO:tasks.workunit.client.0.smithi038.stdout:1/256: mknod da/dd/d10/c59 0 2024-03-24T00:48:46.180 INFO:tasks.workunit.client.0.smithi038.stdout:1/257: fsync da/f58 0 2024-03-24T00:48:46.180 INFO:tasks.workunit.client.0.smithi038.stdout:8/299: rmdir d0 39 2024-03-24T00:48:46.180 INFO:tasks.workunit.client.0.smithi038.stdout:8/300: write d0/d1d/d27/d4d/f53 [570651,12182] 0 2024-03-24T00:48:46.180 INFO:tasks.workunit.client.1.smithi088.stdout:8/317: dwrite d9/dc/d11/d24/d29/f2c [4194304,4194304] 0 2024-03-24T00:48:46.181 INFO:tasks.workunit.client.1.smithi088.stdout:7/269: dwrite d3/f20 [0,4194304] 0 2024-03-24T00:48:46.185 INFO:tasks.workunit.client.0.smithi038.stdout:4/263: mknod da/d38/c50 0 2024-03-24T00:48:46.185 INFO:tasks.workunit.client.0.smithi038.stdout:4/264: write da/d12/d37/dd/f34 [285175,60598] 0 2024-03-24T00:48:46.186 INFO:tasks.workunit.client.0.smithi038.stdout:6/340: sync 2024-03-24T00:48:46.186 INFO:tasks.workunit.client.0.smithi038.stdout:6/341: chown d1/da/d1b/d25/d49/d4e/c5d 2034 1 2024-03-24T00:48:46.187 INFO:tasks.workunit.client.0.smithi038.stdout:0/263: sync 2024-03-24T00:48:46.190 INFO:tasks.workunit.client.1.smithi088.stdout:7/270: dread d3/fd [0,4194304] 0 2024-03-24T00:48:46.192 INFO:tasks.workunit.client.0.smithi038.stdout:7/287: read d2/d2c/f4b [992798,109704] 0 2024-03-24T00:48:46.192 INFO:tasks.workunit.client.0.smithi038.stdout:9/353: dwrite d12/d28/f4c [0,4194304] 0 2024-03-24T00:48:46.203 INFO:tasks.workunit.client.1.smithi088.stdout:1/339: dread d6/d27/d1e/d3f/f4c [0,4194304] 0 2024-03-24T00:48:46.203 INFO:tasks.workunit.client.1.smithi088.stdout:1/340: chown f2 11440480 1 2024-03-24T00:48:46.204 INFO:tasks.workunit.client.0.smithi038.stdout:3/321: symlink d3/d2a/l6d 0 2024-03-24T00:48:46.205 INFO:tasks.workunit.client.0.smithi038.stdout:3/322: creat d3/d30/d5d/f6e x:0 0 0 2024-03-24T00:48:46.205 INFO:tasks.workunit.client.0.smithi038.stdout:3/323: chown d3/d9/dc/df/f31 509 1 2024-03-24T00:48:46.206 INFO:tasks.workunit.client.1.smithi088.stdout:7/271: write d3/d31/f38 [1121765,31278] 0 2024-03-24T00:48:46.211 INFO:tasks.workunit.client.0.smithi038.stdout:4/265: symlink da/d12/d29/l51 0 2024-03-24T00:48:46.216 INFO:tasks.workunit.client.0.smithi038.stdout:0/264: mknod d0/d12/c57 0 2024-03-24T00:48:46.216 INFO:tasks.workunit.client.0.smithi038.stdout:0/265: chown d0/f7 1065418 1 2024-03-24T00:48:46.217 INFO:tasks.workunit.client.0.smithi038.stdout:0/266: truncate d0/d2d/f2f 1481312 0 2024-03-24T00:48:46.217 INFO:tasks.workunit.client.0.smithi038.stdout:1/258: dread da/dd/d10/d14/f38 [0,4194304] 0 2024-03-24T00:48:46.217 INFO:tasks.workunit.client.0.smithi038.stdout:1/259: chown da/d3c/l1a 365238282 1 2024-03-24T00:48:46.221 INFO:tasks.workunit.client.0.smithi038.stdout:9/354: rename d12/d1a/c21 to d12/d28/d3a/d26/c7b 0 2024-03-24T00:48:46.227 INFO:tasks.workunit.client.0.smithi038.stdout:3/324: unlink d3/d30/l39 0 2024-03-24T00:48:46.227 INFO:tasks.workunit.client.0.smithi038.stdout:8/301: dread d0/d15/f17 [0,4194304] 0 2024-03-24T00:48:46.227 INFO:tasks.workunit.client.0.smithi038.stdout:8/302: write d0/d1d/d20/f60 [498878,70364] 0 2024-03-24T00:48:46.227 INFO:tasks.workunit.client.0.smithi038.stdout:8/303: readlink d0/d1d/d27/l5d 0 2024-03-24T00:48:46.231 INFO:tasks.workunit.client.1.smithi088.stdout:9/316: rename d1/d10/d2c/f35 to d1/d10/d41/f56 0 2024-03-24T00:48:46.231 INFO:tasks.workunit.client.1.smithi088.stdout:9/317: truncate d1/f20 5330042 0 2024-03-24T00:48:46.232 INFO:tasks.workunit.client.0.smithi038.stdout:1/260: read f5 [4640807,94267] 0 2024-03-24T00:48:46.232 INFO:tasks.workunit.client.0.smithi038.stdout:4/266: read da/d12/d24/f26 [434631,100756] 0 2024-03-24T00:48:46.233 INFO:tasks.workunit.client.1.smithi088.stdout:2/305: mknod db/c62 0 2024-03-24T00:48:46.234 INFO:tasks.workunit.client.1.smithi088.stdout:2/306: dread - db/f4f zero size 2024-03-24T00:48:46.234 INFO:tasks.workunit.client.1.smithi088.stdout:2/307: read db/f46 [531949,117168] 0 2024-03-24T00:48:46.234 INFO:tasks.workunit.client.1.smithi088.stdout:2/308: fdatasync db/dc/f2b 0 2024-03-24T00:48:46.238 INFO:tasks.workunit.client.0.smithi038.stdout:9/355: unlink d12/d41/l78 0 2024-03-24T00:48:46.239 INFO:tasks.workunit.client.0.smithi038.stdout:3/325: link d3/d9/dc/df/f3a d3/d9/dc/d51/f6f 0 2024-03-24T00:48:46.239 INFO:tasks.workunit.client.0.smithi038.stdout:3/326: write d3/d30/d5d/f6e [710228,38094] 0 2024-03-24T00:48:46.239 INFO:tasks.workunit.client.0.smithi038.stdout:3/327: readlink d3/d30/l4d 0 2024-03-24T00:48:46.239 INFO:tasks.workunit.client.0.smithi038.stdout:3/328: chown d3/d2a/c5b 167 1 2024-03-24T00:48:46.259 INFO:tasks.workunit.client.1.smithi088.stdout:9/318: dread d1/d22/f3f [0,4194304] 0 2024-03-24T00:48:46.264 INFO:tasks.workunit.client.0.smithi038.stdout:1/261: mkdir da/dd/d2c/d5a 0 2024-03-24T00:48:46.264 INFO:tasks.workunit.client.0.smithi038.stdout:8/304: mknod d0/d45/d66/c6c 0 2024-03-24T00:48:46.267 INFO:tasks.workunit.client.1.smithi088.stdout:8/318: mknod d9/dc/d11/d24/d29/d5b/c70 0 2024-03-24T00:48:46.267 INFO:tasks.workunit.client.1.smithi088.stdout:8/319: stat d9 0 2024-03-24T00:48:46.275 INFO:tasks.workunit.client.0.smithi038.stdout:3/329: mkdir d3/d30/d70 0 2024-03-24T00:48:46.278 INFO:tasks.workunit.client.0.smithi038.stdout:0/267: dread d0/d12/d15/d1a/f1e [0,4194304] 0 2024-03-24T00:48:46.283 INFO:tasks.workunit.client.0.smithi038.stdout:1/262: creat da/dd/d4a/f5b x:0 0 0 2024-03-24T00:48:46.292 INFO:tasks.workunit.client.0.smithi038.stdout:8/305: mkdir d0/d45/d66/d6d 0 2024-03-24T00:48:46.292 INFO:tasks.workunit.client.0.smithi038.stdout:3/330: rename d3/d9/dc/d51/l5c to d3/d9/dc/df/d14/l71 0 2024-03-24T00:48:46.292 INFO:tasks.workunit.client.0.smithi038.stdout:8/306: readlink d0/d1d/d27/l65 0 2024-03-24T00:48:46.293 INFO:tasks.workunit.client.1.smithi088.stdout:1/341: rmdir d6/d27/d1e/d3f/d53 39 2024-03-24T00:48:46.293 INFO:tasks.workunit.client.1.smithi088.stdout:5/310: getdents d1/d10/d1b/d2e 0 2024-03-24T00:48:46.293 INFO:tasks.workunit.client.1.smithi088.stdout:5/311: dread - d1/d10/f56 zero size 2024-03-24T00:48:46.293 INFO:tasks.workunit.client.1.smithi088.stdout:5/312: write d1/d43/d4e/f5c [471103,23210] 0 2024-03-24T00:48:46.293 INFO:tasks.workunit.client.1.smithi088.stdout:5/313: chown d1/d10/d1b/d2f/d57/f5f 12 1 2024-03-24T00:48:46.298 INFO:tasks.workunit.client.0.smithi038.stdout:1/263: mkdir da/dd/d22/d5c 0 2024-03-24T00:48:46.301 INFO:tasks.workunit.client.1.smithi088.stdout:4/336: rmdir d5/d16/d1a 39 2024-03-24T00:48:46.302 INFO:tasks.workunit.client.1.smithi088.stdout:6/354: truncate db/f47 138104 0 2024-03-24T00:48:46.303 INFO:tasks.workunit.client.1.smithi088.stdout:4/337: read d5/d7/fe [136685,100512] 0 2024-03-24T00:48:46.303 INFO:tasks.workunit.client.1.smithi088.stdout:4/338: write d5/d7/d72/f73 [470113,97827] 0 2024-03-24T00:48:46.313 INFO:tasks.workunit.client.1.smithi088.stdout:4/339: write d5/d13/d18/d24/f30 [356435,11933] 0 2024-03-24T00:48:46.313 INFO:tasks.workunit.client.1.smithi088.stdout:4/340: truncate d5/d7/dd/f65 10541 0 2024-03-24T00:48:46.313 INFO:tasks.workunit.client.1.smithi088.stdout:4/341: creat d5/d7/f76 x:0 0 0 2024-03-24T00:48:46.319 INFO:tasks.workunit.client.0.smithi038.stdout:0/268: dread d0/d12/d13/d28/f39 [0,4194304] 0 2024-03-24T00:48:46.319 INFO:tasks.workunit.client.0.smithi038.stdout:0/269: dread - d0/f43 zero size 2024-03-24T00:48:46.319 INFO:tasks.workunit.client.0.smithi038.stdout:0/270: read - d0/d12/f3c zero size 2024-03-24T00:48:46.320 INFO:tasks.workunit.client.0.smithi038.stdout:7/288: dwrite d2/d2c/f4b [0,4194304] 0 2024-03-24T00:48:46.321 INFO:tasks.workunit.client.0.smithi038.stdout:6/342: dwrite d1/da/d1b/d25/f2a [4194304,4194304] 0 2024-03-24T00:48:46.321 INFO:tasks.workunit.client.0.smithi038.stdout:6/343: fdatasync d1/da/d1b/d25/d49/d4e/f55 0 2024-03-24T00:48:46.324 INFO:tasks.workunit.client.0.smithi038.stdout:3/331: symlink d3/d9/dc/df/d14/d19/l72 0 2024-03-24T00:48:46.324 INFO:tasks.workunit.client.1.smithi088.stdout:6/355: dread db/dd/f37 [0,4194304] 0 2024-03-24T00:48:46.325 INFO:tasks.workunit.client.1.smithi088.stdout:8/320: rmdir d9/dc/d11 39 2024-03-24T00:48:46.325 INFO:tasks.workunit.client.1.smithi088.stdout:6/356: creat db/d1d/f68 x:0 0 0 2024-03-24T00:48:46.325 INFO:tasks.workunit.client.1.smithi088.stdout:8/321: creat d9/dc/d11/d24/f71 x:0 0 0 2024-03-24T00:48:46.325 INFO:tasks.workunit.client.1.smithi088.stdout:8/322: write d9/f57 [600340,70380] 0 2024-03-24T00:48:46.325 INFO:tasks.workunit.client.1.smithi088.stdout:8/323: chown d9/dc/l3a 32404323 1 2024-03-24T00:48:46.325 INFO:tasks.workunit.client.0.smithi038.stdout:8/307: symlink d0/d10/d19/l6e 0 2024-03-24T00:48:46.326 INFO:tasks.workunit.client.1.smithi088.stdout:9/319: truncate d1/d10/d29/f45 173343 0 2024-03-24T00:48:46.326 INFO:tasks.workunit.client.1.smithi088.stdout:9/320: dread - d1/d10/d11/d23/f53 zero size 2024-03-24T00:48:46.326 INFO:tasks.workunit.client.1.smithi088.stdout:9/321: stat d1/d10/d2c/c3d 0 2024-03-24T00:48:46.328 INFO:tasks.workunit.client.0.smithi038.stdout:1/264: creat da/dd/d2c/d40/f5d x:0 0 0 2024-03-24T00:48:46.328 INFO:tasks.workunit.client.0.smithi038.stdout:1/265: fdatasync da/dd/d10/d14/f2e 0 2024-03-24T00:48:46.332 INFO:tasks.workunit.client.1.smithi088.stdout:1/342: mknod d6/d27/c63 0 2024-03-24T00:48:46.332 INFO:tasks.workunit.client.1.smithi088.stdout:5/314: mkdir d1/d43/d4e/d5b/d66 0 2024-03-24T00:48:46.333 INFO:tasks.workunit.client.1.smithi088.stdout:1/343: dread - d6/d43/f50 zero size 2024-03-24T00:48:46.333 INFO:tasks.workunit.client.1.smithi088.stdout:1/344: stat d6/l46 0 2024-03-24T00:48:46.333 INFO:tasks.workunit.client.1.smithi088.stdout:5/315: fsync d1/dd/f54 0 2024-03-24T00:48:46.334 INFO:tasks.workunit.client.0.smithi038.stdout:0/271: truncate d0/d12/d13/d28/f39 1384152 0 2024-03-24T00:48:46.335 INFO:tasks.workunit.client.1.smithi088.stdout:8/324: dread d9/dc/d11/d24/f45 [0,4194304] 0 2024-03-24T00:48:46.338 INFO:tasks.workunit.client.0.smithi038.stdout:7/289: unlink d2/d2c/f57 0 2024-03-24T00:48:46.338 INFO:tasks.workunit.client.1.smithi088.stdout:6/357: mknod db/dd/d61/c69 0 2024-03-24T00:48:46.339 INFO:tasks.workunit.client.1.smithi088.stdout:2/309: dwrite db/f46 [0,4194304] 0 2024-03-24T00:48:46.343 INFO:tasks.workunit.client.1.smithi088.stdout:9/322: creat d1/d10/d41/f57 x:0 0 0 2024-03-24T00:48:46.345 INFO:tasks.workunit.client.0.smithi038.stdout:9/356: dwrite d12/d28/d3a/f30 [0,4194304] 0 2024-03-24T00:48:46.345 INFO:tasks.workunit.client.1.smithi088.stdout:2/310: dread db/d21/f2e [0,4194304] 0 2024-03-24T00:48:46.347 INFO:tasks.workunit.client.1.smithi088.stdout:1/345: unlink d6/d27/c1b 0 2024-03-24T00:48:46.349 INFO:tasks.workunit.client.1.smithi088.stdout:8/325: symlink d9/dc/d11/d24/d43/l72 0 2024-03-24T00:48:46.350 INFO:tasks.workunit.client.1.smithi088.stdout:2/311: dread db/d21/f5f [0,4194304] 0 2024-03-24T00:48:46.350 INFO:tasks.workunit.client.1.smithi088.stdout:2/312: read db/f18 [20680,22989] 0 2024-03-24T00:48:46.350 INFO:tasks.workunit.client.1.smithi088.stdout:2/313: stat ca 0 2024-03-24T00:48:46.351 INFO:tasks.workunit.client.1.smithi088.stdout:6/358: mkdir db/d40/d55/d6a 0 2024-03-24T00:48:46.353 INFO:tasks.workunit.client.0.smithi038.stdout:7/290: write d2/d9/df/d33/f30 [3723660,3525] 0 2024-03-24T00:48:46.356 INFO:tasks.workunit.client.1.smithi088.stdout:7/272: dwrite d3/ff [4194304,4194304] 0 2024-03-24T00:48:46.361 INFO:tasks.workunit.client.0.smithi038.stdout:4/267: dwrite da/d12/d37/dd/f3a [0,4194304] 0 2024-03-24T00:48:46.362 INFO:tasks.workunit.client.0.smithi038.stdout:3/332: creat d3/d5a/f73 x:0 0 0 2024-03-24T00:48:46.362 INFO:tasks.workunit.client.0.smithi038.stdout:3/333: stat d3/d5a/c64 0 2024-03-24T00:48:46.363 INFO:tasks.workunit.client.1.smithi088.stdout:8/326: dread d9/f63 [0,4194304] 0 2024-03-24T00:48:46.393 INFO:tasks.workunit.client.0.smithi038.stdout:8/308: mknod d0/d45/c6f 0 2024-03-24T00:48:46.395 INFO:tasks.workunit.client.1.smithi088.stdout:1/346: creat d6/d27/d1e/d5b/d5e/f64 x:0 0 0 2024-03-24T00:48:46.395 INFO:tasks.workunit.client.1.smithi088.stdout:1/347: write d6/d27/d1e/d3f/f54 [956134,75668] 0 2024-03-24T00:48:46.395 INFO:tasks.workunit.client.0.smithi038.stdout:6/344: dwrite d1/f1f [0,4194304] 0 2024-03-24T00:48:46.397 INFO:tasks.workunit.client.1.smithi088.stdout:8/327: dread d9/f20 [0,4194304] 0 2024-03-24T00:48:46.397 INFO:tasks.workunit.client.1.smithi088.stdout:8/328: fsync d9/dc/d11/d24/d29/f51 0 2024-03-24T00:48:46.397 INFO:tasks.workunit.client.1.smithi088.stdout:4/342: dwrite d5/d7/d72/f71 [0,4194304] 0 2024-03-24T00:48:46.401 INFO:tasks.workunit.client.0.smithi038.stdout:1/266: symlink da/dd/d4a/l5e 0 2024-03-24T00:48:46.406 INFO:tasks.workunit.client.1.smithi088.stdout:7/273: creat d3/d31/d56/f5b x:0 0 0 2024-03-24T00:48:46.406 INFO:tasks.workunit.client.1.smithi088.stdout:7/274: readlink d3/l13 0 2024-03-24T00:48:46.406 INFO:tasks.workunit.client.1.smithi088.stdout:6/359: dwrite db/d1d/f48 [0,4194304] 0 2024-03-24T00:48:46.406 INFO:tasks.workunit.client.1.smithi088.stdout:6/360: read db/f13 [2149064,65462] 0 2024-03-24T00:48:46.407 INFO:tasks.workunit.client.1.smithi088.stdout:3/299: sync 2024-03-24T00:48:46.408 INFO:tasks.workunit.client.1.smithi088.stdout:0/330: sync 2024-03-24T00:48:46.408 INFO:tasks.workunit.client.1.smithi088.stdout:3/300: truncate d3/d1a/d1e/d2f/f36 1638289 0 2024-03-24T00:48:46.408 INFO:tasks.workunit.client.1.smithi088.stdout:3/301: dread - d3/d6/d19/d2a/f53 zero size 2024-03-24T00:48:46.408 INFO:tasks.workunit.client.1.smithi088.stdout:3/302: chown d3/d1a/d1e/d48/f50 424 1 2024-03-24T00:48:46.417 INFO:tasks.workunit.client.1.smithi088.stdout:0/331: dread d9/f45 [0,4194304] 0 2024-03-24T00:48:46.419 INFO:tasks.workunit.client.1.smithi088.stdout:1/348: dread d6/d23/f34 [0,4194304] 0 2024-03-24T00:48:46.427 INFO:tasks.workunit.client.1.smithi088.stdout:8/329: write f1 [356070,36613] 0 2024-03-24T00:48:46.436 INFO:tasks.workunit.client.1.smithi088.stdout:4/343: dread f2 [4194304,4194304] 0 2024-03-24T00:48:46.436 INFO:tasks.workunit.client.1.smithi088.stdout:4/344: write d5/d16/d37/f5a [545266,37604] 0 2024-03-24T00:48:46.436 INFO:tasks.workunit.client.1.smithi088.stdout:4/345: creat d5/d13/d18/d24/d3d/f77 x:0 0 0 2024-03-24T00:48:46.437 INFO:tasks.workunit.client.1.smithi088.stdout:4/346: dread - d5/d7/f14 zero size 2024-03-24T00:48:46.440 INFO:tasks.workunit.client.1.smithi088.stdout:5/316: dwrite d1/d10/f52 [0,4194304] 0 2024-03-24T00:48:46.458 INFO:tasks.workunit.client.1.smithi088.stdout:0/332: dwrite d9/dc/de/d19/d44/d46/d65/f58 [4194304,4194304] 0 2024-03-24T00:48:46.462 INFO:tasks.workunit.client.0.smithi038.stdout:5/274: sync 2024-03-24T00:48:46.471 INFO:tasks.workunit.client.0.smithi038.stdout:1/267: dwrite f7 [8388608,4194304] 0 2024-03-24T00:48:46.476 INFO:tasks.workunit.client.0.smithi038.stdout:5/275: dread d6/f18 [0,4194304] 0 2024-03-24T00:48:46.476 INFO:tasks.workunit.client.0.smithi038.stdout:5/276: fsync d6/d16/f2a 0 2024-03-24T00:48:46.477 INFO:tasks.workunit.client.1.smithi088.stdout:0/333: dread d9/dc/de/d19/d44/d46/d65/f35 [4194304,4194304] 0 2024-03-24T00:48:46.505 INFO:tasks.workunit.client.1.smithi088.stdout:8/330: dwrite d9/f57 [0,4194304] 0 2024-03-24T00:48:46.508 INFO:tasks.workunit.client.1.smithi088.stdout:0/334: dwrite d9/d26/f61 [0,4194304] 0 2024-03-24T00:48:46.508 INFO:tasks.workunit.client.1.smithi088.stdout:0/335: truncate d9/dc/f52 721159 0 2024-03-24T00:48:46.508 INFO:tasks.workunit.client.1.smithi088.stdout:0/336: chown d9/f69 43 1 2024-03-24T00:48:46.524 INFO:tasks.workunit.client.0.smithi038.stdout:8/309: creat d0/d1d/d27/d4d/f70 x:0 0 0 2024-03-24T00:48:46.533 INFO:tasks.workunit.client.0.smithi038.stdout:6/345: symlink d1/da/d1b/d25/d49/l65 0 2024-03-24T00:48:46.533 INFO:tasks.workunit.client.0.smithi038.stdout:6/346: chown d1/l19 35981888 1 2024-03-24T00:48:46.534 INFO:tasks.workunit.client.0.smithi038.stdout:6/347: fdatasync d1/da/d1b/d1d/f2d 0 2024-03-24T00:48:46.537 INFO:tasks.workunit.client.0.smithi038.stdout:3/334: getdents d3/d2a 0 2024-03-24T00:48:46.543 INFO:tasks.workunit.client.0.smithi038.stdout:1/268: mkdir da/d3c/d2d/d5f 0 2024-03-24T00:48:46.546 INFO:tasks.workunit.client.0.smithi038.stdout:5/277: mknod d6/d19/d20/d42/d43/d4c/c58 0 2024-03-24T00:48:46.547 INFO:tasks.workunit.client.0.smithi038.stdout:8/310: write d0/d1d/d20/d3c/f47 [750216,105193] 0 2024-03-24T00:48:46.558 INFO:tasks.workunit.client.0.smithi038.stdout:8/311: dread d0/d10/d19/f4f [0,4194304] 0 2024-03-24T00:48:46.559 INFO:tasks.workunit.client.0.smithi038.stdout:3/335: write d3/f25 [687399,19875] 0 2024-03-24T00:48:46.560 INFO:tasks.workunit.client.1.smithi088.stdout:6/361: mknod db/d1d/d64/c6b 0 2024-03-24T00:48:46.563 INFO:tasks.workunit.client.0.smithi038.stdout:6/348: creat d1/da/d1b/d25/d49/d4e/f66 x:0 0 0 2024-03-24T00:48:46.565 INFO:tasks.workunit.client.0.smithi038.stdout:1/269: dread da/f49 [0,4194304] 0 2024-03-24T00:48:46.566 INFO:tasks.workunit.client.1.smithi088.stdout:3/303: mknod d3/d1a/d1e/d2f/c66 0 2024-03-24T00:48:46.566 INFO:tasks.workunit.client.0.smithi038.stdout:1/270: write da/dd/d10/d14/f38 [5223291,126280] 0 2024-03-24T00:48:46.566 INFO:tasks.workunit.client.0.smithi038.stdout:1/271: dread - da/dd/d22/f55 zero size 2024-03-24T00:48:46.574 INFO:tasks.workunit.client.0.smithi038.stdout:5/278: write d6/d19/f1c [53973,121590] 0 2024-03-24T00:48:46.577 INFO:tasks.workunit.client.1.smithi088.stdout:4/347: mkdir d5/d13/d63/d78 0 2024-03-24T00:48:46.579 INFO:tasks.workunit.client.0.smithi038.stdout:1/272: mkdir da/dd/d2c/d41/d60 0 2024-03-24T00:48:46.581 INFO:tasks.workunit.client.1.smithi088.stdout:5/317: symlink d1/d10/d1b/d2f/l67 0 2024-03-24T00:48:46.588 INFO:tasks.workunit.client.0.smithi038.stdout:1/273: symlink da/dd/d2c/d5a/l61 0 2024-03-24T00:48:46.592 INFO:tasks.workunit.client.0.smithi038.stdout:1/274: read da/f23 [488090,38241] 0 2024-03-24T00:48:46.597 INFO:tasks.workunit.client.1.smithi088.stdout:9/323: rename d1/f6 to d1/d22/d27/d39/f58 0 2024-03-24T00:48:46.602 INFO:tasks.workunit.client.0.smithi038.stdout:7/291: mknod d2/d9/da/c68 0 2024-03-24T00:48:46.602 INFO:tasks.workunit.client.0.smithi038.stdout:3/336: symlink d3/d9/dc/df/d14/l74 0 2024-03-24T00:48:46.607 INFO:tasks.workunit.client.1.smithi088.stdout:2/314: symlink db/l63 0 2024-03-24T00:48:46.608 INFO:tasks.workunit.client.1.smithi088.stdout:2/315: write db/f4f [373404,71414] 0 2024-03-24T00:48:46.608 INFO:tasks.workunit.client.0.smithi038.stdout:8/312: dwrite d0/d45/d37/f44 [0,4194304] 0 2024-03-24T00:48:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:8/313: rename d0/d1d/l31 to d0/d10/d19/l71 0 2024-03-24T00:48:46.622 INFO:tasks.workunit.client.1.smithi088.stdout:6/362: creat db/d40/d60/f6c x:0 0 0 2024-03-24T00:48:46.622 INFO:tasks.workunit.client.1.smithi088.stdout:6/363: chown db/f47 120778217 1 2024-03-24T00:48:46.623 INFO:tasks.workunit.client.0.smithi038.stdout:3/337: dread d3/d9/dc/df/f1d [0,4194304] 0 2024-03-24T00:48:46.623 INFO:tasks.workunit.client.0.smithi038.stdout:3/338: chown d3/d5a/c69 639575647 1 2024-03-24T00:48:46.624 INFO:tasks.workunit.client.0.smithi038.stdout:0/272: mkdir d0/d12/d15/d58 0 2024-03-24T00:48:46.624 INFO:tasks.workunit.client.0.smithi038.stdout:0/273: dread - d0/d12/d15/d1a/f2b zero size 2024-03-24T00:48:46.624 INFO:tasks.workunit.client.1.smithi088.stdout:3/304: mkdir d3/d67 0 2024-03-24T00:48:46.624 INFO:tasks.workunit.client.1.smithi088.stdout:3/305: write d3/de/f61 [2345144,130951] 0 2024-03-24T00:48:46.624 INFO:tasks.workunit.client.1.smithi088.stdout:4/348: creat d5/d13/d18/f79 x:0 0 0 2024-03-24T00:48:46.631 INFO:tasks.workunit.client.0.smithi038.stdout:3/339: rmdir d3/d9/dc/df/d14 39 2024-03-24T00:48:46.646 INFO:tasks.workunit.client.1.smithi088.stdout:9/324: mknod d1/d10/d11/c59 0 2024-03-24T00:48:46.646 INFO:tasks.workunit.client.1.smithi088.stdout:1/349: unlink d6/d27/f3c 0 2024-03-24T00:48:46.646 INFO:tasks.workunit.client.1.smithi088.stdout:6/364: unlink db/d40/d56/c44 0 2024-03-24T00:48:46.646 INFO:tasks.workunit.client.1.smithi088.stdout:9/325: write d1/d22/f43 [1518662,101925] 0 2024-03-24T00:48:46.651 INFO:tasks.workunit.client.1.smithi088.stdout:3/306: dread d3/d6/f11 [0,4194304] 0 2024-03-24T00:48:46.651 INFO:tasks.workunit.client.1.smithi088.stdout:3/307: chown d3/d32 265 1 2024-03-24T00:48:46.651 INFO:tasks.workunit.client.1.smithi088.stdout:3/308: write d3/d6/f11 [2501977,81278] 0 2024-03-24T00:48:46.651 INFO:tasks.workunit.client.1.smithi088.stdout:3/309: dread - d3/d1a/f5a zero size 2024-03-24T00:48:46.656 INFO:tasks.workunit.client.1.smithi088.stdout:1/350: write d6/d27/d1e/d3f/f54 [388801,58133] 0 2024-03-24T00:48:46.656 INFO:tasks.workunit.client.1.smithi088.stdout:4/349: unlink d5/d7/dd/l34 0 2024-03-24T00:48:46.656 INFO:tasks.workunit.client.1.smithi088.stdout:9/326: rename d1/d22/c52 to d1/d15/c5a 0 2024-03-24T00:48:46.660 INFO:tasks.workunit.client.1.smithi088.stdout:1/351: link d6/d27/d1e/f39 d6/d27/d1e/d58/f65 0 2024-03-24T00:48:46.663 INFO:tasks.workunit.client.0.smithi038.stdout:8/314: dread d0/d1d/d20/f58 [0,4194304] 0 2024-03-24T00:48:46.675 INFO:tasks.workunit.client.0.smithi038.stdout:0/274: rename d0/d12/d15/d1a/c26 to d0/d12/c59 0 2024-03-24T00:48:46.675 INFO:tasks.workunit.client.0.smithi038.stdout:0/275: dread - d0/f43 zero size 2024-03-24T00:48:46.687 INFO:tasks.workunit.client.0.smithi038.stdout:1/275: dwrite f4 [8388608,4194304] 0 2024-03-24T00:48:46.692 INFO:tasks.workunit.client.0.smithi038.stdout:1/276: mknod da/dd/d2c/c62 0 2024-03-24T00:48:46.693 INFO:tasks.workunit.client.1.smithi088.stdout:8/331: mkdir d9/dc/d11/d24/d29/d73 0 2024-03-24T00:48:46.694 INFO:tasks.workunit.client.1.smithi088.stdout:8/332: write d9/dc/d11/d24/d43/d4a/f55 [439905,118622] 0 2024-03-24T00:48:46.695 INFO:tasks.workunit.client.0.smithi038.stdout:0/276: read d0/d12/d15/d1a/f1e [300608,126842] 0 2024-03-24T00:48:46.695 INFO:tasks.workunit.client.1.smithi088.stdout:4/350: unlink d5/f4d 0 2024-03-24T00:48:46.698 INFO:tasks.workunit.client.1.smithi088.stdout:1/352: dread d6/d23/f34 [0,4194304] 0 2024-03-24T00:48:46.707 INFO:tasks.workunit.client.1.smithi088.stdout:2/316: dwrite db/d21/d4a/f4b [0,4194304] 0 2024-03-24T00:48:46.714 INFO:tasks.workunit.client.0.smithi038.stdout:1/277: write da/f16 [2321900,28387] 0 2024-03-24T00:48:46.714 INFO:tasks.workunit.client.0.smithi038.stdout:1/278: fdatasync f4 0 2024-03-24T00:48:46.715 INFO:tasks.workunit.client.0.smithi038.stdout:8/315: dwrite d0/d45/f55 [0,4194304] 0 2024-03-24T00:48:46.715 INFO:tasks.workunit.client.0.smithi038.stdout:8/316: fdatasync d0/f16 0 2024-03-24T00:48:46.715 INFO:tasks.workunit.client.0.smithi038.stdout:8/317: write d0/d1d/d27/d4d/f70 [695702,2200] 0 2024-03-24T00:48:46.716 INFO:tasks.workunit.client.0.smithi038.stdout:8/318: chown d0 105 1 2024-03-24T00:48:46.719 INFO:tasks.workunit.client.0.smithi038.stdout:1/279: creat da/dd/d2c/f63 x:0 0 0 2024-03-24T00:48:46.720 INFO:tasks.workunit.client.1.smithi088.stdout:1/353: dread d6/d27/d1e/d3f/f54 [0,4194304] 0 2024-03-24T00:48:46.723 INFO:tasks.workunit.client.1.smithi088.stdout:2/317: creat db/d21/d4a/d4e/f64 x:0 0 0 2024-03-24T00:48:46.723 INFO:tasks.workunit.client.1.smithi088.stdout:2/318: fdatasync db/f3c 0 2024-03-24T00:48:46.728 INFO:tasks.workunit.client.1.smithi088.stdout:4/351: symlink d5/d7/l7a 0 2024-03-24T00:48:46.728 INFO:tasks.workunit.client.1.smithi088.stdout:2/319: getdents db/d21/d35/d49 0 2024-03-24T00:48:46.729 INFO:tasks.workunit.client.1.smithi088.stdout:1/354: dread d6/d27/d1e/f22 [0,4194304] 0 2024-03-24T00:48:46.729 INFO:tasks.workunit.client.1.smithi088.stdout:1/355: creat d6/d27/d1e/d5b/d5e/f66 x:0 0 0 2024-03-24T00:48:46.730 INFO:tasks.workunit.client.0.smithi038.stdout:7/292: dwrite d2/f1f [0,4194304] 0 2024-03-24T00:48:46.732 INFO:tasks.workunit.client.0.smithi038.stdout:8/319: rename d0/d45/c6f to d0/d10/d19/d62/c72 0 2024-03-24T00:48:46.734 INFO:tasks.workunit.client.0.smithi038.stdout:1/280: unlink da/dd/d10/d14/f38 0 2024-03-24T00:48:46.734 INFO:tasks.workunit.client.0.smithi038.stdout:1/281: write da/f49 [1459304,49732] 0 2024-03-24T00:48:46.734 INFO:tasks.workunit.client.0.smithi038.stdout:1/282: chown da/d3c/c20 439455 1 2024-03-24T00:48:46.735 INFO:tasks.workunit.client.1.smithi088.stdout:2/320: rmdir db/d21 39 2024-03-24T00:48:46.740 INFO:tasks.workunit.client.0.smithi038.stdout:7/293: rename d2/d9/d16/f19 to d2/d2c/f69 0 2024-03-24T00:48:46.741 INFO:tasks.workunit.client.0.smithi038.stdout:5/279: mknod d6/c59 0 2024-03-24T00:48:46.754 INFO:tasks.workunit.client.0.smithi038.stdout:8/320: mknod d0/d45/d66/c73 0 2024-03-24T00:48:46.757 INFO:tasks.workunit.client.1.smithi088.stdout:8/333: dwrite d9/dc/d11/d24/d29/d36/f4e [0,4194304] 0 2024-03-24T00:48:46.758 INFO:tasks.workunit.client.1.smithi088.stdout:1/356: mkdir d6/d27/d67 0 2024-03-24T00:48:46.758 INFO:tasks.workunit.client.1.smithi088.stdout:2/321: write db/d21/d4a/f4b [3414248,52839] 0 2024-03-24T00:48:46.765 INFO:tasks.workunit.client.0.smithi038.stdout:8/321: mknod d0/c74 0 2024-03-24T00:48:46.775 INFO:tasks.workunit.client.0.smithi038.stdout:8/322: dread d0/d1d/d20/f60 [0,4194304] 0 2024-03-24T00:48:46.784 INFO:tasks.workunit.client.1.smithi088.stdout:1/357: write d6/d27/d1e/d58/f65 [306415,72614] 0 2024-03-24T00:48:46.786 INFO:tasks.workunit.client.0.smithi038.stdout:3/340: dwrite d3/d9/dc/df/f27 [0,4194304] 0 2024-03-24T00:48:46.786 INFO:tasks.workunit.client.0.smithi038.stdout:3/341: creat d3/d9/dc/d51/f75 x:0 0 0 2024-03-24T00:48:46.790 INFO:tasks.workunit.client.1.smithi088.stdout:6/365: rmdir db/d40/d60 39 2024-03-24T00:48:46.792 INFO:tasks.workunit.client.0.smithi038.stdout:6/349: creat d1/da/f67 x:0 0 0 2024-03-24T00:48:46.800 INFO:tasks.workunit.client.1.smithi088.stdout:1/358: dread d6/d27/d1e/d3f/f4c [0,4194304] 0 2024-03-24T00:48:46.800 INFO:tasks.workunit.client.1.smithi088.stdout:1/359: stat d6/d27/d1e/d5b/d5e 0 2024-03-24T00:48:46.801 INFO:tasks.workunit.client.1.smithi088.stdout:5/318: mkdir d1/d10/d68 0 2024-03-24T00:48:46.802 INFO:tasks.workunit.client.1.smithi088.stdout:6/366: rmdir db/d40/d56 39 2024-03-24T00:48:46.802 INFO:tasks.workunit.client.1.smithi088.stdout:6/367: truncate db/d40/f5a 594647 0 2024-03-24T00:48:46.807 INFO:tasks.workunit.client.0.smithi038.stdout:4/268: mknod da/d12/c52 0 2024-03-24T00:48:46.811 INFO:tasks.workunit.client.1.smithi088.stdout:5/319: mkdir d1/d19/d1e/d26/d69 0 2024-03-24T00:48:46.811 INFO:tasks.workunit.client.1.smithi088.stdout:3/310: rename d3/d1a/c3f to d3/de/df/d31/c68 0 2024-03-24T00:48:46.811 INFO:tasks.workunit.client.1.smithi088.stdout:6/368: mkdir db/dd/d61/d6d 0 2024-03-24T00:48:46.822 INFO:tasks.workunit.client.1.smithi088.stdout:9/327: rename d1/d10/d11/d28/l30 to d1/d15/l5b 0 2024-03-24T00:48:46.830 INFO:tasks.workunit.client.0.smithi038.stdout:4/269: unlink da/d12/d37/dd/d15/d33/c48 0 2024-03-24T00:48:46.830 INFO:tasks.workunit.client.0.smithi038.stdout:4/270: write da/f1e [140869,76637] 0 2024-03-24T00:48:46.834 INFO:tasks.workunit.client.1.smithi088.stdout:3/311: link d3/de/f17 d3/d6/d19/d2a/f69 0 2024-03-24T00:48:46.835 INFO:tasks.workunit.client.1.smithi088.stdout:8/334: dwrite d9/f20 [4194304,4194304] 0 2024-03-24T00:48:46.840 INFO:tasks.workunit.client.1.smithi088.stdout:7/275: symlink d3/l5c 0 2024-03-24T00:48:46.840 INFO:tasks.workunit.client.1.smithi088.stdout:7/276: readlink d3/l22 0 2024-03-24T00:48:46.840 INFO:tasks.workunit.client.1.smithi088.stdout:7/277: chown d3/d32/f42 28719 1 2024-03-24T00:48:46.844 INFO:tasks.workunit.client.0.smithi038.stdout:4/271: rename da/d38/f4d to da/d12/d37/dd/d15/d4c/f53 0 2024-03-24T00:48:46.847 INFO:tasks.workunit.client.1.smithi088.stdout:9/328: read d1/f5 [3075055,94929] 0 2024-03-24T00:48:46.847 INFO:tasks.workunit.client.1.smithi088.stdout:9/329: dread - d1/d22/f51 zero size 2024-03-24T00:48:46.855 INFO:tasks.workunit.client.1.smithi088.stdout:1/360: rename d6/d27/f1a to d6/f68 0 2024-03-24T00:48:46.866 INFO:tasks.workunit.client.0.smithi038.stdout:4/272: creat da/d12/d37/f54 x:0 0 0 2024-03-24T00:48:46.868 INFO:tasks.workunit.client.1.smithi088.stdout:3/312: symlink d3/d1a/d1e/d48/d4b/l6a 0 2024-03-24T00:48:46.868 INFO:tasks.workunit.client.1.smithi088.stdout:7/278: dread d3/f28 [0,4194304] 0 2024-03-24T00:48:46.868 INFO:tasks.workunit.client.1.smithi088.stdout:7/279: readlink d3/l22 0 2024-03-24T00:48:46.873 INFO:tasks.workunit.client.0.smithi038.stdout:3/342: dread d3/f8 [4194304,4194304] 0 2024-03-24T00:48:46.874 INFO:tasks.workunit.client.1.smithi088.stdout:4/352: dwrite d5/d16/f29 [0,4194304] 0 2024-03-24T00:48:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:7/294: dwrite d2/d9/df/d33/f30 [0,4194304] 0 2024-03-24T00:48:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:4/273: dread da/d12/d37/dd/f34 [0,4194304] 0 2024-03-24T00:48:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:4/274: chown da/d12/d24/c2f 627315574 1 2024-03-24T00:48:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:1/283: dwrite f7 [12582912,4194304] 0 2024-03-24T00:48:46.883 INFO:tasks.workunit.client.1.smithi088.stdout:8/335: mkdir d9/dc/d11/d41/d54/d74 0 2024-03-24T00:48:46.885 INFO:tasks.workunit.client.0.smithi038.stdout:3/343: rename d3/d30/d5d/f6e to d3/d30/d5d/d6a/f76 0 2024-03-24T00:48:46.885 INFO:tasks.workunit.client.0.smithi038.stdout:5/280: dwrite d6/d19/f1c [4194304,4194304] 0 2024-03-24T00:48:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:8/323: dwrite d0/d45/f55 [0,4194304] 0 2024-03-24T00:48:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:6/350: dwrite d1/f1f [0,4194304] 0 2024-03-24T00:48:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:6/351: chown d1/da/d1b/d1d/l4d 84741 1 2024-03-24T00:48:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:6/352: read d1/f7 [1907231,121218] 0 2024-03-24T00:48:46.892 INFO:tasks.workunit.client.1.smithi088.stdout:2/322: dwrite db/dc/f2c [0,4194304] 0 2024-03-24T00:48:46.896 INFO:tasks.workunit.client.0.smithi038.stdout:7/295: link d2/d9/df/c10 d2/d9/df/c6a 0 2024-03-24T00:48:46.896 INFO:tasks.workunit.client.0.smithi038.stdout:7/296: stat d2/ce 0 2024-03-24T00:48:46.900 INFO:tasks.workunit.client.1.smithi088.stdout:9/330: unlink d1/d10/d11/l32 0 2024-03-24T00:48:46.903 INFO:tasks.workunit.client.0.smithi038.stdout:9/357: sync 2024-03-24T00:48:46.903 INFO:tasks.workunit.client.0.smithi038.stdout:2/316: sync 2024-03-24T00:48:46.903 INFO:tasks.workunit.client.0.smithi038.stdout:9/358: dread - d12/d28/d3a/d26/f5a zero size 2024-03-24T00:48:46.904 INFO:tasks.workunit.client.0.smithi038.stdout:9/359: read f6 [3895673,71214] 0 2024-03-24T00:48:46.904 INFO:tasks.workunit.client.0.smithi038.stdout:9/360: truncate d12/f1e 642353 0 2024-03-24T00:48:46.907 INFO:tasks.workunit.client.1.smithi088.stdout:0/337: sync 2024-03-24T00:48:46.907 INFO:tasks.workunit.client.1.smithi088.stdout:0/338: chown d9/d26/c60 47 1 2024-03-24T00:48:46.907 INFO:tasks.workunit.client.0.smithi038.stdout:1/284: write da/d3c/f13 [13687851,105620] 0 2024-03-24T00:48:46.914 INFO:tasks.workunit.client.0.smithi038.stdout:8/324: read d0/d45/f55 [691625,96339] 0 2024-03-24T00:48:46.914 INFO:tasks.workunit.client.0.smithi038.stdout:8/325: fsync d0/d45/d37/f44 0 2024-03-24T00:48:46.920 INFO:tasks.workunit.client.0.smithi038.stdout:3/344: mkdir d3/d9/d77 0 2024-03-24T00:48:46.921 INFO:tasks.workunit.client.1.smithi088.stdout:1/361: mknod d6/c69 0 2024-03-24T00:48:46.921 INFO:tasks.workunit.client.1.smithi088.stdout:3/313: stat d3/d1a/l21 0 2024-03-24T00:48:46.921 INFO:tasks.workunit.client.1.smithi088.stdout:1/362: write d6/d27/d1e/d58/f51 [2119283,75193] 0 2024-03-24T00:48:46.922 INFO:tasks.workunit.client.0.smithi038.stdout:5/281: mknod d6/d19/c5a 0 2024-03-24T00:48:46.923 INFO:tasks.workunit.client.1.smithi088.stdout:7/280: mknod d3/c5d 0 2024-03-24T00:48:46.928 INFO:tasks.workunit.client.0.smithi038.stdout:6/353: mkdir d1/d37/d68 0 2024-03-24T00:48:46.928 INFO:tasks.workunit.client.0.smithi038.stdout:6/354: chown d1/da/f34 2548 1 2024-03-24T00:48:46.928 INFO:tasks.workunit.client.0.smithi038.stdout:6/355: chown d1/da/d1b 4 1 2024-03-24T00:48:46.928 INFO:tasks.workunit.client.0.smithi038.stdout:6/356: write d1/da/d1b/d1d/f43 [524296,130235] 0 2024-03-24T00:48:46.938 INFO:tasks.workunit.client.0.smithi038.stdout:1/285: write da/dd/f1c [3295939,24953] 0 2024-03-24T00:48:46.945 INFO:tasks.workunit.client.0.smithi038.stdout:4/275: rmdir da/d12/d37 39 2024-03-24T00:48:46.949 INFO:tasks.workunit.client.1.smithi088.stdout:2/323: creat db/d54/f65 x:0 0 0 2024-03-24T00:48:46.949 INFO:tasks.workunit.client.1.smithi088.stdout:1/363: creat d6/d27/d67/f6a x:0 0 0 2024-03-24T00:48:46.949 INFO:tasks.workunit.client.1.smithi088.stdout:1/364: fsync d6/d27/d1e/f39 0 2024-03-24T00:48:46.949 INFO:tasks.workunit.client.1.smithi088.stdout:1/365: dread - d6/d43/f50 zero size 2024-03-24T00:48:46.949 INFO:tasks.workunit.client.1.smithi088.stdout:1/366: readlink d6/l12 0 2024-03-24T00:48:46.950 INFO:tasks.workunit.client.1.smithi088.stdout:7/281: rename d3/f14 to d3/d32/d3f/d4b/f5e 0 2024-03-24T00:48:46.951 INFO:tasks.workunit.client.0.smithi038.stdout:7/297: rename d2/d9/da/c38 to d2/d9/da/dc/c6b 0 2024-03-24T00:48:46.951 INFO:tasks.workunit.client.0.smithi038.stdout:7/298: creat d2/d2c/f6c x:0 0 0 2024-03-24T00:48:46.954 INFO:tasks.workunit.client.0.smithi038.stdout:9/361: creat d12/d55/d56/f7c x:0 0 0 2024-03-24T00:48:46.954 INFO:tasks.workunit.client.0.smithi038.stdout:9/362: chown d12/l22 67982548 1 2024-03-24T00:48:46.954 INFO:tasks.workunit.client.0.smithi038.stdout:9/363: dread - d12/d28/d3a/f6d zero size 2024-03-24T00:48:46.954 INFO:tasks.workunit.client.0.smithi038.stdout:9/364: chown d12/d28/d3a/d26/f5a 1462028 1 2024-03-24T00:48:46.957 INFO:tasks.workunit.client.0.smithi038.stdout:3/345: creat d3/f78 x:0 0 0 2024-03-24T00:48:46.958 INFO:tasks.workunit.client.0.smithi038.stdout:3/346: creat d3/d5a/f79 x:0 0 0 2024-03-24T00:48:46.958 INFO:tasks.workunit.client.0.smithi038.stdout:3/347: dread - d3/d9/dc/d51/f75 zero size 2024-03-24T00:48:46.958 INFO:tasks.workunit.client.0.smithi038.stdout:3/348: dread - d3/d30/f34 zero size 2024-03-24T00:48:46.958 INFO:tasks.workunit.client.0.smithi038.stdout:3/349: chown d3/d9/dc/df/c43 60 1 2024-03-24T00:48:46.958 INFO:tasks.workunit.client.0.smithi038.stdout:1/286: dread f8 [0,4194304] 0 2024-03-24T00:48:46.959 INFO:tasks.workunit.client.1.smithi088.stdout:0/339: rmdir d9/d26 39 2024-03-24T00:48:46.959 INFO:tasks.workunit.client.0.smithi038.stdout:5/282: mkdir d6/d19/d20/d42/d43/d4c/d5b 0 2024-03-24T00:48:46.968 INFO:tasks.workunit.client.0.smithi038.stdout:7/299: rename d2/d9/da/f14 to d2/d9/da/dc/d4a/f6d 0 2024-03-24T00:48:46.975 INFO:tasks.workunit.client.1.smithi088.stdout:0/340: getdents d9/dc/de/d19/d44/d46/d65 0 2024-03-24T00:48:46.978 INFO:tasks.workunit.client.0.smithi038.stdout:5/283: mknod d6/d19/d20/d33/c5c 0 2024-03-24T00:48:46.981 INFO:tasks.workunit.client.1.smithi088.stdout:9/331: dread d1/f7 [0,4194304] 0 2024-03-24T00:48:46.981 INFO:tasks.workunit.client.1.smithi088.stdout:9/332: write d1/f8 [8186652,67605] 0 2024-03-24T00:48:46.984 INFO:tasks.workunit.client.1.smithi088.stdout:8/336: dwrite d9/dc/d11/d41/d54/d5d/f60 [0,4194304] 0 2024-03-24T00:48:46.997 INFO:tasks.workunit.client.0.smithi038.stdout:2/317: rename d4/f41 to d4/d27/d54/d67/f71 0 2024-03-24T00:48:47.000 INFO:tasks.workunit.client.1.smithi088.stdout:7/282: dread d3/f1f [0,4194304] 0 2024-03-24T00:48:47.000 INFO:tasks.workunit.client.0.smithi038.stdout:7/300: mknod d2/d9/da/d23/c6e 0 2024-03-24T00:48:47.001 INFO:tasks.workunit.client.1.smithi088.stdout:9/333: truncate d1/d22/d27/d39/f4d 283868 0 2024-03-24T00:48:47.009 INFO:tasks.workunit.client.0.smithi038.stdout:8/326: dwrite d0/d45/d37/f61 [0,4194304] 0 2024-03-24T00:48:47.009 INFO:tasks.workunit.client.0.smithi038.stdout:8/327: creat d0/d1d/d27/d4d/f75 x:0 0 0 2024-03-24T00:48:47.013 INFO:tasks.workunit.client.1.smithi088.stdout:8/337: rename d9/c18 to d9/dc/d11/d24/d29/d5b/c75 0 2024-03-24T00:48:47.015 INFO:tasks.workunit.client.0.smithi038.stdout:2/318: creat d4/d27/f72 x:0 0 0 2024-03-24T00:48:47.016 INFO:tasks.workunit.client.1.smithi088.stdout:7/283: link d3/f20 d3/f5f 0 2024-03-24T00:48:47.017 INFO:tasks.workunit.client.0.smithi038.stdout:7/301: creat d2/d9/d16/f6f x:0 0 0 2024-03-24T00:48:47.018 INFO:tasks.workunit.client.1.smithi088.stdout:8/338: creat d9/dc/d11/d24/d29/d73/f76 x:0 0 0 2024-03-24T00:48:47.018 INFO:tasks.workunit.client.1.smithi088.stdout:8/339: write d9/dc/f16 [665478,108276] 0 2024-03-24T00:48:47.019 INFO:tasks.workunit.client.1.smithi088.stdout:7/284: creat d3/d32/d3f/d4b/f60 x:0 0 0 2024-03-24T00:48:47.019 INFO:tasks.workunit.client.1.smithi088.stdout:7/285: readlink d3/l11 0 2024-03-24T00:48:47.023 INFO:tasks.workunit.client.1.smithi088.stdout:8/340: unlink d9/dc/d11/d24/f40 0 2024-03-24T00:48:47.025 INFO:tasks.workunit.client.0.smithi038.stdout:8/328: getdents d0/d45/d66 0 2024-03-24T00:48:47.026 INFO:tasks.workunit.client.0.smithi038.stdout:5/284: write d6/d1e/f55 [2037791,119459] 0 2024-03-24T00:48:47.027 INFO:tasks.workunit.client.1.smithi088.stdout:0/341: write d9/dc/de/d19/d44/f50 [308770,116715] 0 2024-03-24T00:48:47.027 INFO:tasks.workunit.client.1.smithi088.stdout:0/342: stat d9/dc/de/d19/d44/d46/d65/f39 0 2024-03-24T00:48:47.028 INFO:tasks.workunit.client.1.smithi088.stdout:0/343: chown d9/dc/d4f/d62/f6a 12982015 1 2024-03-24T00:48:47.028 INFO:tasks.workunit.client.0.smithi038.stdout:7/302: getdents d2/d9 0 2024-03-24T00:48:47.029 INFO:tasks.workunit.client.0.smithi038.stdout:2/319: link d4/d27/d2f/d30/d3e/l46 d4/d27/d54/l73 0 2024-03-24T00:48:47.032 INFO:tasks.workunit.client.0.smithi038.stdout:8/329: creat d0/f76 x:0 0 0 2024-03-24T00:48:47.034 INFO:tasks.workunit.client.0.smithi038.stdout:5/285: rmdir d6/d16/d14 39 2024-03-24T00:48:47.036 INFO:tasks.workunit.client.0.smithi038.stdout:7/303: creat d2/d9/df/d65/f70 x:0 0 0 2024-03-24T00:48:47.038 INFO:tasks.workunit.client.0.smithi038.stdout:2/320: symlink d4/d18/d5b/l74 0 2024-03-24T00:48:47.038 INFO:tasks.workunit.client.1.smithi088.stdout:1/367: dwrite d6/d27/f3a [0,4194304] 0 2024-03-24T00:48:47.039 INFO:tasks.workunit.client.1.smithi088.stdout:1/368: creat d6/d27/d67/f6b x:0 0 0 2024-03-24T00:48:47.039 INFO:tasks.workunit.client.1.smithi088.stdout:7/286: dread d3/f1d [0,4194304] 0 2024-03-24T00:48:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:8/330: mknod d0/d45/d66/c77 0 2024-03-24T00:48:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:8/331: write d0/d10/d19/f4f [909751,88951] 0 2024-03-24T00:48:47.044 INFO:tasks.workunit.client.0.smithi038.stdout:1/287: dwrite da/f23 [0,4194304] 0 2024-03-24T00:48:47.045 INFO:tasks.workunit.client.0.smithi038.stdout:5/286: truncate d6/f9 792866 0 2024-03-24T00:48:47.051 INFO:tasks.workunit.client.0.smithi038.stdout:4/276: dwrite da/d38/f4a [0,4194304] 0 2024-03-24T00:48:47.051 INFO:tasks.workunit.client.0.smithi038.stdout:6/357: dwrite d1/da/d1b/d1d/f43 [0,4194304] 0 2024-03-24T00:48:47.052 INFO:tasks.workunit.client.1.smithi088.stdout:3/314: dwrite d3/d6/d58/f63 [0,4194304] 0 2024-03-24T00:48:47.052 INFO:tasks.workunit.client.1.smithi088.stdout:4/353: dwrite d5/d7/d72/f73 [0,4194304] 0 2024-03-24T00:48:47.052 INFO:tasks.workunit.client.1.smithi088.stdout:3/315: stat d3/de/f16 0 2024-03-24T00:48:47.054 INFO:tasks.workunit.client.1.smithi088.stdout:0/344: write d9/f1c [564457,62263] 0 2024-03-24T00:48:47.054 INFO:tasks.workunit.client.1.smithi088.stdout:0/345: chown d9/dc/de/f3e 0 1 2024-03-24T00:48:47.055 INFO:tasks.workunit.client.1.smithi088.stdout:2/324: dwrite db/f38 [0,4194304] 0 2024-03-24T00:48:47.061 INFO:tasks.workunit.client.0.smithi038.stdout:2/321: creat d4/d27/d2f/f75 x:0 0 0 2024-03-24T00:48:47.071 INFO:tasks.workunit.client.0.smithi038.stdout:5/287: mknod d6/d16/d21/c5d 0 2024-03-24T00:48:47.072 INFO:tasks.workunit.client.0.smithi038.stdout:5/288: readlink d6/d16/d21/d2d/l40 0 2024-03-24T00:48:47.085 INFO:tasks.workunit.client.1.smithi088.stdout:4/354: rename d5/d17/f33 to d5/d16/f7b 0 2024-03-24T00:48:47.085 INFO:tasks.workunit.client.1.smithi088.stdout:4/355: truncate d5/d16/f57 327318 0 2024-03-24T00:48:47.088 INFO:tasks.workunit.client.1.smithi088.stdout:3/316: link d3/de/df/f15 d3/d1a/d1e/d2f/d45/f6b 0 2024-03-24T00:48:47.094 INFO:tasks.workunit.client.1.smithi088.stdout:0/346: mknod d9/dc/de/d19/d44/d46/d65/d2a/c6b 0 2024-03-24T00:48:47.099 INFO:tasks.workunit.client.0.smithi038.stdout:4/277: link da/d38/l44 da/d27/l55 0 2024-03-24T00:48:47.102 INFO:tasks.workunit.client.1.smithi088.stdout:2/325: mknod db/dd/c66 0 2024-03-24T00:48:47.102 INFO:tasks.workunit.client.1.smithi088.stdout:2/326: dread - db/dc/f2b zero size 2024-03-24T00:48:47.102 INFO:tasks.workunit.client.1.smithi088.stdout:4/356: dread f4 [0,4194304] 0 2024-03-24T00:48:47.102 INFO:tasks.workunit.client.1.smithi088.stdout:3/317: dread d3/d6/d27/d3c/f59 [0,4194304] 0 2024-03-24T00:48:47.103 INFO:tasks.workunit.client.1.smithi088.stdout:1/369: write d6/d23/f2d [2106492,3137] 0 2024-03-24T00:48:47.103 INFO:tasks.workunit.client.1.smithi088.stdout:8/341: dwrite d9/dc/d11/d24/d29/d5b/f34 [8388608,4194304] 0 2024-03-24T00:48:47.105 INFO:tasks.workunit.client.0.smithi038.stdout:5/289: truncate d6/d1e/f1f 2271724 0 2024-03-24T00:48:47.106 INFO:tasks.workunit.client.1.smithi088.stdout:3/318: rename d3/d6/l41 to d3/d67/l6c 0 2024-03-24T00:48:47.108 INFO:tasks.workunit.client.1.smithi088.stdout:1/370: mknod d6/d27/d1e/d58/c6c 0 2024-03-24T00:48:47.108 INFO:tasks.workunit.client.1.smithi088.stdout:1/371: read - d6/d43/f4f zero size 2024-03-24T00:48:47.109 INFO:tasks.workunit.client.1.smithi088.stdout:8/342: symlink d9/dc/d11/d41/d54/d5d/l77 0 2024-03-24T00:48:47.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:46 smithi038 ceph-mon[29271]: pgmap v29: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 145 MiB/s rd, 181 MiB/s wr, 564 op/s 2024-03-24T00:48:47.114 INFO:tasks.workunit.client.1.smithi088.stdout:3/319: mknod d3/d6/d19/d2a/c6d 0 2024-03-24T00:48:47.114 INFO:tasks.workunit.client.1.smithi088.stdout:3/320: chown d3/d6/d19/d2a/d5f/f65 11090 1 2024-03-24T00:48:47.114 INFO:tasks.workunit.client.1.smithi088.stdout:3/321: chown d3/de/df/f15 156 1 2024-03-24T00:48:47.115 INFO:tasks.workunit.client.1.smithi088.stdout:1/372: creat d6/f6d x:0 0 0 2024-03-24T00:48:47.116 INFO:tasks.workunit.client.1.smithi088.stdout:5/320: sync 2024-03-24T00:48:47.117 INFO:tasks.workunit.client.1.smithi088.stdout:6/369: sync 2024-03-24T00:48:47.117 INFO:tasks.workunit.client.1.smithi088.stdout:6/370: readlink l7 0 2024-03-24T00:48:47.118 INFO:tasks.workunit.client.1.smithi088.stdout:3/322: mkdir d3/d6/d27/d56/d6e 0 2024-03-24T00:48:47.121 INFO:tasks.workunit.client.1.smithi088.stdout:2/327: dread db/dd/f2f [0,4194304] 0 2024-03-24T00:48:47.121 INFO:tasks.workunit.client.1.smithi088.stdout:8/343: rename d9/dc/d11/d24/d29/c4c to d9/dc/d11/c78 0 2024-03-24T00:48:47.122 INFO:tasks.workunit.client.1.smithi088.stdout:1/373: dread d6/f2a [0,4194304] 0 2024-03-24T00:48:47.122 INFO:tasks.workunit.client.1.smithi088.stdout:1/374: fdatasync d6/d23/f33 0 2024-03-24T00:48:47.122 INFO:tasks.workunit.client.1.smithi088.stdout:5/321: mknod d1/d19/d1e/d26/d69/c6a 0 2024-03-24T00:48:47.123 INFO:tasks.workunit.client.1.smithi088.stdout:6/371: creat db/d1d/d23/d3c/d4b/f6e x:0 0 0 2024-03-24T00:48:47.127 INFO:tasks.workunit.client.1.smithi088.stdout:2/328: rename db/d21/c42 to db/dc/c67 0 2024-03-24T00:48:47.127 INFO:tasks.workunit.client.1.smithi088.stdout:1/375: mkdir d6/d6e 0 2024-03-24T00:48:47.129 INFO:tasks.workunit.client.1.smithi088.stdout:5/322: symlink d1/d10/d1b/d2f/l6b 0 2024-03-24T00:48:47.129 INFO:tasks.workunit.client.1.smithi088.stdout:5/323: read d1/d10/d1b/d2f/f38 [3886771,105054] 0 2024-03-24T00:48:47.130 INFO:tasks.workunit.client.1.smithi088.stdout:6/372: creat db/d1d/d50/f6f x:0 0 0 2024-03-24T00:48:47.132 INFO:tasks.workunit.client.0.smithi038.stdout:1/288: write f4 [8896197,2273] 0 2024-03-24T00:48:47.132 INFO:tasks.workunit.client.0.smithi038.stdout:1/289: write da/dd/d22/f4f [29031,19663] 0 2024-03-24T00:48:47.139 INFO:tasks.workunit.client.0.smithi038.stdout:8/332: dwrite d0/d1d/d27/d4d/f53 [0,4194304] 0 2024-03-24T00:48:47.141 INFO:tasks.workunit.client.0.smithi038.stdout:7/304: dwrite d2/d9/da/dc/d4a/f6d [0,4194304] 0 2024-03-24T00:48:47.141 INFO:tasks.workunit.client.0.smithi038.stdout:1/290: unlink l3 0 2024-03-24T00:48:47.142 INFO:tasks.workunit.client.1.smithi088.stdout:1/376: symlink d6/d6e/l6f 0 2024-03-24T00:48:47.142 INFO:tasks.workunit.client.0.smithi038.stdout:0/277: sync 2024-03-24T00:48:47.143 INFO:tasks.workunit.client.0.smithi038.stdout:0/278: write d0/fc [292382,65249] 0 2024-03-24T00:48:47.143 INFO:tasks.workunit.client.1.smithi088.stdout:5/324: unlink d1/d10/c3e 0 2024-03-24T00:48:47.147 INFO:tasks.workunit.client.0.smithi038.stdout:2/322: dwrite d4/f2b [4194304,4194304] 0 2024-03-24T00:48:47.153 INFO:tasks.workunit.client.1.smithi088.stdout:5/325: unlink d1/dd/f23 0 2024-03-24T00:48:47.159 INFO:tasks.workunit.client.0.smithi038.stdout:7/305: rename d2/d2c/f55 to d2/d9/df/f71 0 2024-03-24T00:48:47.159 INFO:tasks.workunit.client.0.smithi038.stdout:7/306: dread - d2/d9/da/dc/d4a/f67 zero size 2024-03-24T00:48:47.159 INFO:tasks.workunit.client.1.smithi088.stdout:7/287: dwrite d3/d32/d34/f37 [4194304,4194304] 0 2024-03-24T00:48:47.159 INFO:tasks.workunit.client.1.smithi088.stdout:7/288: fsync d3/ff 0 2024-03-24T00:48:47.163 INFO:tasks.workunit.client.1.smithi088.stdout:1/377: dread d6/d27/d1e/d58/f48 [0,4194304] 0 2024-03-24T00:48:47.168 INFO:tasks.workunit.client.0.smithi038.stdout:8/333: write d0/d15/f17 [4035567,52998] 0 2024-03-24T00:48:47.169 INFO:tasks.workunit.client.1.smithi088.stdout:5/326: rmdir d1/d10/d68 0 2024-03-24T00:48:47.169 INFO:tasks.workunit.client.1.smithi088.stdout:5/327: fdatasync d1/d19/f45 0 2024-03-24T00:48:47.169 INFO:tasks.workunit.client.1.smithi088.stdout:5/328: write d1/d19/f45 [1582746,39696] 0 2024-03-24T00:48:47.171 INFO:tasks.workunit.client.0.smithi038.stdout:1/291: creat da/d3c/d2d/d5f/f64 x:0 0 0 2024-03-24T00:48:47.173 INFO:tasks.workunit.client.0.smithi038.stdout:0/279: mknod d0/d12/d15/c5a 0 2024-03-24T00:48:47.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:46 smithi088 ceph-mon[36610]: pgmap v29: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 145 MiB/s rd, 181 MiB/s wr, 564 op/s 2024-03-24T00:48:47.187 INFO:tasks.workunit.client.0.smithi038.stdout:7/307: mkdir d2/d9/da/d23/d2b/d5b/d72 0 2024-03-24T00:48:47.194 INFO:tasks.workunit.client.0.smithi038.stdout:1/292: creat da/dd/d2c/d41/f65 x:0 0 0 2024-03-24T00:48:47.207 INFO:tasks.workunit.client.1.smithi088.stdout:7/289: creat d3/d32/d3f/f61 x:0 0 0 2024-03-24T00:48:47.207 INFO:tasks.workunit.client.1.smithi088.stdout:7/290: readlink d3/l12 0 2024-03-24T00:48:47.208 INFO:tasks.workunit.client.1.smithi088.stdout:7/291: stat d3/d32/d3f/f61 0 2024-03-24T00:48:47.208 INFO:tasks.workunit.client.0.smithi038.stdout:1/293: symlink da/dd/d2c/d5a/l66 0 2024-03-24T00:48:47.209 INFO:tasks.workunit.client.1.smithi088.stdout:8/344: dread d9/dc/d11/f2f [0,4194304] 0 2024-03-24T00:48:47.209 INFO:tasks.workunit.client.1.smithi088.stdout:8/345: truncate d9/dc/d11/d24/d29/d36/f4d 800977 0 2024-03-24T00:48:47.209 INFO:tasks.workunit.client.1.smithi088.stdout:8/346: fdatasync d9/f2b 0 2024-03-24T00:48:47.209 INFO:tasks.workunit.client.1.smithi088.stdout:5/329: creat d1/d19/f6c x:0 0 0 2024-03-24T00:48:47.215 INFO:tasks.workunit.client.1.smithi088.stdout:4/357: dwrite d5/d7/f14 [0,4194304] 0 2024-03-24T00:48:47.216 INFO:tasks.workunit.client.1.smithi088.stdout:4/358: stat d5/d7/f44 0 2024-03-24T00:48:47.223 INFO:tasks.workunit.client.0.smithi038.stdout:4/278: dwrite da/d12/d24/f26 [0,4194304] 0 2024-03-24T00:48:47.224 INFO:tasks.workunit.client.0.smithi038.stdout:5/290: dwrite d6/d23/f32 [0,4194304] 0 2024-03-24T00:48:47.226 INFO:tasks.workunit.client.1.smithi088.stdout:2/329: dwrite db/f55 [0,4194304] 0 2024-03-24T00:48:47.226 INFO:tasks.workunit.client.1.smithi088.stdout:2/330: fsync db/d54/f65 0 2024-03-24T00:48:47.226 INFO:tasks.workunit.client.1.smithi088.stdout:2/331: readlink db/l63 0 2024-03-24T00:48:47.226 INFO:tasks.workunit.client.1.smithi088.stdout:2/332: fdatasync db/d54/f65 0 2024-03-24T00:48:47.232 INFO:tasks.workunit.client.0.smithi038.stdout:5/291: rename d6/d16/d21/c5d to d6/d16/d21/c5e 0 2024-03-24T00:48:47.238 INFO:tasks.workunit.client.1.smithi088.stdout:3/323: dwrite d3/d6/d19/d2a/d5f/f65 [0,4194304] 0 2024-03-24T00:48:47.249 INFO:tasks.workunit.client.1.smithi088.stdout:7/292: write d3/f1b [3300313,26013] 0 2024-03-24T00:48:47.252 INFO:tasks.workunit.client.0.smithi038.stdout:1/294: write da/dd/f2f [200141,15150] 0 2024-03-24T00:48:47.252 INFO:tasks.workunit.client.0.smithi038.stdout:1/295: fdatasync da/d3c/d2d/f36 0 2024-03-24T00:48:47.260 INFO:tasks.workunit.client.0.smithi038.stdout:4/279: dread da/d38/f4a [0,4194304] 0 2024-03-24T00:48:47.260 INFO:tasks.workunit.client.0.smithi038.stdout:5/292: dread d6/f9 [0,4194304] 0 2024-03-24T00:48:47.266 INFO:tasks.workunit.client.0.smithi038.stdout:4/280: mkdir da/d56 0 2024-03-24T00:48:47.268 INFO:tasks.workunit.client.0.smithi038.stdout:5/293: getdents d6/d19/d20/d33 0 2024-03-24T00:48:47.268 INFO:tasks.workunit.client.0.smithi038.stdout:5/294: readlink d6/d19/d20/l30 0 2024-03-24T00:48:47.270 INFO:tasks.workunit.client.0.smithi038.stdout:4/281: rmdir da/d56 0 2024-03-24T00:48:47.271 INFO:tasks.workunit.client.0.smithi038.stdout:5/295: creat d6/d16/d21/f5f x:0 0 0 2024-03-24T00:48:47.271 INFO:tasks.workunit.client.0.smithi038.stdout:5/296: write d6/f18 [2813443,126516] 0 2024-03-24T00:48:47.272 INFO:tasks.workunit.client.0.smithi038.stdout:4/282: creat da/d12/f57 x:0 0 0 2024-03-24T00:48:47.275 INFO:tasks.workunit.client.1.smithi088.stdout:3/324: dread d3/d6/d27/d3c/f30 [0,4194304] 0 2024-03-24T00:48:47.275 INFO:tasks.workunit.client.1.smithi088.stdout:2/333: rmdir db/d54 39 2024-03-24T00:48:47.275 INFO:tasks.workunit.client.1.smithi088.stdout:2/334: stat db/dd/l1e 0 2024-03-24T00:48:47.277 INFO:tasks.workunit.client.1.smithi088.stdout:5/330: dwrite d1/d10/d1b/d2e/f65 [0,4194304] 0 2024-03-24T00:48:47.277 INFO:tasks.workunit.client.0.smithi038.stdout:8/334: dwrite d0/d45/f55 [0,4194304] 0 2024-03-24T00:48:47.278 INFO:tasks.workunit.client.0.smithi038.stdout:1/296: dread f7 [8388608,4194304] 0 2024-03-24T00:48:47.278 INFO:tasks.workunit.client.0.smithi038.stdout:1/297: creat da/d3c/d2d/d5f/f67 x:0 0 0 2024-03-24T00:48:47.279 INFO:tasks.workunit.client.0.smithi038.stdout:5/297: rmdir d6/d16/d3c 39 2024-03-24T00:48:47.279 INFO:tasks.workunit.client.0.smithi038.stdout:5/298: stat d6/d16/d14/c31 0 2024-03-24T00:48:47.282 INFO:tasks.workunit.client.1.smithi088.stdout:6/373: dwrite db/dd/f41 [4194304,4194304] 0 2024-03-24T00:48:47.283 INFO:tasks.workunit.client.0.smithi038.stdout:0/280: dwrite d0/d1b/f4a [0,4194304] 0 2024-03-24T00:48:47.284 INFO:tasks.workunit.client.1.smithi088.stdout:1/378: dwrite d6/d27/f1d [0,4194304] 0 2024-03-24T00:48:47.287 INFO:tasks.workunit.client.1.smithi088.stdout:8/347: dwrite f2 [4194304,4194304] 0 2024-03-24T00:48:47.287 INFO:tasks.workunit.client.1.smithi088.stdout:8/348: dread - d9/dc/d11/d24/f58 zero size 2024-03-24T00:48:47.287 INFO:tasks.workunit.client.1.smithi088.stdout:8/349: truncate d9/dc/d11/d24/f45 2027169 0 2024-03-24T00:48:47.287 INFO:tasks.workunit.client.1.smithi088.stdout:8/350: truncate d9/dc/d11/d24/d43/d4a/f55 1589287 0 2024-03-24T00:48:47.288 INFO:tasks.workunit.client.0.smithi038.stdout:1/298: getdents da/dd/d10 0 2024-03-24T00:48:47.303 INFO:tasks.workunit.client.0.smithi038.stdout:4/283: dread da/d12/d24/f30 [0,4194304] 0 2024-03-24T00:48:47.307 INFO:tasks.workunit.client.0.smithi038.stdout:2/323: dwrite d4/d27/d2f/d30/d3e/f61 [0,4194304] 0 2024-03-24T00:48:47.310 INFO:tasks.workunit.client.0.smithi038.stdout:7/308: dwrite d2/d2c/f6c [0,4194304] 0 2024-03-24T00:48:47.312 INFO:tasks.workunit.client.1.smithi088.stdout:7/293: dwrite d3/d32/d3f/d4b/f60 [0,4194304] 0 2024-03-24T00:48:47.315 INFO:tasks.workunit.client.1.smithi088.stdout:6/374: symlink db/d40/d56/l70 0 2024-03-24T00:48:47.317 INFO:tasks.workunit.client.0.smithi038.stdout:0/281: rename d0/d2d/c33 to d0/c5b 0 2024-03-24T00:48:47.317 INFO:tasks.workunit.client.0.smithi038.stdout:0/282: dread - d0/d12/f3c zero size 2024-03-24T00:48:47.329 INFO:tasks.workunit.client.1.smithi088.stdout:2/335: dwrite db/f3c [0,4194304] 0 2024-03-24T00:48:47.333 INFO:tasks.workunit.client.0.smithi038.stdout:5/299: mkdir d6/d60 0 2024-03-24T00:48:47.333 INFO:tasks.workunit.client.0.smithi038.stdout:5/300: write d6/d19/f1c [1812577,47750] 0 2024-03-24T00:48:47.333 INFO:tasks.workunit.client.0.smithi038.stdout:5/301: chown d6/d19/d20/d42/d43/d50/l51 115 1 2024-03-24T00:48:47.333 INFO:tasks.workunit.client.0.smithi038.stdout:5/302: chown d6/d19/d20/d42/d43/d50/l51 15664 1 2024-03-24T00:48:47.334 INFO:tasks.workunit.client.0.smithi038.stdout:1/299: mknod da/c68 0 2024-03-24T00:48:47.334 INFO:tasks.workunit.client.0.smithi038.stdout:1/300: chown da/l3f 78781559 1 2024-03-24T00:48:47.334 INFO:tasks.workunit.client.0.smithi038.stdout:1/301: chown da 1 1 2024-03-24T00:48:47.344 INFO:tasks.workunit.client.1.smithi088.stdout:1/379: mkdir d6/d27/d1e/d3f/d70 0 2024-03-24T00:48:47.346 INFO:tasks.workunit.client.1.smithi088.stdout:8/351: mknod d9/dc/d11/d41/d54/d74/c79 0 2024-03-24T00:48:47.346 INFO:tasks.workunit.client.1.smithi088.stdout:8/352: chown d9/dc/d11/d41/d54/d5d/f60 325721 1 2024-03-24T00:48:47.349 INFO:tasks.workunit.client.1.smithi088.stdout:3/325: dwrite d3/d6/d27/d3c/f42 [0,4194304] 0 2024-03-24T00:48:47.350 INFO:tasks.workunit.client.1.smithi088.stdout:9/334: sync 2024-03-24T00:48:47.354 INFO:tasks.workunit.client.1.smithi088.stdout:6/375: mkdir db/d1d/d64/d71 0 2024-03-24T00:48:47.354 INFO:tasks.workunit.client.1.smithi088.stdout:5/331: dwrite d1/dd/f15 [0,4194304] 0 2024-03-24T00:48:47.355 INFO:tasks.workunit.client.0.smithi038.stdout:8/335: dwrite d0/d45/f55 [0,4194304] 0 2024-03-24T00:48:47.356 INFO:tasks.workunit.client.1.smithi088.stdout:2/336: mknod db/d21/d4a/d4e/c68 0 2024-03-24T00:48:47.360 INFO:tasks.workunit.client.1.smithi088.stdout:6/376: dread db/d1d/f62 [0,4194304] 0 2024-03-24T00:48:47.364 INFO:tasks.workunit.client.1.smithi088.stdout:3/326: readlink d3/d6/l1b 0 2024-03-24T00:48:47.364 INFO:tasks.workunit.client.1.smithi088.stdout:3/327: chown d3/d6/d27/d56 1475612623 1 2024-03-24T00:48:47.368 INFO:tasks.workunit.client.0.smithi038.stdout:5/303: rename d6/fa to d6/d19/d20/d42/d43/d4c/f61 0 2024-03-24T00:48:47.368 INFO:tasks.workunit.client.0.smithi038.stdout:5/304: chown d6/d23 109361757 1 2024-03-24T00:48:47.370 INFO:tasks.workunit.client.0.smithi038.stdout:7/309: creat d2/d9/da/f73 x:0 0 0 2024-03-24T00:48:47.370 INFO:tasks.workunit.client.0.smithi038.stdout:7/310: chown l1 2007024267 1 2024-03-24T00:48:47.370 INFO:tasks.workunit.client.0.smithi038.stdout:7/311: creat d2/d2c/f74 x:0 0 0 2024-03-24T00:48:47.380 INFO:tasks.workunit.client.1.smithi088.stdout:5/332: creat d1/d19/d1e/f6d x:0 0 0 2024-03-24T00:48:47.380 INFO:tasks.workunit.client.1.smithi088.stdout:0/347: sync 2024-03-24T00:48:47.381 INFO:tasks.workunit.client.1.smithi088.stdout:2/337: symlink db/d21/l69 0 2024-03-24T00:48:47.381 INFO:tasks.workunit.client.1.smithi088.stdout:2/338: stat db/dd/l1e 0 2024-03-24T00:48:47.382 INFO:tasks.workunit.client.0.smithi038.stdout:5/305: creat d6/d19/d20/d42/d43/d4c/f62 x:0 0 0 2024-03-24T00:48:47.382 INFO:tasks.workunit.client.0.smithi038.stdout:6/358: sync 2024-03-24T00:48:47.382 INFO:tasks.workunit.client.0.smithi038.stdout:3/350: sync 2024-03-24T00:48:47.382 INFO:tasks.workunit.client.0.smithi038.stdout:6/359: fsync d1/da/d1b/d1d/f43 0 2024-03-24T00:48:47.383 INFO:tasks.workunit.client.0.smithi038.stdout:9/365: sync 2024-03-24T00:48:47.383 INFO:tasks.workunit.client.1.smithi088.stdout:1/380: write d6/f10 [1078364,16531] 0 2024-03-24T00:48:47.383 INFO:tasks.workunit.client.1.smithi088.stdout:1/381: write d6/d27/d1e/d3f/f57 [96276,49611] 0 2024-03-24T00:48:47.384 INFO:tasks.workunit.client.1.smithi088.stdout:7/294: dwrite d3/d31/f47 [0,4194304] 0 2024-03-24T00:48:47.384 INFO:tasks.workunit.client.1.smithi088.stdout:7/295: dread - d3/d32/f48 zero size 2024-03-24T00:48:47.384 INFO:tasks.workunit.client.1.smithi088.stdout:0/348: creat d9/dc/de/f6c x:0 0 0 2024-03-24T00:48:47.384 INFO:tasks.workunit.client.1.smithi088.stdout:0/349: fdatasync d9/d26/f3c 0 2024-03-24T00:48:47.384 INFO:tasks.workunit.client.1.smithi088.stdout:2/339: mknod db/d54/c6a 0 2024-03-24T00:48:47.386 INFO:tasks.workunit.client.0.smithi038.stdout:5/306: getdents d6/d19/d20/d42/d43 0 2024-03-24T00:48:47.388 INFO:tasks.workunit.client.0.smithi038.stdout:7/312: dread d2/d9/f59 [0,4194304] 0 2024-03-24T00:48:47.389 INFO:tasks.workunit.client.0.smithi038.stdout:6/360: unlink d1/da/d1b/d1d/f2d 0 2024-03-24T00:48:47.391 INFO:tasks.workunit.client.1.smithi088.stdout:9/335: dread d1/d10/f1e [0,4194304] 0 2024-03-24T00:48:47.393 INFO:tasks.workunit.client.0.smithi038.stdout:7/313: mkdir d2/d9/df/d75 0 2024-03-24T00:48:47.393 INFO:tasks.workunit.client.0.smithi038.stdout:7/314: read d2/f4 [1014871,15255] 0 2024-03-24T00:48:47.397 INFO:tasks.workunit.client.1.smithi088.stdout:6/377: dread db/d1d/d23/d3c/d43/f39 [0,4194304] 0 2024-03-24T00:48:47.397 INFO:tasks.workunit.client.1.smithi088.stdout:6/378: chown db/dd/l14 708827 1 2024-03-24T00:48:47.397 INFO:tasks.workunit.client.1.smithi088.stdout:6/379: dread - db/d40/d60/f6c zero size 2024-03-24T00:48:47.398 INFO:tasks.workunit.client.0.smithi038.stdout:6/361: mkdir d1/da/d61/d69 0 2024-03-24T00:48:47.398 INFO:tasks.workunit.client.0.smithi038.stdout:6/362: read d1/f1c [468619,130993] 0 2024-03-24T00:48:47.401 INFO:tasks.workunit.client.0.smithi038.stdout:7/315: mknod d2/d9/df/d75/c76 0 2024-03-24T00:48:47.401 INFO:tasks.workunit.client.0.smithi038.stdout:7/316: creat d2/d9/d16/f77 x:0 0 0 2024-03-24T00:48:47.401 INFO:tasks.workunit.client.0.smithi038.stdout:7/317: write d2/f63 [120691,47036] 0 2024-03-24T00:48:47.403 INFO:tasks.workunit.client.1.smithi088.stdout:0/350: rmdir d9/dc/d4f 39 2024-03-24T00:48:47.403 INFO:tasks.workunit.client.1.smithi088.stdout:0/351: dread - d9/dc/d4f/d62/f6a zero size 2024-03-24T00:48:47.403 INFO:tasks.workunit.client.1.smithi088.stdout:0/352: creat d9/dc/d4f/d62/f6d x:0 0 0 2024-03-24T00:48:47.404 INFO:tasks.workunit.client.0.smithi038.stdout:6/363: rename d1/da/f33 to d1/da/d1b/f6a 0 2024-03-24T00:48:47.407 INFO:tasks.workunit.client.1.smithi088.stdout:2/340: unlink db/d21/f45 0 2024-03-24T00:48:47.407 INFO:tasks.workunit.client.1.smithi088.stdout:7/296: write d3/f35 [7755890,104106] 0 2024-03-24T00:48:47.408 INFO:tasks.workunit.client.1.smithi088.stdout:7/297: write d3/f27 [387438,89839] 0 2024-03-24T00:48:47.408 INFO:tasks.workunit.client.1.smithi088.stdout:9/336: symlink d1/d22/d27/d39/l5c 0 2024-03-24T00:48:47.410 INFO:tasks.workunit.client.0.smithi038.stdout:7/318: truncate d2/d9/f59 632489 0 2024-03-24T00:48:47.412 INFO:tasks.workunit.client.0.smithi038.stdout:3/351: dread d3/d9/f58 [0,4194304] 0 2024-03-24T00:48:47.413 INFO:tasks.workunit.client.0.smithi038.stdout:6/364: link d1/f21 d1/f6b 0 2024-03-24T00:48:47.413 INFO:tasks.workunit.client.0.smithi038.stdout:6/365: stat d1/d37 0 2024-03-24T00:48:47.414 INFO:tasks.workunit.client.1.smithi088.stdout:2/341: stat db/d21/l29 0 2024-03-24T00:48:47.414 INFO:tasks.workunit.client.1.smithi088.stdout:2/342: truncate db/d21/d4a/d4e/f64 436315 0 2024-03-24T00:48:47.414 INFO:tasks.workunit.client.1.smithi088.stdout:2/343: write db/d21/f5f [1590438,35597] 0 2024-03-24T00:48:47.415 INFO:tasks.workunit.client.1.smithi088.stdout:5/333: write d1/d10/d1b/d2f/f4a [1641333,114407] 0 2024-03-24T00:48:47.416 INFO:tasks.workunit.client.0.smithi038.stdout:7/319: rename d2/d9/da/dc/fd to d2/d9/df/d75/f78 0 2024-03-24T00:48:47.417 INFO:tasks.workunit.client.1.smithi088.stdout:0/353: dread d9/d26/f38 [0,4194304] 0 2024-03-24T00:48:47.417 INFO:tasks.workunit.client.0.smithi038.stdout:1/302: write da/dd/d22/f25 [1322087,18244] 0 2024-03-24T00:48:47.421 INFO:tasks.workunit.client.1.smithi088.stdout:2/344: write db/dd/f2f [1468449,106400] 0 2024-03-24T00:48:47.421 INFO:tasks.workunit.client.1.smithi088.stdout:2/345: write db/d21/d4a/d4e/f64 [1370312,127097] 0 2024-03-24T00:48:47.422 INFO:tasks.workunit.client.0.smithi038.stdout:3/352: creat d3/d30/f7a x:0 0 0 2024-03-24T00:48:47.422 INFO:tasks.workunit.client.0.smithi038.stdout:3/353: chown d3/f8 13686 1 2024-03-24T00:48:47.422 INFO:tasks.workunit.client.0.smithi038.stdout:3/354: truncate d3/d5a/f79 705223 0 2024-03-24T00:48:47.422 INFO:tasks.workunit.client.0.smithi038.stdout:3/355: dread - d3/d9/dc/df/f3a zero size 2024-03-24T00:48:47.422 INFO:tasks.workunit.client.0.smithi038.stdout:3/356: dread - d3/d9/dc/d51/f52 zero size 2024-03-24T00:48:47.422 INFO:tasks.workunit.client.0.smithi038.stdout:3/357: read - d3/d9/dc/df/f6b zero size 2024-03-24T00:48:47.426 INFO:tasks.workunit.client.0.smithi038.stdout:6/366: truncate d1/da/d1b/d1d/f24 168803 0 2024-03-24T00:48:47.428 INFO:tasks.workunit.client.1.smithi088.stdout:7/298: unlink d3/l39 0 2024-03-24T00:48:47.428 INFO:tasks.workunit.client.1.smithi088.stdout:7/299: fdatasync d3/f29 0 2024-03-24T00:48:47.428 INFO:tasks.workunit.client.1.smithi088.stdout:7/300: dread - d3/d32/f48 zero size 2024-03-24T00:48:47.428 INFO:tasks.workunit.client.1.smithi088.stdout:7/301: dread - d3/d32/f44 zero size 2024-03-24T00:48:47.428 INFO:tasks.workunit.client.1.smithi088.stdout:7/302: write d3/d31/f3c [279339,68994] 0 2024-03-24T00:48:47.428 INFO:tasks.workunit.client.1.smithi088.stdout:3/328: dwrite f2 [4194304,4194304] 0 2024-03-24T00:48:47.429 INFO:tasks.workunit.client.1.smithi088.stdout:3/329: fsync f1 0 2024-03-24T00:48:47.429 INFO:tasks.workunit.client.1.smithi088.stdout:9/337: creat d1/d10/d11/f5d x:0 0 0 2024-03-24T00:48:47.429 INFO:tasks.workunit.client.0.smithi038.stdout:7/320: symlink d2/d9/df/d5d/l79 0 2024-03-24T00:48:47.429 INFO:tasks.workunit.client.0.smithi038.stdout:7/321: write d2/d9/da/f36 [1045077,102042] 0 2024-03-24T00:48:47.430 INFO:tasks.workunit.client.1.smithi088.stdout:5/334: rmdir d1/d10/d1b/d2e/d46 39 2024-03-24T00:48:47.431 INFO:tasks.workunit.client.1.smithi088.stdout:4/359: sync 2024-03-24T00:48:47.435 INFO:tasks.workunit.client.1.smithi088.stdout:0/354: truncate d9/dc/de/f1d 37837 0 2024-03-24T00:48:47.436 INFO:tasks.workunit.client.1.smithi088.stdout:1/382: read d6/d27/d1e/d58/f32 [2662517,108381] 0 2024-03-24T00:48:47.436 INFO:tasks.workunit.client.1.smithi088.stdout:1/383: readlink d6/d23/l49 0 2024-03-24T00:48:47.436 INFO:tasks.workunit.client.1.smithi088.stdout:1/384: stat d6/d27/d1e/d3f/f41 0 2024-03-24T00:48:47.438 INFO:tasks.workunit.client.0.smithi038.stdout:4/284: dwrite da/d12/f4e [4194304,4194304] 0 2024-03-24T00:48:47.438 INFO:tasks.workunit.client.0.smithi038.stdout:4/285: truncate da/d12/f3f 4379899 0 2024-03-24T00:48:47.440 INFO:tasks.workunit.client.0.smithi038.stdout:3/358: creat d3/d30/d70/f7b x:0 0 0 2024-03-24T00:48:47.442 INFO:tasks.workunit.client.0.smithi038.stdout:0/283: dwrite d0/d12/d13/d28/f48 [0,4194304] 0 2024-03-24T00:48:47.442 INFO:tasks.workunit.client.0.smithi038.stdout:0/284: dread - d0/f43 zero size 2024-03-24T00:48:47.450 INFO:tasks.workunit.client.1.smithi088.stdout:1/385: mknod d6/c71 0 2024-03-24T00:48:47.452 INFO:tasks.workunit.client.0.smithi038.stdout:6/367: symlink d1/da/d1b/d25/d49/l6c 0 2024-03-24T00:48:47.453 INFO:tasks.workunit.client.0.smithi038.stdout:7/322: link d2/d9/df/d75/c76 d2/d9/da/d23/c7a 0 2024-03-24T00:48:47.453 INFO:tasks.workunit.client.0.smithi038.stdout:7/323: chown d2/d9/c11 2167449 1 2024-03-24T00:48:47.454 INFO:tasks.workunit.client.1.smithi088.stdout:1/386: write d6/d27/d1e/d5b/d5e/f66 [635882,96209] 0 2024-03-24T00:48:47.454 INFO:tasks.workunit.client.1.smithi088.stdout:9/338: link d1/d15/c1a d1/d10/d11/d28/c5e 0 2024-03-24T00:48:47.454 INFO:tasks.workunit.client.1.smithi088.stdout:9/339: creat d1/d10/d11/f5f x:0 0 0 2024-03-24T00:48:47.455 INFO:tasks.workunit.client.0.smithi038.stdout:3/359: mkdir d3/d30/d5d/d7c 0 2024-03-24T00:48:47.458 INFO:tasks.workunit.client.1.smithi088.stdout:5/335: read d1/dd/f13 [1749564,71929] 0 2024-03-24T00:48:47.459 INFO:tasks.workunit.client.1.smithi088.stdout:8/353: dwrite d9/f3c [0,4194304] 0 2024-03-24T00:48:47.460 INFO:tasks.workunit.client.1.smithi088.stdout:8/354: chown d9/dc/d11/d24/d29/d39 0 1 2024-03-24T00:48:47.460 INFO:tasks.workunit.client.0.smithi038.stdout:6/368: link d1/da/l54 d1/da/d61/d69/l6d 0 2024-03-24T00:48:47.460 INFO:tasks.workunit.client.0.smithi038.stdout:7/324: mknod d2/d9/df/d33/d37/c7b 0 2024-03-24T00:48:47.460 INFO:tasks.workunit.client.0.smithi038.stdout:7/325: chown d2/d9/d16/l60 791799313 1 2024-03-24T00:48:47.462 INFO:tasks.workunit.client.0.smithi038.stdout:3/360: rename d3/d9/dc/df/f5f to d3/d9/dc/d51/f7d 0 2024-03-24T00:48:47.462 INFO:tasks.workunit.client.1.smithi088.stdout:2/346: dread db/f38 [0,4194304] 0 2024-03-24T00:48:47.463 INFO:tasks.workunit.client.1.smithi088.stdout:2/347: creat db/d21/d4a/d4e/f6b x:0 0 0 2024-03-24T00:48:47.465 INFO:tasks.workunit.client.0.smithi038.stdout:7/326: creat d2/d9/f7c x:0 0 0 2024-03-24T00:48:47.467 INFO:tasks.workunit.client.1.smithi088.stdout:5/336: truncate d1/fb 1016394 0 2024-03-24T00:48:47.469 INFO:tasks.workunit.client.0.smithi038.stdout:7/327: mkdir d2/d9/df/d75/d7d 0 2024-03-24T00:48:47.471 INFO:tasks.workunit.client.1.smithi088.stdout:8/355: dread d9/dc/d11/d24/d29/d36/f4d [0,4194304] 0 2024-03-24T00:48:47.473 INFO:tasks.workunit.client.0.smithi038.stdout:4/286: write da/d38/f3c [3962222,49155] 0 2024-03-24T00:48:47.473 INFO:tasks.workunit.client.0.smithi038.stdout:4/287: truncate da/d12/f57 842775 0 2024-03-24T00:48:47.473 INFO:tasks.workunit.client.0.smithi038.stdout:4/288: dread - da/d12/d37/f54 zero size 2024-03-24T00:48:47.478 INFO:tasks.workunit.client.1.smithi088.stdout:4/360: dread d5/d16/d1a/f3e [0,4194304] 0 2024-03-24T00:48:47.480 INFO:tasks.workunit.client.0.smithi038.stdout:7/328: symlink d2/d9/da/d23/d2b/d5b/d72/l7e 0 2024-03-24T00:48:47.481 INFO:tasks.workunit.client.1.smithi088.stdout:6/380: dwrite f8 [4194304,4194304] 0 2024-03-24T00:48:47.483 INFO:tasks.workunit.client.1.smithi088.stdout:5/337: unlink d1/d29/d3f/c59 0 2024-03-24T00:48:47.486 INFO:tasks.workunit.client.1.smithi088.stdout:5/338: write d1/d19/d1e/f47 [372952,9365] 0 2024-03-24T00:48:47.486 INFO:tasks.workunit.client.1.smithi088.stdout:5/339: getdents d1/d43/d4e 0 2024-03-24T00:48:47.486 INFO:tasks.workunit.client.1.smithi088.stdout:8/356: truncate d9/dc/d11/f2f 1987019 0 2024-03-24T00:48:47.488 INFO:tasks.workunit.client.1.smithi088.stdout:4/361: creat d5/f7c x:0 0 0 2024-03-24T00:48:47.489 INFO:tasks.workunit.client.0.smithi038.stdout:4/289: getdents da/d12 0 2024-03-24T00:48:47.495 INFO:tasks.workunit.client.1.smithi088.stdout:6/381: unlink db/d40/d60/f6c 0 2024-03-24T00:48:47.495 INFO:tasks.workunit.client.1.smithi088.stdout:6/382: getdents db/d1d/d64/d71 0 2024-03-24T00:48:47.502 INFO:tasks.workunit.client.0.smithi038.stdout:6/369: dread d1/da/f34 [0,4194304] 0 2024-03-24T00:48:47.504 INFO:tasks.workunit.client.0.smithi038.stdout:4/290: dread da/d12/d37/dd/f34 [0,4194304] 0 2024-03-24T00:48:47.506 INFO:tasks.workunit.client.0.smithi038.stdout:1/303: dwrite da/dd/d2c/d41/f65 [0,4194304] 0 2024-03-24T00:48:47.511 INFO:tasks.workunit.client.1.smithi088.stdout:7/303: dwrite d3/fd [0,4194304] 0 2024-03-24T00:48:47.511 INFO:tasks.workunit.client.1.smithi088.stdout:3/330: dwrite d3/de/df/d49/f54 [4194304,4194304] 0 2024-03-24T00:48:47.512 INFO:tasks.workunit.client.0.smithi038.stdout:0/285: dwrite d0/d12/d42/f51 [0,4194304] 0 2024-03-24T00:48:47.512 INFO:tasks.workunit.client.0.smithi038.stdout:8/336: dwrite d0/d1d/d27/d4d/f75 [0,4194304] 0 2024-03-24T00:48:47.520 INFO:tasks.workunit.client.1.smithi088.stdout:0/355: dwrite d9/dc/de/f11 [0,4194304] 0 2024-03-24T00:48:47.522 INFO:tasks.workunit.client.0.smithi038.stdout:3/361: dwrite d3/d9/dc/df/f12 [0,4194304] 0 2024-03-24T00:48:47.540 INFO:tasks.workunit.client.1.smithi088.stdout:1/387: dwrite f0 [0,4194304] 0 2024-03-24T00:48:47.540 INFO:tasks.workunit.client.1.smithi088.stdout:1/388: write d6/d27/d67/f6a [135388,97770] 0 2024-03-24T00:48:47.540 INFO:tasks.workunit.client.1.smithi088.stdout:1/389: chown d6/f6d 74938 1 2024-03-24T00:48:47.542 INFO:tasks.workunit.client.1.smithi088.stdout:9/340: dread d1/f8 [4194304,4194304] 0 2024-03-24T00:48:47.556 INFO:tasks.workunit.client.1.smithi088.stdout:0/356: write d9/d26/f61 [4168386,14924] 0 2024-03-24T00:48:47.556 INFO:tasks.workunit.client.1.smithi088.stdout:3/331: dwrite d3/de/df/f25 [8388608,4194304] 0 2024-03-24T00:48:47.556 INFO:tasks.workunit.client.1.smithi088.stdout:3/332: readlink d3/de/l35 0 2024-03-24T00:48:47.570 INFO:tasks.workunit.client.1.smithi088.stdout:5/340: dwrite d1/d19/f45 [0,4194304] 0 2024-03-24T00:48:47.570 INFO:tasks.workunit.client.1.smithi088.stdout:5/341: readlink d1/d10/l40 0 2024-03-24T00:48:47.571 INFO:tasks.workunit.client.0.smithi038.stdout:3/362: dwrite d3/d9/dc/d51/f6f [0,4194304] 0 2024-03-24T00:48:47.571 INFO:tasks.workunit.client.0.smithi038.stdout:3/363: chown d3/d9/dc/d51/f7d 33214138 1 2024-03-24T00:48:47.571 INFO:tasks.workunit.client.0.smithi038.stdout:3/364: stat d3/d9/dc/df/d14/f53 0 2024-03-24T00:48:47.571 INFO:tasks.workunit.client.0.smithi038.stdout:3/365: readlink d3/l4f 0 2024-03-24T00:48:47.571 INFO:tasks.workunit.client.1.smithi088.stdout:0/357: read d9/dc/de/d19/f1a [598979,100503] 0 2024-03-24T00:48:47.603 INFO:tasks.workunit.client.0.smithi038.stdout:7/329: getdents d2 0 2024-03-24T00:48:47.606 INFO:tasks.workunit.client.1.smithi088.stdout:2/348: unlink db/d21/d4a/d4e/f6b 0 2024-03-24T00:48:47.607 INFO:tasks.workunit.client.0.smithi038.stdout:6/370: creat d1/da/d61/d69/f6e x:0 0 0 2024-03-24T00:48:47.611 INFO:tasks.workunit.client.1.smithi088.stdout:8/357: mknod d9/dc/d11/d24/d29/d5b/c7a 0 2024-03-24T00:48:47.611 INFO:tasks.workunit.client.0.smithi038.stdout:4/291: mkdir da/d27/d58 0 2024-03-24T00:48:47.613 INFO:tasks.workunit.client.1.smithi088.stdout:4/362: truncate d5/d17/f1c 3389920 0 2024-03-24T00:48:47.616 INFO:tasks.workunit.client.1.smithi088.stdout:6/383: mkdir db/d1d/d64/d72 0 2024-03-24T00:48:47.616 INFO:tasks.workunit.client.1.smithi088.stdout:6/384: write db/dd/f1e [6007179,100604] 0 2024-03-24T00:48:47.620 INFO:tasks.workunit.client.0.smithi038.stdout:1/304: rename da/c68 to da/dd/d2c/d40/c69 0 2024-03-24T00:48:47.622 INFO:tasks.workunit.client.0.smithi038.stdout:7/330: read d2/d9/df/d33/d37/f4c [1841839,31760] 0 2024-03-24T00:48:47.631 INFO:tasks.workunit.client.1.smithi088.stdout:7/304: unlink d3/d32/d34/f36 0 2024-03-24T00:48:47.631 INFO:tasks.workunit.client.1.smithi088.stdout:7/305: stat d3/f35 0 2024-03-24T00:48:47.631 INFO:tasks.workunit.client.1.smithi088.stdout:7/306: fdatasync d3/f2d 0 2024-03-24T00:48:47.631 INFO:tasks.workunit.client.1.smithi088.stdout:7/307: chown d3/l13 65565026 1 2024-03-24T00:48:47.631 INFO:tasks.workunit.client.1.smithi088.stdout:1/390: symlink d6/d23/l72 0 2024-03-24T00:48:47.631 INFO:tasks.workunit.client.1.smithi088.stdout:1/391: stat d6/d27/d1e/d3f/d53 0 2024-03-24T00:48:47.632 INFO:tasks.workunit.client.1.smithi088.stdout:9/341: symlink d1/d15/l60 0 2024-03-24T00:48:47.634 INFO:tasks.workunit.client.0.smithi038.stdout:3/366: dwrite d3/d9/dc/df/d14/d19/f47 [0,4194304] 0 2024-03-24T00:48:47.634 INFO:tasks.workunit.client.0.smithi038.stdout:3/367: creat d3/d30/f7e x:0 0 0 2024-03-24T00:48:47.638 INFO:tasks.workunit.client.1.smithi088.stdout:3/333: mkdir d3/de/d6f 0 2024-03-24T00:48:47.638 INFO:tasks.workunit.client.1.smithi088.stdout:3/334: chown d3/d6/d58/f63 1573699 1 2024-03-24T00:48:47.641 INFO:tasks.workunit.client.1.smithi088.stdout:7/308: dread d3/f17 [0,4194304] 0 2024-03-24T00:48:47.641 INFO:tasks.workunit.client.1.smithi088.stdout:7/309: fdatasync d3/f51 0 2024-03-24T00:48:47.650 INFO:tasks.workunit.client.1.smithi088.stdout:5/342: truncate d1/f31 3150315 0 2024-03-24T00:48:47.650 INFO:tasks.workunit.client.1.smithi088.stdout:5/343: chown d1/d10/d1b/d2e/c3d 149 1 2024-03-24T00:48:47.652 INFO:tasks.workunit.client.0.smithi038.stdout:6/371: symlink d1/da/d1b/d25/d49/l6f 0 2024-03-24T00:48:47.654 INFO:tasks.workunit.client.1.smithi088.stdout:0/358: creat d9/dc/de/d19/d44/f6e x:0 0 0 2024-03-24T00:48:47.659 INFO:tasks.workunit.client.0.smithi038.stdout:0/286: rename d0/d12/d15/d1a/f1e to d0/d14/f5c 0 2024-03-24T00:48:47.659 INFO:tasks.workunit.client.1.smithi088.stdout:7/310: dread d3/fc [4194304,4194304] 0 2024-03-24T00:48:47.662 INFO:tasks.workunit.client.1.smithi088.stdout:4/363: mkdir d5/d13/d18/d6e/d7d 0 2024-03-24T00:48:47.663 INFO:tasks.workunit.client.1.smithi088.stdout:8/358: dwrite d9/f37 [0,4194304] 0 2024-03-24T00:48:47.670 INFO:tasks.workunit.client.0.smithi038.stdout:3/368: creat d3/d9/d77/f7f x:0 0 0 2024-03-24T00:48:47.678 INFO:tasks.workunit.client.1.smithi088.stdout:6/385: mknod db/d40/d55/c73 0 2024-03-24T00:48:47.678 INFO:tasks.workunit.client.1.smithi088.stdout:6/386: read - db/d1d/f68 zero size 2024-03-24T00:48:47.679 INFO:tasks.workunit.client.0.smithi038.stdout:6/372: getdents d1/da/d1b 0 2024-03-24T00:48:47.679 INFO:tasks.workunit.client.0.smithi038.stdout:8/337: rename d0/ld to d0/d45/d37/d5f/l78 0 2024-03-24T00:48:47.679 INFO:tasks.workunit.client.1.smithi088.stdout:1/392: truncate d6/d27/d1e/d3f/f4b 803685 0 2024-03-24T00:48:47.679 INFO:tasks.workunit.client.1.smithi088.stdout:9/342: mknod d1/d15/c61 0 2024-03-24T00:48:47.679 INFO:tasks.workunit.client.1.smithi088.stdout:9/343: stat d1/d10/d29/f45 0 2024-03-24T00:48:47.680 INFO:tasks.workunit.client.1.smithi088.stdout:9/344: chown d1/d15 155 1 2024-03-24T00:48:47.680 INFO:tasks.workunit.client.1.smithi088.stdout:1/393: write d6/d43/f4f [978740,125259] 0 2024-03-24T00:48:47.682 INFO:tasks.workunit.client.1.smithi088.stdout:3/335: mkdir d3/de/df/d49/d70 0 2024-03-24T00:48:47.682 INFO:tasks.workunit.client.1.smithi088.stdout:3/336: fsync d3/de/f17 0 2024-03-24T00:48:47.682 INFO:tasks.workunit.client.0.smithi038.stdout:0/287: truncate d0/f2 2115269 0 2024-03-24T00:48:47.685 INFO:tasks.workunit.client.1.smithi088.stdout:5/344: rename d1/d10/f56 to d1/d43/d4e/f6e 0 2024-03-24T00:48:47.690 INFO:tasks.workunit.client.0.smithi038.stdout:8/338: mkdir d0/d1d/d20/d79 0 2024-03-24T00:48:47.693 INFO:tasks.workunit.client.1.smithi088.stdout:0/359: mkdir d9/dc/de/d19/d44/d46/d65/d6f 0 2024-03-24T00:48:47.696 INFO:tasks.workunit.client.1.smithi088.stdout:7/311: getdents d3/d32 0 2024-03-24T00:48:47.696 INFO:tasks.workunit.client.0.smithi038.stdout:1/305: rename da/d3c/c37 to da/dd/d2c/c6a 0 2024-03-24T00:48:47.700 INFO:tasks.workunit.client.0.smithi038.stdout:3/369: mkdir d3/d80 0 2024-03-24T00:48:47.703 INFO:tasks.workunit.client.0.smithi038.stdout:6/373: read d1/da/d1b/d1d/f43 [1822060,25848] 0 2024-03-24T00:48:47.705 INFO:tasks.workunit.client.0.smithi038.stdout:4/292: dwrite da/f3b [0,4194304] 0 2024-03-24T00:48:47.707 INFO:tasks.workunit.client.1.smithi088.stdout:2/349: dwrite db/dc/f2c [0,4194304] 0 2024-03-24T00:48:47.708 INFO:tasks.workunit.client.0.smithi038.stdout:7/331: rename d2/d2c/f4b to d2/d9/df/d5d/f7f 0 2024-03-24T00:48:47.717 INFO:tasks.workunit.client.1.smithi088.stdout:9/345: mkdir d1/d22/d27/d39/d62 0 2024-03-24T00:48:47.717 INFO:tasks.workunit.client.1.smithi088.stdout:9/346: fsync d1/d10/d11/f5d 0 2024-03-24T00:48:47.720 INFO:tasks.workunit.client.1.smithi088.stdout:1/394: rename d6/d23/f2d to d6/d23/f73 0 2024-03-24T00:48:47.727 INFO:tasks.workunit.client.1.smithi088.stdout:3/337: dread d3/de/df/f25 [8388608,4194304] 0 2024-03-24T00:48:47.731 INFO:tasks.workunit.client.1.smithi088.stdout:2/350: read db/f3c [2877543,80316] 0 2024-03-24T00:48:47.735 INFO:tasks.workunit.client.1.smithi088.stdout:6/387: dwrite db/d40/d56/f51 [0,4194304] 0 2024-03-24T00:48:47.735 INFO:tasks.workunit.client.1.smithi088.stdout:6/388: chown db/dd/d61/c69 53435314 1 2024-03-24T00:48:47.735 INFO:tasks.workunit.client.1.smithi088.stdout:9/347: dread d1/d10/f34 [0,4194304] 0 2024-03-24T00:48:47.735 INFO:tasks.workunit.client.1.smithi088.stdout:9/348: chown d1/d22/d27/d39/l5c 6 1 2024-03-24T00:48:47.735 INFO:tasks.workunit.client.1.smithi088.stdout:9/349: stat d1/d22/d27 0 2024-03-24T00:48:47.735 INFO:tasks.workunit.client.1.smithi088.stdout:5/345: mkdir d1/dd/d6f 0 2024-03-24T00:48:47.742 INFO:tasks.workunit.client.1.smithi088.stdout:0/360: getdents d9/d26 0 2024-03-24T00:48:47.742 INFO:tasks.workunit.client.1.smithi088.stdout:0/361: fsync d9/dc/de/f6c 0 2024-03-24T00:48:47.742 INFO:tasks.workunit.client.1.smithi088.stdout:0/362: truncate d9/dc/d4f/d62/f6a 954656 0 2024-03-24T00:48:47.742 INFO:tasks.workunit.client.1.smithi088.stdout:0/363: chown d9/dc/c54 2944 1 2024-03-24T00:48:47.743 INFO:tasks.workunit.client.0.smithi038.stdout:4/293: link da/d12/d24/c2f da/d12/d24/c59 0 2024-03-24T00:48:47.747 INFO:tasks.workunit.client.0.smithi038.stdout:8/339: rename d0/d15/f17 to d0/d1d/d27/d4d/f7a 0 2024-03-24T00:48:47.747 INFO:tasks.workunit.client.0.smithi038.stdout:8/340: write d0/d1d/d20/f23 [3311058,105556] 0 2024-03-24T00:48:47.747 INFO:tasks.workunit.client.0.smithi038.stdout:8/341: creat d0/d1d/d27/d4d/f7b x:0 0 0 2024-03-24T00:48:47.747 INFO:tasks.workunit.client.0.smithi038.stdout:8/342: write d0/f16 [662013,65509] 0 2024-03-24T00:48:47.750 INFO:tasks.workunit.client.0.smithi038.stdout:1/306: dread da/dd/f1c [0,4194304] 0 2024-03-24T00:48:47.750 INFO:tasks.workunit.client.0.smithi038.stdout:1/307: chown da/f21 16 1 2024-03-24T00:48:47.750 INFO:tasks.workunit.client.0.smithi038.stdout:1/308: creat da/dd/d4a/f6b x:0 0 0 2024-03-24T00:48:47.753 INFO:tasks.workunit.client.0.smithi038.stdout:3/370: rename d3/d9/f1c to d3/d9/dc/f81 0 2024-03-24T00:48:47.756 INFO:tasks.workunit.client.1.smithi088.stdout:7/312: symlink d3/d31/l62 0 2024-03-24T00:48:47.756 INFO:tasks.workunit.client.1.smithi088.stdout:7/313: fdatasync d3/f27 0 2024-03-24T00:48:47.756 INFO:tasks.workunit.client.1.smithi088.stdout:8/359: rmdir d9/dc/d11/d24/d29/d73 39 2024-03-24T00:48:47.757 INFO:tasks.workunit.client.0.smithi038.stdout:7/332: dwrite d2/d9/f59 [0,4194304] 0 2024-03-24T00:48:47.758 INFO:tasks.workunit.client.1.smithi088.stdout:0/364: dread d9/dc/de/d19/f3b [0,4194304] 0 2024-03-24T00:48:47.758 INFO:tasks.workunit.client.0.smithi038.stdout:6/374: dwrite d1/da/d1b/d2f/f62 [0,4194304] 0 2024-03-24T00:48:47.779 INFO:tasks.workunit.client.1.smithi088.stdout:2/351: dwrite db/f46 [0,4194304] 0 2024-03-24T00:48:47.781 INFO:tasks.workunit.client.0.smithi038.stdout:3/371: mkdir d3/d9/dc/df/d14/d19/d82 0 2024-03-24T00:48:47.781 INFO:tasks.workunit.client.0.smithi038.stdout:3/372: truncate d3/d9/dc/df/d14/d19/f42 252942 0 2024-03-24T00:48:47.781 INFO:tasks.workunit.client.0.smithi038.stdout:3/373: write d3/d9/dc/df/d14/f53 [315044,792] 0 2024-03-24T00:48:47.797 INFO:tasks.workunit.client.0.smithi038.stdout:0/288: dwrite d0/d12/d15/f22 [0,4194304] 0 2024-03-24T00:48:47.797 INFO:tasks.workunit.client.0.smithi038.stdout:0/289: fsync d0/d12/d15/d1a/f2b 0 2024-03-24T00:48:47.797 INFO:tasks.workunit.client.0.smithi038.stdout:0/290: readlink d0/l10 0 2024-03-24T00:48:47.799 INFO:tasks.workunit.client.1.smithi088.stdout:1/395: rmdir d6/d27/d1e/d3f/d70 0 2024-03-24T00:48:47.799 INFO:tasks.workunit.client.1.smithi088.stdout:1/396: chown d6/f8 4 1 2024-03-24T00:48:47.800 INFO:tasks.workunit.client.1.smithi088.stdout:1/397: chown d6/d27 0 1 2024-03-24T00:48:47.801 INFO:tasks.workunit.client.0.smithi038.stdout:1/309: dread da/dd/d22/f4f [0,4194304] 0 2024-03-24T00:48:47.802 INFO:tasks.workunit.client.0.smithi038.stdout:7/333: rename d2/d9/df/d33/d37/f4c to d2/d2c/f80 0 2024-03-24T00:48:47.803 INFO:tasks.workunit.client.0.smithi038.stdout:1/310: read da/f35 [444498,102085] 0 2024-03-24T00:48:47.805 INFO:tasks.workunit.client.1.smithi088.stdout:3/338: unlink d3/d1a/d1e/d2f/f5b 0 2024-03-24T00:48:47.805 INFO:tasks.workunit.client.0.smithi038.stdout:6/375: mknod d1/da/d61/d69/c70 0 2024-03-24T00:48:47.808 INFO:tasks.workunit.client.0.smithi038.stdout:8/343: truncate d0/d1d/d20/f23 290958 0 2024-03-24T00:48:47.811 INFO:tasks.workunit.client.1.smithi088.stdout:9/350: rename d1/d15/c61 to d1/d22/d27/d39/d62/c63 0 2024-03-24T00:48:47.815 INFO:tasks.workunit.client.1.smithi088.stdout:5/346: creat d1/d10/d1b/f70 x:0 0 0 2024-03-24T00:48:47.820 INFO:tasks.workunit.client.0.smithi038.stdout:1/311: dread da/dd/d22/f4c [0,4194304] 0 2024-03-24T00:48:47.821 INFO:tasks.workunit.client.0.smithi038.stdout:1/312: creat da/dd/d2c/d40/f6c x:0 0 0 2024-03-24T00:48:47.823 INFO:tasks.workunit.client.1.smithi088.stdout:0/365: link d9/dc/de/f11 d9/dc/de/d19/d44/d46/d59/f70 0 2024-03-24T00:48:47.823 INFO:tasks.workunit.client.1.smithi088.stdout:0/366: stat d9/dc/de/d19/d44/f50 0 2024-03-24T00:48:47.825 INFO:tasks.workunit.client.1.smithi088.stdout:2/352: symlink db/l6c 0 2024-03-24T00:48:47.825 INFO:tasks.workunit.client.1.smithi088.stdout:7/314: dwrite d3/d32/d34/f37 [0,4194304] 0 2024-03-24T00:48:47.825 INFO:tasks.workunit.client.1.smithi088.stdout:2/353: stat db/c62 0 2024-03-24T00:48:47.827 INFO:tasks.workunit.client.0.smithi038.stdout:0/291: mknod d0/d12/d13/d28/c5d 0 2024-03-24T00:48:47.827 INFO:tasks.workunit.client.0.smithi038.stdout:3/374: write d3/d9/dc/d51/f7d [444952,119020] 0 2024-03-24T00:48:47.827 INFO:tasks.workunit.client.1.smithi088.stdout:1/398: mknod d6/d6e/c74 0 2024-03-24T00:48:47.827 INFO:tasks.workunit.client.1.smithi088.stdout:1/399: write d6/d43/f4f [1088408,10050] 0 2024-03-24T00:48:47.828 INFO:tasks.workunit.client.1.smithi088.stdout:1/400: chown d6/d27/d1e/d3f/d53 1967 1 2024-03-24T00:48:47.830 INFO:tasks.workunit.client.1.smithi088.stdout:3/339: link d3/de/f29 d3/d6/d19/d2a/f71 0 2024-03-24T00:48:47.834 INFO:tasks.workunit.client.1.smithi088.stdout:6/389: dwrite db/d1d/d23/d3c/d43/f39 [4194304,4194304] 0 2024-03-24T00:48:47.834 INFO:tasks.workunit.client.1.smithi088.stdout:5/347: dread d1/d19/f45 [0,4194304] 0 2024-03-24T00:48:47.834 INFO:tasks.workunit.client.1.smithi088.stdout:5/348: creat d1/d10/d1b/d2f/d57/f71 x:0 0 0 2024-03-24T00:48:47.835 INFO:tasks.workunit.client.0.smithi038.stdout:7/334: write d2/d9/f20 [2735254,74056] 0 2024-03-24T00:48:47.835 INFO:tasks.workunit.client.0.smithi038.stdout:7/335: chown d2/d9/da/dc/d4a 57036 1 2024-03-24T00:48:47.835 INFO:tasks.workunit.client.0.smithi038.stdout:7/336: stat d2/d9/da/d23/d2b/d5b/l5e 0 2024-03-24T00:48:47.835 INFO:tasks.workunit.client.0.smithi038.stdout:7/337: write d2/d9/df/d65/f70 [109739,66362] 0 2024-03-24T00:48:47.835 INFO:tasks.workunit.client.0.smithi038.stdout:7/338: dread - d2/d9/da/f73 zero size 2024-03-24T00:48:47.835 INFO:tasks.workunit.client.0.smithi038.stdout:4/294: dwrite da/d12/f3f [0,4194304] 0 2024-03-24T00:48:47.835 INFO:tasks.workunit.client.0.smithi038.stdout:6/376: rename d1/l36 to d1/da/d1b/d1d/d3c/l71 0 2024-03-24T00:48:47.836 INFO:tasks.workunit.client.0.smithi038.stdout:6/377: readlink d1/d37/l5f 0 2024-03-24T00:48:47.838 INFO:tasks.workunit.client.1.smithi088.stdout:9/351: rename d1/d10/d2c/c3d to d1/d10/d11/c64 0 2024-03-24T00:48:47.838 INFO:tasks.workunit.client.1.smithi088.stdout:9/352: fsync d1/d22/f51 0 2024-03-24T00:48:47.838 INFO:tasks.workunit.client.1.smithi088.stdout:9/353: write d1/f7 [2141091,89927] 0 2024-03-24T00:48:47.838 INFO:tasks.workunit.client.1.smithi088.stdout:9/354: chown d1/d10/l42 0 1 2024-03-24T00:48:47.839 INFO:tasks.workunit.client.0.smithi038.stdout:8/344: truncate d0/d45/f55 2667310 0 2024-03-24T00:48:47.839 INFO:tasks.workunit.client.0.smithi038.stdout:8/345: chown d0/d1d/c5e 12862233 1 2024-03-24T00:48:47.839 INFO:tasks.workunit.client.0.smithi038.stdout:8/346: chown d0/d1d/d27/l3f 75121 1 2024-03-24T00:48:47.839 INFO:tasks.workunit.client.0.smithi038.stdout:8/347: chown d0/d45/d37/f61 0 1 2024-03-24T00:48:47.839 INFO:tasks.workunit.client.0.smithi038.stdout:8/348: dread - d0/d15/f63 zero size 2024-03-24T00:48:47.840 INFO:tasks.workunit.client.1.smithi088.stdout:8/360: getdents d9/dc/d11/d24/d29/d39 0 2024-03-24T00:48:47.842 INFO:tasks.workunit.client.0.smithi038.stdout:3/375: dread d3/d9/f3c [0,4194304] 0 2024-03-24T00:48:47.843 INFO:tasks.workunit.client.1.smithi088.stdout:0/367: mknod d9/dc/de/d19/d44/d46/d65/d2a/c71 0 2024-03-24T00:48:47.845 INFO:tasks.workunit.client.0.smithi038.stdout:1/313: mknod da/dd/d2c/d41/c6d 0 2024-03-24T00:48:47.845 INFO:tasks.workunit.client.0.smithi038.stdout:1/314: dread - da/dd/d10/d14/f4d zero size 2024-03-24T00:48:47.845 INFO:tasks.workunit.client.0.smithi038.stdout:1/315: chown da/dd/d10 7869 1 2024-03-24T00:48:47.847 INFO:tasks.workunit.client.1.smithi088.stdout:7/315: link d3/l13 d3/d32/l63 0 2024-03-24T00:48:47.848 INFO:tasks.workunit.client.0.smithi038.stdout:0/292: mkdir d0/d12/d42/d5e 0 2024-03-24T00:48:47.850 INFO:tasks.workunit.client.0.smithi038.stdout:9/366: sync 2024-03-24T00:48:47.850 INFO:tasks.workunit.client.0.smithi038.stdout:9/367: chown d12/d28/d3a 16429745 1 2024-03-24T00:48:47.850 INFO:tasks.workunit.client.0.smithi038.stdout:2/324: sync 2024-03-24T00:48:47.850 INFO:tasks.workunit.client.0.smithi038.stdout:2/325: chown d4/l17 5 1 2024-03-24T00:48:47.850 INFO:tasks.workunit.client.0.smithi038.stdout:2/326: chown d4/d18/d26/f70 1 1 2024-03-24T00:48:47.850 INFO:tasks.workunit.client.0.smithi038.stdout:2/327: chown d4/d18/d26/d37/f43 7653 1 2024-03-24T00:48:47.851 INFO:tasks.workunit.client.1.smithi088.stdout:2/354: creat db/d54/f6d x:0 0 0 2024-03-24T00:48:47.851 INFO:tasks.workunit.client.1.smithi088.stdout:2/355: chown db/d54/f65 1888320 1 2024-03-24T00:48:47.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/295: creat da/d12/d37/dd/d15/d4c/f5a x:0 0 0 2024-03-24T00:48:47.854 INFO:tasks.workunit.client.0.smithi038.stdout:5/307: sync 2024-03-24T00:48:47.854 INFO:tasks.workunit.client.1.smithi088.stdout:1/401: creat d6/d27/d1e/d5b/f75 x:0 0 0 2024-03-24T00:48:47.854 INFO:tasks.workunit.client.1.smithi088.stdout:1/402: chown d6/d27/d1e/d3f/d53 509009 1 2024-03-24T00:48:47.855 INFO:tasks.workunit.client.0.smithi038.stdout:6/378: mkdir d1/da/d1b/d25/d49/d72 0 2024-03-24T00:48:47.856 INFO:tasks.workunit.client.0.smithi038.stdout:5/308: creat d6/d1e/f63 x:0 0 0 2024-03-24T00:48:47.856 INFO:tasks.workunit.client.0.smithi038.stdout:5/309: fsync d6/d1e/f55 0 2024-03-24T00:48:47.856 INFO:tasks.workunit.client.0.smithi038.stdout:5/310: stat d6/d16/d21/d2d/l40 0 2024-03-24T00:48:47.856 INFO:tasks.workunit.client.0.smithi038.stdout:5/311: truncate d6/d16/d21/f5f 977938 0 2024-03-24T00:48:47.858 INFO:tasks.workunit.client.1.smithi088.stdout:3/340: creat d3/de/df/d31/f72 x:0 0 0 2024-03-24T00:48:47.858 INFO:tasks.workunit.client.1.smithi088.stdout:3/341: creat d3/d6/d27/d3c/f73 x:0 0 0 2024-03-24T00:48:47.858 INFO:tasks.workunit.client.1.smithi088.stdout:3/342: readlink d3/d6/d27/d3c/l26 0 2024-03-24T00:48:47.858 INFO:tasks.workunit.client.0.smithi038.stdout:3/376: symlink d3/d9/dc/df/l83 0 2024-03-24T00:48:47.862 INFO:tasks.workunit.client.0.smithi038.stdout:1/316: link da/f56 da/dd/d22/d5c/f6e 0 2024-03-24T00:48:47.862 INFO:tasks.workunit.client.0.smithi038.stdout:1/317: rename da/d3c to da/d3c/d6f 22 2024-03-24T00:48:47.862 INFO:tasks.workunit.client.0.smithi038.stdout:1/318: truncate da/f16 4674879 0 2024-03-24T00:48:47.862 INFO:tasks.workunit.client.0.smithi038.stdout:1/319: chown da/dd/d10/d14 347 1 2024-03-24T00:48:47.862 INFO:tasks.workunit.client.0.smithi038.stdout:1/320: chown da/dd/d2c/c6a 2000002706 1 2024-03-24T00:48:47.862 INFO:tasks.workunit.client.0.smithi038.stdout:1/321: fsync da/f23 0 2024-03-24T00:48:47.863 INFO:tasks.workunit.client.1.smithi088.stdout:1/403: dread d6/d27/d1e/f22 [0,4194304] 0 2024-03-24T00:48:47.863 INFO:tasks.workunit.client.1.smithi088.stdout:1/404: write d6/d27/d1e/f39 [1232971,91411] 0 2024-03-24T00:48:47.866 INFO:tasks.workunit.client.0.smithi038.stdout:0/293: mkdir d0/d2d/d5f 0 2024-03-24T00:48:47.867 INFO:tasks.workunit.client.0.smithi038.stdout:9/368: unlink d12/d41/c7a 0 2024-03-24T00:48:47.871 INFO:tasks.workunit.client.1.smithi088.stdout:7/316: dread d3/d32/d34/f37 [4194304,4194304] 0 2024-03-24T00:48:47.871 INFO:tasks.workunit.client.1.smithi088.stdout:7/317: dread - d3/f2f zero size 2024-03-24T00:48:47.871 INFO:tasks.workunit.client.1.smithi088.stdout:7/318: read d3/f30 [871200,26547] 0 2024-03-24T00:48:47.872 INFO:tasks.workunit.client.0.smithi038.stdout:2/328: mkdir d4/d76 0 2024-03-24T00:48:47.872 INFO:tasks.workunit.client.0.smithi038.stdout:2/329: write d4/d27/f72 [549014,3060] 0 2024-03-24T00:48:47.875 INFO:tasks.workunit.client.0.smithi038.stdout:4/296: unlink da/d12/d37/dd/c43 0 2024-03-24T00:48:47.878 INFO:tasks.workunit.client.0.smithi038.stdout:6/379: mkdir d1/da/d73 0 2024-03-24T00:48:47.883 INFO:tasks.workunit.client.0.smithi038.stdout:5/312: mkdir d6/d19/d20/d64 0 2024-03-24T00:48:47.885 INFO:tasks.workunit.client.0.smithi038.stdout:3/377: mknod d3/d2a/c84 0 2024-03-24T00:48:47.886 INFO:tasks.workunit.client.1.smithi088.stdout:5/349: symlink d1/d29/d3f/l72 0 2024-03-24T00:48:47.887 INFO:tasks.workunit.client.0.smithi038.stdout:3/378: fsync d3/d9/dc/df/d14/d19/f47 0 2024-03-24T00:48:47.887 INFO:tasks.workunit.client.0.smithi038.stdout:3/379: chown d3/d5a 218059 1 2024-03-24T00:48:47.887 INFO:tasks.workunit.client.0.smithi038.stdout:3/380: fsync d3/d5a/f79 0 2024-03-24T00:48:47.893 INFO:tasks.workunit.client.0.smithi038.stdout:1/322: symlink da/dd/d22/d5c/l70 0 2024-03-24T00:48:47.895 INFO:tasks.workunit.client.1.smithi088.stdout:9/355: symlink d1/d10/l65 0 2024-03-24T00:48:47.895 INFO:tasks.workunit.client.1.smithi088.stdout:8/361: mkdir d9/dc/d11/d24/d29/d36/d6a/d7b 0 2024-03-24T00:48:47.895 INFO:tasks.workunit.client.1.smithi088.stdout:9/356: chown d1/d15/c19 0 1 2024-03-24T00:48:47.895 INFO:tasks.workunit.client.1.smithi088.stdout:9/357: dread - d1/d22/f51 zero size 2024-03-24T00:48:47.895 INFO:tasks.workunit.client.1.smithi088.stdout:8/362: truncate d9/d6b/f6c 355915 0 2024-03-24T00:48:47.895 INFO:tasks.workunit.client.1.smithi088.stdout:9/358: creat d1/d15/d21/f66 x:0 0 0 2024-03-24T00:48:47.895 INFO:tasks.workunit.client.1.smithi088.stdout:9/359: chown d1/cd 6 1 2024-03-24T00:48:47.895 INFO:tasks.workunit.client.1.smithi088.stdout:8/363: write d9/dc/d11/d24/d43/d4a/f55 [1945585,119104] 0 2024-03-24T00:48:47.895 INFO:tasks.workunit.client.1.smithi088.stdout:8/364: dread - d9/dc/d11/d24/d29/d5b/f53 zero size 2024-03-24T00:48:47.899 INFO:tasks.workunit.client.1.smithi088.stdout:0/368: creat d9/dc/d4f/f72 x:0 0 0 2024-03-24T00:48:47.899 INFO:tasks.workunit.client.1.smithi088.stdout:0/369: chown d9/dc/d4f/f72 57698749 1 2024-03-24T00:48:47.899 INFO:tasks.workunit.client.1.smithi088.stdout:0/370: fsync d9/dc/de/d19/d44/f50 0 2024-03-24T00:48:47.909 INFO:tasks.workunit.client.1.smithi088.stdout:2/356: link db/dc/f1b db/d21/d4a/d4e/f6e 0 2024-03-24T00:48:47.912 INFO:tasks.workunit.client.0.smithi038.stdout:3/381: dread d3/d9/f16 [0,4194304] 0 2024-03-24T00:48:47.912 INFO:tasks.workunit.client.0.smithi038.stdout:3/382: chown d3/d80 1033192 1 2024-03-24T00:48:47.912 INFO:tasks.workunit.client.0.smithi038.stdout:4/297: mkdir da/d12/d29/d5b 0 2024-03-24T00:48:47.912 INFO:tasks.workunit.client.1.smithi088.stdout:4/364: sync 2024-03-24T00:48:47.913 INFO:tasks.workunit.client.1.smithi088.stdout:4/365: fsync d5/d13/d18/d24/f30 0 2024-03-24T00:48:47.914 INFO:tasks.workunit.client.1.smithi088.stdout:3/343: mknod d3/d6/d27/d3c/c74 0 2024-03-24T00:48:47.921 INFO:tasks.workunit.client.0.smithi038.stdout:5/313: dread d6/d23/f3b [0,4194304] 0 2024-03-24T00:48:47.921 INFO:tasks.workunit.client.0.smithi038.stdout:0/294: rename d0/d12/d13 to d0/d12/d60 0 2024-03-24T00:48:47.922 INFO:tasks.workunit.client.0.smithi038.stdout:0/295: write d0/f5 [1082439,124706] 0 2024-03-24T00:48:47.923 INFO:tasks.workunit.client.0.smithi038.stdout:4/298: rename da/d38/f41 to da/d12/f5c 0 2024-03-24T00:48:47.929 INFO:tasks.workunit.client.1.smithi088.stdout:7/319: link d3/d32/l49 d3/l64 0 2024-03-24T00:48:47.929 INFO:tasks.workunit.client.1.smithi088.stdout:6/390: rmdir db/d1d/d23/d3c/d4b 39 2024-03-24T00:48:47.929 INFO:tasks.workunit.client.1.smithi088.stdout:7/320: readlink d3/d31/l62 0 2024-03-24T00:48:47.930 INFO:tasks.workunit.client.1.smithi088.stdout:5/350: rename d1/d19/c3a to d1/d10/d1b/c73 0 2024-03-24T00:48:47.930 INFO:tasks.workunit.client.1.smithi088.stdout:5/351: readlink d1/d29/l34 0 2024-03-24T00:48:47.932 INFO:tasks.workunit.client.1.smithi088.stdout:9/360: unlink d1/d15/c1a 0 2024-03-24T00:48:47.932 INFO:tasks.workunit.client.1.smithi088.stdout:9/361: creat d1/d10/d2c/f67 x:0 0 0 2024-03-24T00:48:47.936 INFO:tasks.workunit.client.0.smithi038.stdout:6/380: truncate d1/da/d1b/d1d/f24 301258 0 2024-03-24T00:48:47.936 INFO:tasks.workunit.client.0.smithi038.stdout:5/314: rename d6/d19/c5a to d6/d16/d21/d2d/c65 0 2024-03-24T00:48:47.936 INFO:tasks.workunit.client.0.smithi038.stdout:6/381: truncate d1/da/d1b/f2b 1152649 0 2024-03-24T00:48:47.946 INFO:tasks.workunit.client.0.smithi038.stdout:8/349: dwrite d0/d45/d37/f61 [4194304,4194304] 0 2024-03-24T00:48:47.950 INFO:tasks.workunit.client.0.smithi038.stdout:2/330: dwrite d4/d18/d26/f4e [4194304,4194304] 0 2024-03-24T00:48:47.952 INFO:tasks.workunit.client.1.smithi088.stdout:8/365: mkdir d9/dc/d11/d24/d29/d5b/d6f/d7c 0 2024-03-24T00:48:47.953 INFO:tasks.workunit.client.1.smithi088.stdout:3/344: getdents d3/d32 0 2024-03-24T00:48:47.953 INFO:tasks.workunit.client.1.smithi088.stdout:0/371: truncate d9/dc/de/d19/d44/f50 398267 0 2024-03-24T00:48:47.953 INFO:tasks.workunit.client.1.smithi088.stdout:3/345: chown d3/d6/d27/d3c/l26 0 1 2024-03-24T00:48:47.955 INFO:tasks.workunit.client.1.smithi088.stdout:1/405: dwrite d6/f10 [0,4194304] 0 2024-03-24T00:48:47.955 INFO:tasks.workunit.client.0.smithi038.stdout:9/369: dwrite d12/d1a/f49 [0,4194304] 0 2024-03-24T00:48:47.957 INFO:tasks.workunit.client.0.smithi038.stdout:6/382: symlink d1/da/d1b/d25/d49/l74 0 2024-03-24T00:48:47.957 INFO:tasks.workunit.client.0.smithi038.stdout:6/383: read d1/da/d1b/f6a [654778,109477] 0 2024-03-24T00:48:47.964 INFO:tasks.workunit.client.1.smithi088.stdout:7/321: unlink d3/f1b 0 2024-03-24T00:48:47.964 INFO:tasks.workunit.client.1.smithi088.stdout:7/322: stat d3/fc 0 2024-03-24T00:48:47.967 INFO:tasks.workunit.client.0.smithi038.stdout:1/323: dwrite f7 [0,4194304] 0 2024-03-24T00:48:47.968 INFO:tasks.workunit.client.1.smithi088.stdout:6/391: mkdir db/dd/d61/d6d/d74 0 2024-03-24T00:48:47.971 INFO:tasks.workunit.client.0.smithi038.stdout:3/383: dwrite d3/d9/f58 [0,4194304] 0 2024-03-24T00:48:47.971 INFO:tasks.workunit.client.0.smithi038.stdout:3/384: chown d3/c63 703 1 2024-03-24T00:48:47.977 INFO:tasks.workunit.client.1.smithi088.stdout:8/366: truncate d9/dc/d11/d24/d29/d5b/f34 7780916 0 2024-03-24T00:48:47.980 INFO:tasks.workunit.client.0.smithi038.stdout:7/339: dwrite d2/d9/f20 [4194304,4194304] 0 2024-03-24T00:48:47.982 INFO:tasks.workunit.client.1.smithi088.stdout:0/372: symlink d9/dc/d4f/d62/l73 0 2024-03-24T00:48:47.984 INFO:tasks.workunit.client.0.smithi038.stdout:3/385: dread d3/d30/d5d/d6a/f76 [0,4194304] 0 2024-03-24T00:48:47.985 INFO:tasks.workunit.client.0.smithi038.stdout:5/315: symlink d6/l66 0 2024-03-24T00:48:47.985 INFO:tasks.workunit.client.0.smithi038.stdout:5/316: chown d6/d19/d20/d42/d4b/f4f 170 1 2024-03-24T00:48:47.986 INFO:tasks.workunit.client.1.smithi088.stdout:9/362: write d1/f20 [3772603,59627] 0 2024-03-24T00:48:47.986 INFO:tasks.workunit.client.1.smithi088.stdout:7/323: dread d3/f2d [4194304,4194304] 0 2024-03-24T00:48:47.987 INFO:tasks.workunit.client.1.smithi088.stdout:7/324: fdatasync d3/d32/d3f/d4b/f60 0 2024-03-24T00:48:47.987 INFO:tasks.workunit.client.1.smithi088.stdout:7/325: dread - d3/f45 zero size 2024-03-24T00:48:47.987 INFO:tasks.workunit.client.0.smithi038.stdout:2/331: dread d4/f10 [0,4194304] 0 2024-03-24T00:48:47.988 INFO:tasks.workunit.client.1.smithi088.stdout:3/346: unlink d3/d1a/l21 0 2024-03-24T00:48:47.990 INFO:tasks.workunit.client.1.smithi088.stdout:2/357: dwrite db/f20 [0,4194304] 0 2024-03-24T00:48:47.991 INFO:tasks.workunit.client.1.smithi088.stdout:6/392: write db/d1d/f48 [2453333,124164] 0 2024-03-24T00:48:47.994 INFO:tasks.workunit.client.0.smithi038.stdout:9/370: link d12/d1a/f20 d12/d41/f7d 0 2024-03-24T00:48:47.994 INFO:tasks.workunit.client.0.smithi038.stdout:9/371: dread - d12/d41/f58 zero size 2024-03-24T00:48:47.994 INFO:tasks.workunit.client.0.smithi038.stdout:9/372: write d12/d28/f36 [1678009,100201] 0 2024-03-24T00:48:47.996 INFO:tasks.workunit.client.0.smithi038.stdout:0/296: dwrite d0/d12/d60/d28/f39 [0,4194304] 0 2024-03-24T00:48:47.996 INFO:tasks.workunit.client.0.smithi038.stdout:0/297: write d0/d2d/f2f [2438786,71726] 0 2024-03-24T00:48:47.996 INFO:tasks.workunit.client.0.smithi038.stdout:0/298: chown d0/l29 51327 1 2024-03-24T00:48:47.997 INFO:tasks.workunit.client.0.smithi038.stdout:6/384: mkdir d1/da/d73/d75 0 2024-03-24T00:48:47.997 INFO:tasks.workunit.client.0.smithi038.stdout:6/385: read d1/da/d1b/d25/d49/d4e/f55 [1011714,60301] 0 2024-03-24T00:48:47.998 INFO:tasks.workunit.client.0.smithi038.stdout:9/373: write d12/f35 [486784,13682] 0 2024-03-24T00:48:47.998 INFO:tasks.workunit.client.0.smithi038.stdout:9/374: chown f8 4087017 1 2024-03-24T00:48:48.000 INFO:tasks.workunit.client.1.smithi088.stdout:1/406: creat d6/d27/d1e/d5b/d5d/f76 x:0 0 0 2024-03-24T00:48:48.001 INFO:tasks.workunit.client.1.smithi088.stdout:2/358: dread db/d21/d4a/d4e/f64 [0,4194304] 0 2024-03-24T00:48:48.008 INFO:tasks.workunit.client.1.smithi088.stdout:8/367: link d9/dc/d11/d24/d29/d5b/f53 d9/dc/d11/d24/d29/d5b/f7d 0 2024-03-24T00:48:48.014 INFO:tasks.workunit.client.1.smithi088.stdout:0/373: mknod d9/dc/de/d19/d44/d46/d65/c74 0 2024-03-24T00:48:48.020 INFO:tasks.workunit.client.0.smithi038.stdout:1/324: dread da/dd/d22/f25 [0,4194304] 0 2024-03-24T00:48:48.021 INFO:tasks.workunit.client.1.smithi088.stdout:7/326: symlink d3/d31/d56/l65 0 2024-03-24T00:48:48.026 INFO:tasks.workunit.client.0.smithi038.stdout:7/340: rename d2/d9/da/dc/c6b to d2/d9/d16/c81 0 2024-03-24T00:48:48.026 INFO:tasks.workunit.client.0.smithi038.stdout:8/350: dwrite d0/d45/d37/f61 [0,4194304] 0 2024-03-24T00:48:48.026 INFO:tasks.workunit.client.0.smithi038.stdout:8/351: chown d0/d1d/d27/l59 110 1 2024-03-24T00:48:48.026 INFO:tasks.workunit.client.0.smithi038.stdout:8/352: fdatasync d0/d15/f63 0 2024-03-24T00:48:48.026 INFO:tasks.workunit.client.0.smithi038.stdout:8/353: truncate d0/f76 440527 0 2024-03-24T00:48:48.027 INFO:tasks.workunit.client.1.smithi088.stdout:3/347: link d3/d6/f3b d3/de/d6f/f75 0 2024-03-24T00:48:48.028 INFO:tasks.workunit.client.1.smithi088.stdout:3/348: read d3/d6/d27/d3c/f39 [246011,104983] 0 2024-03-24T00:48:48.028 INFO:tasks.workunit.client.0.smithi038.stdout:5/317: mkdir d6/d16/d67 0 2024-03-24T00:48:48.033 INFO:tasks.workunit.client.0.smithi038.stdout:2/332: creat d4/d76/f77 x:0 0 0 2024-03-24T00:48:48.034 INFO:tasks.workunit.client.1.smithi088.stdout:1/407: link d6/d27/d1e/l31 d6/d27/d1e/d5b/d5d/l77 0 2024-03-24T00:48:48.044 INFO:tasks.workunit.client.1.smithi088.stdout:2/359: mkdir db/dc/d6f 0 2024-03-24T00:48:48.044 INFO:tasks.workunit.client.1.smithi088.stdout:7/327: read d3/f33 [755551,128856] 0 2024-03-24T00:48:48.046 INFO:tasks.workunit.client.0.smithi038.stdout:5/318: write d6/f26 [811376,91454] 0 2024-03-24T00:48:48.047 INFO:tasks.workunit.client.0.smithi038.stdout:5/319: read d6/f9 [394538,58626] 0 2024-03-24T00:48:48.050 INFO:tasks.workunit.client.0.smithi038.stdout:6/386: rmdir d1/da/d1b/d1d/d3c 39 2024-03-24T00:48:48.051 INFO:tasks.workunit.client.1.smithi088.stdout:8/368: symlink d9/dc/d11/d24/d29/d5b/d6f/l7e 0 2024-03-24T00:48:48.051 INFO:tasks.workunit.client.1.smithi088.stdout:8/369: chown d9/dc/l3a 5831087 1 2024-03-24T00:48:48.051 INFO:tasks.workunit.client.1.smithi088.stdout:8/370: read - d9/dc/d11/d24/f71 zero size 2024-03-24T00:48:48.051 INFO:tasks.workunit.client.1.smithi088.stdout:8/371: fsync d9/dc/d11/d24/d29/f51 0 2024-03-24T00:48:48.055 INFO:tasks.workunit.client.0.smithi038.stdout:3/386: rename d3/d9/dc/d51/f6f to d3/d9/d77/f85 0 2024-03-24T00:48:48.058 INFO:tasks.workunit.client.1.smithi088.stdout:0/374: dwrite d9/d26/f3c [0,4194304] 0 2024-03-24T00:48:48.058 INFO:tasks.workunit.client.1.smithi088.stdout:0/375: chown d9/cd 30593 1 2024-03-24T00:48:48.060 INFO:tasks.workunit.client.0.smithi038.stdout:8/354: unlink d0/d15/c4e 0 2024-03-24T00:48:48.063 INFO:tasks.workunit.client.0.smithi038.stdout:5/320: mkdir d6/d19/d20/d42/d43/d68 0 2024-03-24T00:48:48.063 INFO:tasks.workunit.client.0.smithi038.stdout:5/321: dread - d6/d19/d20/d42/d43/d4c/f62 zero size 2024-03-24T00:48:48.067 INFO:tasks.workunit.client.0.smithi038.stdout:0/299: rename d0/d12/d15/d1a/f2b to d0/d12/d42/f61 0 2024-03-24T00:48:48.068 INFO:tasks.workunit.client.1.smithi088.stdout:3/349: mkdir d3/d1a/d1e/d76 0 2024-03-24T00:48:48.069 INFO:tasks.workunit.client.0.smithi038.stdout:6/387: dread d1/da/d1b/f2b [0,4194304] 0 2024-03-24T00:48:48.069 INFO:tasks.workunit.client.1.smithi088.stdout:6/393: getdents db/d1d/d23/d5c 0 2024-03-24T00:48:48.072 INFO:tasks.workunit.client.0.smithi038.stdout:5/322: mknod d6/d19/d20/d33/c69 0 2024-03-24T00:48:48.072 INFO:tasks.workunit.client.0.smithi038.stdout:5/323: write d6/d1e/f63 [568592,76909] 0 2024-03-24T00:48:48.074 INFO:tasks.workunit.client.0.smithi038.stdout:7/341: dread d2/d9/df/d33/f31 [0,4194304] 0 2024-03-24T00:48:48.074 INFO:tasks.workunit.client.0.smithi038.stdout:7/342: chown d2/d9/df 4041130 1 2024-03-24T00:48:48.080 INFO:tasks.workunit.client.0.smithi038.stdout:1/325: rename da/f49 to da/dd/d22/f71 0 2024-03-24T00:48:48.082 INFO:tasks.workunit.client.1.smithi088.stdout:1/408: creat d6/d27/d1e/d58/f78 x:0 0 0 2024-03-24T00:48:48.085 INFO:tasks.workunit.client.1.smithi088.stdout:6/394: dread db/dd/f2b [0,4194304] 0 2024-03-24T00:48:48.085 INFO:tasks.workunit.client.1.smithi088.stdout:6/395: stat db/dd/c1b 0 2024-03-24T00:48:48.086 INFO:tasks.workunit.client.0.smithi038.stdout:6/388: creat d1/da/d1b/d25/d49/d72/f76 x:0 0 0 2024-03-24T00:48:48.086 INFO:tasks.workunit.client.0.smithi038.stdout:9/375: dwrite d12/d28/d6c/f74 [0,4194304] 0 2024-03-24T00:48:48.086 INFO:tasks.workunit.client.0.smithi038.stdout:9/376: fsync d12/d41/f58 0 2024-03-24T00:48:48.086 INFO:tasks.workunit.client.0.smithi038.stdout:9/377: write d12/d41/f5b [1160304,224] 0 2024-03-24T00:48:48.087 INFO:tasks.workunit.client.1.smithi088.stdout:2/360: dwrite db/dc/f2c [0,4194304] 0 2024-03-24T00:48:48.091 INFO:tasks.workunit.client.0.smithi038.stdout:7/343: mknod d2/d9/df/c82 0 2024-03-24T00:48:48.091 INFO:tasks.workunit.client.0.smithi038.stdout:7/344: chown d2/d2c/f4d 3316 1 2024-03-24T00:48:48.096 INFO:tasks.workunit.client.0.smithi038.stdout:1/326: link da/dd/d2c/d41/f65 da/f72 0 2024-03-24T00:48:48.098 INFO:tasks.workunit.client.1.smithi088.stdout:9/363: rmdir d1/d10/d11/d28 39 2024-03-24T00:48:48.098 INFO:tasks.workunit.client.1.smithi088.stdout:8/372: symlink d9/dc/d11/d24/d29/d36/d6a/l7f 0 2024-03-24T00:48:48.098 INFO:tasks.workunit.client.1.smithi088.stdout:9/364: write d1/d10/d11/d23/f53 [918366,24811] 0 2024-03-24T00:48:48.098 INFO:tasks.workunit.client.1.smithi088.stdout:7/328: creat d3/d32/d3f/d4b/f66 x:0 0 0 2024-03-24T00:48:48.098 INFO:tasks.workunit.client.1.smithi088.stdout:9/365: fdatasync d1/f5 0 2024-03-24T00:48:48.099 INFO:tasks.workunit.client.0.smithi038.stdout:3/387: dwrite d3/d9/f58 [0,4194304] 0 2024-03-24T00:48:48.105 INFO:tasks.workunit.client.1.smithi088.stdout:9/366: dread d1/d10/f34 [0,4194304] 0 2024-03-24T00:48:48.108 INFO:tasks.workunit.client.0.smithi038.stdout:8/355: rename d0/d45/c52 to d0/d45/d37/c7c 0 2024-03-24T00:48:48.108 INFO:tasks.workunit.client.0.smithi038.stdout:8/356: fdatasync d0/d1d/d27/d4d/f75 0 2024-03-24T00:48:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:47 smithi038 ceph-mon[29271]: pgmap v30: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 138 MiB/s rd, 166 MiB/s wr, 243 op/s 2024-03-24T00:48:48.116 INFO:tasks.workunit.client.0.smithi038.stdout:6/389: truncate d1/da/d1b/d25/f2a 1109710 0 2024-03-24T00:48:48.121 INFO:tasks.workunit.client.0.smithi038.stdout:9/378: mkdir d12/d55/d56/d7e 0 2024-03-24T00:48:48.127 INFO:tasks.workunit.client.0.smithi038.stdout:8/357: mknod d0/d45/d37/d5f/c7d 0 2024-03-24T00:48:48.127 INFO:tasks.workunit.client.0.smithi038.stdout:7/345: mknod d2/d9/d16/c83 0 2024-03-24T00:48:48.131 INFO:tasks.workunit.client.1.smithi088.stdout:0/376: rename d9/dc/de/d19/f3b to d9/dc/de/d19/d44/d46/d65/d2a/f75 0 2024-03-24T00:48:48.131 INFO:tasks.workunit.client.1.smithi088.stdout:0/377: write d9/dc/de/f3e [184776,73301] 0 2024-03-24T00:48:48.131 INFO:tasks.workunit.client.1.smithi088.stdout:1/409: creat d6/d6e/f79 x:0 0 0 2024-03-24T00:48:48.131 INFO:tasks.workunit.client.1.smithi088.stdout:3/350: truncate d3/de/df/d31/f72 462647 0 2024-03-24T00:48:48.131 INFO:tasks.workunit.client.1.smithi088.stdout:2/361: mknod db/d54/c70 0 2024-03-24T00:48:48.142 INFO:tasks.workunit.client.1.smithi088.stdout:6/396: rename db/d1d/d50/l52 to db/l75 0 2024-03-24T00:48:48.142 INFO:tasks.workunit.client.1.smithi088.stdout:9/367: write d1/d10/d11/d28/f4f [824675,27748] 0 2024-03-24T00:48:48.143 INFO:tasks.workunit.client.1.smithi088.stdout:6/397: truncate db/d1d/d23/d3c/d4b/f6e 962019 0 2024-03-24T00:48:48.143 INFO:tasks.workunit.client.1.smithi088.stdout:9/368: dread - d1/d10/d41/f57 zero size 2024-03-24T00:48:48.143 INFO:tasks.workunit.client.1.smithi088.stdout:9/369: fsync d1/d10/d11/d23/f54 0 2024-03-24T00:48:48.143 INFO:tasks.workunit.client.1.smithi088.stdout:1/410: mknod d6/d27/d1e/c7a 0 2024-03-24T00:48:48.151 INFO:tasks.workunit.client.1.smithi088.stdout:2/362: link db/d21/d4a/d4e/f64 db/d21/d35/d49/f71 0 2024-03-24T00:48:48.151 INFO:tasks.workunit.client.1.smithi088.stdout:0/378: write d9/dc/de/d19/d44/d46/d65/d2a/f5e [2163795,78989] 0 2024-03-24T00:48:48.151 INFO:tasks.workunit.client.1.smithi088.stdout:8/373: mkdir d9/dc/d11/d80 0 2024-03-24T00:48:48.152 INFO:tasks.workunit.client.1.smithi088.stdout:8/374: chown d9/dc/c19 5 1 2024-03-24T00:48:48.152 INFO:tasks.workunit.client.1.smithi088.stdout:0/379: write d9/f45 [533653,35415] 0 2024-03-24T00:48:48.153 INFO:tasks.workunit.client.1.smithi088.stdout:7/329: dread d3/f2a [0,4194304] 0 2024-03-24T00:48:48.155 INFO:tasks.workunit.client.0.smithi038.stdout:7/346: dread d2/d9/da/f36 [0,4194304] 0 2024-03-24T00:48:48.157 INFO:tasks.workunit.client.1.smithi088.stdout:6/398: rmdir db/d1d 39 2024-03-24T00:48:48.166 INFO:tasks.workunit.client.1.smithi088.stdout:3/351: creat d3/d6/d19/f77 x:0 0 0 2024-03-24T00:48:48.167 INFO:tasks.workunit.client.0.smithi038.stdout:9/379: dread d12/d41/f5b [0,4194304] 0 2024-03-24T00:48:48.167 INFO:tasks.workunit.client.0.smithi038.stdout:7/347: link d2/d9/df/c51 d2/d9/da/dc/c84 0 2024-03-24T00:48:48.170 INFO:tasks.workunit.client.1.smithi088.stdout:9/370: creat d1/d15/f68 x:0 0 0 2024-03-24T00:48:48.170 INFO:tasks.workunit.client.1.smithi088.stdout:9/371: chown d1/d10/d29 679970296 1 2024-03-24T00:48:48.175 INFO:tasks.workunit.client.1.smithi088.stdout:2/363: rename db/dc/l1c to db/d21/d35/d49/l72 0 2024-03-24T00:48:48.175 INFO:tasks.workunit.client.1.smithi088.stdout:0/380: rename d9 to d9/dc/d4f/d76 22 2024-03-24T00:48:48.176 INFO:tasks.workunit.client.1.smithi088.stdout:2/364: read - db/d54/f6d zero size 2024-03-24T00:48:48.176 INFO:tasks.workunit.client.1.smithi088.stdout:8/375: mkdir d9/dc/d11/d24/d43/d4a/d81 0 2024-03-24T00:48:48.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:47 smithi088 ceph-mon[36610]: pgmap v30: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 138 MiB/s rd, 166 MiB/s wr, 243 op/s 2024-03-24T00:48:48.180 INFO:tasks.workunit.client.0.smithi038.stdout:1/327: dwrite da/dd/d10/d14/f47 [0,4194304] 0 2024-03-24T00:48:48.181 INFO:tasks.workunit.client.0.smithi038.stdout:5/324: dwrite d6/d19/d20/d42/d4b/f4f [0,4194304] 0 2024-03-24T00:48:48.184 INFO:tasks.workunit.client.1.smithi088.stdout:7/330: write d3/f2f [882336,90394] 0 2024-03-24T00:48:48.184 INFO:tasks.workunit.client.1.smithi088.stdout:7/331: chown d3/d32/d3f/d4b/f66 3 1 2024-03-24T00:48:48.185 INFO:tasks.workunit.client.1.smithi088.stdout:4/366: sync 2024-03-24T00:48:48.185 INFO:tasks.workunit.client.1.smithi088.stdout:5/352: sync 2024-03-24T00:48:48.186 INFO:tasks.workunit.client.0.smithi038.stdout:9/380: dread d12/d41/f5b [0,4194304] 0 2024-03-24T00:48:48.186 INFO:tasks.workunit.client.0.smithi038.stdout:9/381: truncate d12/d28/d3a/f40 132074 0 2024-03-24T00:48:48.187 INFO:tasks.workunit.client.0.smithi038.stdout:0/300: write d0/f7 [1443381,101390] 0 2024-03-24T00:48:48.189 INFO:tasks.workunit.client.0.smithi038.stdout:2/333: dwrite d4/d18/f2e [0,4194304] 0 2024-03-24T00:48:48.189 INFO:tasks.workunit.client.0.smithi038.stdout:2/334: readlink d4/d27/d54/l73 0 2024-03-24T00:48:48.189 INFO:tasks.workunit.client.0.smithi038.stdout:2/335: creat d4/d27/d2f/d30/d3e/f78 x:0 0 0 2024-03-24T00:48:48.189 INFO:tasks.workunit.client.0.smithi038.stdout:2/336: fdatasync d4/d18/d26/f4e 0 2024-03-24T00:48:48.191 INFO:tasks.workunit.client.0.smithi038.stdout:8/358: dread d0/f76 [0,4194304] 0 2024-03-24T00:48:48.198 INFO:tasks.workunit.client.1.smithi088.stdout:1/411: getdents d6/d27/d67 0 2024-03-24T00:48:48.200 INFO:tasks.workunit.client.1.smithi088.stdout:0/381: symlink d9/d15/l77 0 2024-03-24T00:48:48.201 INFO:tasks.workunit.client.1.smithi088.stdout:6/399: write db/dd/f37 [2536625,58494] 0 2024-03-24T00:48:48.202 INFO:tasks.workunit.client.0.smithi038.stdout:1/328: creat da/dd/d2c/d5a/f73 x:0 0 0 2024-03-24T00:48:48.205 INFO:tasks.workunit.client.0.smithi038.stdout:5/325: link d6/d19/d20/d42/d43/d4c/c58 d6/d19/d20/d42/c6a 0 2024-03-24T00:48:48.206 INFO:tasks.workunit.client.1.smithi088.stdout:8/376: truncate d9/dc/d11/d24/f2a 1074566 0 2024-03-24T00:48:48.210 INFO:tasks.workunit.client.1.smithi088.stdout:7/332: unlink d3/f33 0 2024-03-24T00:48:48.211 INFO:tasks.workunit.client.1.smithi088.stdout:4/367: link d5/d17/f1d d5/d13/d18/d6e/d7d/f7e 0 2024-03-24T00:48:48.212 INFO:tasks.workunit.client.1.smithi088.stdout:4/368: truncate d5/d13/d18/d6e/d7d/f7e 447856 0 2024-03-24T00:48:48.214 INFO:tasks.workunit.client.0.smithi038.stdout:3/388: dwrite d3/d9/dc/df/d14/d19/f47 [0,4194304] 0 2024-03-24T00:48:48.214 INFO:tasks.workunit.client.0.smithi038.stdout:9/382: link d12/l13 d12/d55/d56/l7f 0 2024-03-24T00:48:48.219 INFO:tasks.workunit.client.1.smithi088.stdout:9/372: mknod d1/d10/d11/d28/c69 0 2024-03-24T00:48:48.220 INFO:tasks.workunit.client.1.smithi088.stdout:9/373: chown d1/d22/d27/l4c 2406162 1 2024-03-24T00:48:48.226 INFO:tasks.workunit.client.0.smithi038.stdout:8/359: read d0/d45/d66/f40 [3217642,9113] 0 2024-03-24T00:48:48.227 INFO:tasks.workunit.client.1.smithi088.stdout:0/382: creat d9/dc/de/d19/d44/d46/d65/d6f/f78 x:0 0 0 2024-03-24T00:48:48.227 INFO:tasks.workunit.client.0.smithi038.stdout:2/337: mknod d4/d18/d5b/c79 0 2024-03-24T00:48:48.228 INFO:tasks.workunit.client.1.smithi088.stdout:2/365: mkdir db/d21/d73 0 2024-03-24T00:48:48.228 INFO:tasks.workunit.client.1.smithi088.stdout:2/366: fdatasync db/dc/f2c 0 2024-03-24T00:48:48.234 INFO:tasks.workunit.client.1.smithi088.stdout:3/352: dwrite d3/d6/d27/d3c/f73 [0,4194304] 0 2024-03-24T00:48:48.240 INFO:tasks.workunit.client.0.smithi038.stdout:9/383: mkdir d12/d1a/d80 0 2024-03-24T00:48:48.245 INFO:tasks.workunit.client.1.smithi088.stdout:7/333: write d3/f27 [687834,103650] 0 2024-03-24T00:48:48.245 INFO:tasks.workunit.client.1.smithi088.stdout:6/400: dwrite db/d40/d56/f30 [0,4194304] 0 2024-03-24T00:48:48.245 INFO:tasks.workunit.client.1.smithi088.stdout:6/401: fdatasync db/dd/f41 0 2024-03-24T00:48:48.248 INFO:tasks.workunit.client.1.smithi088.stdout:4/369: symlink d5/d7/l7f 0 2024-03-24T00:48:48.257 INFO:tasks.workunit.client.1.smithi088.stdout:8/377: dwrite d9/dc/d11/d41/d65/f6e [0,4194304] 0 2024-03-24T00:48:48.258 INFO:tasks.workunit.client.1.smithi088.stdout:8/378: write d9/dc/f52 [808030,42002] 0 2024-03-24T00:48:48.264 INFO:tasks.workunit.client.1.smithi088.stdout:1/412: getdents d6 0 2024-03-24T00:48:48.267 INFO:tasks.workunit.client.1.smithi088.stdout:4/370: dread d5/d16/f7b [0,4194304] 0 2024-03-24T00:48:48.267 INFO:tasks.workunit.client.1.smithi088.stdout:0/383: symlink d9/dc/l79 0 2024-03-24T00:48:48.271 INFO:tasks.workunit.client.1.smithi088.stdout:4/371: write d5/d16/d37/f48 [50156,68251] 0 2024-03-24T00:48:48.272 INFO:tasks.workunit.client.1.smithi088.stdout:2/367: creat db/d21/f74 x:0 0 0 2024-03-24T00:48:48.278 INFO:tasks.workunit.client.0.smithi038.stdout:6/390: dwrite d1/da/d1b/d1d/f2c [0,4194304] 0 2024-03-24T00:48:48.278 INFO:tasks.workunit.client.0.smithi038.stdout:6/391: chown d1/da/d1b/d2f/f62 12 1 2024-03-24T00:48:48.278 INFO:tasks.workunit.client.0.smithi038.stdout:6/392: chown d1/d37/l4c 33872985 1 2024-03-24T00:48:48.279 INFO:tasks.workunit.client.0.smithi038.stdout:8/360: unlink d0/d15/l2d 0 2024-03-24T00:48:48.282 INFO:tasks.workunit.client.0.smithi038.stdout:2/338: link d4/f13 d4/f7a 0 2024-03-24T00:48:48.292 INFO:tasks.workunit.client.0.smithi038.stdout:9/384: chown d12/d28/f51 15613593 1 2024-03-24T00:48:48.292 INFO:tasks.workunit.client.0.smithi038.stdout:9/385: chown d12/d41/d57 0 1 2024-03-24T00:48:48.292 INFO:tasks.workunit.client.0.smithi038.stdout:9/386: truncate d12/d28/d3a/f77 720933 0 2024-03-24T00:48:48.300 INFO:tasks.workunit.client.0.smithi038.stdout:6/393: mknod d1/da/d73/c77 0 2024-03-24T00:48:48.303 INFO:tasks.workunit.client.1.smithi088.stdout:7/334: fsync d3/f45 0 2024-03-24T00:48:48.304 INFO:tasks.workunit.client.1.smithi088.stdout:6/402: getdents db/d1d 0 2024-03-24T00:48:48.309 INFO:tasks.workunit.client.0.smithi038.stdout:9/387: write d12/d1a/f49 [3336435,21836] 0 2024-03-24T00:48:48.309 INFO:tasks.workunit.client.0.smithi038.stdout:2/339: getdents d4/d27/d2f/d30 0 2024-03-24T00:48:48.312 INFO:tasks.workunit.client.0.smithi038.stdout:1/329: rename da/d3c/d2d/d5f to da/dd/d2c/d5a/d74 0 2024-03-24T00:48:48.312 INFO:tasks.workunit.client.0.smithi038.stdout:1/330: chown da/d3c/d2d/c3e 161 1 2024-03-24T00:48:48.313 INFO:tasks.workunit.client.0.smithi038.stdout:3/389: dwrite d3/f6 [4194304,4194304] 0 2024-03-24T00:48:48.313 INFO:tasks.workunit.client.0.smithi038.stdout:3/390: readlink d3/d9/dc/df/l83 0 2024-03-24T00:48:48.313 INFO:tasks.workunit.client.0.smithi038.stdout:3/391: write d3/d9/d77/f7f [1012972,32441] 0 2024-03-24T00:48:48.316 INFO:tasks.workunit.client.0.smithi038.stdout:6/394: symlink d1/da/d1b/d25/l78 0 2024-03-24T00:48:48.320 INFO:tasks.workunit.client.1.smithi088.stdout:2/368: dwrite db/f46 [0,4194304] 0 2024-03-24T00:48:48.320 INFO:tasks.workunit.client.0.smithi038.stdout:8/361: unlink d0/d1d/f3e 0 2024-03-24T00:48:48.327 INFO:tasks.workunit.client.1.smithi088.stdout:8/379: rename d9/dc/ce to d9/dc/d11/d41/d65/c82 0 2024-03-24T00:48:48.329 INFO:tasks.workunit.client.0.smithi038.stdout:9/388: unlink d12/f6f 0 2024-03-24T00:48:48.330 INFO:tasks.workunit.client.0.smithi038.stdout:9/389: dread - d12/d55/f6e zero size 2024-03-24T00:48:48.330 INFO:tasks.workunit.client.0.smithi038.stdout:9/390: chown d12/d28/d6c/d75 857753 1 2024-03-24T00:48:48.330 INFO:tasks.workunit.client.0.smithi038.stdout:9/391: fsync d12/d28/d3a/f77 0 2024-03-24T00:48:48.331 INFO:tasks.workunit.client.1.smithi088.stdout:9/374: link d1/d10/d11/d28/c5e d1/d10/d41/c6a 0 2024-03-24T00:48:48.333 INFO:tasks.workunit.client.0.smithi038.stdout:2/340: creat d4/d18/f7b x:0 0 0 2024-03-24T00:48:48.334 INFO:tasks.workunit.client.1.smithi088.stdout:0/384: creat d9/dc/f7a x:0 0 0 2024-03-24T00:48:48.335 INFO:tasks.workunit.client.1.smithi088.stdout:0/385: chown c5 1352 1 2024-03-24T00:48:48.335 INFO:tasks.workunit.client.0.smithi038.stdout:5/326: rename d6/d19/d4d to d6/d19/d20/d64/d6b 0 2024-03-24T00:48:48.337 INFO:tasks.workunit.client.0.smithi038.stdout:6/395: mkdir d1/da/d61/d79 0 2024-03-24T00:48:48.343 INFO:tasks.workunit.client.1.smithi088.stdout:4/372: symlink d5/d7/l80 0 2024-03-24T00:48:48.343 INFO:tasks.workunit.client.1.smithi088.stdout:4/373: chown d5/d13/d18/f79 12183637 1 2024-03-24T00:48:48.344 INFO:tasks.workunit.client.1.smithi088.stdout:3/353: rmdir d3/d6/d27/d3c 39 2024-03-24T00:48:48.348 INFO:tasks.workunit.client.0.smithi038.stdout:5/327: symlink d6/d1e/l6c 0 2024-03-24T00:48:48.348 INFO:tasks.workunit.client.0.smithi038.stdout:1/331: rename da/d3c/f2a to da/dd/d2c/d5a/d74/f75 0 2024-03-24T00:48:48.348 INFO:tasks.workunit.client.0.smithi038.stdout:1/332: dread - da/dd/d10/d34/f42 zero size 2024-03-24T00:48:48.348 INFO:tasks.workunit.client.0.smithi038.stdout:1/333: stat da/dd/d10/d34/f39 0 2024-03-24T00:48:48.348 INFO:tasks.workunit.client.0.smithi038.stdout:1/334: readlink l6 0 2024-03-24T00:48:48.349 INFO:tasks.workunit.client.1.smithi088.stdout:7/335: fdatasync d3/f2b 0 2024-03-24T00:48:48.352 INFO:tasks.workunit.client.1.smithi088.stdout:6/403: creat db/d1d/d64/d72/f76 x:0 0 0 2024-03-24T00:48:48.354 INFO:tasks.workunit.client.1.smithi088.stdout:2/369: mkdir db/d21/d4a/d75 0 2024-03-24T00:48:48.357 INFO:tasks.workunit.client.0.smithi038.stdout:1/335: rename da/d3c/d2d to da/dd/d4a/d76 0 2024-03-24T00:48:48.359 INFO:tasks.workunit.client.1.smithi088.stdout:9/375: mknod d1/d22/d27/d39/c6b 0 2024-03-24T00:48:48.364 INFO:tasks.workunit.client.0.smithi038.stdout:1/336: symlink da/dd/d2c/d40/l77 0 2024-03-24T00:48:48.364 INFO:tasks.workunit.client.0.smithi038.stdout:1/337: stat da/f23 0 2024-03-24T00:48:48.364 INFO:tasks.workunit.client.0.smithi038.stdout:1/338: dread - da/f56 zero size 2024-03-24T00:48:48.364 INFO:tasks.workunit.client.0.smithi038.stdout:1/339: write da/dd/d2c/d40/f53 [818962,50984] 0 2024-03-24T00:48:48.364 INFO:tasks.workunit.client.0.smithi038.stdout:1/340: chown da/dd/d22/d5c/l70 545038163 1 2024-03-24T00:48:48.365 INFO:tasks.workunit.client.1.smithi088.stdout:4/374: chown d5/d7/d72/c3b 30 1 2024-03-24T00:48:48.368 INFO:tasks.workunit.client.1.smithi088.stdout:7/336: readlink d3/l64 0 2024-03-24T00:48:48.369 INFO:tasks.workunit.client.0.smithi038.stdout:6/396: dread d1/da/d1b/d1d/f43 [0,4194304] 0 2024-03-24T00:48:48.372 INFO:tasks.workunit.client.0.smithi038.stdout:5/328: dread d6/d23/f3b [0,4194304] 0 2024-03-24T00:48:48.374 INFO:tasks.workunit.client.0.smithi038.stdout:1/341: symlink da/dd/d22/l78 0 2024-03-24T00:48:48.374 INFO:tasks.workunit.client.0.smithi038.stdout:1/342: fdatasync f5 0 2024-03-24T00:48:48.375 INFO:tasks.workunit.client.1.smithi088.stdout:6/404: write db/d1d/d23/d3c/d43/f39 [7732575,117527] 0 2024-03-24T00:48:48.376 INFO:tasks.workunit.client.0.smithi038.stdout:5/329: mkdir d6/d19/d20/d42/d43/d4c/d6d 0 2024-03-24T00:48:48.380 INFO:tasks.workunit.client.0.smithi038.stdout:1/343: mknod da/dd/d10/d34/c79 0 2024-03-24T00:48:48.381 INFO:tasks.workunit.client.1.smithi088.stdout:0/386: dread d9/d26/f3c [0,4194304] 0 2024-03-24T00:48:48.381 INFO:tasks.workunit.client.1.smithi088.stdout:5/353: sync 2024-03-24T00:48:48.382 INFO:tasks.workunit.client.0.smithi038.stdout:8/362: dwrite d0/d45/d37/f44 [0,4194304] 0 2024-03-24T00:48:48.383 INFO:tasks.workunit.client.1.smithi088.stdout:4/375: rmdir d5/d7/d15/d60 39 2024-03-24T00:48:48.385 INFO:tasks.workunit.client.0.smithi038.stdout:9/392: dwrite d12/d1a/f49 [0,4194304] 0 2024-03-24T00:48:48.385 INFO:tasks.workunit.client.0.smithi038.stdout:9/393: write d12/d55/d56/f7c [947467,62807] 0 2024-03-24T00:48:48.388 INFO:tasks.workunit.client.1.smithi088.stdout:8/380: dread d9/f3e [0,4194304] 0 2024-03-24T00:48:48.389 INFO:tasks.workunit.client.1.smithi088.stdout:2/370: rename db/dc/f30 to db/d21/f76 0 2024-03-24T00:48:48.392 INFO:tasks.workunit.client.1.smithi088.stdout:1/413: sync 2024-03-24T00:48:48.392 INFO:tasks.workunit.client.1.smithi088.stdout:1/414: read - d6/d27/d1e/d5b/d5d/f76 zero size 2024-03-24T00:48:48.392 INFO:tasks.workunit.client.1.smithi088.stdout:1/415: chown d6/d27/d1e/d5b/f75 4862 1 2024-03-24T00:48:48.392 INFO:tasks.workunit.client.1.smithi088.stdout:1/416: readlink d6/lb 0 2024-03-24T00:48:48.392 INFO:tasks.workunit.client.1.smithi088.stdout:7/337: mknod d3/d31/c67 0 2024-03-24T00:48:48.395 INFO:tasks.workunit.client.1.smithi088.stdout:8/381: dread d9/dc/d11/d24/d29/d39/f48 [0,4194304] 0 2024-03-24T00:48:48.398 INFO:tasks.workunit.client.0.smithi038.stdout:1/344: getdents da/dd/d2c 0 2024-03-24T00:48:48.400 INFO:tasks.workunit.client.1.smithi088.stdout:0/387: write d9/dc/de/d19/d44/d46/d65/f58 [6285679,100224] 0 2024-03-24T00:48:48.402 INFO:tasks.workunit.client.1.smithi088.stdout:2/371: rmdir db/dc/d6f 0 2024-03-24T00:48:48.407 INFO:tasks.workunit.client.1.smithi088.stdout:1/417: symlink d6/d43/l7b 0 2024-03-24T00:48:48.407 INFO:tasks.workunit.client.1.smithi088.stdout:1/418: creat d6/d27/d1e/d58/f7c x:0 0 0 2024-03-24T00:48:48.407 INFO:tasks.workunit.client.1.smithi088.stdout:1/419: write d6/d27/d1e/d5b/d5d/f76 [64767,107333] 0 2024-03-24T00:48:48.408 INFO:tasks.workunit.client.1.smithi088.stdout:5/354: dread d1/ff [0,4194304] 0 2024-03-24T00:48:48.408 INFO:tasks.workunit.client.0.smithi038.stdout:9/394: rmdir d12/d55/d56/d7e 0 2024-03-24T00:48:48.411 INFO:tasks.workunit.client.1.smithi088.stdout:9/376: dwrite d1/f4 [0,4194304] 0 2024-03-24T00:48:48.412 INFO:tasks.workunit.client.1.smithi088.stdout:9/377: creat d1/d10/d29/f6c x:0 0 0 2024-03-24T00:48:48.412 INFO:tasks.workunit.client.0.smithi038.stdout:1/345: mkdir da/dd/d7a 0 2024-03-24T00:48:48.417 INFO:tasks.workunit.client.1.smithi088.stdout:8/382: creat d9/dc/d11/d24/d29/d5b/d6f/f83 x:0 0 0 2024-03-24T00:48:48.417 INFO:tasks.workunit.client.1.smithi088.stdout:8/383: chown d9/c68 62788425 1 2024-03-24T00:48:48.423 INFO:tasks.workunit.client.0.smithi038.stdout:6/397: write d1/da/d1b/d1d/f2c [2880964,92691] 0 2024-03-24T00:48:48.431 INFO:tasks.workunit.client.1.smithi088.stdout:5/355: getdents d1/d19 0 2024-03-24T00:48:48.433 INFO:tasks.workunit.client.1.smithi088.stdout:5/356: rmdir d1/d10/d1b/d2e/d60 39 2024-03-24T00:48:48.437 INFO:tasks.workunit.client.0.smithi038.stdout:6/398: truncate d1/da/d1b/d2f/f62 2172537 0 2024-03-24T00:48:48.441 INFO:tasks.workunit.client.1.smithi088.stdout:5/357: symlink d1/d10/d1b/d2e/l74 0 2024-03-24T00:48:48.441 INFO:tasks.workunit.client.1.smithi088.stdout:5/358: chown d1/d10/c11 1 1 2024-03-24T00:48:48.443 INFO:tasks.workunit.client.1.smithi088.stdout:2/372: read db/dd/f1f [3945766,129703] 0 2024-03-24T00:48:48.446 INFO:tasks.workunit.client.1.smithi088.stdout:5/359: rename d1/d10/d1b/d2f/f58 to d1/d43/d4e/d5b/d66/f75 0 2024-03-24T00:48:48.447 INFO:tasks.workunit.client.1.smithi088.stdout:5/360: creat d1/d19/d1e/f76 x:0 0 0 2024-03-24T00:48:48.451 INFO:tasks.workunit.client.1.smithi088.stdout:2/373: link db/d21/d4a/l58 db/dd/l77 0 2024-03-24T00:48:48.451 INFO:tasks.workunit.client.1.smithi088.stdout:2/374: truncate db/dc/f2b 169705 0 2024-03-24T00:48:48.451 INFO:tasks.workunit.client.1.smithi088.stdout:6/405: dwrite db/d40/d56/f51 [0,4194304] 0 2024-03-24T00:48:48.451 INFO:tasks.workunit.client.1.smithi088.stdout:5/361: unlink d1/d29/d3f/l72 0 2024-03-24T00:48:48.451 INFO:tasks.workunit.client.1.smithi088.stdout:5/362: write d1/d10/d1b/d2f/f4a [429616,118070] 0 2024-03-24T00:48:48.452 INFO:tasks.workunit.client.1.smithi088.stdout:5/363: chown d1/d43/d4e/d5b/d66 6265530 1 2024-03-24T00:48:48.459 INFO:tasks.workunit.client.1.smithi088.stdout:8/384: dread d9/dc/f16 [0,4194304] 0 2024-03-24T00:48:48.459 INFO:tasks.workunit.client.1.smithi088.stdout:8/385: fdatasync d9/dc/d11/d24/d29/d5b/f34 0 2024-03-24T00:48:48.464 INFO:tasks.workunit.client.0.smithi038.stdout:9/395: dread d12/d1a/f3c [0,4194304] 0 2024-03-24T00:48:48.464 INFO:tasks.workunit.client.0.smithi038.stdout:5/330: dwrite d6/f3f [4194304,4194304] 0 2024-03-24T00:48:48.465 INFO:tasks.workunit.client.1.smithi088.stdout:1/420: dwrite d6/f10 [0,4194304] 0 2024-03-24T00:48:48.469 INFO:tasks.workunit.client.0.smithi038.stdout:9/396: mkdir d12/d81 0 2024-03-24T00:48:48.469 INFO:tasks.workunit.client.0.smithi038.stdout:9/397: truncate d12/d28/d3a/f77 766687 0 2024-03-24T00:48:48.469 INFO:tasks.workunit.client.0.smithi038.stdout:1/346: dwrite da/d3c/f13 [4194304,4194304] 0 2024-03-24T00:48:48.475 INFO:tasks.workunit.client.1.smithi088.stdout:2/375: dread db/dd/f28 [0,4194304] 0 2024-03-24T00:48:48.476 INFO:tasks.workunit.client.1.smithi088.stdout:2/376: creat db/d54/f78 x:0 0 0 2024-03-24T00:48:48.478 INFO:tasks.workunit.client.1.smithi088.stdout:2/377: rmdir db/d21/d4a/d75 0 2024-03-24T00:48:48.481 INFO:tasks.workunit.client.0.smithi038.stdout:8/363: dwrite d0/d1d/d20/d3c/f47 [0,4194304] 0 2024-03-24T00:48:48.481 INFO:tasks.workunit.client.1.smithi088.stdout:9/378: dwrite d1/d22/d27/d39/f4d [0,4194304] 0 2024-03-24T00:48:48.481 INFO:tasks.workunit.client.1.smithi088.stdout:2/378: link db/d21/d4a/d4e/f6e db/d21/d35/f79 0 2024-03-24T00:48:48.487 INFO:tasks.workunit.client.1.smithi088.stdout:1/421: write d6/f68 [2284543,80118] 0 2024-03-24T00:48:48.487 INFO:tasks.workunit.client.1.smithi088.stdout:5/364: dread d1/fa [0,4194304] 0 2024-03-24T00:48:48.488 INFO:tasks.workunit.client.1.smithi088.stdout:5/365: write d1/d19/d1e/f76 [634166,51348] 0 2024-03-24T00:48:48.488 INFO:tasks.workunit.client.0.smithi038.stdout:8/364: rename d0/d45/d66/c6c to d0/d1d/d20/d3c/c7e 0 2024-03-24T00:48:48.490 INFO:tasks.workunit.client.0.smithi038.stdout:1/347: write da/f23 [2187414,100807] 0 2024-03-24T00:48:48.490 INFO:tasks.workunit.client.0.smithi038.stdout:5/331: dread d6/d23/f3b [4194304,4194304] 0 2024-03-24T00:48:48.490 INFO:tasks.workunit.client.0.smithi038.stdout:5/332: chown d6/d19/d20/d42/d43 8 1 2024-03-24T00:48:48.491 INFO:tasks.workunit.client.1.smithi088.stdout:6/406: write db/d40/d56/f30 [539045,86373] 0 2024-03-24T00:48:48.491 INFO:tasks.workunit.client.1.smithi088.stdout:6/407: write db/d40/d56/f31 [1240887,78241] 0 2024-03-24T00:48:48.491 INFO:tasks.workunit.client.1.smithi088.stdout:6/408: creat db/d1d/d50/f77 x:0 0 0 2024-03-24T00:48:48.494 INFO:tasks.workunit.client.0.smithi038.stdout:8/365: symlink d0/d10/d19/d62/l7f 0 2024-03-24T00:48:48.499 INFO:tasks.workunit.client.1.smithi088.stdout:6/409: dread db/d1d/f62 [0,4194304] 0 2024-03-24T00:48:48.499 INFO:tasks.workunit.client.1.smithi088.stdout:6/410: chown db/d40/d56/l70 32498477 1 2024-03-24T00:48:48.500 INFO:tasks.workunit.client.0.smithi038.stdout:1/348: unlink da/dd/d22/c3a 0 2024-03-24T00:48:48.502 INFO:tasks.workunit.client.0.smithi038.stdout:1/349: write da/dd/d10/d14/f2e [18640,99401] 0 2024-03-24T00:48:48.502 INFO:tasks.workunit.client.0.smithi038.stdout:1/350: dread - da/dd/d2c/d40/f6c zero size 2024-03-24T00:48:48.502 INFO:tasks.workunit.client.0.smithi038.stdout:1/351: stat da/dd/d2c/d5a/d74/f64 0 2024-03-24T00:48:48.503 INFO:tasks.workunit.client.0.smithi038.stdout:1/352: chown da/dd/d10/d14/f4d 143271 1 2024-03-24T00:48:48.503 INFO:tasks.workunit.client.1.smithi088.stdout:1/422: mkdir d6/d27/d1e/d5b/d5d/d61/d7d 0 2024-03-24T00:48:48.504 INFO:tasks.workunit.client.0.smithi038.stdout:5/333: truncate d6/d23/f32 2671582 0 2024-03-24T00:48:48.504 INFO:tasks.workunit.client.0.smithi038.stdout:5/334: write d6/d1e/f63 [1679655,54240] 0 2024-03-24T00:48:48.506 INFO:tasks.workunit.client.0.smithi038.stdout:9/398: dwrite d12/d41/f5b [0,4194304] 0 2024-03-24T00:48:48.509 INFO:tasks.workunit.client.0.smithi038.stdout:8/366: mknod d0/d45/d37/c80 0 2024-03-24T00:48:48.511 INFO:tasks.workunit.client.0.smithi038.stdout:5/335: mkdir d6/d19/d20/d42/d43/d68/d6e 0 2024-03-24T00:48:48.514 INFO:tasks.workunit.client.1.smithi088.stdout:3/354: sync 2024-03-24T00:48:48.524 INFO:tasks.workunit.client.1.smithi088.stdout:2/379: rmdir db/d54 39 2024-03-24T00:48:48.527 INFO:tasks.workunit.client.0.smithi038.stdout:8/367: write d0/d1d/d20/d3c/f47 [40362,36256] 0 2024-03-24T00:48:48.527 INFO:tasks.workunit.client.0.smithi038.stdout:8/368: dread - d0/d15/f54 zero size 2024-03-24T00:48:48.529 INFO:tasks.workunit.client.0.smithi038.stdout:9/399: dread d12/d28/d3a/d26/f3d [0,4194304] 0 2024-03-24T00:48:48.530 INFO:tasks.workunit.client.1.smithi088.stdout:9/379: rename d1/d10/d41 to d1/d10/d29/d6d 0 2024-03-24T00:48:48.531 INFO:tasks.workunit.client.0.smithi038.stdout:5/336: mkdir d6/d19/d20/d33/d6f 0 2024-03-24T00:48:48.532 INFO:tasks.workunit.client.1.smithi088.stdout:3/355: chown d3/d1a/d1e/d2f/d45/c4d 0 1 2024-03-24T00:48:48.534 INFO:tasks.workunit.client.1.smithi088.stdout:6/411: truncate db/d40/d56/f30 3616556 0 2024-03-24T00:48:48.534 INFO:tasks.workunit.client.1.smithi088.stdout:6/412: write db/d1d/d23/d3c/d4b/f6e [1920254,75064] 0 2024-03-24T00:48:48.536 INFO:tasks.workunit.client.0.smithi038.stdout:9/400: write d12/d55/d56/f7c [238727,25827] 0 2024-03-24T00:48:48.536 INFO:tasks.workunit.client.1.smithi088.stdout:5/366: rename d1/d43/d4e/f5c to d1/d29/f77 0 2024-03-24T00:48:48.536 INFO:tasks.workunit.client.1.smithi088.stdout:2/380: dread db/d21/f5f [0,4194304] 0 2024-03-24T00:48:48.537 INFO:tasks.workunit.client.0.smithi038.stdout:5/337: mkdir d6/d19/d20/d70 0 2024-03-24T00:48:48.538 INFO:tasks.workunit.client.1.smithi088.stdout:9/380: unlink d1/d10/d11/d23/l3c 0 2024-03-24T00:48:48.539 INFO:tasks.workunit.client.1.smithi088.stdout:9/381: fsync d1/d22/f43 0 2024-03-24T00:48:48.539 INFO:tasks.workunit.client.1.smithi088.stdout:9/382: fdatasync d1/d10/d11/d23/f53 0 2024-03-24T00:48:48.540 INFO:tasks.workunit.client.0.smithi038.stdout:8/369: mkdir d0/d1d/d27/d4d/d81 0 2024-03-24T00:48:48.542 INFO:tasks.workunit.client.0.smithi038.stdout:9/401: creat d12/d28/f82 x:0 0 0 2024-03-24T00:48:48.542 INFO:tasks.workunit.client.0.smithi038.stdout:9/402: write d12/d28/d3a/f4b [1574268,60070] 0 2024-03-24T00:48:48.545 INFO:tasks.workunit.client.0.smithi038.stdout:5/338: mknod d6/d60/c71 0 2024-03-24T00:48:48.545 INFO:tasks.workunit.client.0.smithi038.stdout:5/339: creat d6/d19/d20/d42/f72 x:0 0 0 2024-03-24T00:48:48.545 INFO:tasks.workunit.client.0.smithi038.stdout:5/340: truncate d6/d19/d20/d42/d43/d4c/f62 634729 0 2024-03-24T00:48:48.547 INFO:tasks.workunit.client.1.smithi088.stdout:5/367: unlink d1/d19/d1e/c32 0 2024-03-24T00:48:48.547 INFO:tasks.workunit.client.1.smithi088.stdout:1/423: rename d6/d27/d1e/d58/f65 to d6/d27/d1e/d5b/d5d/d61/f7e 0 2024-03-24T00:48:48.547 INFO:tasks.workunit.client.1.smithi088.stdout:7/338: sync 2024-03-24T00:48:48.547 INFO:tasks.workunit.client.1.smithi088.stdout:4/376: sync 2024-03-24T00:48:48.547 INFO:tasks.workunit.client.1.smithi088.stdout:4/377: fdatasync d5/d16/d37/f6f 0 2024-03-24T00:48:48.548 INFO:tasks.workunit.client.1.smithi088.stdout:2/381: symlink db/d21/d4a/l7a 0 2024-03-24T00:48:48.548 INFO:tasks.workunit.client.1.smithi088.stdout:2/382: write db/dd/f17 [4629405,19862] 0 2024-03-24T00:48:48.548 INFO:tasks.workunit.client.0.smithi038.stdout:8/370: mknod d0/d45/d66/c82 0 2024-03-24T00:48:48.552 INFO:tasks.workunit.client.1.smithi088.stdout:7/339: chown d3/f30 27 1 2024-03-24T00:48:48.553 INFO:tasks.workunit.client.0.smithi038.stdout:9/403: truncate d12/d41/f7d 1821115 0 2024-03-24T00:48:48.556 INFO:tasks.workunit.client.1.smithi088.stdout:4/378: creat d5/d17/f81 x:0 0 0 2024-03-24T00:48:48.559 INFO:tasks.workunit.client.0.smithi038.stdout:5/341: symlink d6/d19/d20/d42/d43/d50/l73 0 2024-03-24T00:48:48.565 INFO:tasks.workunit.client.1.smithi088.stdout:5/368: dread d1/d10/d1b/d2f/f4a [0,4194304] 0 2024-03-24T00:48:48.565 INFO:tasks.workunit.client.1.smithi088.stdout:8/386: dwrite d9/dc/d11/d24/f2a [0,4194304] 0 2024-03-24T00:48:48.565 INFO:tasks.workunit.client.1.smithi088.stdout:8/387: dread - d9/dc/d11/d24/d29/d5b/f53 zero size 2024-03-24T00:48:48.568 INFO:tasks.workunit.client.1.smithi088.stdout:1/424: rmdir d6/d43 39 2024-03-24T00:48:48.574 INFO:tasks.workunit.client.1.smithi088.stdout:4/379: dread d5/d13/d18/d24/d3d/f55 [0,4194304] 0 2024-03-24T00:48:48.582 INFO:tasks.workunit.client.1.smithi088.stdout:7/340: rename d3/d32/d3f/l46 to d3/d32/d3f/l68 0 2024-03-24T00:48:48.584 INFO:tasks.workunit.client.1.smithi088.stdout:3/356: rename d3/de/df/d49/f4e to d3/d6/f78 0 2024-03-24T00:48:48.584 INFO:tasks.workunit.client.1.smithi088.stdout:5/369: write d1/d10/f17 [1952996,111222] 0 2024-03-24T00:48:48.587 INFO:tasks.workunit.client.0.smithi038.stdout:8/371: rename d0/d1d/c1e to d0/d10/d19/d62/c83 0 2024-03-24T00:48:48.588 INFO:tasks.workunit.client.1.smithi088.stdout:2/383: dwrite db/f46 [0,4194304] 0 2024-03-24T00:48:48.591 INFO:tasks.workunit.client.0.smithi038.stdout:5/342: getdents d6/d1e 0 2024-03-24T00:48:48.594 INFO:tasks.workunit.client.0.smithi038.stdout:8/372: creat d0/d45/d66/f84 x:0 0 0 2024-03-24T00:48:48.594 INFO:tasks.workunit.client.1.smithi088.stdout:6/413: dwrite db/d1d/d23/d3c/d43/f28 [0,4194304] 0 2024-03-24T00:48:48.599 INFO:tasks.workunit.client.0.smithi038.stdout:8/373: rename d0/d45/d66/f84 to d0/d45/d37/d5f/d68/f85 0 2024-03-24T00:48:48.602 INFO:tasks.workunit.client.0.smithi038.stdout:1/353: dwrite f4 [12582912,4194304] 0 2024-03-24T00:48:48.607 INFO:tasks.workunit.client.1.smithi088.stdout:5/370: rename d1/d10/d1b/d2e/d60 to d1/d10/d1b/d2f/d57/d78 0 2024-03-24T00:48:48.607 INFO:tasks.workunit.client.0.smithi038.stdout:5/343: dread d6/d23/f32 [0,4194304] 0 2024-03-24T00:48:48.608 INFO:tasks.workunit.client.0.smithi038.stdout:8/374: symlink d0/d45/d66/d6d/l86 0 2024-03-24T00:48:48.611 INFO:tasks.workunit.client.1.smithi088.stdout:2/384: symlink db/d21/d4a/l7b 0 2024-03-24T00:48:48.611 INFO:tasks.workunit.client.1.smithi088.stdout:2/385: truncate db/d21/f5f 1989934 0 2024-03-24T00:48:48.616 INFO:tasks.workunit.client.0.smithi038.stdout:9/404: truncate d12/d55/d56/f7c 162743 0 2024-03-24T00:48:48.621 INFO:tasks.workunit.client.1.smithi088.stdout:8/388: write d9/dc/f44 [4170882,28547] 0 2024-03-24T00:48:48.622 INFO:tasks.workunit.client.1.smithi088.stdout:9/383: dwrite d1/d15/d21/f3a [0,4194304] 0 2024-03-24T00:48:48.622 INFO:tasks.workunit.client.1.smithi088.stdout:0/388: sync 2024-03-24T00:48:48.622 INFO:tasks.workunit.client.1.smithi088.stdout:0/389: read - d9/dc/f7a zero size 2024-03-24T00:48:48.624 INFO:tasks.workunit.client.0.smithi038.stdout:5/344: dread d6/f3f [4194304,4194304] 0 2024-03-24T00:48:48.624 INFO:tasks.workunit.client.0.smithi038.stdout:5/345: chown d6/f26 613 1 2024-03-24T00:48:48.624 INFO:tasks.workunit.client.0.smithi038.stdout:5/346: creat d6/d19/d20/d42/d43/d4c/f74 x:0 0 0 2024-03-24T00:48:48.626 INFO:tasks.workunit.client.1.smithi088.stdout:7/341: rmdir d3/d32/d3f/d4b 39 2024-03-24T00:48:48.626 INFO:tasks.workunit.client.1.smithi088.stdout:3/357: symlink d3/d67/l79 0 2024-03-24T00:48:48.637 INFO:tasks.workunit.client.1.smithi088.stdout:8/389: truncate d9/f17 1624043 0 2024-03-24T00:48:48.641 INFO:tasks.workunit.client.1.smithi088.stdout:9/384: rename d1/d15/l5b to d1/d10/d11/l6e 0 2024-03-24T00:48:48.641 INFO:tasks.workunit.client.1.smithi088.stdout:9/385: write d1/d10/d11/f5d [379314,46143] 0 2024-03-24T00:48:48.641 INFO:tasks.workunit.client.0.smithi038.stdout:6/399: sync 2024-03-24T00:48:48.641 INFO:tasks.workunit.client.0.smithi038.stdout:7/348: sync 2024-03-24T00:48:48.641 INFO:tasks.workunit.client.0.smithi038.stdout:3/392: sync 2024-03-24T00:48:48.641 INFO:tasks.workunit.client.0.smithi038.stdout:7/349: chown d2/d9/df/d33/f3c 3262113 1 2024-03-24T00:48:48.641 INFO:tasks.workunit.client.0.smithi038.stdout:7/350: fsync d2/f1f 0 2024-03-24T00:48:48.642 INFO:tasks.workunit.client.0.smithi038.stdout:7/351: dread - d2/d9/df/d33/f27 zero size 2024-03-24T00:48:48.642 INFO:tasks.workunit.client.0.smithi038.stdout:2/341: sync 2024-03-24T00:48:48.642 INFO:tasks.workunit.client.0.smithi038.stdout:7/352: write d2/d2c/f62 [527908,51564] 0 2024-03-24T00:48:48.644 INFO:tasks.workunit.client.0.smithi038.stdout:1/354: link da/dd/d4a/f5b da/dd/d2c/d5a/f7b 0 2024-03-24T00:48:48.650 INFO:tasks.workunit.client.1.smithi088.stdout:0/390: mknod d9/dc/de/d19/c7b 0 2024-03-24T00:48:48.655 INFO:tasks.workunit.client.1.smithi088.stdout:7/342: unlink d3/d32/f54 0 2024-03-24T00:48:48.658 INFO:tasks.workunit.client.0.smithi038.stdout:9/405: creat d12/d1a/d80/f83 x:0 0 0 2024-03-24T00:48:48.658 INFO:tasks.workunit.client.0.smithi038.stdout:4/299: sync 2024-03-24T00:48:48.658 INFO:tasks.workunit.client.0.smithi038.stdout:0/301: sync 2024-03-24T00:48:48.663 INFO:tasks.workunit.client.1.smithi088.stdout:5/371: dwrite d1/d10/d1b/f70 [0,4194304] 0 2024-03-24T00:48:48.663 INFO:tasks.workunit.client.1.smithi088.stdout:5/372: fsync d1/d10/d1b/f70 0 2024-03-24T00:48:48.663 INFO:tasks.workunit.client.1.smithi088.stdout:5/373: write d1/d43/d4e/d5b/f63 [154492,74881] 0 2024-03-24T00:48:48.663 INFO:tasks.workunit.client.1.smithi088.stdout:3/358: creat d3/de/d6f/f7a x:0 0 0 2024-03-24T00:48:48.663 INFO:tasks.workunit.client.1.smithi088.stdout:3/359: chown d3/d6/d19/d2a/d5f/f65 3612 1 2024-03-24T00:48:48.664 INFO:tasks.workunit.client.1.smithi088.stdout:3/360: truncate d3/de/d6f/f7a 132129 0 2024-03-24T00:48:48.665 INFO:tasks.workunit.client.0.smithi038.stdout:8/375: dwrite d0/d1d/d27/d4d/f7b [0,4194304] 0 2024-03-24T00:48:48.665 INFO:tasks.workunit.client.0.smithi038.stdout:8/376: chown d0/d1d/d20/c39 2 1 2024-03-24T00:48:48.669 INFO:tasks.workunit.client.0.smithi038.stdout:9/406: fdatasync d12/d28/d3a/f4b 0 2024-03-24T00:48:48.677 INFO:tasks.workunit.client.0.smithi038.stdout:5/347: rename d6/d16/d3c/f56 to d6/d19/d20/d42/d43/d68/f75 0 2024-03-24T00:48:48.677 INFO:tasks.workunit.client.0.smithi038.stdout:3/393: mkdir d3/d9/dc/df/d14/d19/d82/d86 0 2024-03-24T00:48:48.678 INFO:tasks.workunit.client.0.smithi038.stdout:3/394: write d3/d30/f34 [675037,91283] 0 2024-03-24T00:48:48.680 INFO:tasks.workunit.client.1.smithi088.stdout:2/386: dwrite db/dd/f28 [0,4194304] 0 2024-03-24T00:48:48.683 INFO:tasks.workunit.client.0.smithi038.stdout:2/342: mkdir d4/d27/d2f/d30/d3e/d7c 0 2024-03-24T00:48:48.686 INFO:tasks.workunit.client.0.smithi038.stdout:5/348: write d6/d23/f32 [6813,74151] 0 2024-03-24T00:48:48.691 INFO:tasks.workunit.client.0.smithi038.stdout:7/353: creat d2/d9/df/d75/f85 x:0 0 0 2024-03-24T00:48:48.695 INFO:tasks.workunit.client.1.smithi088.stdout:9/386: rename d1/c33 to d1/d22/d27/d39/d62/c6f 0 2024-03-24T00:48:48.695 INFO:tasks.workunit.client.1.smithi088.stdout:9/387: creat d1/d10/d11/d23/f70 x:0 0 0 2024-03-24T00:48:48.696 INFO:tasks.workunit.client.1.smithi088.stdout:9/388: write d1/d10/d11/d23/f70 [757281,61708] 0 2024-03-24T00:48:48.696 INFO:tasks.workunit.client.1.smithi088.stdout:1/425: dwrite d6/d27/d67/f6a [0,4194304] 0 2024-03-24T00:48:48.697 INFO:tasks.workunit.client.1.smithi088.stdout:4/380: dwrite d5/d7/f76 [0,4194304] 0 2024-03-24T00:48:48.697 INFO:tasks.workunit.client.1.smithi088.stdout:4/381: dread - d5/d13/d18/f19 zero size 2024-03-24T00:48:48.698 INFO:tasks.workunit.client.1.smithi088.stdout:0/391: rmdir d9/dc/de/d19/d44/d46/d65/d2a/d2e 39 2024-03-24T00:48:48.699 INFO:tasks.workunit.client.0.smithi038.stdout:0/302: mknod d0/c62 0 2024-03-24T00:48:48.716 INFO:tasks.workunit.client.1.smithi088.stdout:7/343: truncate d3/f2f 139189 0 2024-03-24T00:48:48.727 INFO:tasks.workunit.client.1.smithi088.stdout:6/414: dwrite db/d1d/d50/f77 [0,4194304] 0 2024-03-24T00:48:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:9/407: readlink l7 0 2024-03-24T00:48:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:9/408: chown d12/d28/d3a/l44 685233768 1 2024-03-24T00:48:48.734 INFO:tasks.workunit.client.1.smithi088.stdout:5/374: mkdir d1/d10/d79 0 2024-03-24T00:48:48.737 INFO:tasks.workunit.client.0.smithi038.stdout:6/400: rename d1/da/d1b/d25/d49/d4e/d51/l5a to d1/da/d1b/d25/d49/d72/l7a 0 2024-03-24T00:48:48.737 INFO:tasks.workunit.client.0.smithi038.stdout:6/401: fdatasync d1/f6 0 2024-03-24T00:48:48.737 INFO:tasks.workunit.client.0.smithi038.stdout:6/402: stat d1/c23 0 2024-03-24T00:48:48.737 INFO:tasks.workunit.client.1.smithi088.stdout:2/387: dwrite db/dd/f28 [0,4194304] 0 2024-03-24T00:48:48.737 INFO:tasks.workunit.client.1.smithi088.stdout:2/388: fdatasync db/dc/f2b 0 2024-03-24T00:48:48.740 INFO:tasks.workunit.client.1.smithi088.stdout:3/361: truncate d3/de/d6f/f75 3413118 0 2024-03-24T00:48:48.740 INFO:tasks.workunit.client.1.smithi088.stdout:3/362: chown d3/d6/d19/d2a/l23 87211 1 2024-03-24T00:48:48.740 INFO:tasks.workunit.client.1.smithi088.stdout:3/363: dread - d3/de/f29 zero size 2024-03-24T00:48:48.743 INFO:tasks.workunit.client.0.smithi038.stdout:3/395: symlink d3/d9/dc/l87 0 2024-03-24T00:48:48.747 INFO:tasks.workunit.client.1.smithi088.stdout:8/390: getdents d9/dc/d11/d41/d65 0 2024-03-24T00:48:48.749 INFO:tasks.workunit.client.0.smithi038.stdout:5/349: getdents d6/d19/d20/d42/d43/d68 0 2024-03-24T00:48:48.752 INFO:tasks.workunit.client.0.smithi038.stdout:0/303: mknod d0/d12/d15/c63 0 2024-03-24T00:48:48.754 INFO:tasks.workunit.client.0.smithi038.stdout:9/409: mkdir d12/d81/d84 0 2024-03-24T00:48:48.759 INFO:tasks.workunit.client.1.smithi088.stdout:6/415: fsync db/d40/d56/f31 0 2024-03-24T00:48:48.760 INFO:tasks.workunit.client.1.smithi088.stdout:2/389: dread db/dd/f17 [0,4194304] 0 2024-03-24T00:48:48.760 INFO:tasks.workunit.client.1.smithi088.stdout:2/390: fsync db/d21/f76 0 2024-03-24T00:48:48.763 INFO:tasks.workunit.client.0.smithi038.stdout:6/403: creat d1/da/d1b/d25/d49/f7b x:0 0 0 2024-03-24T00:48:48.763 INFO:tasks.workunit.client.0.smithi038.stdout:6/404: write d1/da/d1b/d25/d49/f7b [901310,49078] 0 2024-03-24T00:48:48.763 INFO:tasks.workunit.client.0.smithi038.stdout:7/354: dwrite d2/d9/da/dc/d4a/f6d [0,4194304] 0 2024-03-24T00:48:48.767 INFO:tasks.workunit.client.1.smithi088.stdout:5/375: dread d1/f3b [0,4194304] 0 2024-03-24T00:48:48.770 INFO:tasks.workunit.client.0.smithi038.stdout:1/355: dwrite da/dd/f1c [4194304,4194304] 0 2024-03-24T00:48:48.771 INFO:tasks.workunit.client.0.smithi038.stdout:4/300: rename da/d12/d29/l46 to da/d38/l5d 0 2024-03-24T00:48:48.773 INFO:tasks.workunit.client.1.smithi088.stdout:1/426: getdents d6/d43 0 2024-03-24T00:48:48.776 INFO:tasks.workunit.client.0.smithi038.stdout:3/396: mknod d3/d9/dc/df/d14/d19/d82/c88 0 2024-03-24T00:48:48.788 INFO:tasks.workunit.client.0.smithi038.stdout:0/304: mkdir d0/d12/d15/d58/d64 0 2024-03-24T00:48:48.797 INFO:tasks.workunit.client.0.smithi038.stdout:4/301: symlink da/d12/l5e 0 2024-03-24T00:48:48.798 INFO:tasks.workunit.client.1.smithi088.stdout:4/382: rename d5/d7/c1f to d5/d16/d1a/c82 0 2024-03-24T00:48:48.801 INFO:tasks.workunit.client.0.smithi038.stdout:1/356: symlink da/dd/d2c/d5a/d74/l7c 0 2024-03-24T00:48:48.803 INFO:tasks.workunit.client.0.smithi038.stdout:5/350: link d6/ce d6/d19/d20/d64/d6b/c76 0 2024-03-24T00:48:48.803 INFO:tasks.workunit.client.0.smithi038.stdout:5/351: write d6/f9 [738821,119249] 0 2024-03-24T00:48:48.803 INFO:tasks.workunit.client.1.smithi088.stdout:0/392: symlink d9/dc/de/d19/d44/d46/d65/d2a/l7c 0 2024-03-24T00:48:48.804 INFO:tasks.workunit.client.1.smithi088.stdout:0/393: creat d9/dc/de/d19/d44/f7d x:0 0 0 2024-03-24T00:48:48.804 INFO:tasks.workunit.client.1.smithi088.stdout:5/376: dread d1/f3b [0,4194304] 0 2024-03-24T00:48:48.804 INFO:tasks.workunit.client.1.smithi088.stdout:7/344: mkdir d3/d32/d3f/d4b/d69 0 2024-03-24T00:48:48.804 INFO:tasks.workunit.client.1.smithi088.stdout:7/345: stat d3/d31/f3c 0 2024-03-24T00:48:48.811 INFO:tasks.workunit.client.0.smithi038.stdout:0/305: creat d0/d2d/f65 x:0 0 0 2024-03-24T00:48:48.811 INFO:tasks.workunit.client.0.smithi038.stdout:8/377: dwrite d0/d1d/f22 [0,4194304] 0 2024-03-24T00:48:48.811 INFO:tasks.workunit.client.0.smithi038.stdout:5/352: mkdir d6/d19/d20/d64/d6b/d77 0 2024-03-24T00:48:48.811 INFO:tasks.workunit.client.0.smithi038.stdout:4/302: truncate da/f3b 3594380 0 2024-03-24T00:48:48.817 INFO:tasks.workunit.client.1.smithi088.stdout:9/389: rmdir d1/d15 39 2024-03-24T00:48:48.817 INFO:tasks.workunit.client.1.smithi088.stdout:1/427: symlink d6/d27/l7f 0 2024-03-24T00:48:48.817 INFO:tasks.workunit.client.1.smithi088.stdout:2/391: creat db/d21/d4a/f7c x:0 0 0 2024-03-24T00:48:48.817 INFO:tasks.workunit.client.0.smithi038.stdout:1/357: rename da/dd/f2f to da/dd/d10/d14/f7d 0 2024-03-24T00:48:48.817 INFO:tasks.workunit.client.0.smithi038.stdout:2/343: dwrite d4/d18/d26/f70 [0,4194304] 0 2024-03-24T00:48:48.821 INFO:tasks.workunit.client.0.smithi038.stdout:9/410: dread d12/d28/f4c [0,4194304] 0 2024-03-24T00:48:48.821 INFO:tasks.workunit.client.0.smithi038.stdout:5/353: link d6/d19/l2f d6/d19/d20/d64/d6b/l78 0 2024-03-24T00:48:48.827 INFO:tasks.workunit.client.1.smithi088.stdout:8/391: rename d9/dc/d11/d41/d65 to d9/dc/d11/d24/d29/d39/d84 0 2024-03-24T00:48:48.830 INFO:tasks.workunit.client.1.smithi088.stdout:3/364: dwrite d3/d6/d27/d3c/f59 [0,4194304] 0 2024-03-24T00:48:48.831 INFO:tasks.workunit.client.1.smithi088.stdout:6/416: dwrite db/d1d/d23/d3c/d43/f34 [4194304,4194304] 0 2024-03-24T00:48:48.832 INFO:tasks.workunit.client.0.smithi038.stdout:0/306: symlink d0/d12/d60/l66 0 2024-03-24T00:48:48.832 INFO:tasks.workunit.client.0.smithi038.stdout:0/307: stat d0/f43 0 2024-03-24T00:48:48.836 INFO:tasks.workunit.client.1.smithi088.stdout:9/390: write d1/d10/f3e [1524577,90345] 0 2024-03-24T00:48:48.839 INFO:tasks.workunit.client.0.smithi038.stdout:6/405: dwrite d1/da/d1b/d2f/f62 [0,4194304] 0 2024-03-24T00:48:48.841 INFO:tasks.workunit.client.1.smithi088.stdout:5/377: unlink d1/d10/d1b/d2e/l55 0 2024-03-24T00:48:48.841 INFO:tasks.workunit.client.1.smithi088.stdout:5/378: chown d1/d19/d1e/d26/d69/c6a 38 1 2024-03-24T00:48:48.841 INFO:tasks.workunit.client.1.smithi088.stdout:5/379: chown d1/dd/l12 11 1 2024-03-24T00:48:48.842 INFO:tasks.workunit.client.1.smithi088.stdout:9/391: fdatasync d1/d10/f3e 0 2024-03-24T00:48:48.843 INFO:tasks.workunit.client.0.smithi038.stdout:2/344: dread d4/d27/f33 [4194304,4194304] 0 2024-03-24T00:48:48.846 INFO:tasks.workunit.client.0.smithi038.stdout:4/303: rmdir da/d12/d37/dd/d15 39 2024-03-24T00:48:48.846 INFO:tasks.workunit.client.0.smithi038.stdout:4/304: stat da/d12/d37/dd/d15/d4c 0 2024-03-24T00:48:48.849 INFO:tasks.workunit.client.0.smithi038.stdout:1/358: read da/dd/d22/f4c [1875117,78605] 0 2024-03-24T00:48:48.852 INFO:tasks.workunit.client.0.smithi038.stdout:1/359: write da/dd/d22/f71 [1526171,53920] 0 2024-03-24T00:48:48.852 INFO:tasks.workunit.client.0.smithi038.stdout:1/360: chown da/d3c/c20 464 1 2024-03-24T00:48:48.856 INFO:tasks.workunit.client.1.smithi088.stdout:7/346: creat d3/d32/f6a x:0 0 0 2024-03-24T00:48:48.859 INFO:tasks.workunit.client.1.smithi088.stdout:9/392: write d1/d22/f49 [4078290,122757] 0 2024-03-24T00:48:48.860 INFO:tasks.workunit.client.0.smithi038.stdout:0/308: dread d0/f5 [0,4194304] 0 2024-03-24T00:48:48.860 INFO:tasks.workunit.client.0.smithi038.stdout:0/309: chown d0/d1b/c30 797166495 1 2024-03-24T00:48:48.861 INFO:tasks.workunit.client.1.smithi088.stdout:1/428: mkdir d6/d27/d1e/d5b/d80 0 2024-03-24T00:48:48.865 INFO:tasks.workunit.client.1.smithi088.stdout:9/393: dread d1/d10/d29/f45 [0,4194304] 0 2024-03-24T00:48:48.868 INFO:tasks.workunit.client.1.smithi088.stdout:5/380: write d1/d43/d4e/d5b/d66/f75 [2885901,90758] 0 2024-03-24T00:48:48.869 INFO:tasks.workunit.client.1.smithi088.stdout:4/383: rename d5/f7c to d5/d17/f83 0 2024-03-24T00:48:48.871 INFO:tasks.workunit.client.0.smithi038.stdout:9/411: symlink d12/d41/d57/l85 0 2024-03-24T00:48:48.872 INFO:tasks.workunit.client.1.smithi088.stdout:8/392: mknod d9/dc/d11/d24/d29/d36/d6a/c85 0 2024-03-24T00:48:48.874 INFO:tasks.workunit.client.0.smithi038.stdout:5/354: mkdir d6/d19/d20/d42/d79 0 2024-03-24T00:48:48.877 INFO:tasks.workunit.client.0.smithi038.stdout:6/406: mknod d1/da/d1b/d25/d49/d4e/c7c 0 2024-03-24T00:48:48.877 INFO:tasks.workunit.client.0.smithi038.stdout:6/407: dread - d1/da/d1b/d25/d49/d4e/f66 zero size 2024-03-24T00:48:48.878 INFO:tasks.workunit.client.1.smithi088.stdout:4/384: dread d5/f9 [0,4194304] 0 2024-03-24T00:48:48.881 INFO:tasks.workunit.client.0.smithi038.stdout:2/345: truncate d4/d27/d54/d67/f71 496770 0 2024-03-24T00:48:48.883 INFO:tasks.workunit.client.1.smithi088.stdout:3/365: creat d3/de/df/d49/f7b x:0 0 0 2024-03-24T00:48:48.885 INFO:tasks.workunit.client.1.smithi088.stdout:2/392: dwrite db/dd/f2f [0,4194304] 0 2024-03-24T00:48:48.885 INFO:tasks.workunit.client.1.smithi088.stdout:2/393: chown ca 2335 1 2024-03-24T00:48:48.885 INFO:tasks.workunit.client.0.smithi038.stdout:7/355: dwrite d2/d2c/f4d [0,4194304] 0 2024-03-24T00:48:48.893 INFO:tasks.workunit.client.1.smithi088.stdout:6/417: creat db/d1d/d23/d5c/f78 x:0 0 0 2024-03-24T00:48:48.893 INFO:tasks.workunit.client.1.smithi088.stdout:6/418: stat f8 0 2024-03-24T00:48:48.896 INFO:tasks.workunit.client.0.smithi038.stdout:4/305: mkdir da/d12/d24/d5f 0 2024-03-24T00:48:48.896 INFO:tasks.workunit.client.0.smithi038.stdout:4/306: chown da/d12/d24 24 1 2024-03-24T00:48:48.896 INFO:tasks.workunit.client.0.smithi038.stdout:4/307: chown da/d12/d29/l39 5 1 2024-03-24T00:48:48.900 INFO:tasks.workunit.client.1.smithi088.stdout:0/394: dwrite d9/dc/d4f/d62/f6a [0,4194304] 0 2024-03-24T00:48:48.901 INFO:tasks.workunit.client.1.smithi088.stdout:1/429: read d6/d27/f1d [4035318,33554] 0 2024-03-24T00:48:48.906 INFO:tasks.workunit.client.0.smithi038.stdout:0/310: creat d0/d12/d42/d5e/f67 x:0 0 0 2024-03-24T00:48:48.906 INFO:tasks.workunit.client.0.smithi038.stdout:0/311: chown d0/d14/c3f 36341 1 2024-03-24T00:48:48.906 INFO:tasks.workunit.client.0.smithi038.stdout:0/312: chown d0/l29 1 1 2024-03-24T00:48:48.906 INFO:tasks.workunit.client.0.smithi038.stdout:0/313: chown d0/d1b/c45 842226561 1 2024-03-24T00:48:48.916 INFO:tasks.workunit.client.1.smithi088.stdout:7/347: mknod d3/d32/d3f/c6b 0 2024-03-24T00:48:48.918 INFO:tasks.workunit.client.0.smithi038.stdout:3/397: dwrite d3/d30/d5d/f5e [0,4194304] 0 2024-03-24T00:48:48.925 INFO:tasks.workunit.client.1.smithi088.stdout:5/381: truncate d1/d19/d1e/d26/f5e 384165 0 2024-03-24T00:48:48.925 INFO:tasks.workunit.client.1.smithi088.stdout:5/382: chown d1/dd 27 1 2024-03-24T00:48:48.926 INFO:tasks.workunit.client.0.smithi038.stdout:9/412: creat d12/d28/d3a/d26/f86 x:0 0 0 2024-03-24T00:48:48.929 INFO:tasks.workunit.client.0.smithi038.stdout:6/408: mknod d1/d37/c7d 0 2024-03-24T00:48:48.937 INFO:tasks.workunit.client.1.smithi088.stdout:9/394: dwrite d1/d10/d11/f17 [0,4194304] 0 2024-03-24T00:48:48.937 INFO:tasks.workunit.client.1.smithi088.stdout:9/395: chown d1/d10/d11/c59 20031177 1 2024-03-24T00:48:48.937 INFO:tasks.workunit.client.1.smithi088.stdout:9/396: dread - d1/d10/d2c/f67 zero size 2024-03-24T00:48:48.938 INFO:tasks.workunit.client.1.smithi088.stdout:8/393: link d9/dc/d11/d24/f2a d9/dc/d11/d24/d29/d36/f86 0 2024-03-24T00:48:48.939 INFO:tasks.workunit.client.1.smithi088.stdout:8/394: write d9/dc/d11/d24/f45 [2541530,104383] 0 2024-03-24T00:48:48.945 INFO:tasks.workunit.client.0.smithi038.stdout:3/398: dread d3/f25 [0,4194304] 0 2024-03-24T00:48:48.945 INFO:tasks.workunit.client.0.smithi038.stdout:3/399: write d3/d30/d70/f7b [199087,115829] 0 2024-03-24T00:48:48.947 INFO:tasks.workunit.client.1.smithi088.stdout:1/430: dread d6/d27/d1e/d3f/f4b [0,4194304] 0 2024-03-24T00:48:48.947 INFO:tasks.workunit.client.1.smithi088.stdout:1/431: fsync d6/d27/d1e/d58/f32 0 2024-03-24T00:48:48.947 INFO:tasks.workunit.client.1.smithi088.stdout:1/432: creat d6/d27/d1e/d58/f81 x:0 0 0 2024-03-24T00:48:48.948 INFO:tasks.workunit.client.1.smithi088.stdout:1/433: read d6/f68 [2952849,39133] 0 2024-03-24T00:48:48.948 INFO:tasks.workunit.client.1.smithi088.stdout:1/434: fdatasync f2 0 2024-03-24T00:48:48.948 INFO:tasks.workunit.client.0.smithi038.stdout:8/378: dwrite d0/f32 [4194304,4194304] 0 2024-03-24T00:48:48.949 INFO:tasks.workunit.client.1.smithi088.stdout:5/383: dread d1/d10/d1b/d2f/f4a [0,4194304] 0 2024-03-24T00:48:48.949 INFO:tasks.workunit.client.1.smithi088.stdout:5/384: creat d1/d10/d1b/d2f/f7a x:0 0 0 2024-03-24T00:48:48.950 INFO:tasks.workunit.client.0.smithi038.stdout:5/355: dwrite d6/d19/d20/d42/d43/d4c/f74 [0,4194304] 0 2024-03-24T00:48:48.951 INFO:tasks.workunit.client.0.smithi038.stdout:1/361: write da/dd/d10/d14/f47 [16029160,18423] 0 2024-03-24T00:48:48.958 INFO:tasks.workunit.client.0.smithi038.stdout:4/308: dwrite da/f1d [0,4194304] 0 2024-03-24T00:48:48.960 INFO:tasks.workunit.client.0.smithi038.stdout:4/309: write da/d12/f57 [166468,46859] 0 2024-03-24T00:48:48.960 INFO:tasks.workunit.client.0.smithi038.stdout:4/310: fdatasync da/d12/d37/dd/f3a 0 2024-03-24T00:48:48.960 INFO:tasks.workunit.client.0.smithi038.stdout:4/311: write da/d12/d37/dd/d15/d4c/f53 [576601,48281] 0 2024-03-24T00:48:48.960 INFO:tasks.workunit.client.0.smithi038.stdout:4/312: chown da/d12/d37/dd/d15/f17 1961 1 2024-03-24T00:48:48.962 INFO:tasks.workunit.client.0.smithi038.stdout:1/362: dread da/dd/d10/d14/f7d [0,4194304] 0 2024-03-24T00:48:48.965 INFO:tasks.workunit.client.1.smithi088.stdout:1/435: fsync d6/f68 0 2024-03-24T00:48:48.966 INFO:tasks.workunit.client.0.smithi038.stdout:7/356: link d2/ce d2/d2c/c86 0 2024-03-24T00:48:48.970 INFO:tasks.workunit.client.1.smithi088.stdout:6/419: dwrite db/dd/f37 [0,4194304] 0 2024-03-24T00:48:48.970 INFO:tasks.workunit.client.1.smithi088.stdout:6/420: write db/dd/f2b [1722198,98691] 0 2024-03-24T00:48:48.972 INFO:tasks.workunit.client.1.smithi088.stdout:2/394: unlink db/dc/f1b 0 2024-03-24T00:48:48.972 INFO:tasks.workunit.client.1.smithi088.stdout:2/395: creat db/d21/d35/f7d x:0 0 0 2024-03-24T00:48:48.976 INFO:tasks.workunit.client.1.smithi088.stdout:3/366: dwrite d3/d6/d27/d3c/f73 [0,4194304] 0 2024-03-24T00:48:48.982 INFO:tasks.workunit.client.0.smithi038.stdout:0/314: creat d0/d12/f68 x:0 0 0 2024-03-24T00:48:48.989 INFO:tasks.workunit.client.0.smithi038.stdout:5/356: read d6/d19/d20/d42/d43/d68/f75 [1775719,20274] 0 2024-03-24T00:48:48.997 INFO:tasks.workunit.client.0.smithi038.stdout:3/400: dread d3/d30/f34 [0,4194304] 0 2024-03-24T00:48:48.997 INFO:tasks.workunit.client.0.smithi038.stdout:3/401: chown d3/d5a/f79 0 1 2024-03-24T00:48:48.998 INFO:tasks.workunit.client.1.smithi088.stdout:0/395: mkdir d9/dc/de/d7e 0 2024-03-24T00:48:48.999 INFO:tasks.workunit.client.1.smithi088.stdout:5/385: dwrite d1/d19/f6c [0,4194304] 0 2024-03-24T00:48:48.999 INFO:tasks.workunit.client.1.smithi088.stdout:6/421: dread db/d40/d56/f31 [0,4194304] 0 2024-03-24T00:48:48.999 INFO:tasks.workunit.client.1.smithi088.stdout:6/422: chown db/dd 59103 1 2024-03-24T00:48:49.000 INFO:tasks.workunit.client.0.smithi038.stdout:9/413: rmdir d12 39 2024-03-24T00:48:49.000 INFO:tasks.workunit.client.0.smithi038.stdout:3/402: read d3/d9/dc/df/d14/d19/f46 [304628,129168] 0 2024-03-24T00:48:49.000 INFO:tasks.workunit.client.0.smithi038.stdout:3/403: fsync d3/f3d 0 2024-03-24T00:48:49.000 INFO:tasks.workunit.client.0.smithi038.stdout:3/404: stat d3/d2a/f59 0 2024-03-24T00:48:49.000 INFO:tasks.workunit.client.0.smithi038.stdout:3/405: write d3/f6 [3803523,126793] 0 2024-03-24T00:48:49.000 INFO:tasks.workunit.client.0.smithi038.stdout:3/406: fsync d3/d9/dc/df/f1d 0 2024-03-24T00:48:49.005 INFO:tasks.workunit.client.0.smithi038.stdout:6/409: mknod d1/d37/d68/c7e 0 2024-03-24T00:48:49.005 INFO:tasks.workunit.client.0.smithi038.stdout:6/410: creat d1/da/d1b/d2f/f7f x:0 0 0 2024-03-24T00:48:49.006 INFO:tasks.workunit.client.1.smithi088.stdout:9/397: rmdir d1/d15/d47 39 2024-03-24T00:48:49.008 INFO:tasks.workunit.client.0.smithi038.stdout:2/346: getdents d4/d27/d2f 0 2024-03-24T00:48:49.011 INFO:tasks.workunit.client.1.smithi088.stdout:8/395: symlink d9/dc/d11/d24/d29/l87 0 2024-03-24T00:48:49.013 INFO:tasks.workunit.client.1.smithi088.stdout:6/423: write db/f13 [7177933,7573] 0 2024-03-24T00:48:49.013 INFO:tasks.workunit.client.1.smithi088.stdout:6/424: write db/d40/f4d [1220065,51595] 0 2024-03-24T00:48:49.016 INFO:tasks.workunit.client.0.smithi038.stdout:8/379: creat d0/d45/d37/f87 x:0 0 0 2024-03-24T00:48:49.019 INFO:tasks.workunit.client.1.smithi088.stdout:4/385: mknod d5/d7/dd/c84 0 2024-03-24T00:48:49.023 INFO:tasks.workunit.client.1.smithi088.stdout:0/396: dread d9/dc/de/d19/d44/d46/d65/f39 [0,4194304] 0 2024-03-24T00:48:49.029 INFO:tasks.workunit.client.0.smithi038.stdout:1/363: creat da/f7e x:0 0 0 2024-03-24T00:48:49.029 INFO:tasks.workunit.client.1.smithi088.stdout:1/436: symlink d6/d27/d1e/d5b/l82 0 2024-03-24T00:48:49.033 INFO:tasks.workunit.client.0.smithi038.stdout:7/357: symlink d2/d9/da/d23/l87 0 2024-03-24T00:48:49.038 INFO:tasks.workunit.client.1.smithi088.stdout:1/437: dread d6/d27/d1e/f22 [0,4194304] 0 2024-03-24T00:48:49.044 INFO:tasks.workunit.client.1.smithi088.stdout:2/396: mknod db/d54/c7e 0 2024-03-24T00:48:49.056 INFO:tasks.workunit.client.1.smithi088.stdout:3/367: link d3/d6/f78 d3/d1a/d1e/d76/f7c 0 2024-03-24T00:48:49.061 INFO:tasks.workunit.client.0.smithi038.stdout:0/315: creat d0/f69 x:0 0 0 2024-03-24T00:48:49.062 INFO:tasks.workunit.client.0.smithi038.stdout:5/357: creat d6/d19/d20/d33/d6f/f7a x:0 0 0 2024-03-24T00:48:49.063 INFO:tasks.workunit.client.1.smithi088.stdout:5/386: mknod d1/d10/d4f/c7b 0 2024-03-24T00:48:49.073 INFO:tasks.workunit.client.0.smithi038.stdout:6/411: symlink d1/da/d73/d75/l80 0 2024-03-24T00:48:49.073 INFO:tasks.workunit.client.0.smithi038.stdout:6/412: stat d1/da/d1b/d25/c46 0 2024-03-24T00:48:49.073 INFO:tasks.workunit.client.0.smithi038.stdout:6/413: dread - d1/da/d1b/d2f/f3d zero size 2024-03-24T00:48:49.075 INFO:tasks.workunit.client.0.smithi038.stdout:2/347: mknod d4/d18/d38/d63/c7d 0 2024-03-24T00:48:49.076 INFO:tasks.workunit.client.1.smithi088.stdout:7/348: rmdir d3 39 2024-03-24T00:48:49.077 INFO:tasks.workunit.client.0.smithi038.stdout:8/380: creat d0/d15/d2c/f88 x:0 0 0 2024-03-24T00:48:49.079 INFO:tasks.workunit.client.1.smithi088.stdout:9/398: creat d1/d10/d11/d28/f71 x:0 0 0 2024-03-24T00:48:49.079 INFO:tasks.workunit.client.1.smithi088.stdout:9/399: fsync d1/d10/d11/d23/f70 0 2024-03-24T00:48:49.081 INFO:tasks.workunit.client.0.smithi038.stdout:1/364: creat da/dd/d22/d5c/f7f x:0 0 0 2024-03-24T00:48:49.081 INFO:tasks.workunit.client.0.smithi038.stdout:1/365: write da/dd/d4a/f6b [484596,95330] 0 2024-03-24T00:48:49.081 INFO:tasks.workunit.client.0.smithi038.stdout:1/366: write da/dd/d22/f55 [175030,75317] 0 2024-03-24T00:48:49.083 INFO:tasks.workunit.client.1.smithi088.stdout:8/396: rename d9/dc/d11/d24/d29/d39/f48 to d9/dc/d11/d24/d29/d36/f88 0 2024-03-24T00:48:49.083 INFO:tasks.workunit.client.1.smithi088.stdout:8/397: truncate d9/dc/d11/d24/f71 21459 0 2024-03-24T00:48:49.084 INFO:tasks.workunit.client.0.smithi038.stdout:7/358: symlink d2/d9/l88 0 2024-03-24T00:48:49.084 INFO:tasks.workunit.client.1.smithi088.stdout:4/386: rmdir d5/d13/d18/d6e 39 2024-03-24T00:48:49.085 INFO:tasks.workunit.client.1.smithi088.stdout:4/387: chown d5/d7/dd/f75 0 1 2024-03-24T00:48:49.085 INFO:tasks.workunit.client.1.smithi088.stdout:4/388: chown d5/d16/d37/f48 116927241 1 2024-03-24T00:48:49.086 INFO:tasks.workunit.client.1.smithi088.stdout:0/397: creat d9/dc/de/d19/f7f x:0 0 0 2024-03-24T00:48:49.091 INFO:tasks.workunit.client.1.smithi088.stdout:9/400: dread d1/d10/f3e [0,4194304] 0 2024-03-24T00:48:49.097 INFO:tasks.workunit.client.0.smithi038.stdout:5/358: mknod d6/d19/d20/d42/d43/d4c/c7b 0 2024-03-24T00:48:49.098 INFO:tasks.workunit.client.1.smithi088.stdout:8/398: dread d9/f28 [0,4194304] 0 2024-03-24T00:48:49.099 INFO:tasks.workunit.client.0.smithi038.stdout:6/414: creat d1/da/d1b/d25/f81 x:0 0 0 2024-03-24T00:48:49.103 INFO:tasks.workunit.client.0.smithi038.stdout:2/348: rename d4/d18/d26/d37/f40 to d4/d18/d38/d63/f7e 0 2024-03-24T00:48:49.104 INFO:tasks.workunit.client.0.smithi038.stdout:1/367: mknod da/d3c/c80 0 2024-03-24T00:48:49.104 INFO:tasks.workunit.client.0.smithi038.stdout:1/368: chown da/dd/d2c/d41/f65 15 1 2024-03-24T00:48:49.107 INFO:tasks.workunit.client.1.smithi088.stdout:1/438: symlink d6/l83 0 2024-03-24T00:48:49.108 INFO:tasks.workunit.client.0.smithi038.stdout:2/349: truncate d4/d18/d26/f2c 1116558 0 2024-03-24T00:48:49.113 INFO:tasks.workunit.client.0.smithi038.stdout:1/369: rename da/dd/d10/d14/f7d to da/d3c/f81 0 2024-03-24T00:48:49.120 INFO:tasks.workunit.client.1.smithi088.stdout:3/368: symlink d3/d1a/d1e/d2f/d45/l7d 0 2024-03-24T00:48:49.120 INFO:tasks.workunit.client.1.smithi088.stdout:3/369: fsync d3/d1a/f5a 0 2024-03-24T00:48:49.121 INFO:tasks.workunit.client.0.smithi038.stdout:5/359: dread d6/d23/f3b [0,4194304] 0 2024-03-24T00:48:49.121 INFO:tasks.workunit.client.1.smithi088.stdout:5/387: mknod d1/d10/d4f/c7c 0 2024-03-24T00:48:49.122 INFO:tasks.workunit.client.1.smithi088.stdout:1/439: dread d6/f3e [0,4194304] 0 2024-03-24T00:48:49.122 INFO:tasks.workunit.client.1.smithi088.stdout:1/440: write d6/d27/d1e/d5b/d5d/d61/f7e [230958,87016] 0 2024-03-24T00:48:49.122 INFO:tasks.workunit.client.1.smithi088.stdout:1/441: write d6/d27/d67/f6b [419049,87726] 0 2024-03-24T00:48:49.125 INFO:tasks.workunit.client.1.smithi088.stdout:7/349: rename d3/f45 to d3/d32/d3f/d4b/f6c 0 2024-03-24T00:48:49.125 INFO:tasks.workunit.client.1.smithi088.stdout:0/398: creat d9/f80 x:0 0 0 2024-03-24T00:48:49.125 INFO:tasks.workunit.client.1.smithi088.stdout:7/350: dread - d3/d32/d3f/d4b/f4c zero size 2024-03-24T00:48:49.126 INFO:tasks.workunit.client.0.smithi038.stdout:5/360: mknod d6/d19/d20/d42/c7c 0 2024-03-24T00:48:49.126 INFO:tasks.workunit.client.0.smithi038.stdout:5/361: write d6/d19/d20/d42/f72 [73455,57046] 0 2024-03-24T00:48:49.128 INFO:tasks.workunit.client.1.smithi088.stdout:5/388: mknod d1/d29/d3f/c7d 0 2024-03-24T00:48:49.128 INFO:tasks.workunit.client.1.smithi088.stdout:9/401: mkdir d1/d22/d72 0 2024-03-24T00:48:49.128 INFO:tasks.workunit.client.1.smithi088.stdout:5/389: write d1/d10/d1b/d2f/f7a [151651,112069] 0 2024-03-24T00:48:49.128 INFO:tasks.workunit.client.1.smithi088.stdout:5/390: dread - d1/d29/f50 zero size 2024-03-24T00:48:49.129 INFO:tasks.workunit.client.0.smithi038.stdout:5/362: link d6/d19/d20/d42/c6a d6/d19/d20/d42/d43/d68/c7d 0 2024-03-24T00:48:49.132 INFO:tasks.workunit.client.0.smithi038.stdout:5/363: write d6/d1e/f55 [440343,28001] 0 2024-03-24T00:48:49.137 INFO:tasks.workunit.client.1.smithi088.stdout:0/399: mknod d9/dc/de/d19/c81 0 2024-03-24T00:48:49.138 INFO:tasks.workunit.client.1.smithi088.stdout:8/399: dwrite d9/dc/d11/d24/f71 [0,4194304] 0 2024-03-24T00:48:49.140 INFO:tasks.workunit.client.1.smithi088.stdout:0/400: symlink d9/dc/de/d19/l82 0 2024-03-24T00:48:49.140 INFO:tasks.workunit.client.1.smithi088.stdout:0/401: stat d9/dc/de 0 2024-03-24T00:48:49.142 INFO:tasks.workunit.client.1.smithi088.stdout:8/400: mkdir d9/dc/d11/d41/d54/d89 0 2024-03-24T00:48:49.148 INFO:tasks.workunit.client.1.smithi088.stdout:8/401: unlink d9/dc/d11/d41/d54/d74/c79 0 2024-03-24T00:48:49.149 INFO:tasks.workunit.client.1.smithi088.stdout:3/370: dread d3/de/df/f25 [4194304,4194304] 0 2024-03-24T00:48:49.149 INFO:tasks.workunit.client.1.smithi088.stdout:3/371: write f2 [9410188,51980] 0 2024-03-24T00:48:49.149 INFO:tasks.workunit.client.1.smithi088.stdout:3/372: stat d3/d6/d58/f63 0 2024-03-24T00:48:49.150 INFO:tasks.workunit.client.1.smithi088.stdout:3/373: write d3/d6/d27/d3c/f30 [506654,615] 0 2024-03-24T00:48:49.150 INFO:tasks.workunit.client.1.smithi088.stdout:3/374: fdatasync d3/d6/d19/d2a/f53 0 2024-03-24T00:48:49.150 INFO:tasks.workunit.client.1.smithi088.stdout:3/375: chown d3/de/df/d49/f54 3707 1 2024-03-24T00:48:49.150 INFO:tasks.workunit.client.1.smithi088.stdout:3/376: dread - d3/d6/d19/f77 zero size 2024-03-24T00:48:49.150 INFO:tasks.workunit.client.1.smithi088.stdout:8/402: mknod d9/dc/d11/d24/d29/d36/c8a 0 2024-03-24T00:48:49.150 INFO:tasks.workunit.client.1.smithi088.stdout:8/403: readlink d9/dc/d11/d41/d54/d5d/l77 0 2024-03-24T00:48:49.151 INFO:tasks.workunit.client.1.smithi088.stdout:8/404: mkdir d9/dc/d11/d24/d29/d5b/d8b 0 2024-03-24T00:48:49.151 INFO:tasks.workunit.client.1.smithi088.stdout:1/442: write f0 [1295515,47375] 0 2024-03-24T00:48:49.158 INFO:tasks.workunit.client.1.smithi088.stdout:9/402: dread d1/d22/f49 [4194304,4194304] 0 2024-03-24T00:48:49.164 INFO:tasks.workunit.client.1.smithi088.stdout:1/443: mknod d6/d43/c84 0 2024-03-24T00:48:49.164 INFO:tasks.workunit.client.1.smithi088.stdout:1/444: write d6/d27/d1e/d5b/d5e/f66 [1627184,65368] 0 2024-03-24T00:48:49.168 INFO:tasks.workunit.client.1.smithi088.stdout:9/403: symlink d1/d22/d72/l73 0 2024-03-24T00:48:49.172 INFO:tasks.workunit.client.1.smithi088.stdout:1/445: mknod d6/c85 0 2024-03-24T00:48:49.173 INFO:tasks.workunit.client.1.smithi088.stdout:5/391: dread d1/d19/f6c [0,4194304] 0 2024-03-24T00:48:49.173 INFO:tasks.workunit.client.1.smithi088.stdout:5/392: readlink d1/d10/d1b/d2f/l67 0 2024-03-24T00:48:49.174 INFO:tasks.workunit.client.1.smithi088.stdout:5/393: creat d1/d10/d1b/d2f/f7e x:0 0 0 2024-03-24T00:48:49.177 INFO:tasks.workunit.client.0.smithi038.stdout:3/407: dwrite d3/f49 [0,4194304] 0 2024-03-24T00:48:49.177 INFO:tasks.workunit.client.0.smithi038.stdout:3/408: stat d3/d9/dc/df/d14/f54 0 2024-03-24T00:48:49.178 INFO:tasks.workunit.client.1.smithi088.stdout:9/404: mknod d1/d22/c74 0 2024-03-24T00:48:49.178 INFO:tasks.workunit.client.1.smithi088.stdout:9/405: chown d1/le 3391 1 2024-03-24T00:48:49.181 INFO:tasks.workunit.client.1.smithi088.stdout:9/406: creat d1/d10/d2c/f75 x:0 0 0 2024-03-24T00:48:49.184 INFO:tasks.workunit.client.1.smithi088.stdout:1/446: rename d6/d27/c63 to d6/c86 0 2024-03-24T00:48:49.201 INFO:tasks.workunit.client.1.smithi088.stdout:3/377: write d3/d6/d58/f63 [640767,105097] 0 2024-03-24T00:48:49.202 INFO:tasks.workunit.client.0.smithi038.stdout:1/370: dwrite da/dd/d2c/d40/f53 [0,4194304] 0 2024-03-24T00:48:49.202 INFO:tasks.workunit.client.1.smithi088.stdout:5/394: read d1/d19/d1e/d26/f5e [381756,58478] 0 2024-03-24T00:48:49.209 INFO:tasks.workunit.client.0.smithi038.stdout:9/414: dread d12/d28/d6c/f74 [0,4194304] 0 2024-03-24T00:48:49.211 INFO:tasks.workunit.client.1.smithi088.stdout:9/407: mknod d1/d22/d27/d39/c76 0 2024-03-24T00:48:49.212 INFO:tasks.workunit.client.1.smithi088.stdout:9/408: read - d1/d15/d21/f66 zero size 2024-03-24T00:48:49.213 INFO:tasks.workunit.client.0.smithi038.stdout:9/415: symlink d12/d1a/d80/l87 0 2024-03-24T00:48:49.218 INFO:tasks.workunit.client.1.smithi088.stdout:1/447: mknod d6/d27/d1e/d5b/c87 0 2024-03-24T00:48:49.218 INFO:tasks.workunit.client.1.smithi088.stdout:1/448: stat d6/d43/l5f 0 2024-03-24T00:48:49.220 INFO:tasks.workunit.client.0.smithi038.stdout:9/416: mkdir d12/d81/d84/d88 0 2024-03-24T00:48:49.220 INFO:tasks.workunit.client.0.smithi038.stdout:9/417: getdents d12/d28/d6c/d75 0 2024-03-24T00:48:49.222 INFO:tasks.workunit.client.1.smithi088.stdout:5/395: symlink d1/d43/d4e/d5b/l7f 0 2024-03-24T00:48:49.222 INFO:tasks.workunit.client.1.smithi088.stdout:5/396: dread - d1/d10/d1b/d2f/d57/f71 zero size 2024-03-24T00:48:49.224 INFO:tasks.workunit.client.1.smithi088.stdout:9/409: mkdir d1/d10/d77 0 2024-03-24T00:48:49.225 INFO:tasks.workunit.client.0.smithi038.stdout:1/371: truncate da/dd/f1c 8371948 0 2024-03-24T00:48:49.229 INFO:tasks.workunit.client.0.smithi038.stdout:9/418: rename f8 to d12/d41/d57/f89 0 2024-03-24T00:48:49.229 INFO:tasks.workunit.client.0.smithi038.stdout:9/419: chown d12/d1a/l38 2775487 1 2024-03-24T00:48:49.232 INFO:tasks.workunit.client.1.smithi088.stdout:1/449: mkdir d6/d27/d67/d88 0 2024-03-24T00:48:49.232 INFO:tasks.workunit.client.1.smithi088.stdout:1/450: dread - d6/d27/d1e/d58/f7c zero size 2024-03-24T00:48:49.232 INFO:tasks.workunit.client.1.smithi088.stdout:5/397: unlink d1/d43/d4e/d5b/d66/f75 0 2024-03-24T00:48:49.232 INFO:tasks.workunit.client.1.smithi088.stdout:5/398: fdatasync d1/d19/f30 0 2024-03-24T00:48:49.240 INFO:tasks.workunit.client.0.smithi038.stdout:9/420: getdents d12/d55 0 2024-03-24T00:48:49.244 INFO:tasks.workunit.client.1.smithi088.stdout:5/399: creat d1/d10/d1b/d2e/d46/f80 x:0 0 0 2024-03-24T00:48:49.244 INFO:tasks.workunit.client.1.smithi088.stdout:5/400: fdatasync d1/d10/d1b/d2f/f38 0 2024-03-24T00:48:49.244 INFO:tasks.workunit.client.1.smithi088.stdout:7/351: dwrite d3/f2c [0,4194304] 0 2024-03-24T00:48:49.246 INFO:tasks.workunit.client.0.smithi038.stdout:0/316: dwrite d0/d12/d42/f51 [0,4194304] 0 2024-03-24T00:48:49.246 INFO:tasks.workunit.client.0.smithi038.stdout:0/317: readlink d0/d12/d42/d44/l54 0 2024-03-24T00:48:49.246 INFO:tasks.workunit.client.0.smithi038.stdout:0/318: chown d0/d12/c19 376987051 1 2024-03-24T00:48:49.246 INFO:tasks.workunit.client.1.smithi088.stdout:8/405: dwrite d9/dc/d11/d24/d29/d5b/f53 [0,4194304] 0 2024-03-24T00:48:49.253 INFO:tasks.workunit.client.1.smithi088.stdout:7/352: dread d3/d32/d34/f3b [0,4194304] 0 2024-03-24T00:48:49.254 INFO:tasks.workunit.client.1.smithi088.stdout:5/401: mkdir d1/d10/d81 0 2024-03-24T00:48:49.254 INFO:tasks.workunit.client.1.smithi088.stdout:5/402: truncate d1/fa 8971695 0 2024-03-24T00:48:49.254 INFO:tasks.workunit.client.1.smithi088.stdout:5/403: stat d1/d43/d4e/d5b/f63 0 2024-03-24T00:48:49.263 INFO:tasks.workunit.client.0.smithi038.stdout:0/319: dread d0/d2d/f2f [0,4194304] 0 2024-03-24T00:48:49.267 INFO:tasks.workunit.client.0.smithi038.stdout:5/364: dwrite d6/d19/d20/d42/d43/d4c/f74 [4194304,4194304] 0 2024-03-24T00:48:49.277 INFO:tasks.workunit.client.0.smithi038.stdout:8/381: dwrite d0/d45/f55 [0,4194304] 0 2024-03-24T00:48:49.277 INFO:tasks.workunit.client.0.smithi038.stdout:2/350: dread d4/f23 [0,4194304] 0 2024-03-24T00:48:49.279 INFO:tasks.workunit.client.1.smithi088.stdout:0/402: dwrite d9/dc/de/d19/d44/f4a [4194304,4194304] 0 2024-03-24T00:48:49.280 INFO:tasks.workunit.client.0.smithi038.stdout:7/359: dwrite d2/d9/df/d33/d37/f40 [0,4194304] 0 2024-03-24T00:48:49.280 INFO:tasks.workunit.client.0.smithi038.stdout:7/360: fdatasync d2/d2c/f74 0 2024-03-24T00:48:49.281 INFO:tasks.workunit.client.0.smithi038.stdout:0/320: dread d0/d2d/f2f [0,4194304] 0 2024-03-24T00:48:49.281 INFO:tasks.workunit.client.0.smithi038.stdout:0/321: chown d0/d12/d15/l4b 975314 1 2024-03-24T00:48:49.288 INFO:tasks.workunit.client.1.smithi088.stdout:0/403: dread d9/dc/de/f14 [0,4194304] 0 2024-03-24T00:48:49.291 INFO:tasks.workunit.client.1.smithi088.stdout:3/378: dwrite d3/d6/d27/f40 [0,4194304] 0 2024-03-24T00:48:49.295 INFO:tasks.workunit.client.0.smithi038.stdout:6/415: dwrite d1/f1c [0,4194304] 0 2024-03-24T00:48:49.298 INFO:tasks.workunit.client.1.smithi088.stdout:5/404: creat d1/d19/d1e/d26/d69/f82 x:0 0 0 2024-03-24T00:48:49.307 INFO:tasks.workunit.client.0.smithi038.stdout:5/365: truncate d6/d23/f32 734769 0 2024-03-24T00:48:49.312 INFO:tasks.workunit.client.1.smithi088.stdout:3/379: write d3/d6/d27/d3c/f42 [553193,49237] 0 2024-03-24T00:48:49.312 INFO:tasks.workunit.client.0.smithi038.stdout:9/421: rmdir d12/d28/d6c/d75 0 2024-03-24T00:48:49.313 INFO:tasks.workunit.client.0.smithi038.stdout:8/382: getdents d0/d45 0 2024-03-24T00:48:49.313 INFO:tasks.workunit.client.0.smithi038.stdout:8/383: dread - d0/d15/d2c/f88 zero size 2024-03-24T00:48:49.319 INFO:tasks.workunit.client.0.smithi038.stdout:7/361: dread d2/d9/df/f35 [0,4194304] 0 2024-03-24T00:48:49.330 INFO:tasks.workunit.client.1.smithi088.stdout:5/405: symlink d1/d19/d1e/d26/l83 0 2024-03-24T00:48:49.330 INFO:tasks.workunit.client.1.smithi088.stdout:5/406: chown d1/d43/d4e/d5b/l7f 635449 1 2024-03-24T00:48:49.330 INFO:tasks.workunit.client.1.smithi088.stdout:5/407: chown d1/d10/c11 31204992 1 2024-03-24T00:48:49.330 INFO:tasks.workunit.client.1.smithi088.stdout:5/408: truncate d1/fb 1637688 0 2024-03-24T00:48:49.330 INFO:tasks.workunit.client.1.smithi088.stdout:8/406: dwrite d9/dc/d11/d24/d43/f50 [0,4194304] 0 2024-03-24T00:48:49.330 INFO:tasks.workunit.client.1.smithi088.stdout:8/407: fsync d9/dc/d11/d24/f45 0 2024-03-24T00:48:49.331 INFO:tasks.workunit.client.0.smithi038.stdout:6/416: dread d1/da/d1b/f26 [0,4194304] 0 2024-03-24T00:48:49.331 INFO:tasks.workunit.client.0.smithi038.stdout:6/417: creat d1/da/f82 x:0 0 0 2024-03-24T00:48:49.334 INFO:tasks.workunit.client.0.smithi038.stdout:5/366: dread d6/d19/f1c [4194304,4194304] 0 2024-03-24T00:48:49.343 INFO:tasks.workunit.client.0.smithi038.stdout:9/422: mknod d12/d81/d84/d88/c8a 0 2024-03-24T00:48:49.351 INFO:tasks.workunit.client.0.smithi038.stdout:8/384: read d0/d1d/d27/d4d/f7b [540626,24230] 0 2024-03-24T00:48:49.352 INFO:tasks.workunit.client.0.smithi038.stdout:7/362: rename d2/d9/df/c82 to d2/d9/da/d23/d2b/d5b/c89 0 2024-03-24T00:48:49.354 INFO:tasks.workunit.client.1.smithi088.stdout:3/380: rmdir d3/d6/d58 39 2024-03-24T00:48:49.355 INFO:tasks.workunit.client.1.smithi088.stdout:0/404: dwrite d9/dc/de/d19/d44/f6e [0,4194304] 0 2024-03-24T00:48:49.355 INFO:tasks.workunit.client.1.smithi088.stdout:0/405: readlink d9/dc/d4f/d62/l73 0 2024-03-24T00:48:49.355 INFO:tasks.workunit.client.1.smithi088.stdout:0/406: stat d9/f45 0 2024-03-24T00:48:49.355 INFO:tasks.workunit.client.1.smithi088.stdout:5/409: dread d1/ff [4194304,4194304] 0 2024-03-24T00:48:49.371 INFO:tasks.workunit.client.0.smithi038.stdout:9/423: getdents d12/d81 0 2024-03-24T00:48:49.371 INFO:tasks.workunit.client.0.smithi038.stdout:9/424: readlink l7 0 2024-03-24T00:48:49.376 INFO:tasks.workunit.client.0.smithi038.stdout:7/363: mkdir d2/d9/da/dc/d8a 0 2024-03-24T00:48:49.377 INFO:tasks.workunit.client.1.smithi088.stdout:3/381: dread d3/d6/d27/d3c/f59 [0,4194304] 0 2024-03-24T00:48:49.384 INFO:tasks.workunit.client.0.smithi038.stdout:8/385: dread d0/d1d/d20/f58 [4194304,4194304] 0 2024-03-24T00:48:49.384 INFO:tasks.workunit.client.0.smithi038.stdout:9/425: creat d12/d28/d71/f8b x:0 0 0 2024-03-24T00:48:49.385 INFO:tasks.workunit.client.1.smithi088.stdout:8/408: dwrite d9/dc/f16 [0,4194304] 0 2024-03-24T00:48:49.385 INFO:tasks.workunit.client.1.smithi088.stdout:8/409: write d9/dc/d11/d41/d56/f59 [513312,50391] 0 2024-03-24T00:48:49.388 INFO:tasks.workunit.client.0.smithi038.stdout:1/372: dwrite da/dd/d2c/d5a/d74/f67 [0,4194304] 0 2024-03-24T00:48:49.388 INFO:tasks.workunit.client.0.smithi038.stdout:1/373: stat da/dd/d2c/d5a/d74/f64 0 2024-03-24T00:48:49.391 INFO:tasks.workunit.client.0.smithi038.stdout:7/364: rename d2/d9/d16/c24 to d2/d9/da/d23/d2b/c8b 0 2024-03-24T00:48:49.391 INFO:tasks.workunit.client.0.smithi038.stdout:7/365: dread - d2/d9/df/d75/f85 zero size 2024-03-24T00:48:49.391 INFO:tasks.workunit.client.0.smithi038.stdout:7/366: write d2/d2c/f62 [131545,34868] 0 2024-03-24T00:48:49.392 INFO:tasks.workunit.client.1.smithi088.stdout:0/407: rmdir d9/dc/d4f 39 2024-03-24T00:48:49.398 INFO:tasks.workunit.client.0.smithi038.stdout:9/426: link f4 d12/d1a/f8c 0 2024-03-24T00:48:49.399 INFO:tasks.workunit.client.1.smithi088.stdout:9/410: dwrite d1/d10/d11/d28/f4f [0,4194304] 0 2024-03-24T00:48:49.403 INFO:tasks.workunit.client.1.smithi088.stdout:3/382: rename d3/d6/d27/d3c/f73 to d3/de/df/d31/f7e 0 2024-03-24T00:48:49.403 INFO:tasks.workunit.client.1.smithi088.stdout:3/383: fdatasync d3/d6/f11 0 2024-03-24T00:48:49.405 INFO:tasks.workunit.client.1.smithi088.stdout:6/425: sync 2024-03-24T00:48:49.408 INFO:tasks.workunit.client.1.smithi088.stdout:2/397: sync 2024-03-24T00:48:49.408 INFO:tasks.workunit.client.1.smithi088.stdout:4/389: sync 2024-03-24T00:48:49.408 INFO:tasks.workunit.client.1.smithi088.stdout:4/390: creat d5/f85 x:0 0 0 2024-03-24T00:48:49.411 INFO:tasks.workunit.client.1.smithi088.stdout:8/410: unlink l6 0 2024-03-24T00:48:49.415 INFO:tasks.workunit.client.0.smithi038.stdout:1/374: mkdir da/dd/d7a/d82 0 2024-03-24T00:48:49.415 INFO:tasks.workunit.client.0.smithi038.stdout:1/375: chown da/dd/d4a/d76/f36 4623525 1 2024-03-24T00:48:49.420 INFO:tasks.workunit.client.0.smithi038.stdout:7/367: stat d2/d9/c4e 0 2024-03-24T00:48:49.425 INFO:tasks.workunit.client.1.smithi088.stdout:3/384: mknod d3/de/df/d49/c7f 0 2024-03-24T00:48:49.425 INFO:tasks.workunit.client.1.smithi088.stdout:3/385: fdatasync d3/d6/d27/d3c/f39 0 2024-03-24T00:48:49.427 INFO:tasks.workunit.client.1.smithi088.stdout:6/426: rename db/d1d/d23/l5e to db/d40/d56/l79 0 2024-03-24T00:48:49.429 INFO:tasks.workunit.client.1.smithi088.stdout:2/398: rmdir db 39 2024-03-24T00:48:49.435 INFO:tasks.workunit.client.0.smithi038.stdout:9/427: mknod d12/d1a/d5f/c8d 0 2024-03-24T00:48:49.435 INFO:tasks.workunit.client.0.smithi038.stdout:4/313: sync 2024-03-24T00:48:49.436 INFO:tasks.workunit.client.0.smithi038.stdout:9/428: write d12/d55/d56/f7c [999451,108747] 0 2024-03-24T00:48:49.441 INFO:tasks.workunit.client.1.smithi088.stdout:4/391: creat d5/d13/f86 x:0 0 0 2024-03-24T00:48:49.445 INFO:tasks.workunit.client.0.smithi038.stdout:7/368: symlink d2/d9/df/d75/l8c 0 2024-03-24T00:48:49.445 INFO:tasks.workunit.client.1.smithi088.stdout:3/386: mkdir d3/d6/d27/d3c/d80 0 2024-03-24T00:48:49.447 INFO:tasks.workunit.client.0.smithi038.stdout:7/369: dread - d2/d9/df/d33/f27 zero size 2024-03-24T00:48:49.447 INFO:tasks.workunit.client.0.smithi038.stdout:4/314: mknod da/d12/d29/c60 0 2024-03-24T00:48:49.450 INFO:tasks.workunit.client.1.smithi088.stdout:8/411: write d9/dc/d11/d24/f71 [2708215,85396] 0 2024-03-24T00:48:49.461 INFO:tasks.workunit.client.1.smithi088.stdout:2/399: truncate db/d21/d35/f3d 3202201 0 2024-03-24T00:48:49.464 INFO:tasks.workunit.client.0.smithi038.stdout:4/315: dread - da/d12/d37/dd/d15/d33/f4b zero size 2024-03-24T00:48:49.464 INFO:tasks.workunit.client.0.smithi038.stdout:4/316: creat da/d38/f61 x:0 0 0 2024-03-24T00:48:49.465 INFO:tasks.workunit.client.0.smithi038.stdout:4/317: readlink da/d12/d37/dd/l20 0 2024-03-24T00:48:49.465 INFO:tasks.workunit.client.0.smithi038.stdout:4/318: chown da/d38/l5d 901479 1 2024-03-24T00:48:49.465 INFO:tasks.workunit.client.1.smithi088.stdout:3/387: creat d3/de/f81 x:0 0 0 2024-03-24T00:48:49.465 INFO:tasks.workunit.client.1.smithi088.stdout:3/388: chown d3/d1a/d1e/f28 515700988 1 2024-03-24T00:48:49.472 INFO:tasks.workunit.client.0.smithi038.stdout:8/386: dwrite d0/d45/d37/d5f/d68/f85 [0,4194304] 0 2024-03-24T00:48:49.473 INFO:tasks.workunit.client.1.smithi088.stdout:8/412: symlink d9/dc/d11/d41/l8c 0 2024-03-24T00:48:49.473 INFO:tasks.workunit.client.1.smithi088.stdout:8/413: chown d9/dc/d11/d24/f45 0 1 2024-03-24T00:48:49.487 INFO:tasks.workunit.client.0.smithi038.stdout:5/367: dwrite d6/d19/d20/d42/d4b/f4f [0,4194304] 0 2024-03-24T00:48:49.490 INFO:tasks.workunit.client.1.smithi088.stdout:0/408: dwrite d9/dc/de/d19/d44/d46/d65/f68 [0,4194304] 0 2024-03-24T00:48:49.491 INFO:tasks.workunit.client.1.smithi088.stdout:2/400: symlink db/d21/d4a/l7f 0 2024-03-24T00:48:49.491 INFO:tasks.workunit.client.1.smithi088.stdout:2/401: readlink db/dc/l26 0 2024-03-24T00:48:49.503 INFO:tasks.workunit.client.0.smithi038.stdout:9/429: write d12/d28/d3a/f50 [4943310,104064] 0 2024-03-24T00:48:49.506 INFO:tasks.workunit.client.0.smithi038.stdout:7/370: rmdir d2/d2c 39 2024-03-24T00:48:49.508 INFO:tasks.workunit.client.1.smithi088.stdout:3/389: rename d3/d6/d19/d2a/f69 to d3/d6/d27/d56/f82 0 2024-03-24T00:48:49.508 INFO:tasks.workunit.client.1.smithi088.stdout:8/414: rename d9/dc/d11/d24/d29/d36 to d9/dc/d11/d24/d29/d36/d6a/d7b/d8d 22 2024-03-24T00:48:49.508 INFO:tasks.workunit.client.1.smithi088.stdout:8/415: getdents d9/dc/d11/d80 0 2024-03-24T00:48:49.515 INFO:tasks.workunit.client.1.smithi088.stdout:0/409: mknod d9/dc/d4f/d63/c83 0 2024-03-24T00:48:49.517 INFO:tasks.workunit.client.0.smithi038.stdout:9/430: dread d12/d28/d3a/f45 [0,4194304] 0 2024-03-24T00:48:49.519 INFO:tasks.workunit.client.1.smithi088.stdout:7/353: sync 2024-03-24T00:48:49.519 INFO:tasks.workunit.client.1.smithi088.stdout:7/354: truncate d3/d32/d3f/f40 1142560 0 2024-03-24T00:48:49.519 INFO:tasks.workunit.client.1.smithi088.stdout:7/355: fdatasync d3/f2a 0 2024-03-24T00:48:49.519 INFO:tasks.workunit.client.1.smithi088.stdout:1/451: sync 2024-03-24T00:48:49.519 INFO:tasks.workunit.client.1.smithi088.stdout:1/452: chown f0 2171644 1 2024-03-24T00:48:49.520 INFO:tasks.workunit.client.0.smithi038.stdout:1/376: dwrite da/dd/f1c [0,4194304] 0 2024-03-24T00:48:49.520 INFO:tasks.workunit.client.1.smithi088.stdout:9/411: dwrite d1/d10/f34 [0,4194304] 0 2024-03-24T00:48:49.520 INFO:tasks.workunit.client.1.smithi088.stdout:6/427: dwrite db/d1d/d50/f6f [0,4194304] 0 2024-03-24T00:48:49.522 INFO:tasks.workunit.client.1.smithi088.stdout:2/402: symlink db/l80 0 2024-03-24T00:48:49.522 INFO:tasks.workunit.client.1.smithi088.stdout:2/403: creat db/d21/d4a/d4e/f81 x:0 0 0 2024-03-24T00:48:49.523 INFO:tasks.workunit.client.0.smithi038.stdout:6/418: dwrite d1/da/d1b/d1d/f24 [0,4194304] 0 2024-03-24T00:48:49.528 INFO:tasks.workunit.client.1.smithi088.stdout:3/390: truncate d3/de/f29 2867166 0 2024-03-24T00:48:49.528 INFO:tasks.workunit.client.1.smithi088.stdout:3/391: write d3/d6/f78 [180742,55282] 0 2024-03-24T00:48:49.528 INFO:tasks.workunit.client.0.smithi038.stdout:4/319: dwrite da/f1d [0,4194304] 0 2024-03-24T00:48:49.530 INFO:tasks.workunit.client.0.smithi038.stdout:4/320: write da/d12/f2c [4529694,130102] 0 2024-03-24T00:48:49.536 INFO:tasks.workunit.client.1.smithi088.stdout:0/410: dread d9/dc/de/d19/d44/f4a [4194304,4194304] 0 2024-03-24T00:48:49.536 INFO:tasks.workunit.client.1.smithi088.stdout:8/416: rename d9/dc/c15 to d9/dc/d11/d24/d29/d39/c8e 0 2024-03-24T00:48:49.540 INFO:tasks.workunit.client.0.smithi038.stdout:5/368: symlink d6/d16/d3c/l7e 0 2024-03-24T00:48:49.540 INFO:tasks.workunit.client.0.smithi038.stdout:5/369: write d6/d23/f53 [2647034,109306] 0 2024-03-24T00:48:49.540 INFO:tasks.workunit.client.0.smithi038.stdout:9/431: getdents d12/d1a/d5f 0 2024-03-24T00:48:49.540 INFO:tasks.workunit.client.0.smithi038.stdout:9/432: truncate d12/d28/d3a/f6d 938977 0 2024-03-24T00:48:49.547 INFO:tasks.workunit.client.1.smithi088.stdout:4/392: write d5/d17/f1d [364593,28589] 0 2024-03-24T00:48:49.551 INFO:tasks.workunit.client.1.smithi088.stdout:1/453: mknod d6/d27/d1e/d5b/d5e/c89 0 2024-03-24T00:48:49.575 INFO:tasks.workunit.client.0.smithi038.stdout:9/433: creat d12/d81/d84/f8e x:0 0 0 2024-03-24T00:48:49.576 INFO:tasks.workunit.client.0.smithi038.stdout:5/370: dread d6/f18 [0,4194304] 0 2024-03-24T00:48:49.580 INFO:tasks.workunit.client.0.smithi038.stdout:1/377: rmdir da/dd/d10/d14 39 2024-03-24T00:48:49.584 INFO:tasks.workunit.client.0.smithi038.stdout:9/434: mkdir d12/d28/d3a/d8f 0 2024-03-24T00:48:49.584 INFO:tasks.workunit.client.1.smithi088.stdout:9/412: symlink d1/d10/d2c/l78 0 2024-03-24T00:48:49.586 INFO:tasks.workunit.client.1.smithi088.stdout:5/410: sync 2024-03-24T00:48:49.590 INFO:tasks.workunit.client.0.smithi038.stdout:5/371: creat d6/d19/d20/d42/d43/d68/d6e/f7f x:0 0 0 2024-03-24T00:48:49.590 INFO:tasks.workunit.client.1.smithi088.stdout:2/404: read - db/d21/d35/f60 zero size 2024-03-24T00:48:49.591 INFO:tasks.workunit.client.1.smithi088.stdout:2/405: write db/d54/f6d [284622,18170] 0 2024-03-24T00:48:49.592 INFO:tasks.workunit.client.0.smithi038.stdout:1/378: mknod da/dd/d22/d5c/c83 0 2024-03-24T00:48:49.592 INFO:tasks.workunit.client.0.smithi038.stdout:1/379: chown da/dd/d2c/d5a/d74/l7c 894 1 2024-03-24T00:48:49.593 INFO:tasks.workunit.client.0.smithi038.stdout:1/380: fdatasync da/dd/d2c/d40/f6c 0 2024-03-24T00:48:49.593 INFO:tasks.workunit.client.0.smithi038.stdout:1/381: chown da/c32 67 1 2024-03-24T00:48:49.593 INFO:tasks.workunit.client.0.smithi038.stdout:1/382: write da/f16 [320099,18572] 0 2024-03-24T00:48:49.594 INFO:tasks.workunit.client.0.smithi038.stdout:9/435: symlink d12/d81/d84/l90 0 2024-03-24T00:48:49.594 INFO:tasks.workunit.client.1.smithi088.stdout:0/411: mkdir d9/dc/de/d19/d84 0 2024-03-24T00:48:49.595 INFO:tasks.workunit.client.1.smithi088.stdout:0/412: write d9/dc/de/d19/d44/d46/d65/f68 [4892618,109175] 0 2024-03-24T00:48:49.595 INFO:tasks.workunit.client.1.smithi088.stdout:0/413: write d9/dc/d4f/d62/f6d [165904,50423] 0 2024-03-24T00:48:49.596 INFO:tasks.workunit.client.0.smithi038.stdout:1/383: creat da/dd/d10/f84 x:0 0 0 2024-03-24T00:48:49.596 INFO:tasks.workunit.client.0.smithi038.stdout:1/384: truncate da/dd/d2c/d40/f6c 964127 0 2024-03-24T00:48:49.596 INFO:tasks.workunit.client.1.smithi088.stdout:6/428: dwrite db/d1d/f68 [0,4194304] 0 2024-03-24T00:48:49.599 INFO:tasks.workunit.client.1.smithi088.stdout:8/417: mkdir d9/dc/d11/d24/d29/d36/d6a/d7b/d8f 0 2024-03-24T00:48:49.600 INFO:tasks.workunit.client.0.smithi038.stdout:1/385: rename da/dd/d10/d34/f42 to da/dd/f85 0 2024-03-24T00:48:49.601 INFO:tasks.workunit.client.1.smithi088.stdout:4/393: symlink d5/d16/l87 0 2024-03-24T00:48:49.602 INFO:tasks.workunit.client.1.smithi088.stdout:4/394: stat d5/d7/d15/d74 0 2024-03-24T00:48:49.602 INFO:tasks.workunit.client.1.smithi088.stdout:4/395: chown d5/f56 844051 1 2024-03-24T00:48:49.603 INFO:tasks.workunit.client.1.smithi088.stdout:1/454: symlink d6/d27/d1e/d3f/d53/l8a 0 2024-03-24T00:48:49.604 INFO:tasks.workunit.client.0.smithi038.stdout:1/386: mkdir da/d3c/d86 0 2024-03-24T00:48:49.605 INFO:tasks.workunit.client.1.smithi088.stdout:5/411: rmdir d1/d19 39 2024-03-24T00:48:49.606 INFO:tasks.workunit.client.1.smithi088.stdout:9/413: write d1/d22/f43 [929354,121357] 0 2024-03-24T00:48:49.608 INFO:tasks.workunit.client.1.smithi088.stdout:9/414: dread - d1/d22/f51 zero size 2024-03-24T00:48:49.608 INFO:tasks.workunit.client.1.smithi088.stdout:2/406: unlink db/d54/f78 0 2024-03-24T00:48:49.609 INFO:tasks.workunit.client.0.smithi038.stdout:1/387: unlink da/d3c/f13 0 2024-03-24T00:48:49.609 INFO:tasks.workunit.client.0.smithi038.stdout:1/388: write da/f16 [954406,16321] 0 2024-03-24T00:48:49.610 INFO:tasks.workunit.client.0.smithi038.stdout:1/389: chown da/d3c/c80 5800236 1 2024-03-24T00:48:49.613 INFO:tasks.workunit.client.1.smithi088.stdout:0/414: dread d9/dc/de/d19/d44/d46/d65/d2a/f75 [0,4194304] 0 2024-03-24T00:48:49.613 INFO:tasks.workunit.client.1.smithi088.stdout:0/415: truncate d9/dc/de/d19/f48 4406418 0 2024-03-24T00:48:49.614 INFO:tasks.workunit.client.1.smithi088.stdout:3/392: write d3/de/f29 [1135839,25426] 0 2024-03-24T00:48:49.614 INFO:tasks.workunit.client.1.smithi088.stdout:3/393: readlink d3/d6/d27/l64 0 2024-03-24T00:48:49.622 INFO:tasks.workunit.client.1.smithi088.stdout:6/429: link db/l75 db/dd/d61/l7a 0 2024-03-24T00:48:49.622 INFO:tasks.workunit.client.1.smithi088.stdout:5/412: mkdir d1/d19/d48/d84 0 2024-03-24T00:48:49.622 INFO:tasks.workunit.client.1.smithi088.stdout:0/416: rename d9/dc/de/d19/d44/d46/d65/d2a/f5e to d9/f85 0 2024-03-24T00:48:49.622 INFO:tasks.workunit.client.1.smithi088.stdout:5/413: chown d1/d10/c24 1479714944 1 2024-03-24T00:48:49.622 INFO:tasks.workunit.client.1.smithi088.stdout:5/414: chown d1/d10/d1b/d2f/f7a 1847091237 1 2024-03-24T00:48:49.622 INFO:tasks.workunit.client.1.smithi088.stdout:5/415: chown d1/d19/d1e/d26/f5e 24 1 2024-03-24T00:48:49.622 INFO:tasks.workunit.client.1.smithi088.stdout:8/418: creat d9/dc/d11/d41/d54/d74/f90 x:0 0 0 2024-03-24T00:48:49.623 INFO:tasks.workunit.client.1.smithi088.stdout:8/419: fsync d9/dc/d11/d24/d29/f2c 0 2024-03-24T00:48:49.623 INFO:tasks.workunit.client.1.smithi088.stdout:8/420: chown d9/dc/d11/d24/d29/d5b/c70 6 1 2024-03-24T00:48:49.623 INFO:tasks.workunit.client.1.smithi088.stdout:5/416: write d1/d19/d1e/d26/d69/f82 [114144,58339] 0 2024-03-24T00:48:49.623 INFO:tasks.workunit.client.1.smithi088.stdout:5/417: fdatasync d1/d29/f77 0 2024-03-24T00:48:49.623 INFO:tasks.workunit.client.1.smithi088.stdout:5/418: dread - d1/d29/d3f/f41 zero size 2024-03-24T00:48:49.624 INFO:tasks.workunit.client.1.smithi088.stdout:0/417: write d9/dc/de/f14 [387125,93764] 0 2024-03-24T00:48:49.624 INFO:tasks.workunit.client.1.smithi088.stdout:0/418: chown d9/d26/f3c 1 1 2024-03-24T00:48:49.625 INFO:tasks.workunit.client.1.smithi088.stdout:3/394: symlink d3/d67/l83 0 2024-03-24T00:48:49.626 INFO:tasks.workunit.client.1.smithi088.stdout:6/430: truncate db/d1d/d23/d3c/d4b/f6e 1767296 0 2024-03-24T00:48:49.628 INFO:tasks.workunit.client.1.smithi088.stdout:9/415: read d1/d10/d11/d28/f4f [4103398,19802] 0 2024-03-24T00:48:49.629 INFO:tasks.workunit.client.1.smithi088.stdout:8/421: rename d9/dc/d11/d24/d29/d36/f46 to d9/dc/d11/d24/d43/d4a/d81/f91 0 2024-03-24T00:48:49.629 INFO:tasks.workunit.client.1.smithi088.stdout:8/422: readlink d9/dc/d11/d24/d43/l72 0 2024-03-24T00:48:49.629 INFO:tasks.workunit.client.1.smithi088.stdout:8/423: dread - d9/dc/d11/d24/f58 zero size 2024-03-24T00:48:49.629 INFO:tasks.workunit.client.1.smithi088.stdout:8/424: write d9/f3e [4386956,65017] 0 2024-03-24T00:48:49.629 INFO:tasks.workunit.client.1.smithi088.stdout:8/425: write d9/dc/d11/d24/d29/d5b/f1e [94786,80280] 0 2024-03-24T00:48:49.631 INFO:tasks.workunit.client.1.smithi088.stdout:0/419: link d9/dc/de/d19/d44/d46/d65/l25 d9/dc/l86 0 2024-03-24T00:48:49.631 INFO:tasks.workunit.client.1.smithi088.stdout:0/420: read - d9/dc/de/f6c zero size 2024-03-24T00:48:49.631 INFO:tasks.workunit.client.1.smithi088.stdout:0/421: chown d9/d15 242760 1 2024-03-24T00:48:49.632 INFO:tasks.workunit.client.1.smithi088.stdout:2/407: fsync db/d54/f6d 0 2024-03-24T00:48:49.632 INFO:tasks.workunit.client.1.smithi088.stdout:2/408: dread - db/d54/f65 zero size 2024-03-24T00:48:49.635 INFO:tasks.workunit.client.1.smithi088.stdout:3/395: unlink d3/d6/d19/d2a/c6d 0 2024-03-24T00:48:49.636 INFO:tasks.workunit.client.1.smithi088.stdout:9/416: creat d1/d22/d72/f79 x:0 0 0 2024-03-24T00:48:49.638 INFO:tasks.workunit.client.1.smithi088.stdout:7/356: dwrite d3/d32/d3f/f40 [0,4194304] 0 2024-03-24T00:48:49.641 INFO:tasks.workunit.client.0.smithi038.stdout:6/419: dwrite d1/f8 [0,4194304] 0 2024-03-24T00:48:49.641 INFO:tasks.workunit.client.0.smithi038.stdout:6/420: chown d1/da/d1b/d25/d49/d72/f76 7 1 2024-03-24T00:48:49.645 INFO:tasks.workunit.client.0.smithi038.stdout:1/390: rmdir da/dd/d10 39 2024-03-24T00:48:49.646 INFO:tasks.workunit.client.0.smithi038.stdout:1/391: truncate da/f7e 845635 0 2024-03-24T00:48:49.646 INFO:tasks.workunit.client.1.smithi088.stdout:8/426: link d9/dc/d11/d24/c2d d9/dc/d11/d24/d43/c92 0 2024-03-24T00:48:49.647 INFO:tasks.workunit.client.1.smithi088.stdout:6/431: dread db/dd/f1e [0,4194304] 0 2024-03-24T00:48:49.648 INFO:tasks.workunit.client.1.smithi088.stdout:2/409: dread db/d21/d4a/d4e/f64 [0,4194304] 0 2024-03-24T00:48:49.649 INFO:tasks.workunit.client.0.smithi038.stdout:4/321: dwrite f9 [0,4194304] 0 2024-03-24T00:48:49.649 INFO:tasks.workunit.client.0.smithi038.stdout:4/322: stat da/l35 0 2024-03-24T00:48:49.651 INFO:tasks.workunit.client.0.smithi038.stdout:6/421: mkdir d1/da/d73/d83 0 2024-03-24T00:48:49.655 INFO:tasks.workunit.client.0.smithi038.stdout:5/372: write d6/f3f [7237382,102809] 0 2024-03-24T00:48:49.655 INFO:tasks.workunit.client.0.smithi038.stdout:8/387: dwrite d0/d45/d37/f48 [0,4194304] 0 2024-03-24T00:48:49.655 INFO:tasks.workunit.client.0.smithi038.stdout:7/371: dwrite d2/f1f [4194304,4194304] 0 2024-03-24T00:48:49.658 INFO:tasks.workunit.client.1.smithi088.stdout:5/419: write d1/d19/d1e/f42 [3859363,56823] 0 2024-03-24T00:48:49.660 INFO:tasks.workunit.client.1.smithi088.stdout:5/420: creat d1/d19/d1e/d26/d69/f85 x:0 0 0 2024-03-24T00:48:49.660 INFO:tasks.workunit.client.1.smithi088.stdout:9/417: mkdir d1/d22/d27/d39/d62/d7a 0 2024-03-24T00:48:49.671 INFO:tasks.workunit.client.1.smithi088.stdout:0/422: dread d9/dc/de/d19/d44/f6e [0,4194304] 0 2024-03-24T00:48:49.671 INFO:tasks.workunit.client.1.smithi088.stdout:0/423: write d9/f45 [190648,103807] 0 2024-03-24T00:48:49.675 INFO:tasks.workunit.client.0.smithi038.stdout:1/392: rename da/d3c/c20 to da/dd/d2c/d5a/d74/c87 0 2024-03-24T00:48:49.676 INFO:tasks.workunit.client.0.smithi038.stdout:6/422: write d1/da/f1a [2388955,67155] 0 2024-03-24T00:48:49.687 INFO:tasks.workunit.client.0.smithi038.stdout:9/436: dread d12/d28/d3a/f50 [4194304,4194304] 0 2024-03-24T00:48:49.697 INFO:tasks.workunit.client.0.smithi038.stdout:4/323: creat da/d12/d37/f62 x:0 0 0 2024-03-24T00:48:49.697 INFO:tasks.workunit.client.0.smithi038.stdout:4/324: creat da/d38/f63 x:0 0 0 2024-03-24T00:48:49.697 INFO:tasks.workunit.client.0.smithi038.stdout:4/325: fdatasync f9 0 2024-03-24T00:48:49.697 INFO:tasks.workunit.client.1.smithi088.stdout:4/396: dwrite d5/d16/f7b [0,4194304] 0 2024-03-24T00:48:49.698 INFO:tasks.workunit.client.0.smithi038.stdout:4/326: write da/d12/f57 [1155537,100181] 0 2024-03-24T00:48:49.698 INFO:tasks.workunit.client.1.smithi088.stdout:1/455: dwrite d6/d27/d1e/d5b/f75 [0,4194304] 0 2024-03-24T00:48:49.699 INFO:tasks.workunit.client.1.smithi088.stdout:7/357: creat d3/d31/f6d x:0 0 0 2024-03-24T00:48:49.699 INFO:tasks.workunit.client.1.smithi088.stdout:7/358: creat d3/d31/f6e x:0 0 0 2024-03-24T00:48:49.702 INFO:tasks.workunit.client.1.smithi088.stdout:3/396: dwrite d3/d6/d27/d3c/f30 [0,4194304] 0 2024-03-24T00:48:49.702 INFO:tasks.workunit.client.1.smithi088.stdout:3/397: stat d3/d6/d27/d56 0 2024-03-24T00:48:49.702 INFO:tasks.workunit.client.1.smithi088.stdout:1/456: write d6/d27/d1e/d58/f48 [850818,115493] 0 2024-03-24T00:48:49.705 INFO:tasks.workunit.client.1.smithi088.stdout:8/427: rmdir d9/dc/d11/d41/d54/d89 0 2024-03-24T00:48:49.705 INFO:tasks.workunit.client.1.smithi088.stdout:8/428: chown d9/dc/d11/d24/l25 115749 1 2024-03-24T00:48:49.705 INFO:tasks.workunit.client.1.smithi088.stdout:8/429: stat d9/dc/d11/d24/d29/d36 0 2024-03-24T00:48:49.713 INFO:tasks.workunit.client.0.smithi038.stdout:7/372: link d2/d9/df/d75/l8c d2/d9/df/d75/d7d/l8d 0 2024-03-24T00:48:49.714 INFO:tasks.workunit.client.0.smithi038.stdout:7/373: fdatasync d2/f4 0 2024-03-24T00:48:49.716 INFO:tasks.workunit.client.1.smithi088.stdout:5/421: dwrite d1/d10/d1b/d2f/f7e [0,4194304] 0 2024-03-24T00:48:49.717 INFO:tasks.workunit.client.0.smithi038.stdout:5/373: dwrite d6/d19/d20/d42/f72 [0,4194304] 0 2024-03-24T00:48:49.719 INFO:tasks.workunit.client.1.smithi088.stdout:6/432: mknod db/dd/d61/d6d/c7b 0 2024-03-24T00:48:49.735 INFO:tasks.workunit.client.1.smithi088.stdout:9/418: rename d1/d15/d21/f3a to d1/d15/d47/f7b 0 2024-03-24T00:48:49.739 INFO:tasks.workunit.client.1.smithi088.stdout:5/422: dread d1/d10/d1b/f70 [0,4194304] 0 2024-03-24T00:48:49.745 INFO:tasks.workunit.client.0.smithi038.stdout:9/437: dwrite d12/d41/f7d [0,4194304] 0 2024-03-24T00:48:49.745 INFO:tasks.workunit.client.0.smithi038.stdout:8/388: dwrite d0/f16 [0,4194304] 0 2024-03-24T00:48:49.745 INFO:tasks.workunit.client.1.smithi088.stdout:0/424: mknod d9/dc/c87 0 2024-03-24T00:48:49.745 INFO:tasks.workunit.client.0.smithi038.stdout:9/438: write f4 [5607359,127565] 0 2024-03-24T00:48:49.745 INFO:tasks.workunit.client.0.smithi038.stdout:9/439: chown d12/d41/c46 4953 1 2024-03-24T00:48:49.755 INFO:tasks.workunit.client.1.smithi088.stdout:8/430: dwrite d9/dc/d11/d24/f58 [0,4194304] 0 2024-03-24T00:48:49.756 INFO:tasks.workunit.client.1.smithi088.stdout:8/431: write d9/dc/d11/d41/d56/f66 [103556,67367] 0 2024-03-24T00:48:49.757 INFO:tasks.workunit.client.0.smithi038.stdout:1/393: link da/d3c/f81 da/dd/d2c/d5a/d74/f88 0 2024-03-24T00:48:49.764 INFO:tasks.workunit.client.1.smithi088.stdout:5/423: dread d1/f3b [0,4194304] 0 2024-03-24T00:48:49.764 INFO:tasks.workunit.client.0.smithi038.stdout:6/423: mkdir d1/d37/d84 0 2024-03-24T00:48:49.773 INFO:tasks.workunit.client.0.smithi038.stdout:4/327: creat da/d12/d37/dd/d15/f64 x:0 0 0 2024-03-24T00:48:49.773 INFO:tasks.workunit.client.0.smithi038.stdout:8/389: read d0/d45/f55 [1069550,60583] 0 2024-03-24T00:48:49.778 INFO:tasks.workunit.client.0.smithi038.stdout:3/409: sync 2024-03-24T00:48:49.778 INFO:tasks.workunit.client.0.smithi038.stdout:7/374: link d2/d2c/l34 d2/d9/df/d65/l8e 0 2024-03-24T00:48:49.778 INFO:tasks.workunit.client.0.smithi038.stdout:0/322: sync 2024-03-24T00:48:49.778 INFO:tasks.workunit.client.0.smithi038.stdout:2/351: sync 2024-03-24T00:48:49.778 INFO:tasks.workunit.client.0.smithi038.stdout:2/352: fsync d4/d18/d38/f45 0 2024-03-24T00:48:49.779 INFO:tasks.workunit.client.1.smithi088.stdout:7/359: creat d3/d32/d3f/f6f x:0 0 0 2024-03-24T00:48:49.779 INFO:tasks.workunit.client.1.smithi088.stdout:7/360: stat d3/d31/f3c 0 2024-03-24T00:48:49.779 INFO:tasks.workunit.client.1.smithi088.stdout:7/361: creat d3/d31/f70 x:0 0 0 2024-03-24T00:48:49.788 INFO:tasks.workunit.client.1.smithi088.stdout:3/398: creat d3/d1a/d1e/d76/f84 x:0 0 0 2024-03-24T00:48:49.790 INFO:tasks.workunit.client.0.smithi038.stdout:5/374: mknod d6/d19/d20/c80 0 2024-03-24T00:48:49.800 INFO:tasks.workunit.client.0.smithi038.stdout:6/424: dread d1/da/d1b/d25/d49/d4e/f55 [0,4194304] 0 2024-03-24T00:48:49.801 INFO:tasks.workunit.client.0.smithi038.stdout:8/390: creat d0/d45/f89 x:0 0 0 2024-03-24T00:48:49.802 INFO:tasks.workunit.client.1.smithi088.stdout:6/433: mknod db/d40/d55/c7c 0 2024-03-24T00:48:49.810 INFO:tasks.workunit.client.0.smithi038.stdout:4/328: mknod da/d12/d37/dd/d15/c65 0 2024-03-24T00:48:49.819 INFO:tasks.workunit.client.0.smithi038.stdout:9/440: dwrite d12/d41/f4f [0,4194304] 0 2024-03-24T00:48:49.824 INFO:tasks.workunit.client.0.smithi038.stdout:7/375: link d2/d9/df/f71 d2/d9/df/d33/d37/d3a/f8f 0 2024-03-24T00:48:49.829 INFO:tasks.workunit.client.1.smithi088.stdout:1/457: rename d6/d43/c84 to d6/d6e/c8b 0 2024-03-24T00:48:49.829 INFO:tasks.workunit.client.1.smithi088.stdout:0/425: getdents d9/dc/d4f 0 2024-03-24T00:48:49.829 INFO:tasks.workunit.client.1.smithi088.stdout:9/419: rmdir d1/d15/d47 39 2024-03-24T00:48:49.834 INFO:tasks.workunit.client.0.smithi038.stdout:0/323: rmdir d0/d2d/d5f 0 2024-03-24T00:48:49.834 INFO:tasks.workunit.client.0.smithi038.stdout:4/329: write da/d12/d37/dd/f21 [2543915,100463] 0 2024-03-24T00:48:49.835 INFO:tasks.workunit.client.0.smithi038.stdout:2/353: unlink d4/d27/c2d 0 2024-03-24T00:48:49.835 INFO:tasks.workunit.client.0.smithi038.stdout:2/354: creat d4/d27/d54/d53/f7f x:0 0 0 2024-03-24T00:48:49.836 INFO:tasks.workunit.client.1.smithi088.stdout:1/458: dread d6/d27/d67/f6b [0,4194304] 0 2024-03-24T00:48:49.837 INFO:tasks.workunit.client.1.smithi088.stdout:4/397: mkdir d5/d7/dd/d88 0 2024-03-24T00:48:49.837 INFO:tasks.workunit.client.1.smithi088.stdout:1/459: getdents d6/d27/d1e/d5b/d5d/d61/d7d 0 2024-03-24T00:48:49.838 INFO:tasks.workunit.client.0.smithi038.stdout:8/391: creat d0/d1d/f8a x:0 0 0 2024-03-24T00:48:49.850 INFO:tasks.workunit.client.0.smithi038.stdout:0/324: mknod d0/d12/d15/d1a/c6a 0 2024-03-24T00:48:49.850 INFO:tasks.workunit.client.0.smithi038.stdout:0/325: write d0/d1b/d2c/f3b [461071,77703] 0 2024-03-24T00:48:49.851 INFO:tasks.workunit.client.0.smithi038.stdout:4/330: mknod da/d12/d37/dd/d15/d4c/c66 0 2024-03-24T00:48:49.851 INFO:tasks.workunit.client.0.smithi038.stdout:4/331: chown da/d12/d29 0 1 2024-03-24T00:48:49.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/332: fdatasync da/d12/d37/dd/f34 0 2024-03-24T00:48:49.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/333: chown da 69 1 2024-03-24T00:48:49.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/334: write da/d12/d37/dd/d15/f17 [4771802,126220] 0 2024-03-24T00:48:49.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/335: fdatasync da/d12/f5c 0 2024-03-24T00:48:49.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/336: fdatasync da/d12/d37/dd/f3a 0 2024-03-24T00:48:49.854 INFO:tasks.workunit.client.1.smithi088.stdout:8/432: creat d9/dc/d11/d24/d29/d39/f93 x:0 0 0 2024-03-24T00:48:49.854 INFO:tasks.workunit.client.1.smithi088.stdout:5/424: symlink d1/d29/d3f/l86 0 2024-03-24T00:48:49.854 INFO:tasks.workunit.client.1.smithi088.stdout:5/425: fsync d1/ff 0 2024-03-24T00:48:49.855 INFO:tasks.workunit.client.0.smithi038.stdout:2/355: unlink d4/d27/l3f 0 2024-03-24T00:48:49.859 INFO:tasks.workunit.client.0.smithi038.stdout:5/375: truncate d6/d19/d20/d42/f72 1213523 0 2024-03-24T00:48:49.872 INFO:tasks.workunit.client.0.smithi038.stdout:0/326: rename d0/f43 to d0/d1b/d2c/f6b 0 2024-03-24T00:48:49.874 INFO:tasks.workunit.client.1.smithi088.stdout:9/420: symlink d1/d10/l7c 0 2024-03-24T00:48:49.874 INFO:tasks.workunit.client.1.smithi088.stdout:9/421: creat d1/d10/d2c/f7d x:0 0 0 2024-03-24T00:48:49.874 INFO:tasks.workunit.client.1.smithi088.stdout:9/422: readlink d1/d10/d29/d6d/l55 0 2024-03-24T00:48:49.874 INFO:tasks.workunit.client.1.smithi088.stdout:9/423: fdatasync d1/d10/d29/f2e 0 2024-03-24T00:48:49.882 INFO:tasks.workunit.client.0.smithi038.stdout:2/356: symlink d4/d27/d54/l80 0 2024-03-24T00:48:49.883 INFO:tasks.workunit.client.0.smithi038.stdout:5/376: creat d6/d19/d20/d42/d43/d50/f81 x:0 0 0 2024-03-24T00:48:49.884 INFO:tasks.workunit.client.1.smithi088.stdout:8/433: mknod d9/c94 0 2024-03-24T00:48:49.888 INFO:tasks.workunit.client.1.smithi088.stdout:6/434: rmdir db/dd 39 2024-03-24T00:48:49.890 INFO:tasks.workunit.client.1.smithi088.stdout:5/426: dread d1/d19/d1e/f42 [0,4194304] 0 2024-03-24T00:48:49.890 INFO:tasks.workunit.client.1.smithi088.stdout:5/427: chown d1/d10/d1b/d2f/d57/f5f 179491985 1 2024-03-24T00:48:49.890 INFO:tasks.workunit.client.1.smithi088.stdout:5/428: fdatasync d1/d19/d1e/d26/d69/f85 0 2024-03-24T00:48:49.890 INFO:tasks.workunit.client.0.smithi038.stdout:2/357: mknod d4/d18/d5b/c81 0 2024-03-24T00:48:49.892 INFO:tasks.workunit.client.1.smithi088.stdout:6/435: write db/dd/f2b [345272,122068] 0 2024-03-24T00:48:49.900 INFO:tasks.workunit.client.1.smithi088.stdout:1/460: dread d6/d27/d1e/d58/f3d [0,4194304] 0 2024-03-24T00:48:49.900 INFO:tasks.workunit.client.1.smithi088.stdout:1/461: write d6/f8 [2807998,3364] 0 2024-03-24T00:48:49.901 INFO:tasks.workunit.client.0.smithi038.stdout:2/358: creat d4/d18/d38/f82 x:0 0 0 2024-03-24T00:48:49.901 INFO:tasks.workunit.client.0.smithi038.stdout:2/359: dread - d4/f15 zero size 2024-03-24T00:48:49.901 INFO:tasks.workunit.client.0.smithi038.stdout:2/360: chown d4/d27/d54/l5d 2 1 2024-03-24T00:48:49.903 INFO:tasks.workunit.client.1.smithi088.stdout:3/399: rename d3/de/df to d3/de/d6f/d85 0 2024-03-24T00:48:49.905 INFO:tasks.workunit.client.1.smithi088.stdout:2/410: sync 2024-03-24T00:48:49.908 INFO:tasks.workunit.client.0.smithi038.stdout:3/410: dwrite d3/d9/dc/d51/f75 [0,4194304] 0 2024-03-24T00:48:49.910 INFO:tasks.workunit.client.1.smithi088.stdout:4/398: rmdir d5/d13/d18/d24/d3d 39 2024-03-24T00:48:49.911 INFO:tasks.workunit.client.1.smithi088.stdout:4/399: chown d5/d16/f7b 354 1 2024-03-24T00:48:49.915 INFO:tasks.workunit.client.0.smithi038.stdout:7/376: dwrite d2/d9/df/d75/f85 [0,4194304] 0 2024-03-24T00:48:49.915 INFO:tasks.workunit.client.0.smithi038.stdout:7/377: stat d2/d9/da/dc/c12 0 2024-03-24T00:48:49.915 INFO:tasks.workunit.client.0.smithi038.stdout:2/361: mkdir d4/d18/d26/d37/d69/d83 0 2024-03-24T00:48:49.915 INFO:tasks.workunit.client.0.smithi038.stdout:2/362: fdatasync d4/ff 0 2024-03-24T00:48:49.920 INFO:tasks.workunit.client.1.smithi088.stdout:5/429: symlink d1/d10/d1b/d2f/d57/l87 0 2024-03-24T00:48:49.920 INFO:tasks.workunit.client.1.smithi088.stdout:6/436: rmdir db/d1d 39 2024-03-24T00:48:49.920 INFO:tasks.workunit.client.1.smithi088.stdout:6/437: chown db/d40/d56/l79 11918 1 2024-03-24T00:48:49.920 INFO:tasks.workunit.client.1.smithi088.stdout:5/430: read d1/dd/f13 [4092341,26806] 0 2024-03-24T00:48:49.923 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:49 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:49.923 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:49 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:49.923 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:49 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:48:49.923 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:49 smithi038 ceph-mon[29271]: pgmap v31: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 138 MiB/s rd, 166 MiB/s wr, 243 op/s 2024-03-24T00:48:49.937 INFO:tasks.workunit.client.0.smithi038.stdout:3/411: mkdir d3/d9/dc/df/d14/d19/d82/d89 0 2024-03-24T00:48:49.937 INFO:tasks.workunit.client.0.smithi038.stdout:3/412: chown d3/d30/c67 48825265 1 2024-03-24T00:48:49.941 INFO:tasks.workunit.client.1.smithi088.stdout:9/424: rename d1/d10/d11/c4e to d1/d10/d29/c7e 0 2024-03-24T00:48:49.953 INFO:tasks.workunit.client.1.smithi088.stdout:1/462: dread d6/d27/d1e/d5b/d5d/d61/f7e [0,4194304] 0 2024-03-24T00:48:49.953 INFO:tasks.workunit.client.1.smithi088.stdout:1/463: write d6/d27/d1e/d5b/d5e/f64 [378851,121946] 0 2024-03-24T00:48:49.959 INFO:tasks.workunit.client.1.smithi088.stdout:5/431: mkdir d1/d29/d3f/d88 0 2024-03-24T00:48:49.959 INFO:tasks.workunit.client.0.smithi038.stdout:9/441: dwrite d12/d28/d3a/d26/f3d [4194304,4194304] 0 2024-03-24T00:48:49.959 INFO:tasks.workunit.client.0.smithi038.stdout:9/442: stat d12/c2d 0 2024-03-24T00:48:49.959 INFO:tasks.workunit.client.0.smithi038.stdout:1/394: dwrite da/dd/d10/f84 [0,4194304] 0 2024-03-24T00:48:49.960 INFO:tasks.workunit.client.1.smithi088.stdout:7/362: sync 2024-03-24T00:48:49.962 INFO:tasks.workunit.client.1.smithi088.stdout:4/400: dread d5/d7/f76 [0,4194304] 0 2024-03-24T00:48:49.962 INFO:tasks.workunit.client.1.smithi088.stdout:4/401: fdatasync d5/d7/dd/f65 0 2024-03-24T00:48:49.969 INFO:tasks.workunit.client.1.smithi088.stdout:9/425: unlink d1/d22/d27/d39/f4d 0 2024-03-24T00:48:49.971 INFO:tasks.workunit.client.1.smithi088.stdout:8/434: dwrite d9/dc/d11/d24/d29/f51 [4194304,4194304] 0 2024-03-24T00:48:49.971 INFO:tasks.workunit.client.1.smithi088.stdout:8/435: chown d9/dc/d11/d24/d29/d73/f76 2062990206 1 2024-03-24T00:48:49.973 INFO:tasks.workunit.client.0.smithi038.stdout:2/363: symlink d4/d18/d26/d37/d69/d83/l84 0 2024-03-24T00:48:49.973 INFO:tasks.workunit.client.0.smithi038.stdout:2/364: write d4/d27/d54/d53/f64 [529767,66670] 0 2024-03-24T00:48:49.976 INFO:tasks.workunit.client.1.smithi088.stdout:1/464: symlink d6/d23/l8c 0 2024-03-24T00:48:49.979 INFO:tasks.workunit.client.1.smithi088.stdout:8/436: dread d9/d6b/f6c [0,4194304] 0 2024-03-24T00:48:49.979 INFO:tasks.workunit.client.1.smithi088.stdout:8/437: dread - d9/dc/d11/d24/f5e zero size 2024-03-24T00:48:49.986 INFO:tasks.workunit.client.1.smithi088.stdout:7/363: mknod d3/d32/d3f/d4b/d69/c71 0 2024-03-24T00:48:49.987 INFO:tasks.workunit.client.1.smithi088.stdout:9/426: dread d1/f7 [4194304,4194304] 0 2024-03-24T00:48:49.992 INFO:tasks.workunit.client.1.smithi088.stdout:0/426: dwrite d9/dc/de/d19/f48 [0,4194304] 0 2024-03-24T00:48:49.998 INFO:tasks.workunit.client.1.smithi088.stdout:4/402: link d5/d16/d37/f48 d5/d17/f89 0 2024-03-24T00:48:49.998 INFO:tasks.workunit.client.1.smithi088.stdout:4/403: fdatasync d5/d7/d72/f73 0 2024-03-24T00:48:50.007 INFO:tasks.workunit.client.0.smithi038.stdout:7/378: dwrite d2/d9/df/f71 [4194304,4194304] 0 2024-03-24T00:48:50.008 INFO:tasks.workunit.client.0.smithi038.stdout:9/443: mknod d12/c91 0 2024-03-24T00:48:50.008 INFO:tasks.workunit.client.1.smithi088.stdout:6/438: dwrite db/d40/d56/f51 [0,4194304] 0 2024-03-24T00:48:50.009 INFO:tasks.workunit.client.1.smithi088.stdout:6/439: write db/d1d/d23/d5c/f78 [209209,73023] 0 2024-03-24T00:48:50.009 INFO:tasks.workunit.client.0.smithi038.stdout:1/395: getdents da/dd/d10/d34 0 2024-03-24T00:48:50.010 INFO:tasks.workunit.client.0.smithi038.stdout:1/396: truncate da/dd/d2c/d5a/f7b 307304 0 2024-03-24T00:48:50.011 INFO:tasks.workunit.client.1.smithi088.stdout:8/438: mkdir d9/dc/d11/d24/d43/d4a/d95 0 2024-03-24T00:48:50.012 INFO:tasks.workunit.client.1.smithi088.stdout:8/439: chown d9/dc/d11/d24/d29/d39/d84 161097 1 2024-03-24T00:48:50.012 INFO:tasks.workunit.client.1.smithi088.stdout:8/440: readlink d9/dc/d11/d24/d43/l72 0 2024-03-24T00:48:50.012 INFO:tasks.workunit.client.1.smithi088.stdout:8/441: truncate d9/dc/d11/d41/d56/f66 655182 0 2024-03-24T00:48:50.015 INFO:tasks.workunit.client.1.smithi088.stdout:7/364: rename d3/d32/f6a to d3/d32/d34/f72 0 2024-03-24T00:48:50.017 INFO:tasks.workunit.client.1.smithi088.stdout:5/432: dwrite d1/d10/d1b/f70 [4194304,4194304] 0 2024-03-24T00:48:50.031 INFO:tasks.workunit.client.0.smithi038.stdout:2/365: creat d4/d18/d26/f85 x:0 0 0 2024-03-24T00:48:50.031 INFO:tasks.workunit.client.0.smithi038.stdout:2/366: write d4/d18/d38/f45 [51692,47986] 0 2024-03-24T00:48:50.034 INFO:tasks.workunit.client.0.smithi038.stdout:9/444: unlink d12/d28/d3a/f40 0 2024-03-24T00:48:50.035 INFO:tasks.workunit.client.0.smithi038.stdout:9/445: dread - d12/d28/d71/f8b zero size 2024-03-24T00:48:50.035 INFO:tasks.workunit.client.0.smithi038.stdout:9/446: chown d12/d28/d3a/f50 279859295 1 2024-03-24T00:48:50.035 INFO:tasks.workunit.client.0.smithi038.stdout:9/447: write d12/d41/f58 [993390,15458] 0 2024-03-24T00:48:50.040 INFO:tasks.workunit.client.0.smithi038.stdout:2/367: rmdir d4/d27/d54/d67 39 2024-03-24T00:48:50.041 INFO:tasks.workunit.client.1.smithi088.stdout:8/442: creat d9/dc/d11/d24/d29/d5b/d8b/f96 x:0 0 0 2024-03-24T00:48:50.041 INFO:tasks.workunit.client.1.smithi088.stdout:7/365: readlink d3/l4f 0 2024-03-24T00:48:50.046 INFO:tasks.workunit.client.1.smithi088.stdout:6/440: read db/d1d/d50/f77 [3684208,7485] 0 2024-03-24T00:48:50.047 INFO:tasks.workunit.client.0.smithi038.stdout:1/397: getdents da/dd/d2c/d41 0 2024-03-24T00:48:50.052 INFO:tasks.workunit.client.1.smithi088.stdout:0/427: dwrite d9/d26/f3c [4194304,4194304] 0 2024-03-24T00:48:50.056 INFO:tasks.workunit.client.0.smithi038.stdout:2/368: mkdir d4/d27/d54/d86 0 2024-03-24T00:48:50.059 INFO:tasks.workunit.client.1.smithi088.stdout:9/427: rename d1/d10/d11/f5d to d1/d15/f7f 0 2024-03-24T00:48:50.060 INFO:tasks.workunit.client.1.smithi088.stdout:5/433: truncate d1/d10/d1b/d2f/f7e 1847900 0 2024-03-24T00:48:50.060 INFO:tasks.workunit.client.1.smithi088.stdout:7/366: rename d3/d32/d3f/d4b/d69 to d3/d32/d3f/d4b/d73 0 2024-03-24T00:48:50.060 INFO:tasks.workunit.client.1.smithi088.stdout:6/441: dread db/d1d/d23/d3c/d4b/f6e [0,4194304] 0 2024-03-24T00:48:50.063 INFO:tasks.workunit.client.1.smithi088.stdout:9/428: mkdir d1/d15/d80 0 2024-03-24T00:48:50.069 INFO:tasks.workunit.client.1.smithi088.stdout:5/434: mknod d1/d10/d1b/c89 0 2024-03-24T00:48:50.069 INFO:tasks.workunit.client.1.smithi088.stdout:4/404: rename d5/d7/cb to d5/d13/c8a 0 2024-03-24T00:48:50.069 INFO:tasks.workunit.client.1.smithi088.stdout:4/405: fdatasync d5/d17/f81 0 2024-03-24T00:48:50.072 INFO:tasks.workunit.client.0.smithi038.stdout:4/337: dwrite da/d12/d24/f30 [4194304,4194304] 0 2024-03-24T00:48:50.072 INFO:tasks.workunit.client.0.smithi038.stdout:8/392: dwrite d0/d1d/d20/f60 [0,4194304] 0 2024-03-24T00:48:50.072 INFO:tasks.workunit.client.0.smithi038.stdout:8/393: readlink d0/d1d/d20/l41 0 2024-03-24T00:48:50.072 INFO:tasks.workunit.client.0.smithi038.stdout:8/394: truncate d0/d15/f63 388570 0 2024-03-24T00:48:50.077 INFO:tasks.workunit.client.1.smithi088.stdout:9/429: rename d1/d10/d11/d28/f4f to d1/d10/d11/d23/f81 0 2024-03-24T00:48:50.087 INFO:tasks.workunit.client.1.smithi088.stdout:4/406: mkdir d5/d8b 0 2024-03-24T00:48:50.091 INFO:tasks.workunit.client.0.smithi038.stdout:1/398: dread da/f23 [0,4194304] 0 2024-03-24T00:48:50.091 INFO:tasks.workunit.client.0.smithi038.stdout:2/369: write d4/d18/f22 [316446,14017] 0 2024-03-24T00:48:50.091 INFO:tasks.workunit.client.0.smithi038.stdout:1/399: chown da/dd/d22/f4c 220430847 1 2024-03-24T00:48:50.096 INFO:tasks.workunit.client.1.smithi088.stdout:9/430: creat d1/d15/d21/f82 x:0 0 0 2024-03-24T00:48:50.097 INFO:tasks.workunit.client.0.smithi038.stdout:2/370: rename d4/ff to d4/d27/d54/d67/f87 0 2024-03-24T00:48:50.097 INFO:tasks.workunit.client.0.smithi038.stdout:2/371: write d4/d18/d26/f6d [530325,51749] 0 2024-03-24T00:48:50.101 INFO:tasks.workunit.client.0.smithi038.stdout:1/400: symlink da/dd/d22/d5c/l89 0 2024-03-24T00:48:50.101 INFO:tasks.workunit.client.0.smithi038.stdout:1/401: fsync da/dd/d2c/d40/f6c 0 2024-03-24T00:48:50.103 INFO:tasks.workunit.client.1.smithi088.stdout:9/431: creat d1/d22/d27/f83 x:0 0 0 2024-03-24T00:48:50.104 INFO:tasks.workunit.client.0.smithi038.stdout:1/402: mknod da/dd/d10/d34/c8a 0 2024-03-24T00:48:50.104 INFO:tasks.workunit.client.0.smithi038.stdout:1/403: dread - da/dd/d2c/d5a/d74/f64 zero size 2024-03-24T00:48:50.104 INFO:tasks.workunit.client.0.smithi038.stdout:1/404: write da/dd/d22/f4f [247438,45504] 0 2024-03-24T00:48:50.104 INFO:tasks.workunit.client.0.smithi038.stdout:4/338: dread da/d38/f4a [0,4194304] 0 2024-03-24T00:48:50.104 INFO:tasks.workunit.client.0.smithi038.stdout:1/405: chown da/dd/d2c/d40/f6c 1 1 2024-03-24T00:48:50.104 INFO:tasks.workunit.client.0.smithi038.stdout:1/406: chown da/dd/d10/d14/c28 197105383 1 2024-03-24T00:48:50.106 INFO:tasks.workunit.client.0.smithi038.stdout:7/379: dwrite d2/d9/df/d5d/f7f [0,4194304] 0 2024-03-24T00:48:50.106 INFO:tasks.workunit.client.0.smithi038.stdout:7/380: chown d2/d9/df/d65 92396192 1 2024-03-24T00:48:50.111 INFO:tasks.workunit.client.0.smithi038.stdout:4/339: symlink da/d12/d29/l67 0 2024-03-24T00:48:50.112 INFO:tasks.workunit.client.0.smithi038.stdout:8/395: dread d0/d1d/d20/f23 [0,4194304] 0 2024-03-24T00:48:50.112 INFO:tasks.workunit.client.1.smithi088.stdout:9/432: mkdir d1/d15/d84 0 2024-03-24T00:48:50.112 INFO:tasks.workunit.client.1.smithi088.stdout:9/433: chown d1/d10/d11/d28/f71 166 1 2024-03-24T00:48:50.116 INFO:tasks.workunit.client.0.smithi038.stdout:9/448: dwrite f6 [0,4194304] 0 2024-03-24T00:48:50.119 INFO:tasks.workunit.client.0.smithi038.stdout:1/407: mkdir da/dd/d4a/d8b 0 2024-03-24T00:48:50.123 INFO:tasks.workunit.client.0.smithi038.stdout:7/381: mkdir d2/d2c/d90 0 2024-03-24T00:48:50.123 INFO:tasks.workunit.client.0.smithi038.stdout:7/382: read - d2/d9/da/d23/d2b/f5f zero size 2024-03-24T00:48:50.125 INFO:tasks.workunit.client.0.smithi038.stdout:0/327: dwrite d0/d12/f3c [0,4194304] 0 2024-03-24T00:48:50.126 INFO:tasks.workunit.client.0.smithi038.stdout:7/383: write d2/d9/da/dc/f1c [4508730,72273] 0 2024-03-24T00:48:50.126 INFO:tasks.workunit.client.1.smithi088.stdout:9/434: rmdir d1/d10/d29/d6d 39 2024-03-24T00:48:50.128 INFO:tasks.workunit.client.1.smithi088.stdout:8/443: dwrite d9/dc/d11/d24/d29/d39/f93 [0,4194304] 0 2024-03-24T00:48:50.128 INFO:tasks.workunit.client.1.smithi088.stdout:8/444: getdents d9/dc/d11/d24/d43/d4a/d95 0 2024-03-24T00:48:50.128 INFO:tasks.workunit.client.1.smithi088.stdout:8/445: creat d9/dc/d11/d24/d29/d36/f97 x:0 0 0 2024-03-24T00:48:50.136 INFO:tasks.workunit.client.0.smithi038.stdout:1/408: dread da/dd/d22/f55 [0,4194304] 0 2024-03-24T00:48:50.136 INFO:tasks.workunit.client.0.smithi038.stdout:9/449: creat d12/d28/d3a/d26/d79/f92 x:0 0 0 2024-03-24T00:48:50.136 INFO:tasks.workunit.client.0.smithi038.stdout:9/450: creat d12/d28/d3a/d26/f93 x:0 0 0 2024-03-24T00:48:50.137 INFO:tasks.workunit.client.0.smithi038.stdout:9/451: truncate d12/f35 1395824 0 2024-03-24T00:48:50.137 INFO:tasks.workunit.client.0.smithi038.stdout:1/409: getdents da/dd/d4a/d8b 0 2024-03-24T00:48:50.137 INFO:tasks.workunit.client.0.smithi038.stdout:1/410: chown da/d3c/l1a 14597005 1 2024-03-24T00:48:50.137 INFO:tasks.workunit.client.0.smithi038.stdout:1/411: write da/dd/d2c/d5a/f73 [777120,118454] 0 2024-03-24T00:48:50.137 INFO:tasks.workunit.client.1.smithi088.stdout:7/367: dwrite d3/f35 [8388608,4194304] 0 2024-03-24T00:48:50.138 INFO:tasks.workunit.client.1.smithi088.stdout:9/435: dread d1/d10/d11/f38 [0,4194304] 0 2024-03-24T00:48:50.139 INFO:tasks.workunit.client.0.smithi038.stdout:8/396: dread d0/d1d/d27/d4d/f7b [0,4194304] 0 2024-03-24T00:48:50.143 INFO:tasks.workunit.client.1.smithi088.stdout:5/435: dwrite d1/d10/d1b/d2f/d57/f71 [0,4194304] 0 2024-03-24T00:48:50.143 INFO:tasks.workunit.client.1.smithi088.stdout:5/436: dread - d1/d29/d3f/f41 zero size 2024-03-24T00:48:50.143 INFO:tasks.workunit.client.0.smithi038.stdout:2/372: write d4/d18/f2e [642515,50088] 0 2024-03-24T00:48:50.148 INFO:tasks.workunit.client.1.smithi088.stdout:8/446: symlink d9/dc/d11/d24/d29/d36/d6a/d7b/l98 0 2024-03-24T00:48:50.154 INFO:tasks.workunit.client.1.smithi088.stdout:4/407: dwrite d5/d13/d18/d24/f30 [0,4194304] 0 2024-03-24T00:48:50.159 INFO:tasks.workunit.client.1.smithi088.stdout:9/436: dread d1/f8 [4194304,4194304] 0 2024-03-24T00:48:50.159 INFO:tasks.workunit.client.0.smithi038.stdout:0/328: mknod d0/d14/c6c 0 2024-03-24T00:48:50.161 INFO:tasks.workunit.client.1.smithi088.stdout:5/437: mkdir d1/d10/d1b/d2f/d8a 0 2024-03-24T00:48:50.161 INFO:tasks.workunit.client.1.smithi088.stdout:5/438: fdatasync d1/d43/d4e/d5b/f63 0 2024-03-24T00:48:50.167 INFO:tasks.workunit.client.1.smithi088.stdout:4/408: mknod d5/d13/d18/c8c 0 2024-03-24T00:48:50.167 INFO:tasks.workunit.client.1.smithi088.stdout:4/409: truncate d5/d7/d15/d60/f69 939972 0 2024-03-24T00:48:50.172 INFO:tasks.workunit.client.0.smithi038.stdout:7/384: link d2/d9/df/d33/d37/c3b d2/d9/da/d23/d2b/c91 0 2024-03-24T00:48:50.172 INFO:tasks.workunit.client.0.smithi038.stdout:7/385: fsync d2/d9/f59 0 2024-03-24T00:48:50.174 INFO:tasks.workunit.client.1.smithi088.stdout:9/437: rmdir d1 39 2024-03-24T00:48:50.179 INFO:tasks.workunit.client.0.smithi038.stdout:1/412: mkdir da/dd/d22/d8c 0 2024-03-24T00:48:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:49 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:49 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:49 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:48:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:49 smithi088 ceph-mon[36610]: pgmap v31: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 138 MiB/s rd, 166 MiB/s wr, 243 op/s 2024-03-24T00:48:50.181 INFO:tasks.workunit.client.1.smithi088.stdout:4/410: getdents d5/d13/d18/d24/d3d 0 2024-03-24T00:48:50.182 INFO:tasks.workunit.client.0.smithi038.stdout:4/340: dwrite da/d12/f5c [4194304,4194304] 0 2024-03-24T00:48:50.184 INFO:tasks.workunit.client.0.smithi038.stdout:8/397: unlink d0/d15/c46 0 2024-03-24T00:48:50.187 INFO:tasks.workunit.client.1.smithi088.stdout:9/438: write d1/d10/d11/f17 [3976658,47863] 0 2024-03-24T00:48:50.187 INFO:tasks.workunit.client.0.smithi038.stdout:2/373: rename d4/d27/d54/d86 to d4/d55/d88 0 2024-03-24T00:48:50.188 INFO:tasks.workunit.client.0.smithi038.stdout:2/374: chown d4/d18/d26/f85 1 1 2024-03-24T00:48:50.190 INFO:tasks.workunit.client.1.smithi088.stdout:4/411: unlink d5/d13/d18/l61 0 2024-03-24T00:48:50.193 INFO:tasks.workunit.client.0.smithi038.stdout:0/329: creat d0/d1b/d2c/f6d x:0 0 0 2024-03-24T00:48:50.194 INFO:tasks.workunit.client.1.smithi088.stdout:9/439: mkdir d1/d10/d29/d85 0 2024-03-24T00:48:50.197 INFO:tasks.workunit.client.0.smithi038.stdout:1/413: creat da/dd/d7a/d82/f8d x:0 0 0 2024-03-24T00:48:50.200 INFO:tasks.workunit.client.1.smithi088.stdout:4/412: dread d5/d13/d18/d6e/d7d/f7e [0,4194304] 0 2024-03-24T00:48:50.200 INFO:tasks.workunit.client.1.smithi088.stdout:4/413: fsync d5/d7/d15/d60/f69 0 2024-03-24T00:48:50.202 INFO:tasks.workunit.client.0.smithi038.stdout:6/425: sync 2024-03-24T00:48:50.202 INFO:tasks.workunit.client.0.smithi038.stdout:6/426: write d1/da/f1a [2308115,56831] 0 2024-03-24T00:48:50.202 INFO:tasks.workunit.client.0.smithi038.stdout:6/427: chown d1/d37/c64 1646308425 1 2024-03-24T00:48:50.203 INFO:tasks.workunit.client.1.smithi088.stdout:9/440: rename d1/d10/d11/d23/f53 to d1/d22/d27/d39/f86 0 2024-03-24T00:48:50.203 INFO:tasks.workunit.client.1.smithi088.stdout:9/441: stat d1/d15/l60 0 2024-03-24T00:48:50.204 INFO:tasks.workunit.client.1.smithi088.stdout:3/400: sync 2024-03-24T00:48:50.204 INFO:tasks.workunit.client.1.smithi088.stdout:2/411: sync 2024-03-24T00:48:50.204 INFO:tasks.workunit.client.1.smithi088.stdout:3/401: truncate d3/d6/d27/d3c/f39 1055739 0 2024-03-24T00:48:50.206 INFO:tasks.workunit.client.0.smithi038.stdout:4/341: mkdir da/d12/d24/d68 0 2024-03-24T00:48:50.206 INFO:tasks.workunit.client.1.smithi088.stdout:4/414: symlink d5/d16/l8d 0 2024-03-24T00:48:50.210 INFO:tasks.workunit.client.1.smithi088.stdout:9/442: rename d1/d10/d2c/f67 to d1/d10/d29/d85/f87 0 2024-03-24T00:48:50.210 INFO:tasks.workunit.client.1.smithi088.stdout:9/443: read - d1/d10/d29/d6d/f57 zero size 2024-03-24T00:48:50.210 INFO:tasks.workunit.client.1.smithi088.stdout:2/412: symlink db/d21/d4a/l82 0 2024-03-24T00:48:50.212 INFO:tasks.workunit.client.1.smithi088.stdout:3/402: unlink d3/de/l35 0 2024-03-24T00:48:50.213 INFO:tasks.workunit.client.1.smithi088.stdout:4/415: rename d5/d7/d15/l5d to d5/d7/d15/l8e 0 2024-03-24T00:48:50.216 INFO:tasks.workunit.client.0.smithi038.stdout:3/413: sync 2024-03-24T00:48:50.216 INFO:tasks.workunit.client.0.smithi038.stdout:2/375: unlink d4/c6 0 2024-03-24T00:48:50.216 INFO:tasks.workunit.client.0.smithi038.stdout:2/376: chown d4/d18/d26/d37/f43 3152 1 2024-03-24T00:48:50.216 INFO:tasks.workunit.client.0.smithi038.stdout:2/377: stat d4/f10 0 2024-03-24T00:48:50.216 INFO:tasks.workunit.client.0.smithi038.stdout:2/378: read - d4/d18/f31 zero size 2024-03-24T00:48:50.216 INFO:tasks.workunit.client.0.smithi038.stdout:5/377: sync 2024-03-24T00:48:50.216 INFO:tasks.workunit.client.0.smithi038.stdout:5/378: stat d6/d19/d20/d42/c7c 0 2024-03-24T00:48:50.217 INFO:tasks.workunit.client.0.smithi038.stdout:5/379: read d6/d1e/f1f [1277575,63110] 0 2024-03-24T00:48:50.217 INFO:tasks.workunit.client.0.smithi038.stdout:5/380: write d6/d16/f2a [3431908,34438] 0 2024-03-24T00:48:50.217 INFO:tasks.workunit.client.0.smithi038.stdout:8/398: mknod d0/d45/c8b 0 2024-03-24T00:48:50.217 INFO:tasks.workunit.client.0.smithi038.stdout:8/399: write d0/d15/d2c/f88 [718652,99040] 0 2024-03-24T00:48:50.219 INFO:tasks.workunit.client.0.smithi038.stdout:6/428: creat d1/d37/d68/f85 x:0 0 0 2024-03-24T00:48:50.225 INFO:tasks.workunit.client.0.smithi038.stdout:0/330: write d0/d12/d42/f51 [1308065,73694] 0 2024-03-24T00:48:50.225 INFO:tasks.workunit.client.1.smithi088.stdout:9/444: link d1/d10/l65 d1/d22/d72/l88 0 2024-03-24T00:48:50.225 INFO:tasks.workunit.client.1.smithi088.stdout:3/403: unlink d3/d6/d27/f40 0 2024-03-24T00:48:50.228 INFO:tasks.workunit.client.0.smithi038.stdout:9/452: dwrite d12/d1a/d80/f83 [0,4194304] 0 2024-03-24T00:48:50.229 INFO:tasks.workunit.client.0.smithi038.stdout:1/414: dread da/dd/d2c/d5a/d74/f75 [0,4194304] 0 2024-03-24T00:48:50.230 INFO:tasks.workunit.client.0.smithi038.stdout:4/342: dread da/d12/d37/dd/f2b [0,4194304] 0 2024-03-24T00:48:50.234 INFO:tasks.workunit.client.0.smithi038.stdout:7/386: rename d2/d9/df to d2/d9/da/d23/d2b/d5b/d72/d92 0 2024-03-24T00:48:50.234 INFO:tasks.workunit.client.0.smithi038.stdout:7/387: chown l1 2062453350 1 2024-03-24T00:48:50.236 INFO:tasks.workunit.client.1.smithi088.stdout:4/416: unlink d5/f85 0 2024-03-24T00:48:50.236 INFO:tasks.workunit.client.1.smithi088.stdout:4/417: dread - d5/f46 zero size 2024-03-24T00:48:50.241 INFO:tasks.workunit.client.0.smithi038.stdout:5/381: mknod d6/d16/d21/c82 0 2024-03-24T00:48:50.241 INFO:tasks.workunit.client.0.smithi038.stdout:5/382: write d6/d19/d20/d42/d43/d68/f75 [3617345,62421] 0 2024-03-24T00:48:50.244 INFO:tasks.workunit.client.0.smithi038.stdout:8/400: getdents d0/d1d/d20 0 2024-03-24T00:48:50.245 INFO:tasks.workunit.client.1.smithi088.stdout:5/439: dwrite d1/f3b [4194304,4194304] 0 2024-03-24T00:48:50.245 INFO:tasks.workunit.client.1.smithi088.stdout:5/440: dread - d1/d19/d1e/d26/d69/f85 zero size 2024-03-24T00:48:50.248 INFO:tasks.workunit.client.1.smithi088.stdout:9/445: mknod d1/d10/c89 0 2024-03-24T00:48:50.248 INFO:tasks.workunit.client.1.smithi088.stdout:9/446: read - d1/d10/d2c/f7d zero size 2024-03-24T00:48:50.248 INFO:tasks.workunit.client.1.smithi088.stdout:9/447: stat d1/le 0 2024-03-24T00:48:50.248 INFO:tasks.workunit.client.0.smithi038.stdout:3/414: dread d3/d9/d77/f85 [0,4194304] 0 2024-03-24T00:48:50.249 INFO:tasks.workunit.client.0.smithi038.stdout:3/415: creat d3/d30/d70/f8a x:0 0 0 2024-03-24T00:48:50.249 INFO:tasks.workunit.client.0.smithi038.stdout:3/416: fdatasync d3/d9/dc/df/f6b 0 2024-03-24T00:48:50.261 INFO:tasks.workunit.client.0.smithi038.stdout:9/453: truncate d12/d28/d3a/f4b 249275 0 2024-03-24T00:48:50.261 INFO:tasks.workunit.client.0.smithi038.stdout:9/454: fdatasync d12/d28/d71/f8b 0 2024-03-24T00:48:50.263 INFO:tasks.workunit.client.0.smithi038.stdout:1/415: symlink da/dd/d2c/d41/d60/l8e 0 2024-03-24T00:48:50.265 INFO:tasks.workunit.client.0.smithi038.stdout:4/343: creat da/d27/f69 x:0 0 0 2024-03-24T00:48:50.266 INFO:tasks.workunit.client.0.smithi038.stdout:4/344: dread - da/d12/d37/f62 zero size 2024-03-24T00:48:50.266 INFO:tasks.workunit.client.0.smithi038.stdout:4/345: truncate da/d12/d37/dd/d15/d4c/f53 1514444 0 2024-03-24T00:48:50.266 INFO:tasks.workunit.client.0.smithi038.stdout:4/346: creat da/d12/d37/dd/d15/d4c/f6a x:0 0 0 2024-03-24T00:48:50.266 INFO:tasks.workunit.client.0.smithi038.stdout:4/347: fdatasync da/d12/d37/dd/d15/f64 0 2024-03-24T00:48:50.266 INFO:tasks.workunit.client.1.smithi088.stdout:5/441: mkdir d1/d29/d3f/d88/d8b 0 2024-03-24T00:48:50.267 INFO:tasks.workunit.client.0.smithi038.stdout:7/388: mkdir d2/d9/da/dc/d93 0 2024-03-24T00:48:50.267 INFO:tasks.workunit.client.0.smithi038.stdout:7/389: stat d2/d9/da/f36 0 2024-03-24T00:48:50.267 INFO:tasks.workunit.client.0.smithi038.stdout:7/390: chown d2/d9/da/dc/c17 380722477 1 2024-03-24T00:48:50.267 INFO:tasks.workunit.client.0.smithi038.stdout:7/391: fsync d2/d9/da/d23/d2b/d5b/d72/d92/f71 0 2024-03-24T00:48:50.267 INFO:tasks.workunit.client.0.smithi038.stdout:7/392: fdatasync d2/f4 0 2024-03-24T00:48:50.272 INFO:tasks.workunit.client.0.smithi038.stdout:5/383: dread d6/f26 [0,4194304] 0 2024-03-24T00:48:50.272 INFO:tasks.workunit.client.0.smithi038.stdout:5/384: chown d6/d19/d20/d42/d43/d68/c7d 29256 1 2024-03-24T00:48:50.273 INFO:tasks.workunit.client.0.smithi038.stdout:6/429: rename d1/da/d1b/d2f/f59 to d1/da/d1b/d1d/d3c/f86 0 2024-03-24T00:48:50.275 INFO:tasks.workunit.client.0.smithi038.stdout:3/417: creat d3/d30/d5d/d6a/f8b x:0 0 0 2024-03-24T00:48:50.276 INFO:tasks.workunit.client.0.smithi038.stdout:1/416: dread da/dd/d2c/d5a/d74/f88 [0,4194304] 0 2024-03-24T00:48:50.276 INFO:tasks.workunit.client.0.smithi038.stdout:1/417: chown l6 33 1 2024-03-24T00:48:50.278 INFO:tasks.workunit.client.0.smithi038.stdout:9/455: mkdir d12/d1a/d94 0 2024-03-24T00:48:50.280 INFO:tasks.workunit.client.1.smithi088.stdout:9/448: rename d1/d10/d11/d28 to d1/d10/d77/d8a 0 2024-03-24T00:48:50.280 INFO:tasks.workunit.client.1.smithi088.stdout:9/449: creat d1/d10/d11/d23/f8b x:0 0 0 2024-03-24T00:48:50.280 INFO:tasks.workunit.client.1.smithi088.stdout:9/450: stat d1/d15/d47/f7b 0 2024-03-24T00:48:50.281 INFO:tasks.workunit.client.0.smithi038.stdout:7/393: unlink d2/d9/da/c68 0 2024-03-24T00:48:50.282 INFO:tasks.workunit.client.0.smithi038.stdout:5/385: rmdir d6/d16 39 2024-03-24T00:48:50.283 INFO:tasks.workunit.client.1.smithi088.stdout:3/404: dwrite d3/d1a/d1e/d2f/d45/f5d [0,4194304] 0 2024-03-24T00:48:50.288 INFO:tasks.workunit.client.0.smithi038.stdout:5/386: write d6/d19/d20/d33/d6f/f7a [110544,121875] 0 2024-03-24T00:48:50.288 INFO:tasks.workunit.client.0.smithi038.stdout:6/430: creat d1/da/d73/d83/f87 x:0 0 0 2024-03-24T00:48:50.289 INFO:tasks.workunit.client.0.smithi038.stdout:8/401: mkdir d0/d1d/d8c 0 2024-03-24T00:48:50.289 INFO:tasks.workunit.client.0.smithi038.stdout:4/348: rename c1 to da/d12/d29/d5b/c6b 0 2024-03-24T00:48:50.290 INFO:tasks.workunit.client.0.smithi038.stdout:8/402: write d0/f76 [39266,74315] 0 2024-03-24T00:48:50.291 INFO:tasks.workunit.client.0.smithi038.stdout:8/403: truncate d0/d45/f89 654465 0 2024-03-24T00:48:50.291 INFO:tasks.workunit.client.0.smithi038.stdout:1/418: stat f4 0 2024-03-24T00:48:50.298 INFO:tasks.workunit.client.0.smithi038.stdout:7/394: truncate d2/d9/da/d23/f39 3989690 0 2024-03-24T00:48:50.303 INFO:tasks.workunit.client.0.smithi038.stdout:4/349: rename da/d27/d58 to da/d12/d24/d68/d6c 0 2024-03-24T00:48:50.307 INFO:tasks.workunit.client.1.smithi088.stdout:4/418: dwrite d5/d13/d18/d24/d3d/f77 [0,4194304] 0 2024-03-24T00:48:50.307 INFO:tasks.workunit.client.1.smithi088.stdout:4/419: dread - d5/d16/d1a/d22/f52 zero size 2024-03-24T00:48:50.308 INFO:tasks.workunit.client.1.smithi088.stdout:3/405: rename d3/l34 to d3/d6/d27/d3c/d80/l86 0 2024-03-24T00:48:50.310 INFO:tasks.workunit.client.0.smithi038.stdout:2/379: dwrite d4/f10 [0,4194304] 0 2024-03-24T00:48:50.311 INFO:tasks.workunit.client.1.smithi088.stdout:1/465: sync 2024-03-24T00:48:50.321 INFO:tasks.workunit.client.1.smithi088.stdout:4/420: rmdir d5/d7/d15 39 2024-03-24T00:48:50.322 INFO:tasks.workunit.client.1.smithi088.stdout:4/421: fdatasync d5/d17/f83 0 2024-03-24T00:48:50.322 INFO:tasks.workunit.client.1.smithi088.stdout:4/422: read - d5/f62 zero size 2024-03-24T00:48:50.322 INFO:tasks.workunit.client.0.smithi038.stdout:8/404: creat d0/d1d/d20/d3c/f8d x:0 0 0 2024-03-24T00:48:50.322 INFO:tasks.workunit.client.0.smithi038.stdout:1/419: truncate da/dd/d10/f1f 8211389 0 2024-03-24T00:48:50.323 INFO:tasks.workunit.client.1.smithi088.stdout:1/466: symlink d6/d6e/l8d 0 2024-03-24T00:48:50.329 INFO:tasks.workunit.client.1.smithi088.stdout:5/442: dwrite d1/d43/d4e/f6e [0,4194304] 0 2024-03-24T00:48:50.329 INFO:tasks.workunit.client.1.smithi088.stdout:9/451: dwrite d1/d10/d11/d23/f70 [0,4194304] 0 2024-03-24T00:48:50.329 INFO:tasks.workunit.client.1.smithi088.stdout:9/452: creat d1/d22/f8c x:0 0 0 2024-03-24T00:48:50.329 INFO:tasks.workunit.client.1.smithi088.stdout:9/453: chown d1/d22/d72/f79 1608109913 1 2024-03-24T00:48:50.330 INFO:tasks.workunit.client.1.smithi088.stdout:9/454: read - d1/d10/d2c/f75 zero size 2024-03-24T00:48:50.330 INFO:tasks.workunit.client.1.smithi088.stdout:9/455: chown d1/d10/d77/d8a/f71 199 1 2024-03-24T00:48:50.330 INFO:tasks.workunit.client.1.smithi088.stdout:9/456: creat d1/d22/d27/f8d x:0 0 0 2024-03-24T00:48:50.330 INFO:tasks.workunit.client.1.smithi088.stdout:9/457: dread - d1/d10/d2c/f75 zero size 2024-03-24T00:48:50.337 INFO:tasks.workunit.client.0.smithi038.stdout:2/380: unlink d4/d27/d2f/f75 0 2024-03-24T00:48:50.337 INFO:tasks.workunit.client.0.smithi038.stdout:2/381: write d4/d27/d54/d67/f87 [852291,63957] 0 2024-03-24T00:48:50.339 INFO:tasks.workunit.client.0.smithi038.stdout:8/405: creat d0/d1d/d27/f8e x:0 0 0 2024-03-24T00:48:50.339 INFO:tasks.workunit.client.1.smithi088.stdout:4/423: write d5/fc [2343607,1512] 0 2024-03-24T00:48:50.342 INFO:tasks.workunit.client.1.smithi088.stdout:1/467: dread d6/f3e [0,4194304] 0 2024-03-24T00:48:50.343 INFO:tasks.workunit.client.0.smithi038.stdout:1/420: mknod da/d3c/d86/c8f 0 2024-03-24T00:48:50.343 INFO:tasks.workunit.client.0.smithi038.stdout:1/421: write da/f16 [328719,121459] 0 2024-03-24T00:48:50.344 INFO:tasks.workunit.client.0.smithi038.stdout:4/350: rmdir da/d12/d37/dd/d15/d4c 39 2024-03-24T00:48:50.345 INFO:tasks.workunit.client.0.smithi038.stdout:0/331: dwrite d0/d1b/f20 [0,4194304] 0 2024-03-24T00:48:50.345 INFO:tasks.workunit.client.0.smithi038.stdout:0/332: write d0/f7 [764052,29870] 0 2024-03-24T00:48:50.346 INFO:tasks.workunit.client.0.smithi038.stdout:7/395: dread d2/d9/da/f41 [0,4194304] 0 2024-03-24T00:48:50.356 INFO:tasks.workunit.client.0.smithi038.stdout:9/456: rename d12/d28/l34 to d12/d28/d6c/l95 0 2024-03-24T00:48:50.356 INFO:tasks.workunit.client.0.smithi038.stdout:9/457: dread - d12/d28/d3a/d26/f93 zero size 2024-03-24T00:48:50.362 INFO:tasks.workunit.client.1.smithi088.stdout:5/443: dread d1/d43/d4e/f6e [0,4194304] 0 2024-03-24T00:48:50.373 INFO:tasks.workunit.client.0.smithi038.stdout:4/351: rename da/d12/d37/dd/d15/d33 to da/d12/d37/dd/d15/d6d 0 2024-03-24T00:48:50.373 INFO:tasks.workunit.client.0.smithi038.stdout:4/352: readlink da/d12/d29/l39 0 2024-03-24T00:48:50.379 INFO:tasks.workunit.client.1.smithi088.stdout:6/442: sync 2024-03-24T00:48:50.379 INFO:tasks.workunit.client.1.smithi088.stdout:0/428: sync 2024-03-24T00:48:50.385 INFO:tasks.workunit.client.0.smithi038.stdout:2/382: mkdir d4/d27/d2f/d89 0 2024-03-24T00:48:50.385 INFO:tasks.workunit.client.1.smithi088.stdout:7/368: sync 2024-03-24T00:48:50.387 INFO:tasks.workunit.client.0.smithi038.stdout:0/333: mknod d0/d12/d15/d1a/c6e 0 2024-03-24T00:48:50.389 INFO:tasks.workunit.client.0.smithi038.stdout:3/418: dwrite d3/d9/f58 [0,4194304] 0 2024-03-24T00:48:50.397 INFO:tasks.workunit.client.0.smithi038.stdout:7/396: dread d2/d2c/f62 [0,4194304] 0 2024-03-24T00:48:50.398 INFO:tasks.workunit.client.1.smithi088.stdout:9/458: mknod d1/d15/d47/c8e 0 2024-03-24T00:48:50.398 INFO:tasks.workunit.client.1.smithi088.stdout:9/459: truncate d1/d22/d72/f79 978759 0 2024-03-24T00:48:50.398 INFO:tasks.workunit.client.1.smithi088.stdout:9/460: chown d1/d10/d29/c4a 3580358 1 2024-03-24T00:48:50.400 INFO:tasks.workunit.client.0.smithi038.stdout:8/406: rename d0/c4 to d0/d1d/d27/d6b/c8f 0 2024-03-24T00:48:50.413 INFO:tasks.workunit.client.0.smithi038.stdout:4/353: link da/d12/d29/l39 da/d12/d24/d5f/l6e 0 2024-03-24T00:48:50.414 INFO:tasks.workunit.client.1.smithi088.stdout:3/406: dwrite d3/de/d6f/f75 [0,4194304] 0 2024-03-24T00:48:50.415 INFO:tasks.workunit.client.1.smithi088.stdout:6/443: mknod db/dd/d61/d6d/c7d 0 2024-03-24T00:48:50.417 INFO:tasks.workunit.client.0.smithi038.stdout:6/431: dwrite d1/da/d1b/d25/d49/d4e/f55 [0,4194304] 0 2024-03-24T00:48:50.419 INFO:tasks.workunit.client.0.smithi038.stdout:2/383: creat d4/d55/d88/f8a x:0 0 0 2024-03-24T00:48:50.420 INFO:tasks.workunit.client.1.smithi088.stdout:0/429: mkdir d9/dc/de/d19/d44/d46/d65/d2a/d2e/d88 0 2024-03-24T00:48:50.420 INFO:tasks.workunit.client.1.smithi088.stdout:7/369: truncate d3/f2d 8642489 0 2024-03-24T00:48:50.423 INFO:tasks.workunit.client.1.smithi088.stdout:9/461: unlink d1/d10/d11/f38 0 2024-03-24T00:48:50.424 INFO:tasks.workunit.client.0.smithi038.stdout:3/419: creat d3/f8c x:0 0 0 2024-03-24T00:48:50.424 INFO:tasks.workunit.client.0.smithi038.stdout:2/384: creat d4/d27/d2f/d30/d3e/f8b x:0 0 0 2024-03-24T00:48:50.427 INFO:tasks.workunit.client.1.smithi088.stdout:3/407: unlink d3/d6/d58/f63 0 2024-03-24T00:48:50.430 INFO:tasks.workunit.client.1.smithi088.stdout:0/430: creat d9/dc/de/d7e/f89 x:0 0 0 2024-03-24T00:48:50.431 INFO:tasks.workunit.client.1.smithi088.stdout:0/431: read d9/dc/de/d19/d44/d46/d65/d2a/d2e/f5f [3598303,12668] 0 2024-03-24T00:48:50.431 INFO:tasks.workunit.client.1.smithi088.stdout:0/432: fsync d9/dc/de/d19/d44/f4a 0 2024-03-24T00:48:50.434 INFO:tasks.workunit.client.1.smithi088.stdout:7/370: mkdir d3/d32/d3f/d4b/d74 0 2024-03-24T00:48:50.434 INFO:tasks.workunit.client.1.smithi088.stdout:7/371: write d3/d31/f6e [799708,22571] 0 2024-03-24T00:48:50.435 INFO:tasks.workunit.client.0.smithi038.stdout:7/397: symlink d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/l94 0 2024-03-24T00:48:50.437 INFO:tasks.workunit.client.1.smithi088.stdout:0/433: rmdir d9/dc/de/d19/d44/d46/d65/d2a/d2e/d88 0 2024-03-24T00:48:50.438 INFO:tasks.workunit.client.1.smithi088.stdout:3/408: creat d3/f87 x:0 0 0 2024-03-24T00:48:50.438 INFO:tasks.workunit.client.1.smithi088.stdout:3/409: write d3/d1a/d1e/d2f/f36 [2125733,121929] 0 2024-03-24T00:48:50.438 INFO:tasks.workunit.client.1.smithi088.stdout:3/410: truncate d3/de/f16 4363617 0 2024-03-24T00:48:50.439 INFO:tasks.workunit.client.1.smithi088.stdout:0/434: creat d9/dc/d4f/d63/f8a x:0 0 0 2024-03-24T00:48:50.441 INFO:tasks.workunit.client.0.smithi038.stdout:1/422: rename da/dd/d2c/d41/c6d to da/dd/d2c/d5a/d74/c90 0 2024-03-24T00:48:50.441 INFO:tasks.workunit.client.0.smithi038.stdout:1/423: fdatasync da/dd/d7a/d82/f8d 0 2024-03-24T00:48:50.442 INFO:tasks.workunit.client.0.smithi038.stdout:1/424: chown da/f23 267 1 2024-03-24T00:48:50.443 INFO:tasks.workunit.client.0.smithi038.stdout:0/334: rename d0/fc to d0/d2d/f6f 0 2024-03-24T00:48:50.445 INFO:tasks.workunit.client.0.smithi038.stdout:4/354: link da/d12/d37/f54 da/d12/d24/d5f/f6f 0 2024-03-24T00:48:50.446 INFO:tasks.workunit.client.0.smithi038.stdout:1/425: creat da/dd/d7a/f91 x:0 0 0 2024-03-24T00:48:50.446 INFO:tasks.workunit.client.0.smithi038.stdout:1/426: chown da/d3c/d86/c8f 32172811 1 2024-03-24T00:48:50.446 INFO:tasks.workunit.client.0.smithi038.stdout:1/427: creat da/dd/d2c/d5a/f92 x:0 0 0 2024-03-24T00:48:50.447 INFO:tasks.workunit.client.0.smithi038.stdout:1/428: write da/dd/f85 [262709,88409] 0 2024-03-24T00:48:50.452 INFO:tasks.workunit.client.0.smithi038.stdout:1/429: write da/dd/d4a/d76/f36 [28711,30032] 0 2024-03-24T00:48:50.455 INFO:tasks.workunit.client.0.smithi038.stdout:1/430: symlink da/dd/d2c/d41/d60/l93 0 2024-03-24T00:48:50.456 INFO:tasks.workunit.client.0.smithi038.stdout:1/431: read da/dd/f85 [282139,50103] 0 2024-03-24T00:48:50.456 INFO:tasks.workunit.client.0.smithi038.stdout:1/432: write da/dd/d22/d5c/f6e [665736,67633] 0 2024-03-24T00:48:50.457 INFO:tasks.workunit.client.0.smithi038.stdout:0/335: write d0/f2 [647601,46383] 0 2024-03-24T00:48:50.467 INFO:tasks.workunit.client.0.smithi038.stdout:1/433: mkdir da/dd/d22/d94 0 2024-03-24T00:48:50.467 INFO:tasks.workunit.client.0.smithi038.stdout:1/434: write da/dd/d10/d14/f47 [11852442,96919] 0 2024-03-24T00:48:50.468 INFO:tasks.workunit.client.1.smithi088.stdout:8/447: sync 2024-03-24T00:48:50.468 INFO:tasks.workunit.client.1.smithi088.stdout:2/413: sync 2024-03-24T00:48:50.475 INFO:tasks.workunit.client.0.smithi038.stdout:7/398: dread d2/d9/da/d23/d2b/d5b/d72/d92/f29 [0,4194304] 0 2024-03-24T00:48:50.475 INFO:tasks.workunit.client.0.smithi038.stdout:7/399: readlink d2/d9/d16/l1e 0 2024-03-24T00:48:50.477 INFO:tasks.workunit.client.1.smithi088.stdout:2/414: creat db/d21/d73/f83 x:0 0 0 2024-03-24T00:48:50.478 INFO:tasks.workunit.client.0.smithi038.stdout:7/400: write d2/d9/da/d23/d2b/d5b/d72/d92/d75/f85 [5081142,23391] 0 2024-03-24T00:48:50.484 INFO:tasks.workunit.client.1.smithi088.stdout:7/372: read d3/d32/d3f/f40 [3130547,91112] 0 2024-03-24T00:48:50.492 INFO:tasks.workunit.client.1.smithi088.stdout:8/448: symlink d9/dc/d11/l99 0 2024-03-24T00:48:50.499 INFO:tasks.workunit.client.0.smithi038.stdout:1/435: write da/dd/d2c/d41/f65 [89875,71250] 0 2024-03-24T00:48:50.500 INFO:tasks.workunit.client.0.smithi038.stdout:5/387: dwrite d6/d19/d20/d42/d43/d4c/f74 [0,4194304] 0 2024-03-24T00:48:50.501 INFO:tasks.workunit.client.1.smithi088.stdout:2/415: link db/d21/c4c db/c84 0 2024-03-24T00:48:50.506 INFO:tasks.workunit.client.0.smithi038.stdout:7/401: getdents d2/d9/da/dc 0 2024-03-24T00:48:50.506 INFO:tasks.workunit.client.0.smithi038.stdout:7/402: stat d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37 0 2024-03-24T00:48:50.511 INFO:tasks.workunit.client.1.smithi088.stdout:8/449: dread d9/dc/d11/d24/d29/d5b/f1e [0,4194304] 0 2024-03-24T00:48:50.512 INFO:tasks.workunit.client.0.smithi038.stdout:1/436: rmdir da/dd 39 2024-03-24T00:48:50.514 INFO:tasks.workunit.client.0.smithi038.stdout:1/437: chown da/dd/d22/c29 232176 1 2024-03-24T00:48:50.521 INFO:tasks.workunit.client.1.smithi088.stdout:2/416: creat db/dd/f85 x:0 0 0 2024-03-24T00:48:50.522 INFO:tasks.workunit.client.0.smithi038.stdout:5/388: mknod d6/d19/d20/d42/d43/d48/c83 0 2024-03-24T00:48:50.528 INFO:tasks.workunit.client.1.smithi088.stdout:8/450: creat d9/dc/d11/d41/d54/f9a x:0 0 0 2024-03-24T00:48:50.531 INFO:tasks.workunit.client.1.smithi088.stdout:0/435: dwrite d9/f80 [0,4194304] 0 2024-03-24T00:48:50.532 INFO:tasks.workunit.client.1.smithi088.stdout:0/436: chown c6 887 1 2024-03-24T00:48:50.532 INFO:tasks.workunit.client.1.smithi088.stdout:0/437: fdatasync d9/dc/de/f3e 0 2024-03-24T00:48:50.537 INFO:tasks.workunit.client.1.smithi088.stdout:4/424: dwrite d5/d17/f89 [0,4194304] 0 2024-03-24T00:48:50.537 INFO:tasks.workunit.client.1.smithi088.stdout:4/425: write d5/d13/d18/d24/f53 [514672,28599] 0 2024-03-24T00:48:50.537 INFO:tasks.workunit.client.0.smithi038.stdout:7/403: link d2/d9/da/d23/l87 d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/l95 0 2024-03-24T00:48:50.538 INFO:tasks.workunit.client.1.smithi088.stdout:5/444: dwrite d1/d10/d1b/d2f/f7e [0,4194304] 0 2024-03-24T00:48:50.541 INFO:tasks.workunit.client.1.smithi088.stdout:7/373: rename d3/d31/f47 to d3/f75 0 2024-03-24T00:48:50.541 INFO:tasks.workunit.client.1.smithi088.stdout:0/438: rename d9/dc/de/d19/d44/d46/d65/d2a to d9/dc/de/d19/d44/d46/d65/d2a/d2e/d8b 22 2024-03-24T00:48:50.541 INFO:tasks.workunit.client.0.smithi038.stdout:9/458: dwrite d12/d28/d3a/f2b [0,4194304] 0 2024-03-24T00:48:50.541 INFO:tasks.workunit.client.0.smithi038.stdout:9/459: chown d12/f42 80 1 2024-03-24T00:48:50.545 INFO:tasks.workunit.client.0.smithi038.stdout:5/389: unlink d6/d19/d20/d42/c6a 0 2024-03-24T00:48:50.548 INFO:tasks.workunit.client.1.smithi088.stdout:8/451: mknod d9/dc/d11/d24/d29/d73/c9b 0 2024-03-24T00:48:50.549 INFO:tasks.workunit.client.0.smithi038.stdout:1/438: dread da/dd/d22/f25 [0,4194304] 0 2024-03-24T00:48:50.549 INFO:tasks.workunit.client.0.smithi038.stdout:1/439: chown da/d3c 113 1 2024-03-24T00:48:50.549 INFO:tasks.workunit.client.1.smithi088.stdout:6/444: dwrite db/f19 [0,4194304] 0 2024-03-24T00:48:50.552 INFO:tasks.workunit.client.0.smithi038.stdout:7/404: creat d2/d9/da/d23/d2b/d5b/d72/d92/d33/f96 x:0 0 0 2024-03-24T00:48:50.552 INFO:tasks.workunit.client.1.smithi088.stdout:3/411: write d3/d1a/d1e/f28 [3245703,86881] 0 2024-03-24T00:48:50.554 INFO:tasks.workunit.client.0.smithi038.stdout:7/405: fsync d2/d9/da/d23/f39 0 2024-03-24T00:48:50.554 INFO:tasks.workunit.client.0.smithi038.stdout:7/406: stat d2/d2c/f56 0 2024-03-24T00:48:50.555 INFO:tasks.workunit.client.1.smithi088.stdout:2/417: dread db/f38 [0,4194304] 0 2024-03-24T00:48:50.555 INFO:tasks.workunit.client.1.smithi088.stdout:2/418: write db/d21/d4a/f4b [1761275,80966] 0 2024-03-24T00:48:50.556 INFO:tasks.workunit.client.0.smithi038.stdout:9/460: creat d12/d81/f96 x:0 0 0 2024-03-24T00:48:50.557 INFO:tasks.workunit.client.0.smithi038.stdout:9/461: creat d12/d1a/d80/f97 x:0 0 0 2024-03-24T00:48:50.566 INFO:tasks.workunit.client.1.smithi088.stdout:4/426: symlink d5/d7/d15/d74/l8f 0 2024-03-24T00:48:50.567 INFO:tasks.workunit.client.1.smithi088.stdout:7/374: write d3/d32/f43 [7780025,36981] 0 2024-03-24T00:48:50.567 INFO:tasks.workunit.client.1.smithi088.stdout:7/375: dread - d3/d32/f44 zero size 2024-03-24T00:48:50.567 INFO:tasks.workunit.client.0.smithi038.stdout:5/390: mkdir d6/d19/d20/d42/d43/d84 0 2024-03-24T00:48:50.567 INFO:tasks.workunit.client.0.smithi038.stdout:5/391: creat d6/d16/d21/f85 x:0 0 0 2024-03-24T00:48:50.567 INFO:tasks.workunit.client.0.smithi038.stdout:5/392: write d6/d16/d21/f85 [648179,25037] 0 2024-03-24T00:48:50.567 INFO:tasks.workunit.client.0.smithi038.stdout:3/420: write d3/d9/f3c [3345563,85429] 0 2024-03-24T00:48:50.568 INFO:tasks.workunit.client.0.smithi038.stdout:2/385: dwrite f2 [4194304,4194304] 0 2024-03-24T00:48:50.568 INFO:tasks.workunit.client.0.smithi038.stdout:6/432: dwrite d1/da/d1b/d25/f44 [0,4194304] 0 2024-03-24T00:48:50.568 INFO:tasks.workunit.client.0.smithi038.stdout:6/433: chown d1/da/d1b/d2f/l32 715 1 2024-03-24T00:48:50.568 INFO:tasks.workunit.client.0.smithi038.stdout:1/440: rmdir da/dd/d22/d8c 0 2024-03-24T00:48:50.568 INFO:tasks.workunit.client.0.smithi038.stdout:1/441: write da/dd/d10/f84 [4581338,21237] 0 2024-03-24T00:48:50.568 INFO:tasks.workunit.client.0.smithi038.stdout:2/386: creat d4/d18/d26/f8c x:0 0 0 2024-03-24T00:48:50.568 INFO:tasks.workunit.client.0.smithi038.stdout:3/421: read d3/d9/f26 [95384,57822] 0 2024-03-24T00:48:50.568 INFO:tasks.workunit.client.0.smithi038.stdout:3/422: chown d3/d9/dc/df/f1d 214372 1 2024-03-24T00:48:50.574 INFO:tasks.workunit.client.0.smithi038.stdout:5/393: dread d6/d16/d21/f5f [0,4194304] 0 2024-03-24T00:48:50.575 INFO:tasks.workunit.client.0.smithi038.stdout:8/407: dwrite d0/d1d/d27/f8e [0,4194304] 0 2024-03-24T00:48:50.575 INFO:tasks.workunit.client.0.smithi038.stdout:4/355: dwrite da/d38/f61 [0,4194304] 0 2024-03-24T00:48:50.575 INFO:tasks.workunit.client.0.smithi038.stdout:8/408: chown d0/c1 46 1 2024-03-24T00:48:50.575 INFO:tasks.workunit.client.0.smithi038.stdout:4/356: write da/d27/f69 [942095,108288] 0 2024-03-24T00:48:50.576 INFO:tasks.workunit.client.1.smithi088.stdout:5/445: link d1/d19/f2c d1/d19/d1e/d26/d69/f8c 0 2024-03-24T00:48:50.576 INFO:tasks.workunit.client.1.smithi088.stdout:5/446: write d1/d19/f30 [7095993,73303] 0 2024-03-24T00:48:50.576 INFO:tasks.workunit.client.1.smithi088.stdout:5/447: write d1/d19/d1e/d26/d69/f8c [3166586,19515] 0 2024-03-24T00:48:50.576 INFO:tasks.workunit.client.1.smithi088.stdout:5/448: readlink d1/l8 0 2024-03-24T00:48:50.576 INFO:tasks.workunit.client.1.smithi088.stdout:5/449: creat d1/d10/d1b/d2f/d57/f8d x:0 0 0 2024-03-24T00:48:50.576 INFO:tasks.workunit.client.1.smithi088.stdout:0/439: getdents d9/dc/de/d19/d44/d46/d65/d2a 0 2024-03-24T00:48:50.583 INFO:tasks.workunit.client.0.smithi038.stdout:0/336: dwrite d0/d12/d60/d28/f48 [4194304,4194304] 0 2024-03-24T00:48:50.586 INFO:tasks.workunit.client.1.smithi088.stdout:8/452: mkdir d9/dc/d11/d24/d29/d5b/d6f/d7c/d9c 0 2024-03-24T00:48:50.587 INFO:tasks.workunit.client.0.smithi038.stdout:7/407: read d2/f13 [3439303,26585] 0 2024-03-24T00:48:50.588 INFO:tasks.workunit.client.0.smithi038.stdout:7/408: dread - d2/d9/da/f73 zero size 2024-03-24T00:48:50.589 INFO:tasks.workunit.client.1.smithi088.stdout:3/412: dread d3/d6/d27/d3c/f30 [0,4194304] 0 2024-03-24T00:48:50.598 INFO:tasks.workunit.client.0.smithi038.stdout:9/462: read d12/d28/d3a/f30 [3318004,55559] 0 2024-03-24T00:48:50.606 INFO:tasks.workunit.client.0.smithi038.stdout:1/442: link da/dd/d2c/f63 da/dd/f95 0 2024-03-24T00:48:50.606 INFO:tasks.workunit.client.0.smithi038.stdout:1/443: truncate da/dd/d2c/d5a/f92 147114 0 2024-03-24T00:48:50.623 INFO:tasks.workunit.client.0.smithi038.stdout:2/387: creat d4/d27/d2f/d30/f8d x:0 0 0 2024-03-24T00:48:50.623 INFO:tasks.workunit.client.0.smithi038.stdout:2/388: creat d4/d18/d26/f8e x:0 0 0 2024-03-24T00:48:50.639 INFO:tasks.workunit.client.0.smithi038.stdout:3/423: rmdir d3/d9/dc/df/d14/d19/d82 39 2024-03-24T00:48:50.640 INFO:tasks.workunit.client.1.smithi088.stdout:6/445: link db/d40/d56/c3e db/d1d/d64/d72/c7e 0 2024-03-24T00:48:50.640 INFO:tasks.workunit.client.1.smithi088.stdout:6/446: write db/d1d/d50/f77 [809895,29427] 0 2024-03-24T00:48:50.646 INFO:tasks.workunit.client.1.smithi088.stdout:6/447: dread db/d1d/f58 [0,4194304] 0 2024-03-24T00:48:50.646 INFO:tasks.workunit.client.1.smithi088.stdout:6/448: readlink db/dd/l14 0 2024-03-24T00:48:50.656 INFO:tasks.workunit.client.1.smithi088.stdout:6/449: write db/d1d/d23/d3c/d4b/f6e [1352819,50540] 0 2024-03-24T00:48:50.656 INFO:tasks.workunit.client.1.smithi088.stdout:6/450: readlink l7 0 2024-03-24T00:48:50.657 INFO:tasks.workunit.client.1.smithi088.stdout:8/453: dwrite d9/dc/d11/d24/d29/d5b/f34 [4194304,4194304] 0 2024-03-24T00:48:50.658 INFO:tasks.workunit.client.1.smithi088.stdout:0/440: dwrite d9/dc/de/d19/d44/d46/f57 [4194304,4194304] 0 2024-03-24T00:48:50.659 INFO:tasks.workunit.client.1.smithi088.stdout:0/441: write d9/dc/de/d19/d44/d46/f57 [9106475,32559] 0 2024-03-24T00:48:50.659 INFO:tasks.workunit.client.1.smithi088.stdout:0/442: creat d9/dc/de/d7e/f8c x:0 0 0 2024-03-24T00:48:50.659 INFO:tasks.workunit.client.1.smithi088.stdout:0/443: write d9/dc/de/d19/d44/d46/d65/d6f/f78 [97782,84622] 0 2024-03-24T00:48:50.659 INFO:tasks.workunit.client.1.smithi088.stdout:0/444: fsync d9/f55 0 2024-03-24T00:48:50.659 INFO:tasks.workunit.client.1.smithi088.stdout:0/445: fsync d9/dc/d4f/f72 0 2024-03-24T00:48:50.664 INFO:tasks.workunit.client.1.smithi088.stdout:4/427: creat d5/d13/d18/d24/f90 x:0 0 0 2024-03-24T00:48:50.665 INFO:tasks.workunit.client.1.smithi088.stdout:7/376: mkdir d3/d32/d76 0 2024-03-24T00:48:50.672 INFO:tasks.workunit.client.1.smithi088.stdout:5/450: mkdir d1/d29/d8e 0 2024-03-24T00:48:50.672 INFO:tasks.workunit.client.0.smithi038.stdout:6/434: getdents d1/da/d1b/d25/d49/d4e 0 2024-03-24T00:48:50.672 INFO:tasks.workunit.client.0.smithi038.stdout:6/435: getdents d1/da/d1b 0 2024-03-24T00:48:50.673 INFO:tasks.workunit.client.1.smithi088.stdout:5/451: write d1/d29/f77 [1106604,103540] 0 2024-03-24T00:48:50.673 INFO:tasks.workunit.client.1.smithi088.stdout:5/452: stat d1/d19/d1e/f42 0 2024-03-24T00:48:50.674 INFO:tasks.workunit.client.1.smithi088.stdout:3/413: creat d3/d6/d19/d2a/f88 x:0 0 0 2024-03-24T00:48:50.676 INFO:tasks.workunit.client.0.smithi038.stdout:9/463: truncate d12/d28/f4d 2173354 0 2024-03-24T00:48:50.678 INFO:tasks.workunit.client.1.smithi088.stdout:6/451: rename db/d40/d56/c4e to db/d40/d56/c7f 0 2024-03-24T00:48:50.688 INFO:tasks.workunit.client.1.smithi088.stdout:2/419: mknod db/d21/d35/d49/c86 0 2024-03-24T00:48:50.696 INFO:tasks.workunit.client.1.smithi088.stdout:0/446: getdents d9/dc/d4f/d62 0 2024-03-24T00:48:50.696 INFO:tasks.workunit.client.1.smithi088.stdout:0/447: chown d9/dc/f52 42190575 1 2024-03-24T00:48:50.699 INFO:tasks.workunit.client.1.smithi088.stdout:1/468: sync 2024-03-24T00:48:50.699 INFO:tasks.workunit.client.1.smithi088.stdout:9/462: sync 2024-03-24T00:48:50.699 INFO:tasks.workunit.client.1.smithi088.stdout:9/463: write d1/d10/d29/d6d/f56 [600270,80990] 0 2024-03-24T00:48:50.703 INFO:tasks.workunit.client.1.smithi088.stdout:4/428: mkdir d5/d17/d91 0 2024-03-24T00:48:50.704 INFO:tasks.workunit.client.0.smithi038.stdout:9/464: dread d12/d28/d3a/d26/f3d [4194304,4194304] 0 2024-03-24T00:48:50.707 INFO:tasks.workunit.client.0.smithi038.stdout:1/444: dwrite da/dd/d2c/d5a/d74/f88 [0,4194304] 0 2024-03-24T00:48:50.708 INFO:tasks.workunit.client.1.smithi088.stdout:7/377: unlink d3/d32/d3f/d4b/f66 0 2024-03-24T00:48:50.710 INFO:tasks.workunit.client.1.smithi088.stdout:3/414: mkdir d3/de/d6f/d89 0 2024-03-24T00:48:50.713 INFO:tasks.workunit.client.0.smithi038.stdout:2/389: mkdir d4/d27/d8f 0 2024-03-24T00:48:50.713 INFO:tasks.workunit.client.0.smithi038.stdout:2/390: write d4/f13 [1313685,58083] 0 2024-03-24T00:48:50.714 INFO:tasks.workunit.client.0.smithi038.stdout:2/391: write d4/d27/d2f/d30/d3e/f6e [8568428,57323] 0 2024-03-24T00:48:50.714 INFO:tasks.workunit.client.0.smithi038.stdout:2/392: chown d4 637 1 2024-03-24T00:48:50.714 INFO:tasks.workunit.client.0.smithi038.stdout:3/424: dwrite d3/f8 [0,4194304] 0 2024-03-24T00:48:50.715 INFO:tasks.workunit.client.1.smithi088.stdout:6/452: link db/c1a db/d1d/d64/c80 0 2024-03-24T00:48:50.715 INFO:tasks.workunit.client.1.smithi088.stdout:6/453: write db/d1d/f48 [1275239,57352] 0 2024-03-24T00:48:50.716 INFO:tasks.workunit.client.1.smithi088.stdout:6/454: chown db/d1d/d64/d72/c7e 5 1 2024-03-24T00:48:50.716 INFO:tasks.workunit.client.1.smithi088.stdout:6/455: dread - db/d1d/d64/d72/f76 zero size 2024-03-24T00:48:50.717 INFO:tasks.workunit.client.0.smithi038.stdout:3/425: creat d3/d9/f8d x:0 0 0 2024-03-24T00:48:50.719 INFO:tasks.workunit.client.0.smithi038.stdout:3/426: write d3/f6 [3853531,17874] 0 2024-03-24T00:48:50.719 INFO:tasks.workunit.client.0.smithi038.stdout:3/427: chown d3/d9/dc/df/f31 42 1 2024-03-24T00:48:50.721 INFO:tasks.workunit.client.1.smithi088.stdout:4/429: dread d5/d13/d18/d24/f30 [0,4194304] 0 2024-03-24T00:48:50.725 INFO:tasks.workunit.client.1.smithi088.stdout:8/454: rename d9/dc/d11/c26 to d9/dc/d11/d24/d29/d36/c9d 0 2024-03-24T00:48:50.727 INFO:tasks.workunit.client.1.smithi088.stdout:1/469: getdents d6/d27/d67 0 2024-03-24T00:48:50.731 INFO:tasks.workunit.client.1.smithi088.stdout:9/464: symlink d1/d10/d11/d23/l8f 0 2024-03-24T00:48:50.731 INFO:tasks.workunit.client.1.smithi088.stdout:7/378: rmdir d3/d32/d34 39 2024-03-24T00:48:50.733 INFO:tasks.workunit.client.0.smithi038.stdout:8/409: mknod d0/d1d/c90 0 2024-03-24T00:48:50.733 INFO:tasks.workunit.client.0.smithi038.stdout:8/410: dread - d0/d1d/d20/d3c/f8d zero size 2024-03-24T00:48:50.733 INFO:tasks.workunit.client.0.smithi038.stdout:8/411: dread - d0/d1d/d20/d3c/f8d zero size 2024-03-24T00:48:50.734 INFO:tasks.workunit.client.1.smithi088.stdout:3/415: creat d3/d6/d27/d56/d6e/f8a x:0 0 0 2024-03-24T00:48:50.737 INFO:tasks.workunit.client.0.smithi038.stdout:5/394: mkdir d6/d86 0 2024-03-24T00:48:50.742 INFO:tasks.workunit.client.0.smithi038.stdout:0/337: getdents d0/d12 0 2024-03-24T00:48:50.751 INFO:tasks.workunit.client.1.smithi088.stdout:0/448: getdents d9/dc/d4f/d63 0 2024-03-24T00:48:50.751 INFO:tasks.workunit.client.1.smithi088.stdout:1/470: creat d6/d27/f8e x:0 0 0 2024-03-24T00:48:50.751 INFO:tasks.workunit.client.1.smithi088.stdout:0/449: truncate d9/dc/de/d19/d44/f7d 395469 0 2024-03-24T00:48:50.751 INFO:tasks.workunit.client.1.smithi088.stdout:0/450: chown d9/dc/de/d19/d44/d46/d65/d6f 26 1 2024-03-24T00:48:50.751 INFO:tasks.workunit.client.1.smithi088.stdout:1/471: write d6/f2a [720939,45641] 0 2024-03-24T00:48:50.751 INFO:tasks.workunit.client.1.smithi088.stdout:1/472: fdatasync d6/d27/d1e/d58/f81 0 2024-03-24T00:48:50.751 INFO:tasks.workunit.client.1.smithi088.stdout:1/473: write d6/f10 [4787293,25937] 0 2024-03-24T00:48:50.752 INFO:tasks.workunit.client.0.smithi038.stdout:6/436: stat d1/da/d1b/d25/c31 0 2024-03-24T00:48:50.753 INFO:tasks.workunit.client.0.smithi038.stdout:0/338: dread d0/d2d/f6f [0,4194304] 0 2024-03-24T00:48:50.757 INFO:tasks.workunit.client.1.smithi088.stdout:9/465: symlink d1/d22/d72/l90 0 2024-03-24T00:48:50.758 INFO:tasks.workunit.client.1.smithi088.stdout:9/466: dread - d1/d22/d27/f83 zero size 2024-03-24T00:48:50.758 INFO:tasks.workunit.client.0.smithi038.stdout:3/428: write d3/d9/dc/df/d14/d19/f47 [1439939,4428] 0 2024-03-24T00:48:50.759 INFO:tasks.workunit.client.1.smithi088.stdout:2/420: dwrite db/dc/f2b [0,4194304] 0 2024-03-24T00:48:50.770 INFO:tasks.workunit.client.0.smithi038.stdout:8/412: dread d0/d45/f55 [0,4194304] 0 2024-03-24T00:48:50.773 INFO:tasks.workunit.client.1.smithi088.stdout:6/456: dwrite db/dd/f2b [0,4194304] 0 2024-03-24T00:48:50.776 INFO:tasks.workunit.client.1.smithi088.stdout:3/416: symlink d3/d6/d27/l8b 0 2024-03-24T00:48:50.780 INFO:tasks.workunit.client.0.smithi038.stdout:6/437: dread d1/da/d1b/d25/d49/f7b [0,4194304] 0 2024-03-24T00:48:50.783 INFO:tasks.workunit.client.0.smithi038.stdout:1/445: write da/dd/d2c/d5a/d74/f88 [1532443,79803] 0 2024-03-24T00:48:50.789 INFO:tasks.workunit.client.1.smithi088.stdout:1/474: symlink d6/l8f 0 2024-03-24T00:48:50.789 INFO:tasks.workunit.client.1.smithi088.stdout:1/475: dread - d6/d27/d1e/d58/f78 zero size 2024-03-24T00:48:50.794 INFO:tasks.workunit.client.1.smithi088.stdout:4/430: dwrite d5/d16/f57 [0,4194304] 0 2024-03-24T00:48:50.795 INFO:tasks.workunit.client.1.smithi088.stdout:4/431: dread - d5/d13/d18/f19 zero size 2024-03-24T00:48:50.795 INFO:tasks.workunit.client.1.smithi088.stdout:7/379: rmdir d3/d32/d3f/d4b 39 2024-03-24T00:48:50.795 INFO:tasks.workunit.client.1.smithi088.stdout:7/380: dread - d3/d32/d3f/f61 zero size 2024-03-24T00:48:50.795 INFO:tasks.workunit.client.1.smithi088.stdout:7/381: chown d3/f2b 13782 1 2024-03-24T00:48:50.798 INFO:tasks.workunit.client.1.smithi088.stdout:9/467: creat d1/d10/d29/d85/f91 x:0 0 0 2024-03-24T00:48:50.803 INFO:tasks.workunit.client.1.smithi088.stdout:5/453: truncate d1/d19/d1e/d26/d69/f8c 539356 0 2024-03-24T00:48:50.805 INFO:tasks.workunit.client.1.smithi088.stdout:2/421: unlink db/d21/f76 0 2024-03-24T00:48:50.821 INFO:tasks.workunit.client.0.smithi038.stdout:0/339: creat d0/d12/f70 x:0 0 0 2024-03-24T00:48:50.822 INFO:tasks.workunit.client.1.smithi088.stdout:0/451: dwrite d9/dc/de/d19/d44/d46/d65/f58 [4194304,4194304] 0 2024-03-24T00:48:50.822 INFO:tasks.workunit.client.1.smithi088.stdout:0/452: chown d9/dc/de/d19/d44/d46/d65/d2a/c71 230654263 1 2024-03-24T00:48:50.823 INFO:tasks.workunit.client.0.smithi038.stdout:8/413: write d0/d1d/d27/d4d/f75 [2984577,82243] 0 2024-03-24T00:48:50.824 INFO:tasks.workunit.client.0.smithi038.stdout:8/414: fdatasync d0/d1d/d27/d4d/f70 0 2024-03-24T00:48:50.824 INFO:tasks.workunit.client.0.smithi038.stdout:6/438: rename d1/da/d1b/d1d/l4d to d1/da/d1b/d1d/d3c/l88 0 2024-03-24T00:48:50.824 INFO:tasks.workunit.client.0.smithi038.stdout:6/439: dread - d1/da/d1b/d25/d49/d72/f76 zero size 2024-03-24T00:48:50.824 INFO:tasks.workunit.client.0.smithi038.stdout:6/440: write d1/da/d1b/d25/d49/f7b [1893681,72210] 0 2024-03-24T00:48:50.826 INFO:tasks.workunit.client.1.smithi088.stdout:6/457: symlink db/dd/d61/d6d/d74/l81 0 2024-03-24T00:48:50.829 INFO:tasks.workunit.client.1.smithi088.stdout:3/417: symlink d3/d6/d27/d56/d6e/l8c 0 2024-03-24T00:48:50.831 INFO:tasks.workunit.client.0.smithi038.stdout:1/446: mknod da/dd/d7a/d82/c96 0 2024-03-24T00:48:50.831 INFO:tasks.workunit.client.0.smithi038.stdout:1/447: truncate da/f35 1391544 0 2024-03-24T00:48:50.833 INFO:tasks.workunit.client.0.smithi038.stdout:0/340: mknod d0/d12/d60/d28/c71 0 2024-03-24T00:48:50.835 INFO:tasks.workunit.client.0.smithi038.stdout:2/393: dwrite d4/d18/d38/f57 [0,4194304] 0 2024-03-24T00:48:50.835 INFO:tasks.workunit.client.0.smithi038.stdout:2/394: fsync d4/d18/d38/f45 0 2024-03-24T00:48:50.841 INFO:tasks.workunit.client.0.smithi038.stdout:5/395: dwrite d6/f18 [0,4194304] 0 2024-03-24T00:48:50.841 INFO:tasks.workunit.client.0.smithi038.stdout:8/415: creat d0/d45/d37/d5f/d68/f91 x:0 0 0 2024-03-24T00:48:50.842 INFO:tasks.workunit.client.0.smithi038.stdout:8/416: write d0/d45/d37/f87 [57028,10911] 0 2024-03-24T00:48:50.844 INFO:tasks.workunit.client.1.smithi088.stdout:1/476: dwrite d6/f68 [0,4194304] 0 2024-03-24T00:48:50.844 INFO:tasks.workunit.client.1.smithi088.stdout:1/477: stat d6/d27/d1e/d58/f51 0 2024-03-24T00:48:50.844 INFO:tasks.workunit.client.1.smithi088.stdout:1/478: dread - d6/d6e/f79 zero size 2024-03-24T00:48:50.857 INFO:tasks.workunit.client.0.smithi038.stdout:1/448: dread da/dd/d10/f84 [0,4194304] 0 2024-03-24T00:48:50.857 INFO:tasks.workunit.client.0.smithi038.stdout:0/341: truncate d0/d12/d60/d28/f39 3729305 0 2024-03-24T00:48:50.858 INFO:tasks.workunit.client.0.smithi038.stdout:0/342: dread - d0/d1b/d2c/f6b zero size 2024-03-24T00:48:50.864 INFO:tasks.workunit.client.1.smithi088.stdout:4/432: mkdir d5/d7/d92 0 2024-03-24T00:48:50.868 INFO:tasks.workunit.client.0.smithi038.stdout:9/465: dwrite d12/d41/f7d [4194304,4194304] 0 2024-03-24T00:48:50.868 INFO:tasks.workunit.client.0.smithi038.stdout:3/429: dread d3/d9/dc/d51/f7d [0,4194304] 0 2024-03-24T00:48:50.869 INFO:tasks.workunit.client.0.smithi038.stdout:5/396: mkdir d6/d19/d20/d42/d43/d87 0 2024-03-24T00:48:50.869 INFO:tasks.workunit.client.0.smithi038.stdout:5/397: readlink d6/d19/d20/d42/d43/d50/l51 0 2024-03-24T00:48:50.870 INFO:tasks.workunit.client.1.smithi088.stdout:9/468: getdents d1/d15/d47 0 2024-03-24T00:48:50.873 INFO:tasks.workunit.client.0.smithi038.stdout:8/417: truncate d0/d1d/d20/d3c/f47 3358063 0 2024-03-24T00:48:50.873 INFO:tasks.workunit.client.0.smithi038.stdout:8/418: write d0/f76 [1024163,61824] 0 2024-03-24T00:48:50.873 INFO:tasks.workunit.client.0.smithi038.stdout:8/419: write d0/d1d/d27/d4d/f70 [1440634,28008] 0 2024-03-24T00:48:50.875 INFO:tasks.workunit.client.1.smithi088.stdout:2/422: creat db/f87 x:0 0 0 2024-03-24T00:48:50.876 INFO:tasks.workunit.client.0.smithi038.stdout:0/343: symlink d0/d2d/l72 0 2024-03-24T00:48:50.878 INFO:tasks.workunit.client.1.smithi088.stdout:0/453: mkdir d9/dc/de/d19/d8d 0 2024-03-24T00:48:50.878 INFO:tasks.workunit.client.1.smithi088.stdout:0/454: readlink d9/l16 0 2024-03-24T00:48:50.881 INFO:tasks.workunit.client.1.smithi088.stdout:6/458: symlink db/d40/d55/l82 0 2024-03-24T00:48:50.885 INFO:tasks.workunit.client.0.smithi038.stdout:2/395: dwrite d4/d27/d2f/d30/d3e/f8b [0,4194304] 0 2024-03-24T00:48:50.885 INFO:tasks.workunit.client.0.smithi038.stdout:2/396: fsync d4/fa 0 2024-03-24T00:48:50.890 INFO:tasks.workunit.client.1.smithi088.stdout:1/479: mkdir d6/d27/d1e/d5b/d5e/d90 0 2024-03-24T00:48:50.890 INFO:tasks.workunit.client.1.smithi088.stdout:1/480: chown d6/d23/f73 159 1 2024-03-24T00:48:50.890 INFO:tasks.workunit.client.1.smithi088.stdout:1/481: chown d6/d27/d1e/l59 57723 1 2024-03-24T00:48:50.891 INFO:tasks.workunit.client.0.smithi038.stdout:6/441: dwrite d1/f6 [0,4194304] 0 2024-03-24T00:48:50.891 INFO:tasks.workunit.client.0.smithi038.stdout:6/442: stat d1/da/d1b/d25/d49/l6f 0 2024-03-24T00:48:50.894 INFO:tasks.workunit.client.1.smithi088.stdout:7/382: rename d3/d32/d3f/l68 to d3/l77 0 2024-03-24T00:48:50.894 INFO:tasks.workunit.client.1.smithi088.stdout:7/383: truncate d3/d32/f42 574020 0 2024-03-24T00:48:50.896 INFO:tasks.workunit.client.1.smithi088.stdout:9/469: mknod d1/d22/d72/c92 0 2024-03-24T00:48:50.896 INFO:tasks.workunit.client.1.smithi088.stdout:9/470: creat d1/d22/f93 x:0 0 0 2024-03-24T00:48:50.898 INFO:tasks.workunit.client.1.smithi088.stdout:2/423: mkdir db/d54/d88 0 2024-03-24T00:48:50.898 INFO:tasks.workunit.client.1.smithi088.stdout:2/424: creat db/f89 x:0 0 0 2024-03-24T00:48:50.899 INFO:tasks.workunit.client.0.smithi038.stdout:9/466: creat d12/d41/d57/f98 x:0 0 0 2024-03-24T00:48:50.901 INFO:tasks.workunit.client.1.smithi088.stdout:0/455: mknod d9/dc/de/d7e/c8e 0 2024-03-24T00:48:50.901 INFO:tasks.workunit.client.1.smithi088.stdout:0/456: chown d9/dc/de/d19/d44/d46/d59/f70 410656 1 2024-03-24T00:48:50.902 INFO:tasks.workunit.client.0.smithi038.stdout:3/430: symlink d3/d9/dc/df/d14/l8e 0 2024-03-24T00:48:50.902 INFO:tasks.workunit.client.0.smithi038.stdout:3/431: dread - d3/d30/f7a zero size 2024-03-24T00:48:50.902 INFO:tasks.workunit.client.0.smithi038.stdout:3/432: read - d3/d30/d70/f8a zero size 2024-03-24T00:48:50.903 INFO:tasks.workunit.client.1.smithi088.stdout:3/418: rmdir d3/d32 39 2024-03-24T00:48:50.905 INFO:tasks.workunit.client.0.smithi038.stdout:5/398: rename d6/d16/d21/f85 to d6/d19/d20/d33/d6f/f88 0 2024-03-24T00:48:50.915 INFO:tasks.workunit.client.1.smithi088.stdout:4/433: dread d5/d7/d72/f71 [0,4194304] 0 2024-03-24T00:48:50.915 INFO:tasks.workunit.client.1.smithi088.stdout:4/434: chown d5/d13/f12 0 1 2024-03-24T00:48:50.915 INFO:tasks.workunit.client.0.smithi038.stdout:8/420: creat d0/d10/f92 x:0 0 0 2024-03-24T00:48:50.917 INFO:tasks.workunit.client.1.smithi088.stdout:8/455: sync 2024-03-24T00:48:50.921 INFO:tasks.workunit.client.0.smithi038.stdout:4/357: sync 2024-03-24T00:48:50.921 INFO:tasks.workunit.client.0.smithi038.stdout:7/409: sync 2024-03-24T00:48:50.925 INFO:tasks.workunit.client.1.smithi088.stdout:0/457: mkdir d9/d26/d8f 0 2024-03-24T00:48:50.925 INFO:tasks.workunit.client.0.smithi038.stdout:1/449: dwrite da/dd/d22/d5c/f7f [0,4194304] 0 2024-03-24T00:48:50.935 INFO:tasks.workunit.client.0.smithi038.stdout:9/467: mkdir d12/d28/d71/d99 0 2024-03-24T00:48:50.935 INFO:tasks.workunit.client.0.smithi038.stdout:6/443: rmdir d1/da/d61 39 2024-03-24T00:48:50.935 INFO:tasks.workunit.client.0.smithi038.stdout:3/433: symlink d3/d9/dc/df/l8f 0 2024-03-24T00:48:50.935 INFO:tasks.workunit.client.0.smithi038.stdout:2/397: rename d4/d18/d26/d37/c5e to d4/d27/d54/d67/c90 0 2024-03-24T00:48:50.935 INFO:tasks.workunit.client.0.smithi038.stdout:2/398: readlink d4/d27/d2f/d30/l34 0 2024-03-24T00:48:50.936 INFO:tasks.workunit.client.0.smithi038.stdout:2/399: fdatasync d4/d27/d54/d53/f64 0 2024-03-24T00:48:50.936 INFO:tasks.workunit.client.0.smithi038.stdout:5/399: dread d6/f18 [0,4194304] 0 2024-03-24T00:48:50.938 INFO:tasks.workunit.client.1.smithi088.stdout:9/471: link d1/d22/d27/d39/f86 d1/d10/d77/d8a/f94 0 2024-03-24T00:48:50.938 INFO:tasks.workunit.client.1.smithi088.stdout:9/472: chown d1/d15 513210 1 2024-03-24T00:48:50.941 INFO:tasks.workunit.client.0.smithi038.stdout:8/421: truncate d0/d45/f89 71321 0 2024-03-24T00:48:50.941 INFO:tasks.workunit.client.0.smithi038.stdout:8/422: fsync d0/d1d/d20/d3c/f8d 0 2024-03-24T00:48:50.942 INFO:tasks.workunit.client.1.smithi088.stdout:5/454: rename d1/d19/d1e/d26 to d1/d29/d3f/d88/d8f 0 2024-03-24T00:48:50.944 INFO:tasks.workunit.client.0.smithi038.stdout:0/344: getdents d0/d12/d42/d5e 0 2024-03-24T00:48:50.944 INFO:tasks.workunit.client.1.smithi088.stdout:9/473: mknod d1/d10/d11/d23/c95 0 2024-03-24T00:48:50.947 INFO:tasks.workunit.client.1.smithi088.stdout:1/482: dwrite d6/d27/d1e/d58/f51 [0,4194304] 0 2024-03-24T00:48:50.947 INFO:tasks.workunit.client.1.smithi088.stdout:1/483: chown d6/d23/f4a 0 1 2024-03-24T00:48:50.947 INFO:tasks.workunit.client.1.smithi088.stdout:1/484: fsync d6/d27/d1e/d5b/d5e/f60 0 2024-03-24T00:48:50.950 INFO:tasks.workunit.client.0.smithi038.stdout:7/410: creat d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/d3a/f97 x:0 0 0 2024-03-24T00:48:50.957 INFO:tasks.workunit.client.1.smithi088.stdout:7/384: read d3/fd [334315,65830] 0 2024-03-24T00:48:50.957 INFO:tasks.workunit.client.1.smithi088.stdout:7/385: readlink d3/d32/l49 0 2024-03-24T00:48:50.959 INFO:tasks.workunit.client.1.smithi088.stdout:5/455: link d1/d29/d3f/c7d d1/dd/d6f/c90 0 2024-03-24T00:48:50.961 INFO:tasks.workunit.client.0.smithi038.stdout:4/358: mkdir da/d12/d37/dd/d15/d4c/d70 0 2024-03-24T00:48:50.971 INFO:tasks.workunit.client.1.smithi088.stdout:1/485: rmdir d6/d27/d1e/d5b/d5d/d61 39 2024-03-24T00:48:50.972 INFO:tasks.workunit.client.0.smithi038.stdout:8/423: dread d0/d1d/d27/d4d/f53 [0,4194304] 0 2024-03-24T00:48:50.975 INFO:tasks.workunit.client.0.smithi038.stdout:3/434: truncate d3/f25 5051617 0 2024-03-24T00:48:50.976 INFO:tasks.workunit.client.0.smithi038.stdout:3/435: fdatasync d3/d30/f33 0 2024-03-24T00:48:50.977 INFO:tasks.workunit.client.1.smithi088.stdout:7/386: mknod d3/d31/d56/c78 0 2024-03-24T00:48:50.980 INFO:tasks.workunit.client.1.smithi088.stdout:0/458: dwrite d9/dc/de/d19/d44/f4a [4194304,4194304] 0 2024-03-24T00:48:50.981 INFO:tasks.workunit.client.0.smithi038.stdout:2/400: unlink d4/f23 0 2024-03-24T00:48:50.981 INFO:tasks.workunit.client.0.smithi038.stdout:5/400: truncate d6/d19/d20/d42/d43/d4c/f74 1365976 0 2024-03-24T00:48:50.982 INFO:tasks.workunit.client.1.smithi088.stdout:6/459: rename db/d1d/d64 to db/d3d/d83 0 2024-03-24T00:48:50.982 INFO:tasks.workunit.client.1.smithi088.stdout:6/460: chown db/c1a 0 1 2024-03-24T00:48:50.983 INFO:tasks.workunit.client.0.smithi038.stdout:6/444: rename d1/da/f14 to d1/da/f89 0 2024-03-24T00:48:50.989 INFO:tasks.workunit.client.1.smithi088.stdout:8/456: write d9/dc/d11/d24/d29/d5b/f34 [4855458,35364] 0 2024-03-24T00:48:50.989 INFO:tasks.workunit.client.1.smithi088.stdout:8/457: chown d9/dc/d11/d41/d56/f66 1348 1 2024-03-24T00:48:50.989 INFO:tasks.workunit.client.1.smithi088.stdout:8/458: write d9/dc/d11/d24/d29/d5b/d6f/f83 [353684,77315] 0 2024-03-24T00:48:50.989 INFO:tasks.workunit.client.1.smithi088.stdout:8/459: chown d9/dc/d11/d24/d29/d39/f93 7 1 2024-03-24T00:48:50.990 INFO:tasks.workunit.client.0.smithi038.stdout:0/345: link d0/l9 d0/l73 0 2024-03-24T00:48:50.990 INFO:tasks.workunit.client.0.smithi038.stdout:0/346: readlink d0/l73 0 2024-03-24T00:48:50.990 INFO:tasks.workunit.client.0.smithi038.stdout:0/347: write d0/d12/d42/f61 [1047065,15350] 0 2024-03-24T00:48:50.990 INFO:tasks.workunit.client.0.smithi038.stdout:0/348: creat d0/d12/d42/f74 x:0 0 0 2024-03-24T00:48:50.995 INFO:tasks.workunit.client.1.smithi088.stdout:7/387: truncate d3/f24 351058 0 2024-03-24T00:48:50.995 INFO:tasks.workunit.client.1.smithi088.stdout:7/388: truncate d3/d32/d34/f72 995703 0 2024-03-24T00:48:50.997 INFO:tasks.workunit.client.0.smithi038.stdout:7/411: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d98 0 2024-03-24T00:48:50.998 INFO:tasks.workunit.client.0.smithi038.stdout:1/450: dwrite da/dd/f1c [0,4194304] 0 2024-03-24T00:48:50.999 INFO:tasks.workunit.client.1.smithi088.stdout:0/459: mknod d9/dc/de/d19/d44/d46/d65/c90 0 2024-03-24T00:48:50.999 INFO:tasks.workunit.client.1.smithi088.stdout:0/460: fdatasync d9/dc/de/f14 0 2024-03-24T00:48:50.999 INFO:tasks.workunit.client.1.smithi088.stdout:7/389: write d3/d32/f42 [482913,65605] 0 2024-03-24T00:48:51.001 INFO:tasks.workunit.client.1.smithi088.stdout:4/435: dwrite d5/f4b [0,4194304] 0 2024-03-24T00:48:51.002 INFO:tasks.workunit.client.0.smithi038.stdout:8/424: unlink d0/d45/d66/c30 0 2024-03-24T00:48:51.002 INFO:tasks.workunit.client.0.smithi038.stdout:8/425: creat d0/d45/d66/f93 x:0 0 0 2024-03-24T00:48:51.002 INFO:tasks.workunit.client.1.smithi088.stdout:2/425: rename db/l6c to db/dc/l8a 0 2024-03-24T00:48:51.009 INFO:tasks.workunit.client.1.smithi088.stdout:9/474: dwrite d1/d10/d29/d6d/f57 [0,4194304] 0 2024-03-24T00:48:51.011 INFO:tasks.workunit.client.1.smithi088.stdout:6/461: mknod db/d1d/d50/c84 0 2024-03-24T00:48:51.011 INFO:tasks.workunit.client.1.smithi088.stdout:1/486: rmdir d6/d27/d1e/d5b/d5d 39 2024-03-24T00:48:51.011 INFO:tasks.workunit.client.1.smithi088.stdout:1/487: chown d6/d27/d1e/d3f/f57 44225 1 2024-03-24T00:48:51.011 INFO:tasks.workunit.client.1.smithi088.stdout:8/460: read d9/dc/d11/d24/d29/d36/f4e [1552881,5636] 0 2024-03-24T00:48:51.012 INFO:tasks.workunit.client.0.smithi038.stdout:3/436: write d3/d9/dc/df/f12 [3581597,54547] 0 2024-03-24T00:48:51.013 INFO:tasks.workunit.client.0.smithi038.stdout:2/401: rename d4/fc to d4/d27/f91 0 2024-03-24T00:48:51.013 INFO:tasks.workunit.client.0.smithi038.stdout:5/401: mknod d6/d19/d20/d70/c89 0 2024-03-24T00:48:51.017 INFO:tasks.workunit.client.0.smithi038.stdout:4/359: dwrite da/d27/f69 [0,4194304] 0 2024-03-24T00:48:51.017 INFO:tasks.workunit.client.0.smithi038.stdout:4/360: dread - da/d12/d37/dd/d15/d6d/f4b zero size 2024-03-24T00:48:51.025 INFO:tasks.workunit.client.0.smithi038.stdout:6/445: truncate d1/f7 2931553 0 2024-03-24T00:48:51.025 INFO:tasks.workunit.client.0.smithi038.stdout:6/446: dread - d1/da/d1b/d2f/f3d zero size 2024-03-24T00:48:51.029 INFO:tasks.workunit.client.1.smithi088.stdout:0/461: truncate d9/dc/f52 540682 0 2024-03-24T00:48:51.029 INFO:tasks.workunit.client.1.smithi088.stdout:0/462: fdatasync d9/dc/f49 0 2024-03-24T00:48:51.030 INFO:tasks.workunit.client.1.smithi088.stdout:7/390: chown d3/f28 21422626 1 2024-03-24T00:48:51.035 INFO:tasks.workunit.client.0.smithi038.stdout:7/412: mkdir d2/d2c/d99 0 2024-03-24T00:48:51.036 INFO:tasks.workunit.client.0.smithi038.stdout:0/349: dwrite d0/d1b/d2c/f3b [0,4194304] 0 2024-03-24T00:48:51.038 INFO:tasks.workunit.client.0.smithi038.stdout:3/437: fsync d3/d9/dc/df/f12 0 2024-03-24T00:48:51.038 INFO:tasks.workunit.client.0.smithi038.stdout:1/451: creat da/dd/d2c/d5a/d74/f97 x:0 0 0 2024-03-24T00:48:51.042 INFO:tasks.workunit.client.1.smithi088.stdout:3/419: rename d3/de/d6f/d85/f25 to d3/d6/d27/f8d 0 2024-03-24T00:48:51.045 INFO:tasks.workunit.client.0.smithi038.stdout:5/402: link d6/d16/d14/c31 d6/d16/c8a 0 2024-03-24T00:48:51.046 INFO:tasks.workunit.client.1.smithi088.stdout:6/462: creat db/d1d/d23/d3c/d43/f85 x:0 0 0 2024-03-24T00:48:51.048 INFO:tasks.workunit.client.0.smithi038.stdout:8/426: rename d0/d1d/d20/d3c/f8d to d0/d15/f94 0 2024-03-24T00:48:51.049 INFO:tasks.workunit.client.1.smithi088.stdout:1/488: symlink d6/d27/d1e/d5b/l91 0 2024-03-24T00:48:51.051 INFO:tasks.workunit.client.1.smithi088.stdout:8/461: symlink d9/dc/d11/d24/d43/d4a/l9e 0 2024-03-24T00:48:51.051 INFO:tasks.workunit.client.1.smithi088.stdout:8/462: fsync d9/d6b/f6c 0 2024-03-24T00:48:51.053 INFO:tasks.workunit.client.0.smithi038.stdout:7/413: dread d2/f1f [0,4194304] 0 2024-03-24T00:48:51.053 INFO:tasks.workunit.client.0.smithi038.stdout:7/414: readlink l1 0 2024-03-24T00:48:51.061 INFO:tasks.workunit.client.1.smithi088.stdout:3/420: dread d3/d6/d27/d56/f82 [0,4194304] 0 2024-03-24T00:48:51.062 INFO:tasks.workunit.client.1.smithi088.stdout:3/421: fdatasync d3/d6/d27/d3c/f30 0 2024-03-24T00:48:51.079 INFO:tasks.workunit.client.1.smithi088.stdout:3/422: dread d3/d1a/d1e/f28 [0,4194304] 0 2024-03-24T00:48:51.091 INFO:tasks.workunit.client.0.smithi038.stdout:4/361: dwrite da/d12/d24/f26 [0,4194304] 0 2024-03-24T00:48:51.094 INFO:tasks.workunit.client.0.smithi038.stdout:2/402: dwrite d4/d27/f91 [0,4194304] 0 2024-03-24T00:48:51.097 INFO:tasks.workunit.client.0.smithi038.stdout:1/452: dwrite da/dd/d10/d34/f39 [0,4194304] 0 2024-03-24T00:48:51.121 INFO:tasks.workunit.client.0.smithi038.stdout:6/447: mknod d1/d37/d84/c8a 0 2024-03-24T00:48:51.127 INFO:tasks.workunit.client.0.smithi038.stdout:3/438: rmdir d3/d9/dc/df/d14/d19/d82/d86 0 2024-03-24T00:48:51.127 INFO:tasks.workunit.client.0.smithi038.stdout:3/439: dread - d3/d30/f7a zero size 2024-03-24T00:48:51.131 INFO:tasks.workunit.client.0.smithi038.stdout:5/403: unlink d6/d19/d20/d33/c69 0 2024-03-24T00:48:51.134 INFO:tasks.workunit.client.0.smithi038.stdout:7/415: stat d2/d9/d16/l1e 0 2024-03-24T00:48:51.134 INFO:tasks.workunit.client.1.smithi088.stdout:0/463: mkdir d9/dc/de/d7e/d91 0 2024-03-24T00:48:51.136 INFO:tasks.workunit.client.1.smithi088.stdout:7/391: fsync d3/f29 0 2024-03-24T00:48:51.140 INFO:tasks.workunit.client.1.smithi088.stdout:5/456: rename d1/d43/d4e/f6e to d1/d10/d1b/d2e/f91 0 2024-03-24T00:48:51.140 INFO:tasks.workunit.client.1.smithi088.stdout:5/457: readlink d1/d10/d1b/d2f/d57/l61 0 2024-03-24T00:48:51.144 INFO:tasks.workunit.client.1.smithi088.stdout:2/426: mknod db/d21/c8b 0 2024-03-24T00:48:51.147 INFO:tasks.workunit.client.1.smithi088.stdout:6/463: getdents db/dd/d61/d6d/d74 0 2024-03-24T00:48:51.148 INFO:tasks.workunit.client.1.smithi088.stdout:6/464: write db/d1d/f62 [473175,121216] 0 2024-03-24T00:48:51.148 INFO:tasks.workunit.client.0.smithi038.stdout:4/362: rmdir da 39 2024-03-24T00:48:51.149 INFO:tasks.workunit.client.1.smithi088.stdout:1/489: creat d6/d27/d67/d88/f92 x:0 0 0 2024-03-24T00:48:51.150 INFO:tasks.workunit.client.1.smithi088.stdout:1/490: readlink l4 0 2024-03-24T00:48:51.152 INFO:tasks.workunit.client.1.smithi088.stdout:8/463: truncate d9/f57 3216011 0 2024-03-24T00:48:51.152 INFO:tasks.workunit.client.1.smithi088.stdout:8/464: fdatasync d9/dc/d11/d41/d54/d74/f90 0 2024-03-24T00:48:51.155 INFO:tasks.workunit.client.0.smithi038.stdout:2/403: unlink d4/d76/f77 0 2024-03-24T00:48:51.156 INFO:tasks.workunit.client.0.smithi038.stdout:2/404: dread - d4/d27/d54/d53/f6b zero size 2024-03-24T00:48:51.162 INFO:tasks.workunit.client.1.smithi088.stdout:0/464: creat d9/dc/de/d19/d44/d46/d65/d2a/f92 x:0 0 0 2024-03-24T00:48:51.162 INFO:tasks.workunit.client.1.smithi088.stdout:0/465: stat d9/f80 0 2024-03-24T00:48:51.168 INFO:tasks.workunit.client.1.smithi088.stdout:2/427: dread db/d21/d4a/d4e/f64 [0,4194304] 0 2024-03-24T00:48:51.172 INFO:tasks.workunit.client.1.smithi088.stdout:7/392: mkdir d3/d32/d3f/d4b/d74/d79 0 2024-03-24T00:48:51.173 INFO:tasks.workunit.client.0.smithi038.stdout:1/453: rename da/dd/d10 to da/dd/d7a/d82/d98 0 2024-03-24T00:48:51.174 INFO:tasks.workunit.client.1.smithi088.stdout:8/465: dread d9/dc/d11/d24/d43/f50 [0,4194304] 0 2024-03-24T00:48:51.182 INFO:tasks.workunit.client.0.smithi038.stdout:6/448: link d1/da/d61/d69/f6e d1/d37/f8b 0 2024-03-24T00:48:51.183 INFO:tasks.workunit.client.1.smithi088.stdout:6/465: dread db/d1d/d50/f6f [0,4194304] 0 2024-03-24T00:48:51.183 INFO:tasks.workunit.client.1.smithi088.stdout:4/436: rename d5/d7/l80 to d5/d13/l93 0 2024-03-24T00:48:51.183 INFO:tasks.workunit.client.1.smithi088.stdout:4/437: readlink d5/d7/d15/d74/l8f 0 2024-03-24T00:48:51.183 INFO:tasks.workunit.client.1.smithi088.stdout:4/438: chown d5/d16/d1a/f3e 28 1 2024-03-24T00:48:51.183 INFO:tasks.workunit.client.0.smithi038.stdout:7/416: mkdir d2/d2c/d9a 0 2024-03-24T00:48:51.186 INFO:tasks.workunit.client.0.smithi038.stdout:4/363: mkdir da/d12/d29/d71 0 2024-03-24T00:48:51.188 INFO:tasks.workunit.client.1.smithi088.stdout:1/491: getdents d6/d27/d1e/d5b/d5d/d61 0 2024-03-24T00:48:51.189 INFO:tasks.workunit.client.1.smithi088.stdout:1/492: write d6/d27/d1e/d3f/f4c [2173257,82346] 0 2024-03-24T00:48:51.194 INFO:tasks.workunit.client.0.smithi038.stdout:6/449: mknod d1/da/d1b/d25/d49/d4e/d51/c8c 0 2024-03-24T00:48:51.199 INFO:tasks.workunit.client.1.smithi088.stdout:2/428: link db/dc/f2b db/d21/d35/d49/f8c 0 2024-03-24T00:48:51.199 INFO:tasks.workunit.client.1.smithi088.stdout:2/429: truncate db/f87 98537 0 2024-03-24T00:48:51.199 INFO:tasks.workunit.client.1.smithi088.stdout:2/430: readlink db/d21/d4a/l82 0 2024-03-24T00:48:51.200 INFO:tasks.workunit.client.0.smithi038.stdout:7/417: mkdir d2/d9/da/d9b 0 2024-03-24T00:48:51.200 INFO:tasks.workunit.client.0.smithi038.stdout:1/454: dread da/dd/d2c/d5a/f92 [0,4194304] 0 2024-03-24T00:48:51.201 INFO:tasks.workunit.client.1.smithi088.stdout:7/393: truncate d3/d31/f6e 303816 0 2024-03-24T00:48:51.201 INFO:tasks.workunit.client.1.smithi088.stdout:7/394: creat d3/d31/f7a x:0 0 0 2024-03-24T00:48:51.203 INFO:tasks.workunit.client.0.smithi038.stdout:6/450: symlink d1/da/d73/d75/l8d 0 2024-03-24T00:48:51.203 INFO:tasks.workunit.client.0.smithi038.stdout:5/404: dwrite d6/d23/f32 [0,4194304] 0 2024-03-24T00:48:51.204 INFO:tasks.workunit.client.0.smithi038.stdout:5/405: readlink d6/d16/d3c/l7e 0 2024-03-24T00:48:51.207 INFO:tasks.workunit.client.0.smithi038.stdout:7/418: symlink d2/d9/da/dc/d93/l9c 0 2024-03-24T00:48:51.208 INFO:tasks.workunit.client.0.smithi038.stdout:1/455: symlink da/dd/d4a/d76/l99 0 2024-03-24T00:48:51.208 INFO:tasks.workunit.client.1.smithi088.stdout:8/466: symlink d9/d6b/l9f 0 2024-03-24T00:48:51.208 INFO:tasks.workunit.client.1.smithi088.stdout:9/475: sync 2024-03-24T00:48:51.208 INFO:tasks.workunit.client.1.smithi088.stdout:3/423: rename d3/d6/f11 to d3/de/d6f/d85/d49/f8e 0 2024-03-24T00:48:51.213 INFO:tasks.workunit.client.1.smithi088.stdout:4/439: mkdir d5/d16/d94 0 2024-03-24T00:48:51.213 INFO:tasks.workunit.client.1.smithi088.stdout:6/466: write db/dd/f10 [2841047,47284] 0 2024-03-24T00:48:51.213 INFO:tasks.workunit.client.1.smithi088.stdout:1/493: read d6/d27/d1e/f22 [17060,130565] 0 2024-03-24T00:48:51.213 INFO:tasks.workunit.client.1.smithi088.stdout:2/431: read db/d21/d4a/d4e/f64 [67943,42551] 0 2024-03-24T00:48:51.214 INFO:tasks.workunit.client.0.smithi038.stdout:5/406: rename d6/f3f to d6/d19/d20/d42/d43/d68/f8b 0 2024-03-24T00:48:51.214 INFO:tasks.workunit.client.0.smithi038.stdout:5/407: dread - d6/d19/d20/d42/d43/d68/d6e/f7f zero size 2024-03-24T00:48:51.214 INFO:tasks.workunit.client.0.smithi038.stdout:7/419: creat d2/d9/da/d23/d2b/d5b/d72/d92/d5d/f9d x:0 0 0 2024-03-24T00:48:51.214 INFO:tasks.workunit.client.0.smithi038.stdout:7/420: chown d2/d2c 309232 1 2024-03-24T00:48:51.214 INFO:tasks.workunit.client.1.smithi088.stdout:5/458: rmdir d1/dd 39 2024-03-24T00:48:51.216 INFO:tasks.workunit.client.0.smithi038.stdout:1/456: creat da/dd/d7a/d82/d98/f9a x:0 0 0 2024-03-24T00:48:51.219 INFO:tasks.workunit.client.0.smithi038.stdout:7/421: link d2/d9/da/d23/d2b/d5b/d72/d92/d33/f30 d2/d9/da/d23/d2b/d5b/d72/d92/f9e 0 2024-03-24T00:48:51.219 INFO:tasks.workunit.client.0.smithi038.stdout:7/422: dread - d2/d9/da/d23/d2b/d5b/d72/d92/d33/f27 zero size 2024-03-24T00:48:51.221 INFO:tasks.workunit.client.0.smithi038.stdout:4/364: read da/d12/d37/dd/f3a [2863461,98637] 0 2024-03-24T00:48:51.221 INFO:tasks.workunit.client.0.smithi038.stdout:2/405: dwrite d4/d27/d2f/d30/f8d [0,4194304] 0 2024-03-24T00:48:51.222 INFO:tasks.workunit.client.0.smithi038.stdout:1/457: rename da/dd/d22/l26 to da/dd/d4a/d8b/l9b 0 2024-03-24T00:48:51.222 INFO:tasks.workunit.client.0.smithi038.stdout:1/458: fdatasync da/f35 0 2024-03-24T00:48:51.222 INFO:tasks.workunit.client.0.smithi038.stdout:1/459: write da/dd/d7a/d82/f8d [886830,39804] 0 2024-03-24T00:48:51.224 INFO:tasks.workunit.client.1.smithi088.stdout:7/395: unlink d3/d32/d3f/f40 0 2024-03-24T00:48:51.224 INFO:tasks.workunit.client.1.smithi088.stdout:7/396: write d3/f35 [4565083,123298] 0 2024-03-24T00:48:51.224 INFO:tasks.workunit.client.0.smithi038.stdout:7/423: creat d2/d9/da/d23/d2b/d5b/d72/d92/d98/f9f x:0 0 0 2024-03-24T00:48:51.229 INFO:tasks.workunit.client.1.smithi088.stdout:9/476: link d1/ff d1/d22/f96 0 2024-03-24T00:48:51.230 INFO:tasks.workunit.client.1.smithi088.stdout:8/467: write d9/dc/d11/d24/d29/d5b/f53 [2522396,24764] 0 2024-03-24T00:48:51.230 INFO:tasks.workunit.client.1.smithi088.stdout:8/468: fsync d9/f63 0 2024-03-24T00:48:51.230 INFO:tasks.workunit.client.1.smithi088.stdout:8/469: write d9/dc/d11/d41/d54/d74/f90 [592305,62460] 0 2024-03-24T00:48:51.230 INFO:tasks.workunit.client.1.smithi088.stdout:8/470: chown d9/d6b/f6c 0 1 2024-03-24T00:48:51.230 INFO:tasks.workunit.client.1.smithi088.stdout:8/471: stat d9/dc/d11/l1d 0 2024-03-24T00:48:51.231 INFO:tasks.workunit.client.1.smithi088.stdout:3/424: rename d3/d1a/d1e/d2f/c66 to d3/de/d6f/d85/c8f 0 2024-03-24T00:48:51.234 INFO:tasks.workunit.client.1.smithi088.stdout:1/494: getdents d6 0 2024-03-24T00:48:51.234 INFO:tasks.workunit.client.1.smithi088.stdout:1/495: truncate d6/d27/d1e/d58/f7c 161672 0 2024-03-24T00:48:51.235 INFO:tasks.workunit.client.0.smithi038.stdout:2/406: unlink d4/d18/d38/d63/c7d 0 2024-03-24T00:48:51.240 INFO:tasks.workunit.client.1.smithi088.stdout:4/440: write d5/d7/f44 [827955,55429] 0 2024-03-24T00:48:51.240 INFO:tasks.workunit.client.1.smithi088.stdout:5/459: creat d1/d19/d48/f92 x:0 0 0 2024-03-24T00:48:51.240 INFO:tasks.workunit.client.1.smithi088.stdout:8/472: creat d9/dc/d11/d24/d29/d36/d6a/d7b/fa0 x:0 0 0 2024-03-24T00:48:51.241 INFO:tasks.workunit.client.0.smithi038.stdout:2/407: rename d4/f16 to d4/d18/d38/d63/f92 0 2024-03-24T00:48:51.241 INFO:tasks.workunit.client.1.smithi088.stdout:3/425: mkdir d3/de/d6f/d85/d90 0 2024-03-24T00:48:51.242 INFO:tasks.workunit.client.0.smithi038.stdout:1/460: getdents da 0 2024-03-24T00:48:51.246 INFO:tasks.workunit.client.1.smithi088.stdout:0/466: dwrite d9/dc/de/d19/d44/d46/d65/f39 [0,4194304] 0 2024-03-24T00:48:51.246 INFO:tasks.workunit.client.1.smithi088.stdout:0/467: creat d9/dc/de/d19/f93 x:0 0 0 2024-03-24T00:48:51.246 INFO:tasks.workunit.client.1.smithi088.stdout:0/468: write d9/dc/d4f/d62/f6a [5029048,34010] 0 2024-03-24T00:48:51.246 INFO:tasks.workunit.client.1.smithi088.stdout:0/469: creat d9/dc/de/d19/d44/f94 x:0 0 0 2024-03-24T00:48:51.247 INFO:tasks.workunit.client.0.smithi038.stdout:7/424: getdents d2/d9 0 2024-03-24T00:48:51.248 INFO:tasks.workunit.client.0.smithi038.stdout:3/440: dwrite d3/d9/dc/df/d14/d19/f28 [0,4194304] 0 2024-03-24T00:48:51.255 INFO:tasks.workunit.client.1.smithi088.stdout:8/473: mknod d9/dc/d11/d24/ca1 0 2024-03-24T00:48:51.255 INFO:tasks.workunit.client.0.smithi038.stdout:1/461: rename da/dd/l57 to da/l9c 0 2024-03-24T00:48:51.255 INFO:tasks.workunit.client.1.smithi088.stdout:7/397: rename d3/d31/f6d to d3/d31/d56/f7b 0 2024-03-24T00:48:51.258 INFO:tasks.workunit.client.1.smithi088.stdout:6/467: dread db/d1d/d23/d3c/d43/f28 [0,4194304] 0 2024-03-24T00:48:51.258 INFO:tasks.workunit.client.1.smithi088.stdout:6/468: dread - db/d1d/d23/d3c/d43/f85 zero size 2024-03-24T00:48:51.259 INFO:tasks.workunit.client.1.smithi088.stdout:3/426: mknod d3/de/d6f/c91 0 2024-03-24T00:48:51.260 INFO:tasks.workunit.client.0.smithi038.stdout:7/425: creat d2/d9/da/d23/d2b/d5b/d72/d92/d65/fa0 x:0 0 0 2024-03-24T00:48:51.260 INFO:tasks.workunit.client.1.smithi088.stdout:5/460: truncate d1/dd/f13 7283256 0 2024-03-24T00:48:51.262 INFO:tasks.workunit.client.1.smithi088.stdout:4/441: write d5/d13/d18/d24/d3d/f55 [3890694,88446] 0 2024-03-24T00:48:51.262 INFO:tasks.workunit.client.1.smithi088.stdout:4/442: truncate d5/d16/d37/f6f 43628 0 2024-03-24T00:48:51.264 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:51 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:51.264 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:51 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:51.267 INFO:tasks.workunit.client.1.smithi088.stdout:8/474: truncate d9/dc/d11/d24/d29/d39/d84/f6e 4129159 0 2024-03-24T00:48:51.269 INFO:tasks.workunit.client.1.smithi088.stdout:2/432: dread db/d21/d35/f79 [0,4194304] 0 2024-03-24T00:48:51.271 INFO:tasks.workunit.client.1.smithi088.stdout:6/469: creat db/d40/d60/f86 x:0 0 0 2024-03-24T00:48:51.272 INFO:tasks.workunit.client.0.smithi038.stdout:6/451: dwrite d1/da/fb [0,4194304] 0 2024-03-24T00:48:51.273 INFO:tasks.workunit.client.0.smithi038.stdout:6/452: creat d1/da/d1b/d25/f8e x:0 0 0 2024-03-24T00:48:51.273 INFO:tasks.workunit.client.0.smithi038.stdout:3/441: mkdir d3/d30/d70/d90 0 2024-03-24T00:48:51.273 INFO:tasks.workunit.client.0.smithi038.stdout:3/442: fsync d3/f8 0 2024-03-24T00:48:51.279 INFO:tasks.workunit.client.0.smithi038.stdout:5/408: dwrite d6/d1e/f63 [0,4194304] 0 2024-03-24T00:48:51.280 INFO:tasks.workunit.client.0.smithi038.stdout:4/365: dread da/d12/f4e [0,4194304] 0 2024-03-24T00:48:51.282 INFO:tasks.workunit.client.1.smithi088.stdout:5/461: mkdir d1/d10/d1b/d2f/d93 0 2024-03-24T00:48:51.282 INFO:tasks.workunit.client.1.smithi088.stdout:5/462: read - d1/d19/d1e/f6d zero size 2024-03-24T00:48:51.282 INFO:tasks.workunit.client.1.smithi088.stdout:3/427: write d3/d6/f2b [4748203,72888] 0 2024-03-24T00:48:51.282 INFO:tasks.workunit.client.1.smithi088.stdout:3/428: fsync d3/d6/d27/d56/f82 0 2024-03-24T00:48:51.292 INFO:tasks.workunit.client.1.smithi088.stdout:6/470: dread f8 [4194304,4194304] 0 2024-03-24T00:48:51.292 INFO:tasks.workunit.client.1.smithi088.stdout:6/471: read db/d40/d56/f30 [364415,11175] 0 2024-03-24T00:48:51.292 INFO:tasks.workunit.client.1.smithi088.stdout:6/472: readlink db/dd/d61/d6d/d74/l81 0 2024-03-24T00:48:51.292 INFO:tasks.workunit.client.1.smithi088.stdout:6/473: write db/d1d/f48 [393180,66779] 0 2024-03-24T00:48:51.292 INFO:tasks.workunit.client.1.smithi088.stdout:4/443: mkdir d5/d7/d15/d74/d95 0 2024-03-24T00:48:51.299 INFO:tasks.workunit.client.0.smithi038.stdout:7/426: dread d2/d9/da/d23/f39 [0,4194304] 0 2024-03-24T00:48:51.299 INFO:tasks.workunit.client.0.smithi038.stdout:7/427: fdatasync d2/d2c/f80 0 2024-03-24T00:48:51.307 INFO:tasks.workunit.client.1.smithi088.stdout:5/463: read d1/d10/d1b/d2f/d57/f71 [470204,45564] 0 2024-03-24T00:48:51.310 INFO:tasks.workunit.client.0.smithi038.stdout:5/409: dread d6/d23/f32 [0,4194304] 0 2024-03-24T00:48:51.311 INFO:tasks.workunit.client.1.smithi088.stdout:9/477: dwrite d1/d15/f7f [0,4194304] 0 2024-03-24T00:48:51.311 INFO:tasks.workunit.client.1.smithi088.stdout:9/478: creat d1/d22/d27/f97 x:0 0 0 2024-03-24T00:48:51.317 INFO:tasks.workunit.client.0.smithi038.stdout:3/443: dread d3/d9/d77/f85 [0,4194304] 0 2024-03-24T00:48:51.334 INFO:tasks.workunit.client.0.smithi038.stdout:7/428: dread d2/d2c/f80 [0,4194304] 0 2024-03-24T00:48:51.335 INFO:tasks.workunit.client.0.smithi038.stdout:7/429: chown d2/d9/d16/c83 6 1 2024-03-24T00:48:51.335 INFO:tasks.workunit.client.1.smithi088.stdout:2/433: unlink db/dc/l57 0 2024-03-24T00:48:51.335 INFO:tasks.workunit.client.1.smithi088.stdout:2/434: write db/d21/d4a/f7c [315104,37506] 0 2024-03-24T00:48:51.336 INFO:tasks.workunit.client.0.smithi038.stdout:7/430: fdatasync d2/d2c/f56 0 2024-03-24T00:48:51.342 INFO:tasks.workunit.client.0.smithi038.stdout:5/410: write d6/d23/f3b [1078178,123270] 0 2024-03-24T00:48:51.344 INFO:tasks.workunit.client.0.smithi038.stdout:4/366: truncate da/d12/f3f 3274181 0 2024-03-24T00:48:51.344 INFO:tasks.workunit.client.0.smithi038.stdout:4/367: truncate da/d12/d37/dd/d15/d4c/f6a 666372 0 2024-03-24T00:48:51.344 INFO:tasks.workunit.client.0.smithi038.stdout:4/368: fsync da/d12/d37/dd/d15/d4c/f5a 0 2024-03-24T00:48:51.344 INFO:tasks.workunit.client.1.smithi088.stdout:8/475: dwrite d9/dc/d11/d24/d29/d36/f4d [0,4194304] 0 2024-03-24T00:48:51.348 INFO:tasks.workunit.client.1.smithi088.stdout:3/429: symlink d3/d1a/d1e/d48/d4b/l92 0 2024-03-24T00:48:51.348 INFO:tasks.workunit.client.1.smithi088.stdout:3/430: creat d3/d1a/d1e/d76/f93 x:0 0 0 2024-03-24T00:48:51.348 INFO:tasks.workunit.client.1.smithi088.stdout:5/464: mkdir d1/d29/d3f/d88/d8b/d94 0 2024-03-24T00:48:51.350 INFO:tasks.workunit.client.1.smithi088.stdout:1/496: dwrite d6/f2a [0,4194304] 0 2024-03-24T00:48:51.361 INFO:tasks.workunit.client.1.smithi088.stdout:9/479: mkdir d1/d22/d27/d39/d98 0 2024-03-24T00:48:51.364 INFO:tasks.workunit.client.1.smithi088.stdout:6/474: dwrite db/f13 [0,4194304] 0 2024-03-24T00:48:51.365 INFO:tasks.workunit.client.0.smithi038.stdout:6/453: getdents d1 0 2024-03-24T00:48:51.365 INFO:tasks.workunit.client.0.smithi038.stdout:6/454: creat d1/d37/d68/f8f x:0 0 0 2024-03-24T00:48:51.365 INFO:tasks.workunit.client.0.smithi038.stdout:3/444: mknod d3/d9/dc/c91 0 2024-03-24T00:48:51.366 INFO:tasks.workunit.client.0.smithi038.stdout:2/408: dwrite d4/f13 [0,4194304] 0 2024-03-24T00:48:51.366 INFO:tasks.workunit.client.0.smithi038.stdout:2/409: fdatasync d4/d18/d26/f62 0 2024-03-24T00:48:51.374 INFO:tasks.workunit.client.0.smithi038.stdout:1/462: dwrite da/d3c/f81 [0,4194304] 0 2024-03-24T00:48:51.375 INFO:tasks.workunit.client.1.smithi088.stdout:4/444: dwrite d5/d13/d18/d24/d3d/f42 [0,4194304] 0 2024-03-24T00:48:51.376 INFO:tasks.workunit.client.1.smithi088.stdout:4/445: creat d5/d13/f96 x:0 0 0 2024-03-24T00:48:51.376 INFO:tasks.workunit.client.1.smithi088.stdout:4/446: write d5/d13/d18/d24/f90 [692376,113475] 0 2024-03-24T00:48:51.400 INFO:tasks.workunit.client.0.smithi038.stdout:5/411: dread d6/d19/d20/d42/d43/d68/f75 [0,4194304] 0 2024-03-24T00:48:51.400 INFO:tasks.workunit.client.0.smithi038.stdout:5/412: chown d6/d16/d21/d2d/c46 50873 1 2024-03-24T00:48:51.407 INFO:tasks.workunit.client.0.smithi038.stdout:1/463: dread f4 [8388608,4194304] 0 2024-03-24T00:48:51.411 INFO:tasks.workunit.client.0.smithi038.stdout:4/369: dwrite f9 [0,4194304] 0 2024-03-24T00:48:51.415 INFO:tasks.workunit.client.1.smithi088.stdout:2/435: rename db/d21/d4a/d4e/f61 to db/d21/d73/f8d 0 2024-03-24T00:48:51.415 INFO:tasks.workunit.client.1.smithi088.stdout:2/436: truncate db/d21/d35/d49/f71 1517100 0 2024-03-24T00:48:51.423 INFO:tasks.workunit.client.0.smithi038.stdout:5/413: dread d6/d23/f53 [4194304,4194304] 0 2024-03-24T00:48:51.423 INFO:tasks.workunit.client.0.smithi038.stdout:6/455: unlink d1/da/f82 0 2024-03-24T00:48:51.426 INFO:tasks.workunit.client.0.smithi038.stdout:3/445: mknod d3/d5a/c92 0 2024-03-24T00:48:51.429 INFO:tasks.workunit.client.0.smithi038.stdout:7/431: dwrite d2/d9/da/dc/f1c [0,4194304] 0 2024-03-24T00:48:51.429 INFO:tasks.workunit.client.0.smithi038.stdout:7/432: fsync d2/d9/da/d23/d2b/d5b/d72/d92/d33/f96 0 2024-03-24T00:48:51.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:51 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:51.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:51 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:48:51.430 INFO:tasks.workunit.client.1.smithi088.stdout:5/465: mknod d1/d43/d4e/d5b/c95 0 2024-03-24T00:48:51.432 INFO:tasks.workunit.client.0.smithi038.stdout:5/414: dread d6/d19/d20/d42/d43/d4c/f74 [0,4194304] 0 2024-03-24T00:48:51.434 INFO:tasks.workunit.client.0.smithi038.stdout:9/468: sync 2024-03-24T00:48:51.440 INFO:tasks.workunit.client.0.smithi038.stdout:6/456: dread d1/da/d1b/d25/d49/d4e/f55 [0,4194304] 0 2024-03-24T00:48:51.440 INFO:tasks.workunit.client.0.smithi038.stdout:6/457: dread - d1/da/d1b/d25/d49/d4e/f66 zero size 2024-03-24T00:48:51.446 INFO:tasks.workunit.client.1.smithi088.stdout:5/466: write d1/d10/d1b/d2e/f91 [1770504,86605] 0 2024-03-24T00:48:51.456 INFO:tasks.workunit.client.1.smithi088.stdout:9/480: creat d1/d10/d77/d8a/f99 x:0 0 0 2024-03-24T00:48:51.456 INFO:tasks.workunit.client.1.smithi088.stdout:9/481: readlink d1/d10/l46 0 2024-03-24T00:48:51.500 INFO:tasks.workunit.client.0.smithi038.stdout:2/410: rename d4/d27/d54/l73 to d4/l93 0 2024-03-24T00:48:51.509 INFO:tasks.workunit.client.0.smithi038.stdout:7/433: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d75/f78 [4194304,4194304] 0 2024-03-24T00:48:51.522 INFO:tasks.workunit.client.1.smithi088.stdout:6/475: mkdir db/dd/d87 0 2024-03-24T00:48:51.523 INFO:tasks.workunit.client.1.smithi088.stdout:5/467: dwrite d1/dd/f54 [0,4194304] 0 2024-03-24T00:48:51.523 INFO:tasks.workunit.client.1.smithi088.stdout:5/468: fsync d1/fa 0 2024-03-24T00:48:51.523 INFO:tasks.workunit.client.1.smithi088.stdout:5/469: chown d1/d29/f77 15723 1 2024-03-24T00:48:51.528 INFO:tasks.workunit.client.0.smithi038.stdout:6/458: dwrite d1/da/d1b/d2f/f7f [0,4194304] 0 2024-03-24T00:48:51.529 INFO:tasks.workunit.client.0.smithi038.stdout:2/411: dread d4/f1b [0,4194304] 0 2024-03-24T00:48:51.529 INFO:tasks.workunit.client.0.smithi038.stdout:2/412: creat d4/d27/f94 x:0 0 0 2024-03-24T00:48:51.529 INFO:tasks.workunit.client.0.smithi038.stdout:2/413: write d4/d27/d54/d67/f71 [457960,96185] 0 2024-03-24T00:48:51.529 INFO:tasks.workunit.client.0.smithi038.stdout:2/414: creat d4/d27/d54/d53/f95 x:0 0 0 2024-03-24T00:48:51.529 INFO:tasks.workunit.client.0.smithi038.stdout:2/415: dread - d4/d18/d38/f82 zero size 2024-03-24T00:48:51.540 INFO:tasks.workunit.client.0.smithi038.stdout:5/415: mknod d6/d16/d14/c8c 0 2024-03-24T00:48:51.545 INFO:tasks.workunit.client.0.smithi038.stdout:1/464: rename da/dd/d2c/d5a/l61 to da/dd/d7a/d82/d98/l9d 0 2024-03-24T00:48:51.549 INFO:tasks.workunit.client.0.smithi038.stdout:0/350: sync 2024-03-24T00:48:51.549 INFO:tasks.workunit.client.0.smithi038.stdout:2/416: dread d4/f7a [0,4194304] 0 2024-03-24T00:48:51.552 INFO:tasks.workunit.client.0.smithi038.stdout:7/434: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d5d/ca1 0 2024-03-24T00:48:51.554 INFO:tasks.workunit.client.0.smithi038.stdout:8/427: sync 2024-03-24T00:48:51.561 INFO:tasks.workunit.client.0.smithi038.stdout:0/351: read d0/d2d/f2f [45152,98358] 0 2024-03-24T00:48:51.561 INFO:tasks.workunit.client.0.smithi038.stdout:5/416: dread d6/d19/d20/d42/d43/d68/f75 [0,4194304] 0 2024-03-24T00:48:51.561 INFO:tasks.workunit.client.0.smithi038.stdout:5/417: fsync d6/f26 0 2024-03-24T00:48:51.566 INFO:tasks.workunit.client.1.smithi088.stdout:4/447: rename d5/d13/d18/l5c to d5/d13/d18/d6e/d7d/l97 0 2024-03-24T00:48:51.566 INFO:tasks.workunit.client.1.smithi088.stdout:4/448: dread - d5/f46 zero size 2024-03-24T00:48:51.568 INFO:tasks.workunit.client.0.smithi038.stdout:4/370: symlink da/d12/d37/dd/d15/d6d/l72 0 2024-03-24T00:48:51.571 INFO:tasks.workunit.client.1.smithi088.stdout:2/437: mkdir db/d54/d88/d8e 0 2024-03-24T00:48:51.571 INFO:tasks.workunit.client.1.smithi088.stdout:2/438: truncate db/f4f 1119109 0 2024-03-24T00:48:51.571 INFO:tasks.workunit.client.1.smithi088.stdout:2/439: chown db/d21/d4a/f53 0 1 2024-03-24T00:48:51.571 INFO:tasks.workunit.client.1.smithi088.stdout:2/440: write db/f3c [1685816,109989] 0 2024-03-24T00:48:51.571 INFO:tasks.workunit.client.1.smithi088.stdout:2/441: chown db/d21/d35/d49/l51 1529394440 1 2024-03-24T00:48:51.571 INFO:tasks.workunit.client.1.smithi088.stdout:2/442: write db/f87 [658581,37283] 0 2024-03-24T00:48:51.574 INFO:tasks.workunit.client.1.smithi088.stdout:8/476: unlink d9/dc/d11/c13 0 2024-03-24T00:48:51.575 INFO:tasks.workunit.client.0.smithi038.stdout:1/465: mkdir da/dd/d7a/d82/d98/d9e 0 2024-03-24T00:48:51.580 INFO:tasks.workunit.client.0.smithi038.stdout:2/417: mkdir d4/d18/d96 0 2024-03-24T00:48:51.583 INFO:tasks.workunit.client.0.smithi038.stdout:7/435: getdents d2/d9/da/d23/d2b/d5b/d72/d92 0 2024-03-24T00:48:51.583 INFO:tasks.workunit.client.0.smithi038.stdout:7/436: write d2/f13 [1854168,112616] 0 2024-03-24T00:48:51.586 INFO:tasks.workunit.client.0.smithi038.stdout:8/428: mknod d0/d45/d37/d5f/c95 0 2024-03-24T00:48:51.591 INFO:tasks.workunit.client.0.smithi038.stdout:4/371: mkdir da/d12/d29/d5b/d73 0 2024-03-24T00:48:51.594 INFO:tasks.workunit.client.0.smithi038.stdout:2/418: write d4/f13 [1769419,118027] 0 2024-03-24T00:48:51.598 INFO:tasks.workunit.client.0.smithi038.stdout:1/466: symlink da/dd/d2c/d5a/l9f 0 2024-03-24T00:48:51.598 INFO:tasks.workunit.client.1.smithi088.stdout:9/482: truncate d1/d10/f34 2925673 0 2024-03-24T00:48:51.598 INFO:tasks.workunit.client.1.smithi088.stdout:9/483: readlink d1/d10/l42 0 2024-03-24T00:48:51.603 INFO:tasks.workunit.client.0.smithi038.stdout:7/437: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/ca2 0 2024-03-24T00:48:51.606 INFO:tasks.workunit.client.1.smithi088.stdout:5/470: link d1/d10/d1b/d2f/d57/l61 d1/d29/d3f/l96 0 2024-03-24T00:48:51.611 INFO:tasks.workunit.client.0.smithi038.stdout:4/372: symlink da/d27/l74 0 2024-03-24T00:48:51.615 INFO:tasks.workunit.client.1.smithi088.stdout:9/484: dread d1/d22/f49 [0,4194304] 0 2024-03-24T00:48:51.615 INFO:tasks.workunit.client.1.smithi088.stdout:9/485: dread - d1/d22/d27/f83 zero size 2024-03-24T00:48:51.619 INFO:tasks.workunit.client.0.smithi038.stdout:2/419: symlink d4/d27/l97 0 2024-03-24T00:48:51.627 INFO:tasks.workunit.client.0.smithi038.stdout:1/467: dread da/dd/d7a/d82/d98/f15 [12582912,4194304] 0 2024-03-24T00:48:51.627 INFO:tasks.workunit.client.0.smithi038.stdout:1/468: fsync da/dd/d2c/d5a/f7b 0 2024-03-24T00:48:51.627 INFO:tasks.workunit.client.0.smithi038.stdout:1/469: chown da/dd/d2c/d5a/d74/f97 123326917 1 2024-03-24T00:48:51.628 INFO:tasks.workunit.client.1.smithi088.stdout:2/443: mkdir db/d21/d4a/d4e/d8f 0 2024-03-24T00:48:51.634 INFO:tasks.workunit.client.0.smithi038.stdout:4/373: creat da/d12/d24/d5f/f75 x:0 0 0 2024-03-24T00:48:51.637 INFO:tasks.workunit.client.1.smithi088.stdout:4/449: dwrite d5/d13/d18/d24/f53 [0,4194304] 0 2024-03-24T00:48:51.638 INFO:tasks.workunit.client.1.smithi088.stdout:4/450: chown d5/d7/dd/f75 678665270 1 2024-03-24T00:48:51.640 INFO:tasks.workunit.client.1.smithi088.stdout:8/477: dwrite d9/dc/d11/f2f [0,4194304] 0 2024-03-24T00:48:51.640 INFO:tasks.workunit.client.1.smithi088.stdout:5/471: creat d1/d10/d81/f97 x:0 0 0 2024-03-24T00:48:51.640 INFO:tasks.workunit.client.1.smithi088.stdout:8/478: write d9/dc/d11/d24/d29/d73/f76 [1036416,128526] 0 2024-03-24T00:48:51.647 INFO:tasks.workunit.client.0.smithi038.stdout:2/420: symlink d4/d18/d26/d37/d69/d83/l98 0 2024-03-24T00:48:51.647 INFO:tasks.workunit.client.0.smithi038.stdout:2/421: creat d4/d76/f99 x:0 0 0 2024-03-24T00:48:51.652 INFO:tasks.workunit.client.0.smithi038.stdout:1/470: creat da/dd/d22/d94/fa0 x:0 0 0 2024-03-24T00:48:51.661 INFO:tasks.workunit.client.1.smithi088.stdout:4/451: symlink d5/d7/d15/d74/l98 0 2024-03-24T00:48:51.663 INFO:tasks.workunit.client.0.smithi038.stdout:1/471: symlink da/dd/d2c/d41/d60/la1 0 2024-03-24T00:48:51.663 INFO:tasks.workunit.client.0.smithi038.stdout:1/472: dread - da/dd/d2c/d5a/d74/f64 zero size 2024-03-24T00:48:51.664 INFO:tasks.workunit.client.0.smithi038.stdout:2/422: write d4/fa [3319489,63233] 0 2024-03-24T00:48:51.666 INFO:tasks.workunit.client.0.smithi038.stdout:0/352: dwrite d0/f5 [0,4194304] 0 2024-03-24T00:48:51.666 INFO:tasks.workunit.client.0.smithi038.stdout:0/353: chown d0/d12/d15/f22 929 1 2024-03-24T00:48:51.666 INFO:tasks.workunit.client.0.smithi038.stdout:0/354: stat d0/d12/d42/d5e 0 2024-03-24T00:48:51.668 INFO:tasks.workunit.client.1.smithi088.stdout:8/479: symlink d9/dc/d11/d24/d29/d5b/la2 0 2024-03-24T00:48:51.669 INFO:tasks.workunit.client.1.smithi088.stdout:2/444: getdents db 0 2024-03-24T00:48:51.669 INFO:tasks.workunit.client.1.smithi088.stdout:2/445: fdatasync db/f18 0 2024-03-24T00:48:51.672 INFO:tasks.workunit.client.0.smithi038.stdout:8/429: dwrite d0/d45/f55 [0,4194304] 0 2024-03-24T00:48:51.672 INFO:tasks.workunit.client.0.smithi038.stdout:8/430: chown d0/d1d/d20/l41 1343851 1 2024-03-24T00:48:51.672 INFO:tasks.workunit.client.1.smithi088.stdout:5/472: dread d1/d29/f50 [0,4194304] 0 2024-03-24T00:48:51.674 INFO:tasks.workunit.client.0.smithi038.stdout:1/473: symlink da/dd/d22/d94/la2 0 2024-03-24T00:48:51.677 INFO:tasks.workunit.client.1.smithi088.stdout:4/452: write d5/f56 [3515740,105132] 0 2024-03-24T00:48:51.679 INFO:tasks.workunit.client.0.smithi038.stdout:2/423: creat d4/d27/d2f/d89/f9a x:0 0 0 2024-03-24T00:48:51.681 INFO:tasks.workunit.client.1.smithi088.stdout:5/473: dread d1/d19/f2c [0,4194304] 0 2024-03-24T00:48:51.688 INFO:tasks.workunit.client.0.smithi038.stdout:1/474: mknod da/dd/d7a/d82/ca3 0 2024-03-24T00:48:51.689 INFO:tasks.workunit.client.0.smithi038.stdout:1/475: write da/dd/d2c/d5a/f7b [1113592,62668] 0 2024-03-24T00:48:51.689 INFO:tasks.workunit.client.0.smithi038.stdout:1/476: read da/dd/d2c/d5a/f92 [93353,30395] 0 2024-03-24T00:48:51.693 INFO:tasks.workunit.client.0.smithi038.stdout:2/424: truncate d4/d27/f47 7492769 0 2024-03-24T00:48:51.697 INFO:tasks.workunit.client.0.smithi038.stdout:1/477: truncate da/dd/d22/f71 679245 0 2024-03-24T00:48:51.697 INFO:tasks.workunit.client.0.smithi038.stdout:1/478: readlink da/d3c/l3b 0 2024-03-24T00:48:51.697 INFO:tasks.workunit.client.0.smithi038.stdout:1/479: truncate da/dd/d7a/d82/d98/f15 16873805 0 2024-03-24T00:48:51.699 INFO:tasks.workunit.client.1.smithi088.stdout:4/453: creat d5/d7/d15/d74/d95/f99 x:0 0 0 2024-03-24T00:48:51.699 INFO:tasks.workunit.client.1.smithi088.stdout:4/454: fsync d5/d16/f29 0 2024-03-24T00:48:51.699 INFO:tasks.workunit.client.1.smithi088.stdout:4/455: readlink d5/d16/l20 0 2024-03-24T00:48:51.700 INFO:tasks.workunit.client.0.smithi038.stdout:2/425: mkdir d4/d27/d2f/d30/d3e/d9b 0 2024-03-24T00:48:51.702 INFO:tasks.workunit.client.0.smithi038.stdout:1/480: rmdir da 39 2024-03-24T00:48:51.705 INFO:tasks.workunit.client.0.smithi038.stdout:2/426: creat d4/d27/d2f/d30/d3e/d9b/f9c x:0 0 0 2024-03-24T00:48:51.707 INFO:tasks.workunit.client.1.smithi088.stdout:1/497: mkdir d6/d27/d1e/d5b/d93 0 2024-03-24T00:48:51.708 INFO:tasks.workunit.client.1.smithi088.stdout:5/474: mkdir d1/dd/d6f/d98 0 2024-03-24T00:48:51.710 INFO:tasks.workunit.client.0.smithi038.stdout:2/427: symlink d4/d27/d2f/l9d 0 2024-03-24T00:48:51.710 INFO:tasks.workunit.client.0.smithi038.stdout:2/428: stat d4/d27/d2f/d30/l34 0 2024-03-24T00:48:51.716 INFO:tasks.workunit.client.1.smithi088.stdout:1/498: dread d6/d27/d1e/d3f/f54 [0,4194304] 0 2024-03-24T00:48:51.718 INFO:tasks.workunit.client.1.smithi088.stdout:4/456: creat d5/d16/d1a/d22/f9a x:0 0 0 2024-03-24T00:48:51.718 INFO:tasks.workunit.client.1.smithi088.stdout:5/475: symlink d1/d10/d81/l99 0 2024-03-24T00:48:51.719 INFO:tasks.workunit.client.0.smithi038.stdout:2/429: mkdir d4/d27/d8f/d9e 0 2024-03-24T00:48:51.719 INFO:tasks.workunit.client.0.smithi038.stdout:2/430: stat d4/d27/d54/d53 0 2024-03-24T00:48:51.719 INFO:tasks.workunit.client.0.smithi038.stdout:1/481: write da/d3c/f81 [2290735,80843] 0 2024-03-24T00:48:51.720 INFO:tasks.workunit.client.1.smithi088.stdout:6/476: rename db/d40/d60 to db/dd/d61/d6d/d74/d88 0 2024-03-24T00:48:51.723 INFO:tasks.workunit.client.1.smithi088.stdout:1/499: symlink d6/d27/d1e/l94 0 2024-03-24T00:48:51.724 INFO:tasks.workunit.client.1.smithi088.stdout:4/457: symlink d5/d13/d18/d6e/d7d/l9b 0 2024-03-24T00:48:51.725 INFO:tasks.workunit.client.1.smithi088.stdout:5/476: symlink d1/dd/d6f/l9a 0 2024-03-24T00:48:51.725 INFO:tasks.workunit.client.1.smithi088.stdout:5/477: fsync d1/d19/d1e/f6d 0 2024-03-24T00:48:51.725 INFO:tasks.workunit.client.1.smithi088.stdout:5/478: chown d1/d29/d3f/d88/d8f/d69/c6a 57527 1 2024-03-24T00:48:51.725 INFO:tasks.workunit.client.1.smithi088.stdout:5/479: creat d1/d19/d1e/f9b x:0 0 0 2024-03-24T00:48:51.726 INFO:tasks.workunit.client.1.smithi088.stdout:6/477: rename db/d40/d55/d6a to db/dd/d89 0 2024-03-24T00:48:51.726 INFO:tasks.workunit.client.1.smithi088.stdout:6/478: chown db 107739713 1 2024-03-24T00:48:51.736 INFO:tasks.workunit.client.0.smithi038.stdout:3/446: rename d3/d9/f44 to d3/d9/dc/df/d14/d19/d82/f93 0 2024-03-24T00:48:51.737 INFO:tasks.workunit.client.0.smithi038.stdout:3/447: truncate d3/d9/f8d 90589 0 2024-03-24T00:48:51.737 INFO:tasks.workunit.client.0.smithi038.stdout:3/448: stat d3/f5 0 2024-03-24T00:48:51.737 INFO:tasks.workunit.client.0.smithi038.stdout:3/449: creat d3/d9/d77/f94 x:0 0 0 2024-03-24T00:48:51.737 INFO:tasks.workunit.client.0.smithi038.stdout:3/450: creat d3/d30/d5d/f95 x:0 0 0 2024-03-24T00:48:51.737 INFO:tasks.workunit.client.0.smithi038.stdout:3/451: write d3/d30/d5d/f95 [245606,55214] 0 2024-03-24T00:48:51.737 INFO:tasks.workunit.client.1.smithi088.stdout:5/480: getdents d1/d43 0 2024-03-24T00:48:51.741 INFO:tasks.workunit.client.0.smithi038.stdout:9/469: rename l7 to d12/d41/l9a 0 2024-03-24T00:48:51.743 INFO:tasks.workunit.client.1.smithi088.stdout:6/479: creat db/dd/d61/d6d/d74/d88/f8a x:0 0 0 2024-03-24T00:48:51.743 INFO:tasks.workunit.client.1.smithi088.stdout:6/480: dread - db/dd/d61/d6d/d74/d88/f8a zero size 2024-03-24T00:48:51.743 INFO:tasks.workunit.client.1.smithi088.stdout:6/481: chown db/d1d/d23/d3c/d43/l32 2 1 2024-03-24T00:48:51.743 INFO:tasks.workunit.client.0.smithi038.stdout:9/470: write d12/d28/d3a/f77 [371030,49519] 0 2024-03-24T00:48:51.744 INFO:tasks.workunit.client.1.smithi088.stdout:4/458: dread d5/d7/dd/f21 [0,4194304] 0 2024-03-24T00:48:51.749 INFO:tasks.workunit.client.0.smithi038.stdout:6/459: rename d1/da/d1b/d1d/f43 to d1/d37/f90 0 2024-03-24T00:48:51.752 INFO:tasks.workunit.client.1.smithi088.stdout:6/482: creat db/dd/d61/d6d/d74/f8b x:0 0 0 2024-03-24T00:48:51.752 INFO:tasks.workunit.client.0.smithi038.stdout:9/471: getdents d12/d1a 0 2024-03-24T00:48:51.755 INFO:tasks.workunit.client.1.smithi088.stdout:6/483: mknod db/d1d/d23/c8c 0 2024-03-24T00:48:51.757 INFO:tasks.workunit.client.0.smithi038.stdout:7/438: rename d2/f13 to d2/d9/da/d23/fa3 0 2024-03-24T00:48:51.757 INFO:tasks.workunit.client.0.smithi038.stdout:7/439: chown d2/d9/da/d23/d2b/d5b/d72/d92/c6a 5 1 2024-03-24T00:48:51.761 INFO:tasks.workunit.client.0.smithi038.stdout:4/374: rename f9 to da/d38/f76 0 2024-03-24T00:48:51.762 INFO:tasks.workunit.client.0.smithi038.stdout:7/440: truncate d2/d9/da/d23/d2b/d5b/d72/d92/f71 2497093 0 2024-03-24T00:48:51.764 INFO:tasks.workunit.client.0.smithi038.stdout:6/460: getdents d1/da/d61/d69 0 2024-03-24T00:48:51.764 INFO:tasks.workunit.client.0.smithi038.stdout:6/461: truncate d1/da/d1b/d25/d49/d72/f76 332695 0 2024-03-24T00:48:51.766 INFO:tasks.workunit.client.0.smithi038.stdout:0/355: rename d0/d12/d15/l4b to d0/d12/d15/d58/d64/l75 0 2024-03-24T00:48:51.766 INFO:tasks.workunit.client.0.smithi038.stdout:0/356: chown d0/d12/d15/d1a/c6a 34984632 1 2024-03-24T00:48:51.767 INFO:tasks.workunit.client.0.smithi038.stdout:0/357: readlink d0/d12/d15/d1a/l1c 0 2024-03-24T00:48:51.767 INFO:tasks.workunit.client.0.smithi038.stdout:2/431: dwrite d4/f1b [4194304,4194304] 0 2024-03-24T00:48:51.769 INFO:tasks.workunit.client.0.smithi038.stdout:7/441: creat d2/d9/da/d23/d2b/d5b/fa4 x:0 0 0 2024-03-24T00:48:51.769 INFO:tasks.workunit.client.0.smithi038.stdout:2/432: read d4/d27/d54/d53/f64 [96434,87320] 0 2024-03-24T00:48:51.769 INFO:tasks.workunit.client.0.smithi038.stdout:6/462: creat d1/da/d1b/f91 x:0 0 0 2024-03-24T00:48:51.771 INFO:tasks.workunit.client.1.smithi088.stdout:4/459: write d5/d7/d72/f71 [3902108,33616] 0 2024-03-24T00:48:51.772 INFO:tasks.workunit.client.1.smithi088.stdout:4/460: creat d5/d13/d18/d24/f9c x:0 0 0 2024-03-24T00:48:51.772 INFO:tasks.workunit.client.1.smithi088.stdout:4/461: chown d5/d13/d18/f1b 90436 1 2024-03-24T00:48:51.773 INFO:tasks.workunit.client.0.smithi038.stdout:0/358: truncate d0/d1b/f4a 3920733 0 2024-03-24T00:48:51.777 INFO:tasks.workunit.client.0.smithi038.stdout:1/482: rename da/dd/d4a/d76/l99 to da/dd/d2c/la4 0 2024-03-24T00:48:51.783 INFO:tasks.workunit.client.0.smithi038.stdout:7/442: creat d2/d9/da/dc/d8a/fa5 x:0 0 0 2024-03-24T00:48:51.783 INFO:tasks.workunit.client.1.smithi088.stdout:4/462: mknod d5/d16/d37/c9d 0 2024-03-24T00:48:51.789 INFO:tasks.workunit.client.0.smithi038.stdout:5/418: symlink d6/d19/d20/d42/l8d 0 2024-03-24T00:48:51.789 INFO:tasks.workunit.client.0.smithi038.stdout:5/419: stat d6/d19/d20/d42/l8d 0 2024-03-24T00:48:51.789 INFO:tasks.workunit.client.0.smithi038.stdout:5/420: chown d6/d19/d20/d70 214311 1 2024-03-24T00:48:51.795 INFO:tasks.workunit.client.0.smithi038.stdout:2/433: unlink d4/d27/d54/d53/f5f 0 2024-03-24T00:48:51.796 INFO:tasks.workunit.client.0.smithi038.stdout:2/434: creat d4/d27/d2f/d89/f9f x:0 0 0 2024-03-24T00:48:51.797 INFO:tasks.workunit.client.0.smithi038.stdout:6/463: mkdir d1/da/d1b/d25/d92 0 2024-03-24T00:48:51.797 INFO:tasks.workunit.client.0.smithi038.stdout:6/464: stat d1/da/d1b/d25/c31 0 2024-03-24T00:48:51.799 INFO:tasks.workunit.client.1.smithi088.stdout:8/480: rmdir d9/dc/d11/d24 39 2024-03-24T00:48:51.799 INFO:tasks.workunit.client.0.smithi038.stdout:5/421: dread d6/d19/d20/d33/d6f/f7a [0,4194304] 0 2024-03-24T00:48:51.803 INFO:tasks.workunit.client.0.smithi038.stdout:3/452: rename d3/d9/d77/f94 to d3/d9/f96 0 2024-03-24T00:48:51.804 INFO:tasks.workunit.client.0.smithi038.stdout:1/483: creat da/dd/d7a/d82/d98/d34/fa5 x:0 0 0 2024-03-24T00:48:51.805 INFO:tasks.workunit.client.0.smithi038.stdout:3/453: write d3/d5a/f79 [46060,69264] 0 2024-03-24T00:48:51.806 INFO:tasks.workunit.client.1.smithi088.stdout:2/446: write db/d21/d35/f7d [596609,25942] 0 2024-03-24T00:48:51.807 INFO:tasks.workunit.client.0.smithi038.stdout:2/435: creat d4/d18/d26/d37/d69/fa0 x:0 0 0 2024-03-24T00:48:51.808 INFO:tasks.workunit.client.0.smithi038.stdout:5/422: creat d6/d16/d14/f8e x:0 0 0 2024-03-24T00:48:51.808 INFO:tasks.workunit.client.0.smithi038.stdout:5/423: stat d6/d19/d20/d42/d43/d4c/f62 0 2024-03-24T00:48:51.810 INFO:tasks.workunit.client.1.smithi088.stdout:9/486: unlink d1/d22/c74 0 2024-03-24T00:48:51.811 INFO:tasks.workunit.client.0.smithi038.stdout:1/484: symlink da/dd/d7a/la6 0 2024-03-24T00:48:51.814 INFO:tasks.workunit.client.0.smithi038.stdout:9/472: rename d12/d1a/f8c to d12/d55/f9b 0 2024-03-24T00:48:51.815 INFO:tasks.workunit.client.0.smithi038.stdout:9/473: read d12/d28/d3a/f77 [330140,103435] 0 2024-03-24T00:48:51.816 INFO:tasks.workunit.client.0.smithi038.stdout:8/431: unlink d0/d45/d37/l50 0 2024-03-24T00:48:51.816 INFO:tasks.workunit.client.0.smithi038.stdout:8/432: truncate d0/d45/d37/d5f/d68/f91 745528 0 2024-03-24T00:48:51.821 INFO:tasks.workunit.client.0.smithi038.stdout:3/454: truncate d3/d2a/f4b 3486446 0 2024-03-24T00:48:51.822 INFO:tasks.workunit.client.0.smithi038.stdout:2/436: mkdir d4/d18/da1 0 2024-03-24T00:48:51.827 INFO:tasks.workunit.client.0.smithi038.stdout:5/424: rmdir d6/d19/d20/d70 39 2024-03-24T00:48:51.829 INFO:tasks.workunit.client.0.smithi038.stdout:1/485: creat da/dd/d7a/d82/d98/d9e/fa7 x:0 0 0 2024-03-24T00:48:51.834 INFO:tasks.workunit.client.0.smithi038.stdout:6/465: rename d1/da/d73/d83/f87 to d1/f93 0 2024-03-24T00:48:51.835 INFO:tasks.workunit.client.1.smithi088.stdout:9/487: write d1/d22/f49 [7455405,27568] 0 2024-03-24T00:48:51.835 INFO:tasks.workunit.client.1.smithi088.stdout:9/488: chown d1/d15/d47/c8e 0 1 2024-03-24T00:48:51.837 INFO:tasks.workunit.client.0.smithi038.stdout:8/433: mknod d0/d10/c96 0 2024-03-24T00:48:51.837 INFO:tasks.workunit.client.0.smithi038.stdout:8/434: creat d0/d45/d66/f97 x:0 0 0 2024-03-24T00:48:51.840 INFO:tasks.workunit.client.1.smithi088.stdout:5/481: dwrite d1/d10/d1b/f70 [4194304,4194304] 0 2024-03-24T00:48:51.840 INFO:tasks.workunit.client.0.smithi038.stdout:3/455: link d3/d2a/c5b d3/d9/dc/d51/c97 0 2024-03-24T00:48:51.841 INFO:tasks.workunit.client.0.smithi038.stdout:3/456: fsync f1 0 2024-03-24T00:48:51.844 INFO:tasks.workunit.client.1.smithi088.stdout:7/398: sync 2024-03-24T00:48:51.844 INFO:tasks.workunit.client.1.smithi088.stdout:3/431: sync 2024-03-24T00:48:51.845 INFO:tasks.workunit.client.1.smithi088.stdout:0/470: sync 2024-03-24T00:48:51.846 INFO:tasks.workunit.client.0.smithi038.stdout:2/437: creat d4/d27/d8f/d9e/fa2 x:0 0 0 2024-03-24T00:48:51.846 INFO:tasks.workunit.client.0.smithi038.stdout:2/438: chown d4/c5 388 1 2024-03-24T00:48:51.846 INFO:tasks.workunit.client.0.smithi038.stdout:5/425: creat d6/d16/d67/f8f x:0 0 0 2024-03-24T00:48:51.854 INFO:tasks.workunit.client.1.smithi088.stdout:5/482: getdents d1/d29/d3f/d88/d8f/d69 0 2024-03-24T00:48:51.861 INFO:tasks.workunit.client.0.smithi038.stdout:0/359: dwrite d0/d12/d42/f61 [0,4194304] 0 2024-03-24T00:48:51.863 INFO:tasks.workunit.client.1.smithi088.stdout:1/500: rename d6/d27/d1e to d6/d43/d95 0 2024-03-24T00:48:51.864 INFO:tasks.workunit.client.0.smithi038.stdout:4/375: dwrite da/d12/d37/dd/d15/d4c/f5a [0,4194304] 0 2024-03-24T00:48:51.864 INFO:tasks.workunit.client.0.smithi038.stdout:4/376: fdatasync da/d12/d24/d5f/f6f 0 2024-03-24T00:48:51.864 INFO:tasks.workunit.client.0.smithi038.stdout:4/377: fdatasync da/d12/f4e 0 2024-03-24T00:48:51.864 INFO:tasks.workunit.client.0.smithi038.stdout:4/378: write da/d12/d37/dd/f21 [328014,66351] 0 2024-03-24T00:48:51.864 INFO:tasks.workunit.client.0.smithi038.stdout:4/379: chown da/d38 137807 1 2024-03-24T00:48:51.866 INFO:tasks.workunit.client.0.smithi038.stdout:9/474: mkdir d12/d28/d9c 0 2024-03-24T00:48:51.866 INFO:tasks.workunit.client.0.smithi038.stdout:9/475: write d12/d55/d56/f7c [1099565,113442] 0 2024-03-24T00:48:51.866 INFO:tasks.workunit.client.0.smithi038.stdout:9/476: write d12/d41/f5b [4541304,41381] 0 2024-03-24T00:48:51.866 INFO:tasks.workunit.client.0.smithi038.stdout:9/477: stat d12/d1a/f3c 0 2024-03-24T00:48:51.866 INFO:tasks.workunit.client.0.smithi038.stdout:9/478: chown d12/d28/c59 74 1 2024-03-24T00:48:51.873 INFO:tasks.workunit.client.1.smithi088.stdout:8/481: dwrite d9/f37 [0,4194304] 0 2024-03-24T00:48:51.874 INFO:tasks.workunit.client.1.smithi088.stdout:8/482: fdatasync d9/dc/f16 0 2024-03-24T00:48:51.874 INFO:tasks.workunit.client.1.smithi088.stdout:8/483: read d9/dc/d11/d24/d29/d5b/f53 [3816049,35792] 0 2024-03-24T00:48:51.874 INFO:tasks.workunit.client.1.smithi088.stdout:8/484: chown d9/dc/d11/d24/l33 30 1 2024-03-24T00:48:51.875 INFO:tasks.workunit.client.1.smithi088.stdout:7/399: dread d3/f5f [4194304,4194304] 0 2024-03-24T00:48:51.878 INFO:tasks.workunit.client.1.smithi088.stdout:3/432: rmdir d3/d1a/d1e/d48/d4b 39 2024-03-24T00:48:51.878 INFO:tasks.workunit.client.1.smithi088.stdout:0/471: symlink d9/d26/d8f/l95 0 2024-03-24T00:48:51.880 INFO:tasks.workunit.client.1.smithi088.stdout:2/447: dwrite db/f89 [0,4194304] 0 2024-03-24T00:48:51.881 INFO:tasks.workunit.client.0.smithi038.stdout:6/466: creat d1/da/d1b/d1d/f94 x:0 0 0 2024-03-24T00:48:51.881 INFO:tasks.workunit.client.0.smithi038.stdout:6/467: chown d1/da/d1b/d25/d49/d4e/d51 0 1 2024-03-24T00:48:51.883 INFO:tasks.workunit.client.1.smithi088.stdout:5/483: rmdir d1/d10/d1b/d2e 39 2024-03-24T00:48:51.891 INFO:tasks.workunit.client.1.smithi088.stdout:8/485: read d9/f2b [639793,53558] 0 2024-03-24T00:48:51.896 INFO:tasks.workunit.client.1.smithi088.stdout:6/484: rename db/d1d/f68 to db/d1d/d50/f8d 0 2024-03-24T00:48:51.899 INFO:tasks.workunit.client.0.smithi038.stdout:8/435: link d0/d1d/d20/f58 d0/d15/f98 0 2024-03-24T00:48:51.906 INFO:tasks.workunit.client.1.smithi088.stdout:2/448: dread db/dd/f2f [0,4194304] 0 2024-03-24T00:48:51.911 INFO:tasks.workunit.client.0.smithi038.stdout:2/439: rmdir d4/d55/d88 39 2024-03-24T00:48:51.914 INFO:tasks.workunit.client.1.smithi088.stdout:5/484: dread d1/d10/d1b/f70 [4194304,4194304] 0 2024-03-24T00:48:51.914 INFO:tasks.workunit.client.1.smithi088.stdout:1/501: mknod d6/d43/d95/d5b/d80/c96 0 2024-03-24T00:48:51.914 INFO:tasks.workunit.client.1.smithi088.stdout:1/502: readlink d6/d43/d95/d5b/l82 0 2024-03-24T00:48:51.924 INFO:tasks.workunit.client.0.smithi038.stdout:7/443: dwrite d2/f1f [8388608,4194304] 0 2024-03-24T00:48:51.924 INFO:tasks.workunit.client.0.smithi038.stdout:7/444: write d2/d9/f20 [352741,83824] 0 2024-03-24T00:48:51.925 INFO:tasks.workunit.client.0.smithi038.stdout:7/445: truncate d2/d9/da/dc/d4a/f67 418235 0 2024-03-24T00:48:51.935 INFO:tasks.workunit.client.1.smithi088.stdout:8/486: dread d9/dc/d11/d41/d54/d74/f90 [0,4194304] 0 2024-03-24T00:48:51.935 INFO:tasks.workunit.client.1.smithi088.stdout:8/487: write d9/dc/d11/d24/d29/d5b/f1e [4506635,115995] 0 2024-03-24T00:48:51.941 INFO:tasks.workunit.client.1.smithi088.stdout:7/400: mkdir d3/d32/d34/d7c 0 2024-03-24T00:48:51.943 INFO:tasks.workunit.client.1.smithi088.stdout:3/433: creat d3/de/d6f/d85/d31/f94 x:0 0 0 2024-03-24T00:48:51.949 INFO:tasks.workunit.client.0.smithi038.stdout:1/486: getdents da/dd/d2c 0 2024-03-24T00:48:51.957 INFO:tasks.workunit.client.1.smithi088.stdout:8/488: write d9/dc/d11/d24/d29/f2c [8026337,1563] 0 2024-03-24T00:48:51.958 INFO:tasks.workunit.client.0.smithi038.stdout:2/440: dread d4/fa [0,4194304] 0 2024-03-24T00:48:51.958 INFO:tasks.workunit.client.0.smithi038.stdout:2/441: chown d4/d27/d2f/d30/d3e/f78 34903 1 2024-03-24T00:48:51.958 INFO:tasks.workunit.client.0.smithi038.stdout:8/436: dread d0/d1d/d27/f8e [0,4194304] 0 2024-03-24T00:48:51.959 INFO:tasks.workunit.client.0.smithi038.stdout:0/360: dwrite d0/d1b/d2c/f6b [0,4194304] 0 2024-03-24T00:48:51.960 INFO:tasks.workunit.client.0.smithi038.stdout:3/457: dwrite d3/d9/dc/df/f3a [4194304,4194304] 0 2024-03-24T00:48:51.960 INFO:tasks.workunit.client.0.smithi038.stdout:3/458: chown d3/d9/dc/d51/f75 3799591 1 2024-03-24T00:48:51.960 INFO:tasks.workunit.client.0.smithi038.stdout:3/459: fsync d3/f8c 0 2024-03-24T00:48:51.966 INFO:tasks.workunit.client.0.smithi038.stdout:5/426: dwrite d6/d19/d20/d33/d6f/f7a [0,4194304] 0 2024-03-24T00:48:51.968 INFO:tasks.workunit.client.1.smithi088.stdout:6/485: mkdir db/d8e 0 2024-03-24T00:48:51.971 INFO:tasks.workunit.client.1.smithi088.stdout:0/472: dwrite d9/f56 [0,4194304] 0 2024-03-24T00:48:51.972 INFO:tasks.workunit.client.0.smithi038.stdout:9/479: dwrite d12/d28/d3a/d26/f86 [0,4194304] 0 2024-03-24T00:48:51.972 INFO:tasks.workunit.client.0.smithi038.stdout:9/480: dread - d12/d28/d3a/d26/f5a zero size 2024-03-24T00:48:51.975 INFO:tasks.workunit.client.0.smithi038.stdout:4/380: creat da/d12/d37/dd/d15/f77 x:0 0 0 2024-03-24T00:48:51.978 INFO:tasks.workunit.client.0.smithi038.stdout:4/381: write da/d12/d37/dd/d15/d4c/f53 [791596,99799] 0 2024-03-24T00:48:51.979 INFO:tasks.workunit.client.1.smithi088.stdout:1/503: dwrite d6/d27/f3a [0,4194304] 0 2024-03-24T00:48:51.980 INFO:tasks.workunit.client.1.smithi088.stdout:4/463: rename d5/d17/f89 to d5/d13/d18/f9e 0 2024-03-24T00:48:51.983 INFO:tasks.workunit.client.1.smithi088.stdout:8/489: dread d9/dc/d11/d24/d29/d5b/f1e [0,4194304] 0 2024-03-24T00:48:51.983 INFO:tasks.workunit.client.1.smithi088.stdout:8/490: chown d9/f37 763 1 2024-03-24T00:48:51.994 INFO:tasks.workunit.client.1.smithi088.stdout:2/449: creat db/d21/d35/f90 x:0 0 0 2024-03-24T00:48:51.994 INFO:tasks.workunit.client.1.smithi088.stdout:2/450: readlink db/dc/l23 0 2024-03-24T00:48:51.998 INFO:tasks.workunit.client.0.smithi038.stdout:6/468: link d1/da/d1b/d25/d49/d4e/f66 d1/da/d1b/d25/d49/d4e/f95 0 2024-03-24T00:48:51.998 INFO:tasks.workunit.client.0.smithi038.stdout:6/469: chown d1/d37/d68/f85 26 1 2024-03-24T00:48:52.001 INFO:tasks.workunit.client.1.smithi088.stdout:5/485: link d1/d29/d3f/c7d d1/d19/c9c 0 2024-03-24T00:48:52.001 INFO:tasks.workunit.client.1.smithi088.stdout:5/486: fdatasync d1/d29/f50 0 2024-03-24T00:48:52.002 INFO:tasks.workunit.client.1.smithi088.stdout:5/487: chown d1/d29/d3f 154961 1 2024-03-24T00:48:52.002 INFO:tasks.workunit.client.1.smithi088.stdout:8/491: dread d9/dc/d11/d41/d54/d5d/f60 [0,4194304] 0 2024-03-24T00:48:52.002 INFO:tasks.workunit.client.1.smithi088.stdout:8/492: fsync d9/dc/f5c 0 2024-03-24T00:48:52.012 INFO:tasks.workunit.client.0.smithi038.stdout:7/446: rename d2/d2c/l34 to d2/d9/da/d23/d2b/d5b/d72/d92/d5d/la6 0 2024-03-24T00:48:52.013 INFO:tasks.workunit.client.1.smithi088.stdout:2/451: dread db/d21/f44 [0,4194304] 0 2024-03-24T00:48:52.020 INFO:tasks.workunit.client.0.smithi038.stdout:1/487: mknod da/dd/d22/d5c/ca8 0 2024-03-24T00:48:52.020 INFO:tasks.workunit.client.0.smithi038.stdout:1/488: write da/dd/d2c/d5a/f7b [151372,9540] 0 2024-03-24T00:48:52.022 INFO:tasks.workunit.client.1.smithi088.stdout:6/486: creat db/d40/f8f x:0 0 0 2024-03-24T00:48:52.022 INFO:tasks.workunit.client.1.smithi088.stdout:6/487: write db/d40/f5a [1361340,18389] 0 2024-03-24T00:48:52.029 INFO:tasks.workunit.client.0.smithi038.stdout:8/437: mknod d0/d10/c99 0 2024-03-24T00:48:52.037 INFO:tasks.workunit.client.0.smithi038.stdout:1/489: dread da/dd/f85 [0,4194304] 0 2024-03-24T00:48:52.041 INFO:tasks.workunit.client.0.smithi038.stdout:5/427: mkdir d6/d19/d20/d64/d6b/d90 0 2024-03-24T00:48:52.054 INFO:tasks.workunit.client.0.smithi038.stdout:2/442: dwrite d4/d27/d54/d53/f7f [0,4194304] 0 2024-03-24T00:48:52.058 INFO:tasks.workunit.client.1.smithi088.stdout:1/504: mknod d6/d43/c97 0 2024-03-24T00:48:52.058 INFO:tasks.workunit.client.1.smithi088.stdout:9/489: rename d1/d10/d29/d6d/c6a to d1/d22/d27/d39/d98/c9a 0 2024-03-24T00:48:52.063 INFO:tasks.workunit.client.1.smithi088.stdout:8/493: truncate d9/dc/d11/d24/f71 3397471 0 2024-03-24T00:48:52.069 INFO:tasks.workunit.client.0.smithi038.stdout:8/438: dread d0/d45/f55 [0,4194304] 0 2024-03-24T00:48:52.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/439: write d0/d15/f63 [608145,121990] 0 2024-03-24T00:48:52.075 INFO:tasks.workunit.client.0.smithi038.stdout:9/481: mkdir d12/d41/d9d 0 2024-03-24T00:48:52.076 INFO:tasks.workunit.client.0.smithi038.stdout:4/382: rename da/d12/d24/d5f/l6e to da/d12/d24/d5f/l78 0 2024-03-24T00:48:52.082 INFO:tasks.workunit.client.0.smithi038.stdout:6/470: dwrite d1/d37/f90 [0,4194304] 0 2024-03-24T00:48:52.083 INFO:tasks.workunit.client.0.smithi038.stdout:6/471: stat d1/da/d1b/d25/d49/l65 0 2024-03-24T00:48:52.083 INFO:tasks.workunit.client.0.smithi038.stdout:6/472: write d1/da/f89 [1205270,50173] 0 2024-03-24T00:48:52.083 INFO:tasks.workunit.client.0.smithi038.stdout:6/473: stat d1/da/d73/d75 0 2024-03-24T00:48:52.087 INFO:tasks.workunit.client.0.smithi038.stdout:7/447: rename d2/d9/da/d23/d2b/d5b/fa4 to d2/d9/fa7 0 2024-03-24T00:48:52.093 INFO:tasks.workunit.client.1.smithi088.stdout:2/452: mkdir db/d91 0 2024-03-24T00:48:52.093 INFO:tasks.workunit.client.1.smithi088.stdout:2/453: chown db/d21/d4a/d4e/f6e 105308109 1 2024-03-24T00:48:52.095 INFO:tasks.workunit.client.0.smithi038.stdout:1/490: mknod da/dd/d2c/d40/ca9 0 2024-03-24T00:48:52.097 INFO:tasks.workunit.client.1.smithi088.stdout:0/473: rename d9/dc/de/d19/d44/d46/d65/l37 to d9/dc/d4f/l96 0 2024-03-24T00:48:52.097 INFO:tasks.workunit.client.0.smithi038.stdout:9/482: stat d12/d55/f67 0 2024-03-24T00:48:52.098 INFO:tasks.workunit.client.1.smithi088.stdout:9/490: mkdir d1/d10/d9b 0 2024-03-24T00:48:52.101 INFO:tasks.workunit.client.1.smithi088.stdout:8/494: mknod d9/dc/d11/d24/d43/d4a/d81/ca3 0 2024-03-24T00:48:52.101 INFO:tasks.workunit.client.1.smithi088.stdout:8/495: creat d9/dc/d11/d41/d54/d5d/fa4 x:0 0 0 2024-03-24T00:48:52.102 INFO:tasks.workunit.client.0.smithi038.stdout:2/443: mkdir d4/da3 0 2024-03-24T00:48:52.104 INFO:tasks.workunit.client.1.smithi088.stdout:2/454: symlink db/d21/d4a/d4e/d8f/l92 0 2024-03-24T00:48:52.106 INFO:tasks.workunit.client.0.smithi038.stdout:0/361: rename d0/d12/d60/l21 to d0/d12/d42/l76 0 2024-03-24T00:48:52.106 INFO:tasks.workunit.client.0.smithi038.stdout:0/362: readlink d0/d12/d42/l76 0 2024-03-24T00:48:52.107 INFO:tasks.workunit.client.0.smithi038.stdout:0/363: fdatasync d0/f5 0 2024-03-24T00:48:52.111 INFO:tasks.workunit.client.1.smithi088.stdout:4/464: rename d5/d13/d18/l70 to d5/d17/l9f 0 2024-03-24T00:48:52.113 INFO:tasks.workunit.client.1.smithi088.stdout:3/434: write d3/de/f16 [902493,72768] 0 2024-03-24T00:48:52.118 INFO:tasks.workunit.client.1.smithi088.stdout:0/474: mkdir d9/dc/de/d19/d8d/d97 0 2024-03-24T00:48:52.118 INFO:tasks.workunit.client.1.smithi088.stdout:0/475: fsync d9/dc/de/d19/d44/d46/d59/f70 0 2024-03-24T00:48:52.118 INFO:tasks.workunit.client.1.smithi088.stdout:0/476: truncate d9/dc/de/f14 932904 0 2024-03-24T00:48:52.119 INFO:tasks.workunit.client.1.smithi088.stdout:0/477: truncate d9/dc/de/d19/f7f 740718 0 2024-03-24T00:48:52.120 INFO:tasks.workunit.client.1.smithi088.stdout:9/491: mkdir d1/d15/d21/d9c 0 2024-03-24T00:48:52.120 INFO:tasks.workunit.client.1.smithi088.stdout:9/492: truncate d1/d15/d21/f82 199265 0 2024-03-24T00:48:52.120 INFO:tasks.workunit.client.1.smithi088.stdout:9/493: dread - d1/d10/d2c/f7d zero size 2024-03-24T00:48:52.130 INFO:tasks.workunit.client.1.smithi088.stdout:8/496: mkdir d9/dc/d11/d24/d43/d4a/d95/da5 0 2024-03-24T00:48:52.136 INFO:tasks.workunit.client.0.smithi038.stdout:1/491: symlink da/dd/d22/laa 0 2024-03-24T00:48:52.143 INFO:tasks.workunit.client.1.smithi088.stdout:2/455: creat db/dc/f93 x:0 0 0 2024-03-24T00:48:52.147 INFO:tasks.workunit.client.0.smithi038.stdout:9/483: symlink d12/d28/d3a/d8f/l9e 0 2024-03-24T00:48:52.147 INFO:tasks.workunit.client.0.smithi038.stdout:9/484: chown d12/d55/d56/c66 3940848 1 2024-03-24T00:48:52.147 INFO:tasks.workunit.client.0.smithi038.stdout:9/485: fdatasync d12/f42 0 2024-03-24T00:48:52.152 INFO:tasks.workunit.client.0.smithi038.stdout:1/492: dread da/dd/d7a/d82/d98/d14/f2e [0,4194304] 0 2024-03-24T00:48:52.161 INFO:tasks.workunit.client.1.smithi088.stdout:3/435: creat d3/f95 x:0 0 0 2024-03-24T00:48:52.170 INFO:tasks.workunit.client.1.smithi088.stdout:9/494: unlink d1/c9 0 2024-03-24T00:48:52.171 INFO:tasks.workunit.client.1.smithi088.stdout:9/495: truncate d1/d15/f7f 4286088 0 2024-03-24T00:48:52.171 INFO:tasks.workunit.client.1.smithi088.stdout:2/456: dread f1 [0,4194304] 0 2024-03-24T00:48:52.171 INFO:tasks.workunit.client.1.smithi088.stdout:2/457: chown db/f18 71286 1 2024-03-24T00:48:52.172 INFO:tasks.workunit.client.1.smithi088.stdout:3/436: dread d3/de/d6f/d85/d49/f8e [4194304,4194304] 0 2024-03-24T00:48:52.172 INFO:tasks.workunit.client.1.smithi088.stdout:3/437: readlink d3/d67/l6c 0 2024-03-24T00:48:52.172 INFO:tasks.workunit.client.1.smithi088.stdout:3/438: dread - d3/d1a/d1e/d76/f93 zero size 2024-03-24T00:48:52.174 INFO:tasks.workunit.client.0.smithi038.stdout:3/460: rename d3/d9/dc/df/c65 to d3/d9/dc/df/d14/d19/d82/d89/c98 0 2024-03-24T00:48:52.174 INFO:tasks.workunit.client.1.smithi088.stdout:8/497: truncate d9/dc/d11/d24/d29/d36/f86 1274740 0 2024-03-24T00:48:52.178 INFO:tasks.workunit.client.0.smithi038.stdout:0/364: mknod d0/c77 0 2024-03-24T00:48:52.186 INFO:tasks.workunit.client.0.smithi038.stdout:4/383: dwrite da/d12/d37/dd/d15/f64 [0,4194304] 0 2024-03-24T00:48:52.187 INFO:tasks.workunit.client.1.smithi088.stdout:4/465: dwrite d5/d17/f1d [0,4194304] 0 2024-03-24T00:48:52.187 INFO:tasks.workunit.client.1.smithi088.stdout:4/466: creat d5/fa0 x:0 0 0 2024-03-24T00:48:52.187 INFO:tasks.workunit.client.1.smithi088.stdout:4/467: creat d5/d13/d18/d24/fa1 x:0 0 0 2024-03-24T00:48:52.190 INFO:tasks.workunit.client.1.smithi088.stdout:9/496: mkdir d1/d22/d27/d39/d62/d7a/d9d 0 2024-03-24T00:48:52.190 INFO:tasks.workunit.client.1.smithi088.stdout:9/497: creat d1/d22/d72/f9e x:0 0 0 2024-03-24T00:48:52.192 INFO:tasks.workunit.client.1.smithi088.stdout:0/478: dwrite d9/f1c [0,4194304] 0 2024-03-24T00:48:52.193 INFO:tasks.workunit.client.0.smithi038.stdout:1/493: mkdir da/dd/d22/dab 0 2024-03-24T00:48:52.193 INFO:tasks.workunit.client.0.smithi038.stdout:1/494: chown da/dd/d22 28 1 2024-03-24T00:48:52.194 INFO:tasks.workunit.client.0.smithi038.stdout:1/495: write da/dd/d2c/f63 [809769,82242] 0 2024-03-24T00:48:52.194 INFO:tasks.workunit.client.0.smithi038.stdout:1/496: read da/dd/d2c/d5a/d74/f88 [2322795,109976] 0 2024-03-24T00:48:52.196 INFO:tasks.workunit.client.1.smithi088.stdout:7/401: sync 2024-03-24T00:48:52.201 INFO:tasks.workunit.client.1.smithi088.stdout:2/458: rmdir db/d91 0 2024-03-24T00:48:52.205 INFO:tasks.workunit.client.1.smithi088.stdout:3/439: mkdir d3/de/d6f/d85/d49/d70/d96 0 2024-03-24T00:48:52.205 INFO:tasks.workunit.client.1.smithi088.stdout:8/498: truncate d9/dc/d11/d24/d43/d4a/d81/f91 3148898 0 2024-03-24T00:48:52.207 INFO:tasks.workunit.client.1.smithi088.stdout:5/488: sync 2024-03-24T00:48:52.210 INFO:tasks.workunit.client.1.smithi088.stdout:7/402: dread d3/f27 [0,4194304] 0 2024-03-24T00:48:52.215 INFO:tasks.workunit.client.0.smithi038.stdout:6/474: dwrite d1/da/d1b/d2f/f3d [0,4194304] 0 2024-03-24T00:48:52.218 INFO:tasks.workunit.client.1.smithi088.stdout:9/498: write d1/d10/f1e [577946,68679] 0 2024-03-24T00:48:52.220 INFO:tasks.workunit.client.1.smithi088.stdout:4/468: mknod d5/d7/d15/d74/ca2 0 2024-03-24T00:48:52.224 INFO:tasks.workunit.client.1.smithi088.stdout:6/488: rename db/d40 to db/d1d/d50/d90 0 2024-03-24T00:48:52.224 INFO:tasks.workunit.client.1.smithi088.stdout:6/489: fsync db/d1d/d50/d90/d56/f51 0 2024-03-24T00:48:52.224 INFO:tasks.workunit.client.1.smithi088.stdout:6/490: truncate db/d1d/d23/d5c/f67 1014861 0 2024-03-24T00:48:52.227 INFO:tasks.workunit.client.0.smithi038.stdout:2/444: rmdir d4/d27 39 2024-03-24T00:48:52.227 INFO:tasks.workunit.client.0.smithi038.stdout:2/445: getdents d4/d18/d96 0 2024-03-24T00:48:52.227 INFO:tasks.workunit.client.0.smithi038.stdout:2/446: write d4/d18/d26/f8e [33208,104944] 0 2024-03-24T00:48:52.234 INFO:tasks.workunit.client.1.smithi088.stdout:0/479: creat d9/dc/de/d19/d8d/f98 x:0 0 0 2024-03-24T00:48:52.234 INFO:tasks.workunit.client.1.smithi088.stdout:0/480: fsync d9/dc/de/d19/d44/f6e 0 2024-03-24T00:48:52.234 INFO:tasks.workunit.client.1.smithi088.stdout:2/459: mknod db/d21/d4a/c94 0 2024-03-24T00:48:52.239 INFO:tasks.workunit.client.0.smithi038.stdout:0/365: creat d0/f78 x:0 0 0 2024-03-24T00:48:52.241 INFO:tasks.workunit.client.1.smithi088.stdout:5/489: mkdir d1/d10/d1b/d9d 0 2024-03-24T00:48:52.250 INFO:tasks.workunit.client.1.smithi088.stdout:4/469: dread d5/f9 [0,4194304] 0 2024-03-24T00:48:52.252 INFO:tasks.workunit.client.0.smithi038.stdout:2/447: dread d4/d18/d26/f4e [4194304,4194304] 0 2024-03-24T00:48:52.252 INFO:tasks.workunit.client.0.smithi038.stdout:2/448: dread - d4/d18/d26/f8c zero size 2024-03-24T00:48:52.253 INFO:tasks.workunit.client.1.smithi088.stdout:9/499: creat d1/d22/d27/d39/d62/d7a/d9d/f9f x:0 0 0 2024-03-24T00:48:52.253 INFO:tasks.workunit.client.1.smithi088.stdout:9/500: truncate d1/d10/d11/f5f 258125 0 2024-03-24T00:48:52.253 INFO:tasks.workunit.client.1.smithi088.stdout:6/491: dread db/f19 [0,4194304] 0 2024-03-24T00:48:52.256 INFO:tasks.workunit.client.1.smithi088.stdout:1/505: rename d6/l83 to d6/d6e/l98 0 2024-03-24T00:48:52.293 INFO:tasks.workunit.client.0.smithi038.stdout:4/384: link da/f1e da/d12/d24/d68/d6c/f79 0 2024-03-24T00:48:52.293 INFO:tasks.workunit.client.0.smithi038.stdout:9/486: getdents d12/d41/d57 0 2024-03-24T00:48:52.293 INFO:tasks.workunit.client.0.smithi038.stdout:1/497: symlink da/dd/d22/d5c/lac 0 2024-03-24T00:48:52.293 INFO:tasks.workunit.client.0.smithi038.stdout:6/475: symlink d1/da/d73/d75/l96 0 2024-03-24T00:48:52.293 INFO:tasks.workunit.client.0.smithi038.stdout:0/366: symlink d0/d14/l79 0 2024-03-24T00:48:52.293 INFO:tasks.workunit.client.0.smithi038.stdout:0/367: readlink d0/d1b/l56 0 2024-03-24T00:48:52.294 INFO:tasks.workunit.client.1.smithi088.stdout:2/460: creat db/d21/d35/f95 x:0 0 0 2024-03-24T00:48:52.294 INFO:tasks.workunit.client.1.smithi088.stdout:2/461: fsync db/d21/f44 0 2024-03-24T00:48:52.294 INFO:tasks.workunit.client.1.smithi088.stdout:7/403: dread d3/f1f [4194304,4194304] 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:8/499: dwrite d9/f37 [4194304,4194304] 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:2/462: write db/d21/d4a/f7c [115060,122560] 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:5/490: mkdir d1/d10/d1b/d2f/d57/d9e 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:3/440: dwrite d3/de/d6f/d85/d49/f54 [0,4194304] 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:3/441: chown d3/de/d6f/d85/c8f 73069 1 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:9/501: write d1/d10/d29/f2e [452910,91219] 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:9/502: creat d1/d10/d11/d23/fa0 x:0 0 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:9/503: truncate d1/d22/d72/f9e 678491 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:5/491: dread d1/d29/d3f/d88/d8f/d69/f8c [0,4194304] 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:6/492: creat db/dd/d61/d6d/f91 x:0 0 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:1/506: unlink d6/d23/f4a 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:3/442: read d3/d6/d19/d2a/d5f/f65 [2539282,63214] 0 2024-03-24T00:48:52.295 INFO:tasks.workunit.client.1.smithi088.stdout:3/443: write d3/d6/d27/d56/d6e/f8a [726954,43212] 0 2024-03-24T00:48:52.296 INFO:tasks.workunit.client.0.smithi038.stdout:2/449: rmdir d4/d27/d2f/d30/d3e/d9b 39 2024-03-24T00:48:52.301 INFO:tasks.workunit.client.1.smithi088.stdout:8/500: mknod d9/dc/d11/d24/d29/d39/d84/ca6 0 2024-03-24T00:48:52.307 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:52 smithi038 ceph-mon[29271]: pgmap v32: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 205 MiB/s rd, 254 MiB/s wr, 367 op/s 2024-03-24T00:48:52.308 INFO:tasks.workunit.client.0.smithi038.stdout:4/385: unlink da/d12/f2c 0 2024-03-24T00:48:52.309 INFO:tasks.workunit.client.0.smithi038.stdout:6/476: read d1/f6 [909626,70644] 0 2024-03-24T00:48:52.309 INFO:tasks.workunit.client.0.smithi038.stdout:6/477: write d1/da/d1b/f2b [924469,4430] 0 2024-03-24T00:48:52.310 INFO:tasks.workunit.client.1.smithi088.stdout:0/481: dwrite d9/dc/de/f5c [0,4194304] 0 2024-03-24T00:48:52.315 INFO:tasks.workunit.client.0.smithi038.stdout:9/487: rename d12/d55/d56/c68 to d12/d55/c9f 0 2024-03-24T00:48:52.315 INFO:tasks.workunit.client.1.smithi088.stdout:4/470: dwrite d5/d7/f44 [0,4194304] 0 2024-03-24T00:48:52.315 INFO:tasks.workunit.client.1.smithi088.stdout:4/471: chown d5/d7/dd/f75 1875122322 1 2024-03-24T00:48:52.318 INFO:tasks.workunit.client.1.smithi088.stdout:9/504: chown d1/d10/d77/d8a/c24 26 1 2024-03-24T00:48:52.320 INFO:tasks.workunit.client.0.smithi038.stdout:0/368: creat d0/d12/d15/d1a/f7a x:0 0 0 2024-03-24T00:48:52.322 INFO:tasks.workunit.client.0.smithi038.stdout:6/478: symlink d1/da/d73/l97 0 2024-03-24T00:48:52.322 INFO:tasks.workunit.client.0.smithi038.stdout:6/479: dread - d1/da/d1b/d25/d49/d4e/f66 zero size 2024-03-24T00:48:52.322 INFO:tasks.workunit.client.0.smithi038.stdout:6/480: chown d1/f6b 8581 1 2024-03-24T00:48:52.325 INFO:tasks.workunit.client.1.smithi088.stdout:1/507: mknod d6/d43/d95/d5b/d5d/c99 0 2024-03-24T00:48:52.325 INFO:tasks.workunit.client.1.smithi088.stdout:5/492: truncate d1/d10/d1b/d2f/f4a 1415736 0 2024-03-24T00:48:52.328 INFO:tasks.workunit.client.0.smithi038.stdout:0/369: symlink d0/d12/d60/d28/l7b 0 2024-03-24T00:48:52.331 INFO:tasks.workunit.client.1.smithi088.stdout:7/404: mknod d3/c7d 0 2024-03-24T00:48:52.331 INFO:tasks.workunit.client.1.smithi088.stdout:7/405: creat d3/d31/f7e x:0 0 0 2024-03-24T00:48:52.336 INFO:tasks.workunit.client.0.smithi038.stdout:6/481: symlink d1/da/d1b/d25/d92/l98 0 2024-03-24T00:48:52.340 INFO:tasks.workunit.client.1.smithi088.stdout:0/482: symlink d9/d15/l99 0 2024-03-24T00:48:52.341 INFO:tasks.workunit.client.0.smithi038.stdout:3/461: dwrite d3/d9/dc/df/d14/d19/d82/f93 [0,4194304] 0 2024-03-24T00:48:52.341 INFO:tasks.workunit.client.1.smithi088.stdout:0/483: creat d9/dc/de/d19/d44/d46/d65/d6f/f9a x:0 0 0 2024-03-24T00:48:52.376 INFO:tasks.workunit.client.1.smithi088.stdout:2/463: rename db/d21/d4a to db/dd/d96 0 2024-03-24T00:48:52.380 INFO:tasks.workunit.client.0.smithi038.stdout:3/462: mkdir d3/d30/d70/d90/d99 0 2024-03-24T00:48:52.385 INFO:tasks.workunit.client.0.smithi038.stdout:3/463: chown d3/d30/d70/d90/d99 3804 1 2024-03-24T00:48:52.385 INFO:tasks.workunit.client.0.smithi038.stdout:4/386: dread da/d12/d37/dd/d15/d4c/f5a [0,4194304] 0 2024-03-24T00:48:52.388 INFO:tasks.workunit.client.1.smithi088.stdout:5/493: unlink d1/d10/d1b/d2e/f65 0 2024-03-24T00:48:52.388 INFO:tasks.workunit.client.1.smithi088.stdout:5/494: chown d1/d29/l34 47781882 1 2024-03-24T00:48:52.395 INFO:tasks.workunit.client.1.smithi088.stdout:9/505: dwrite d1/d22/f93 [0,4194304] 0 2024-03-24T00:48:52.396 INFO:tasks.workunit.client.1.smithi088.stdout:3/444: dwrite d3/de/d6f/d85/d31/f7e [4194304,4194304] 0 2024-03-24T00:48:52.397 INFO:tasks.workunit.client.1.smithi088.stdout:6/493: dwrite db/d1d/d50/d90/f5a [0,4194304] 0 2024-03-24T00:48:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:1/498: dwrite da/dd/d22/d5c/f7f [0,4194304] 0 2024-03-24T00:48:52.398 INFO:tasks.workunit.client.1.smithi088.stdout:5/495: dread d1/d19/d1e/f76 [0,4194304] 0 2024-03-24T00:48:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:3/464: dread d3/d9/f2b [0,4194304] 0 2024-03-24T00:48:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:3/465: chown d3/d9/dc/df/d14/f53 204 1 2024-03-24T00:48:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:3/466: write d3/d9/dc/d51/f52 [491880,36147] 0 2024-03-24T00:48:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:3/467: truncate d3/d30/d70/f7b 436449 0 2024-03-24T00:48:52.401 INFO:tasks.workunit.client.1.smithi088.stdout:7/406: mkdir d3/d32/d3f/d4b/d74/d79/d7f 0 2024-03-24T00:48:52.401 INFO:tasks.workunit.client.1.smithi088.stdout:7/407: chown d3/f24 129456574 1 2024-03-24T00:48:52.402 INFO:tasks.workunit.client.1.smithi088.stdout:1/508: dwrite d6/d43/d95/d58/f7c [0,4194304] 0 2024-03-24T00:48:52.403 INFO:tasks.workunit.client.0.smithi038.stdout:9/488: dwrite d12/d1a/d80/f97 [0,4194304] 0 2024-03-24T00:48:52.405 INFO:tasks.workunit.client.0.smithi038.stdout:1/499: truncate f5 1553295 0 2024-03-24T00:48:52.410 INFO:tasks.workunit.client.1.smithi088.stdout:0/484: rmdir d9/dc/de/d19 39 2024-03-24T00:48:52.412 INFO:tasks.workunit.client.0.smithi038.stdout:6/482: dwrite d1/f7 [0,4194304] 0 2024-03-24T00:48:52.412 INFO:tasks.workunit.client.0.smithi038.stdout:2/450: dwrite d4/d18/f7b [0,4194304] 0 2024-03-24T00:48:52.422 INFO:tasks.workunit.client.1.smithi088.stdout:2/464: creat db/d54/d88/f97 x:0 0 0 2024-03-24T00:48:52.429 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:52 smithi088 ceph-mon[36610]: pgmap v32: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 205 MiB/s rd, 254 MiB/s wr, 367 op/s 2024-03-24T00:48:52.446 INFO:tasks.workunit.client.0.smithi038.stdout:6/483: symlink d1/da/d1b/d2f/l99 0 2024-03-24T00:48:52.446 INFO:tasks.workunit.client.0.smithi038.stdout:2/451: creat d4/d18/da1/fa4 x:0 0 0 2024-03-24T00:48:52.456 INFO:tasks.workunit.client.1.smithi088.stdout:8/501: dwrite d9/dc/d11/d24/d29/d36/f97 [0,4194304] 0 2024-03-24T00:48:52.456 INFO:tasks.workunit.client.1.smithi088.stdout:8/502: read d9/f2b [3953448,121313] 0 2024-03-24T00:48:52.456 INFO:tasks.workunit.client.1.smithi088.stdout:9/506: truncate d1/f7 4600876 0 2024-03-24T00:48:52.461 INFO:tasks.workunit.client.1.smithi088.stdout:6/494: creat db/d1d/d23/d3c/f92 x:0 0 0 2024-03-24T00:48:52.461 INFO:tasks.workunit.client.1.smithi088.stdout:6/495: read db/d1d/d23/d3c/d4b/f6e [463875,15552] 0 2024-03-24T00:48:52.462 INFO:tasks.workunit.client.1.smithi088.stdout:3/445: dwrite d3/de/d6f/d85/d31/f7e [4194304,4194304] 0 2024-03-24T00:48:52.462 INFO:tasks.workunit.client.1.smithi088.stdout:3/446: fsync d3/d6/d19/d2a/f53 0 2024-03-24T00:48:52.465 INFO:tasks.workunit.client.0.smithi038.stdout:2/452: creat d4/d27/d54/d53/fa5 x:0 0 0 2024-03-24T00:48:52.469 INFO:tasks.workunit.client.0.smithi038.stdout:1/500: dread f7 [0,4194304] 0 2024-03-24T00:48:52.471 INFO:tasks.workunit.client.1.smithi088.stdout:5/496: rmdir d1/d29/d3f/d88/d8f 39 2024-03-24T00:48:52.476 INFO:tasks.workunit.client.0.smithi038.stdout:4/387: dwrite da/d38/f3c [0,4194304] 0 2024-03-24T00:48:52.478 INFO:tasks.workunit.client.1.smithi088.stdout:9/507: dread d1/d15/f37 [0,4194304] 0 2024-03-24T00:48:52.479 INFO:tasks.workunit.client.0.smithi038.stdout:6/484: dread d1/da/d1b/f2b [0,4194304] 0 2024-03-24T00:48:52.479 INFO:tasks.workunit.client.0.smithi038.stdout:6/485: stat d1/da/d61/d79 0 2024-03-24T00:48:52.479 INFO:tasks.workunit.client.1.smithi088.stdout:7/408: rmdir d3/d32/d3f/d4b/d73 39 2024-03-24T00:48:52.484 INFO:tasks.workunit.client.0.smithi038.stdout:3/468: dwrite d3/d9/f2b [0,4194304] 0 2024-03-24T00:48:52.495 INFO:tasks.workunit.client.0.smithi038.stdout:9/489: dwrite d12/d28/d3a/f30 [0,4194304] 0 2024-03-24T00:48:52.495 INFO:tasks.workunit.client.0.smithi038.stdout:9/490: write d12/f42 [857319,18600] 0 2024-03-24T00:48:52.495 INFO:tasks.workunit.client.1.smithi088.stdout:1/509: unlink d6/d27/l36 0 2024-03-24T00:48:52.495 INFO:tasks.workunit.client.1.smithi088.stdout:1/510: fdatasync d6/f3e 0 2024-03-24T00:48:52.497 INFO:tasks.workunit.client.0.smithi038.stdout:4/388: write da/d12/d37/dd/d15/f64 [3737976,21986] 0 2024-03-24T00:48:52.498 INFO:tasks.workunit.client.0.smithi038.stdout:4/389: write da/d12/f5c [8491623,13182] 0 2024-03-24T00:48:52.498 INFO:tasks.workunit.client.0.smithi038.stdout:4/390: fdatasync da/f3b 0 2024-03-24T00:48:52.498 INFO:tasks.workunit.client.1.smithi088.stdout:0/485: mkdir d9/dc/d4f/d62/d9b 0 2024-03-24T00:48:52.499 INFO:tasks.workunit.client.1.smithi088.stdout:0/486: stat d9/dc/de/d19/d44/d46/d65/d6f 0 2024-03-24T00:48:52.499 INFO:tasks.workunit.client.1.smithi088.stdout:2/465: getdents db/d21/d35/d49 0 2024-03-24T00:48:52.508 INFO:tasks.workunit.client.1.smithi088.stdout:8/503: rename d9/dc/f5c to d9/dc/d11/d24/d29/d5b/d6f/fa7 0 2024-03-24T00:48:52.517 INFO:tasks.workunit.client.0.smithi038.stdout:4/391: dread da/d12/f3f [0,4194304] 0 2024-03-24T00:48:52.517 INFO:tasks.workunit.client.0.smithi038.stdout:4/392: chown da/d12/d37/dd/f2b 0 1 2024-03-24T00:48:52.523 INFO:tasks.workunit.client.1.smithi088.stdout:6/496: mknod db/c93 0 2024-03-24T00:48:52.528 INFO:tasks.workunit.client.1.smithi088.stdout:3/447: rmdir d3/de/d6f/d85 39 2024-03-24T00:48:52.530 INFO:tasks.workunit.client.1.smithi088.stdout:5/497: rmdir d1/d29/d3f/d88/d8b 39 2024-03-24T00:48:52.530 INFO:tasks.workunit.client.1.smithi088.stdout:5/498: write d1/d19/f30 [8173046,107518] 0 2024-03-24T00:48:52.536 INFO:tasks.workunit.client.1.smithi088.stdout:9/508: mknod d1/d15/d80/ca1 0 2024-03-24T00:48:52.537 INFO:tasks.workunit.client.1.smithi088.stdout:9/509: chown d1/d15/d21/f82 17689 1 2024-03-24T00:48:52.537 INFO:tasks.workunit.client.1.smithi088.stdout:9/510: chown d1/cd 462 1 2024-03-24T00:48:52.542 INFO:tasks.workunit.client.1.smithi088.stdout:1/511: link d6/d23/c42 d6/d43/c9a 0 2024-03-24T00:48:52.545 INFO:tasks.workunit.client.1.smithi088.stdout:7/409: dwrite d3/d32/d3f/f6f [0,4194304] 0 2024-03-24T00:48:52.558 INFO:tasks.workunit.client.0.smithi038.stdout:3/469: link d3/d5a/c64 d3/d9/dc/df/d14/c9a 0 2024-03-24T00:48:52.558 INFO:tasks.workunit.client.0.smithi038.stdout:3/470: chown c0 269909 1 2024-03-24T00:48:52.558 INFO:tasks.workunit.client.1.smithi088.stdout:2/466: link db/dd/d96/l7b db/dd/d96/l98 0 2024-03-24T00:48:52.558 INFO:tasks.workunit.client.1.smithi088.stdout:8/504: mknod d9/dc/d11/d24/d29/d5b/ca8 0 2024-03-24T00:48:52.560 INFO:tasks.workunit.client.1.smithi088.stdout:6/497: creat db/d8e/f94 x:0 0 0 2024-03-24T00:48:52.560 INFO:tasks.workunit.client.1.smithi088.stdout:6/498: creat db/d1d/d23/d5c/f95 x:0 0 0 2024-03-24T00:48:52.560 INFO:tasks.workunit.client.1.smithi088.stdout:6/499: fdatasync db/d1d/d23/d3c/d43/f39 0 2024-03-24T00:48:52.560 INFO:tasks.workunit.client.1.smithi088.stdout:6/500: chown db/d1d/d23/d3c/d43/c36 542884 1 2024-03-24T00:48:52.561 INFO:tasks.workunit.client.1.smithi088.stdout:5/499: mkdir d1/d10/d1b/d2e/d9f 0 2024-03-24T00:48:52.561 INFO:tasks.workunit.client.1.smithi088.stdout:5/500: write d1/d29/f50 [4286181,79441] 0 2024-03-24T00:48:52.562 INFO:tasks.workunit.client.0.smithi038.stdout:9/491: truncate fc 4754989 0 2024-03-24T00:48:52.565 INFO:tasks.workunit.client.1.smithi088.stdout:9/511: rename d1/d22/d27/d39/d62/c63 to d1/d15/d80/ca2 0 2024-03-24T00:48:52.567 INFO:tasks.workunit.client.1.smithi088.stdout:1/512: mkdir d6/d27/d67/d9b 0 2024-03-24T00:48:52.567 INFO:tasks.workunit.client.1.smithi088.stdout:4/472: sync 2024-03-24T00:48:52.567 INFO:tasks.workunit.client.1.smithi088.stdout:4/473: readlink d5/d7/d15/d74/l8f 0 2024-03-24T00:48:52.567 INFO:tasks.workunit.client.1.smithi088.stdout:4/474: write d5/d7/dd/f21 [1078415,49036] 0 2024-03-24T00:48:52.570 INFO:tasks.workunit.client.0.smithi038.stdout:9/492: creat d12/d41/d57/fa0 x:0 0 0 2024-03-24T00:48:52.570 INFO:tasks.workunit.client.0.smithi038.stdout:9/493: fdatasync d12/d41/f6a 0 2024-03-24T00:48:52.570 INFO:tasks.workunit.client.0.smithi038.stdout:9/494: chown d12/d28/f51 6302257 1 2024-03-24T00:48:52.576 INFO:tasks.workunit.client.0.smithi038.stdout:1/501: dwrite da/dd/d22/d5c/f6e [0,4194304] 0 2024-03-24T00:48:52.579 INFO:tasks.workunit.client.0.smithi038.stdout:3/471: symlink d3/l9b 0 2024-03-24T00:48:52.579 INFO:tasks.workunit.client.0.smithi038.stdout:3/472: chown d3/d9/dc/df/d14/d19/f6c 27730 1 2024-03-24T00:48:52.579 INFO:tasks.workunit.client.0.smithi038.stdout:3/473: truncate d3/d9/dc/df/d14/d19/f46 1644994 0 2024-03-24T00:48:52.581 INFO:tasks.workunit.client.1.smithi088.stdout:2/467: rmdir db/dd/d96/d4e/d8f 39 2024-03-24T00:48:52.581 INFO:tasks.workunit.client.1.smithi088.stdout:2/468: fdatasync db/f18 0 2024-03-24T00:48:52.581 INFO:tasks.workunit.client.1.smithi088.stdout:7/410: truncate d3/d32/d34/f72 308417 0 2024-03-24T00:48:52.583 INFO:tasks.workunit.client.0.smithi038.stdout:9/495: unlink d12/d28/d6c/l95 0 2024-03-24T00:48:52.583 INFO:tasks.workunit.client.0.smithi038.stdout:9/496: creat d12/d81/d84/fa1 x:0 0 0 2024-03-24T00:48:52.583 INFO:tasks.workunit.client.0.smithi038.stdout:9/497: fsync d12/d41/f7d 0 2024-03-24T00:48:52.583 INFO:tasks.workunit.client.0.smithi038.stdout:9/498: dread - d12/d28/d3a/d26/f4a zero size 2024-03-24T00:48:52.583 INFO:tasks.workunit.client.0.smithi038.stdout:9/499: chown d12/d28/d71/d99 9930183 1 2024-03-24T00:48:52.586 INFO:tasks.workunit.client.0.smithi038.stdout:6/486: rmdir d1/da/d73/d83 0 2024-03-24T00:48:52.588 INFO:tasks.workunit.client.1.smithi088.stdout:5/501: mkdir d1/dd/d6f/d98/da0 0 2024-03-24T00:48:52.596 INFO:tasks.workunit.client.0.smithi038.stdout:1/502: dread f5 [0,4194304] 0 2024-03-24T00:48:52.596 INFO:tasks.workunit.client.0.smithi038.stdout:1/503: readlink da/d3c/l1a 0 2024-03-24T00:48:52.598 INFO:tasks.workunit.client.1.smithi088.stdout:2/469: dread db/dd/d96/d4e/f64 [0,4194304] 0 2024-03-24T00:48:52.598 INFO:tasks.workunit.client.1.smithi088.stdout:2/470: chown db/l12 38088497 1 2024-03-24T00:48:52.598 INFO:tasks.workunit.client.1.smithi088.stdout:2/471: readlink db/l80 0 2024-03-24T00:48:52.598 INFO:tasks.workunit.client.1.smithi088.stdout:8/505: rename d9/dc/d11/d24/d29/d5b/d6f/d7c to d9/dc/d11/d24/d29/da9 0 2024-03-24T00:48:52.601 INFO:tasks.workunit.client.0.smithi038.stdout:4/393: dwrite da/d38/f3c [0,4194304] 0 2024-03-24T00:48:52.602 INFO:tasks.workunit.client.0.smithi038.stdout:4/394: write da/d12/f57 [1866153,80909] 0 2024-03-24T00:48:52.602 INFO:tasks.workunit.client.0.smithi038.stdout:2/453: dwrite d4/d18/d38/f82 [0,4194304] 0 2024-03-24T00:48:52.604 INFO:tasks.workunit.client.0.smithi038.stdout:3/474: mknod d3/d80/c9c 0 2024-03-24T00:48:52.605 INFO:tasks.workunit.client.1.smithi088.stdout:1/513: mknod d6/d43/d95/c9c 0 2024-03-24T00:48:52.609 INFO:tasks.workunit.client.1.smithi088.stdout:4/475: symlink d5/d13/d18/d6e/la3 0 2024-03-24T00:48:52.614 INFO:tasks.workunit.client.1.smithi088.stdout:5/502: truncate d1/fb 56857 0 2024-03-24T00:48:52.615 INFO:tasks.workunit.client.1.smithi088.stdout:2/472: mkdir db/dd/d96/d99 0 2024-03-24T00:48:52.615 INFO:tasks.workunit.client.1.smithi088.stdout:2/473: truncate db/dc/f93 825136 0 2024-03-24T00:48:52.618 INFO:tasks.workunit.client.1.smithi088.stdout:6/501: rename db/d1d/d50/d90/d55/c73 to db/d1d/d23/d5c/c96 0 2024-03-24T00:48:52.620 INFO:tasks.workunit.client.0.smithi038.stdout:9/500: mknod d12/d1a/ca2 0 2024-03-24T00:48:52.624 INFO:tasks.workunit.client.1.smithi088.stdout:8/506: unlink d9/dc/d11/d24/d29/d5b/c70 0 2024-03-24T00:48:52.626 INFO:tasks.workunit.client.1.smithi088.stdout:7/411: dwrite d3/d32/d3f/d4b/f6c [0,4194304] 0 2024-03-24T00:48:52.632 INFO:tasks.workunit.client.1.smithi088.stdout:1/514: dread d6/d23/f34 [0,4194304] 0 2024-03-24T00:48:52.632 INFO:tasks.workunit.client.1.smithi088.stdout:1/515: write d6/d43/f50 [197601,96821] 0 2024-03-24T00:48:52.632 INFO:tasks.workunit.client.1.smithi088.stdout:1/516: chown d6/d43/d95/d58/f81 65 1 2024-03-24T00:48:52.632 INFO:tasks.workunit.client.1.smithi088.stdout:1/517: truncate d6/d43/d95/d3f/f57 739348 0 2024-03-24T00:48:52.632 INFO:tasks.workunit.client.0.smithi038.stdout:1/504: unlink da/dd/d2c/d40/l77 0 2024-03-24T00:48:52.634 INFO:tasks.workunit.client.1.smithi088.stdout:1/518: write d6/d43/d95/d5b/d5e/f66 [628971,2002] 0 2024-03-24T00:48:52.638 INFO:tasks.workunit.client.1.smithi088.stdout:4/476: symlink d5/d7/d15/la4 0 2024-03-24T00:48:52.638 INFO:tasks.workunit.client.1.smithi088.stdout:4/477: chown d5/d13/f86 1143 1 2024-03-24T00:48:52.638 INFO:tasks.workunit.client.1.smithi088.stdout:2/474: symlink db/dd/d96/d99/l9a 0 2024-03-24T00:48:52.638 INFO:tasks.workunit.client.1.smithi088.stdout:2/475: truncate db/dc/f2c 4526538 0 2024-03-24T00:48:52.638 INFO:tasks.workunit.client.1.smithi088.stdout:2/476: fsync db/d21/d35/f95 0 2024-03-24T00:48:52.638 INFO:tasks.workunit.client.1.smithi088.stdout:2/477: chown db/dd/c34 435950 1 2024-03-24T00:48:52.639 INFO:tasks.workunit.client.0.smithi038.stdout:4/395: creat da/d12/d37/dd/d15/d4c/f7a x:0 0 0 2024-03-24T00:48:52.639 INFO:tasks.workunit.client.0.smithi038.stdout:4/396: chown da/d12/d37/dd/d15/d4c/f53 7843905 1 2024-03-24T00:48:52.639 INFO:tasks.workunit.client.0.smithi038.stdout:2/454: mkdir d4/d18/d26/d37/d69/da6 0 2024-03-24T00:48:52.640 INFO:tasks.workunit.client.1.smithi088.stdout:8/507: creat d9/dc/d11/d24/d29/d39/d84/faa x:0 0 0 2024-03-24T00:48:52.642 INFO:tasks.workunit.client.0.smithi038.stdout:9/501: link d12/c91 d12/d28/d6c/ca3 0 2024-03-24T00:48:52.642 INFO:tasks.workunit.client.0.smithi038.stdout:9/502: read - d12/d81/d84/fa1 zero size 2024-03-24T00:48:52.648 INFO:tasks.workunit.client.0.smithi038.stdout:6/487: dwrite d1/da/d1b/f91 [0,4194304] 0 2024-03-24T00:48:52.648 INFO:tasks.workunit.client.0.smithi038.stdout:3/475: dread d3/d9/fb [0,4194304] 0 2024-03-24T00:48:52.648 INFO:tasks.workunit.client.0.smithi038.stdout:3/476: chown d3/d9/dc/df/d14/d19/l72 133524577 1 2024-03-24T00:48:52.658 INFO:tasks.workunit.client.0.smithi038.stdout:1/505: creat da/dd/d4a/d76/fad x:0 0 0 2024-03-24T00:48:52.658 INFO:tasks.workunit.client.0.smithi038.stdout:1/506: chown da/dd 171836 1 2024-03-24T00:48:52.659 INFO:tasks.workunit.client.1.smithi088.stdout:7/412: readlink d3/l13 0 2024-03-24T00:48:52.659 INFO:tasks.workunit.client.0.smithi038.stdout:4/397: dread da/d12/d37/dd/d15/f64 [0,4194304] 0 2024-03-24T00:48:52.660 INFO:tasks.workunit.client.0.smithi038.stdout:4/398: read da/d12/d37/dd/d15/d4c/f53 [662393,89513] 0 2024-03-24T00:48:52.660 INFO:tasks.workunit.client.0.smithi038.stdout:4/399: write da/d12/d24/d68/d6c/f79 [311722,76842] 0 2024-03-24T00:48:52.663 INFO:tasks.workunit.client.0.smithi038.stdout:2/455: creat d4/d27/d2f/d30/d3e/d9b/fa7 x:0 0 0 2024-03-24T00:48:52.666 INFO:tasks.workunit.client.0.smithi038.stdout:9/503: rename d12/d1a/l38 to d12/d55/d56/la4 0 2024-03-24T00:48:52.694 INFO:tasks.workunit.client.1.smithi088.stdout:9/512: rename d1/d22/d27/d39/d62/d7a/d9d to d1/d22/d27/d39/d98/da3 0 2024-03-24T00:48:52.694 INFO:tasks.workunit.client.1.smithi088.stdout:9/513: truncate d1/d10/d2c/f75 751614 0 2024-03-24T00:48:52.695 INFO:tasks.workunit.client.1.smithi088.stdout:7/413: creat d3/d32/d3f/d4b/d74/d79/d7f/f80 x:0 0 0 2024-03-24T00:48:52.698 INFO:tasks.workunit.client.0.smithi038.stdout:6/488: symlink d1/da/d1b/d1d/l9a 0 2024-03-24T00:48:52.701 INFO:tasks.workunit.client.0.smithi038.stdout:3/477: symlink d3/d80/l9d 0 2024-03-24T00:48:52.701 INFO:tasks.workunit.client.0.smithi038.stdout:3/478: creat d3/d30/d5d/d6a/f9e x:0 0 0 2024-03-24T00:48:52.701 INFO:tasks.workunit.client.0.smithi038.stdout:3/479: chown d3/d9/dc/df/f1d 67473108 1 2024-03-24T00:48:52.701 INFO:tasks.workunit.client.0.smithi038.stdout:3/480: chown d3/f49 6 1 2024-03-24T00:48:52.701 INFO:tasks.workunit.client.0.smithi038.stdout:3/481: chown d3/d9/dc/df/d14 474134 1 2024-03-24T00:48:52.701 INFO:tasks.workunit.client.0.smithi038.stdout:3/482: write d3/f25 [897976,65528] 0 2024-03-24T00:48:52.701 INFO:tasks.workunit.client.0.smithi038.stdout:3/483: chown d3/d9/dc/df/d14/d19/c3f 2 1 2024-03-24T00:48:52.702 INFO:tasks.workunit.client.1.smithi088.stdout:6/502: dwrite db/dd/d61/d6d/f91 [0,4194304] 0 2024-03-24T00:48:52.702 INFO:tasks.workunit.client.1.smithi088.stdout:9/514: unlink d1/la 0 2024-03-24T00:48:52.709 INFO:tasks.workunit.client.1.smithi088.stdout:7/414: chown d3/c5d 6730940 1 2024-03-24T00:48:52.711 INFO:tasks.workunit.client.0.smithi038.stdout:0/370: sync 2024-03-24T00:48:52.711 INFO:tasks.workunit.client.0.smithi038.stdout:0/371: write d0/d12/d42/f51 [1826031,50597] 0 2024-03-24T00:48:52.711 INFO:tasks.workunit.client.0.smithi038.stdout:0/372: fsync d0/d14/f5c 0 2024-03-24T00:48:52.712 INFO:tasks.workunit.client.1.smithi088.stdout:4/478: dwrite d5/d7/f44 [0,4194304] 0 2024-03-24T00:48:52.712 INFO:tasks.workunit.client.0.smithi038.stdout:7/448: sync 2024-03-24T00:48:52.712 INFO:tasks.workunit.client.0.smithi038.stdout:7/449: truncate d2/d9/da/dc/d4a/f6d 5535202 0 2024-03-24T00:48:52.713 INFO:tasks.workunit.client.0.smithi038.stdout:5/428: sync 2024-03-24T00:48:52.715 INFO:tasks.workunit.client.1.smithi088.stdout:1/519: dwrite d6/d23/f29 [0,4194304] 0 2024-03-24T00:48:52.720 INFO:tasks.workunit.client.1.smithi088.stdout:6/503: stat db/dd/d61/l7a 0 2024-03-24T00:48:52.721 INFO:tasks.workunit.client.0.smithi038.stdout:8/440: sync 2024-03-24T00:48:52.721 INFO:tasks.workunit.client.0.smithi038.stdout:2/456: creat d4/d18/d5b/fa8 x:0 0 0 2024-03-24T00:48:52.721 INFO:tasks.workunit.client.0.smithi038.stdout:2/457: write d4/d27/d8f/d9e/fa2 [464532,2485] 0 2024-03-24T00:48:52.721 INFO:tasks.workunit.client.0.smithi038.stdout:9/504: link d12/d28/d3a/f6d d12/d81/fa5 0 2024-03-24T00:48:52.721 INFO:tasks.workunit.client.0.smithi038.stdout:6/489: rename d1/da/d1b/d25/d49/l6c to d1/l9b 0 2024-03-24T00:48:52.721 INFO:tasks.workunit.client.1.smithi088.stdout:7/415: creat d3/d32/d76/f81 x:0 0 0 2024-03-24T00:48:52.722 INFO:tasks.workunit.client.0.smithi038.stdout:5/429: write d6/d19/d20/d33/d6f/f88 [411397,85831] 0 2024-03-24T00:48:52.722 INFO:tasks.workunit.client.0.smithi038.stdout:5/430: chown d6/d16/d67 109 1 2024-03-24T00:48:52.723 INFO:tasks.workunit.client.1.smithi088.stdout:9/515: dread d1/d15/f37 [0,4194304] 0 2024-03-24T00:48:52.724 INFO:tasks.workunit.client.0.smithi038.stdout:3/484: symlink d3/d80/l9f 0 2024-03-24T00:48:52.726 INFO:tasks.workunit.client.0.smithi038.stdout:0/373: truncate d0/f5 461583 0 2024-03-24T00:48:52.733 INFO:tasks.workunit.client.1.smithi088.stdout:1/520: unlink d6/d43/d95/d3f/f54 0 2024-03-24T00:48:52.736 INFO:tasks.workunit.client.0.smithi038.stdout:4/400: dwrite da/f3b [0,4194304] 0 2024-03-24T00:48:52.736 INFO:tasks.workunit.client.0.smithi038.stdout:4/401: readlink da/d27/l74 0 2024-03-24T00:48:52.736 INFO:tasks.workunit.client.0.smithi038.stdout:4/402: fsync da/d12/d37/dd/d15/f77 0 2024-03-24T00:48:52.742 INFO:tasks.workunit.client.0.smithi038.stdout:9/505: mknod d12/d41/ca6 0 2024-03-24T00:48:52.744 INFO:tasks.workunit.client.1.smithi088.stdout:4/479: read d5/d7/f76 [2524011,85500] 0 2024-03-24T00:48:52.744 INFO:tasks.workunit.client.1.smithi088.stdout:4/480: creat d5/d13/fa5 x:0 0 0 2024-03-24T00:48:52.745 INFO:tasks.workunit.client.1.smithi088.stdout:4/481: fsync d5/f9 0 2024-03-24T00:48:52.746 INFO:tasks.workunit.client.1.smithi088.stdout:7/416: creat d3/d32/d3f/d4b/d74/f82 x:0 0 0 2024-03-24T00:48:52.746 INFO:tasks.workunit.client.1.smithi088.stdout:7/417: creat d3/d31/f83 x:0 0 0 2024-03-24T00:48:52.747 INFO:tasks.workunit.client.1.smithi088.stdout:8/508: dwrite d9/dc/d11/d24/f2a [0,4194304] 0 2024-03-24T00:48:52.751 INFO:tasks.workunit.client.0.smithi038.stdout:6/490: creat d1/da/d1b/d25/d49/d4e/d51/f9c x:0 0 0 2024-03-24T00:48:52.751 INFO:tasks.workunit.client.1.smithi088.stdout:9/516: unlink d1/d10/d77/d8a/c69 0 2024-03-24T00:48:52.751 INFO:tasks.workunit.client.1.smithi088.stdout:9/517: readlink d1/le 0 2024-03-24T00:48:52.753 INFO:tasks.workunit.client.1.smithi088.stdout:1/521: symlink d6/d27/d67/d9b/l9d 0 2024-03-24T00:48:52.757 INFO:tasks.workunit.client.0.smithi038.stdout:8/441: dread d0/d1d/d20/d3c/f47 [0,4194304] 0 2024-03-24T00:48:52.759 INFO:tasks.workunit.client.0.smithi038.stdout:3/485: unlink d3/d9/f8d 0 2024-03-24T00:48:52.763 INFO:tasks.workunit.client.1.smithi088.stdout:7/418: dread d3/d32/d3f/f6f [0,4194304] 0 2024-03-24T00:48:52.764 INFO:tasks.workunit.client.0.smithi038.stdout:2/458: dread d4/d27/d2f/d30/d3e/f6e [0,4194304] 0 2024-03-24T00:48:52.767 INFO:tasks.workunit.client.0.smithi038.stdout:4/403: link da/d38/l44 da/d12/d37/l7b 0 2024-03-24T00:48:52.768 INFO:tasks.workunit.client.0.smithi038.stdout:1/507: dwrite da/dd/d2c/d5a/d74/f75 [0,4194304] 0 2024-03-24T00:48:52.770 INFO:tasks.workunit.client.1.smithi088.stdout:4/482: chown d5/d7/d72/c64 10505108 1 2024-03-24T00:48:52.773 INFO:tasks.workunit.client.0.smithi038.stdout:9/506: link d12/l47 d12/d41/d9d/la7 0 2024-03-24T00:48:52.774 INFO:tasks.workunit.client.1.smithi088.stdout:6/504: dwrite db/d1d/d50/d90/d56/f51 [0,4194304] 0 2024-03-24T00:48:52.777 INFO:tasks.workunit.client.1.smithi088.stdout:0/487: sync 2024-03-24T00:48:52.780 INFO:tasks.workunit.client.0.smithi038.stdout:8/442: creat d0/d10/d19/f9a x:0 0 0 2024-03-24T00:48:52.784 INFO:tasks.workunit.client.0.smithi038.stdout:1/508: dread da/dd/d2c/d5a/f73 [0,4194304] 0 2024-03-24T00:48:52.785 INFO:tasks.workunit.client.0.smithi038.stdout:6/491: dread d1/da/f34 [0,4194304] 0 2024-03-24T00:48:52.785 INFO:tasks.workunit.client.0.smithi038.stdout:6/492: dread - d1/da/f3b zero size 2024-03-24T00:48:52.790 INFO:tasks.workunit.client.1.smithi088.stdout:8/509: creat d9/dc/d11/d24/d29/da9/fab x:0 0 0 2024-03-24T00:48:52.790 INFO:tasks.workunit.client.1.smithi088.stdout:8/510: fdatasync d9/dc/d11/d41/d54/d74/f90 0 2024-03-24T00:48:52.791 INFO:tasks.workunit.client.0.smithi038.stdout:3/486: mkdir d3/d30/d70/d90/da0 0 2024-03-24T00:48:52.791 INFO:tasks.workunit.client.0.smithi038.stdout:0/374: rmdir d0/d12/d15/d1a 39 2024-03-24T00:48:52.796 INFO:tasks.workunit.client.0.smithi038.stdout:2/459: symlink d4/d27/la9 0 2024-03-24T00:48:52.797 INFO:tasks.workunit.client.1.smithi088.stdout:9/518: rename d1/d10/d2c/l78 to d1/d15/la4 0 2024-03-24T00:48:52.802 INFO:tasks.workunit.client.0.smithi038.stdout:4/404: mkdir da/d12/d24/d68/d6c/d7c 0 2024-03-24T00:48:52.803 INFO:tasks.workunit.client.1.smithi088.stdout:7/419: mknod d3/d32/d3f/d4b/d74/d79/c84 0 2024-03-24T00:48:52.817 INFO:tasks.workunit.client.1.smithi088.stdout:3/448: sync 2024-03-24T00:48:52.819 INFO:tasks.workunit.client.0.smithi038.stdout:1/509: truncate f7 1254046 0 2024-03-24T00:48:52.823 INFO:tasks.workunit.client.1.smithi088.stdout:4/483: mkdir d5/d16/d94/da6 0 2024-03-24T00:48:52.823 INFO:tasks.workunit.client.1.smithi088.stdout:4/484: chown d5/d16/d37 36418221 1 2024-03-24T00:48:52.823 INFO:tasks.workunit.client.1.smithi088.stdout:4/485: stat d5/d7/d15 0 2024-03-24T00:48:52.827 INFO:tasks.workunit.client.1.smithi088.stdout:3/449: dread d3/d1a/d1e/d76/f7c [0,4194304] 0 2024-03-24T00:48:52.833 INFO:tasks.workunit.client.0.smithi038.stdout:6/493: symlink d1/da/d61/d69/l9d 0 2024-03-24T00:48:52.834 INFO:tasks.workunit.client.0.smithi038.stdout:6/494: fdatasync d1/da/d1b/d2f/f3d 0 2024-03-24T00:48:52.834 INFO:tasks.workunit.client.0.smithi038.stdout:0/375: creat d0/d1b/f7c x:0 0 0 2024-03-24T00:48:52.834 INFO:tasks.workunit.client.0.smithi038.stdout:0/376: chown d0/d14/c6c 962 1 2024-03-24T00:48:52.834 INFO:tasks.workunit.client.0.smithi038.stdout:0/377: fdatasync d0/d12/f68 0 2024-03-24T00:48:52.840 INFO:tasks.workunit.client.1.smithi088.stdout:0/488: mknod d9/dc/de/d19/d8d/c9c 0 2024-03-24T00:48:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:7/450: dwrite d2/d9/da/dc/f1c [0,4194304] 0 2024-03-24T00:48:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:7/451: fsync d2/d9/da/d23/d2b/f5f 0 2024-03-24T00:48:52.847 INFO:tasks.workunit.client.0.smithi038.stdout:2/460: mknod d4/d18/d5b/caa 0 2024-03-24T00:48:52.849 INFO:tasks.workunit.client.1.smithi088.stdout:8/511: rename d9/dc/d11/d24/l33 to d9/dc/d11/d24/d43/d4a/d95/da5/lac 0 2024-03-24T00:48:52.854 INFO:tasks.workunit.client.1.smithi088.stdout:1/522: dwrite d6/d43/d95/f39 [4194304,4194304] 0 2024-03-24T00:48:52.856 INFO:tasks.workunit.client.0.smithi038.stdout:9/507: rmdir d12/d1a/d80 39 2024-03-24T00:48:52.864 INFO:tasks.workunit.client.1.smithi088.stdout:9/519: mkdir d1/d15/d80/da5 0 2024-03-24T00:48:52.864 INFO:tasks.workunit.client.1.smithi088.stdout:9/520: chown d1/d22/f49 83 1 2024-03-24T00:48:52.864 INFO:tasks.workunit.client.1.smithi088.stdout:9/521: truncate d1/d10/d2c/f7d 220190 0 2024-03-24T00:48:52.864 INFO:tasks.workunit.client.1.smithi088.stdout:5/503: sync 2024-03-24T00:48:52.866 INFO:tasks.workunit.client.0.smithi038.stdout:0/378: truncate d0/d2d/f2f 1619642 0 2024-03-24T00:48:52.869 INFO:tasks.workunit.client.1.smithi088.stdout:3/450: creat d3/f97 x:0 0 0 2024-03-24T00:48:52.869 INFO:tasks.workunit.client.1.smithi088.stdout:3/451: read d3/d6/d27/d3c/f42 [1732281,69312] 0 2024-03-24T00:48:52.869 INFO:tasks.workunit.client.1.smithi088.stdout:3/452: fdatasync d3/d6/d27/f8d 0 2024-03-24T00:48:52.871 INFO:tasks.workunit.client.0.smithi038.stdout:7/452: rmdir d2/d9/da/d23/d2b 39 2024-03-24T00:48:52.871 INFO:tasks.workunit.client.0.smithi038.stdout:7/453: read - d2/d9/da/d23/d2b/d5b/d72/d92/d33/f3c zero size 2024-03-24T00:48:52.875 INFO:tasks.workunit.client.0.smithi038.stdout:2/461: mkdir d4/d27/d54/dab 0 2024-03-24T00:48:52.876 INFO:tasks.workunit.client.0.smithi038.stdout:6/495: write d1/da/d1b/f91 [2030459,89361] 0 2024-03-24T00:48:52.876 INFO:tasks.workunit.client.0.smithi038.stdout:6/496: fdatasync d1/da/d1b/d2f/f7f 0 2024-03-24T00:48:52.876 INFO:tasks.workunit.client.0.smithi038.stdout:5/431: dwrite d6/d19/d20/d42/d43/d4c/f62 [0,4194304] 0 2024-03-24T00:48:52.876 INFO:tasks.workunit.client.0.smithi038.stdout:5/432: fdatasync d6/d1e/f55 0 2024-03-24T00:48:52.876 INFO:tasks.workunit.client.0.smithi038.stdout:5/433: chown d6/d23 1 1 2024-03-24T00:48:52.881 INFO:tasks.workunit.client.1.smithi088.stdout:6/505: rmdir db/d1d/d23/d3c 39 2024-03-24T00:48:52.882 INFO:tasks.workunit.client.1.smithi088.stdout:6/506: truncate db/d1d/d50/f77 4405742 0 2024-03-24T00:48:52.887 INFO:tasks.workunit.client.1.smithi088.stdout:0/489: creat d9/dc/d4f/d62/f9d x:0 0 0 2024-03-24T00:48:52.897 INFO:tasks.workunit.client.1.smithi088.stdout:7/420: dwrite d3/d32/d3f/d4b/f4c [0,4194304] 0 2024-03-24T00:48:52.898 INFO:tasks.workunit.client.0.smithi038.stdout:0/379: creat d0/d12/d15/d58/d64/f7d x:0 0 0 2024-03-24T00:48:52.899 INFO:tasks.workunit.client.0.smithi038.stdout:5/434: dread d6/d19/d20/d33/d6f/f7a [0,4194304] 0 2024-03-24T00:48:52.920 INFO:tasks.workunit.client.1.smithi088.stdout:4/486: rename d5/d7/f76 to d5/d13/fa7 0 2024-03-24T00:48:52.925 INFO:tasks.workunit.client.0.smithi038.stdout:3/487: dwrite d3/d5a/f79 [0,4194304] 0 2024-03-24T00:48:52.930 INFO:tasks.workunit.client.1.smithi088.stdout:8/512: unlink d9/f17 0 2024-03-24T00:48:52.932 INFO:tasks.workunit.client.0.smithi038.stdout:0/380: dread d0/d12/f3c [0,4194304] 0 2024-03-24T00:48:52.933 INFO:tasks.workunit.client.0.smithi038.stdout:4/405: dwrite da/d12/d37/dd/d15/f17 [0,4194304] 0 2024-03-24T00:48:52.938 INFO:tasks.workunit.client.0.smithi038.stdout:2/462: dwrite d4/f24 [0,4194304] 0 2024-03-24T00:48:52.938 INFO:tasks.workunit.client.0.smithi038.stdout:2/463: truncate d4/f50 1079633 0 2024-03-24T00:48:52.944 INFO:tasks.workunit.client.1.smithi088.stdout:1/523: stat d6/d27/l2c 0 2024-03-24T00:48:52.945 INFO:tasks.workunit.client.1.smithi088.stdout:1/524: dread - d6/d27/d67/d88/f92 zero size 2024-03-24T00:48:52.948 INFO:tasks.workunit.client.0.smithi038.stdout:6/497: mknod d1/da/d1b/d2f/c9e 0 2024-03-24T00:48:52.948 INFO:tasks.workunit.client.0.smithi038.stdout:6/498: write d1/d37/d68/f8f [205006,55131] 0 2024-03-24T00:48:52.948 INFO:tasks.workunit.client.0.smithi038.stdout:3/488: write d3/d9/dc/df/f27 [3073154,6005] 0 2024-03-24T00:48:52.948 INFO:tasks.workunit.client.0.smithi038.stdout:1/510: dwrite da/dd/d7a/d82/d98/f9a [0,4194304] 0 2024-03-24T00:48:52.949 INFO:tasks.workunit.client.0.smithi038.stdout:9/508: mkdir d12/d28/da8 0 2024-03-24T00:48:52.952 INFO:tasks.workunit.client.0.smithi038.stdout:5/435: creat d6/d19/d20/d42/f91 x:0 0 0 2024-03-24T00:48:52.955 INFO:tasks.workunit.client.1.smithi088.stdout:0/490: unlink d9/dc/de/d19/d44/d46/d65/d6f/f78 0 2024-03-24T00:48:52.958 INFO:tasks.workunit.client.1.smithi088.stdout:2/478: sync 2024-03-24T00:48:52.958 INFO:tasks.workunit.client.1.smithi088.stdout:2/479: chown db/d21/d73 73 1 2024-03-24T00:48:52.958 INFO:tasks.workunit.client.1.smithi088.stdout:2/480: fsync db/d21/d35/d49/f71 0 2024-03-24T00:48:52.960 INFO:tasks.workunit.client.1.smithi088.stdout:6/507: dwrite db/dd/d61/d6d/d74/d88/f86 [0,4194304] 0 2024-03-24T00:48:52.961 INFO:tasks.workunit.client.1.smithi088.stdout:2/481: read db/d21/f33 [50326,7863] 0 2024-03-24T00:48:52.964 INFO:tasks.workunit.client.1.smithi088.stdout:1/525: write d6/f2a [3564841,35567] 0 2024-03-24T00:48:52.965 INFO:tasks.workunit.client.0.smithi038.stdout:7/454: getdents d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/d3a 0 2024-03-24T00:48:52.972 INFO:tasks.workunit.client.1.smithi088.stdout:8/513: creat d9/dc/d11/d24/d29/d5b/fad x:0 0 0 2024-03-24T00:48:52.973 INFO:tasks.workunit.client.0.smithi038.stdout:4/406: creat da/f7d x:0 0 0 2024-03-24T00:48:52.975 INFO:tasks.workunit.client.1.smithi088.stdout:3/453: mknod d3/c98 0 2024-03-24T00:48:52.977 INFO:tasks.workunit.client.1.smithi088.stdout:0/491: symlink d9/dc/de/d7e/l9e 0 2024-03-24T00:48:52.978 INFO:tasks.workunit.client.0.smithi038.stdout:2/464: mknod d4/d27/d54/d67/cac 0 2024-03-24T00:48:52.982 INFO:tasks.workunit.client.1.smithi088.stdout:6/508: rmdir db/d3d/d83 39 2024-03-24T00:48:52.983 INFO:tasks.workunit.client.0.smithi038.stdout:3/489: symlink d3/d30/d5d/la1 0 2024-03-24T00:48:52.984 INFO:tasks.workunit.client.1.smithi088.stdout:6/509: read db/d1d/f62 [167443,18726] 0 2024-03-24T00:48:52.984 INFO:tasks.workunit.client.1.smithi088.stdout:2/482: creat db/d54/d88/d8e/f9b x:0 0 0 2024-03-24T00:48:52.995 INFO:tasks.workunit.client.1.smithi088.stdout:2/483: write db/f38 [1606079,27282] 0 2024-03-24T00:48:52.998 INFO:tasks.workunit.client.0.smithi038.stdout:1/511: mkdir da/dd/d2c/d41/d60/dae 0 2024-03-24T00:48:53.001 INFO:tasks.workunit.client.0.smithi038.stdout:9/509: rmdir d12 39 2024-03-24T00:48:53.003 INFO:tasks.workunit.client.0.smithi038.stdout:5/436: mkdir d6/d19/d20/d42/d43/d50/d92 0 2024-03-24T00:48:53.004 INFO:tasks.workunit.client.0.smithi038.stdout:4/407: dread da/d12/f5c [0,4194304] 0 2024-03-24T00:48:53.004 INFO:tasks.workunit.client.0.smithi038.stdout:4/408: creat da/d12/d24/d5f/f7e x:0 0 0 2024-03-24T00:48:53.004 INFO:tasks.workunit.client.0.smithi038.stdout:4/409: write da/d12/d37/dd/d15/d4c/f53 [753224,66016] 0 2024-03-24T00:48:53.005 INFO:tasks.workunit.client.1.smithi088.stdout:5/504: rename d1/d29 to d1/dd/d6f/da1 0 2024-03-24T00:48:53.005 INFO:tasks.workunit.client.1.smithi088.stdout:5/505: write d1/fa [2400167,15586] 0 2024-03-24T00:48:53.005 INFO:tasks.workunit.client.1.smithi088.stdout:4/487: getdents d5/d17 0 2024-03-24T00:48:53.005 INFO:tasks.workunit.client.1.smithi088.stdout:4/488: fsync d5/d13/d18/f32 0 2024-03-24T00:48:53.005 INFO:tasks.workunit.client.1.smithi088.stdout:4/489: chown d5/d13/d18/f1b 1 1 2024-03-24T00:48:53.005 INFO:tasks.workunit.client.1.smithi088.stdout:8/514: symlink d9/dc/d11/d80/lae 0 2024-03-24T00:48:53.005 INFO:tasks.workunit.client.1.smithi088.stdout:3/454: dread d3/d6/f2b [0,4194304] 0 2024-03-24T00:48:53.006 INFO:tasks.workunit.client.0.smithi038.stdout:2/465: mkdir d4/d18/d26/d37/d69/d83/dad 0 2024-03-24T00:48:53.006 INFO:tasks.workunit.client.0.smithi038.stdout:2/466: chown d4/d18/d38/d63 3301981 1 2024-03-24T00:48:53.006 INFO:tasks.workunit.client.0.smithi038.stdout:2/467: creat d4/d27/d54/d53/fae x:0 0 0 2024-03-24T00:48:53.006 INFO:tasks.workunit.client.1.smithi088.stdout:9/522: rename d1/d10/d29/f6c to d1/d10/d29/fa6 0 2024-03-24T00:48:53.008 INFO:tasks.workunit.client.1.smithi088.stdout:6/510: dread db/d1d/d50/d90/f5a [0,4194304] 0 2024-03-24T00:48:53.009 INFO:tasks.workunit.client.1.smithi088.stdout:0/492: unlink d9/dc/de/f40 0 2024-03-24T00:48:53.016 INFO:tasks.workunit.client.1.smithi088.stdout:7/421: rename d3/d31/l3a to d3/d32/d76/l85 0 2024-03-24T00:48:53.016 INFO:tasks.workunit.client.1.smithi088.stdout:7/422: stat d3/d31/f83 0 2024-03-24T00:48:53.017 INFO:tasks.workunit.client.0.smithi038.stdout:3/490: dread d3/d9/dc/df/f31 [0,4194304] 0 2024-03-24T00:48:53.017 INFO:tasks.workunit.client.0.smithi038.stdout:3/491: chown d3/f49 187782502 1 2024-03-24T00:48:53.017 INFO:tasks.workunit.client.0.smithi038.stdout:3/492: readlink d3/d9/dc/df/d14/l56 0 2024-03-24T00:48:53.017 INFO:tasks.workunit.client.0.smithi038.stdout:3/493: fdatasync d3/f5 0 2024-03-24T00:48:53.018 INFO:tasks.workunit.client.1.smithi088.stdout:2/484: write db/f46 [2768972,60455] 0 2024-03-24T00:48:53.018 INFO:tasks.workunit.client.0.smithi038.stdout:1/512: truncate da/f7e 594200 0 2024-03-24T00:48:53.019 INFO:tasks.workunit.client.1.smithi088.stdout:4/490: mknod d5/ca8 0 2024-03-24T00:48:53.019 INFO:tasks.workunit.client.1.smithi088.stdout:4/491: chown d5/d16/c6c 8050517 1 2024-03-24T00:48:53.019 INFO:tasks.workunit.client.0.smithi038.stdout:6/499: dwrite d1/da/d1b/f26 [0,4194304] 0 2024-03-24T00:48:53.022 INFO:tasks.workunit.client.1.smithi088.stdout:8/515: creat d9/dc/d11/d24/d29/d73/faf x:0 0 0 2024-03-24T00:48:53.022 INFO:tasks.workunit.client.0.smithi038.stdout:9/510: rename d12/d55/f6e to d12/d81/d84/d88/fa9 0 2024-03-24T00:48:53.022 INFO:tasks.workunit.client.0.smithi038.stdout:7/455: dread d2/d9/da/d23/d2b/d5b/d72/d92/d75/f78 [4194304,4194304] 0 2024-03-24T00:48:53.023 INFO:tasks.workunit.client.1.smithi088.stdout:3/455: creat d3/de/d6f/d85/d49/f99 x:0 0 0 2024-03-24T00:48:53.023 INFO:tasks.workunit.client.1.smithi088.stdout:3/456: chown d3/d6/d27/d3c/f59 275 1 2024-03-24T00:48:53.024 INFO:tasks.workunit.client.1.smithi088.stdout:9/523: mkdir d1/d15/d47/da7 0 2024-03-24T00:48:53.025 INFO:tasks.workunit.client.1.smithi088.stdout:0/493: getdents d9/dc/de/d19/d44/d46/d59 0 2024-03-24T00:48:53.025 INFO:tasks.workunit.client.1.smithi088.stdout:0/494: fsync d9/f55 0 2024-03-24T00:48:53.026 INFO:tasks.workunit.client.0.smithi038.stdout:2/468: mknod d4/d27/d2f/d89/caf 0 2024-03-24T00:48:53.026 INFO:tasks.workunit.client.0.smithi038.stdout:2/469: fdatasync d4/d76/f99 0 2024-03-24T00:48:53.027 INFO:tasks.workunit.client.1.smithi088.stdout:7/423: mknod d3/d32/d34/d7c/c86 0 2024-03-24T00:48:53.027 INFO:tasks.workunit.client.1.smithi088.stdout:7/424: creat d3/d32/d76/f87 x:0 0 0 2024-03-24T00:48:53.027 INFO:tasks.workunit.client.1.smithi088.stdout:7/425: chown d3/d32/l55 4430 1 2024-03-24T00:48:53.028 INFO:tasks.workunit.client.0.smithi038.stdout:3/494: dread d3/d9/dc/d51/f52 [0,4194304] 0 2024-03-24T00:48:53.029 INFO:tasks.workunit.client.1.smithi088.stdout:6/511: read db/d1d/d23/d3c/d43/f28 [1280529,109328] 0 2024-03-24T00:48:53.029 INFO:tasks.workunit.client.1.smithi088.stdout:2/485: mkdir db/dd/d96/d4e/d8f/d9c 0 2024-03-24T00:48:53.029 INFO:tasks.workunit.client.1.smithi088.stdout:2/486: read db/dd/f1f [2236859,22762] 0 2024-03-24T00:48:53.033 INFO:tasks.workunit.client.1.smithi088.stdout:3/457: getdents d3/d6/d27/d3c 0 2024-03-24T00:48:53.033 INFO:tasks.workunit.client.1.smithi088.stdout:3/458: write d3/d6/d19/d2a/f88 [324116,82148] 0 2024-03-24T00:48:53.034 INFO:tasks.workunit.client.0.smithi038.stdout:9/511: rmdir d12/d1a/d94 0 2024-03-24T00:48:53.034 INFO:tasks.workunit.client.0.smithi038.stdout:9/512: dread - d12/d81/d84/f8e zero size 2024-03-24T00:48:53.036 INFO:tasks.workunit.client.0.smithi038.stdout:5/437: rename d6/d19/d20/d42/d43/d68/f75 to d6/d16/d3c/f93 0 2024-03-24T00:48:53.037 INFO:tasks.workunit.client.1.smithi088.stdout:9/524: mknod d1/d10/d11/ca8 0 2024-03-24T00:48:53.042 INFO:tasks.workunit.client.1.smithi088.stdout:0/495: creat d9/dc/de/d7e/d91/f9f x:0 0 0 2024-03-24T00:48:53.042 INFO:tasks.workunit.client.1.smithi088.stdout:0/496: readlink d9/dc/de/d19/l82 0 2024-03-24T00:48:53.043 INFO:tasks.workunit.client.0.smithi038.stdout:7/456: mkdir d2/d9/da/dc/da8 0 2024-03-24T00:48:53.044 INFO:tasks.workunit.client.0.smithi038.stdout:4/410: symlink da/d12/d37/dd/d15/d6d/l7f 0 2024-03-24T00:48:53.044 INFO:tasks.workunit.client.0.smithi038.stdout:4/411: fdatasync da/f3b 0 2024-03-24T00:48:53.048 INFO:tasks.workunit.client.0.smithi038.stdout:2/470: link d4/d18/d38/f5a d4/d18/d26/d37/d69/d83/fb0 0 2024-03-24T00:48:53.050 INFO:tasks.workunit.client.1.smithi088.stdout:2/487: write db/d21/d35/f5a [3080586,32786] 0 2024-03-24T00:48:53.050 INFO:tasks.workunit.client.1.smithi088.stdout:2/488: fsync db/d21/d35/f7d 0 2024-03-24T00:48:53.050 INFO:tasks.workunit.client.1.smithi088.stdout:2/489: fdatasync db/d21/f5f 0 2024-03-24T00:48:53.050 INFO:tasks.workunit.client.1.smithi088.stdout:2/490: fsync db/d54/f65 0 2024-03-24T00:48:53.052 INFO:tasks.workunit.client.0.smithi038.stdout:9/513: mknod d12/d28/d9c/caa 0 2024-03-24T00:48:53.052 INFO:tasks.workunit.client.0.smithi038.stdout:9/514: stat d12/d28/d3a/d26/f5a 0 2024-03-24T00:48:53.052 INFO:tasks.workunit.client.0.smithi038.stdout:9/515: read - d12/d81/d84/fa1 zero size 2024-03-24T00:48:53.052 INFO:tasks.workunit.client.1.smithi088.stdout:4/492: read d5/d16/d37/f48 [463940,104434] 0 2024-03-24T00:48:53.052 INFO:tasks.workunit.client.1.smithi088.stdout:4/493: stat d5/d13/d18/f79 0 2024-03-24T00:48:53.054 INFO:tasks.workunit.client.1.smithi088.stdout:6/512: symlink db/d1d/d50/d90/d55/l97 0 2024-03-24T00:48:53.056 INFO:tasks.workunit.client.0.smithi038.stdout:5/438: symlink d6/d19/l94 0 2024-03-24T00:48:53.057 INFO:tasks.workunit.client.0.smithi038.stdout:5/439: write d6/d1e/f63 [5237063,126592] 0 2024-03-24T00:48:53.060 INFO:tasks.workunit.client.0.smithi038.stdout:7/457: link d2/d9/da/d23/d2b/d5b/d72/d92/c10 d2/d9/da/d23/ca9 0 2024-03-24T00:48:53.067 INFO:tasks.workunit.client.0.smithi038.stdout:4/412: symlink da/d12/d24/d68/d6c/l80 0 2024-03-24T00:48:53.074 INFO:tasks.workunit.client.1.smithi088.stdout:3/459: creat d3/de/d6f/d85/f9a x:0 0 0 2024-03-24T00:48:53.074 INFO:tasks.workunit.client.1.smithi088.stdout:5/506: write d1/fb [459262,70405] 0 2024-03-24T00:48:53.084 INFO:tasks.workunit.client.0.smithi038.stdout:9/516: rename d12/d41/d57/l85 to d12/d28/d6c/lab 0 2024-03-24T00:48:53.084 INFO:tasks.workunit.client.0.smithi038.stdout:5/440: read d6/d1e/f63 [2020983,111429] 0 2024-03-24T00:48:53.084 INFO:tasks.workunit.client.0.smithi038.stdout:1/513: write da/dd/d2c/d5a/d74/f67 [1508960,12548] 0 2024-03-24T00:48:53.084 INFO:tasks.workunit.client.0.smithi038.stdout:9/517: dread - d12/d28/d3a/d26/f5a zero size 2024-03-24T00:48:53.092 INFO:tasks.workunit.client.1.smithi088.stdout:9/525: mknod d1/d22/ca9 0 2024-03-24T00:48:53.093 INFO:tasks.workunit.client.1.smithi088.stdout:9/526: dread - d1/d10/d77/d8a/f99 zero size 2024-03-24T00:48:53.093 INFO:tasks.workunit.client.1.smithi088.stdout:9/527: dread - d1/d10/d11/d23/f8b zero size 2024-03-24T00:48:53.094 INFO:tasks.workunit.client.1.smithi088.stdout:9/528: write d1/d10/d29/fa6 [713806,39478] 0 2024-03-24T00:48:53.094 INFO:tasks.workunit.client.1.smithi088.stdout:0/497: mkdir d9/dc/de/da0 0 2024-03-24T00:48:53.096 INFO:tasks.workunit.client.0.smithi038.stdout:5/441: rmdir d6/d19/d20/d42/d43/d4c/d5b 0 2024-03-24T00:48:53.096 INFO:tasks.workunit.client.1.smithi088.stdout:2/491: link db/dd/d96/l98 db/l9d 0 2024-03-24T00:48:53.100 INFO:tasks.workunit.client.0.smithi038.stdout:1/514: creat da/dd/d7a/d82/d98/d14/faf x:0 0 0 2024-03-24T00:48:53.100 INFO:tasks.workunit.client.0.smithi038.stdout:1/515: creat da/dd/d7a/d82/d98/d9e/fb0 x:0 0 0 2024-03-24T00:48:53.101 INFO:tasks.workunit.client.1.smithi088.stdout:4/494: truncate d5/d13/d18/d24/f30 3452108 0 2024-03-24T00:48:53.103 INFO:tasks.workunit.client.0.smithi038.stdout:1/516: write da/f7e [88265,98852] 0 2024-03-24T00:48:53.103 INFO:tasks.workunit.client.0.smithi038.stdout:1/517: write da/dd/d7a/d82/d98/d9e/fa7 [650288,58923] 0 2024-03-24T00:48:53.104 INFO:tasks.workunit.client.1.smithi088.stdout:6/513: link db/d1d/d50/d90/f8f db/dd/d87/f98 0 2024-03-24T00:48:53.104 INFO:tasks.workunit.client.1.smithi088.stdout:6/514: fdatasync db/d1d/d23/d3c/d4b/f6e 0 2024-03-24T00:48:53.104 INFO:tasks.workunit.client.1.smithi088.stdout:6/515: stat db/d1d/d23/d3c/l45 0 2024-03-24T00:48:53.104 INFO:tasks.workunit.client.1.smithi088.stdout:6/516: chown db/dd/d61/d6d/d74/f8b 19280 1 2024-03-24T00:48:53.107 INFO:tasks.workunit.client.0.smithi038.stdout:2/471: rmdir d4/d18/d96 0 2024-03-24T00:48:53.109 INFO:tasks.workunit.client.1.smithi088.stdout:8/516: rename d9/dc/d11/d41/d54/d74 to d9/dc/d11/d80/db0 0 2024-03-24T00:48:53.109 INFO:tasks.workunit.client.1.smithi088.stdout:8/517: chown f2 613161870 1 2024-03-24T00:48:53.112 INFO:tasks.workunit.client.1.smithi088.stdout:5/507: creat d1/d10/d1b/d2f/d57/d9e/fa2 x:0 0 0 2024-03-24T00:48:53.112 INFO:tasks.workunit.client.1.smithi088.stdout:5/508: stat d1/dd/d6f/l9a 0 2024-03-24T00:48:53.115 INFO:tasks.workunit.client.1.smithi088.stdout:2/492: creat db/d21/d35/f9e x:0 0 0 2024-03-24T00:48:53.117 INFO:tasks.workunit.client.0.smithi038.stdout:1/518: rename f8 to da/dd/fb1 0 2024-03-24T00:48:53.121 INFO:tasks.workunit.client.1.smithi088.stdout:6/517: mkdir db/d1d/d50/d90/d55/d99 0 2024-03-24T00:48:53.121 INFO:tasks.workunit.client.1.smithi088.stdout:8/518: symlink d9/dc/d11/d24/d29/da9/lb1 0 2024-03-24T00:48:53.121 INFO:tasks.workunit.client.1.smithi088.stdout:5/509: creat d1/d10/d1b/d9d/fa3 x:0 0 0 2024-03-24T00:48:53.122 INFO:tasks.workunit.client.1.smithi088.stdout:5/510: stat d1/d10/d1b/d2f/f7e 0 2024-03-24T00:48:53.123 INFO:tasks.workunit.client.0.smithi038.stdout:1/519: symlink da/dd/d2c/d41/lb2 0 2024-03-24T00:48:53.132 INFO:tasks.workunit.client.0.smithi038.stdout:1/520: write f4 [140493,32170] 0 2024-03-24T00:48:53.139 INFO:tasks.workunit.client.1.smithi088.stdout:9/529: truncate d1/d10/d77/d8a/f36 481039 0 2024-03-24T00:48:53.139 INFO:tasks.workunit.client.1.smithi088.stdout:6/518: truncate db/d1d/d50/f6f 3751355 0 2024-03-24T00:48:53.139 INFO:tasks.workunit.client.1.smithi088.stdout:6/519: truncate db/f13 8973642 0 2024-03-24T00:48:53.141 INFO:tasks.workunit.client.1.smithi088.stdout:8/519: truncate f1 1505914 0 2024-03-24T00:48:53.149 INFO:tasks.workunit.client.1.smithi088.stdout:5/511: mknod d1/dd/d6f/ca4 0 2024-03-24T00:48:53.157 INFO:tasks.workunit.client.1.smithi088.stdout:4/495: mkdir d5/d7/dd/da9 0 2024-03-24T00:48:53.162 INFO:tasks.workunit.client.0.smithi038.stdout:6/500: dwrite d1/da/d1b/d25/d49/d4e/f66 [0,4194304] 0 2024-03-24T00:48:53.165 INFO:tasks.workunit.client.1.smithi088.stdout:6/520: write db/d1d/d50/d90/f5a [1240951,16321] 0 2024-03-24T00:48:53.166 INFO:tasks.workunit.client.0.smithi038.stdout:1/521: dread da/f21 [0,4194304] 0 2024-03-24T00:48:53.170 INFO:tasks.workunit.client.0.smithi038.stdout:9/518: getdents d12/d28/d9c 0 2024-03-24T00:48:53.171 INFO:tasks.workunit.client.1.smithi088.stdout:9/530: rmdir d1/d22/d27/d39/d62 39 2024-03-24T00:48:53.171 INFO:tasks.workunit.client.1.smithi088.stdout:9/531: chown d1/d10/d77/d8a 304701 1 2024-03-24T00:48:53.171 INFO:tasks.workunit.client.1.smithi088.stdout:7/426: rename d3/l19 to d3/d32/d34/d7c/l88 0 2024-03-24T00:48:53.171 INFO:tasks.workunit.client.1.smithi088.stdout:8/520: truncate d9/dc/f52 424344 0 2024-03-24T00:48:53.171 INFO:tasks.workunit.client.1.smithi088.stdout:8/521: truncate d9/dc/f16 4338456 0 2024-03-24T00:48:53.173 INFO:tasks.workunit.client.0.smithi038.stdout:7/458: dwrite d2/d9/f54 [4194304,4194304] 0 2024-03-24T00:48:53.177 INFO:tasks.workunit.client.1.smithi088.stdout:4/496: mknod d5/d8b/caa 0 2024-03-24T00:48:53.186 INFO:tasks.workunit.client.0.smithi038.stdout:3/495: dwrite d3/f8 [4194304,4194304] 0 2024-03-24T00:48:53.187 INFO:tasks.workunit.client.0.smithi038.stdout:7/459: dread d2/d9/da/d23/d2b/d5b/d72/d92/d75/f85 [4194304,4194304] 0 2024-03-24T00:48:53.192 INFO:tasks.workunit.client.0.smithi038.stdout:4/413: dwrite da/d12/d37/dd/d15/d4c/f5a [0,4194304] 0 2024-03-24T00:48:53.192 INFO:tasks.workunit.client.0.smithi038.stdout:4/414: write da/d12/f57 [383909,96235] 0 2024-03-24T00:48:53.193 INFO:tasks.workunit.client.0.smithi038.stdout:4/415: creat da/d38/f81 x:0 0 0 2024-03-24T00:48:53.193 INFO:tasks.workunit.client.0.smithi038.stdout:1/522: creat da/d3c/d86/fb3 x:0 0 0 2024-03-24T00:48:53.194 INFO:tasks.workunit.client.0.smithi038.stdout:1/523: creat da/dd/d7a/d82/fb4 x:0 0 0 2024-03-24T00:48:53.195 INFO:tasks.workunit.client.1.smithi088.stdout:3/460: dwrite d3/d6/d27/d3c/f59 [4194304,4194304] 0 2024-03-24T00:48:53.195 INFO:tasks.workunit.client.1.smithi088.stdout:2/493: dread db/d21/d35/f5a [0,4194304] 0 2024-03-24T00:48:53.198 INFO:tasks.workunit.client.1.smithi088.stdout:6/521: creat db/d1d/d23/d3c/d43/f9a x:0 0 0 2024-03-24T00:48:53.202 INFO:tasks.workunit.client.0.smithi038.stdout:5/442: dwrite d6/d19/d20/d33/d6f/f7a [0,4194304] 0 2024-03-24T00:48:53.202 INFO:tasks.workunit.client.0.smithi038.stdout:5/443: fsync d6/f18 0 2024-03-24T00:48:53.207 INFO:tasks.workunit.client.0.smithi038.stdout:9/519: symlink d12/d28/d71/lac 0 2024-03-24T00:48:53.211 INFO:tasks.workunit.client.0.smithi038.stdout:3/496: dread d3/d9/dc/df/d14/d19/d82/f93 [0,4194304] 0 2024-03-24T00:48:53.215 INFO:tasks.workunit.client.0.smithi038.stdout:7/460: dread d2/d2c/f69 [0,4194304] 0 2024-03-24T00:48:53.219 INFO:tasks.workunit.client.0.smithi038.stdout:2/472: dwrite d4/d18/d26/f4e [0,4194304] 0 2024-03-24T00:48:53.221 INFO:tasks.workunit.client.0.smithi038.stdout:1/524: dread da/dd/d22/f4f [0,4194304] 0 2024-03-24T00:48:53.222 INFO:tasks.workunit.client.1.smithi088.stdout:7/427: mkdir d3/d32/d3f/d4b/d74/d79/d7f/d89 0 2024-03-24T00:48:53.222 INFO:tasks.workunit.client.1.smithi088.stdout:7/428: write d3/d31/f70 [428670,92431] 0 2024-03-24T00:48:53.230 INFO:tasks.workunit.client.1.smithi088.stdout:1/526: sync 2024-03-24T00:48:53.230 INFO:tasks.workunit.client.1.smithi088.stdout:1/527: write d6/d27/d67/d88/f92 [140207,106996] 0 2024-03-24T00:48:53.235 INFO:tasks.workunit.client.0.smithi038.stdout:1/525: fsync da/dd/fb1 0 2024-03-24T00:48:53.235 INFO:tasks.workunit.client.0.smithi038.stdout:1/526: chown da/dd/d2c/d41/d60/l8e 164 1 2024-03-24T00:48:53.240 INFO:tasks.workunit.client.0.smithi038.stdout:3/497: dread d3/f5 [0,4194304] 0 2024-03-24T00:48:53.241 INFO:tasks.workunit.client.1.smithi088.stdout:2/494: symlink db/dd/d96/d99/l9f 0 2024-03-24T00:48:53.248 INFO:tasks.workunit.client.1.smithi088.stdout:5/512: dwrite d1/d19/d1e/f47 [0,4194304] 0 2024-03-24T00:48:53.248 INFO:tasks.workunit.client.1.smithi088.stdout:5/513: creat d1/d10/d1b/d2f/d57/fa5 x:0 0 0 2024-03-24T00:48:53.258 INFO:tasks.workunit.client.1.smithi088.stdout:0/498: rename d9/dc/d4f to d9/d26/d8f/da1 0 2024-03-24T00:48:53.260 INFO:tasks.workunit.client.1.smithi088.stdout:9/532: dwrite d1/f7 [0,4194304] 0 2024-03-24T00:48:53.262 INFO:tasks.workunit.client.1.smithi088.stdout:7/429: chown d3/f20 0 1 2024-03-24T00:48:53.263 INFO:tasks.workunit.client.0.smithi038.stdout:1/527: dread da/dd/d22/d5c/f7f [0,4194304] 0 2024-03-24T00:48:53.270 INFO:tasks.workunit.client.0.smithi038.stdout:3/498: dread d3/d2a/f59 [0,4194304] 0 2024-03-24T00:48:53.272 INFO:tasks.workunit.client.1.smithi088.stdout:1/528: symlink d6/d6e/l9e 0 2024-03-24T00:48:53.280 INFO:tasks.workunit.client.1.smithi088.stdout:4/497: dwrite d5/d7/d15/d60/f69 [0,4194304] 0 2024-03-24T00:48:53.284 INFO:tasks.workunit.client.0.smithi038.stdout:4/416: dwrite da/d12/f5c [0,4194304] 0 2024-03-24T00:48:53.286 INFO:tasks.workunit.client.1.smithi088.stdout:2/495: dwrite db/d21/d73/f8d [0,4194304] 0 2024-03-24T00:48:53.286 INFO:tasks.workunit.client.1.smithi088.stdout:2/496: dread - db/d21/d35/f60 zero size 2024-03-24T00:48:53.296 INFO:tasks.workunit.client.1.smithi088.stdout:0/499: dread d9/dc/de/d19/d44/f6e [0,4194304] 0 2024-03-24T00:48:53.306 INFO:tasks.workunit.client.0.smithi038.stdout:5/444: dread d6/d19/d20/d42/d43/d4c/f61 [4194304,4194304] 0 2024-03-24T00:48:53.310 INFO:tasks.workunit.client.1.smithi088.stdout:6/522: rename db/dd/f66 to db/dd/d61/d6d/d74/d88/f9b 0 2024-03-24T00:48:53.310 INFO:tasks.workunit.client.1.smithi088.stdout:6/523: truncate db/f19 4472874 0 2024-03-24T00:48:53.312 INFO:tasks.workunit.client.1.smithi088.stdout:3/461: dwrite d3/d1a/d1e/d48/f50 [0,4194304] 0 2024-03-24T00:48:53.313 INFO:tasks.workunit.client.1.smithi088.stdout:3/462: chown d3/d6/d27/d3c/c74 23587855 1 2024-03-24T00:48:53.317 INFO:tasks.workunit.client.0.smithi038.stdout:5/445: write d6/d19/d20/d42/f72 [64300,126486] 0 2024-03-24T00:48:53.324 INFO:tasks.workunit.client.0.smithi038.stdout:9/520: mknod d12/d28/d6c/cad 0 2024-03-24T00:48:53.325 INFO:tasks.workunit.client.1.smithi088.stdout:9/533: dread - d1/d10/d29/d85/f91 zero size 2024-03-24T00:48:53.328 INFO:tasks.workunit.client.0.smithi038.stdout:7/461: symlink d2/d9/da/d23/d2b/d5b/d72/d92/d65/laa 0 2024-03-24T00:48:53.330 INFO:tasks.workunit.client.1.smithi088.stdout:0/500: write d9/f55 [4965815,1483] 0 2024-03-24T00:48:53.332 INFO:tasks.workunit.client.0.smithi038.stdout:2/473: link d4/d18/d5b/fa8 d4/d76/fb1 0 2024-03-24T00:48:53.340 INFO:tasks.workunit.client.1.smithi088.stdout:7/430: unlink d3/d32/f44 0 2024-03-24T00:48:53.340 INFO:tasks.workunit.client.1.smithi088.stdout:7/431: fsync d3/fd 0 2024-03-24T00:48:53.342 INFO:tasks.workunit.client.0.smithi038.stdout:1/528: symlink da/dd/d7a/d82/lb5 0 2024-03-24T00:48:53.344 INFO:tasks.workunit.client.1.smithi088.stdout:8/522: dwrite d9/dc/f52 [0,4194304] 0 2024-03-24T00:48:53.344 INFO:tasks.workunit.client.1.smithi088.stdout:8/523: dread - d9/dc/d11/d24/d29/da9/fab zero size 2024-03-24T00:48:53.346 INFO:tasks.workunit.client.1.smithi088.stdout:2/497: creat db/dd/d96/d4e/fa0 x:0 0 0 2024-03-24T00:48:53.347 INFO:tasks.workunit.client.0.smithi038.stdout:3/499: rename d3/d30/d5d/d6a/f9e to d3/d9/dc/df/d14/d19/d82/d89/fa2 0 2024-03-24T00:48:53.347 INFO:tasks.workunit.client.1.smithi088.stdout:4/498: dwrite d5/d7/f14 [4194304,4194304] 0 2024-03-24T00:48:53.348 INFO:tasks.workunit.client.0.smithi038.stdout:5/446: read d6/d19/d20/d33/d6f/f7a [910857,40549] 0 2024-03-24T00:48:53.348 INFO:tasks.workunit.client.1.smithi088.stdout:4/499: creat d5/d13/d18/d24/fab x:0 0 0 2024-03-24T00:48:53.349 INFO:tasks.workunit.client.1.smithi088.stdout:8/524: dread d9/dc/d11/d24/d29/d36/f88 [0,4194304] 0 2024-03-24T00:48:53.352 INFO:tasks.workunit.client.1.smithi088.stdout:5/514: rename d1/d10/d1b/d2f/f38 to d1/d19/fa6 0 2024-03-24T00:48:53.352 INFO:tasks.workunit.client.0.smithi038.stdout:4/417: link da/d12/f57 da/d12/d24/d68/f82 0 2024-03-24T00:48:53.352 INFO:tasks.workunit.client.1.smithi088.stdout:5/515: write d1/dd/d6f/da1/d3f/d88/d8f/d69/f8c [1457792,106234] 0 2024-03-24T00:48:53.354 INFO:tasks.workunit.client.1.smithi088.stdout:2/498: dread db/d21/f2e [0,4194304] 0 2024-03-24T00:48:53.354 INFO:tasks.workunit.client.1.smithi088.stdout:2/499: read db/f38 [1529462,61654] 0 2024-03-24T00:48:53.354 INFO:tasks.workunit.client.1.smithi088.stdout:2/500: fsync db/dd/d96/f7c 0 2024-03-24T00:48:53.356 INFO:tasks.workunit.client.1.smithi088.stdout:6/524: creat db/dd/d61/d6d/d74/d88/f9c x:0 0 0 2024-03-24T00:48:53.356 INFO:tasks.workunit.client.1.smithi088.stdout:6/525: getdents db/dd/d89 0 2024-03-24T00:48:53.363 INFO:tasks.workunit.client.1.smithi088.stdout:3/463: link d3/f95 d3/d1a/d1e/f9b 0 2024-03-24T00:48:53.363 INFO:tasks.workunit.client.1.smithi088.stdout:3/464: dread - d3/de/d6f/d85/d31/f94 zero size 2024-03-24T00:48:53.365 INFO:tasks.workunit.client.0.smithi038.stdout:9/521: truncate f4 1453900 0 2024-03-24T00:48:53.365 INFO:tasks.workunit.client.0.smithi038.stdout:9/522: readlink d12/d28/l6b 0 2024-03-24T00:48:53.365 INFO:tasks.workunit.client.0.smithi038.stdout:7/462: truncate d2/d9/da/f1d 2445155 0 2024-03-24T00:48:53.367 INFO:tasks.workunit.client.0.smithi038.stdout:1/529: mkdir da/dd/d2c/db6 0 2024-03-24T00:48:53.368 INFO:tasks.workunit.client.0.smithi038.stdout:1/530: dread - da/dd/d2c/d40/f5d zero size 2024-03-24T00:48:53.368 INFO:tasks.workunit.client.1.smithi088.stdout:9/534: dwrite d1/d10/d77/d8a/f99 [0,4194304] 0 2024-03-24T00:48:53.371 INFO:tasks.workunit.client.1.smithi088.stdout:0/501: mknod d9/dc/de/d19/d84/ca2 0 2024-03-24T00:48:53.373 INFO:tasks.workunit.client.1.smithi088.stdout:6/526: dread db/dd/f37 [4194304,4194304] 0 2024-03-24T00:48:53.373 INFO:tasks.workunit.client.1.smithi088.stdout:6/527: stat db/d1d/d50/c84 0 2024-03-24T00:48:53.375 INFO:tasks.workunit.client.0.smithi038.stdout:8/443: sync 2024-03-24T00:48:53.375 INFO:tasks.workunit.client.0.smithi038.stdout:8/444: fsync d0/d15/d2c/f88 0 2024-03-24T00:48:53.375 INFO:tasks.workunit.client.0.smithi038.stdout:8/445: fsync d0/d45/f89 0 2024-03-24T00:48:53.378 INFO:tasks.workunit.client.0.smithi038.stdout:5/447: symlink d6/d60/l95 0 2024-03-24T00:48:53.378 INFO:tasks.workunit.client.0.smithi038.stdout:3/500: creat d3/d9/dc/df/d14/d19/d82/d89/fa3 x:0 0 0 2024-03-24T00:48:53.381 INFO:tasks.workunit.client.0.smithi038.stdout:0/381: sync 2024-03-24T00:48:53.381 INFO:tasks.workunit.client.0.smithi038.stdout:0/382: truncate d0/d12/d60/d28/f39 3734375 0 2024-03-24T00:48:53.383 INFO:tasks.workunit.client.0.smithi038.stdout:9/523: creat d12/d28/da8/fae x:0 0 0 2024-03-24T00:48:53.383 INFO:tasks.workunit.client.1.smithi088.stdout:7/432: write d3/d32/d3f/d4b/f5e [2781748,118313] 0 2024-03-24T00:48:53.393 INFO:tasks.workunit.client.0.smithi038.stdout:3/501: mkdir d3/d9/dc/df/d14/d19/da4 0 2024-03-24T00:48:53.393 INFO:tasks.workunit.client.0.smithi038.stdout:3/502: creat d3/d5a/fa5 x:0 0 0 2024-03-24T00:48:53.393 INFO:tasks.workunit.client.1.smithi088.stdout:4/500: mkdir d5/dac 0 2024-03-24T00:48:53.394 INFO:tasks.workunit.client.0.smithi038.stdout:1/531: write da/dd/d2c/d5a/d74/f75 [2700243,73429] 0 2024-03-24T00:48:53.398 INFO:tasks.workunit.client.1.smithi088.stdout:8/525: creat d9/dc/d11/d24/d43/d4a/d81/fb2 x:0 0 0 2024-03-24T00:48:53.400 INFO:tasks.workunit.client.0.smithi038.stdout:9/524: unlink d12/d28/f4d 0 2024-03-24T00:48:53.407 INFO:tasks.workunit.client.1.smithi088.stdout:3/465: symlink d3/d6/d19/l9c 0 2024-03-24T00:48:53.408 INFO:tasks.workunit.client.1.smithi088.stdout:7/433: dread d3/f2c [0,4194304] 0 2024-03-24T00:48:53.414 INFO:tasks.workunit.client.0.smithi038.stdout:7/463: rename d2/d9/da/d23/d2b/d5b/d72/d92/d65 to d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab 0 2024-03-24T00:48:53.430 INFO:tasks.workunit.client.0.smithi038.stdout:7/464: symlink d2/d2c/d99/lac 0 2024-03-24T00:48:53.430 INFO:tasks.workunit.client.1.smithi088.stdout:5/516: dread d1/d19/fa6 [0,4194304] 0 2024-03-24T00:48:53.431 INFO:tasks.workunit.client.1.smithi088.stdout:1/529: dwrite d6/d27/d67/f6a [0,4194304] 0 2024-03-24T00:48:53.440 INFO:tasks.workunit.client.1.smithi088.stdout:3/466: write d3/d6/f3b [696064,31210] 0 2024-03-24T00:48:53.442 INFO:tasks.workunit.client.1.smithi088.stdout:0/502: mknod d9/dc/de/d7e/ca3 0 2024-03-24T00:48:53.444 INFO:tasks.workunit.client.1.smithi088.stdout:6/528: getdents db/d3d/d83/d72 0 2024-03-24T00:48:53.446 INFO:tasks.workunit.client.1.smithi088.stdout:4/501: rename d5/d16/d1a/d22/f9a to d5/d7/d92/fad 0 2024-03-24T00:48:53.454 INFO:tasks.workunit.client.1.smithi088.stdout:5/517: truncate d1/f31 1253560 0 2024-03-24T00:48:53.457 INFO:tasks.workunit.client.0.smithi038.stdout:2/474: dwrite d4/d18/d38/d63/f7e [0,4194304] 0 2024-03-24T00:48:53.462 INFO:tasks.workunit.client.0.smithi038.stdout:7/465: write d2/d9/da/d23/d2b/d5b/d72/d92/d75/f78 [3881861,18732] 0 2024-03-24T00:48:53.466 INFO:tasks.workunit.client.1.smithi088.stdout:3/467: rmdir d3/d1a/d1e/d2f 39 2024-03-24T00:48:53.473 INFO:tasks.workunit.client.1.smithi088.stdout:9/535: dwrite d1/d15/d21/f82 [0,4194304] 0 2024-03-24T00:48:53.473 INFO:tasks.workunit.client.1.smithi088.stdout:2/501: dwrite db/d21/f44 [0,4194304] 0 2024-03-24T00:48:53.474 INFO:tasks.workunit.client.1.smithi088.stdout:9/536: write d1/d15/f7f [4194314,98913] 0 2024-03-24T00:48:53.482 INFO:tasks.workunit.client.1.smithi088.stdout:6/529: rename db/d1d/d50/d90/d56/c3e to db/d1d/d23/d3c/c9d 0 2024-03-24T00:48:53.482 INFO:tasks.workunit.client.0.smithi038.stdout:5/448: dread d6/d19/d20/d42/d43/d4c/f74 [0,4194304] 0 2024-03-24T00:48:53.482 INFO:tasks.workunit.client.0.smithi038.stdout:5/449: write d6/d16/d67/f8f [1034051,10571] 0 2024-03-24T00:48:53.482 INFO:tasks.workunit.client.0.smithi038.stdout:5/450: fdatasync d6/d19/d20/d42/d43/d50/f81 0 2024-03-24T00:48:53.484 INFO:tasks.workunit.client.0.smithi038.stdout:2/475: rmdir d4/d27/d2f/d30/d3e 39 2024-03-24T00:48:53.487 INFO:tasks.workunit.client.0.smithi038.stdout:7/466: rename d2/d9/d16/f6f to d2/d9/da/d23/d2b/d5b/d72/d92/d98/fad 0 2024-03-24T00:48:53.487 INFO:tasks.workunit.client.1.smithi088.stdout:4/502: dread d5/d16/f57 [0,4194304] 0 2024-03-24T00:48:53.488 INFO:tasks.workunit.client.1.smithi088.stdout:4/503: creat d5/d16/d37/fae x:0 0 0 2024-03-24T00:48:53.488 INFO:tasks.workunit.client.1.smithi088.stdout:4/504: write d5/d16/d37/f6d [554395,33377] 0 2024-03-24T00:48:53.491 INFO:tasks.workunit.client.1.smithi088.stdout:4/505: fsync d5/d13/d18/d24/fab 0 2024-03-24T00:48:53.498 INFO:tasks.workunit.client.1.smithi088.stdout:5/518: rmdir d1/d43/d4e 39 2024-03-24T00:48:53.502 INFO:tasks.workunit.client.1.smithi088.stdout:3/468: stat d3/d6/d27/d3c/d80/l86 0 2024-03-24T00:48:53.502 INFO:tasks.workunit.client.1.smithi088.stdout:3/469: write d3/d6/d27/d56/d6e/f8a [813850,32353] 0 2024-03-24T00:48:53.509 INFO:tasks.workunit.client.0.smithi038.stdout:7/467: rename d2/c3e to d2/d9/d16/cae 0 2024-03-24T00:48:53.521 INFO:tasks.workunit.client.0.smithi038.stdout:8/446: dread d0/d1d/d20/d3c/f47 [0,4194304] 0 2024-03-24T00:48:53.522 INFO:tasks.workunit.client.1.smithi088.stdout:5/519: write d1/dd/f15 [635408,47702] 0 2024-03-24T00:48:53.524 INFO:tasks.workunit.client.0.smithi038.stdout:7/468: rename d2/d9/f20 to d2/d9/da/d23/d2b/d5b/faf 0 2024-03-24T00:48:53.525 INFO:tasks.workunit.client.1.smithi088.stdout:2/502: mknod db/d54/d88/ca1 0 2024-03-24T00:48:53.525 INFO:tasks.workunit.client.1.smithi088.stdout:2/503: readlink db/dd/d96/l58 0 2024-03-24T00:48:53.525 INFO:tasks.workunit.client.1.smithi088.stdout:2/504: dread - db/d54/d88/f97 zero size 2024-03-24T00:48:53.525 INFO:tasks.workunit.client.1.smithi088.stdout:2/505: chown db/d21/d35/d49/c86 43 1 2024-03-24T00:48:53.525 INFO:tasks.workunit.client.1.smithi088.stdout:2/506: stat db/dd/d96/d4e/d8f 0 2024-03-24T00:48:53.525 INFO:tasks.workunit.client.1.smithi088.stdout:2/507: fdatasync db/dd/d96/d4e/f81 0 2024-03-24T00:48:53.527 INFO:tasks.workunit.client.1.smithi088.stdout:9/537: mknod d1/d15/d47/caa 0 2024-03-24T00:48:53.527 INFO:tasks.workunit.client.1.smithi088.stdout:9/538: chown d1/d10/d2c/f75 725 1 2024-03-24T00:48:53.531 INFO:tasks.workunit.client.0.smithi038.stdout:8/447: creat d0/d45/d66/d6d/f9b x:0 0 0 2024-03-24T00:48:53.531 INFO:tasks.workunit.client.1.smithi088.stdout:6/530: mkdir db/dd/d87/d9e 0 2024-03-24T00:48:53.531 INFO:tasks.workunit.client.1.smithi088.stdout:6/531: readlink db/d1d/d23/d3c/d43/l32 0 2024-03-24T00:48:53.533 INFO:tasks.workunit.client.1.smithi088.stdout:2/508: dread db/f87 [0,4194304] 0 2024-03-24T00:48:53.536 INFO:tasks.workunit.client.0.smithi038.stdout:7/469: creat d2/d9/fb0 x:0 0 0 2024-03-24T00:48:53.536 INFO:tasks.workunit.client.0.smithi038.stdout:8/448: getdents d0/d45/d37/d5f 0 2024-03-24T00:48:53.536 INFO:tasks.workunit.client.0.smithi038.stdout:8/449: read - d0/d45/d66/f97 zero size 2024-03-24T00:48:53.538 INFO:tasks.workunit.client.0.smithi038.stdout:0/383: dwrite d0/d12/d42/f51 [0,4194304] 0 2024-03-24T00:48:53.539 INFO:tasks.workunit.client.1.smithi088.stdout:4/506: creat d5/d16/faf x:0 0 0 2024-03-24T00:48:53.539 INFO:tasks.workunit.client.1.smithi088.stdout:4/507: readlink d5/d13/d18/d24/l58 0 2024-03-24T00:48:53.540 INFO:tasks.workunit.client.1.smithi088.stdout:5/520: truncate d1/dd/d6f/da1/d3f/d88/d8f/f5e 27255 0 2024-03-24T00:48:53.541 INFO:tasks.workunit.client.1.smithi088.stdout:6/532: creat db/d3d/d83/f9f x:0 0 0 2024-03-24T00:48:53.541 INFO:tasks.workunit.client.0.smithi038.stdout:8/450: mknod d0/d45/c9c 0 2024-03-24T00:48:53.546 INFO:tasks.workunit.client.1.smithi088.stdout:5/521: rmdir d1/d19 39 2024-03-24T00:48:53.546 INFO:tasks.workunit.client.1.smithi088.stdout:2/509: creat db/dd/d96/fa2 x:0 0 0 2024-03-24T00:48:53.549 INFO:tasks.workunit.client.1.smithi088.stdout:2/510: symlink db/dd/d96/d4e/la3 0 2024-03-24T00:48:53.551 INFO:tasks.workunit.client.1.smithi088.stdout:5/522: creat d1/fa7 x:0 0 0 2024-03-24T00:48:53.558 INFO:tasks.workunit.client.0.smithi038.stdout:8/451: write d0/d10/d19/f4f [110866,53291] 0 2024-03-24T00:48:53.559 INFO:tasks.workunit.client.1.smithi088.stdout:2/511: unlink db/dd/d96/d99/l9a 0 2024-03-24T00:48:53.559 INFO:tasks.workunit.client.1.smithi088.stdout:2/512: fsync db/dc/f2b 0 2024-03-24T00:48:53.559 INFO:tasks.workunit.client.1.smithi088.stdout:2/513: stat db/dd/d96/f7c 0 2024-03-24T00:48:53.559 INFO:tasks.workunit.client.0.smithi038.stdout:8/452: write d0/d10/d19/f9a [857058,33111] 0 2024-03-24T00:48:53.559 INFO:tasks.workunit.client.0.smithi038.stdout:8/453: chown d0/d45/d66/f40 10966 1 2024-03-24T00:48:53.560 INFO:tasks.workunit.client.0.smithi038.stdout:8/454: readlink d0/d10/d19/d62/l7f 0 2024-03-24T00:48:53.561 INFO:tasks.workunit.client.0.smithi038.stdout:5/451: dwrite d6/d19/f1c [0,4194304] 0 2024-03-24T00:48:53.561 INFO:tasks.workunit.client.0.smithi038.stdout:7/470: write d2/d9/da/d23/d2b/d5b/d72/d92/f58 [1107571,98840] 0 2024-03-24T00:48:53.563 INFO:tasks.workunit.client.1.smithi088.stdout:5/523: symlink d1/d10/d1b/la8 0 2024-03-24T00:48:53.567 INFO:tasks.workunit.client.1.smithi088.stdout:7/434: write d3/f24 [732581,40113] 0 2024-03-24T00:48:53.567 INFO:tasks.workunit.client.1.smithi088.stdout:2/514: mknod db/dc/ca4 0 2024-03-24T00:48:53.567 INFO:tasks.workunit.client.0.smithi038.stdout:8/455: symlink d0/d1d/d27/l9d 0 2024-03-24T00:48:53.567 INFO:tasks.workunit.client.0.smithi038.stdout:8/456: readlink d0/d15/d2c/l57 0 2024-03-24T00:48:53.571 INFO:tasks.workunit.client.0.smithi038.stdout:5/452: mkdir d6/d19/d20/d42/d43/d68/d96 0 2024-03-24T00:48:53.580 INFO:tasks.workunit.client.0.smithi038.stdout:9/525: dwrite d12/d81/d84/f8e [0,4194304] 0 2024-03-24T00:48:53.597 INFO:tasks.workunit.client.0.smithi038.stdout:4/418: dwrite da/d12/d37/dd/f34 [0,4194304] 0 2024-03-24T00:48:53.599 INFO:tasks.workunit.client.1.smithi088.stdout:2/515: dread db/d21/d35/f5a [0,4194304] 0 2024-03-24T00:48:53.599 INFO:tasks.workunit.client.1.smithi088.stdout:2/516: write db/d21/d35/f3d [118562,31365] 0 2024-03-24T00:48:53.599 INFO:tasks.workunit.client.1.smithi088.stdout:2/517: creat db/dd/fa5 x:0 0 0 2024-03-24T00:48:53.599 INFO:tasks.workunit.client.1.smithi088.stdout:2/518: stat db/dd/d96 0 2024-03-24T00:48:53.609 INFO:tasks.workunit.client.0.smithi038.stdout:4/419: mkdir da/d12/d37/d83 0 2024-03-24T00:48:53.609 INFO:tasks.workunit.client.0.smithi038.stdout:9/526: dread d12/d1a/d80/f97 [0,4194304] 0 2024-03-24T00:48:53.609 INFO:tasks.workunit.client.0.smithi038.stdout:9/527: truncate f6 1932233 0 2024-03-24T00:48:53.613 INFO:tasks.workunit.client.0.smithi038.stdout:4/420: mknod da/d38/c84 0 2024-03-24T00:48:53.617 INFO:tasks.workunit.client.0.smithi038.stdout:9/528: symlink d12/d1a/laf 0 2024-03-24T00:48:53.617 INFO:tasks.workunit.client.0.smithi038.stdout:9/529: write d12/d41/d57/f89 [5024023,113041] 0 2024-03-24T00:48:53.617 INFO:tasks.workunit.client.0.smithi038.stdout:9/530: write d12/d55/d56/f7c [1844044,13329] 0 2024-03-24T00:48:53.618 INFO:tasks.workunit.client.0.smithi038.stdout:5/453: dread d6/d16/d21/f5f [0,4194304] 0 2024-03-24T00:48:53.618 INFO:tasks.workunit.client.0.smithi038.stdout:5/454: write d6/d1e/f63 [1862424,12283] 0 2024-03-24T00:48:53.618 INFO:tasks.workunit.client.0.smithi038.stdout:5/455: truncate d6/d16/d14/f8e 642176 0 2024-03-24T00:48:53.621 INFO:tasks.workunit.client.0.smithi038.stdout:1/532: dwrite da/dd/d22/d5c/f7f [0,4194304] 0 2024-03-24T00:48:53.621 INFO:tasks.workunit.client.0.smithi038.stdout:1/533: readlink da/dd/d2c/d5a/l9f 0 2024-03-24T00:48:53.623 INFO:tasks.workunit.client.0.smithi038.stdout:9/531: symlink d12/d1a/d5f/lb0 0 2024-03-24T00:48:53.623 INFO:tasks.workunit.client.0.smithi038.stdout:9/532: chown d12/d28/d3a/c60 5286 1 2024-03-24T00:48:53.623 INFO:tasks.workunit.client.0.smithi038.stdout:9/533: creat d12/d28/da8/fb1 x:0 0 0 2024-03-24T00:48:53.629 INFO:tasks.workunit.client.0.smithi038.stdout:1/534: truncate da/dd/fb1 1979232 0 2024-03-24T00:48:53.639 INFO:tasks.workunit.client.1.smithi088.stdout:5/524: dread d1/dd/d6f/da1/f50 [0,4194304] 0 2024-03-24T00:48:53.642 INFO:tasks.workunit.client.0.smithi038.stdout:9/534: rmdir d12/d28/d3a 39 2024-03-24T00:48:53.645 INFO:tasks.workunit.client.0.smithi038.stdout:7/471: read d2/d9/da/f1d [613360,50504] 0 2024-03-24T00:48:53.652 INFO:tasks.workunit.client.0.smithi038.stdout:7/472: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1 0 2024-03-24T00:48:53.652 INFO:tasks.workunit.client.0.smithi038.stdout:9/535: getdents d12/d28/d71 0 2024-03-24T00:48:53.654 INFO:tasks.workunit.client.1.smithi088.stdout:3/470: dwrite d3/d6/f3b [0,4194304] 0 2024-03-24T00:48:53.654 INFO:tasks.workunit.client.1.smithi088.stdout:3/471: chown d3/d1a/d1e/d2f/f36 15377014 1 2024-03-24T00:48:53.654 INFO:tasks.workunit.client.0.smithi038.stdout:9/536: mknod d12/d28/d71/cb2 0 2024-03-24T00:48:53.658 INFO:tasks.workunit.client.1.smithi088.stdout:1/530: dwrite d6/d27/d67/f6b [0,4194304] 0 2024-03-24T00:48:53.658 INFO:tasks.workunit.client.1.smithi088.stdout:1/531: fdatasync d6/d43/d95/f39 0 2024-03-24T00:48:53.660 INFO:tasks.workunit.client.0.smithi038.stdout:2/476: dwrite d4/f13 [0,4194304] 0 2024-03-24T00:48:53.671 INFO:tasks.workunit.client.0.smithi038.stdout:9/537: mknod d12/d41/cb3 0 2024-03-24T00:48:53.671 INFO:tasks.workunit.client.0.smithi038.stdout:9/538: chown d12/d28/f82 8 1 2024-03-24T00:48:53.671 INFO:tasks.workunit.client.0.smithi038.stdout:9/539: write d12/d81/d84/d88/fa9 [499500,66583] 0 2024-03-24T00:48:53.671 INFO:tasks.workunit.client.0.smithi038.stdout:9/540: chown d12/d28/d6c 80398502 1 2024-03-24T00:48:53.681 INFO:tasks.workunit.client.1.smithi088.stdout:3/472: read d3/de/d6f/f75 [2953956,108932] 0 2024-03-24T00:48:53.683 INFO:tasks.workunit.client.0.smithi038.stdout:2/477: creat d4/d55/fb2 x:0 0 0 2024-03-24T00:48:53.684 INFO:tasks.workunit.client.1.smithi088.stdout:3/473: stat d3/d6/d27/d3c/c3a 0 2024-03-24T00:48:53.687 INFO:tasks.workunit.client.1.smithi088.stdout:9/539: dwrite d1/d22/d27/d39/d98/da3/f9f [0,4194304] 0 2024-03-24T00:48:53.688 INFO:tasks.workunit.client.0.smithi038.stdout:5/456: dwrite d6/d19/d20/d42/f91 [0,4194304] 0 2024-03-24T00:48:53.689 INFO:tasks.workunit.client.0.smithi038.stdout:9/541: creat d12/d1a/d80/fb4 x:0 0 0 2024-03-24T00:48:53.698 INFO:tasks.workunit.client.1.smithi088.stdout:8/526: dwrite d9/d6b/f6c [0,4194304] 0 2024-03-24T00:48:53.698 INFO:tasks.workunit.client.1.smithi088.stdout:8/527: chown d9/f2b 348658 1 2024-03-24T00:48:53.698 INFO:tasks.workunit.client.1.smithi088.stdout:0/503: dwrite d9/dc/de/d7e/d91/f9f [0,4194304] 0 2024-03-24T00:48:53.709 INFO:tasks.workunit.client.0.smithi038.stdout:0/384: dwrite d0/d12/d60/d28/f39 [0,4194304] 0 2024-03-24T00:48:53.709 INFO:tasks.workunit.client.0.smithi038.stdout:0/385: chown d0/d12/d42/l76 26 1 2024-03-24T00:48:53.709 INFO:tasks.workunit.client.0.smithi038.stdout:0/386: stat d0/f69 0 2024-03-24T00:48:53.710 INFO:tasks.workunit.client.0.smithi038.stdout:8/457: dwrite d0/d15/f54 [0,4194304] 0 2024-03-24T00:48:53.710 INFO:tasks.workunit.client.0.smithi038.stdout:8/458: stat d0/d45/d37/d5f/d68/f85 0 2024-03-24T00:48:53.713 INFO:tasks.workunit.client.1.smithi088.stdout:4/508: dwrite d5/d16/d1a/f1e [0,4194304] 0 2024-03-24T00:48:53.714 INFO:tasks.workunit.client.0.smithi038.stdout:5/457: write d6/d19/f1c [2269443,75522] 0 2024-03-24T00:48:53.724 INFO:tasks.workunit.client.1.smithi088.stdout:1/532: dwrite d6/d43/d95/d58/f78 [0,4194304] 0 2024-03-24T00:48:53.724 INFO:tasks.workunit.client.1.smithi088.stdout:1/533: fsync d6/f6d 0 2024-03-24T00:48:53.724 INFO:tasks.workunit.client.1.smithi088.stdout:1/534: fdatasync d6/d43/f4f 0 2024-03-24T00:48:53.724 INFO:tasks.workunit.client.1.smithi088.stdout:1/535: fdatasync d6/d43/d95/d58/f78 0 2024-03-24T00:48:53.744 INFO:tasks.workunit.client.1.smithi088.stdout:3/474: rmdir d3/de 39 2024-03-24T00:48:53.744 INFO:tasks.workunit.client.0.smithi038.stdout:1/535: dwrite da/f21 [0,4194304] 0 2024-03-24T00:48:53.744 INFO:tasks.workunit.client.0.smithi038.stdout:1/536: stat da/f7e 0 2024-03-24T00:48:53.745 INFO:tasks.workunit.client.0.smithi038.stdout:1/537: chown da/dd/d2c/d5a/d74/f67 2 1 2024-03-24T00:48:53.745 INFO:tasks.workunit.client.0.smithi038.stdout:1/538: fsync da/dd/d22/d5c/f7f 0 2024-03-24T00:48:53.754 INFO:tasks.workunit.client.0.smithi038.stdout:4/421: dwrite da/d12/d24/f26 [0,4194304] 0 2024-03-24T00:48:53.768 INFO:tasks.workunit.client.0.smithi038.stdout:7/473: dwrite d2/d9/da/f47 [0,4194304] 0 2024-03-24T00:48:53.770 INFO:tasks.workunit.client.1.smithi088.stdout:5/525: dwrite d1/dd/f54 [4194304,4194304] 0 2024-03-24T00:48:53.782 INFO:tasks.workunit.client.0.smithi038.stdout:2/478: dwrite d4/d27/f33 [4194304,4194304] 0 2024-03-24T00:48:53.786 INFO:tasks.workunit.client.0.smithi038.stdout:9/542: dwrite d12/d1a/f3c [0,4194304] 0 2024-03-24T00:48:53.788 INFO:tasks.workunit.client.1.smithi088.stdout:5/526: dread d1/d19/d1e/f47 [0,4194304] 0 2024-03-24T00:48:53.789 INFO:tasks.workunit.client.0.smithi038.stdout:0/387: symlink d0/d12/d42/d44/l7e 0 2024-03-24T00:48:53.789 INFO:tasks.workunit.client.0.smithi038.stdout:8/459: mkdir d0/d1d/d20/d3c/d9e 0 2024-03-24T00:48:53.789 INFO:tasks.workunit.client.0.smithi038.stdout:8/460: creat d0/d1d/d27/d4d/f9f x:0 0 0 2024-03-24T00:48:53.789 INFO:tasks.workunit.client.0.smithi038.stdout:8/461: creat d0/d45/d37/d5f/d68/fa0 x:0 0 0 2024-03-24T00:48:53.798 INFO:tasks.workunit.client.0.smithi038.stdout:5/458: rmdir d6/d19/d20/d64/d6b/d90 0 2024-03-24T00:48:53.798 INFO:tasks.workunit.client.0.smithi038.stdout:5/459: write d6/d19/d20/d42/d43/d4c/f74 [2410666,4119] 0 2024-03-24T00:48:53.798 INFO:tasks.workunit.client.1.smithi088.stdout:1/536: dwrite d6/d43/d95/d5b/f75 [0,4194304] 0 2024-03-24T00:48:53.807 INFO:tasks.workunit.client.1.smithi088.stdout:0/504: truncate d9/dc/de/d19/d44/f4a 5656816 0 2024-03-24T00:48:53.818 INFO:tasks.workunit.client.0.smithi038.stdout:1/539: symlink da/dd/d2c/d40/lb7 0 2024-03-24T00:48:53.821 INFO:tasks.workunit.client.1.smithi088.stdout:9/540: dwrite d1/d10/d29/d6d/f57 [0,4194304] 0 2024-03-24T00:48:53.822 INFO:tasks.workunit.client.1.smithi088.stdout:8/528: dread d9/dc/d11/d24/d43/d4a/d81/f91 [0,4194304] 0 2024-03-24T00:48:53.828 INFO:tasks.workunit.client.0.smithi038.stdout:2/479: dread d4/d27/f2a [0,4194304] 0 2024-03-24T00:48:53.829 INFO:tasks.workunit.client.0.smithi038.stdout:2/480: read d4/d18/d26/d37/d69/d83/fb0 [95509,115815] 0 2024-03-24T00:48:53.829 INFO:tasks.workunit.client.0.smithi038.stdout:2/481: chown d4/d18/d26/d37/d69/d83/l84 37 1 2024-03-24T00:48:53.832 INFO:tasks.workunit.client.0.smithi038.stdout:4/422: rename da/d12/f57 to da/d12/d24/d68/d6c/f85 0 2024-03-24T00:48:53.833 INFO:tasks.workunit.client.0.smithi038.stdout:4/423: chown da/d12/d29/d5b/c6b 153764065 1 2024-03-24T00:48:53.837 INFO:tasks.workunit.client.0.smithi038.stdout:9/543: dwrite fe [0,4194304] 0 2024-03-24T00:48:53.837 INFO:tasks.workunit.client.0.smithi038.stdout:9/544: fsync d12/d1a/f49 0 2024-03-24T00:48:53.837 INFO:tasks.workunit.client.0.smithi038.stdout:9/545: chown d12/d1a/d5f/c65 59436 1 2024-03-24T00:48:53.844 INFO:tasks.workunit.client.0.smithi038.stdout:7/474: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/db2 0 2024-03-24T00:48:53.847 INFO:tasks.workunit.client.1.smithi088.stdout:4/509: getdents d5/d16/d94 0 2024-03-24T00:48:53.853 INFO:tasks.workunit.client.0.smithi038.stdout:1/540: dread da/dd/d22/d5c/f6e [0,4194304] 0 2024-03-24T00:48:53.853 INFO:tasks.workunit.client.0.smithi038.stdout:1/541: fdatasync da/f7e 0 2024-03-24T00:48:53.854 INFO:tasks.workunit.client.0.smithi038.stdout:0/388: mknod d0/d12/d42/d5e/c7f 0 2024-03-24T00:48:53.859 INFO:tasks.workunit.client.1.smithi088.stdout:3/475: creat d3/d67/f9d x:0 0 0 2024-03-24T00:48:53.860 INFO:tasks.workunit.client.1.smithi088.stdout:3/476: stat d3/d6/d27/d56/d6e 0 2024-03-24T00:48:53.860 INFO:tasks.workunit.client.1.smithi088.stdout:8/529: dwrite d9/dc/d11/d41/d54/f9a [0,4194304] 0 2024-03-24T00:48:53.860 INFO:tasks.workunit.client.1.smithi088.stdout:8/530: chown d9/dc/d11/d24/d43/d4a/d81/ca3 2049485381 1 2024-03-24T00:48:53.864 INFO:tasks.workunit.client.0.smithi038.stdout:5/460: truncate d6/d23/f3b 593768 0 2024-03-24T00:48:53.867 INFO:tasks.workunit.client.1.smithi088.stdout:5/527: unlink d1/d10/l1a 0 2024-03-24T00:48:53.869 INFO:tasks.workunit.client.0.smithi038.stdout:4/424: symlink da/d12/d24/d68/d6c/l86 0 2024-03-24T00:48:53.869 INFO:tasks.workunit.client.0.smithi038.stdout:4/425: write da/d12/d37/dd/f2b [436638,76896] 0 2024-03-24T00:48:53.874 INFO:tasks.workunit.client.0.smithi038.stdout:9/546: link d12/d1a/f20 d12/d55/fb5 0 2024-03-24T00:48:53.881 INFO:tasks.workunit.client.0.smithi038.stdout:6/501: sync 2024-03-24T00:48:53.881 INFO:tasks.workunit.client.0.smithi038.stdout:6/502: readlink d1/l9b 0 2024-03-24T00:48:53.882 INFO:tasks.workunit.client.0.smithi038.stdout:8/462: dwrite d0/d45/d66/d6d/f9b [0,4194304] 0 2024-03-24T00:48:53.882 INFO:tasks.workunit.client.0.smithi038.stdout:7/475: unlink d2/d9/da/d23/d2b/d5b/faf 0 2024-03-24T00:48:53.883 INFO:tasks.workunit.client.1.smithi088.stdout:8/531: dread d9/dc/f52 [0,4194304] 0 2024-03-24T00:48:53.885 INFO:tasks.workunit.client.0.smithi038.stdout:1/542: creat da/dd/d2c/db6/fb8 x:0 0 0 2024-03-24T00:48:53.885 INFO:tasks.workunit.client.0.smithi038.stdout:1/543: chown da/dd/f1c 0 1 2024-03-24T00:48:53.885 INFO:tasks.workunit.client.0.smithi038.stdout:1/544: truncate da/dd/fb1 2568881 0 2024-03-24T00:48:53.891 INFO:tasks.workunit.client.0.smithi038.stdout:0/389: stat d0/d14/c55 0 2024-03-24T00:48:53.891 INFO:tasks.workunit.client.0.smithi038.stdout:0/390: readlink d0/d12/d42/d44/l7e 0 2024-03-24T00:48:53.891 INFO:tasks.workunit.client.0.smithi038.stdout:5/461: mkdir d6/d1e/d97 0 2024-03-24T00:48:53.896 INFO:tasks.workunit.client.1.smithi088.stdout:1/537: creat d6/d43/f9f x:0 0 0 2024-03-24T00:48:53.898 INFO:tasks.workunit.client.0.smithi038.stdout:4/426: creat da/d12/f87 x:0 0 0 2024-03-24T00:48:53.899 INFO:tasks.workunit.client.0.smithi038.stdout:6/503: dread d1/da/d1b/d25/d49/d4e/f66 [0,4194304] 0 2024-03-24T00:48:53.901 INFO:tasks.workunit.client.0.smithi038.stdout:9/547: symlink d12/d1a/d5f/lb6 0 2024-03-24T00:48:53.903 INFO:tasks.workunit.client.0.smithi038.stdout:4/427: fdatasync da/d12/d37/dd/f2b 0 2024-03-24T00:48:53.904 INFO:tasks.workunit.client.1.smithi088.stdout:0/505: unlink d9/dc/de/d19/d44/d46/d65/f39 0 2024-03-24T00:48:53.904 INFO:tasks.workunit.client.1.smithi088.stdout:4/510: dwrite d5/d7/d92/fad [0,4194304] 0 2024-03-24T00:48:53.911 INFO:tasks.workunit.client.0.smithi038.stdout:1/545: write da/dd/d2c/d40/f53 [150660,60579] 0 2024-03-24T00:48:53.911 INFO:tasks.workunit.client.0.smithi038.stdout:0/391: write d0/fa [3168414,64382] 0 2024-03-24T00:48:53.911 INFO:tasks.workunit.client.0.smithi038.stdout:0/392: fdatasync d0/d2d/f65 0 2024-03-24T00:48:53.913 INFO:tasks.workunit.client.1.smithi088.stdout:9/541: rename d1/d22/d27/d39/d62 to d1/d10/d9b/dab 0 2024-03-24T00:48:53.915 INFO:tasks.workunit.client.1.smithi088.stdout:0/506: read d9/dc/de/d7e/d91/f9f [2998010,73329] 0 2024-03-24T00:48:53.919 INFO:tasks.workunit.client.0.smithi038.stdout:1/546: dread da/dd/d22/f55 [0,4194304] 0 2024-03-24T00:48:53.919 INFO:tasks.workunit.client.0.smithi038.stdout:1/547: dread - da/dd/d7a/d82/fb4 zero size 2024-03-24T00:48:53.919 INFO:tasks.workunit.client.0.smithi038.stdout:1/548: truncate da/dd/d2c/d5a/d74/f64 301520 0 2024-03-24T00:48:53.923 INFO:tasks.workunit.client.0.smithi038.stdout:8/463: creat d0/d1d/d27/d6b/fa1 x:0 0 0 2024-03-24T00:48:53.926 INFO:tasks.workunit.client.1.smithi088.stdout:5/528: unlink d1/d43/d4e/d5b/l7f 0 2024-03-24T00:48:53.928 INFO:tasks.workunit.client.0.smithi038.stdout:3/503: sync 2024-03-24T00:48:53.928 INFO:tasks.workunit.client.0.smithi038.stdout:0/393: dread d0/d1b/d2c/f6b [0,4194304] 0 2024-03-24T00:48:53.930 INFO:tasks.workunit.client.0.smithi038.stdout:7/476: link d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/c3b d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/cb3 0 2024-03-24T00:48:53.930 INFO:tasks.workunit.client.0.smithi038.stdout:7/477: readlink d2/d9/l88 0 2024-03-24T00:48:53.932 INFO:tasks.workunit.client.0.smithi038.stdout:5/462: mknod d6/d19/c98 0 2024-03-24T00:48:53.939 INFO:tasks.workunit.client.0.smithi038.stdout:6/504: rename d1/da/d1b/d25/d49/f7b to d1/da/d73/d75/f9f 0 2024-03-24T00:48:53.939 INFO:tasks.workunit.client.0.smithi038.stdout:6/505: chown d1/da/d1b/d2f 13501 1 2024-03-24T00:48:53.942 INFO:tasks.workunit.client.1.smithi088.stdout:4/511: truncate d5/d17/f1d 3198634 0 2024-03-24T00:48:53.944 INFO:tasks.workunit.client.1.smithi088.stdout:5/529: write d1/d10/f52 [4146785,85672] 0 2024-03-24T00:48:53.945 INFO:tasks.workunit.client.1.smithi088.stdout:5/530: write d1/dd/d6f/da1/d3f/d88/d8f/d69/f85 [833216,77632] 0 2024-03-24T00:48:53.946 INFO:tasks.workunit.client.0.smithi038.stdout:4/428: creat da/d12/d37/dd/d15/d6d/f88 x:0 0 0 2024-03-24T00:48:53.950 INFO:tasks.workunit.client.0.smithi038.stdout:3/504: write d3/d30/d5d/f5e [3232602,106047] 0 2024-03-24T00:48:53.951 INFO:tasks.workunit.client.1.smithi088.stdout:3/477: rename d3/de/d6f/d85/l46 to d3/d6/d27/l9e 0 2024-03-24T00:48:53.955 INFO:tasks.workunit.client.0.smithi038.stdout:8/464: fsync d0/d1d/d27/d4d/f9f 0 2024-03-24T00:48:53.955 INFO:tasks.workunit.client.0.smithi038.stdout:8/465: chown d0/d45/d37/d5f/d68 488 1 2024-03-24T00:48:53.955 INFO:tasks.workunit.client.0.smithi038.stdout:8/466: write d0/d45/d37/d5f/d68/fa0 [792034,44909] 0 2024-03-24T00:48:53.955 INFO:tasks.workunit.client.0.smithi038.stdout:8/467: chown d0/d45/d37/c80 1778 1 2024-03-24T00:48:53.955 INFO:tasks.workunit.client.0.smithi038.stdout:8/468: chown d0/d1d/d27/d4d/d81 5075 1 2024-03-24T00:48:53.960 INFO:tasks.workunit.client.1.smithi088.stdout:4/512: symlink d5/d16/d94/lb0 0 2024-03-24T00:48:53.960 INFO:tasks.workunit.client.1.smithi088.stdout:4/513: creat d5/d16/fb1 x:0 0 0 2024-03-24T00:48:53.964 INFO:tasks.workunit.client.0.smithi038.stdout:1/549: mknod da/dd/d7a/d82/d98/d14/cb9 0 2024-03-24T00:48:53.969 INFO:tasks.workunit.client.1.smithi088.stdout:9/542: dwrite d1/f7 [0,4194304] 0 2024-03-24T00:48:53.969 INFO:tasks.workunit.client.1.smithi088.stdout:9/543: chown d1/d10/d11/c13 1 1 2024-03-24T00:48:53.978 INFO:tasks.workunit.client.1.smithi088.stdout:3/478: symlink d3/d6/d27/d56/l9f 0 2024-03-24T00:48:53.978 INFO:tasks.workunit.client.1.smithi088.stdout:3/479: truncate d3/d1a/d1e/d76/f93 11510 0 2024-03-24T00:48:53.978 INFO:tasks.workunit.client.1.smithi088.stdout:3/480: fsync f1 0 2024-03-24T00:48:53.985 INFO:tasks.workunit.client.0.smithi038.stdout:7/478: unlink d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/c3b 0 2024-03-24T00:48:53.985 INFO:tasks.workunit.client.0.smithi038.stdout:7/479: chown d2/f4 3 1 2024-03-24T00:48:53.986 INFO:tasks.workunit.client.1.smithi088.stdout:4/514: write d5/d16/d1a/f3e [2306916,17459] 0 2024-03-24T00:48:53.987 INFO:tasks.workunit.client.1.smithi088.stdout:4/515: dread - d5/d13/d18/d24/f9c zero size 2024-03-24T00:48:53.987 INFO:tasks.workunit.client.1.smithi088.stdout:4/516: readlink d5/d7/d15/d74/l98 0 2024-03-24T00:48:53.987 INFO:tasks.workunit.client.1.smithi088.stdout:2/519: sync 2024-03-24T00:48:53.987 INFO:tasks.workunit.client.1.smithi088.stdout:7/435: sync 2024-03-24T00:48:53.987 INFO:tasks.workunit.client.1.smithi088.stdout:6/533: sync 2024-03-24T00:48:53.988 INFO:tasks.workunit.client.0.smithi038.stdout:5/463: symlink d6/d19/d20/d42/d43/l99 0 2024-03-24T00:48:53.998 INFO:tasks.workunit.client.1.smithi088.stdout:3/481: creat d3/de/d6f/d89/fa0 x:0 0 0 2024-03-24T00:48:54.001 INFO:tasks.workunit.client.0.smithi038.stdout:6/506: link d1/da/l54 d1/d37/d68/la0 0 2024-03-24T00:48:54.006 INFO:tasks.workunit.client.0.smithi038.stdout:0/394: dwrite d0/d12/d15/d58/d64/f7d [0,4194304] 0 2024-03-24T00:48:54.007 INFO:tasks.workunit.client.1.smithi088.stdout:5/531: dread d1/dd/d6f/da1/d3f/d88/d8f/d69/f82 [0,4194304] 0 2024-03-24T00:48:54.007 INFO:tasks.workunit.client.1.smithi088.stdout:5/532: chown d1/dd/d6f/da1/d3f/d88/d8f/f5e 111 1 2024-03-24T00:48:54.007 INFO:tasks.workunit.client.1.smithi088.stdout:0/507: dwrite d9/dc/de/d19/d44/d46/d65/f35 [0,4194304] 0 2024-03-24T00:48:54.015 INFO:tasks.workunit.client.0.smithi038.stdout:4/429: mkdir da/d27/d89 0 2024-03-24T00:48:54.028 INFO:tasks.workunit.client.1.smithi088.stdout:6/534: unlink db/d1d/d50/d90/f46 0 2024-03-24T00:48:54.028 INFO:tasks.workunit.client.1.smithi088.stdout:6/535: chown db/d1d/d23/d3c/d43 1257424 1 2024-03-24T00:48:54.028 INFO:tasks.workunit.client.1.smithi088.stdout:6/536: fdatasync db/d1d/f62 0 2024-03-24T00:48:54.031 INFO:tasks.workunit.client.1.smithi088.stdout:9/544: dwrite d1/d10/d11/d23/f70 [0,4194304] 0 2024-03-24T00:48:54.031 INFO:tasks.workunit.client.0.smithi038.stdout:3/505: rename d3/d80 to d3/d9/dc/da6 0 2024-03-24T00:48:54.031 INFO:tasks.workunit.client.0.smithi038.stdout:3/506: fsync d3/d30/f7e 0 2024-03-24T00:48:54.037 INFO:tasks.workunit.client.0.smithi038.stdout:1/550: creat da/dd/d7a/d82/d98/fba x:0 0 0 2024-03-24T00:48:54.038 INFO:tasks.workunit.client.1.smithi088.stdout:5/533: write d1/d19/d1e/f76 [171884,62504] 0 2024-03-24T00:48:54.039 INFO:tasks.workunit.client.1.smithi088.stdout:2/520: unlink db/d21/d35/d49/c86 0 2024-03-24T00:48:54.040 INFO:tasks.workunit.client.1.smithi088.stdout:8/532: rename d9/dc/d11/d24/d43/d4a/d81 to d9/dc/d11/d41/d54/d5d/db3 0 2024-03-24T00:48:54.043 INFO:tasks.workunit.client.0.smithi038.stdout:7/480: truncate d2/f1f 6755443 0 2024-03-24T00:48:54.043 INFO:tasks.workunit.client.0.smithi038.stdout:7/481: truncate d2/d9/f7c 978260 0 2024-03-24T00:48:54.048 INFO:tasks.workunit.client.1.smithi088.stdout:3/482: symlink d3/d67/la1 0 2024-03-24T00:48:54.048 INFO:tasks.workunit.client.1.smithi088.stdout:0/508: creat d9/dc/de/d19/d44/fa4 x:0 0 0 2024-03-24T00:48:54.048 INFO:tasks.workunit.client.1.smithi088.stdout:0/509: chown d9/dc/de/d19/d44/d46/d65/f58 75504 1 2024-03-24T00:48:54.048 INFO:tasks.workunit.client.1.smithi088.stdout:3/483: write d3/d6/d19/f52 [331738,117932] 0 2024-03-24T00:48:54.048 INFO:tasks.workunit.client.1.smithi088.stdout:3/484: dread - d3/de/d6f/d85/f9a zero size 2024-03-24T00:48:54.048 INFO:tasks.workunit.client.1.smithi088.stdout:0/510: truncate d9/dc/de/f6c 772966 0 2024-03-24T00:48:54.048 INFO:tasks.workunit.client.1.smithi088.stdout:6/537: dread db/dd/d61/d6d/d74/d88/f86 [0,4194304] 0 2024-03-24T00:48:54.048 INFO:tasks.workunit.client.1.smithi088.stdout:6/538: readlink db/d1d/d50/d90/d55/l82 0 2024-03-24T00:48:54.059 INFO:tasks.workunit.client.0.smithi038.stdout:5/464: dwrite d6/f9 [0,4194304] 0 2024-03-24T00:48:54.059 INFO:tasks.workunit.client.1.smithi088.stdout:8/533: write d9/f63 [695524,19031] 0 2024-03-24T00:48:54.060 INFO:tasks.workunit.client.1.smithi088.stdout:9/545: link d1/d15/f68 d1/d10/d29/d85/fac 0 2024-03-24T00:48:54.060 INFO:tasks.workunit.client.1.smithi088.stdout:9/546: read d1/d10/f3e [4149722,49629] 0 2024-03-24T00:48:54.060 INFO:tasks.workunit.client.1.smithi088.stdout:9/547: truncate d1/d10/d11/d23/f54 956895 0 2024-03-24T00:48:54.060 INFO:tasks.workunit.client.1.smithi088.stdout:9/548: write d1/d15/d21/f66 [523817,87158] 0 2024-03-24T00:48:54.065 INFO:tasks.workunit.client.1.smithi088.stdout:5/534: creat d1/d10/d1b/d2f/d93/fa9 x:0 0 0 2024-03-24T00:48:54.066 INFO:tasks.workunit.client.1.smithi088.stdout:5/535: write d1/d19/f6c [5126727,125403] 0 2024-03-24T00:48:54.066 INFO:tasks.workunit.client.1.smithi088.stdout:5/536: chown d1/d10/d81/f97 3070 1 2024-03-24T00:48:54.066 INFO:tasks.workunit.client.1.smithi088.stdout:5/537: fsync d1/d10/d81/f97 0 2024-03-24T00:48:54.069 INFO:tasks.workunit.client.1.smithi088.stdout:7/436: dwrite d3/f28 [0,4194304] 0 2024-03-24T00:48:54.069 INFO:tasks.workunit.client.1.smithi088.stdout:2/521: link db/dd/l1e db/d54/la6 0 2024-03-24T00:48:54.070 INFO:tasks.workunit.client.0.smithi038.stdout:6/507: dwrite d1/d37/f90 [0,4194304] 0 2024-03-24T00:48:54.071 INFO:tasks.workunit.client.0.smithi038.stdout:3/507: symlink d3/d2a/la7 0 2024-03-24T00:48:54.071 INFO:tasks.workunit.client.0.smithi038.stdout:3/508: write d3/d5a/f73 [556321,45288] 0 2024-03-24T00:48:54.072 INFO:tasks.workunit.client.1.smithi088.stdout:0/511: rmdir d9/dc/de/da0 0 2024-03-24T00:48:54.074 INFO:tasks.workunit.client.0.smithi038.stdout:0/395: rename d0/d12/d15/c63 to d0/d12/d42/d44/c80 0 2024-03-24T00:48:54.074 INFO:tasks.workunit.client.0.smithi038.stdout:0/396: fdatasync d0/fa 0 2024-03-24T00:48:54.074 INFO:tasks.workunit.client.0.smithi038.stdout:0/397: truncate d0/d12/f68 738005 0 2024-03-24T00:48:54.074 INFO:tasks.workunit.client.0.smithi038.stdout:0/398: chown d0/d2d/f6f 2745 1 2024-03-24T00:48:54.075 INFO:tasks.workunit.client.1.smithi088.stdout:6/539: symlink db/dd/la0 0 2024-03-24T00:48:54.075 INFO:tasks.workunit.client.0.smithi038.stdout:1/551: rmdir da/dd 39 2024-03-24T00:48:54.076 INFO:tasks.workunit.client.0.smithi038.stdout:1/552: write f4 [1148026,60079] 0 2024-03-24T00:48:54.078 INFO:tasks.workunit.client.0.smithi038.stdout:5/465: truncate d6/d19/d20/d42/d43/d4c/f74 1955794 0 2024-03-24T00:48:54.079 INFO:tasks.workunit.client.0.smithi038.stdout:4/430: rmdir da/d12/d37/dd/d15/d4c 39 2024-03-24T00:48:54.082 INFO:tasks.workunit.client.1.smithi088.stdout:9/549: creat d1/d10/fad x:0 0 0 2024-03-24T00:48:54.082 INFO:tasks.workunit.client.1.smithi088.stdout:9/550: fsync d1/d10/f1e 0 2024-03-24T00:48:54.088 INFO:tasks.workunit.client.0.smithi038.stdout:3/509: mkdir d3/d9/dc/df/d14/d19/d82/d89/da8 0 2024-03-24T00:48:54.088 INFO:tasks.workunit.client.0.smithi038.stdout:3/510: truncate d3/d30/f33 1010210 0 2024-03-24T00:48:54.090 INFO:tasks.workunit.client.0.smithi038.stdout:3/511: write d3/f49 [4779319,2985] 0 2024-03-24T00:48:54.090 INFO:tasks.workunit.client.1.smithi088.stdout:5/538: mkdir d1/d10/d1b/d2f/d57/d9e/daa 0 2024-03-24T00:48:54.090 INFO:tasks.workunit.client.1.smithi088.stdout:5/539: stat d1/d19/f30 0 2024-03-24T00:48:54.090 INFO:tasks.workunit.client.1.smithi088.stdout:5/540: write d1/dd/d6f/da1/f50 [4156661,23490] 0 2024-03-24T00:48:54.090 INFO:tasks.workunit.client.1.smithi088.stdout:5/541: write d1/d19/d1e/f76 [898245,85472] 0 2024-03-24T00:48:54.091 INFO:tasks.workunit.client.1.smithi088.stdout:5/542: readlink d1/d19/d1e/l53 0 2024-03-24T00:48:54.092 INFO:tasks.workunit.client.1.smithi088.stdout:2/522: dread db/d21/d35/f3d [0,4194304] 0 2024-03-24T00:48:54.093 INFO:tasks.workunit.client.1.smithi088.stdout:8/534: dread d9/d6b/f6c [0,4194304] 0 2024-03-24T00:48:54.093 INFO:tasks.workunit.client.1.smithi088.stdout:8/535: chown d9/dc/d11/l1d 3475 1 2024-03-24T00:48:54.096 INFO:tasks.workunit.client.1.smithi088.stdout:7/437: mknod d3/d32/d34/d7c/c8a 0 2024-03-24T00:48:54.096 INFO:tasks.workunit.client.1.smithi088.stdout:7/438: dread - d3/d32/d3f/d4b/d74/f82 zero size 2024-03-24T00:48:54.096 INFO:tasks.workunit.client.1.smithi088.stdout:7/439: chown d3/d31/l62 3 1 2024-03-24T00:48:54.098 INFO:tasks.workunit.client.0.smithi038.stdout:0/399: symlink d0/d12/d42/l81 0 2024-03-24T00:48:54.100 INFO:tasks.workunit.client.1.smithi088.stdout:3/485: dwrite d3/d6/f3b [0,4194304] 0 2024-03-24T00:48:54.101 INFO:tasks.workunit.client.1.smithi088.stdout:6/540: dread db/d1d/d23/d3c/d43/f39 [4194304,4194304] 0 2024-03-24T00:48:54.102 INFO:tasks.workunit.client.1.smithi088.stdout:0/512: link d9/dc/de/d19/l42 d9/dc/de/d19/d8d/la5 0 2024-03-24T00:48:54.103 INFO:tasks.workunit.client.0.smithi038.stdout:1/553: truncate da/dd/d22/d5c/f6e 810517 0 2024-03-24T00:48:54.105 INFO:tasks.workunit.client.1.smithi088.stdout:7/440: dread d3/d32/d34/f3b [0,4194304] 0 2024-03-24T00:48:54.105 INFO:tasks.workunit.client.1.smithi088.stdout:7/441: chown d3/d32/f48 880 1 2024-03-24T00:48:54.105 INFO:tasks.workunit.client.1.smithi088.stdout:7/442: read d3/f30 [584349,106801] 0 2024-03-24T00:48:54.105 INFO:tasks.workunit.client.1.smithi088.stdout:7/443: chown d3/f2c 147748841 1 2024-03-24T00:48:54.127 INFO:tasks.workunit.client.0.smithi038.stdout:5/466: creat d6/d60/f9a x:0 0 0 2024-03-24T00:48:54.134 INFO:tasks.workunit.client.0.smithi038.stdout:6/508: rmdir d1/da/d1b/d25/d49/d72 39 2024-03-24T00:48:54.142 INFO:tasks.workunit.client.1.smithi088.stdout:4/517: write d5/d17/f1c [2788848,124117] 0 2024-03-24T00:48:54.154 INFO:tasks.workunit.client.0.smithi038.stdout:4/431: dwrite da/f3b [0,4194304] 0 2024-03-24T00:48:54.154 INFO:tasks.workunit.client.0.smithi038.stdout:4/432: read da/d12/d37/dd/f3a [941085,74595] 0 2024-03-24T00:48:54.184 INFO:tasks.workunit.client.1.smithi088.stdout:9/551: creat d1/d22/d27/d39/d98/fae x:0 0 0 2024-03-24T00:48:54.189 INFO:tasks.workunit.client.1.smithi088.stdout:6/541: dwrite db/dd/d61/d6d/d74/d88/f86 [4194304,4194304] 0 2024-03-24T00:48:54.189 INFO:tasks.workunit.client.1.smithi088.stdout:6/542: fdatasync db/dd/d61/d6d/f91 0 2024-03-24T00:48:54.189 INFO:tasks.workunit.client.1.smithi088.stdout:5/543: link d1/dd/f54 d1/d10/d1b/d2f/d57/d9e/fab 0 2024-03-24T00:48:54.189 INFO:tasks.workunit.client.1.smithi088.stdout:5/544: chown d1/d10/d1b/d9d 4102720 1 2024-03-24T00:48:54.193 INFO:tasks.workunit.client.0.smithi038.stdout:0/400: dwrite d0/d12/d15/d58/d64/f7d [0,4194304] 0 2024-03-24T00:48:54.193 INFO:tasks.workunit.client.0.smithi038.stdout:0/401: chown d0/d1b/f20 9296 1 2024-03-24T00:48:54.193 INFO:tasks.workunit.client.0.smithi038.stdout:0/402: chown d0/c53 16406 1 2024-03-24T00:48:54.193 INFO:tasks.workunit.client.0.smithi038.stdout:0/403: creat d0/d12/d15/d58/d64/f82 x:0 0 0 2024-03-24T00:48:54.194 INFO:tasks.workunit.client.0.smithi038.stdout:5/467: symlink d6/d86/l9b 0 2024-03-24T00:48:54.194 INFO:tasks.workunit.client.0.smithi038.stdout:1/554: mkdir da/dd/d7a/d82/d98/d14/dbb 0 2024-03-24T00:48:54.194 INFO:tasks.workunit.client.0.smithi038.stdout:6/509: dread d1/da/d1b/f91 [0,4194304] 0 2024-03-24T00:48:54.194 INFO:tasks.workunit.client.0.smithi038.stdout:6/510: creat d1/da/d1b/fa1 x:0 0 0 2024-03-24T00:48:54.195 INFO:tasks.workunit.client.0.smithi038.stdout:3/512: dwrite d3/d9/dc/df/f1d [0,4194304] 0 2024-03-24T00:48:54.197 INFO:tasks.workunit.client.0.smithi038.stdout:7/482: rename d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/d3a to d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4 0 2024-03-24T00:48:54.197 INFO:tasks.workunit.client.0.smithi038.stdout:7/483: chown d2/d2c/f62 1314377 1 2024-03-24T00:48:54.201 INFO:tasks.workunit.client.1.smithi088.stdout:2/523: creat db/dd/d96/d4e/d8f/fa7 x:0 0 0 2024-03-24T00:48:54.201 INFO:tasks.workunit.client.1.smithi088.stdout:8/536: creat d9/dc/d11/d41/d54/d5d/db3/fb4 x:0 0 0 2024-03-24T00:48:54.206 INFO:tasks.workunit.client.1.smithi088.stdout:0/513: dwrite d9/dc/de/d19/d44/d46/d65/f35 [0,4194304] 0 2024-03-24T00:48:54.207 INFO:tasks.workunit.client.1.smithi088.stdout:7/444: stat d3/d32/d34/d7c/l88 0 2024-03-24T00:48:54.215 INFO:tasks.workunit.client.1.smithi088.stdout:4/518: link d5/d13/d18/d24/f9c d5/d16/d94/fb2 0 2024-03-24T00:48:54.216 INFO:tasks.workunit.client.0.smithi038.stdout:4/433: dwrite da/d38/f81 [0,4194304] 0 2024-03-24T00:48:54.219 INFO:tasks.workunit.client.0.smithi038.stdout:0/404: unlink d0/d14/c3f 0 2024-03-24T00:48:54.224 INFO:tasks.workunit.client.0.smithi038.stdout:5/468: unlink d6/d60/l95 0 2024-03-24T00:48:54.224 INFO:tasks.workunit.client.0.smithi038.stdout:5/469: creat d6/d19/d20/d42/d43/d4c/f9c x:0 0 0 2024-03-24T00:48:54.224 INFO:tasks.workunit.client.0.smithi038.stdout:5/470: write d6/d19/d20/d42/f72 [1474856,105136] 0 2024-03-24T00:48:54.224 INFO:tasks.workunit.client.0.smithi038.stdout:5/471: creat d6/d19/d20/d42/d43/d4c/f9d x:0 0 0 2024-03-24T00:48:54.225 INFO:tasks.workunit.client.1.smithi088.stdout:9/552: dwrite d1/ff [4194304,4194304] 0 2024-03-24T00:48:54.227 INFO:tasks.workunit.client.0.smithi038.stdout:1/555: creat da/dd/d22/dab/fbc x:0 0 0 2024-03-24T00:48:54.227 INFO:tasks.workunit.client.0.smithi038.stdout:1/556: creat da/d3c/d86/fbd x:0 0 0 2024-03-24T00:48:54.227 INFO:tasks.workunit.client.1.smithi088.stdout:0/514: dread d9/dc/de/d19/d44/d46/d65/f68 [0,4194304] 0 2024-03-24T00:48:54.227 INFO:tasks.workunit.client.1.smithi088.stdout:0/515: chown d9/dc/de/d19/d44/d46/d65/d2a 1248088307 1 2024-03-24T00:48:54.227 INFO:tasks.workunit.client.1.smithi088.stdout:0/516: chown d9/dc/de/d7e/c8e 90958 1 2024-03-24T00:48:54.228 INFO:tasks.workunit.client.1.smithi088.stdout:5/545: symlink d1/d10/d1b/d2e/d46/lac 0 2024-03-24T00:48:54.229 INFO:tasks.workunit.client.1.smithi088.stdout:3/486: getdents d3/de/d6f/d89 0 2024-03-24T00:48:54.229 INFO:tasks.workunit.client.1.smithi088.stdout:3/487: write d3/de/f16 [2959309,101325] 0 2024-03-24T00:48:54.229 INFO:tasks.workunit.client.1.smithi088.stdout:3/488: stat f1 0 2024-03-24T00:48:54.229 INFO:tasks.workunit.client.1.smithi088.stdout:3/489: truncate d3/d6/d27/d56/f82 5593200 0 2024-03-24T00:48:54.230 INFO:tasks.workunit.client.0.smithi038.stdout:6/511: mkdir d1/da/d61/da2 0 2024-03-24T00:48:54.230 INFO:tasks.workunit.client.0.smithi038.stdout:6/512: stat d1/c23 0 2024-03-24T00:48:54.231 INFO:tasks.workunit.client.1.smithi088.stdout:8/537: symlink d9/dc/d11/d41/lb5 0 2024-03-24T00:48:54.232 INFO:tasks.workunit.client.0.smithi038.stdout:3/513: creat d3/d30/fa9 x:0 0 0 2024-03-24T00:48:54.233 INFO:tasks.workunit.client.1.smithi088.stdout:2/524: truncate db/d21/d35/f5a 3305136 0 2024-03-24T00:48:54.234 INFO:tasks.workunit.client.1.smithi088.stdout:1/538: sync 2024-03-24T00:48:54.234 INFO:tasks.workunit.client.0.smithi038.stdout:7/484: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d75/cb5 0 2024-03-24T00:48:54.236 INFO:tasks.workunit.client.1.smithi088.stdout:7/445: rename d3/d32/d76/l85 to d3/d32/d3f/l8b 0 2024-03-24T00:48:54.240 INFO:tasks.workunit.client.1.smithi088.stdout:4/519: mkdir d5/d13/d18/d24/db3 0 2024-03-24T00:48:54.245 INFO:tasks.workunit.client.0.smithi038.stdout:4/434: unlink da/d38/l5d 0 2024-03-24T00:48:54.248 INFO:tasks.workunit.client.0.smithi038.stdout:0/405: mkdir d0/d12/d42/d83 0 2024-03-24T00:48:54.250 INFO:tasks.workunit.client.1.smithi088.stdout:6/543: rmdir db/d1d/d50 39 2024-03-24T00:48:54.251 INFO:tasks.workunit.client.1.smithi088.stdout:9/553: dread d1/ff [4194304,4194304] 0 2024-03-24T00:48:54.251 INFO:tasks.workunit.client.1.smithi088.stdout:5/546: symlink d1/d43/d4e/d5b/lad 0 2024-03-24T00:48:54.251 INFO:tasks.workunit.client.1.smithi088.stdout:5/547: creat d1/d19/d48/fae x:0 0 0 2024-03-24T00:48:54.254 INFO:tasks.workunit.client.1.smithi088.stdout:3/490: creat d3/de/d6f/d89/fa2 x:0 0 0 2024-03-24T00:48:54.254 INFO:tasks.workunit.client.1.smithi088.stdout:3/491: chown d3/de/d6f/d85 773 1 2024-03-24T00:48:54.258 INFO:tasks.workunit.client.1.smithi088.stdout:8/538: symlink d9/dc/d11/d24/d29/d36/d6a/d7b/lb6 0 2024-03-24T00:48:54.259 INFO:tasks.workunit.client.0.smithi038.stdout:5/472: creat d6/d19/d20/d42/d79/f9e x:0 0 0 2024-03-24T00:48:54.271 INFO:tasks.workunit.client.0.smithi038.stdout:1/557: symlink da/dd/d7a/d82/d98/d14/lbe 0 2024-03-24T00:48:54.271 INFO:tasks.workunit.client.0.smithi038.stdout:1/558: stat da/dd/d2c/d41/f65 0 2024-03-24T00:48:54.271 INFO:tasks.workunit.client.0.smithi038.stdout:1/559: truncate da/dd/d4a/d76/fad 839737 0 2024-03-24T00:48:54.271 INFO:tasks.workunit.client.0.smithi038.stdout:1/560: read - da/dd/d7a/f91 zero size 2024-03-24T00:48:54.271 INFO:tasks.workunit.client.0.smithi038.stdout:1/561: chown da/f72 41 1 2024-03-24T00:48:54.271 INFO:tasks.workunit.client.1.smithi088.stdout:4/520: unlink d5/d17/c27 0 2024-03-24T00:48:54.276 INFO:tasks.workunit.client.0.smithi038.stdout:6/513: creat d1/da/d61/d79/fa3 x:0 0 0 2024-03-24T00:48:54.278 INFO:tasks.workunit.client.1.smithi088.stdout:6/544: truncate f8 3114686 0 2024-03-24T00:48:54.278 INFO:tasks.workunit.client.1.smithi088.stdout:6/545: dread - db/dd/d87/f98 zero size 2024-03-24T00:48:54.280 INFO:tasks.workunit.client.0.smithi038.stdout:3/514: truncate d3/d9/dc/df/d14/f54 356014 0 2024-03-24T00:48:54.281 INFO:tasks.workunit.client.1.smithi088.stdout:9/554: write d1/d10/d29/f45 [472383,16771] 0 2024-03-24T00:48:54.281 INFO:tasks.workunit.client.1.smithi088.stdout:9/555: creat d1/d22/d72/faf x:0 0 0 2024-03-24T00:48:54.286 INFO:tasks.workunit.client.0.smithi038.stdout:4/435: mkdir da/d12/d24/d5f/d8a 0 2024-03-24T00:48:54.287 INFO:tasks.workunit.client.0.smithi038.stdout:4/436: write da/d12/d37/dd/f3a [574118,2114] 0 2024-03-24T00:48:54.288 INFO:tasks.workunit.client.1.smithi088.stdout:3/492: mknod d3/de/d6f/d85/d90/ca3 0 2024-03-24T00:48:54.288 INFO:tasks.workunit.client.0.smithi038.stdout:0/406: mkdir d0/d14/d84 0 2024-03-24T00:48:54.289 INFO:tasks.workunit.client.0.smithi038.stdout:0/407: stat d0/d2d/c2e 0 2024-03-24T00:48:54.289 INFO:tasks.workunit.client.0.smithi038.stdout:1/562: mknod da/dd/d2c/d5a/d74/cbf 0 2024-03-24T00:48:54.290 INFO:tasks.workunit.client.1.smithi088.stdout:0/517: dwrite d9/dc/de/f3e [0,4194304] 0 2024-03-24T00:48:54.292 INFO:tasks.workunit.client.0.smithi038.stdout:5/473: creat d6/d1e/f9f x:0 0 0 2024-03-24T00:48:54.292 INFO:tasks.workunit.client.0.smithi038.stdout:1/563: truncate da/dd/d7a/d82/d98/fba 465153 0 2024-03-24T00:48:54.293 INFO:tasks.workunit.client.1.smithi088.stdout:8/539: rmdir d9/dc/d11/d24/d43/d4a/d95/da5 39 2024-03-24T00:48:54.293 INFO:tasks.workunit.client.1.smithi088.stdout:8/540: write d9/dc/d11/d24/d29/d73/f76 [1474871,37303] 0 2024-03-24T00:48:54.293 INFO:tasks.workunit.client.1.smithi088.stdout:7/446: write d3/fc [2041333,116214] 0 2024-03-24T00:48:54.294 INFO:tasks.workunit.client.1.smithi088.stdout:2/525: dwrite db/dd/f17 [0,4194304] 0 2024-03-24T00:48:54.294 INFO:tasks.workunit.client.1.smithi088.stdout:2/526: chown db/dd/d96/c94 25991147 1 2024-03-24T00:48:54.295 INFO:tasks.workunit.client.0.smithi038.stdout:6/514: rename d1/f21 to d1/d37/d84/fa4 0 2024-03-24T00:48:54.296 INFO:tasks.workunit.client.0.smithi038.stdout:6/515: write d1/da/f67 [880564,6391] 0 2024-03-24T00:48:54.296 INFO:tasks.workunit.client.0.smithi038.stdout:6/516: chown d1/da/d1b/d25/l78 0 1 2024-03-24T00:48:54.302 INFO:tasks.workunit.client.0.smithi038.stdout:4/437: dread da/d12/d24/d68/f82 [0,4194304] 0 2024-03-24T00:48:54.312 INFO:tasks.workunit.client.0.smithi038.stdout:3/515: creat d3/d2a/faa x:0 0 0 2024-03-24T00:48:54.316 INFO:tasks.workunit.client.1.smithi088.stdout:4/521: link d5/d13/f96 d5/d7/d15/d74/d95/fb4 0 2024-03-24T00:48:54.316 INFO:tasks.workunit.client.1.smithi088.stdout:4/522: dread - d5/d13/d18/f19 zero size 2024-03-24T00:48:54.323 INFO:tasks.workunit.client.0.smithi038.stdout:0/408: creat d0/d14/d84/f85 x:0 0 0 2024-03-24T00:48:54.323 INFO:tasks.workunit.client.0.smithi038.stdout:0/409: getdents d0/d12/d42/d83 0 2024-03-24T00:48:54.329 INFO:tasks.workunit.client.0.smithi038.stdout:4/438: truncate da/d12/d37/dd/d15/f17 4496537 0 2024-03-24T00:48:54.335 INFO:tasks.workunit.client.0.smithi038.stdout:4/439: symlink da/d12/d29/d5b/l8b 0 2024-03-24T00:48:54.340 INFO:tasks.workunit.client.1.smithi088.stdout:6/546: mkdir db/dd/d61/d6d/d74/d88/da1 0 2024-03-24T00:48:54.340 INFO:tasks.workunit.client.1.smithi088.stdout:6/547: creat db/d1d/d23/d3c/d43/fa2 x:0 0 0 2024-03-24T00:48:54.340 INFO:tasks.workunit.client.1.smithi088.stdout:6/548: readlink db/d1d/d50/d90/d56/l79 0 2024-03-24T00:48:54.340 INFO:tasks.workunit.client.1.smithi088.stdout:6/549: chown db/dd/l25 509192 1 2024-03-24T00:48:54.341 INFO:tasks.workunit.client.1.smithi088.stdout:6/550: stat db/d1d/d50/d90/d56/c4f 0 2024-03-24T00:48:54.345 INFO:tasks.workunit.client.1.smithi088.stdout:0/518: symlink d9/d26/d8f/da1/d62/la6 0 2024-03-24T00:48:54.355 INFO:tasks.workunit.client.0.smithi038.stdout:7/485: dwrite d2/d9/da/dc/d8a/fa5 [0,4194304] 0 2024-03-24T00:48:54.356 INFO:tasks.workunit.client.1.smithi088.stdout:7/447: unlink d3/d32/d3f/d4b/d74/d79/c84 0 2024-03-24T00:48:54.357 INFO:tasks.workunit.client.1.smithi088.stdout:8/541: creat d9/dc/d11/d24/d43/d4a/d95/da5/fb7 x:0 0 0 2024-03-24T00:48:54.357 INFO:tasks.workunit.client.1.smithi088.stdout:8/542: chown d9/dc/d11/d24/d29/d36/c8a 20 1 2024-03-24T00:48:54.365 INFO:tasks.workunit.client.1.smithi088.stdout:4/523: rename d5/f62 to d5/fb5 0 2024-03-24T00:48:54.365 INFO:tasks.workunit.client.1.smithi088.stdout:2/527: rename db/dc/c67 to db/d54/d88/ca8 0 2024-03-24T00:48:54.369 INFO:tasks.workunit.client.1.smithi088.stdout:9/556: mkdir d1/d10/d29/db0 0 2024-03-24T00:48:54.372 INFO:tasks.workunit.client.1.smithi088.stdout:3/493: getdents d3/d1a/d1e/d76 0 2024-03-24T00:48:54.377 INFO:tasks.workunit.client.0.smithi038.stdout:0/410: write d0/d12/d15/d58/d64/f7d [1603827,23921] 0 2024-03-24T00:48:54.383 INFO:tasks.workunit.client.1.smithi088.stdout:6/551: symlink db/d3d/la3 0 2024-03-24T00:48:54.386 INFO:tasks.workunit.client.1.smithi088.stdout:7/448: dread d3/d32/d3f/d4b/f6c [0,4194304] 0 2024-03-24T00:48:54.387 INFO:tasks.workunit.client.1.smithi088.stdout:7/449: fdatasync d3/d31/f3c 0 2024-03-24T00:48:54.387 INFO:tasks.workunit.client.1.smithi088.stdout:0/519: link d9/d26/d8f/da1/d63/f8a d9/d26/d8f/da1/d63/fa7 0 2024-03-24T00:48:54.397 INFO:tasks.workunit.client.1.smithi088.stdout:9/557: symlink d1/d15/lb1 0 2024-03-24T00:48:54.397 INFO:tasks.workunit.client.0.smithi038.stdout:6/517: write d1/da/d1b/d25/d49/d4e/f95 [1678975,69376] 0 2024-03-24T00:48:54.399 INFO:tasks.workunit.client.1.smithi088.stdout:9/558: read d1/d10/d29/f45 [86244,107197] 0 2024-03-24T00:48:54.399 INFO:tasks.workunit.client.1.smithi088.stdout:9/559: dread - d1/d22/f8c zero size 2024-03-24T00:48:54.399 INFO:tasks.workunit.client.1.smithi088.stdout:9/560: read d1/d10/d11/f17 [638079,71329] 0 2024-03-24T00:48:54.400 INFO:tasks.workunit.client.1.smithi088.stdout:6/552: mknod db/d3d/d83/d71/ca4 0 2024-03-24T00:48:54.400 INFO:tasks.workunit.client.1.smithi088.stdout:3/494: mkdir d3/d6/d58/da4 0 2024-03-24T00:48:54.400 INFO:tasks.workunit.client.1.smithi088.stdout:7/450: creat d3/f8c x:0 0 0 2024-03-24T00:48:54.400 INFO:tasks.workunit.client.1.smithi088.stdout:3/495: truncate d3/d6/f78 431761 0 2024-03-24T00:48:54.401 INFO:tasks.workunit.client.0.smithi038.stdout:0/411: rename d0/d12/d15/d58/d64/f82 to d0/d1b/d2c/f86 0 2024-03-24T00:48:54.404 INFO:tasks.workunit.client.1.smithi088.stdout:0/520: mknod d9/dc/ca8 0 2024-03-24T00:48:54.404 INFO:tasks.workunit.client.1.smithi088.stdout:2/528: rmdir db/dd/d96/d4e 39 2024-03-24T00:48:54.404 INFO:tasks.workunit.client.1.smithi088.stdout:2/529: fsync db/d21/f44 0 2024-03-24T00:48:54.404 INFO:tasks.workunit.client.1.smithi088.stdout:2/530: chown db/dc/f2b 13771666 1 2024-03-24T00:48:54.404 INFO:tasks.workunit.client.1.smithi088.stdout:2/531: truncate db/f20 4871126 0 2024-03-24T00:48:54.406 INFO:tasks.workunit.client.0.smithi038.stdout:7/486: read d2/d2c/f4d [3793167,26680] 0 2024-03-24T00:48:54.412 INFO:tasks.workunit.client.1.smithi088.stdout:6/553: creat db/d1d/d23/fa5 x:0 0 0 2024-03-24T00:48:54.416 INFO:tasks.workunit.client.0.smithi038.stdout:6/518: rmdir d1/da/d73 39 2024-03-24T00:48:54.416 INFO:tasks.workunit.client.0.smithi038.stdout:6/519: chown d1/da/d1b/d2f/c3a 634498 1 2024-03-24T00:48:54.418 INFO:tasks.workunit.client.0.smithi038.stdout:1/564: dwrite da/dd/d22/f4c [0,4194304] 0 2024-03-24T00:48:54.418 INFO:tasks.workunit.client.0.smithi038.stdout:5/474: dwrite d6/d19/d20/d42/d43/d50/f81 [0,4194304] 0 2024-03-24T00:48:54.419 INFO:tasks.workunit.client.0.smithi038.stdout:3/516: dwrite d3/f3d [0,4194304] 0 2024-03-24T00:48:54.420 INFO:tasks.workunit.client.1.smithi088.stdout:3/496: symlink d3/d6/d27/d56/d6e/la5 0 2024-03-24T00:48:54.423 INFO:tasks.workunit.client.1.smithi088.stdout:0/521: mknod d9/ca9 0 2024-03-24T00:48:54.426 INFO:tasks.workunit.client.0.smithi038.stdout:7/487: link d2/d2c/f4d d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/fb6 0 2024-03-24T00:48:54.426 INFO:tasks.workunit.client.0.smithi038.stdout:7/488: truncate d2/d9/fa7 225759 0 2024-03-24T00:48:54.426 INFO:tasks.workunit.client.0.smithi038.stdout:0/412: dread d0/d12/d15/f22 [0,4194304] 0 2024-03-24T00:48:54.426 INFO:tasks.workunit.client.0.smithi038.stdout:0/413: creat d0/d1b/f87 x:0 0 0 2024-03-24T00:48:54.426 INFO:tasks.workunit.client.0.smithi038.stdout:0/414: dread - d0/d12/f70 zero size 2024-03-24T00:48:54.428 INFO:tasks.workunit.client.1.smithi088.stdout:6/554: creat db/d1d/d50/d90/d55/d99/fa6 x:0 0 0 2024-03-24T00:48:54.434 INFO:tasks.workunit.client.0.smithi038.stdout:6/520: creat d1/d37/fa5 x:0 0 0 2024-03-24T00:48:54.435 INFO:tasks.workunit.client.1.smithi088.stdout:8/543: dwrite d9/dc/d11/d24/d43/d4a/d95/da5/fb7 [0,4194304] 0 2024-03-24T00:48:54.437 INFO:tasks.workunit.client.1.smithi088.stdout:7/451: dread d3/f21 [0,4194304] 0 2024-03-24T00:48:54.439 INFO:tasks.workunit.client.1.smithi088.stdout:0/522: symlink d9/dc/de/d19/d8d/laa 0 2024-03-24T00:48:54.441 INFO:tasks.workunit.client.1.smithi088.stdout:0/523: write d9/dc/de/d19/f7f [385972,117444] 0 2024-03-24T00:48:54.444 INFO:tasks.workunit.client.0.smithi038.stdout:6/521: fdatasync d1/da/f67 0 2024-03-24T00:48:54.456 INFO:tasks.workunit.client.1.smithi088.stdout:2/532: dwrite db/d21/d35/f95 [0,4194304] 0 2024-03-24T00:48:54.458 INFO:tasks.workunit.client.1.smithi088.stdout:7/452: read d3/f28 [3747838,17113] 0 2024-03-24T00:48:54.461 INFO:tasks.workunit.client.0.smithi038.stdout:4/440: dwrite da/d38/f81 [4194304,4194304] 0 2024-03-24T00:48:54.471 INFO:tasks.workunit.client.0.smithi038.stdout:1/565: mkdir da/dd/d7a/d82/d98/d14/dc0 0 2024-03-24T00:48:54.477 INFO:tasks.workunit.client.0.smithi038.stdout:5/475: link d6/f9 d6/d19/d20/d42/d79/fa0 0 2024-03-24T00:48:54.478 INFO:tasks.workunit.client.1.smithi088.stdout:7/453: write d3/f2c [3365098,39032] 0 2024-03-24T00:48:54.478 INFO:tasks.workunit.client.1.smithi088.stdout:7/454: fdatasync d3/f2a 0 2024-03-24T00:48:54.480 INFO:tasks.workunit.client.0.smithi038.stdout:7/489: dwrite d2/f63 [0,4194304] 0 2024-03-24T00:48:54.481 INFO:tasks.workunit.client.1.smithi088.stdout:3/497: dwrite d3/de/d6f/d85/d49/f54 [8388608,4194304] 0 2024-03-24T00:48:54.481 INFO:tasks.workunit.client.1.smithi088.stdout:3/498: chown d3/de/d6f/d85/d90/ca3 3299821 1 2024-03-24T00:48:54.481 INFO:tasks.workunit.client.1.smithi088.stdout:3/499: chown d3/d6/d19/f77 22575 1 2024-03-24T00:48:54.485 INFO:tasks.workunit.client.1.smithi088.stdout:8/544: mknod d9/dc/d11/d24/d29/d39/cb8 0 2024-03-24T00:48:54.486 INFO:tasks.workunit.client.1.smithi088.stdout:8/545: fsync d9/f20 0 2024-03-24T00:48:54.487 INFO:tasks.workunit.client.1.smithi088.stdout:7/455: dread d3/f30 [0,4194304] 0 2024-03-24T00:48:54.490 INFO:tasks.workunit.client.0.smithi038.stdout:3/517: rmdir d3/d30/d70/d90/da0 0 2024-03-24T00:48:54.494 INFO:tasks.workunit.client.1.smithi088.stdout:0/524: link d9/dc/de/d19/d44/d46/l53 d9/lab 0 2024-03-24T00:48:54.494 INFO:tasks.workunit.client.1.smithi088.stdout:0/525: stat d9/d15 0 2024-03-24T00:48:54.500 INFO:tasks.workunit.client.0.smithi038.stdout:0/415: mkdir d0/d14/d88 0 2024-03-24T00:48:54.500 INFO:tasks.workunit.client.0.smithi038.stdout:0/416: write d0/f69 [524038,19274] 0 2024-03-24T00:48:54.501 INFO:tasks.workunit.client.1.smithi088.stdout:2/533: mknod db/d54/d88/d8e/ca9 0 2024-03-24T00:48:54.502 INFO:tasks.workunit.client.0.smithi038.stdout:6/522: truncate d1/da/d1b/d25/d49/d72/f76 319530 0 2024-03-24T00:48:54.502 INFO:tasks.workunit.client.0.smithi038.stdout:6/523: chown d1/da/f89 8052496 1 2024-03-24T00:48:54.507 INFO:tasks.workunit.client.0.smithi038.stdout:4/441: mknod da/d12/c8c 0 2024-03-24T00:48:54.508 INFO:tasks.workunit.client.1.smithi088.stdout:6/555: truncate db/dd/d61/d6d/d74/d88/f86 7045966 0 2024-03-24T00:48:54.516 INFO:tasks.workunit.client.0.smithi038.stdout:1/566: symlink da/dd/d7a/d82/lc1 0 2024-03-24T00:48:54.523 INFO:tasks.workunit.client.0.smithi038.stdout:5/476: truncate d6/d1e/f1f 1042583 0 2024-03-24T00:48:54.527 INFO:tasks.workunit.client.1.smithi088.stdout:5/548: sync 2024-03-24T00:48:54.527 INFO:tasks.workunit.client.1.smithi088.stdout:1/539: sync 2024-03-24T00:48:54.527 INFO:tasks.workunit.client.1.smithi088.stdout:5/549: fdatasync d1/dd/f13 0 2024-03-24T00:48:54.528 INFO:tasks.workunit.client.0.smithi038.stdout:8/469: sync 2024-03-24T00:48:54.528 INFO:tasks.workunit.client.0.smithi038.stdout:2/482: sync 2024-03-24T00:48:54.528 INFO:tasks.workunit.client.0.smithi038.stdout:8/470: fsync d0/d15/f94 0 2024-03-24T00:48:54.528 INFO:tasks.workunit.client.0.smithi038.stdout:2/483: creat d4/d76/fb3 x:0 0 0 2024-03-24T00:48:54.528 INFO:tasks.workunit.client.0.smithi038.stdout:2/484: chown d4/d27/d2f/d30/f8d 899 1 2024-03-24T00:48:54.528 INFO:tasks.workunit.client.0.smithi038.stdout:2/485: chown d4/d18/f28 2315 1 2024-03-24T00:48:54.528 INFO:tasks.workunit.client.0.smithi038.stdout:2/486: write d4/d18/d26/f8e [1068652,105133] 0 2024-03-24T00:48:54.529 INFO:tasks.workunit.client.0.smithi038.stdout:9/548: sync 2024-03-24T00:48:54.529 INFO:tasks.workunit.client.0.smithi038.stdout:2/487: truncate d4/d55/fb2 144685 0 2024-03-24T00:48:54.530 INFO:tasks.workunit.client.1.smithi088.stdout:3/500: unlink d3/d1a/d1e/d48/l5c 0 2024-03-24T00:48:54.531 INFO:tasks.workunit.client.0.smithi038.stdout:0/417: dread d0/d12/d15/d58/d64/f7d [0,4194304] 0 2024-03-24T00:48:54.531 INFO:tasks.workunit.client.0.smithi038.stdout:0/418: dread - d0/d2d/f65 zero size 2024-03-24T00:48:54.535 INFO:tasks.workunit.client.0.smithi038.stdout:7/490: symlink d2/d9/da/d23/d2b/d5b/d72/d92/d5d/lb7 0 2024-03-24T00:48:54.540 INFO:tasks.workunit.client.0.smithi038.stdout:6/524: dread d1/f1c [0,4194304] 0 2024-03-24T00:48:54.542 INFO:tasks.workunit.client.0.smithi038.stdout:1/567: dread da/dd/d7a/d82/d98/f9a [0,4194304] 0 2024-03-24T00:48:54.542 INFO:tasks.workunit.client.0.smithi038.stdout:1/568: fdatasync da/dd/d7a/d82/d98/d14/f47 0 2024-03-24T00:48:54.545 INFO:tasks.workunit.client.1.smithi088.stdout:7/456: truncate d3/d32/d3f/d4b/f60 4060118 0 2024-03-24T00:48:54.546 INFO:tasks.workunit.client.0.smithi038.stdout:3/518: creat d3/d9/dc/da6/fab x:0 0 0 2024-03-24T00:48:54.546 INFO:tasks.workunit.client.0.smithi038.stdout:3/519: readlink d3/d9/dc/df/l8f 0 2024-03-24T00:48:54.546 INFO:tasks.workunit.client.0.smithi038.stdout:3/520: creat d3/d30/fac x:0 0 0 2024-03-24T00:48:54.558 INFO:tasks.workunit.client.1.smithi088.stdout:1/540: symlink d6/d27/la0 0 2024-03-24T00:48:54.561 INFO:tasks.workunit.client.0.smithi038.stdout:9/549: read d12/d28/d3a/f2b [3240924,62319] 0 2024-03-24T00:48:54.564 INFO:tasks.workunit.client.0.smithi038.stdout:1/569: dread da/f56 [0,4194304] 0 2024-03-24T00:48:54.564 INFO:tasks.workunit.client.0.smithi038.stdout:1/570: write da/dd/d4a/f5b [641044,116311] 0 2024-03-24T00:48:54.567 INFO:tasks.workunit.client.1.smithi088.stdout:5/550: mkdir d1/dd/d6f/da1/d3f/d88/d8b/d94/daf 0 2024-03-24T00:48:54.567 INFO:tasks.workunit.client.1.smithi088.stdout:5/551: chown d1/d19/d1e/f42 262714172 1 2024-03-24T00:48:54.567 INFO:tasks.workunit.client.1.smithi088.stdout:5/552: truncate d1/d19/d1e/f9b 377371 0 2024-03-24T00:48:54.567 INFO:tasks.workunit.client.1.smithi088.stdout:5/553: dread - d1/d19/d1e/f6d zero size 2024-03-24T00:48:54.583 INFO:tasks.workunit.client.1.smithi088.stdout:3/501: rmdir d3/d6/d19 39 2024-03-24T00:48:54.584 INFO:tasks.workunit.client.1.smithi088.stdout:2/534: dwrite db/dd/d96/f4b [0,4194304] 0 2024-03-24T00:48:54.587 INFO:tasks.workunit.client.1.smithi088.stdout:0/526: dwrite d9/dc/de/d19/f48 [0,4194304] 0 2024-03-24T00:48:54.587 INFO:tasks.workunit.client.1.smithi088.stdout:0/527: stat d9/dc/de/f11 0 2024-03-24T00:48:54.587 INFO:tasks.workunit.client.1.smithi088.stdout:7/457: dread d3/f2d [4194304,4194304] 0 2024-03-24T00:48:54.587 INFO:tasks.workunit.client.1.smithi088.stdout:0/528: write d9/f85 [3300775,61253] 0 2024-03-24T00:48:54.587 INFO:tasks.workunit.client.1.smithi088.stdout:0/529: stat d9/dc/l79 0 2024-03-24T00:48:54.587 INFO:tasks.workunit.client.1.smithi088.stdout:8/546: rename d9/dc/d11/d24/d43/d4a/d95 to d9/dc/d11/d24/d29/d5b/db9 0 2024-03-24T00:48:54.587 INFO:tasks.workunit.client.1.smithi088.stdout:8/547: fsync d9/dc/d11/d41/d56/f59 0 2024-03-24T00:48:54.596 INFO:tasks.workunit.client.0.smithi038.stdout:0/419: getdents d0/d12/d15/d1a 0 2024-03-24T00:48:54.597 INFO:tasks.workunit.client.0.smithi038.stdout:7/491: mkdir d2/d9/da/dc/d8a/db8 0 2024-03-24T00:48:54.597 INFO:tasks.workunit.client.0.smithi038.stdout:6/525: symlink d1/da/d1b/d25/d49/d4e/d51/la6 0 2024-03-24T00:48:54.597 INFO:tasks.workunit.client.0.smithi038.stdout:0/420: creat d0/d12/f89 x:0 0 0 2024-03-24T00:48:54.606 INFO:tasks.workunit.client.1.smithi088.stdout:3/502: truncate d3/d1a/d1e/d2f/d45/f5d 3807421 0 2024-03-24T00:48:54.609 INFO:tasks.workunit.client.0.smithi038.stdout:3/521: mkdir d3/d9/dc/df/d14/dad 0 2024-03-24T00:48:54.610 INFO:tasks.workunit.client.1.smithi088.stdout:2/535: creat db/faa x:0 0 0 2024-03-24T00:48:54.614 INFO:tasks.workunit.client.0.smithi038.stdout:4/442: rename da/d12/d37/f62 to da/d12/d37/dd/f8d 0 2024-03-24T00:48:54.614 INFO:tasks.workunit.client.1.smithi088.stdout:7/458: dread d3/d32/d34/f37 [0,4194304] 0 2024-03-24T00:48:54.616 INFO:tasks.workunit.client.1.smithi088.stdout:6/556: dwrite db/d1d/d50/d90/d56/f31 [0,4194304] 0 2024-03-24T00:48:54.617 INFO:tasks.workunit.client.1.smithi088.stdout:1/541: dwrite d6/d27/f8e [0,4194304] 0 2024-03-24T00:48:54.617 INFO:tasks.workunit.client.1.smithi088.stdout:1/542: fsync d6/d6e/f79 0 2024-03-24T00:48:54.618 INFO:tasks.workunit.client.1.smithi088.stdout:5/554: dread d1/f31 [0,4194304] 0 2024-03-24T00:48:54.619 INFO:tasks.workunit.client.0.smithi038.stdout:1/571: dwrite da/f58 [0,4194304] 0 2024-03-24T00:48:54.619 INFO:tasks.workunit.client.0.smithi038.stdout:1/572: write da/dd/d2c/d40/f5d [256144,19279] 0 2024-03-24T00:48:54.621 INFO:tasks.workunit.client.1.smithi088.stdout:0/530: rename d9/dc/de/d7e/ca3 to d9/d26/d8f/da1/cac 0 2024-03-24T00:48:54.621 INFO:tasks.workunit.client.1.smithi088.stdout:0/531: chown d9/f1c 6 1 2024-03-24T00:48:54.621 INFO:tasks.workunit.client.1.smithi088.stdout:0/532: chown d9/dc/de/d19/d44/d46/d65/c74 82 1 2024-03-24T00:48:54.624 INFO:tasks.workunit.client.0.smithi038.stdout:1/573: dread da/f7e [0,4194304] 0 2024-03-24T00:48:54.626 INFO:tasks.workunit.client.1.smithi088.stdout:8/548: symlink d9/dc/d11/d24/d29/d39/d84/lba 0 2024-03-24T00:48:54.627 INFO:tasks.workunit.client.1.smithi088.stdout:4/524: sync 2024-03-24T00:48:54.627 INFO:tasks.workunit.client.1.smithi088.stdout:9/561: sync 2024-03-24T00:48:54.627 INFO:tasks.workunit.client.1.smithi088.stdout:4/525: write d5/d17/f2a [417708,64119] 0 2024-03-24T00:48:54.628 INFO:tasks.workunit.client.1.smithi088.stdout:9/562: write d1/d10/d77/d8a/f71 [719816,120881] 0 2024-03-24T00:48:54.628 INFO:tasks.workunit.client.1.smithi088.stdout:9/563: dread - d1/d15/f68 zero size 2024-03-24T00:48:54.633 INFO:tasks.workunit.client.0.smithi038.stdout:6/526: getdents d1/da/d1b/d25 0 2024-03-24T00:48:54.636 INFO:tasks.workunit.client.1.smithi088.stdout:7/459: mknod d3/d32/d3f/d4b/d74/d79/d7f/d89/c8d 0 2024-03-24T00:48:54.636 INFO:tasks.workunit.client.1.smithi088.stdout:7/460: write d3/d31/f7a [522980,18427] 0 2024-03-24T00:48:54.636 INFO:tasks.workunit.client.1.smithi088.stdout:7/461: write d3/d32/d3f/f61 [824009,44991] 0 2024-03-24T00:48:54.637 INFO:tasks.workunit.client.0.smithi038.stdout:0/421: creat d0/d12/d60/d28/f8a x:0 0 0 2024-03-24T00:48:54.640 INFO:tasks.workunit.client.0.smithi038.stdout:7/492: creat d2/fb9 x:0 0 0 2024-03-24T00:48:54.644 INFO:tasks.workunit.client.0.smithi038.stdout:4/443: mknod da/d12/d29/d5b/c8e 0 2024-03-24T00:48:54.654 INFO:tasks.workunit.client.0.smithi038.stdout:5/477: rename d6/d86 to d6/d19/da1 0 2024-03-24T00:48:54.654 INFO:tasks.workunit.client.0.smithi038.stdout:5/478: chown d6/d16/d14/c8c 16160 1 2024-03-24T00:48:54.660 INFO:tasks.workunit.client.1.smithi088.stdout:1/543: dwrite d6/d43/d95/d5b/d5d/d61/f7e [4194304,4194304] 0 2024-03-24T00:48:54.662 INFO:tasks.workunit.client.1.smithi088.stdout:6/557: symlink db/d1d/d23/d3c/d43/la7 0 2024-03-24T00:48:54.668 INFO:tasks.workunit.client.1.smithi088.stdout:0/533: truncate d9/f1c 1228670 0 2024-03-24T00:48:54.677 INFO:tasks.workunit.client.1.smithi088.stdout:5/555: dwrite d1/d19/d1e/f6d [0,4194304] 0 2024-03-24T00:48:54.677 INFO:tasks.workunit.client.1.smithi088.stdout:5/556: dread - d1/d10/d1b/d2f/d57/f8d zero size 2024-03-24T00:48:54.678 INFO:tasks.workunit.client.1.smithi088.stdout:5/557: fsync d1/dd/d6f/da1/d3f/d88/d8f/d69/f85 0 2024-03-24T00:48:54.678 INFO:tasks.workunit.client.1.smithi088.stdout:5/558: readlink d1/dd/d6f/da1/l34 0 2024-03-24T00:48:54.679 INFO:tasks.workunit.client.0.smithi038.stdout:1/574: link da/dd/d4a/d76/fad da/dd/d4a/fc2 0 2024-03-24T00:48:54.679 INFO:tasks.workunit.client.0.smithi038.stdout:1/575: stat da/dd/d7a/d82/lc1 0 2024-03-24T00:48:54.692 INFO:tasks.workunit.client.1.smithi088.stdout:1/544: dread d6/f10 [0,4194304] 0 2024-03-24T00:48:54.692 INFO:tasks.workunit.client.1.smithi088.stdout:1/545: read - d6/d43/d95/d58/f81 zero size 2024-03-24T00:48:54.695 INFO:tasks.workunit.client.1.smithi088.stdout:8/549: symlink d9/dc/d11/d24/d29/d39/d84/lbb 0 2024-03-24T00:48:54.695 INFO:tasks.workunit.client.1.smithi088.stdout:8/550: fdatasync d9/dc/d11/d24/d29/d39/d84/faa 0 2024-03-24T00:48:54.695 INFO:tasks.workunit.client.0.smithi038.stdout:7/493: link d2/d9/d16/c83 d2/d9/da/d23/d2b/d5b/d72/d92/d5d/cba 0 2024-03-24T00:48:54.695 INFO:tasks.workunit.client.0.smithi038.stdout:7/494: readlink d2/d9/da/d23/d2b/d5b/d72/d92/d5d/l79 0 2024-03-24T00:48:54.696 INFO:tasks.workunit.client.0.smithi038.stdout:1/576: dread da/dd/fb1 [0,4194304] 0 2024-03-24T00:48:54.696 INFO:tasks.workunit.client.0.smithi038.stdout:1/577: readlink da/dd/d2c/d41/d60/la1 0 2024-03-24T00:48:54.699 INFO:tasks.workunit.client.0.smithi038.stdout:8/471: rename d0/d1d/d27/d4d/f9f to d0/d15/d2c/fa2 0 2024-03-24T00:48:54.703 INFO:tasks.workunit.client.1.smithi088.stdout:4/526: rename d5/d16/c6c to d5/d13/d18/d6e/d7d/cb6 0 2024-03-24T00:48:54.704 INFO:tasks.workunit.client.1.smithi088.stdout:4/527: write d5/d17/f1d [3998081,82150] 0 2024-03-24T00:48:54.704 INFO:tasks.workunit.client.1.smithi088.stdout:4/528: dread - d5/d7/d15/d74/d95/f99 zero size 2024-03-24T00:48:54.706 INFO:tasks.workunit.client.1.smithi088.stdout:9/564: creat d1/d15/d80/da5/fb2 x:0 0 0 2024-03-24T00:48:54.710 INFO:tasks.workunit.client.1.smithi088.stdout:3/503: getdents d3/de/d6f/d89 0 2024-03-24T00:48:54.714 INFO:tasks.workunit.client.1.smithi088.stdout:7/462: stat d3/c18 0 2024-03-24T00:48:54.714 INFO:tasks.workunit.client.1.smithi088.stdout:7/463: readlink d3/d32/l55 0 2024-03-24T00:48:54.717 INFO:tasks.workunit.client.1.smithi088.stdout:6/558: truncate db/d1d/d23/d3c/d43/f34 1081756 0 2024-03-24T00:48:54.717 INFO:tasks.workunit.client.1.smithi088.stdout:6/559: write db/dd/d61/d6d/d74/d88/f9c [7871,55805] 0 2024-03-24T00:48:54.717 INFO:tasks.workunit.client.1.smithi088.stdout:8/551: dread d9/dc/d11/d24/f71 [0,4194304] 0 2024-03-24T00:48:54.717 INFO:tasks.workunit.client.1.smithi088.stdout:8/552: fdatasync d9/dc/d11/d24/d29/d36/f86 0 2024-03-24T00:48:54.727 INFO:tasks.workunit.client.1.smithi088.stdout:0/534: dwrite d9/d26/f61 [4194304,4194304] 0 2024-03-24T00:48:54.727 INFO:tasks.workunit.client.1.smithi088.stdout:0/535: truncate d9/dc/de/d19/d8d/f98 98330 0 2024-03-24T00:48:54.727 INFO:tasks.workunit.client.1.smithi088.stdout:0/536: dread - d9/dc/de/d19/f93 zero size 2024-03-24T00:48:54.729 INFO:tasks.workunit.client.1.smithi088.stdout:6/560: dread db/d1d/d23/d5c/f78 [0,4194304] 0 2024-03-24T00:48:54.730 INFO:tasks.workunit.client.0.smithi038.stdout:0/422: rmdir d0/d12 39 2024-03-24T00:48:54.731 INFO:tasks.workunit.client.0.smithi038.stdout:7/495: creat d2/d9/da/d23/d2b/fbb x:0 0 0 2024-03-24T00:48:54.737 INFO:tasks.workunit.client.1.smithi088.stdout:1/546: link d6/d27/d67/d88/f92 d6/d27/d67/d88/fa1 0 2024-03-24T00:48:54.737 INFO:tasks.workunit.client.1.smithi088.stdout:5/559: rename d1/fa7 to d1/d10/d1b/d2e/d46/fb0 0 2024-03-24T00:48:54.737 INFO:tasks.workunit.client.1.smithi088.stdout:4/529: creat d5/d13/d18/d24/d3d/fb7 x:0 0 0 2024-03-24T00:48:54.738 INFO:tasks.workunit.client.0.smithi038.stdout:4/444: mkdir da/d12/d37/dd/d15/d6d/d8f 0 2024-03-24T00:48:54.738 INFO:tasks.workunit.client.0.smithi038.stdout:4/445: write da/d12/d37/dd/d15/d4c/f6a [482165,89201] 0 2024-03-24T00:48:54.745 INFO:tasks.workunit.client.1.smithi088.stdout:5/560: write d1/dd/d6f/da1/d3f/d88/d8f/d69/f8c [1415283,53284] 0 2024-03-24T00:48:54.745 INFO:tasks.workunit.client.0.smithi038.stdout:1/578: creat da/dd/d2c/d41/d60/dae/fc3 x:0 0 0 2024-03-24T00:48:54.746 INFO:tasks.workunit.client.0.smithi038.stdout:8/472: mkdir d0/d1d/d20/d3c/d9e/da3 0 2024-03-24T00:48:54.756 INFO:tasks.workunit.client.0.smithi038.stdout:2/488: rename d4/d27/d54/d53/c6a to d4/da3/cb4 0 2024-03-24T00:48:54.761 INFO:tasks.workunit.client.0.smithi038.stdout:5/479: dwrite d6/d19/d20/d42/d43/d68/d6e/f7f [0,4194304] 0 2024-03-24T00:48:54.776 INFO:tasks.workunit.client.0.smithi038.stdout:0/423: symlink d0/d12/l8b 0 2024-03-24T00:48:54.776 INFO:tasks.workunit.client.0.smithi038.stdout:4/446: unlink da/d38/f81 0 2024-03-24T00:48:54.776 INFO:tasks.workunit.client.0.smithi038.stdout:8/473: mkdir d0/d1d/d27/d6b/da4 0 2024-03-24T00:48:54.777 INFO:tasks.workunit.client.0.smithi038.stdout:6/527: rename d1/da/d1b/d25/d92 to d1/da/d1b/da7 0 2024-03-24T00:48:54.777 INFO:tasks.workunit.client.0.smithi038.stdout:6/528: creat d1/da/d1b/fa8 x:0 0 0 2024-03-24T00:48:54.781 INFO:tasks.workunit.client.0.smithi038.stdout:0/424: write d0/d1b/f20 [177237,116584] 0 2024-03-24T00:48:54.787 INFO:tasks.workunit.client.1.smithi088.stdout:9/565: dwrite d1/d10/d29/d85/fac [0,4194304] 0 2024-03-24T00:48:54.787 INFO:tasks.workunit.client.0.smithi038.stdout:5/480: symlink d6/d16/d21/d2d/la2 0 2024-03-24T00:48:54.790 INFO:tasks.workunit.client.0.smithi038.stdout:4/447: creat da/d12/d24/d68/d6c/d7c/f90 x:0 0 0 2024-03-24T00:48:54.793 INFO:tasks.workunit.client.1.smithi088.stdout:8/553: dwrite d9/dc/d11/d24/d29/d5b/d8b/f96 [0,4194304] 0 2024-03-24T00:48:54.794 INFO:tasks.workunit.client.0.smithi038.stdout:8/474: link d0/d45/d37/d5f/c7d d0/d1d/d27/d4d/ca5 0 2024-03-24T00:48:54.802 INFO:tasks.workunit.client.1.smithi088.stdout:0/537: creat d9/d26/d8f/fad x:0 0 0 2024-03-24T00:48:54.809 INFO:tasks.workunit.client.0.smithi038.stdout:6/529: mknod d1/da/d1b/da7/ca9 0 2024-03-24T00:48:54.810 INFO:tasks.workunit.client.0.smithi038.stdout:6/530: write d1/da/d61/d69/f6e [331703,20608] 0 2024-03-24T00:48:54.817 INFO:tasks.workunit.client.0.smithi038.stdout:0/425: symlink d0/d12/d15/d1a/l8c 0 2024-03-24T00:48:54.817 INFO:tasks.workunit.client.1.smithi088.stdout:7/464: dwrite d3/f20 [0,4194304] 0 2024-03-24T00:48:54.818 INFO:tasks.workunit.client.1.smithi088.stdout:4/530: unlink d5/d16/d37/f6d 0 2024-03-24T00:48:54.818 INFO:tasks.workunit.client.1.smithi088.stdout:4/531: dread - d5/d16/d1a/d22/f52 zero size 2024-03-24T00:48:54.821 INFO:tasks.workunit.client.1.smithi088.stdout:5/561: mkdir d1/d10/d1b/d2f/d57/d9e/db1 0 2024-03-24T00:48:54.825 INFO:tasks.workunit.client.0.smithi038.stdout:5/481: rename d6/d19/d20/d42/d43/d68/d96 to d6/d19/d20/d70/da3 0 2024-03-24T00:48:54.829 INFO:tasks.workunit.client.0.smithi038.stdout:4/448: symlink da/d12/d37/dd/d15/d4c/d70/l91 0 2024-03-24T00:48:54.829 INFO:tasks.workunit.client.0.smithi038.stdout:4/449: stat da/d38/f63 0 2024-03-24T00:48:54.829 INFO:tasks.workunit.client.0.smithi038.stdout:4/450: stat da/d12/d37/dd/d15/d4c/f7a 0 2024-03-24T00:48:54.831 INFO:tasks.workunit.client.0.smithi038.stdout:2/489: dwrite d4/d27/d2f/d30/d3e/f8b [0,4194304] 0 2024-03-24T00:48:54.832 INFO:tasks.workunit.client.1.smithi088.stdout:1/547: dwrite d6/f17 [0,4194304] 0 2024-03-24T00:48:54.832 INFO:tasks.workunit.client.1.smithi088.stdout:1/548: chown d6/d27 13124314 1 2024-03-24T00:48:54.834 INFO:tasks.workunit.client.0.smithi038.stdout:8/475: mkdir d0/d45/d66/da6 0 2024-03-24T00:48:54.834 INFO:tasks.workunit.client.0.smithi038.stdout:8/476: chown d0/d45/d66/c2a 316889 1 2024-03-24T00:48:54.835 INFO:tasks.workunit.client.0.smithi038.stdout:7/496: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d98/f9f [0,4194304] 0 2024-03-24T00:48:54.840 INFO:tasks.workunit.client.0.smithi038.stdout:6/531: creat d1/faa x:0 0 0 2024-03-24T00:48:54.840 INFO:tasks.workunit.client.0.smithi038.stdout:6/532: truncate d1/faa 47043 0 2024-03-24T00:48:54.842 INFO:tasks.workunit.client.1.smithi088.stdout:9/566: unlink d1/d22/f96 0 2024-03-24T00:48:54.849 INFO:tasks.workunit.client.0.smithi038.stdout:1/579: dwrite da/dd/d7a/f91 [0,4194304] 0 2024-03-24T00:48:54.849 INFO:tasks.workunit.client.0.smithi038.stdout:1/580: chown da 17 1 2024-03-24T00:48:54.849 INFO:tasks.workunit.client.0.smithi038.stdout:1/581: truncate f4 2132984 0 2024-03-24T00:48:54.853 INFO:tasks.workunit.client.0.smithi038.stdout:0/426: rename d0/d1b/f4a to d0/d2d/f8d 0 2024-03-24T00:48:54.857 INFO:tasks.workunit.client.1.smithi088.stdout:2/536: sync 2024-03-24T00:48:54.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:54 smithi038 ceph-mon[29271]: pgmap v33: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 154 MiB/s rd, 198 MiB/s wr, 280 op/s 2024-03-24T00:48:54.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:54 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:48:54.860 INFO:tasks.workunit.client.0.smithi038.stdout:4/451: creat da/d12/d29/d71/f92 x:0 0 0 2024-03-24T00:48:54.868 INFO:tasks.workunit.client.1.smithi088.stdout:8/554: mkdir d9/dc/d11/d24/d29/d36/d6a/d7b/d8f/dbc 0 2024-03-24T00:48:54.871 INFO:tasks.workunit.client.0.smithi038.stdout:2/490: link d4/d18/d5b/l74 d4/d55/lb5 0 2024-03-24T00:48:54.871 INFO:tasks.workunit.client.0.smithi038.stdout:2/491: readlink d4/d27/d2f/d30/l60 0 2024-03-24T00:48:54.876 INFO:tasks.workunit.client.1.smithi088.stdout:3/504: rename d3/l22 to d3/d6/d19/d2a/d5f/la6 0 2024-03-24T00:48:54.876 INFO:tasks.workunit.client.1.smithi088.stdout:3/505: stat d3/de/d6f/d85/f9a 0 2024-03-24T00:48:54.881 INFO:tasks.workunit.client.1.smithi088.stdout:7/465: creat d3/d32/f8e x:0 0 0 2024-03-24T00:48:54.882 INFO:tasks.workunit.client.1.smithi088.stdout:2/537: dread db/f89 [0,4194304] 0 2024-03-24T00:48:54.883 INFO:tasks.workunit.client.1.smithi088.stdout:4/532: creat d5/dac/fb8 x:0 0 0 2024-03-24T00:48:54.883 INFO:tasks.workunit.client.1.smithi088.stdout:4/533: write d5/dac/fb8 [250474,125686] 0 2024-03-24T00:48:54.883 INFO:tasks.workunit.client.1.smithi088.stdout:4/534: stat d5/f56 0 2024-03-24T00:48:54.885 INFO:tasks.workunit.client.0.smithi038.stdout:8/477: mknod d0/d1d/d27/d6b/da4/ca7 0 2024-03-24T00:48:54.885 INFO:tasks.workunit.client.0.smithi038.stdout:8/478: fsync d0/f16 0 2024-03-24T00:48:54.890 INFO:tasks.workunit.client.0.smithi038.stdout:7/497: mknod d2/d9/da/d9b/cbc 0 2024-03-24T00:48:54.890 INFO:tasks.workunit.client.0.smithi038.stdout:7/498: chown d2/d2c/f4d 99 1 2024-03-24T00:48:54.890 INFO:tasks.workunit.client.0.smithi038.stdout:6/533: creat d1/da/d73/fab x:0 0 0 2024-03-24T00:48:54.892 INFO:tasks.workunit.client.1.smithi088.stdout:0/538: rename d9/dc/de/d19/c81 to d9/d26/d8f/da1/d62/cae 0 2024-03-24T00:48:54.897 INFO:tasks.workunit.client.1.smithi088.stdout:7/466: mkdir d3/d32/d76/d8f 0 2024-03-24T00:48:54.898 INFO:tasks.workunit.client.1.smithi088.stdout:7/467: read d3/f35 [4577283,100267] 0 2024-03-24T00:48:54.898 INFO:tasks.workunit.client.1.smithi088.stdout:7/468: fsync d3/d32/f5a 0 2024-03-24T00:48:54.898 INFO:tasks.workunit.client.1.smithi088.stdout:7/469: write d3/d32/f42 [15709,117433] 0 2024-03-24T00:48:54.898 INFO:tasks.workunit.client.1.smithi088.stdout:7/470: write d3/d32/f8e [859259,13606] 0 2024-03-24T00:48:54.898 INFO:tasks.workunit.client.1.smithi088.stdout:7/471: write d3/f30 [1275881,28269] 0 2024-03-24T00:48:54.900 INFO:tasks.workunit.client.1.smithi088.stdout:2/538: symlink db/d21/d35/d49/lab 0 2024-03-24T00:48:54.901 INFO:tasks.workunit.client.0.smithi038.stdout:5/482: dwrite d6/d1e/f1f [0,4194304] 0 2024-03-24T00:48:54.907 INFO:tasks.workunit.client.0.smithi038.stdout:8/479: write d0/d45/d37/f48 [2271095,96132] 0 2024-03-24T00:48:54.908 INFO:tasks.workunit.client.1.smithi088.stdout:4/535: mknod d5/d7/dd/d88/cb9 0 2024-03-24T00:48:54.908 INFO:tasks.workunit.client.1.smithi088.stdout:4/536: fdatasync d5/d7/d92/fad 0 2024-03-24T00:48:54.910 INFO:tasks.workunit.client.1.smithi088.stdout:1/549: rename d6/d23/l8c to d6/d43/d95/d5b/d5e/la2 0 2024-03-24T00:48:54.916 INFO:tasks.workunit.client.1.smithi088.stdout:2/539: creat db/d21/d35/d49/fac x:0 0 0 2024-03-24T00:48:54.917 INFO:tasks.workunit.client.1.smithi088.stdout:6/561: sync 2024-03-24T00:48:54.917 INFO:tasks.workunit.client.1.smithi088.stdout:6/562: dread - db/d3d/d83/d72/f76 zero size 2024-03-24T00:48:54.922 INFO:tasks.workunit.client.0.smithi038.stdout:1/582: unlink da/dd/d2c/d40/l45 0 2024-03-24T00:48:54.923 INFO:tasks.workunit.client.0.smithi038.stdout:0/427: mknod d0/d12/d42/c8e 0 2024-03-24T00:48:54.925 INFO:tasks.workunit.client.1.smithi088.stdout:6/563: mknod db/d1d/d50/d90/d55/ca8 0 2024-03-24T00:48:54.925 INFO:tasks.workunit.client.1.smithi088.stdout:2/540: getdents db/d54/d88 0 2024-03-24T00:48:54.925 INFO:tasks.workunit.client.0.smithi038.stdout:4/452: creat da/d12/d24/d68/f93 x:0 0 0 2024-03-24T00:48:54.926 INFO:tasks.workunit.client.0.smithi038.stdout:6/534: dread d1/da/f89 [0,4194304] 0 2024-03-24T00:48:54.929 INFO:tasks.workunit.client.1.smithi088.stdout:6/564: getdents db/d1d/d23/d3c 0 2024-03-24T00:48:54.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:54 smithi088 ceph-mon[36610]: pgmap v33: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 154 MiB/s rd, 198 MiB/s wr, 280 op/s 2024-03-24T00:48:54.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:54 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:48:54.930 INFO:tasks.workunit.client.0.smithi038.stdout:2/492: creat d4/d27/d2f/d30/d3e/d7c/fb6 x:0 0 0 2024-03-24T00:48:54.930 INFO:tasks.workunit.client.0.smithi038.stdout:2/493: readlink d4/d18/d38/d63/l65 0 2024-03-24T00:48:54.932 INFO:tasks.workunit.client.0.smithi038.stdout:8/480: creat d0/d45/fa8 x:0 0 0 2024-03-24T00:48:54.932 INFO:tasks.workunit.client.0.smithi038.stdout:7/499: symlink d2/d9/d16/lbd 0 2024-03-24T00:48:54.933 INFO:tasks.workunit.client.0.smithi038.stdout:0/428: rename d0/d12/l8b to d0/d2d/l8f 0 2024-03-24T00:48:54.933 INFO:tasks.workunit.client.0.smithi038.stdout:0/429: chown d0/d12/d42/d44/l54 508988485 1 2024-03-24T00:48:54.939 INFO:tasks.workunit.client.1.smithi088.stdout:6/565: unlink db/d1d/d23/d3c/d43/f85 0 2024-03-24T00:48:54.940 INFO:tasks.workunit.client.0.smithi038.stdout:6/535: getdents d1/da/d1b/d25 0 2024-03-24T00:48:54.946 INFO:tasks.workunit.client.1.smithi088.stdout:9/567: dwrite d1/d22/d27/f8d [0,4194304] 0 2024-03-24T00:48:54.949 INFO:tasks.workunit.client.1.smithi088.stdout:3/506: dread d3/d6/d19/d2a/f71 [0,4194304] 0 2024-03-24T00:48:54.949 INFO:tasks.workunit.client.1.smithi088.stdout:3/507: dread - d3/d6/d19/f77 zero size 2024-03-24T00:48:54.955 INFO:tasks.workunit.client.0.smithi038.stdout:0/430: creat d0/d12/d60/f90 x:0 0 0 2024-03-24T00:48:54.965 INFO:tasks.workunit.client.0.smithi038.stdout:2/494: dread d4/d27/f39 [0,4194304] 0 2024-03-24T00:48:54.972 INFO:tasks.workunit.client.0.smithi038.stdout:6/536: dread d1/da/fb [0,4194304] 0 2024-03-24T00:48:54.973 INFO:tasks.workunit.client.1.smithi088.stdout:0/539: dwrite d9/dc/de/f3e [0,4194304] 0 2024-03-24T00:48:54.973 INFO:tasks.workunit.client.1.smithi088.stdout:0/540: stat d9/dc/de/d19/c7b 0 2024-03-24T00:48:54.973 INFO:tasks.workunit.client.1.smithi088.stdout:0/541: fdatasync d9/dc/de/d7e/d91/f9f 0 2024-03-24T00:48:54.973 INFO:tasks.workunit.client.1.smithi088.stdout:0/542: truncate d9/d26/d8f/fad 1030643 0 2024-03-24T00:48:54.982 INFO:tasks.workunit.client.0.smithi038.stdout:4/453: rmdir da/d12/d29/d71 39 2024-03-24T00:48:54.982 INFO:tasks.workunit.client.0.smithi038.stdout:7/500: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/cbe 0 2024-03-24T00:48:54.982 INFO:tasks.workunit.client.0.smithi038.stdout:8/481: dread d0/d45/d66/d6d/f9b [0,4194304] 0 2024-03-24T00:48:54.983 INFO:tasks.workunit.client.0.smithi038.stdout:8/482: dread - d0/d45/d66/f93 zero size 2024-03-24T00:48:54.984 INFO:tasks.workunit.client.0.smithi038.stdout:0/431: getdents d0/d12/d15/d58/d64 0 2024-03-24T00:48:54.986 INFO:tasks.workunit.client.0.smithi038.stdout:9/550: sync 2024-03-24T00:48:54.986 INFO:tasks.workunit.client.0.smithi038.stdout:9/551: readlink d12/d28/d3a/l44 0 2024-03-24T00:48:54.986 INFO:tasks.workunit.client.0.smithi038.stdout:3/522: sync 2024-03-24T00:48:54.991 INFO:tasks.workunit.client.1.smithi088.stdout:8/555: dwrite d9/dc/d11/d24/d29/d5b/d6f/f83 [0,4194304] 0 2024-03-24T00:48:54.991 INFO:tasks.workunit.client.1.smithi088.stdout:8/556: read - d9/dc/d11/d24/d29/d73/faf zero size 2024-03-24T00:48:54.997 INFO:tasks.workunit.client.0.smithi038.stdout:6/537: dread d1/da/d1b/d1d/d3c/f86 [0,4194304] 0 2024-03-24T00:48:54.998 INFO:tasks.workunit.client.0.smithi038.stdout:4/454: creat da/d12/d29/d5b/d73/f94 x:0 0 0 2024-03-24T00:48:54.999 INFO:tasks.workunit.client.0.smithi038.stdout:4/455: dread - da/d12/d37/dd/d15/d6d/f4b zero size 2024-03-24T00:48:55.000 INFO:tasks.workunit.client.0.smithi038.stdout:0/432: fdatasync d0/d1b/f20 0 2024-03-24T00:48:55.008 INFO:tasks.workunit.client.0.smithi038.stdout:7/501: write d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f97 [564948,95139] 0 2024-03-24T00:48:55.009 INFO:tasks.workunit.client.1.smithi088.stdout:4/537: dwrite d5/d17/f1c [4194304,4194304] 0 2024-03-24T00:48:55.009 INFO:tasks.workunit.client.1.smithi088.stdout:4/538: chown d5/d13/fa7 3966005 1 2024-03-24T00:48:55.016 INFO:tasks.workunit.client.0.smithi038.stdout:8/483: symlink d0/d1d/d27/d4d/d81/la9 0 2024-03-24T00:48:55.016 INFO:tasks.workunit.client.1.smithi088.stdout:7/472: dwrite d3/d31/f70 [0,4194304] 0 2024-03-24T00:48:55.019 INFO:tasks.workunit.client.1.smithi088.stdout:1/550: dwrite d6/d27/f8e [4194304,4194304] 0 2024-03-24T00:48:55.024 INFO:tasks.workunit.client.0.smithi038.stdout:5/483: dwrite d6/d19/d20/d42/d4b/f4f [0,4194304] 0 2024-03-24T00:48:55.025 INFO:tasks.workunit.client.0.smithi038.stdout:5/484: chown d6/d16/d67/f8f 213643211 1 2024-03-24T00:48:55.028 INFO:tasks.workunit.client.1.smithi088.stdout:9/568: truncate d1/f20 1483260 0 2024-03-24T00:48:55.029 INFO:tasks.workunit.client.0.smithi038.stdout:1/583: dwrite da/f58 [0,4194304] 0 2024-03-24T00:48:55.029 INFO:tasks.workunit.client.0.smithi038.stdout:2/495: dwrite d4/d18/d26/f70 [0,4194304] 0 2024-03-24T00:48:55.036 INFO:tasks.workunit.client.0.smithi038.stdout:3/523: rename d3/d9/dc/df/d14/d19/l72 to d3/d30/d5d/lae 0 2024-03-24T00:48:55.036 INFO:tasks.workunit.client.0.smithi038.stdout:3/524: write d3/d9/dc/d51/f52 [1466828,67760] 0 2024-03-24T00:48:55.037 INFO:tasks.workunit.client.1.smithi088.stdout:6/566: dwrite db/d1d/d23/d5c/f78 [0,4194304] 0 2024-03-24T00:48:55.049 INFO:tasks.workunit.client.0.smithi038.stdout:9/552: dread d12/f1e [0,4194304] 0 2024-03-24T00:48:55.050 INFO:tasks.workunit.client.0.smithi038.stdout:3/525: write d3/f29 [3648627,68914] 0 2024-03-24T00:48:55.051 INFO:tasks.workunit.client.0.smithi038.stdout:6/538: rmdir d1/da/d1b/d25/d49/d4e 39 2024-03-24T00:48:55.051 INFO:tasks.workunit.client.0.smithi038.stdout:6/539: stat d1/da/d1b/d2f 0 2024-03-24T00:48:55.053 INFO:tasks.workunit.client.1.smithi088.stdout:5/562: sync 2024-03-24T00:48:55.053 INFO:tasks.workunit.client.1.smithi088.stdout:5/563: fdatasync d1/d19/d1e/f47 0 2024-03-24T00:48:55.054 INFO:tasks.workunit.client.0.smithi038.stdout:4/456: creat da/d12/d37/dd/f95 x:0 0 0 2024-03-24T00:48:55.054 INFO:tasks.workunit.client.0.smithi038.stdout:0/433: symlink d0/d12/d15/l91 0 2024-03-24T00:48:55.054 INFO:tasks.workunit.client.0.smithi038.stdout:0/434: fsync d0/d12/d60/d28/f39 0 2024-03-24T00:48:55.063 INFO:tasks.workunit.client.0.smithi038.stdout:7/502: creat d2/d9/da/d23/d2b/d5b/d72/fbf x:0 0 0 2024-03-24T00:48:55.067 INFO:tasks.workunit.client.1.smithi088.stdout:8/557: rename d9/f3c to d9/dc/d11/d41/d56/fbd 0 2024-03-24T00:48:55.068 INFO:tasks.workunit.client.0.smithi038.stdout:4/457: dread da/d12/d37/dd/d15/d4c/f5a [0,4194304] 0 2024-03-24T00:48:55.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/484: truncate d0/f16 1005631 0 2024-03-24T00:48:55.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/485: read - d0/d1d/f8a zero size 2024-03-24T00:48:55.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/486: chown d0/d10/d19/l71 143941044 1 2024-03-24T00:48:55.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/487: fdatasync d0/d45/d37/f87 0 2024-03-24T00:48:55.073 INFO:tasks.workunit.client.0.smithi038.stdout:5/485: truncate d6/d19/d20/d42/f91 3101805 0 2024-03-24T00:48:55.087 INFO:tasks.workunit.client.1.smithi088.stdout:4/539: creat d5/d16/d94/fba x:0 0 0 2024-03-24T00:48:55.099 INFO:tasks.workunit.client.0.smithi038.stdout:2/496: creat d4/da3/fb7 x:0 0 0 2024-03-24T00:48:55.100 INFO:tasks.workunit.client.0.smithi038.stdout:2/497: fsync d4/d27/d54/d53/f95 0 2024-03-24T00:48:55.111 INFO:tasks.workunit.client.0.smithi038.stdout:9/553: creat d12/d28/d3a/d26/fb7 x:0 0 0 2024-03-24T00:48:55.112 INFO:tasks.workunit.client.1.smithi088.stdout:7/473: creat d3/d32/d76/d8f/f90 x:0 0 0 2024-03-24T00:48:55.113 INFO:tasks.workunit.client.0.smithi038.stdout:1/584: dwrite da/dd/d7a/d82/d98/f54 [0,4194304] 0 2024-03-24T00:48:55.113 INFO:tasks.workunit.client.0.smithi038.stdout:1/585: readlink da/dd/d2c/d5a/d74/l7c 0 2024-03-24T00:48:55.116 INFO:tasks.workunit.client.0.smithi038.stdout:3/526: rename d3/d30/d5d/la1 to d3/d30/d5d/d6a/laf 0 2024-03-24T00:48:55.119 INFO:tasks.workunit.client.1.smithi088.stdout:1/551: rmdir d6 39 2024-03-24T00:48:55.120 INFO:tasks.workunit.client.0.smithi038.stdout:6/540: creat d1/da/d61/da2/fac x:0 0 0 2024-03-24T00:48:55.120 INFO:tasks.workunit.client.0.smithi038.stdout:6/541: fdatasync d1/d37/d84/fa4 0 2024-03-24T00:48:55.120 INFO:tasks.workunit.client.0.smithi038.stdout:6/542: truncate d1/da/d1b/d2f/f7f 4973599 0 2024-03-24T00:48:55.123 INFO:tasks.workunit.client.0.smithi038.stdout:5/486: creat d6/d19/d20/d42/d43/fa4 x:0 0 0 2024-03-24T00:48:55.123 INFO:tasks.workunit.client.0.smithi038.stdout:5/487: stat d6/d60 0 2024-03-24T00:48:55.127 INFO:tasks.workunit.client.0.smithi038.stdout:8/488: read d0/d1d/d27/f8e [3965996,69236] 0 2024-03-24T00:48:55.131 INFO:tasks.workunit.client.0.smithi038.stdout:9/554: mknod d12/d81/cb8 0 2024-03-24T00:48:55.134 INFO:tasks.workunit.client.1.smithi088.stdout:9/569: creat d1/d15/d80/fb3 x:0 0 0 2024-03-24T00:48:55.141 INFO:tasks.workunit.client.0.smithi038.stdout:3/527: mkdir d3/d30/d5d/d7c/db0 0 2024-03-24T00:48:55.144 INFO:tasks.workunit.client.1.smithi088.stdout:6/567: symlink db/dd/d61/la9 0 2024-03-24T00:48:55.146 INFO:tasks.workunit.client.0.smithi038.stdout:6/543: getdents d1/da/d1b/da7 0 2024-03-24T00:48:55.150 INFO:tasks.workunit.client.0.smithi038.stdout:5/488: creat d6/d16/d3c/fa5 x:0 0 0 2024-03-24T00:48:55.150 INFO:tasks.workunit.client.0.smithi038.stdout:5/489: stat d6/d1e/f63 0 2024-03-24T00:48:55.151 INFO:tasks.workunit.client.0.smithi038.stdout:5/490: truncate d6/d1e/f9f 448532 0 2024-03-24T00:48:55.154 INFO:tasks.workunit.client.0.smithi038.stdout:8/489: mknod d0/d1d/d20/d3c/d9e/da3/caa 0 2024-03-24T00:48:55.155 INFO:tasks.workunit.client.0.smithi038.stdout:9/555: mknod d12/d41/d9d/cb9 0 2024-03-24T00:48:55.155 INFO:tasks.workunit.client.0.smithi038.stdout:8/490: fdatasync d0/d45/d37/d5f/d68/f85 0 2024-03-24T00:48:55.159 INFO:tasks.workunit.client.0.smithi038.stdout:3/528: dread d3/d9/f16 [0,4194304] 0 2024-03-24T00:48:55.163 INFO:tasks.workunit.client.0.smithi038.stdout:5/491: mkdir d6/d16/d21/d2d/da6 0 2024-03-24T00:48:55.163 INFO:tasks.workunit.client.0.smithi038.stdout:5/492: chown d6/d23 218 1 2024-03-24T00:48:55.163 INFO:tasks.workunit.client.0.smithi038.stdout:5/493: creat d6/d19/d20/d42/d43/d50/fa7 x:0 0 0 2024-03-24T00:48:55.170 INFO:tasks.workunit.client.1.smithi088.stdout:4/540: dwrite d5/d13/d18/d24/d3d/f55 [0,4194304] 0 2024-03-24T00:48:55.172 INFO:tasks.workunit.client.1.smithi088.stdout:0/543: rmdir d9/dc 39 2024-03-24T00:48:55.176 INFO:tasks.workunit.client.1.smithi088.stdout:5/564: getdents d1/d43/d4e 0 2024-03-24T00:48:55.176 INFO:tasks.workunit.client.0.smithi038.stdout:9/556: write d12/d81/d84/f8e [775785,13764] 0 2024-03-24T00:48:55.177 INFO:tasks.workunit.client.0.smithi038.stdout:5/494: rename d6/d19/d20/d42/d43/d4c/f61 to d6/d1e/fa8 0 2024-03-24T00:48:55.177 INFO:tasks.workunit.client.0.smithi038.stdout:5/495: fsync d6/d19/d20/d42/d43/d50/f81 0 2024-03-24T00:48:55.177 INFO:tasks.workunit.client.0.smithi038.stdout:5/496: dread - d6/d19/d20/d42/d43/d4c/f9d zero size 2024-03-24T00:48:55.177 INFO:tasks.workunit.client.0.smithi038.stdout:8/491: dread d0/d1d/d20/d3c/f47 [0,4194304] 0 2024-03-24T00:48:55.178 INFO:tasks.workunit.client.0.smithi038.stdout:8/492: creat d0/d10/fab x:0 0 0 2024-03-24T00:48:55.180 INFO:tasks.workunit.client.1.smithi088.stdout:8/558: mknod d9/dc/d11/d24/d29/cbe 0 2024-03-24T00:48:55.180 INFO:tasks.workunit.client.1.smithi088.stdout:8/559: fsync d9/dc/d11/d24/d29/d5b/d6f/f83 0 2024-03-24T00:48:55.180 INFO:tasks.workunit.client.1.smithi088.stdout:8/560: stat d9/dc/d11/d24/d29/d5b/db9 0 2024-03-24T00:48:55.182 INFO:tasks.workunit.client.1.smithi088.stdout:5/565: dread d1/f31 [0,4194304] 0 2024-03-24T00:48:55.183 INFO:tasks.workunit.client.1.smithi088.stdout:7/474: symlink d3/d32/d3f/d4b/d74/l91 0 2024-03-24T00:48:55.185 INFO:tasks.workunit.client.0.smithi038.stdout:5/497: truncate d6/d16/d3c/f93 357048 0 2024-03-24T00:48:55.186 INFO:tasks.workunit.client.1.smithi088.stdout:1/552: rename d6/d43/d95/d58/f3d to d6/d27/d67/fa3 0 2024-03-24T00:48:55.187 INFO:tasks.workunit.client.0.smithi038.stdout:9/557: symlink d12/d55/lba 0 2024-03-24T00:48:55.194 INFO:tasks.workunit.client.1.smithi088.stdout:9/570: creat d1/d10/d77/fb4 x:0 0 0 2024-03-24T00:48:55.196 INFO:tasks.workunit.client.0.smithi038.stdout:8/493: mknod d0/d10/d19/d62/cac 0 2024-03-24T00:48:55.198 INFO:tasks.workunit.client.1.smithi088.stdout:0/544: getdents d9/dc/de/d19/d44/d46 0 2024-03-24T00:48:55.198 INFO:tasks.workunit.client.1.smithi088.stdout:0/545: dread - d9/dc/f7a zero size 2024-03-24T00:48:55.198 INFO:tasks.workunit.client.1.smithi088.stdout:8/561: dread d9/dc/d11/d24/d29/d5b/f53 [0,4194304] 0 2024-03-24T00:48:55.198 INFO:tasks.workunit.client.1.smithi088.stdout:0/546: write d9/dc/de/d19/d44/d46/d65/d2a/f92 [19320,120249] 0 2024-03-24T00:48:55.199 INFO:tasks.workunit.client.0.smithi038.stdout:4/458: dwrite da/d12/d37/dd/f34 [0,4194304] 0 2024-03-24T00:48:55.199 INFO:tasks.workunit.client.0.smithi038.stdout:0/435: dwrite d0/d1b/d2c/f6b [0,4194304] 0 2024-03-24T00:48:55.199 INFO:tasks.workunit.client.0.smithi038.stdout:0/436: write d0/f2 [658615,57893] 0 2024-03-24T00:48:55.200 INFO:tasks.workunit.client.0.smithi038.stdout:0/437: readlink d0/d2d/l37 0 2024-03-24T00:48:55.202 INFO:tasks.workunit.client.1.smithi088.stdout:5/566: mknod d1/d43/d4e/d5b/cb2 0 2024-03-24T00:48:55.203 INFO:tasks.workunit.client.0.smithi038.stdout:5/498: symlink d6/d19/d20/d42/d79/la9 0 2024-03-24T00:48:55.204 INFO:tasks.workunit.client.0.smithi038.stdout:9/558: symlink d12/d41/lbb 0 2024-03-24T00:48:55.206 INFO:tasks.workunit.client.0.smithi038.stdout:1/586: dwrite da/dd/d2c/d5a/d74/f75 [0,4194304] 0 2024-03-24T00:48:55.211 INFO:tasks.workunit.client.0.smithi038.stdout:8/494: rename d0/d1d/d27/d4d/f70 to d0/d10/d19/d62/fad 0 2024-03-24T00:48:55.211 INFO:tasks.workunit.client.1.smithi088.stdout:0/547: dread d9/d26/d8f/da1/d62/f6d [0,4194304] 0 2024-03-24T00:48:55.211 INFO:tasks.workunit.client.1.smithi088.stdout:0/548: chown d9/d26/d8f 239665934 1 2024-03-24T00:48:55.212 INFO:tasks.workunit.client.1.smithi088.stdout:2/541: sync 2024-03-24T00:48:55.212 INFO:tasks.workunit.client.1.smithi088.stdout:3/508: sync 2024-03-24T00:48:55.215 INFO:tasks.workunit.client.0.smithi038.stdout:6/544: dwrite d1/da/f34 [4194304,4194304] 0 2024-03-24T00:48:55.215 INFO:tasks.workunit.client.0.smithi038.stdout:6/545: fdatasync d1/da/d1b/d1d/f4f 0 2024-03-24T00:48:55.224 INFO:tasks.workunit.client.0.smithi038.stdout:5/499: write d6/f18 [2976924,108568] 0 2024-03-24T00:48:55.224 INFO:tasks.workunit.client.0.smithi038.stdout:5/500: readlink d6/d16/d3c/l7e 0 2024-03-24T00:48:55.224 INFO:tasks.workunit.client.0.smithi038.stdout:5/501: chown d6/d19/c98 471189 1 2024-03-24T00:48:55.225 INFO:tasks.workunit.client.1.smithi088.stdout:6/568: rename db/f38 to db/d1d/d23/faa 0 2024-03-24T00:48:55.225 INFO:tasks.workunit.client.1.smithi088.stdout:6/569: read - db/d1d/d23/d3c/d43/f9a zero size 2024-03-24T00:48:55.227 INFO:tasks.workunit.client.0.smithi038.stdout:4/459: mknod da/d12/d37/dd/d15/d6d/d8f/c96 0 2024-03-24T00:48:55.227 INFO:tasks.workunit.client.0.smithi038.stdout:8/495: dread d0/d1d/d20/f23 [0,4194304] 0 2024-03-24T00:48:55.228 INFO:tasks.workunit.client.0.smithi038.stdout:8/496: dread - d0/d10/f92 zero size 2024-03-24T00:48:55.228 INFO:tasks.workunit.client.0.smithi038.stdout:8/497: readlink d0/d1d/d27/l3f 0 2024-03-24T00:48:55.231 INFO:tasks.workunit.client.0.smithi038.stdout:0/438: mkdir d0/d12/d92 0 2024-03-24T00:48:55.236 INFO:tasks.workunit.client.1.smithi088.stdout:1/553: mkdir d6/d27/d67/d88/da4 0 2024-03-24T00:48:55.236 INFO:tasks.workunit.client.1.smithi088.stdout:1/554: dread - d6/d6e/f79 zero size 2024-03-24T00:48:55.245 INFO:tasks.workunit.client.0.smithi038.stdout:1/587: mkdir da/dd/d2c/db6/dc4 0 2024-03-24T00:48:55.248 INFO:tasks.workunit.client.0.smithi038.stdout:1/588: write da/dd/d2c/d40/f6c [413913,111126] 0 2024-03-24T00:48:55.249 INFO:tasks.workunit.client.1.smithi088.stdout:9/571: mkdir d1/db5 0 2024-03-24T00:48:55.250 INFO:tasks.workunit.client.0.smithi038.stdout:6/546: write d1/f7 [922923,61681] 0 2024-03-24T00:48:55.250 INFO:tasks.workunit.client.0.smithi038.stdout:6/547: write d1/da/f89 [390820,72254] 0 2024-03-24T00:48:55.259 INFO:tasks.workunit.client.1.smithi088.stdout:0/549: mknod d9/d26/d8f/da1/caf 0 2024-03-24T00:48:55.261 INFO:tasks.workunit.client.1.smithi088.stdout:3/509: rmdir d3/de/d6f/d85/d90 39 2024-03-24T00:48:55.266 INFO:tasks.workunit.client.1.smithi088.stdout:2/542: mkdir db/dd/d96/d4e/d8f/dad 0 2024-03-24T00:48:55.268 INFO:tasks.workunit.client.1.smithi088.stdout:7/475: dread d3/d32/d3f/d4b/f60 [0,4194304] 0 2024-03-24T00:48:55.269 INFO:tasks.workunit.client.1.smithi088.stdout:7/476: readlink d3/d31/l3d 0 2024-03-24T00:48:55.269 INFO:tasks.workunit.client.1.smithi088.stdout:7/477: write d3/f2a [1041646,6095] 0 2024-03-24T00:48:55.271 INFO:tasks.workunit.client.1.smithi088.stdout:8/562: dwrite d9/dc/d11/d24/f71 [0,4194304] 0 2024-03-24T00:48:55.271 INFO:tasks.workunit.client.1.smithi088.stdout:4/541: rename d5/d7/dd/f21 to d5/d13/d18/d6e/d7d/fbb 0 2024-03-24T00:48:55.274 INFO:tasks.workunit.client.1.smithi088.stdout:1/555: mkdir d6/d43/d95/d5b/d5d/d61/d7d/da5 0 2024-03-24T00:48:55.276 INFO:tasks.workunit.client.1.smithi088.stdout:9/572: symlink d1/d10/d11/lb6 0 2024-03-24T00:48:55.276 INFO:tasks.workunit.client.1.smithi088.stdout:9/573: write d1/d15/f68 [4268346,31713] 0 2024-03-24T00:48:55.281 INFO:tasks.workunit.client.1.smithi088.stdout:0/550: mkdir d9/d26/d8f/da1/d62/d9b/db0 0 2024-03-24T00:48:55.284 INFO:tasks.workunit.client.0.smithi038.stdout:0/439: stat d0/d1b/d2c/c4e 0 2024-03-24T00:48:55.284 INFO:tasks.workunit.client.0.smithi038.stdout:0/440: chown d0/d12/d15/d1a/l1c 321473896 1 2024-03-24T00:48:55.294 INFO:tasks.workunit.client.0.smithi038.stdout:6/548: creat d1/da/d1b/da7/fad x:0 0 0 2024-03-24T00:48:55.294 INFO:tasks.workunit.client.0.smithi038.stdout:1/589: rename da/dd/d4a/f5b to da/dd/d2c/d41/d60/fc5 0 2024-03-24T00:48:55.296 INFO:tasks.workunit.client.0.smithi038.stdout:5/502: rmdir d6/d60 39 2024-03-24T00:48:55.297 INFO:tasks.workunit.client.0.smithi038.stdout:5/503: truncate d6/d19/d20/d42/d79/f9e 539474 0 2024-03-24T00:48:55.300 INFO:tasks.workunit.client.1.smithi088.stdout:8/563: write d9/f32 [166051,34225] 0 2024-03-24T00:48:55.301 INFO:tasks.workunit.client.1.smithi088.stdout:1/556: symlink d6/d27/la6 0 2024-03-24T00:48:55.302 INFO:tasks.workunit.client.1.smithi088.stdout:2/543: link db/d54/la6 db/d21/lae 0 2024-03-24T00:48:55.302 INFO:tasks.workunit.client.1.smithi088.stdout:9/574: stat d1/f20 0 2024-03-24T00:48:55.306 INFO:tasks.workunit.client.1.smithi088.stdout:8/564: unlink d9/dc/d11/d24/f45 0 2024-03-24T00:48:55.310 INFO:tasks.workunit.client.0.smithi038.stdout:4/460: rmdir da/d12/d37/dd 39 2024-03-24T00:48:55.310 INFO:tasks.workunit.client.0.smithi038.stdout:4/461: creat da/d12/d24/d5f/f97 x:0 0 0 2024-03-24T00:48:55.313 INFO:tasks.workunit.client.1.smithi088.stdout:7/478: write d3/ff [3713676,122806] 0 2024-03-24T00:48:55.314 INFO:tasks.workunit.client.0.smithi038.stdout:6/549: symlink d1/da/d73/d75/lae 0 2024-03-24T00:48:55.314 INFO:tasks.workunit.client.0.smithi038.stdout:6/550: stat d1/da/d1b/d25/d49/d72/f76 0 2024-03-24T00:48:55.317 INFO:tasks.workunit.client.1.smithi088.stdout:1/557: creat d6/d43/d95/d5b/fa7 x:0 0 0 2024-03-24T00:48:55.317 INFO:tasks.workunit.client.1.smithi088.stdout:1/558: creat d6/d43/d95/d5b/fa8 x:0 0 0 2024-03-24T00:48:55.318 INFO:tasks.workunit.client.0.smithi038.stdout:1/590: symlink da/dd/d7a/d82/d98/d14/lc6 0 2024-03-24T00:48:55.322 INFO:tasks.workunit.client.1.smithi088.stdout:2/544: link db/dd/d96/l82 db/d21/d35/d49/laf 0 2024-03-24T00:48:55.324 INFO:tasks.workunit.client.0.smithi038.stdout:5/504: rename d6/d1e/fa8 to d6/d19/d20/d33/d6f/faa 0 2024-03-24T00:48:55.325 INFO:tasks.workunit.client.1.smithi088.stdout:9/575: mknod d1/d15/d80/da5/cb7 0 2024-03-24T00:48:55.325 INFO:tasks.workunit.client.1.smithi088.stdout:9/576: write d1/d10/d11/f17 [3296063,8444] 0 2024-03-24T00:48:55.328 INFO:tasks.workunit.client.0.smithi038.stdout:9/559: dwrite d12/d28/f51 [0,4194304] 0 2024-03-24T00:48:55.329 INFO:tasks.workunit.client.0.smithi038.stdout:9/560: read - d12/d28/d3a/d26/fb7 zero size 2024-03-24T00:48:55.329 INFO:tasks.workunit.client.0.smithi038.stdout:9/561: readlink d12/d1a/l5c 0 2024-03-24T00:48:55.331 INFO:tasks.workunit.client.1.smithi088.stdout:7/479: dread d3/f2a [0,4194304] 0 2024-03-24T00:48:55.331 INFO:tasks.workunit.client.1.smithi088.stdout:7/480: dread - d3/d31/d56/f7b zero size 2024-03-24T00:48:55.335 INFO:tasks.workunit.client.0.smithi038.stdout:4/462: creat da/d12/d29/f98 x:0 0 0 2024-03-24T00:48:55.335 INFO:tasks.workunit.client.0.smithi038.stdout:4/463: creat da/d12/d37/f99 x:0 0 0 2024-03-24T00:48:55.335 INFO:tasks.workunit.client.1.smithi088.stdout:8/565: rename d9/dc/d11/d24/d29/d36/d6a/d7b/l98 to d9/dc/d11/d41/d54/d5d/db3/lbf 0 2024-03-24T00:48:55.341 INFO:tasks.workunit.client.0.smithi038.stdout:6/551: unlink d1/da/d1b/d1d/d3c/l60 0 2024-03-24T00:48:55.341 INFO:tasks.workunit.client.0.smithi038.stdout:6/552: creat d1/da/d61/da2/faf x:0 0 0 2024-03-24T00:48:55.342 INFO:tasks.workunit.client.1.smithi088.stdout:2/545: mkdir db/dd/d96/d4e/d8f/d9c/db0 0 2024-03-24T00:48:55.343 INFO:tasks.workunit.client.0.smithi038.stdout:1/591: truncate da/dd/d2c/d5a/f7b 239106 0 2024-03-24T00:48:55.344 INFO:tasks.workunit.client.0.smithi038.stdout:5/505: truncate d6/d19/d20/d42/d4b/f4f 1947265 0 2024-03-24T00:48:55.346 INFO:tasks.workunit.client.0.smithi038.stdout:0/441: dwrite d0/f5 [0,4194304] 0 2024-03-24T00:48:55.347 INFO:tasks.workunit.client.0.smithi038.stdout:0/442: getdents d0/d14/d88 0 2024-03-24T00:48:55.348 INFO:tasks.workunit.client.1.smithi088.stdout:1/559: rename d6/d43/d95/d3f/f4c to d6/d43/d95/d5b/d5d/fa9 0 2024-03-24T00:48:55.350 INFO:tasks.workunit.client.0.smithi038.stdout:6/553: fdatasync d1/da/f89 0 2024-03-24T00:48:55.350 INFO:tasks.workunit.client.0.smithi038.stdout:6/554: readlink d1/da/d61/d69/l6d 0 2024-03-24T00:48:55.351 INFO:tasks.workunit.client.1.smithi088.stdout:4/542: dwrite d5/dac/fb8 [0,4194304] 0 2024-03-24T00:48:55.351 INFO:tasks.workunit.client.0.smithi038.stdout:8/498: dwrite d0/d1d/d20/f60 [0,4194304] 0 2024-03-24T00:48:55.351 INFO:tasks.workunit.client.0.smithi038.stdout:8/499: chown d0/d1d/d20/f58 2035 1 2024-03-24T00:48:55.354 INFO:tasks.workunit.client.1.smithi088.stdout:9/577: unlink d1/d10/d29/c4a 0 2024-03-24T00:48:55.354 INFO:tasks.workunit.client.1.smithi088.stdout:1/560: link d6/d43/d95/d5b/d5d/f76 d6/d43/d95/d3f/d53/faa 0 2024-03-24T00:48:55.354 INFO:tasks.workunit.client.1.smithi088.stdout:8/566: dread d9/f32 [0,4194304] 0 2024-03-24T00:48:55.356 INFO:tasks.workunit.client.0.smithi038.stdout:4/464: rename da/d12/d29/d71 to da/d12/d24/d5f/d9a 0 2024-03-24T00:48:55.356 INFO:tasks.workunit.client.0.smithi038.stdout:9/562: mkdir d12/d28/d6c/dbc 0 2024-03-24T00:48:55.356 INFO:tasks.workunit.client.0.smithi038.stdout:4/465: chown da/d38 241905550 1 2024-03-24T00:48:55.359 INFO:tasks.workunit.client.0.smithi038.stdout:0/443: creat d0/d12/d42/d44/f93 x:0 0 0 2024-03-24T00:48:55.359 INFO:tasks.workunit.client.0.smithi038.stdout:0/444: write d0/d12/d42/f61 [5229098,35607] 0 2024-03-24T00:48:55.360 INFO:tasks.workunit.client.1.smithi088.stdout:0/551: dwrite d9/dc/f49 [0,4194304] 0 2024-03-24T00:48:55.364 INFO:tasks.workunit.client.0.smithi038.stdout:0/445: dread - d0/d2d/f65 zero size 2024-03-24T00:48:55.371 INFO:tasks.workunit.client.1.smithi088.stdout:8/567: rename d9/dc/d11/d24/d29/l30 to d9/dc/d11/d24/d43/lc0 0 2024-03-24T00:48:55.376 INFO:tasks.workunit.client.0.smithi038.stdout:1/592: rename da/dd/d2c/d5a/d74/f67 to da/dd/d4a/d8b/fc7 0 2024-03-24T00:48:55.376 INFO:tasks.workunit.client.0.smithi038.stdout:1/593: read - da/dd/d7a/d82/d98/d14/f4d zero size 2024-03-24T00:48:55.376 INFO:tasks.workunit.client.0.smithi038.stdout:1/594: chown da/dd/d2c/d40/f6c 48 1 2024-03-24T00:48:55.376 INFO:tasks.workunit.client.0.smithi038.stdout:1/595: fsync da/dd/d7a/d82/d98/d14/f4d 0 2024-03-24T00:48:55.376 INFO:tasks.workunit.client.0.smithi038.stdout:1/596: chown da/f35 44953267 1 2024-03-24T00:48:55.376 INFO:tasks.workunit.client.0.smithi038.stdout:1/597: creat da/dd/d2c/d41/d60/dae/fc8 x:0 0 0 2024-03-24T00:48:55.382 INFO:tasks.workunit.client.0.smithi038.stdout:4/466: creat da/d12/d37/dd/d15/d4c/d70/f9b x:0 0 0 2024-03-24T00:48:55.384 INFO:tasks.workunit.client.1.smithi088.stdout:4/543: write d5/d13/d18/d6e/d7d/f7e [570547,21094] 0 2024-03-24T00:48:55.386 INFO:tasks.workunit.client.0.smithi038.stdout:9/563: dread d12/d81/fa5 [0,4194304] 0 2024-03-24T00:48:55.387 INFO:tasks.workunit.client.1.smithi088.stdout:8/568: mkdir d9/dc/d11/d24/d29/d36/dc1 0 2024-03-24T00:48:55.391 INFO:tasks.workunit.client.1.smithi088.stdout:7/481: dwrite d3/d31/f6e [0,4194304] 0 2024-03-24T00:48:55.404 INFO:tasks.workunit.client.0.smithi038.stdout:6/555: dread d1/da/d1b/d1d/d3c/f86 [0,4194304] 0 2024-03-24T00:48:55.404 INFO:tasks.workunit.client.0.smithi038.stdout:6/556: creat d1/da/d61/d79/fb0 x:0 0 0 2024-03-24T00:48:55.405 INFO:tasks.workunit.client.1.smithi088.stdout:0/552: dread d9/d26/f61 [0,4194304] 0 2024-03-24T00:48:55.405 INFO:tasks.workunit.client.1.smithi088.stdout:8/569: symlink d9/dc/d11/d24/d29/d73/lc2 0 2024-03-24T00:48:55.414 INFO:tasks.workunit.client.1.smithi088.stdout:1/561: write d6/d43/d95/d58/f51 [3688518,118047] 0 2024-03-24T00:48:55.414 INFO:tasks.workunit.client.1.smithi088.stdout:2/546: dwrite db/f20 [0,4194304] 0 2024-03-24T00:48:55.414 INFO:tasks.workunit.client.0.smithi038.stdout:6/557: mkdir d1/da/d61/da2/db1 0 2024-03-24T00:48:55.416 INFO:tasks.workunit.client.1.smithi088.stdout:9/578: dread d1/d10/d77/d8a/f36 [0,4194304] 0 2024-03-24T00:48:55.417 INFO:tasks.workunit.client.1.smithi088.stdout:0/553: rename d9/d26/d8f/da1/l67 to d9/dc/de/d7e/d91/lb1 0 2024-03-24T00:48:55.417 INFO:tasks.workunit.client.1.smithi088.stdout:0/554: readlink d9/dc/de/d19/d44/d46/d59/l66 0 2024-03-24T00:48:55.423 INFO:tasks.workunit.client.1.smithi088.stdout:0/555: link d9/dc/de/d19/d44/f7d d9/d26/d8f/da1/d62/d9b/db0/fb2 0 2024-03-24T00:48:55.424 INFO:tasks.workunit.client.0.smithi038.stdout:5/506: dwrite d6/d16/d14/f8e [0,4194304] 0 2024-03-24T00:48:55.432 INFO:tasks.workunit.client.0.smithi038.stdout:8/500: dwrite d0/d45/d66/f40 [0,4194304] 0 2024-03-24T00:48:55.432 INFO:tasks.workunit.client.0.smithi038.stdout:8/501: write d0/d15/f63 [1137739,57030] 0 2024-03-24T00:48:55.432 INFO:tasks.workunit.client.0.smithi038.stdout:8/502: write d0/d1d/d20/d3c/f47 [4219888,23282] 0 2024-03-24T00:48:55.433 INFO:tasks.workunit.client.1.smithi088.stdout:9/579: rename d1/d15/d47 to d1/d10/d9b/dab/db8 0 2024-03-24T00:48:55.440 INFO:tasks.workunit.client.1.smithi088.stdout:1/562: dread d6/d43/d95/d58/f48 [0,4194304] 0 2024-03-24T00:48:55.448 INFO:tasks.workunit.client.0.smithi038.stdout:5/507: rename d6/d19/d20/d33/d6f/faa to d6/d19/d20/d70/da3/fab 0 2024-03-24T00:48:55.451 INFO:tasks.workunit.client.1.smithi088.stdout:0/556: rename d9/dc/de/d19/d44/d46/d65/l4b to d9/d26/d8f/da1/d62/d9b/db0/lb3 0 2024-03-24T00:48:55.455 INFO:tasks.workunit.client.1.smithi088.stdout:1/563: getdents d6/d43/d95/d5b/d5d 0 2024-03-24T00:48:55.456 INFO:tasks.workunit.client.0.smithi038.stdout:9/564: dread d12/d55/f9b [0,4194304] 0 2024-03-24T00:48:55.457 INFO:tasks.workunit.client.1.smithi088.stdout:0/557: mkdir d9/d26/d8f/da1/d62/d9b/db0/db4 0 2024-03-24T00:48:55.458 INFO:tasks.workunit.client.0.smithi038.stdout:5/508: creat d6/d19/da1/fac x:0 0 0 2024-03-24T00:48:55.459 INFO:tasks.workunit.client.1.smithi088.stdout:7/482: dwrite d3/f35 [4194304,4194304] 0 2024-03-24T00:48:55.459 INFO:tasks.workunit.client.1.smithi088.stdout:7/483: truncate d3/d32/d34/f72 309059 0 2024-03-24T00:48:55.461 INFO:tasks.workunit.client.1.smithi088.stdout:0/558: creat d9/dc/de/d19/d44/d46/d65/fb5 x:0 0 0 2024-03-24T00:48:55.461 INFO:tasks.workunit.client.1.smithi088.stdout:0/559: truncate d9/dc/de/d19/d44/f7d 937610 0 2024-03-24T00:48:55.462 INFO:tasks.workunit.client.0.smithi038.stdout:4/467: dwrite da/d12/d37/dd/d15/d4c/d70/f9b [0,4194304] 0 2024-03-24T00:48:55.462 INFO:tasks.workunit.client.0.smithi038.stdout:4/468: readlink da/d12/d37/dd/l20 0 2024-03-24T00:48:55.466 INFO:tasks.workunit.client.0.smithi038.stdout:1/598: dwrite da/dd/d22/f4f [0,4194304] 0 2024-03-24T00:48:55.466 INFO:tasks.workunit.client.0.smithi038.stdout:9/565: creat d12/d28/d6c/fbd x:0 0 0 2024-03-24T00:48:55.470 INFO:tasks.workunit.client.1.smithi088.stdout:8/570: write d9/dc/d11/d24/d43/d4a/f55 [956043,24865] 0 2024-03-24T00:48:55.470 INFO:tasks.workunit.client.1.smithi088.stdout:8/571: write d9/dc/d11/d41/d56/f59 [574372,49068] 0 2024-03-24T00:48:55.471 INFO:tasks.workunit.client.1.smithi088.stdout:7/484: creat d3/d32/d3f/d4b/d73/f92 x:0 0 0 2024-03-24T00:48:55.472 INFO:tasks.workunit.client.1.smithi088.stdout:5/567: sync 2024-03-24T00:48:55.472 INFO:tasks.workunit.client.1.smithi088.stdout:7/485: creat d3/d32/d3f/d4b/d73/f93 x:0 0 0 2024-03-24T00:48:55.476 INFO:tasks.workunit.client.1.smithi088.stdout:6/570: sync 2024-03-24T00:48:55.476 INFO:tasks.workunit.client.0.smithi038.stdout:6/558: dwrite d1/d37/f90 [0,4194304] 0 2024-03-24T00:48:55.476 INFO:tasks.workunit.client.0.smithi038.stdout:6/559: creat d1/da/d1b/d25/d49/d4e/d51/fb2 x:0 0 0 2024-03-24T00:48:55.476 INFO:tasks.workunit.client.0.smithi038.stdout:4/469: dread da/d12/d37/dd/d15/d4c/f53 [0,4194304] 0 2024-03-24T00:48:55.478 INFO:tasks.workunit.client.1.smithi088.stdout:0/560: link d9/dc/de/d19/l42 d9/dc/de/d7e/lb6 0 2024-03-24T00:48:55.478 INFO:tasks.workunit.client.1.smithi088.stdout:3/510: sync 2024-03-24T00:48:55.483 INFO:tasks.workunit.client.1.smithi088.stdout:2/547: dwrite db/d21/d35/f60 [0,4194304] 0 2024-03-24T00:48:55.486 INFO:tasks.workunit.client.0.smithi038.stdout:9/566: unlink d12/d1a/d5f/lb0 0 2024-03-24T00:48:55.486 INFO:tasks.workunit.client.0.smithi038.stdout:9/567: stat f4 0 2024-03-24T00:48:55.487 INFO:tasks.workunit.client.1.smithi088.stdout:5/568: rename d1/d10/d4f/c7b to d1/dd/d6f/da1/d3f/d88/d8b/d94/cb3 0 2024-03-24T00:48:55.488 INFO:tasks.workunit.client.1.smithi088.stdout:6/571: symlink db/d1d/d50/d90/d56/lab 0 2024-03-24T00:48:55.488 INFO:tasks.workunit.client.1.smithi088.stdout:6/572: write db/d3d/d83/d72/f76 [396846,122674] 0 2024-03-24T00:48:55.491 INFO:tasks.workunit.client.1.smithi088.stdout:7/486: getdents d3/d31 0 2024-03-24T00:48:55.496 INFO:tasks.workunit.client.1.smithi088.stdout:5/569: symlink d1/d10/d1b/lb4 0 2024-03-24T00:48:55.498 INFO:tasks.workunit.client.1.smithi088.stdout:6/573: truncate db/d1d/d50/f77 1145577 0 2024-03-24T00:48:55.498 INFO:tasks.workunit.client.1.smithi088.stdout:5/570: creat d1/d10/d1b/d2f/d57/d9e/fb5 x:0 0 0 2024-03-24T00:48:55.499 INFO:tasks.workunit.client.1.smithi088.stdout:1/564: dread d6/f68 [0,4194304] 0 2024-03-24T00:48:55.499 INFO:tasks.workunit.client.1.smithi088.stdout:1/565: chown d6/c5a 24243 1 2024-03-24T00:48:55.501 INFO:tasks.workunit.client.1.smithi088.stdout:6/574: mknod db/dd/cac 0 2024-03-24T00:48:55.503 INFO:tasks.workunit.client.1.smithi088.stdout:5/571: mkdir d1/d10/db6 0 2024-03-24T00:48:55.506 INFO:tasks.workunit.client.1.smithi088.stdout:1/566: creat d6/fab x:0 0 0 2024-03-24T00:48:55.506 INFO:tasks.workunit.client.1.smithi088.stdout:1/567: write f0 [855733,79844] 0 2024-03-24T00:48:55.507 INFO:tasks.workunit.client.1.smithi088.stdout:6/575: mkdir db/dd/d89/dad 0 2024-03-24T00:48:55.508 INFO:tasks.workunit.client.0.smithi038.stdout:6/560: dread d1/da/d1b/d1d/d3c/f86 [0,4194304] 0 2024-03-24T00:48:55.508 INFO:tasks.workunit.client.0.smithi038.stdout:6/561: creat d1/da/d1b/fb3 x:0 0 0 2024-03-24T00:48:55.508 INFO:tasks.workunit.client.0.smithi038.stdout:6/562: chown d1/f6 61 1 2024-03-24T00:48:55.508 INFO:tasks.workunit.client.0.smithi038.stdout:6/563: stat d1/da/d61/d79 0 2024-03-24T00:48:55.509 INFO:tasks.workunit.client.1.smithi088.stdout:6/576: mknod db/dd/d61/d6d/d74/d88/da1/cae 0 2024-03-24T00:48:55.513 INFO:tasks.workunit.client.1.smithi088.stdout:3/511: write d3/d6/d27/d3c/f59 [6951140,118164] 0 2024-03-24T00:48:55.513 INFO:tasks.workunit.client.0.smithi038.stdout:4/470: dread da/d38/f76 [0,4194304] 0 2024-03-24T00:48:55.519 INFO:tasks.workunit.client.0.smithi038.stdout:9/568: read d12/d1a/d80/f97 [1817416,81706] 0 2024-03-24T00:48:55.519 INFO:tasks.workunit.client.1.smithi088.stdout:3/512: rename d3/de/c60 to d3/d6/d58/da4/ca7 0 2024-03-24T00:48:55.519 INFO:tasks.workunit.client.1.smithi088.stdout:3/513: readlink d3/d1a/d1e/d2f/d45/l7d 0 2024-03-24T00:48:55.519 INFO:tasks.workunit.client.1.smithi088.stdout:3/514: write d3/d6/d19/d2a/f53 [794488,102143] 0 2024-03-24T00:48:55.521 INFO:tasks.workunit.client.1.smithi088.stdout:4/544: dwrite d5/d13/d18/d6e/d7d/fbb [0,4194304] 0 2024-03-24T00:48:55.521 INFO:tasks.workunit.client.1.smithi088.stdout:4/545: read - d5/d16/d94/fb2 zero size 2024-03-24T00:48:55.526 INFO:tasks.workunit.client.0.smithi038.stdout:7/503: sync 2024-03-24T00:48:55.527 INFO:tasks.workunit.client.0.smithi038.stdout:7/504: write d2/d9/da/d23/d2b/d5b/d72/d92/d75/f85 [6092724,52319] 0 2024-03-24T00:48:55.528 INFO:tasks.workunit.client.1.smithi088.stdout:3/515: unlink d3/d6/d27/d56/d6e/f8a 0 2024-03-24T00:48:55.531 INFO:tasks.workunit.client.0.smithi038.stdout:6/564: creat d1/da/d1b/d1d/d3c/fb4 x:0 0 0 2024-03-24T00:48:55.531 INFO:tasks.workunit.client.0.smithi038.stdout:6/565: write d1/f5 [3486385,122988] 0 2024-03-24T00:48:55.537 INFO:tasks.workunit.client.1.smithi088.stdout:4/546: symlink d5/d7/d15/d74/lbc 0 2024-03-24T00:48:55.539 INFO:tasks.workunit.client.1.smithi088.stdout:6/577: dread db/d1d/d23/d3c/d43/f39 [0,4194304] 0 2024-03-24T00:48:55.539 INFO:tasks.workunit.client.1.smithi088.stdout:3/516: creat d3/d6/d27/d3c/fa8 x:0 0 0 2024-03-24T00:48:55.539 INFO:tasks.workunit.client.1.smithi088.stdout:0/561: dwrite d9/d26/d8f/da1/d62/f6a [0,4194304] 0 2024-03-24T00:48:55.540 INFO:tasks.workunit.client.0.smithi038.stdout:9/569: creat d12/d28/d71/fbe x:0 0 0 2024-03-24T00:48:55.543 INFO:tasks.workunit.client.1.smithi088.stdout:4/547: rmdir d5/d7/dd/da9 0 2024-03-24T00:48:55.546 INFO:tasks.workunit.client.0.smithi038.stdout:6/566: symlink d1/d37/d84/lb5 0 2024-03-24T00:48:55.546 INFO:tasks.workunit.client.0.smithi038.stdout:7/505: truncate d2/d9/da/d23/d2b/d5b/d72/d92/d98/f9f 3930062 0 2024-03-24T00:48:55.546 INFO:tasks.workunit.client.0.smithi038.stdout:6/567: readlink d1/da/d61/d69/l9d 0 2024-03-24T00:48:55.546 INFO:tasks.workunit.client.0.smithi038.stdout:4/471: getdents da/d12/d24/d5f 0 2024-03-24T00:48:55.546 INFO:tasks.workunit.client.0.smithi038.stdout:4/472: stat da/d12/d37/dd/d15/d6d 0 2024-03-24T00:48:55.546 INFO:tasks.workunit.client.0.smithi038.stdout:4/473: dread - da/d12/d24/d5f/f97 zero size 2024-03-24T00:48:55.546 INFO:tasks.workunit.client.0.smithi038.stdout:4/474: truncate da/d12/d37/dd/d15/d6d/f88 19783 0 2024-03-24T00:48:55.548 INFO:tasks.workunit.client.1.smithi088.stdout:0/562: mknod d9/dc/de/d19/cb7 0 2024-03-24T00:48:55.549 INFO:tasks.workunit.client.1.smithi088.stdout:3/517: dread d3/d6/f78 [0,4194304] 0 2024-03-24T00:48:55.549 INFO:tasks.workunit.client.0.smithi038.stdout:7/506: rename d2/d9/l22 to d2/d9/da/d23/lc0 0 2024-03-24T00:48:55.553 INFO:tasks.workunit.client.0.smithi038.stdout:6/568: truncate d1/f8 2006056 0 2024-03-24T00:48:55.554 INFO:tasks.workunit.client.1.smithi088.stdout:3/518: unlink d3/d6/f78 0 2024-03-24T00:48:55.559 INFO:tasks.workunit.client.0.smithi038.stdout:7/507: link d2/d2c/f62 d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/db2/fc1 0 2024-03-24T00:48:55.559 INFO:tasks.workunit.client.0.smithi038.stdout:7/508: chown d2/fb9 3580 1 2024-03-24T00:48:55.560 INFO:tasks.workunit.client.1.smithi088.stdout:0/563: dread d9/dc/de/d19/d8d/f98 [0,4194304] 0 2024-03-24T00:48:55.563 INFO:tasks.workunit.client.1.smithi088.stdout:2/548: dwrite db/d21/d35/d49/fac [0,4194304] 0 2024-03-24T00:48:55.563 INFO:tasks.workunit.client.1.smithi088.stdout:2/549: readlink db/dd/d96/l58 0 2024-03-24T00:48:55.569 INFO:tasks.workunit.client.1.smithi088.stdout:8/572: dwrite d9/dc/d11/d24/d29/f51 [0,4194304] 0 2024-03-24T00:48:55.569 INFO:tasks.workunit.client.1.smithi088.stdout:8/573: dread - d9/dc/d11/d41/d54/d5d/db3/fb2 zero size 2024-03-24T00:48:55.571 INFO:tasks.workunit.client.0.smithi038.stdout:9/570: dread d12/d55/fb5 [0,4194304] 0 2024-03-24T00:48:55.572 INFO:tasks.workunit.client.0.smithi038.stdout:9/571: dread - d12/d28/f82 zero size 2024-03-24T00:48:55.573 INFO:tasks.workunit.client.0.smithi038.stdout:9/572: creat d12/d28/da8/fbf x:0 0 0 2024-03-24T00:48:55.574 INFO:tasks.workunit.client.1.smithi088.stdout:3/519: creat d3/de/d6f/fa9 x:0 0 0 2024-03-24T00:48:55.574 INFO:tasks.workunit.client.1.smithi088.stdout:4/548: dread d5/d13/d18/f9e [0,4194304] 0 2024-03-24T00:48:55.574 INFO:tasks.workunit.client.1.smithi088.stdout:3/520: chown d3/d6/d27/d3c/f42 125482 1 2024-03-24T00:48:55.574 INFO:tasks.workunit.client.1.smithi088.stdout:3/521: write d3/d1a/f5a [783077,8415] 0 2024-03-24T00:48:55.575 INFO:tasks.workunit.client.0.smithi038.stdout:4/475: write da/d12/d24/f26 [3375661,45639] 0 2024-03-24T00:48:55.577 INFO:tasks.workunit.client.0.smithi038.stdout:5/509: dwrite d6/d19/d20/d42/d43/d50/f81 [0,4194304] 0 2024-03-24T00:48:55.577 INFO:tasks.workunit.client.0.smithi038.stdout:5/510: chown d6/d19/l57 0 1 2024-03-24T00:48:55.582 INFO:tasks.workunit.client.1.smithi088.stdout:0/564: mknod d9/dc/de/d19/d44/d46/cb8 0 2024-03-24T00:48:55.583 INFO:tasks.workunit.client.1.smithi088.stdout:5/572: dread d1/d19/f30 [0,4194304] 0 2024-03-24T00:48:55.595 INFO:tasks.workunit.client.0.smithi038.stdout:4/476: unlink da/d12/d37/dd/d15/d4c/f5a 0 2024-03-24T00:48:55.595 INFO:tasks.workunit.client.1.smithi088.stdout:2/550: stat db/dd/c24 0 2024-03-24T00:48:55.595 INFO:tasks.workunit.client.1.smithi088.stdout:2/551: readlink db/dd/d96/l7a 0 2024-03-24T00:48:55.598 INFO:tasks.workunit.client.1.smithi088.stdout:8/574: creat d9/dc/fc3 x:0 0 0 2024-03-24T00:48:55.601 INFO:tasks.workunit.client.1.smithi088.stdout:7/487: dwrite d3/f28 [0,4194304] 0 2024-03-24T00:48:55.609 INFO:tasks.workunit.client.0.smithi038.stdout:6/569: write d1/da/d1b/f26 [2718240,52211] 0 2024-03-24T00:48:55.609 INFO:tasks.workunit.client.0.smithi038.stdout:6/570: creat d1/da/d1b/da7/fb6 x:0 0 0 2024-03-24T00:48:55.609 INFO:tasks.workunit.client.0.smithi038.stdout:6/571: fsync d1/da/d1b/d2f/f7f 0 2024-03-24T00:48:55.610 INFO:tasks.workunit.client.1.smithi088.stdout:3/522: link d3/de/d6f/f75 d3/d6/d27/d3c/faa 0 2024-03-24T00:48:55.618 INFO:tasks.workunit.client.1.smithi088.stdout:0/565: symlink d9/d26/d8f/da1/d62/lb9 0 2024-03-24T00:48:55.619 INFO:tasks.workunit.client.0.smithi038.stdout:4/477: rename da/d12/d37/dd to da/d12/d37/dd/d15/d6d/d9c 22 2024-03-24T00:48:55.620 INFO:tasks.workunit.client.1.smithi088.stdout:5/573: mknod d1/d10/d1b/d2f/d93/cb7 0 2024-03-24T00:48:55.630 INFO:tasks.workunit.client.0.smithi038.stdout:6/572: rmdir d1/da/d1b/d25/d49/d4e 39 2024-03-24T00:48:55.639 INFO:tasks.workunit.client.1.smithi088.stdout:8/575: mkdir d9/dc/d11/d24/d29/d36/dc1/dc4 0 2024-03-24T00:48:55.641 INFO:tasks.workunit.client.0.smithi038.stdout:4/478: truncate da/d12/d24/f30 1709580 0 2024-03-24T00:48:55.644 INFO:tasks.workunit.client.1.smithi088.stdout:4/549: rename d5/d7/d72/l45 to d5/lbd 0 2024-03-24T00:48:55.646 INFO:tasks.workunit.client.1.smithi088.stdout:7/488: mknod d3/d32/d3f/d4b/d73/c94 0 2024-03-24T00:48:55.646 INFO:tasks.workunit.client.1.smithi088.stdout:7/489: creat d3/d32/d3f/d4b/d74/f95 x:0 0 0 2024-03-24T00:48:55.649 INFO:tasks.workunit.client.1.smithi088.stdout:3/523: symlink d3/de/d6f/d85/d49/d70/lab 0 2024-03-24T00:48:55.651 INFO:tasks.workunit.client.0.smithi038.stdout:6/573: rmdir d1/da/d1b/d1d/d3c 39 2024-03-24T00:48:55.652 INFO:tasks.workunit.client.1.smithi088.stdout:3/524: write d3/d6/d27/d3c/f39 [1002167,83270] 0 2024-03-24T00:48:55.656 INFO:tasks.workunit.client.1.smithi088.stdout:0/566: creat d9/d26/d8f/da1/fba x:0 0 0 2024-03-24T00:48:55.657 INFO:tasks.workunit.client.1.smithi088.stdout:0/567: write d9/dc/de/d19/d44/d46/d65/fb5 [536453,107939] 0 2024-03-24T00:48:55.658 INFO:tasks.workunit.client.0.smithi038.stdout:4/479: write da/d38/f61 [2649800,75705] 0 2024-03-24T00:48:55.659 INFO:tasks.workunit.client.1.smithi088.stdout:5/574: truncate d1/dd/d6f/da1/f77 605391 0 2024-03-24T00:48:55.659 INFO:tasks.workunit.client.1.smithi088.stdout:5/575: chown d1/d10/d1b/d2f/d57/c5a 428151 1 2024-03-24T00:48:55.663 INFO:tasks.workunit.client.1.smithi088.stdout:8/576: creat d9/dc/d11/d24/d29/d5b/db9/fc5 x:0 0 0 2024-03-24T00:48:55.663 INFO:tasks.workunit.client.1.smithi088.stdout:8/577: truncate d9/dc/d11/d24/d29/d5b/db9/fc5 21641 0 2024-03-24T00:48:55.670 INFO:tasks.workunit.client.0.smithi038.stdout:4/480: truncate da/d12/f3f 2907347 0 2024-03-24T00:48:55.670 INFO:tasks.workunit.client.0.smithi038.stdout:4/481: dread - da/d12/d37/f99 zero size 2024-03-24T00:48:55.672 INFO:tasks.workunit.client.1.smithi088.stdout:4/550: symlink d5/d13/d63/lbe 0 2024-03-24T00:48:55.674 INFO:tasks.workunit.client.0.smithi038.stdout:4/482: symlink da/d12/d37/dd/d15/d4c/d70/l9d 0 2024-03-24T00:48:55.674 INFO:tasks.workunit.client.0.smithi038.stdout:4/483: write da/d12/f4e [8564882,66694] 0 2024-03-24T00:48:55.678 INFO:tasks.workunit.client.1.smithi088.stdout:7/490: creat d3/d32/d34/f96 x:0 0 0 2024-03-24T00:48:55.679 INFO:tasks.workunit.client.1.smithi088.stdout:0/568: creat d9/d26/d8f/da1/d62/d9b/fbb x:0 0 0 2024-03-24T00:48:55.686 INFO:tasks.workunit.client.1.smithi088.stdout:7/491: creat d3/d32/d3f/d4b/d74/d79/f97 x:0 0 0 2024-03-24T00:48:55.690 INFO:tasks.workunit.client.1.smithi088.stdout:3/525: rename d3/d6/d27/l9e to d3/lac 0 2024-03-24T00:48:55.690 INFO:tasks.workunit.client.1.smithi088.stdout:7/492: dread - d3/d32/d3f/d4b/d74/f95 zero size 2024-03-24T00:48:55.692 INFO:tasks.workunit.client.0.smithi038.stdout:7/509: dwrite d2/d9/f54 [4194304,4194304] 0 2024-03-24T00:48:55.693 INFO:tasks.workunit.client.1.smithi088.stdout:4/551: rename d5/ca8 to d5/d16/d1a/d22/cbf 0 2024-03-24T00:48:55.697 INFO:tasks.workunit.client.1.smithi088.stdout:0/569: rename d9/dc/de/d19/c7b to d9/d26/d8f/da1/cbc 0 2024-03-24T00:48:55.697 INFO:tasks.workunit.client.1.smithi088.stdout:0/570: fdatasync d9/dc/de/d19/f7f 0 2024-03-24T00:48:55.699 INFO:tasks.workunit.client.0.smithi038.stdout:9/573: dread d12/d81/d84/f8e [0,4194304] 0 2024-03-24T00:48:55.706 INFO:tasks.workunit.client.1.smithi088.stdout:9/580: sync 2024-03-24T00:48:55.715 INFO:tasks.workunit.client.1.smithi088.stdout:3/526: dread d3/d6/f2b [0,4194304] 0 2024-03-24T00:48:55.716 INFO:tasks.workunit.client.1.smithi088.stdout:7/493: getdents d3/d32/d3f/d4b/d74 0 2024-03-24T00:48:55.722 INFO:tasks.workunit.client.1.smithi088.stdout:3/527: getdents d3/d6/d27/d56 0 2024-03-24T00:48:55.722 INFO:tasks.workunit.client.1.smithi088.stdout:3/528: write d3/de/d6f/d89/fa2 [812844,94749] 0 2024-03-24T00:48:55.724 INFO:tasks.workunit.client.0.smithi038.stdout:7/510: dread d2/d9/da/d23/d2b/d5b/d72/d92/f29 [0,4194304] 0 2024-03-24T00:48:55.724 INFO:tasks.workunit.client.0.smithi038.stdout:7/511: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/f9e 0 2024-03-24T00:48:55.725 INFO:tasks.workunit.client.1.smithi088.stdout:1/568: sync 2024-03-24T00:48:55.725 INFO:tasks.workunit.client.1.smithi088.stdout:6/578: sync 2024-03-24T00:48:55.727 INFO:tasks.workunit.client.1.smithi088.stdout:1/569: write d6/d23/f73 [4322169,119938] 0 2024-03-24T00:48:55.733 INFO:tasks.workunit.client.1.smithi088.stdout:3/529: rename d3/d1a/f5a to d3/d32/fad 0 2024-03-24T00:48:55.735 INFO:tasks.workunit.client.0.smithi038.stdout:6/574: dread d1/da/d1b/d2f/f62 [0,4194304] 0 2024-03-24T00:48:55.737 INFO:tasks.workunit.client.1.smithi088.stdout:0/571: read d9/d26/f38 [222890,61031] 0 2024-03-24T00:48:55.739 INFO:tasks.workunit.client.1.smithi088.stdout:7/494: symlink d3/l98 0 2024-03-24T00:48:55.742 INFO:tasks.workunit.client.0.smithi038.stdout:7/512: mknod d2/d9/da/dc/d8a/cc2 0 2024-03-24T00:48:55.744 INFO:tasks.workunit.client.1.smithi088.stdout:6/579: truncate db/d1d/f48 1376174 0 2024-03-24T00:48:55.744 INFO:tasks.workunit.client.1.smithi088.stdout:6/580: chown db/f13 36835265 1 2024-03-24T00:48:55.748 INFO:tasks.workunit.client.1.smithi088.stdout:6/581: unlink db/c42 0 2024-03-24T00:48:55.752 INFO:tasks.workunit.client.1.smithi088.stdout:3/530: dread d3/de/f29 [0,4194304] 0 2024-03-24T00:48:55.756 INFO:tasks.workunit.client.1.smithi088.stdout:7/495: dread d3/f27 [0,4194304] 0 2024-03-24T00:48:55.763 INFO:tasks.workunit.client.1.smithi088.stdout:7/496: rename d3/f2f to d3/d32/d3f/d4b/f99 0 2024-03-24T00:48:55.766 INFO:tasks.workunit.client.1.smithi088.stdout:7/497: symlink d3/d32/d34/l9a 0 2024-03-24T00:48:55.769 INFO:tasks.workunit.client.1.smithi088.stdout:8/578: dwrite d9/d6b/f6c [0,4194304] 0 2024-03-24T00:48:55.771 INFO:tasks.workunit.client.1.smithi088.stdout:7/498: creat d3/d31/f9b x:0 0 0 2024-03-24T00:48:55.771 INFO:tasks.workunit.client.1.smithi088.stdout:8/579: link d9/f3e d9/dc/d11/d24/d29/d39/fc6 0 2024-03-24T00:48:55.774 INFO:tasks.workunit.client.1.smithi088.stdout:8/580: mknod d9/dc/d11/d80/db0/cc7 0 2024-03-24T00:48:55.776 INFO:tasks.workunit.client.0.smithi038.stdout:6/575: dread d1/da/d73/d75/f9f [0,4194304] 0 2024-03-24T00:48:55.776 INFO:tasks.workunit.client.0.smithi038.stdout:6/576: dread - d1/da/d1b/d25/f8e zero size 2024-03-24T00:48:55.777 INFO:tasks.workunit.client.0.smithi038.stdout:6/577: fsync d1/d37/f8b 0 2024-03-24T00:48:55.777 INFO:tasks.workunit.client.0.smithi038.stdout:6/578: fdatasync d1/d37/f8b 0 2024-03-24T00:48:55.778 INFO:tasks.workunit.client.1.smithi088.stdout:8/581: creat d9/d6b/fc8 x:0 0 0 2024-03-24T00:48:55.784 INFO:tasks.workunit.client.1.smithi088.stdout:8/582: write d9/dc/d11/d24/d29/d5b/f1e [1913786,35273] 0 2024-03-24T00:48:55.784 INFO:tasks.workunit.client.1.smithi088.stdout:4/552: dwrite d5/fb5 [0,4194304] 0 2024-03-24T00:48:55.784 INFO:tasks.workunit.client.1.smithi088.stdout:4/553: write d5/d7/dd/f65 [784196,122396] 0 2024-03-24T00:48:55.784 INFO:tasks.workunit.client.1.smithi088.stdout:4/554: chown d5/d13/d18/d24/db3 33526 1 2024-03-24T00:48:55.786 INFO:tasks.workunit.client.0.smithi038.stdout:4/484: dwrite da/f1e [0,4194304] 0 2024-03-24T00:48:55.786 INFO:tasks.workunit.client.0.smithi038.stdout:4/485: truncate da/d12/d37/dd/d15/d4c/f6a 1456106 0 2024-03-24T00:48:55.786 INFO:tasks.workunit.client.0.smithi038.stdout:4/486: truncate da/d12/d24/d5f/f6f 703081 0 2024-03-24T00:48:55.789 INFO:tasks.workunit.client.1.smithi088.stdout:7/499: dread d3/f17 [0,4194304] 0 2024-03-24T00:48:55.793 INFO:tasks.workunit.client.0.smithi038.stdout:4/487: symlink da/d12/d24/d68/d6c/l9e 0 2024-03-24T00:48:55.796 INFO:tasks.workunit.client.1.smithi088.stdout:5/576: dwrite d1/d10/f52 [0,4194304] 0 2024-03-24T00:48:55.796 INFO:tasks.workunit.client.1.smithi088.stdout:7/500: unlink d3/f5f 0 2024-03-24T00:48:55.796 INFO:tasks.workunit.client.0.smithi038.stdout:9/574: dwrite d12/d28/d3a/f4e [0,4194304] 0 2024-03-24T00:48:55.797 INFO:tasks.workunit.client.0.smithi038.stdout:7/513: dread d2/d2c/f4d [0,4194304] 0 2024-03-24T00:48:55.797 INFO:tasks.workunit.client.0.smithi038.stdout:4/488: mknod da/d12/d24/d5f/d8a/c9f 0 2024-03-24T00:48:55.798 INFO:tasks.workunit.client.0.smithi038.stdout:4/489: write da/d12/d29/d5b/d73/f94 [477998,125947] 0 2024-03-24T00:48:55.804 INFO:tasks.workunit.client.1.smithi088.stdout:4/555: dread f4 [0,4194304] 0 2024-03-24T00:48:55.804 INFO:tasks.workunit.client.1.smithi088.stdout:4/556: chown d5/d7/dd/f75 838961 1 2024-03-24T00:48:55.805 INFO:tasks.workunit.client.0.smithi038.stdout:9/575: mkdir d12/d41/dc0 0 2024-03-24T00:48:55.805 INFO:tasks.workunit.client.0.smithi038.stdout:9/576: stat d12/d28/d6c/dbc 0 2024-03-24T00:48:55.806 INFO:tasks.workunit.client.0.smithi038.stdout:7/514: getdents d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4 0 2024-03-24T00:48:55.808 INFO:tasks.workunit.client.0.smithi038.stdout:9/577: truncate d12/d41/f4f 2827909 0 2024-03-24T00:48:55.809 INFO:tasks.workunit.client.0.smithi038.stdout:5/511: dwrite d6/d1e/f63 [4194304,4194304] 0 2024-03-24T00:48:55.809 INFO:tasks.workunit.client.0.smithi038.stdout:5/512: truncate d6/d1e/f9f 696932 0 2024-03-24T00:48:55.809 INFO:tasks.workunit.client.0.smithi038.stdout:5/513: fdatasync d6/d19/d20/d42/f91 0 2024-03-24T00:48:55.817 INFO:tasks.workunit.client.0.smithi038.stdout:9/578: symlink d12/d28/d9c/lc1 0 2024-03-24T00:48:55.817 INFO:tasks.workunit.client.1.smithi088.stdout:9/581: dwrite d1/d10/f34 [0,4194304] 0 2024-03-24T00:48:55.830 INFO:tasks.workunit.client.1.smithi088.stdout:9/582: mknod d1/d10/d77/d8a/cb9 0 2024-03-24T00:48:55.831 INFO:tasks.workunit.client.0.smithi038.stdout:4/490: dread da/d12/d29/d5b/d73/f94 [0,4194304] 0 2024-03-24T00:48:55.833 INFO:tasks.workunit.client.1.smithi088.stdout:3/531: dwrite d3/d6/d19/d2a/f71 [0,4194304] 0 2024-03-24T00:48:55.836 INFO:tasks.workunit.client.1.smithi088.stdout:0/572: dwrite d9/dc/de/f5c [4194304,4194304] 0 2024-03-24T00:48:55.836 INFO:tasks.workunit.client.1.smithi088.stdout:9/583: link d1/d10/d77/d8a/cb9 d1/d10/d2c/cba 0 2024-03-24T00:48:55.836 INFO:tasks.workunit.client.1.smithi088.stdout:0/573: chown d9/d26 18338012 1 2024-03-24T00:48:55.836 INFO:tasks.workunit.client.1.smithi088.stdout:0/574: truncate d9/d26/f5a 218918 0 2024-03-24T00:48:55.836 INFO:tasks.workunit.client.1.smithi088.stdout:0/575: chown d9/d26/d8f/da1/fba 266 1 2024-03-24T00:48:55.838 INFO:tasks.workunit.client.0.smithi038.stdout:6/579: dwrite d1/da/d61/da2/faf [0,4194304] 0 2024-03-24T00:48:55.841 INFO:tasks.workunit.client.1.smithi088.stdout:3/532: unlink d3/d6/c55 0 2024-03-24T00:48:55.847 INFO:tasks.workunit.client.1.smithi088.stdout:9/584: dread d1/d10/d29/fa6 [0,4194304] 0 2024-03-24T00:48:55.847 INFO:tasks.workunit.client.1.smithi088.stdout:9/585: chown d1/d10/d9b/dab/db8/l4b 1069 1 2024-03-24T00:48:55.847 INFO:tasks.workunit.client.1.smithi088.stdout:4/557: write d5/d7/f14 [5250895,62228] 0 2024-03-24T00:48:55.849 INFO:tasks.workunit.client.1.smithi088.stdout:7/501: dread d3/d32/d3f/f6f [0,4194304] 0 2024-03-24T00:48:55.852 INFO:tasks.workunit.client.1.smithi088.stdout:0/576: symlink d9/dc/de/d7e/lbd 0 2024-03-24T00:48:55.860 INFO:tasks.workunit.client.1.smithi088.stdout:4/558: mknod d5/d13/d18/d24/db3/cc0 0 2024-03-24T00:48:55.860 INFO:tasks.workunit.client.1.smithi088.stdout:5/577: dwrite d1/dd/f54 [8388608,4194304] 0 2024-03-24T00:48:55.860 INFO:tasks.workunit.client.1.smithi088.stdout:3/533: dread d3/d6/d27/d3c/faa [0,4194304] 0 2024-03-24T00:48:55.872 INFO:tasks.workunit.client.1.smithi088.stdout:0/577: mknod d9/d26/d8f/cbe 0 2024-03-24T00:48:55.872 INFO:tasks.workunit.client.1.smithi088.stdout:0/578: fdatasync d9/d26/d8f/da1/d62/f6d 0 2024-03-24T00:48:55.874 INFO:tasks.workunit.client.0.smithi038.stdout:3/529: sync 2024-03-24T00:48:55.874 INFO:tasks.workunit.client.0.smithi038.stdout:3/530: dread - d3/d9/dc/df/f6b zero size 2024-03-24T00:48:55.874 INFO:tasks.workunit.client.0.smithi038.stdout:3/531: fsync d3/d9/dc/df/d14/d19/f42 0 2024-03-24T00:48:55.874 INFO:tasks.workunit.client.0.smithi038.stdout:3/532: readlink d3/d9/dc/df/l83 0 2024-03-24T00:48:55.874 INFO:tasks.workunit.client.1.smithi088.stdout:1/570: dwrite d6/d23/f73 [4194304,4194304] 0 2024-03-24T00:48:55.874 INFO:tasks.workunit.client.1.smithi088.stdout:1/571: stat d6/c71 0 2024-03-24T00:48:55.878 INFO:tasks.workunit.client.1.smithi088.stdout:8/583: dwrite d9/dc/d11/d24/d29/d39/d84/f6e [0,4194304] 0 2024-03-24T00:48:55.883 INFO:tasks.workunit.client.1.smithi088.stdout:2/552: sync 2024-03-24T00:48:55.883 INFO:tasks.workunit.client.0.smithi038.stdout:8/503: sync 2024-03-24T00:48:55.883 INFO:tasks.workunit.client.0.smithi038.stdout:8/504: write d0/d15/d2c/fa2 [580461,99364] 0 2024-03-24T00:48:55.883 INFO:tasks.workunit.client.0.smithi038.stdout:2/498: sync 2024-03-24T00:48:55.883 INFO:tasks.workunit.client.0.smithi038.stdout:2/499: fdatasync d4/d27/d2f/d30/d3e/d9b/f9c 0 2024-03-24T00:48:55.883 INFO:tasks.workunit.client.0.smithi038.stdout:0/446: sync 2024-03-24T00:48:55.885 INFO:tasks.workunit.client.0.smithi038.stdout:9/579: dread d12/d28/f36 [0,4194304] 0 2024-03-24T00:48:55.887 INFO:tasks.workunit.client.1.smithi088.stdout:4/559: creat d5/d7/d72/fc1 x:0 0 0 2024-03-24T00:48:55.888 INFO:tasks.workunit.client.1.smithi088.stdout:5/578: rename d1/d10/c24 to d1/d10/d1b/d2f/cb8 0 2024-03-24T00:48:55.889 INFO:tasks.workunit.client.1.smithi088.stdout:0/579: write d9/dc/de/d19/d44/f6e [3293306,24081] 0 2024-03-24T00:48:55.893 INFO:tasks.workunit.client.1.smithi088.stdout:1/572: dread d6/d43/d95/d5b/d5e/f64 [0,4194304] 0 2024-03-24T00:48:55.896 INFO:tasks.workunit.client.0.smithi038.stdout:6/580: creat d1/da/d1b/d25/d49/d72/fb7 x:0 0 0 2024-03-24T00:48:55.897 INFO:tasks.workunit.client.0.smithi038.stdout:8/505: creat d0/d45/d66/da6/fae x:0 0 0 2024-03-24T00:48:55.897 INFO:tasks.workunit.client.0.smithi038.stdout:3/533: unlink d3/d9/dc/df/d14/d19/l4c 0 2024-03-24T00:48:55.901 INFO:tasks.workunit.client.1.smithi088.stdout:8/584: creat d9/dc/d11/d24/d29/d39/fc9 x:0 0 0 2024-03-24T00:48:55.903 INFO:tasks.workunit.client.1.smithi088.stdout:9/586: dwrite d1/d10/d77/d8a/f94 [0,4194304] 0 2024-03-24T00:48:55.907 INFO:tasks.workunit.client.1.smithi088.stdout:2/553: symlink db/lb1 0 2024-03-24T00:48:55.908 INFO:tasks.workunit.client.0.smithi038.stdout:5/514: dwrite d6/d19/d20/d42/f91 [0,4194304] 0 2024-03-24T00:48:55.908 INFO:tasks.workunit.client.1.smithi088.stdout:4/560: rename d5/d16/d1a/d22/l28 to d5/d17/lc2 0 2024-03-24T00:48:55.908 INFO:tasks.workunit.client.1.smithi088.stdout:4/561: chown d5/d16/d1a/d22/l5f 6 1 2024-03-24T00:48:55.908 INFO:tasks.workunit.client.1.smithi088.stdout:4/562: stat d5/d17/c5e 0 2024-03-24T00:48:55.909 INFO:tasks.workunit.client.0.smithi038.stdout:5/515: write d6/f26 [1100824,80514] 0 2024-03-24T00:48:55.909 INFO:tasks.workunit.client.0.smithi038.stdout:5/516: write d6/d19/d20/d33/d6f/f88 [193541,4225] 0 2024-03-24T00:48:55.909 INFO:tasks.workunit.client.0.smithi038.stdout:0/447: rename d0/d2d/c3e to d0/d12/d60/d28/c94 0 2024-03-24T00:48:55.912 INFO:tasks.workunit.client.0.smithi038.stdout:9/580: symlink d12/d1a/d5f/lc2 0 2024-03-24T00:48:55.913 INFO:tasks.workunit.client.1.smithi088.stdout:3/534: dwrite d3/d1a/d1e/d2f/d45/f6b [0,4194304] 0 2024-03-24T00:48:55.917 INFO:tasks.workunit.client.1.smithi088.stdout:2/554: dread db/d54/f6d [0,4194304] 0 2024-03-24T00:48:55.917 INFO:tasks.workunit.client.1.smithi088.stdout:2/555: write db/dc/f2c [5406224,41119] 0 2024-03-24T00:48:55.917 INFO:tasks.workunit.client.1.smithi088.stdout:2/556: fdatasync db/dc/f2b 0 2024-03-24T00:48:55.917 INFO:tasks.workunit.client.1.smithi088.stdout:5/579: mkdir d1/d19/db9 0 2024-03-24T00:48:55.920 INFO:tasks.workunit.client.0.smithi038.stdout:3/534: dread d3/d9/f2b [0,4194304] 0 2024-03-24T00:48:55.921 INFO:tasks.workunit.client.0.smithi038.stdout:8/506: mkdir d0/d1d/d27/d4d/daf 0 2024-03-24T00:48:55.926 INFO:tasks.workunit.client.1.smithi088.stdout:6/582: sync 2024-03-24T00:48:55.927 INFO:tasks.workunit.client.1.smithi088.stdout:1/573: truncate d6/f10 4455270 0 2024-03-24T00:48:55.929 INFO:tasks.workunit.client.1.smithi088.stdout:8/585: creat d9/dc/d11/d24/fca x:0 0 0 2024-03-24T00:48:55.929 INFO:tasks.workunit.client.1.smithi088.stdout:8/586: dread - d9/dc/d11/d41/d54/d5d/db3/fb4 zero size 2024-03-24T00:48:55.929 INFO:tasks.workunit.client.0.smithi038.stdout:1/599: sync 2024-03-24T00:48:55.933 INFO:tasks.workunit.client.1.smithi088.stdout:0/580: dwrite d9/dc/f49 [0,4194304] 0 2024-03-24T00:48:55.936 INFO:tasks.workunit.client.1.smithi088.stdout:4/563: creat d5/d7/d92/fc3 x:0 0 0 2024-03-24T00:48:55.936 INFO:tasks.workunit.client.1.smithi088.stdout:4/564: chown d5/d7/f44 32 1 2024-03-24T00:48:55.937 INFO:tasks.workunit.client.0.smithi038.stdout:8/507: dread d0/f16 [0,4194304] 0 2024-03-24T00:48:55.937 INFO:tasks.workunit.client.0.smithi038.stdout:8/508: truncate d0/d15/f94 752239 0 2024-03-24T00:48:55.943 INFO:tasks.workunit.client.0.smithi038.stdout:6/581: rename d1/da/d1b/da7/fb6 to d1/da/d1b/fb8 0 2024-03-24T00:48:55.943 INFO:tasks.workunit.client.0.smithi038.stdout:6/582: readlink d1/da/d1b/d25/l78 0 2024-03-24T00:48:55.943 INFO:tasks.workunit.client.0.smithi038.stdout:6/583: write d1/f13 [2419727,83585] 0 2024-03-24T00:48:55.943 INFO:tasks.workunit.client.0.smithi038.stdout:9/581: mknod d12/d1a/cc3 0 2024-03-24T00:48:55.945 INFO:tasks.workunit.client.1.smithi088.stdout:5/580: mkdir d1/d10/d1b/d2f/d57/d9e/dba 0 2024-03-24T00:48:55.946 INFO:tasks.workunit.client.0.smithi038.stdout:1/600: truncate da/dd/d22/d5c/f7f 2389433 0 2024-03-24T00:48:55.948 INFO:tasks.workunit.client.0.smithi038.stdout:5/517: getdents d6/d16/d14 0 2024-03-24T00:48:55.949 INFO:tasks.workunit.client.0.smithi038.stdout:3/535: symlink d3/d30/d5d/lb1 0 2024-03-24T00:48:55.956 INFO:tasks.workunit.client.1.smithi088.stdout:6/583: mkdir db/d1d/d23/d3c/d43/daf 0 2024-03-24T00:48:55.956 INFO:tasks.workunit.client.1.smithi088.stdout:6/584: creat db/d1d/d23/fb0 x:0 0 0 2024-03-24T00:48:55.956 INFO:tasks.workunit.client.1.smithi088.stdout:6/585: chown db/f47 3413 1 2024-03-24T00:48:55.957 INFO:tasks.workunit.client.1.smithi088.stdout:8/587: dread d9/f57 [0,4194304] 0 2024-03-24T00:48:55.958 INFO:tasks.workunit.client.0.smithi038.stdout:9/582: chown d12/d28/c2f 248 1 2024-03-24T00:48:55.961 INFO:tasks.workunit.client.0.smithi038.stdout:6/584: creat d1/da/d61/da2/db1/fb9 x:0 0 0 2024-03-24T00:48:55.964 INFO:tasks.workunit.client.1.smithi088.stdout:1/574: write d6/d23/f73 [5100967,44395] 0 2024-03-24T00:48:55.964 INFO:tasks.workunit.client.1.smithi088.stdout:1/575: write d6/d43/f50 [402193,68135] 0 2024-03-24T00:48:55.972 INFO:tasks.workunit.client.1.smithi088.stdout:9/587: dwrite d1/d22/d72/faf [0,4194304] 0 2024-03-24T00:48:55.972 INFO:tasks.workunit.client.1.smithi088.stdout:9/588: creat d1/d10/d77/fbb x:0 0 0 2024-03-24T00:48:55.973 INFO:tasks.workunit.client.1.smithi088.stdout:9/589: write d1/d10/d77/fb4 [851618,65472] 0 2024-03-24T00:48:55.973 INFO:tasks.workunit.client.0.smithi038.stdout:1/601: rename da/dd/d2c to da/dd/d2c/db6/dc4/dc9 22 2024-03-24T00:48:55.973 INFO:tasks.workunit.client.0.smithi038.stdout:1/602: chown da/dd/d22/c30 4289970 1 2024-03-24T00:48:55.973 INFO:tasks.workunit.client.0.smithi038.stdout:8/509: rename d0/d1d/c90 to d0/d1d/d27/d4d/cb0 0 2024-03-24T00:48:55.973 INFO:tasks.workunit.client.0.smithi038.stdout:8/510: write d0/d15/d2c/fa2 [1092423,43432] 0 2024-03-24T00:48:55.973 INFO:tasks.workunit.client.0.smithi038.stdout:8/511: readlink d0/d1d/d27/l9d 0 2024-03-24T00:48:55.974 INFO:tasks.workunit.client.1.smithi088.stdout:4/565: unlink d5/d17/f1c 0 2024-03-24T00:48:55.982 INFO:tasks.workunit.client.1.smithi088.stdout:3/535: rename d3/d1a/d1e/d48 to d3/d1a/d1e/d76/dae 0 2024-03-24T00:48:55.982 INFO:tasks.workunit.client.1.smithi088.stdout:6/586: dread db/dd/f10 [0,4194304] 0 2024-03-24T00:48:55.986 INFO:tasks.workunit.client.0.smithi038.stdout:6/585: dread d1/f7 [0,4194304] 0 2024-03-24T00:48:55.986 INFO:tasks.workunit.client.0.smithi038.stdout:6/586: creat d1/da/d61/d79/fba x:0 0 0 2024-03-24T00:48:55.986 INFO:tasks.workunit.client.0.smithi038.stdout:6/587: dread - d1/da/d1b/fa8 zero size 2024-03-24T00:48:55.991 INFO:tasks.workunit.client.0.smithi038.stdout:5/518: symlink d6/d19/d20/d42/d43/d68/lad 0 2024-03-24T00:48:55.991 INFO:tasks.workunit.client.0.smithi038.stdout:0/448: dwrite d0/d1b/d2c/f86 [0,4194304] 0 2024-03-24T00:48:55.991 INFO:tasks.workunit.client.0.smithi038.stdout:0/449: fdatasync d0/d2d/f6f 0 2024-03-24T00:48:55.991 INFO:tasks.workunit.client.0.smithi038.stdout:0/450: read - d0/d12/d15/d1a/f7a zero size 2024-03-24T00:48:55.992 INFO:tasks.workunit.client.0.smithi038.stdout:3/536: unlink d3/d9/dc/df/d14/l71 0 2024-03-24T00:48:55.992 INFO:tasks.workunit.client.0.smithi038.stdout:3/537: stat d3/d9/fb 0 2024-03-24T00:48:55.992 INFO:tasks.workunit.client.0.smithi038.stdout:3/538: chown d3/d9/dc/df/d14/l74 5 1 2024-03-24T00:48:55.997 INFO:tasks.workunit.client.0.smithi038.stdout:1/603: mknod da/dd/d7a/d82/d98/d14/dc0/cca 0 2024-03-24T00:48:55.998 INFO:tasks.workunit.client.1.smithi088.stdout:8/588: truncate d9/d6b/f6c 1671408 0 2024-03-24T00:48:56.007 INFO:tasks.workunit.client.0.smithi038.stdout:2/500: dwrite d4/d18/d38/f5a [0,4194304] 0 2024-03-24T00:48:56.014 INFO:tasks.workunit.client.1.smithi088.stdout:0/581: getdents d9/d26/d8f/da1/d62/d9b 0 2024-03-24T00:48:56.014 INFO:tasks.workunit.client.1.smithi088.stdout:1/576: creat d6/d27/d67/d9b/fac x:0 0 0 2024-03-24T00:48:56.019 INFO:tasks.workunit.client.0.smithi038.stdout:5/519: symlink d6/d19/d20/d64/d6b/d77/lae 0 2024-03-24T00:48:56.026 INFO:tasks.workunit.client.0.smithi038.stdout:3/539: creat d3/d9/dc/df/d14/d19/d82/d89/da8/fb2 x:0 0 0 2024-03-24T00:48:56.037 INFO:tasks.workunit.client.1.smithi088.stdout:6/587: creat db/d3d/d83/d71/fb1 x:0 0 0 2024-03-24T00:48:56.041 INFO:tasks.workunit.client.0.smithi038.stdout:2/501: truncate d4/d18/d26/f3d 2274508 0 2024-03-24T00:48:56.041 INFO:tasks.workunit.client.0.smithi038.stdout:2/502: write d4/d18/f28 [941296,11602] 0 2024-03-24T00:48:56.041 INFO:tasks.workunit.client.0.smithi038.stdout:2/503: dread - d4/d27/d2f/d30/d3e/f78 zero size 2024-03-24T00:48:56.041 INFO:tasks.workunit.client.0.smithi038.stdout:2/504: dread - d4/d27/d2f/d30/d3e/d7c/fb6 zero size 2024-03-24T00:48:56.041 INFO:tasks.workunit.client.0.smithi038.stdout:2/505: chown d4/d18/d26/d37/d69/d83/dad 157693 1 2024-03-24T00:48:56.042 INFO:tasks.workunit.client.0.smithi038.stdout:2/506: fsync d4/d18/d38/d63/f7e 0 2024-03-24T00:48:56.042 INFO:tasks.workunit.client.0.smithi038.stdout:2/507: creat d4/d27/d2f/d30/d3e/d9b/fb8 x:0 0 0 2024-03-24T00:48:56.042 INFO:tasks.workunit.client.0.smithi038.stdout:2/508: chown d4/da3/fb7 231745 1 2024-03-24T00:48:56.042 INFO:tasks.workunit.client.1.smithi088.stdout:8/589: creat d9/dc/d11/d24/d29/d5b/db9/da5/fcb x:0 0 0 2024-03-24T00:48:56.044 INFO:tasks.workunit.client.0.smithi038.stdout:3/540: creat d3/d9/fb3 x:0 0 0 2024-03-24T00:48:56.044 INFO:tasks.workunit.client.0.smithi038.stdout:3/541: truncate d3/f8c 385733 0 2024-03-24T00:48:56.045 INFO:tasks.workunit.client.0.smithi038.stdout:3/542: creat d3/d9/dc/da6/fb4 x:0 0 0 2024-03-24T00:48:56.048 INFO:tasks.workunit.client.0.smithi038.stdout:1/604: rmdir da/dd/d7a/d82/d98/d34 39 2024-03-24T00:48:56.048 INFO:tasks.workunit.client.0.smithi038.stdout:1/605: creat da/dd/d7a/fcb x:0 0 0 2024-03-24T00:48:56.048 INFO:tasks.workunit.client.0.smithi038.stdout:1/606: chown l2 210 1 2024-03-24T00:48:56.069 INFO:tasks.workunit.client.1.smithi088.stdout:6/588: write db/d1d/d50/d90/d56/f31 [1897256,12685] 0 2024-03-24T00:48:56.071 INFO:tasks.workunit.client.0.smithi038.stdout:0/451: dwrite d0/d1b/d2c/f86 [0,4194304] 0 2024-03-24T00:48:56.078 INFO:tasks.workunit.client.1.smithi088.stdout:8/590: mknod d9/dc/d11/d24/d29/da9/ccc 0 2024-03-24T00:48:56.079 INFO:tasks.workunit.client.1.smithi088.stdout:3/536: dwrite d3/de/d6f/fa9 [0,4194304] 0 2024-03-24T00:48:56.079 INFO:tasks.workunit.client.0.smithi038.stdout:9/583: dwrite d12/d55/d56/f7c [0,4194304] 0 2024-03-24T00:48:56.081 INFO:tasks.workunit.client.1.smithi088.stdout:0/582: dwrite d9/dc/de/f6c [0,4194304] 0 2024-03-24T00:48:56.081 INFO:tasks.workunit.client.1.smithi088.stdout:0/583: stat d9/dc/l21 0 2024-03-24T00:48:56.087 INFO:tasks.workunit.client.1.smithi088.stdout:8/591: dread d9/dc/d11/d24/d29/d5b/d6f/fa7 [0,4194304] 0 2024-03-24T00:48:56.088 INFO:tasks.workunit.client.1.smithi088.stdout:6/589: symlink db/d1d/d50/d90/d55/lb2 0 2024-03-24T00:48:56.090 INFO:tasks.workunit.client.0.smithi038.stdout:0/452: truncate d0/d1b/d2c/f6b 1792266 0 2024-03-24T00:48:56.090 INFO:tasks.workunit.client.0.smithi038.stdout:0/453: truncate d0/f5 4701761 0 2024-03-24T00:48:56.092 INFO:tasks.workunit.client.1.smithi088.stdout:2/557: rename db/d54 to db/dd/d96/d99/db2 0 2024-03-24T00:48:56.097 INFO:tasks.workunit.client.0.smithi038.stdout:9/584: symlink d12/d1a/d5f/lc4 0 2024-03-24T00:48:56.098 INFO:tasks.workunit.client.0.smithi038.stdout:9/585: dread - d12/d28/d3a/d26/f4a zero size 2024-03-24T00:48:56.099 INFO:tasks.workunit.client.0.smithi038.stdout:5/520: dwrite d6/d19/d20/d42/d43/d4c/f62 [0,4194304] 0 2024-03-24T00:48:56.100 INFO:tasks.workunit.client.1.smithi088.stdout:9/590: dwrite d1/d22/d72/f79 [0,4194304] 0 2024-03-24T00:48:56.100 INFO:tasks.workunit.client.1.smithi088.stdout:9/591: dread - d1/d22/d27/d39/d98/fae zero size 2024-03-24T00:48:56.100 INFO:tasks.workunit.client.1.smithi088.stdout:9/592: creat d1/d10/d11/d23/fbc x:0 0 0 2024-03-24T00:48:56.100 INFO:tasks.workunit.client.1.smithi088.stdout:9/593: dread - d1/d22/f8c zero size 2024-03-24T00:48:56.101 INFO:tasks.workunit.client.1.smithi088.stdout:3/537: rmdir d3/de/d6f/d85/d90 39 2024-03-24T00:48:56.104 INFO:tasks.workunit.client.0.smithi038.stdout:9/586: symlink d12/d28/d3a/d26/lc5 0 2024-03-24T00:48:56.104 INFO:tasks.workunit.client.0.smithi038.stdout:8/512: dwrite d0/d45/d66/f93 [0,4194304] 0 2024-03-24T00:48:56.105 INFO:tasks.workunit.client.0.smithi038.stdout:9/587: chown d12/d28/d3a/d26/f3d 170292 1 2024-03-24T00:48:56.106 INFO:tasks.workunit.client.0.smithi038.stdout:9/588: write d12/d28/d3a/f3f [754736,59006] 0 2024-03-24T00:48:56.108 INFO:tasks.workunit.client.0.smithi038.stdout:4/491: dwrite da/d12/d24/f30 [0,4194304] 0 2024-03-24T00:48:56.108 INFO:tasks.workunit.client.0.smithi038.stdout:4/492: creat da/d12/fa0 x:0 0 0 2024-03-24T00:48:56.108 INFO:tasks.workunit.client.0.smithi038.stdout:4/493: creat da/fa1 x:0 0 0 2024-03-24T00:48:56.108 INFO:tasks.workunit.client.0.smithi038.stdout:4/494: creat da/fa2 x:0 0 0 2024-03-24T00:48:56.108 INFO:tasks.workunit.client.0.smithi038.stdout:4/495: fdatasync da/d12/d37/dd/f21 0 2024-03-24T00:48:56.108 INFO:tasks.workunit.client.0.smithi038.stdout:4/496: fsync da/d27/f69 0 2024-03-24T00:48:56.108 INFO:tasks.workunit.client.0.smithi038.stdout:4/497: creat da/d38/fa3 x:0 0 0 2024-03-24T00:48:56.111 INFO:tasks.workunit.client.1.smithi088.stdout:3/538: dread d3/d6/d27/d3c/f39 [0,4194304] 0 2024-03-24T00:48:56.112 INFO:tasks.workunit.client.1.smithi088.stdout:0/584: creat d9/dc/de/d19/fbf x:0 0 0 2024-03-24T00:48:56.112 INFO:tasks.workunit.client.1.smithi088.stdout:0/585: fsync d9/dc/de/d19/d44/d46/d65/fb5 0 2024-03-24T00:48:56.115 INFO:tasks.workunit.client.0.smithi038.stdout:5/521: truncate d6/d19/d20/d42/d4b/f4f 393913 0 2024-03-24T00:48:56.116 INFO:tasks.workunit.client.0.smithi038.stdout:5/522: creat d6/d19/d20/d33/d6f/faf x:0 0 0 2024-03-24T00:48:56.116 INFO:tasks.workunit.client.0.smithi038.stdout:5/523: write d6/d19/d20/d42/d43/d50/fa7 [591739,63125] 0 2024-03-24T00:48:56.116 INFO:tasks.workunit.client.1.smithi088.stdout:6/590: mknod db/d1d/d50/d90/cb3 0 2024-03-24T00:48:56.119 INFO:tasks.workunit.client.0.smithi038.stdout:1/607: dwrite da/dd/d7a/d82/d98/f84 [0,4194304] 0 2024-03-24T00:48:56.119 INFO:tasks.workunit.client.0.smithi038.stdout:1/608: stat da/f23 0 2024-03-24T00:48:56.124 INFO:tasks.workunit.client.1.smithi088.stdout:2/558: symlink db/dd/d96/d99/db2/lb3 0 2024-03-24T00:48:56.124 INFO:tasks.workunit.client.1.smithi088.stdout:2/559: truncate db/dd/d96/d99/db2/f6d 1126077 0 2024-03-24T00:48:56.130 INFO:tasks.workunit.client.0.smithi038.stdout:8/513: mknod d0/d1d/d8c/cb1 0 2024-03-24T00:48:56.136 INFO:tasks.workunit.client.0.smithi038.stdout:9/589: dread fc [0,4194304] 0 2024-03-24T00:48:56.137 INFO:tasks.workunit.client.1.smithi088.stdout:9/594: creat d1/d22/d27/fbd x:0 0 0 2024-03-24T00:48:56.140 INFO:tasks.workunit.client.0.smithi038.stdout:3/543: dwrite d3/d9/f16 [0,4194304] 0 2024-03-24T00:48:56.142 INFO:tasks.workunit.client.1.smithi088.stdout:3/539: mkdir d3/d1a/d1e/d76/dae/d4b/daf 0 2024-03-24T00:48:56.142 INFO:tasks.workunit.client.1.smithi088.stdout:3/540: truncate d3/de/d6f/f7a 629590 0 2024-03-24T00:48:56.142 INFO:tasks.workunit.client.1.smithi088.stdout:3/541: fdatasync d3/d6/d19/d2a/d5f/f65 0 2024-03-24T00:48:56.147 INFO:tasks.workunit.client.1.smithi088.stdout:0/586: symlink d9/d26/d8f/da1/d62/d9b/lc0 0 2024-03-24T00:48:56.152 INFO:tasks.workunit.client.1.smithi088.stdout:6/591: rmdir db/d1d/d50 39 2024-03-24T00:48:56.154 INFO:tasks.workunit.client.1.smithi088.stdout:5/581: sync 2024-03-24T00:48:56.154 INFO:tasks.workunit.client.1.smithi088.stdout:7/502: sync 2024-03-24T00:48:56.154 INFO:tasks.workunit.client.1.smithi088.stdout:4/566: rename d5/d16/l87 to d5/d7/dd/lc4 0 2024-03-24T00:48:56.156 INFO:tasks.workunit.client.1.smithi088.stdout:2/560: creat db/dd/d96/d99/db2/fb4 x:0 0 0 2024-03-24T00:48:56.157 INFO:tasks.workunit.client.1.smithi088.stdout:9/595: truncate d1/d10/d11/d23/f54 1818780 0 2024-03-24T00:48:56.158 INFO:tasks.workunit.client.0.smithi038.stdout:5/524: mkdir d6/d1e/d97/db0 0 2024-03-24T00:48:56.159 INFO:tasks.workunit.client.0.smithi038.stdout:0/454: dwrite d0/d12/d42/d5e/f67 [0,4194304] 0 2024-03-24T00:48:56.160 INFO:tasks.workunit.client.1.smithi088.stdout:3/542: mknod d3/de/cb0 0 2024-03-24T00:48:56.160 INFO:tasks.workunit.client.1.smithi088.stdout:3/543: write d3/f97 [557853,12507] 0 2024-03-24T00:48:56.160 INFO:tasks.workunit.client.1.smithi088.stdout:3/544: chown d3/de/f16 10788 1 2024-03-24T00:48:56.161 INFO:tasks.workunit.client.1.smithi088.stdout:0/587: unlink d9/d26/c60 0 2024-03-24T00:48:56.162 INFO:tasks.workunit.client.0.smithi038.stdout:8/514: rename d0/d15/d2c/fa2 to d0/d1d/d27/d4d/fb2 0 2024-03-24T00:48:56.163 INFO:tasks.workunit.client.1.smithi088.stdout:6/592: link db/d1d/d23/d3c/d4b/c65 db/d1d/d23/d3c/d4b/cb4 0 2024-03-24T00:48:56.163 INFO:tasks.workunit.client.1.smithi088.stdout:9/596: dread d1/d22/f3f [0,4194304] 0 2024-03-24T00:48:56.164 INFO:tasks.workunit.client.1.smithi088.stdout:5/582: mkdir d1/d10/d1b/d2f/d57/d9e/dba/dbb 0 2024-03-24T00:48:56.167 INFO:tasks.workunit.client.0.smithi038.stdout:9/590: write d12/d1a/f20 [7101625,52011] 0 2024-03-24T00:48:56.167 INFO:tasks.workunit.client.0.smithi038.stdout:9/591: chown d12/d28/d3a/l23 177 1 2024-03-24T00:48:56.167 INFO:tasks.workunit.client.0.smithi038.stdout:4/498: unlink da/fa1 0 2024-03-24T00:48:56.167 INFO:tasks.workunit.client.1.smithi088.stdout:4/567: mknod d5/d7/d92/cc5 0 2024-03-24T00:48:56.170 INFO:tasks.workunit.client.1.smithi088.stdout:1/577: rename d6/d27/d67/d9b/fac to d6/d43/d95/d5b/d5d/d61/d7d/fad 0 2024-03-24T00:48:56.172 INFO:tasks.workunit.client.0.smithi038.stdout:0/455: truncate d0/fa 6408175 0 2024-03-24T00:48:56.174 INFO:tasks.workunit.client.0.smithi038.stdout:1/609: dwrite da/dd/d4a/fc2 [0,4194304] 0 2024-03-24T00:48:56.174 INFO:tasks.workunit.client.0.smithi038.stdout:1/610: chown da/f72 29273717 1 2024-03-24T00:48:56.176 INFO:tasks.workunit.client.0.smithi038.stdout:8/515: unlink d0/d45/d37/f44 0 2024-03-24T00:48:56.177 INFO:tasks.workunit.client.1.smithi088.stdout:9/597: symlink d1/d10/d9b/dab/d7a/lbe 0 2024-03-24T00:48:56.177 INFO:tasks.workunit.client.1.smithi088.stdout:3/545: write d3/d1a/d1e/d2f/d45/f6b [3611165,43290] 0 2024-03-24T00:48:56.177 INFO:tasks.workunit.client.1.smithi088.stdout:9/598: creat d1/d22/d27/fbf x:0 0 0 2024-03-24T00:48:56.178 INFO:tasks.workunit.client.1.smithi088.stdout:4/568: write d5/d16/d37/f5a [521011,114468] 0 2024-03-24T00:48:56.181 INFO:tasks.workunit.client.0.smithi038.stdout:9/592: creat d12/d28/d71/d99/fc6 x:0 0 0 2024-03-24T00:48:56.183 INFO:tasks.workunit.client.0.smithi038.stdout:1/611: write f5 [1148866,14186] 0 2024-03-24T00:48:56.184 INFO:tasks.workunit.client.1.smithi088.stdout:1/578: symlink d6/d43/d95/d5b/d5d/d61/lae 0 2024-03-24T00:48:56.185 INFO:tasks.workunit.client.0.smithi038.stdout:4/499: mkdir da/d12/d24/d5f/d8a/da4 0 2024-03-24T00:48:56.185 INFO:tasks.workunit.client.0.smithi038.stdout:4/500: creat da/d12/d37/dd/d15/d4c/d70/fa5 x:0 0 0 2024-03-24T00:48:56.187 INFO:tasks.workunit.client.1.smithi088.stdout:0/588: write d9/dc/de/d19/d44/d46/d65/f68 [2997125,74929] 0 2024-03-24T00:48:56.190 INFO:tasks.workunit.client.1.smithi088.stdout:2/561: rmdir db/dd/d96 39 2024-03-24T00:48:56.190 INFO:tasks.workunit.client.1.smithi088.stdout:2/562: read - db/dd/d96/d4e/fa0 zero size 2024-03-24T00:48:56.191 INFO:tasks.workunit.client.0.smithi038.stdout:8/516: unlink d0/d10/l11 0 2024-03-24T00:48:56.192 INFO:tasks.workunit.client.1.smithi088.stdout:6/593: rmdir db/d1d/d50/d90/d55/d99 39 2024-03-24T00:48:56.193 INFO:tasks.workunit.client.1.smithi088.stdout:9/599: dread d1/d10/d77/d8a/f94 [0,4194304] 0 2024-03-24T00:48:56.203 INFO:tasks.workunit.client.0.smithi038.stdout:1/612: dread da/f72 [0,4194304] 0 2024-03-24T00:48:56.203 INFO:tasks.workunit.client.0.smithi038.stdout:1/613: dread - da/dd/d22/d94/fa0 zero size 2024-03-24T00:48:56.204 INFO:tasks.workunit.client.0.smithi038.stdout:1/614: fdatasync da/dd/d2c/d5a/f92 0 2024-03-24T00:48:56.204 INFO:tasks.workunit.client.0.smithi038.stdout:1/615: stat da/dd/d2c/d5a/d74/f64 0 2024-03-24T00:48:56.204 INFO:tasks.workunit.client.0.smithi038.stdout:1/616: readlink da/l3f 0 2024-03-24T00:48:56.204 INFO:tasks.workunit.client.1.smithi088.stdout:3/546: mknod d3/de/d6f/d85/d31/cb1 0 2024-03-24T00:48:56.208 INFO:tasks.workunit.client.1.smithi088.stdout:4/569: mkdir d5/d16/d1a/d22/dc6 0 2024-03-24T00:48:56.212 INFO:tasks.workunit.client.1.smithi088.stdout:8/592: rename d9/dc/d11/d24/d29/d36/d6a/c85 to d9/dc/d11/ccd 0 2024-03-24T00:48:56.214 INFO:tasks.workunit.client.1.smithi088.stdout:5/583: dwrite d1/d10/d81/f97 [0,4194304] 0 2024-03-24T00:48:56.214 INFO:tasks.workunit.client.1.smithi088.stdout:5/584: read - d1/d10/d1b/d2f/d57/f8d zero size 2024-03-24T00:48:56.216 INFO:tasks.workunit.client.0.smithi038.stdout:9/593: truncate d12/d28/d3a/f2b 39843 0 2024-03-24T00:48:56.222 INFO:tasks.workunit.client.0.smithi038.stdout:4/501: mkdir da/d12/d24/d68/d6c/da6 0 2024-03-24T00:48:56.222 INFO:tasks.workunit.client.0.smithi038.stdout:4/502: creat da/fa7 x:0 0 0 2024-03-24T00:48:56.223 INFO:tasks.workunit.client.1.smithi088.stdout:7/503: dwrite d3/f2b [0,4194304] 0 2024-03-24T00:48:56.223 INFO:tasks.workunit.client.1.smithi088.stdout:7/504: truncate d3/d32/d3f/d4b/d74/d79/f97 535515 0 2024-03-24T00:48:56.223 INFO:tasks.workunit.client.1.smithi088.stdout:6/594: rmdir db/dd/d61 39 2024-03-24T00:48:56.224 INFO:tasks.workunit.client.0.smithi038.stdout:8/517: mknod d0/d1d/d20/d79/cb3 0 2024-03-24T00:48:56.224 INFO:tasks.workunit.client.0.smithi038.stdout:8/518: creat d0/d10/d19/fb4 x:0 0 0 2024-03-24T00:48:56.225 INFO:tasks.workunit.client.1.smithi088.stdout:2/563: dread db/f4f [0,4194304] 0 2024-03-24T00:48:56.227 INFO:tasks.workunit.client.0.smithi038.stdout:8/519: fdatasync d0/d45/d37/f48 0 2024-03-24T00:48:56.228 INFO:tasks.workunit.client.1.smithi088.stdout:9/600: mkdir d1/d10/d77/dc0 0 2024-03-24T00:48:56.231 INFO:tasks.workunit.client.0.smithi038.stdout:1/617: unlink da/dd/d22/f4f 0 2024-03-24T00:48:56.232 INFO:tasks.workunit.client.1.smithi088.stdout:3/547: creat d3/d1a/d1e/d76/dae/d4b/daf/fb2 x:0 0 0 2024-03-24T00:48:56.233 INFO:tasks.workunit.client.1.smithi088.stdout:3/548: chown d3/d1a/d1e/d2f/f36 32591567 1 2024-03-24T00:48:56.233 INFO:tasks.workunit.client.1.smithi088.stdout:3/549: write d3/de/f16 [1802154,114749] 0 2024-03-24T00:48:56.235 INFO:tasks.workunit.client.1.smithi088.stdout:3/550: read d3/d6/d19/d2a/f53 [760444,91143] 0 2024-03-24T00:48:56.239 INFO:tasks.workunit.client.0.smithi038.stdout:3/544: rename d3/d9/dc/df/d14 to d3/d9/dc/da6/db5 0 2024-03-24T00:48:56.241 INFO:tasks.workunit.client.1.smithi088.stdout:8/593: mkdir d9/dc/dce 0 2024-03-24T00:48:56.259 INFO:tasks.workunit.client.1.smithi088.stdout:5/585: mkdir d1/d10/d79/dbc 0 2024-03-24T00:48:56.259 INFO:tasks.workunit.client.0.smithi038.stdout:4/503: mknod da/d12/d37/d83/ca8 0 2024-03-24T00:48:56.264 INFO:tasks.workunit.client.0.smithi038.stdout:1/618: mknod da/dd/d2c/d5a/d74/ccc 0 2024-03-24T00:48:56.265 INFO:tasks.workunit.client.0.smithi038.stdout:1/619: read da/dd/d22/f25 [909552,7078] 0 2024-03-24T00:48:56.265 INFO:tasks.workunit.client.0.smithi038.stdout:7/515: sync 2024-03-24T00:48:56.271 INFO:tasks.workunit.client.0.smithi038.stdout:5/525: rename d6/d16/c8a to d6/d19/d20/d64/d6b/cb1 0 2024-03-24T00:48:56.271 INFO:tasks.workunit.client.0.smithi038.stdout:0/456: rename d0/d12 to d0/d12/d42/d5e/d95 22 2024-03-24T00:48:56.273 INFO:tasks.workunit.client.0.smithi038.stdout:4/504: mknod da/d12/d29/d5b/d73/ca9 0 2024-03-24T00:48:56.278 INFO:tasks.workunit.client.0.smithi038.stdout:1/620: unlink da/dd/d22/f4c 0 2024-03-24T00:48:56.279 INFO:tasks.workunit.client.1.smithi088.stdout:2/564: symlink db/d21/d73/lb5 0 2024-03-24T00:48:56.282 INFO:tasks.workunit.client.0.smithi038.stdout:3/545: rename d3/d9/dc/da6/fb4 to d3/d2a/fb6 0 2024-03-24T00:48:56.283 INFO:tasks.workunit.client.1.smithi088.stdout:9/601: mknod d1/d10/d9b/dab/db8/da7/cc1 0 2024-03-24T00:48:56.296 INFO:tasks.workunit.client.0.smithi038.stdout:4/505: write da/d38/f76 [2898653,57474] 0 2024-03-24T00:48:56.297 INFO:tasks.workunit.client.1.smithi088.stdout:3/551: creat d3/fb3 x:0 0 0 2024-03-24T00:48:56.299 INFO:tasks.workunit.client.0.smithi038.stdout:3/546: mknod d3/d9/dc/da6/db5/cb7 0 2024-03-24T00:48:56.302 INFO:tasks.workunit.client.0.smithi038.stdout:0/457: rename d0/d12/d42/f74 to d0/d14/d84/f96 0 2024-03-24T00:48:56.308 INFO:tasks.workunit.client.1.smithi088.stdout:8/594: dwrite d9/dc/d11/d24/d43/d4a/f55 [0,4194304] 0 2024-03-24T00:48:56.308 INFO:tasks.workunit.client.1.smithi088.stdout:8/595: dread - d9/dc/d11/d24/fca zero size 2024-03-24T00:48:56.311 INFO:tasks.workunit.client.1.smithi088.stdout:5/586: creat d1/dd/d6f/da1/d3f/d88/d8f/fbd x:0 0 0 2024-03-24T00:48:56.311 INFO:tasks.workunit.client.1.smithi088.stdout:2/565: getdents db/dd/d96/d99/db2/d88/d8e 0 2024-03-24T00:48:56.311 INFO:tasks.workunit.client.1.smithi088.stdout:2/566: fdatasync db/d21/d73/f83 0 2024-03-24T00:48:56.312 INFO:tasks.workunit.client.1.smithi088.stdout:2/567: chown db/d21/d35/d49/lab 20562234 1 2024-03-24T00:48:56.313 INFO:tasks.workunit.client.0.smithi038.stdout:8/520: dwrite d0/d1d/d20/d3c/f47 [0,4194304] 0 2024-03-24T00:48:56.313 INFO:tasks.workunit.client.0.smithi038.stdout:8/521: fsync d0/d15/f63 0 2024-03-24T00:48:56.315 INFO:tasks.workunit.client.0.smithi038.stdout:9/594: dwrite f4 [0,4194304] 0 2024-03-24T00:48:56.315 INFO:tasks.workunit.client.1.smithi088.stdout:9/602: link d1/d10/d29/f45 d1/d10/d9b/dab/db8/fc2 0 2024-03-24T00:48:56.315 INFO:tasks.workunit.client.1.smithi088.stdout:9/603: dread - d1/d22/f51 zero size 2024-03-24T00:48:56.316 INFO:tasks.workunit.client.1.smithi088.stdout:1/579: rename d6/d43 to d6/d23/daf 0 2024-03-24T00:48:56.316 INFO:tasks.workunit.client.1.smithi088.stdout:1/580: chown d6/f3e 35334 1 2024-03-24T00:48:56.316 INFO:tasks.workunit.client.0.smithi038.stdout:5/526: rmdir d6/d19/d20/d42 39 2024-03-24T00:48:56.317 INFO:tasks.workunit.client.0.smithi038.stdout:4/506: creat da/d12/d24/d5f/d8a/da4/faa x:0 0 0 2024-03-24T00:48:56.318 INFO:tasks.workunit.client.0.smithi038.stdout:4/507: fsync da/d38/f3c 0 2024-03-24T00:48:56.321 INFO:tasks.workunit.client.0.smithi038.stdout:1/621: rename da/dd/d2c/la4 to da/dd/d7a/d82/lcd 0 2024-03-24T00:48:56.321 INFO:tasks.workunit.client.0.smithi038.stdout:1/622: stat da/dd/d7a/d82/d98/d14/c50 0 2024-03-24T00:48:56.321 INFO:tasks.workunit.client.0.smithi038.stdout:1/623: write da/dd/d7a/d82/d98/d34/fa5 [287408,96814] 0 2024-03-24T00:48:56.322 INFO:tasks.workunit.client.0.smithi038.stdout:1/624: creat da/dd/d22/d94/fce x:0 0 0 2024-03-24T00:48:56.322 INFO:tasks.workunit.client.0.smithi038.stdout:1/625: stat da/dd/d2c/d41/d60/fc5 0 2024-03-24T00:48:56.324 INFO:tasks.workunit.client.1.smithi088.stdout:5/587: mknod d1/d10/d1b/cbe 0 2024-03-24T00:48:56.324 INFO:tasks.workunit.client.1.smithi088.stdout:5/588: chown d1/d10/d4f/c5d 781162544 1 2024-03-24T00:48:56.326 INFO:tasks.workunit.client.1.smithi088.stdout:2/568: creat db/d21/fb6 x:0 0 0 2024-03-24T00:48:56.327 INFO:tasks.workunit.client.1.smithi088.stdout:9/604: unlink d1/d10/d11/d23/f54 0 2024-03-24T00:48:56.327 INFO:tasks.workunit.client.1.smithi088.stdout:9/605: chown d1/d10/d77/fb4 23389 1 2024-03-24T00:48:56.329 INFO:tasks.workunit.client.0.smithi038.stdout:9/595: truncate d12/d1a/d80/f97 3040748 0 2024-03-24T00:48:56.331 INFO:tasks.workunit.client.1.smithi088.stdout:0/589: rename d9/dc/f7a to d9/dc/de/d7e/d91/fc1 0 2024-03-24T00:48:56.331 INFO:tasks.workunit.client.1.smithi088.stdout:0/590: write d9/d26/d8f/da1/d62/f6d [328992,23511] 0 2024-03-24T00:48:56.332 INFO:tasks.workunit.client.1.smithi088.stdout:0/591: creat d9/d26/d8f/da1/fc2 x:0 0 0 2024-03-24T00:48:56.334 INFO:tasks.workunit.client.0.smithi038.stdout:8/522: dread d0/d1d/d27/d4d/f7b [0,4194304] 0 2024-03-24T00:48:56.334 INFO:tasks.workunit.client.0.smithi038.stdout:5/527: creat d6/d19/d20/d70/da3/fb2 x:0 0 0 2024-03-24T00:48:56.334 INFO:tasks.workunit.client.0.smithi038.stdout:8/523: chown d0/d1d/d27/l3f 70164 1 2024-03-24T00:48:56.335 INFO:tasks.workunit.client.1.smithi088.stdout:1/581: getdents d6/d23/daf/d95/d3f 0 2024-03-24T00:48:56.337 INFO:tasks.workunit.client.0.smithi038.stdout:3/547: rename d3/d9/f62 to d3/d9/dc/df/fb8 0 2024-03-24T00:48:56.339 INFO:tasks.workunit.client.1.smithi088.stdout:5/589: creat d1/dd/d6f/da1/d3f/d88/d8b/fbf x:0 0 0 2024-03-24T00:48:56.341 INFO:tasks.workunit.client.0.smithi038.stdout:5/528: read d6/d19/d20/d42/d43/d4c/f74 [322399,74251] 0 2024-03-24T00:48:56.341 INFO:tasks.workunit.client.0.smithi038.stdout:5/529: chown d6/d19/d20/d42/d43/d87 200985 1 2024-03-24T00:48:56.347 INFO:tasks.workunit.client.1.smithi088.stdout:2/569: unlink db/l9d 0 2024-03-24T00:48:56.347 INFO:tasks.workunit.client.0.smithi038.stdout:4/508: dread da/d12/d37/dd/f3a [0,4194304] 0 2024-03-24T00:48:56.347 INFO:tasks.workunit.client.0.smithi038.stdout:4/509: dread - da/d38/f63 zero size 2024-03-24T00:48:56.347 INFO:tasks.workunit.client.0.smithi038.stdout:4/510: chown da/d12/c8c 1483498 1 2024-03-24T00:48:56.347 INFO:tasks.workunit.client.0.smithi038.stdout:1/626: symlink da/lcf 0 2024-03-24T00:48:56.350 INFO:tasks.workunit.client.1.smithi088.stdout:9/606: link d1/d15/d80/da5/fb2 d1/d10/d11/fc3 0 2024-03-24T00:48:56.350 INFO:tasks.workunit.client.0.smithi038.stdout:9/596: creat d12/d28/fc7 x:0 0 0 2024-03-24T00:48:56.352 INFO:tasks.workunit.client.1.smithi088.stdout:4/570: sync 2024-03-24T00:48:56.354 INFO:tasks.workunit.client.0.smithi038.stdout:3/548: rename d3/d9/dc/da6/db5/l41 to d3/d9/dc/da6/db5/lb9 0 2024-03-24T00:48:56.356 INFO:tasks.workunit.client.0.smithi038.stdout:0/458: rename d0/d12/f70 to d0/d14/d88/f97 0 2024-03-24T00:48:56.356 INFO:tasks.workunit.client.0.smithi038.stdout:0/459: chown d0/d14/d88 76973238 1 2024-03-24T00:48:56.356 INFO:tasks.workunit.client.0.smithi038.stdout:0/460: readlink d0/d12/d60/l38 0 2024-03-24T00:48:56.358 INFO:tasks.workunit.client.1.smithi088.stdout:0/592: rmdir d9/dc/de/d7e/d91 39 2024-03-24T00:48:56.362 INFO:tasks.workunit.client.1.smithi088.stdout:1/582: mkdir d6/d27/db0 0 2024-03-24T00:48:56.362 INFO:tasks.workunit.client.1.smithi088.stdout:1/583: write d6/f6d [168300,58510] 0 2024-03-24T00:48:56.362 INFO:tasks.workunit.client.0.smithi038.stdout:4/511: mknod da/d12/d29/d5b/cab 0 2024-03-24T00:48:56.368 INFO:tasks.workunit.client.1.smithi088.stdout:3/552: dwrite d3/de/f16 [4194304,4194304] 0 2024-03-24T00:48:56.368 INFO:tasks.workunit.client.1.smithi088.stdout:3/553: readlink d3/d6/d19/d2a/d5f/la6 0 2024-03-24T00:48:56.369 INFO:tasks.workunit.client.1.smithi088.stdout:5/590: dread d1/d19/d1e/f6d [0,4194304] 0 2024-03-24T00:48:56.369 INFO:tasks.workunit.client.1.smithi088.stdout:5/591: fsync d1/d19/d1e/f9b 0 2024-03-24T00:48:56.369 INFO:tasks.workunit.client.1.smithi088.stdout:5/592: stat d1/dd/d6f/da1/d3f/d88/d8f/f5e 0 2024-03-24T00:48:56.369 INFO:tasks.workunit.client.1.smithi088.stdout:5/593: write d1/dd/d6f/da1/d3f/d88/d8b/fbf [615594,19213] 0 2024-03-24T00:48:56.369 INFO:tasks.workunit.client.1.smithi088.stdout:5/594: read - d1/d10/d1b/d2f/d93/fa9 zero size 2024-03-24T00:48:56.375 INFO:tasks.workunit.client.0.smithi038.stdout:1/627: dread da/d3c/f81 [0,4194304] 0 2024-03-24T00:48:56.383 INFO:tasks.workunit.client.1.smithi088.stdout:2/570: dread db/d21/d35/f79 [0,4194304] 0 2024-03-24T00:48:56.383 INFO:tasks.workunit.client.1.smithi088.stdout:2/571: truncate db/d21/d35/f90 474972 0 2024-03-24T00:48:56.386 INFO:tasks.workunit.client.1.smithi088.stdout:3/554: write d3/d6/d27/d3c/f59 [1533382,117729] 0 2024-03-24T00:48:56.389 INFO:tasks.workunit.client.1.smithi088.stdout:8/596: dwrite d9/dc/d11/d24/d29/d36/f97 [0,4194304] 0 2024-03-24T00:48:56.393 INFO:tasks.workunit.client.0.smithi038.stdout:1/628: write da/dd/d7a/d82/d98/f9a [4150304,125787] 0 2024-03-24T00:48:56.415 INFO:tasks.workunit.client.0.smithi038.stdout:5/530: rmdir d6/d19/d20/d70/da3 39 2024-03-24T00:48:56.415 INFO:tasks.workunit.client.0.smithi038.stdout:5/531: stat d6/d16/d14 0 2024-03-24T00:48:56.417 INFO:tasks.workunit.client.0.smithi038.stdout:0/461: write d0/d12/d60/d28/f48 [5219413,125585] 0 2024-03-24T00:48:56.418 INFO:tasks.workunit.client.0.smithi038.stdout:0/462: chown d0/d12/d15/f22 31687955 1 2024-03-24T00:48:56.418 INFO:tasks.workunit.client.0.smithi038.stdout:0/463: fdatasync d0/d1b/f20 0 2024-03-24T00:48:56.420 INFO:tasks.workunit.client.0.smithi038.stdout:6/588: sync 2024-03-24T00:48:56.422 INFO:tasks.workunit.client.0.smithi038.stdout:3/549: creat d3/d9/dc/da6/db5/d19/d82/fba x:0 0 0 2024-03-24T00:48:56.431 INFO:tasks.workunit.client.0.smithi038.stdout:5/532: dread d6/d19/d20/d42/d79/fa0 [0,4194304] 0 2024-03-24T00:48:56.433 INFO:tasks.workunit.client.1.smithi088.stdout:7/505: rename d3/d32/d3f to d3/d32/d34/d9c 0 2024-03-24T00:48:56.436 INFO:tasks.workunit.client.0.smithi038.stdout:8/524: rename d0/d45/fa8 to d0/d1d/d27/d4d/daf/fb5 0 2024-03-24T00:48:56.436 INFO:tasks.workunit.client.0.smithi038.stdout:8/525: creat d0/d15/d2c/fb6 x:0 0 0 2024-03-24T00:48:56.436 INFO:tasks.workunit.client.0.smithi038.stdout:8/526: truncate d0/d1d/d27/d4d/daf/fb5 996385 0 2024-03-24T00:48:56.437 INFO:tasks.workunit.client.0.smithi038.stdout:6/589: dread d1/f8 [0,4194304] 0 2024-03-24T00:48:56.437 INFO:tasks.workunit.client.1.smithi088.stdout:4/571: creat d5/d16/d94/da6/fc7 x:0 0 0 2024-03-24T00:48:56.437 INFO:tasks.workunit.client.0.smithi038.stdout:0/464: write d0/d12/d42/f51 [1562550,61780] 0 2024-03-24T00:48:56.438 INFO:tasks.workunit.client.0.smithi038.stdout:0/465: chown d0/d1b/d2c 1451174 1 2024-03-24T00:48:56.441 INFO:tasks.workunit.client.0.smithi038.stdout:4/512: mkdir da/d12/d37/d83/dac 0 2024-03-24T00:48:56.447 INFO:tasks.workunit.client.1.smithi088.stdout:1/584: mknod d6/d23/daf/d95/d5b/d5e/d90/cb1 0 2024-03-24T00:48:56.451 INFO:tasks.workunit.client.0.smithi038.stdout:5/533: dread d6/d19/d20/d42/d43/d50/f81 [0,4194304] 0 2024-03-24T00:48:56.491 INFO:tasks.workunit.client.1.smithi088.stdout:5/595: creat d1/d19/fc0 x:0 0 0 2024-03-24T00:48:56.491 INFO:tasks.workunit.client.1.smithi088.stdout:5/596: fdatasync d1/dd/d6f/da1/d3f/d88/d8f/d69/f82 0 2024-03-24T00:48:56.494 INFO:tasks.workunit.client.1.smithi088.stdout:0/593: mknod d9/dc/de/cc3 0 2024-03-24T00:48:56.510 INFO:tasks.workunit.client.1.smithi088.stdout:5/597: dread d1/d10/d1b/d2f/f7e [0,4194304] 0 2024-03-24T00:48:56.512 INFO:tasks.workunit.client.0.smithi038.stdout:1/629: link da/dd/d22/c30 da/dd/d2c/d41/cd0 0 2024-03-24T00:48:56.513 INFO:tasks.workunit.client.1.smithi088.stdout:5/598: write d1/dd/d6f/da1/d3f/d88/d8b/fbf [273167,119340] 0 2024-03-24T00:48:56.513 INFO:tasks.workunit.client.0.smithi038.stdout:3/550: fsync d3/d9/dc/da6/db5/d19/d82/fba 0 2024-03-24T00:48:56.520 INFO:tasks.workunit.client.1.smithi088.stdout:5/599: dread d1/dd/d6f/da1/f77 [0,4194304] 0 2024-03-24T00:48:56.526 INFO:tasks.workunit.client.1.smithi088.stdout:8/597: creat d9/dc/d11/d24/d29/d36/d6a/d7b/d8f/dbc/fcf x:0 0 0 2024-03-24T00:48:56.527 INFO:tasks.workunit.client.1.smithi088.stdout:8/598: stat d9/dc/d11/d24/d29/d5b/db9/fc5 0 2024-03-24T00:48:56.527 INFO:tasks.workunit.client.1.smithi088.stdout:8/599: write d9/f63 [1943946,14943] 0 2024-03-24T00:48:56.571 INFO:tasks.workunit.client.0.smithi038.stdout:9/597: getdents d12/d28/da8 0 2024-03-24T00:48:56.583 INFO:tasks.workunit.client.0.smithi038.stdout:8/527: getdents d0/d1d/d27/d6b/da4 0 2024-03-24T00:48:56.583 INFO:tasks.workunit.client.0.smithi038.stdout:8/528: fdatasync d0/d1d/d20/f58 0 2024-03-24T00:48:56.583 INFO:tasks.workunit.client.0.smithi038.stdout:8/529: write d0/d15/d2c/fb6 [796049,24065] 0 2024-03-24T00:48:56.584 INFO:tasks.workunit.client.0.smithi038.stdout:6/590: truncate d1/da/d1b/d25/d49/d4e/f66 3406339 0 2024-03-24T00:48:56.587 INFO:tasks.workunit.client.1.smithi088.stdout:3/555: rmdir d3/d6/d27/d3c 39 2024-03-24T00:48:56.590 INFO:tasks.workunit.client.0.smithi038.stdout:0/466: creat d0/d12/d15/d1a/f98 x:0 0 0 2024-03-24T00:48:56.594 INFO:tasks.workunit.client.0.smithi038.stdout:4/513: creat da/d12/d37/d83/dac/fad x:0 0 0 2024-03-24T00:48:56.601 INFO:tasks.workunit.client.0.smithi038.stdout:8/530: write d0/d45/d66/f93 [3605407,68040] 0 2024-03-24T00:48:56.601 INFO:tasks.workunit.client.0.smithi038.stdout:8/531: write d0/d1d/d27/d4d/f75 [1464298,52790] 0 2024-03-24T00:48:56.609 INFO:tasks.workunit.client.1.smithi088.stdout:6/595: rename db/d1d/d50/d90/d56/l59 to db/dd/d87/lb5 0 2024-03-24T00:48:56.609 INFO:tasks.workunit.client.1.smithi088.stdout:3/556: write d3/d6/f2b [211468,102892] 0 2024-03-24T00:48:56.610 INFO:tasks.workunit.client.0.smithi038.stdout:1/630: truncate da/dd/d2c/d5a/f73 90573 0 2024-03-24T00:48:56.611 INFO:tasks.workunit.client.1.smithi088.stdout:7/506: getdents d3/d32/d34/d9c/d4b/d74/d79 0 2024-03-24T00:48:56.613 INFO:tasks.workunit.client.0.smithi038.stdout:3/551: symlink d3/d9/dc/lbb 0 2024-03-24T00:48:56.614 INFO:tasks.workunit.client.1.smithi088.stdout:4/572: creat d5/d13/d63/fc8 x:0 0 0 2024-03-24T00:48:56.619 INFO:tasks.workunit.client.0.smithi038.stdout:9/598: creat d12/d28/d6c/dbc/fc8 x:0 0 0 2024-03-24T00:48:56.622 INFO:tasks.workunit.client.1.smithi088.stdout:0/594: mkdir d9/d26/d8f/da1/d62/dc4 0 2024-03-24T00:48:56.625 INFO:tasks.workunit.client.0.smithi038.stdout:4/514: creat da/d12/d24/d68/d6c/fae x:0 0 0 2024-03-24T00:48:56.625 INFO:tasks.workunit.client.0.smithi038.stdout:4/515: write da/d12/d24/d68/d6c/d7c/f90 [893360,110007] 0 2024-03-24T00:48:56.625 INFO:tasks.workunit.client.0.smithi038.stdout:4/516: truncate da/d12/d29/f98 236710 0 2024-03-24T00:48:56.627 INFO:tasks.workunit.client.0.smithi038.stdout:4/517: read da/d12/d37/f54 [92750,72702] 0 2024-03-24T00:48:56.627 INFO:tasks.workunit.client.0.smithi038.stdout:4/518: fdatasync da/d12/d37/d83/dac/fad 0 2024-03-24T00:48:56.627 INFO:tasks.workunit.client.1.smithi088.stdout:5/600: mknod d1/d10/d81/cc1 0 2024-03-24T00:48:56.634 INFO:tasks.workunit.client.0.smithi038.stdout:8/532: truncate d0/d45/d37/d5f/d68/f91 30426 0 2024-03-24T00:48:56.638 INFO:tasks.workunit.client.1.smithi088.stdout:6/596: symlink db/d1d/d23/d3c/d43/lb6 0 2024-03-24T00:48:56.638 INFO:tasks.workunit.client.1.smithi088.stdout:6/597: fdatasync db/d1d/d50/d90/d56/f30 0 2024-03-24T00:48:56.639 INFO:tasks.workunit.client.0.smithi038.stdout:1/631: mknod da/dd/d7a/d82/d98/d14/cd1 0 2024-03-24T00:48:56.643 INFO:tasks.workunit.client.1.smithi088.stdout:3/557: rmdir d3/d6/d58/da4 39 2024-03-24T00:48:56.645 INFO:tasks.workunit.client.0.smithi038.stdout:8/533: unlink d0/d15/d2c/fb6 0 2024-03-24T00:48:56.646 INFO:tasks.workunit.client.0.smithi038.stdout:8/534: chown d0/d10/d19/d62/c83 20 1 2024-03-24T00:48:56.646 INFO:tasks.workunit.client.0.smithi038.stdout:8/535: read d0/d45/d37/f48 [2068563,108887] 0 2024-03-24T00:48:56.648 INFO:tasks.workunit.client.1.smithi088.stdout:4/573: link d5/d16/d1a/d22/l5f d5/d13/d63/d78/lc9 0 2024-03-24T00:48:56.648 INFO:tasks.workunit.client.1.smithi088.stdout:4/574: write d5/d7/f44 [4939634,116231] 0 2024-03-24T00:48:56.649 INFO:tasks.workunit.client.0.smithi038.stdout:1/632: truncate da/f21 3088123 0 2024-03-24T00:48:56.649 INFO:tasks.workunit.client.0.smithi038.stdout:1/633: write da/dd/d2c/d5a/d74/f88 [2945263,25837] 0 2024-03-24T00:48:56.650 INFO:tasks.workunit.client.0.smithi038.stdout:1/634: chown da/dd/d2c/d41/d60/dae/fc8 2320259 1 2024-03-24T00:48:56.652 INFO:tasks.workunit.client.1.smithi088.stdout:5/601: mkdir d1/d19/d48/dc2 0 2024-03-24T00:48:56.653 INFO:tasks.workunit.client.1.smithi088.stdout:0/595: dread d9/d26/d8f/da1/d62/f6a [0,4194304] 0 2024-03-24T00:48:56.657 INFO:tasks.workunit.client.0.smithi038.stdout:9/599: dread d12/d28/d3a/f6d [0,4194304] 0 2024-03-24T00:48:56.657 INFO:tasks.workunit.client.0.smithi038.stdout:9/600: readlink d12/d55/d56/la4 0 2024-03-24T00:48:56.658 INFO:tasks.workunit.client.1.smithi088.stdout:0/596: mknod d9/dc/de/d7e/d91/cc5 0 2024-03-24T00:48:56.658 INFO:tasks.workunit.client.1.smithi088.stdout:0/597: truncate d9/f1c 1258962 0 2024-03-24T00:48:56.660 INFO:tasks.workunit.client.0.smithi038.stdout:1/635: read da/f35 [705580,124164] 0 2024-03-24T00:48:56.660 INFO:tasks.workunit.client.0.smithi038.stdout:1/636: fdatasync da/dd/d7a/d82/d98/d34/fa5 0 2024-03-24T00:48:56.661 INFO:tasks.workunit.client.1.smithi088.stdout:6/598: symlink db/d1d/d50/d90/d55/d99/lb7 0 2024-03-24T00:48:56.662 INFO:tasks.workunit.client.0.smithi038.stdout:9/601: getdents d12/d28/d3a/d26 0 2024-03-24T00:48:56.664 INFO:tasks.workunit.client.0.smithi038.stdout:4/519: dread da/d12/d29/d5b/d73/f94 [0,4194304] 0 2024-03-24T00:48:56.668 INFO:tasks.workunit.client.1.smithi088.stdout:6/599: symlink db/dd/lb8 0 2024-03-24T00:48:56.670 INFO:tasks.workunit.client.1.smithi088.stdout:3/558: dread d3/de/f16 [0,4194304] 0 2024-03-24T00:48:56.671 INFO:tasks.workunit.client.0.smithi038.stdout:9/602: truncate f4 2421785 0 2024-03-24T00:48:56.671 INFO:tasks.workunit.client.0.smithi038.stdout:9/603: readlink d12/d28/d3a/l23 0 2024-03-24T00:48:56.671 INFO:tasks.workunit.client.0.smithi038.stdout:9/604: stat d12/d28/f51 0 2024-03-24T00:48:56.678 INFO:tasks.workunit.client.1.smithi088.stdout:8/600: rmdir d9 39 2024-03-24T00:48:56.679 INFO:tasks.workunit.client.0.smithi038.stdout:3/552: dwrite d3/d9/f58 [0,4194304] 0 2024-03-24T00:48:56.680 INFO:tasks.workunit.client.0.smithi038.stdout:3/553: chown d3/d30/d5d/d6a 235 1 2024-03-24T00:48:56.682 INFO:tasks.workunit.client.0.smithi038.stdout:8/536: rename d0/d10/d19 to d0/d45/d37/d5f/db7 0 2024-03-24T00:48:56.682 INFO:tasks.workunit.client.0.smithi038.stdout:8/537: dread - d0/d1d/d27/d6b/fa1 zero size 2024-03-24T00:48:56.684 INFO:tasks.workunit.client.1.smithi088.stdout:5/602: dread d1/d10/d1b/d2f/d57/d9e/fab [0,4194304] 0 2024-03-24T00:48:56.684 INFO:tasks.workunit.client.1.smithi088.stdout:5/603: write d1/f3b [8491111,77207] 0 2024-03-24T00:48:56.684 INFO:tasks.workunit.client.1.smithi088.stdout:5/604: write d1/d19/f2c [101641,36715] 0 2024-03-24T00:48:56.684 INFO:tasks.workunit.client.1.smithi088.stdout:5/605: write d1/dd/d6f/da1/d3f/d88/d8f/d69/f85 [1063984,53041] 0 2024-03-24T00:48:56.684 INFO:tasks.workunit.client.1.smithi088.stdout:3/559: mknod d3/d6/d58/cb4 0 2024-03-24T00:48:56.685 INFO:tasks.workunit.client.1.smithi088.stdout:8/601: truncate d9/dc/d11/d24/d29/d39/d84/f6e 3158007 0 2024-03-24T00:48:56.685 INFO:tasks.workunit.client.1.smithi088.stdout:8/602: fsync d9/f3e 0 2024-03-24T00:48:56.691 INFO:tasks.workunit.client.1.smithi088.stdout:5/606: mkdir d1/d10/d1b/d2f/d57/d78/dc3 0 2024-03-24T00:48:56.692 INFO:tasks.workunit.client.1.smithi088.stdout:5/607: read d1/dd/f15 [34063,128743] 0 2024-03-24T00:48:56.692 INFO:tasks.workunit.client.1.smithi088.stdout:3/560: chown d3/d1a/d1e/d76/f84 74 1 2024-03-24T00:48:56.694 INFO:tasks.workunit.client.1.smithi088.stdout:9/607: symlink d1/d22/d27/d39/lc4 0 2024-03-24T00:48:56.695 INFO:tasks.workunit.client.1.smithi088.stdout:8/603: symlink d9/dc/d11/d41/ld0 0 2024-03-24T00:48:56.699 INFO:tasks.workunit.client.1.smithi088.stdout:3/561: getdents d3/d1a/d1e/d2f 0 2024-03-24T00:48:56.699 INFO:tasks.workunit.client.1.smithi088.stdout:3/562: fdatasync d3/de/f29 0 2024-03-24T00:48:56.703 INFO:tasks.workunit.client.1.smithi088.stdout:9/608: dread d1/d15/d21/f66 [0,4194304] 0 2024-03-24T00:48:56.704 INFO:tasks.workunit.client.0.smithi038.stdout:3/554: write f1 [5977103,129110] 0 2024-03-24T00:48:56.704 INFO:tasks.workunit.client.1.smithi088.stdout:0/598: unlink d9/dc/de/d19/d44/d46/d65/l43 0 2024-03-24T00:48:56.711 INFO:tasks.workunit.client.1.smithi088.stdout:0/599: symlink d9/dc/de/d19/d44/d46/d65/d2a/lc6 0 2024-03-24T00:48:56.711 INFO:tasks.workunit.client.0.smithi038.stdout:0/467: dwrite d0/d14/f5c [0,4194304] 0 2024-03-24T00:48:56.715 INFO:tasks.workunit.client.1.smithi088.stdout:9/609: truncate d1/f7 2491632 0 2024-03-24T00:48:56.720 INFO:tasks.workunit.client.1.smithi088.stdout:0/600: mkdir d9/d26/d8f/dc7 0 2024-03-24T00:48:56.721 INFO:tasks.workunit.client.1.smithi088.stdout:8/604: dread d9/dc/d11/d24/d29/d39/f93 [0,4194304] 0 2024-03-24T00:48:56.725 INFO:tasks.workunit.client.0.smithi038.stdout:3/555: getdents d3/d9/d77 0 2024-03-24T00:48:56.725 INFO:tasks.workunit.client.0.smithi038.stdout:3/556: chown d3/d9/dc/da6/db5/lb9 10603966 1 2024-03-24T00:48:56.727 INFO:tasks.workunit.client.0.smithi038.stdout:4/520: symlink da/d12/laf 0 2024-03-24T00:48:56.731 INFO:tasks.workunit.client.1.smithi088.stdout:1/585: rename d6/l8f to d6/d23/daf/d95/d3f/d53/lb2 0 2024-03-24T00:48:56.734 INFO:tasks.workunit.client.1.smithi088.stdout:9/610: creat d1/d10/d29/fc5 x:0 0 0 2024-03-24T00:48:56.737 INFO:tasks.workunit.client.1.smithi088.stdout:1/586: creat d6/d27/fb3 x:0 0 0 2024-03-24T00:48:56.739 INFO:tasks.workunit.client.0.smithi038.stdout:3/557: mkdir d3/d30/d5d/d7c/dbc 0 2024-03-24T00:48:56.739 INFO:tasks.workunit.client.0.smithi038.stdout:3/558: truncate d3/d9/fb3 985670 0 2024-03-24T00:48:56.739 INFO:tasks.workunit.client.0.smithi038.stdout:3/559: write d3/d30/fa9 [983720,15322] 0 2024-03-24T00:48:56.739 INFO:tasks.workunit.client.0.smithi038.stdout:3/560: chown d3/d9/dc/df/c43 139279364 1 2024-03-24T00:48:56.741 INFO:tasks.workunit.client.1.smithi088.stdout:7/507: rename d3/c18 to d3/d32/d34/d9c/d4b/d74/d79/d7f/c9d 0 2024-03-24T00:48:56.742 INFO:tasks.workunit.client.1.smithi088.stdout:4/575: dwrite d5/d16/f29 [0,4194304] 0 2024-03-24T00:48:56.742 INFO:tasks.workunit.client.1.smithi088.stdout:6/600: dwrite db/d1d/d23/d3c/d4b/f6e [0,4194304] 0 2024-03-24T00:48:56.742 INFO:tasks.workunit.client.1.smithi088.stdout:6/601: truncate db/d1d/d50/f77 1149386 0 2024-03-24T00:48:56.743 INFO:tasks.workunit.client.0.smithi038.stdout:4/521: mknod da/d12/d37/d83/dac/cb0 0 2024-03-24T00:48:56.743 INFO:tasks.workunit.client.0.smithi038.stdout:4/522: truncate da/d12/d24/d5f/d8a/da4/faa 274659 0 2024-03-24T00:48:56.743 INFO:tasks.workunit.client.0.smithi038.stdout:9/605: dwrite d12/d1a/d80/f83 [0,4194304] 0 2024-03-24T00:48:56.744 INFO:tasks.workunit.client.1.smithi088.stdout:9/611: unlink d1/d10/d29/d85/f87 0 2024-03-24T00:48:56.747 INFO:tasks.workunit.client.1.smithi088.stdout:1/587: dread d6/d27/d67/d88/fa1 [0,4194304] 0 2024-03-24T00:48:56.748 INFO:tasks.workunit.client.1.smithi088.stdout:5/608: rename d1/d10/d1b/c73 to d1/d10/d1b/d2f/d57/d9e/db1/cc4 0 2024-03-24T00:48:56.750 INFO:tasks.workunit.client.0.smithi038.stdout:0/468: truncate d0/d12/d60/d28/f48 2022396 0 2024-03-24T00:48:56.753 INFO:tasks.workunit.client.0.smithi038.stdout:2/509: sync 2024-03-24T00:48:56.756 INFO:tasks.workunit.client.1.smithi088.stdout:6/602: symlink db/dd/d89/lb9 0 2024-03-24T00:48:56.758 INFO:tasks.workunit.client.0.smithi038.stdout:4/523: creat da/d12/d29/fb1 x:0 0 0 2024-03-24T00:48:56.760 INFO:tasks.workunit.client.1.smithi088.stdout:9/612: mknod d1/d10/d2c/cc6 0 2024-03-24T00:48:56.760 INFO:tasks.workunit.client.1.smithi088.stdout:9/613: truncate d1/d15/d80/da5/fb2 98510 0 2024-03-24T00:48:56.761 INFO:tasks.workunit.client.1.smithi088.stdout:4/576: dread d5/d13/f12 [0,4194304] 0 2024-03-24T00:48:56.761 INFO:tasks.workunit.client.1.smithi088.stdout:4/577: chown d5/d7/d72/c3b 33884721 1 2024-03-24T00:48:56.761 INFO:tasks.workunit.client.0.smithi038.stdout:3/561: dread f1 [4194304,4194304] 0 2024-03-24T00:48:56.763 INFO:tasks.workunit.client.0.smithi038.stdout:9/606: rename d12/d1a/d80/f97 to d12/d28/d71/d99/fc9 0 2024-03-24T00:48:56.768 INFO:tasks.workunit.client.1.smithi088.stdout:4/578: symlink d5/d16/d37/lca 0 2024-03-24T00:48:56.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:56 smithi038 ceph-mon[29271]: pgmap v34: 65 pgs: 65 active+clean; 2.2 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 225 MiB/s rd, 279 MiB/s wr, 398 op/s 2024-03-24T00:48:56.773 INFO:tasks.workunit.client.0.smithi038.stdout:0/469: rmdir d0/d12/d42/d5e 39 2024-03-24T00:48:56.774 INFO:tasks.workunit.client.0.smithi038.stdout:3/562: write d3/d30/f34 [144526,110862] 0 2024-03-24T00:48:56.774 INFO:tasks.workunit.client.0.smithi038.stdout:2/510: write d4/d27/f2a [2894661,17813] 0 2024-03-24T00:48:56.774 INFO:tasks.workunit.client.1.smithi088.stdout:7/508: write d3/f29 [3363479,28714] 0 2024-03-24T00:48:56.776 INFO:tasks.workunit.client.1.smithi088.stdout:3/563: rename d3/d6/d27/d56 to d3/d1a/d1e/d2f/d45/db5 0 2024-03-24T00:48:56.776 INFO:tasks.workunit.client.1.smithi088.stdout:3/564: read d3/d1a/d1e/f28 [1992204,20179] 0 2024-03-24T00:48:56.778 INFO:tasks.workunit.client.0.smithi038.stdout:9/607: link d12/d1a/f20 d12/d28/d3a/fca 0 2024-03-24T00:48:56.778 INFO:tasks.workunit.client.0.smithi038.stdout:9/608: fdatasync d12/d55/f9b 0 2024-03-24T00:48:56.779 INFO:tasks.workunit.client.0.smithi038.stdout:9/609: write d12/d28/d71/d99/fc6 [67914,37075] 0 2024-03-24T00:48:56.779 INFO:tasks.workunit.client.0.smithi038.stdout:9/610: chown d12/d55/d56/la4 0 1 2024-03-24T00:48:56.780 INFO:tasks.workunit.client.1.smithi088.stdout:4/579: mknod d5/d13/d18/d24/db3/ccb 0 2024-03-24T00:48:56.781 INFO:tasks.workunit.client.0.smithi038.stdout:8/538: truncate d0/d45/d37/d5f/d68/f91 396251 0 2024-03-24T00:48:56.781 INFO:tasks.workunit.client.0.smithi038.stdout:8/539: chown d0/d10/c96 63490 1 2024-03-24T00:48:56.781 INFO:tasks.workunit.client.0.smithi038.stdout:8/540: chown d0/c14 232453 1 2024-03-24T00:48:56.781 INFO:tasks.workunit.client.0.smithi038.stdout:8/541: fdatasync d0/d45/d37/f61 0 2024-03-24T00:48:56.781 INFO:tasks.workunit.client.0.smithi038.stdout:8/542: truncate d0/d45/d37/d5f/db7/d62/fad 1918003 0 2024-03-24T00:48:56.781 INFO:tasks.workunit.client.0.smithi038.stdout:8/543: chown d0/d45/d66/da6 24599 1 2024-03-24T00:48:56.782 INFO:tasks.workunit.client.0.smithi038.stdout:0/470: link d0/d12/d42/d44/f93 d0/d12/d15/d1a/f99 0 2024-03-24T00:48:56.783 INFO:tasks.workunit.client.0.smithi038.stdout:1/637: mknod da/dd/d2c/d5a/cd2 0 2024-03-24T00:48:56.786 INFO:tasks.workunit.client.0.smithi038.stdout:3/563: link d3/f49 d3/d9/dc/da6/db5/dad/fbd 0 2024-03-24T00:48:56.787 INFO:tasks.workunit.client.1.smithi088.stdout:3/565: dread d3/de/d6f/d85/f15 [0,4194304] 0 2024-03-24T00:48:56.790 INFO:tasks.workunit.client.1.smithi088.stdout:0/601: rename d9/d26/d8f/da1/f72 to d9/d26/d8f/da1/d62/d9b/db0/db4/fc8 0 2024-03-24T00:48:56.794 INFO:tasks.workunit.client.0.smithi038.stdout:2/511: rmdir d4/d18/d26/d37/d69/d83/dad 0 2024-03-24T00:48:56.794 INFO:tasks.workunit.client.0.smithi038.stdout:2/512: write d4/d27/f47 [8125566,69672] 0 2024-03-24T00:48:56.797 INFO:tasks.workunit.client.0.smithi038.stdout:9/611: rmdir d12/d28/d3a/d8f 39 2024-03-24T00:48:56.800 INFO:tasks.workunit.client.0.smithi038.stdout:8/544: mkdir d0/d45/db8 0 2024-03-24T00:48:56.802 INFO:tasks.workunit.client.1.smithi088.stdout:9/614: write d1/d15/f7f [4011452,51285] 0 2024-03-24T00:48:56.802 INFO:tasks.workunit.client.1.smithi088.stdout:7/509: dread d3/d32/d34/f72 [0,4194304] 0 2024-03-24T00:48:56.802 INFO:tasks.workunit.client.1.smithi088.stdout:1/588: dwrite d6/d23/daf/d95/d58/f51 [4194304,4194304] 0 2024-03-24T00:48:56.803 INFO:tasks.workunit.client.0.smithi038.stdout:0/471: mkdir d0/d12/d9a 0 2024-03-24T00:48:56.804 INFO:tasks.workunit.client.1.smithi088.stdout:3/566: rmdir d3/d1a/d1e/d2f 39 2024-03-24T00:48:56.804 INFO:tasks.workunit.client.1.smithi088.stdout:8/605: creat d9/dc/fd1 x:0 0 0 2024-03-24T00:48:56.804 INFO:tasks.workunit.client.1.smithi088.stdout:0/602: symlink d9/dc/de/d19/d44/lc9 0 2024-03-24T00:48:56.805 INFO:tasks.workunit.client.1.smithi088.stdout:8/606: write d9/dc/d11/d24/d29/d39/d84/faa [118281,109692] 0 2024-03-24T00:48:56.805 INFO:tasks.workunit.client.1.smithi088.stdout:5/609: rename d1/dd/d6f/da1/d3f/d88/d8b/d94/daf to d1/d10/d79/dc5 0 2024-03-24T00:48:56.807 INFO:tasks.workunit.client.0.smithi038.stdout:2/513: creat d4/d55/d88/fb9 x:0 0 0 2024-03-24T00:48:56.807 INFO:tasks.workunit.client.0.smithi038.stdout:2/514: chown d4/d18/d26/d37/d69 104258981 1 2024-03-24T00:48:56.810 INFO:tasks.workunit.client.1.smithi088.stdout:9/615: creat d1/d15/d21/d9c/fc7 x:0 0 0 2024-03-24T00:48:56.811 INFO:tasks.workunit.client.0.smithi038.stdout:9/612: symlink d12/d55/lcb 0 2024-03-24T00:48:56.813 INFO:tasks.workunit.client.1.smithi088.stdout:7/510: chown d3/d31/d56/f7b 156 1 2024-03-24T00:48:56.813 INFO:tasks.workunit.client.1.smithi088.stdout:7/511: truncate d3/d32/d76/f81 77575 0 2024-03-24T00:48:56.813 INFO:tasks.workunit.client.1.smithi088.stdout:7/512: chown d3/d32/d34/d7c/c8a 796195606 1 2024-03-24T00:48:56.813 INFO:tasks.workunit.client.1.smithi088.stdout:7/513: read - d3/d32/d34/f96 zero size 2024-03-24T00:48:56.813 INFO:tasks.workunit.client.1.smithi088.stdout:7/514: write d3/d32/d34/d9c/d4b/f99 [529102,96356] 0 2024-03-24T00:48:56.814 INFO:tasks.workunit.client.0.smithi038.stdout:8/545: mknod d0/d1d/d20/d3c/d9e/cb9 0 2024-03-24T00:48:56.814 INFO:tasks.workunit.client.0.smithi038.stdout:8/546: chown d0/cf 171 1 2024-03-24T00:48:56.815 INFO:tasks.workunit.client.1.smithi088.stdout:1/589: creat d6/fb4 x:0 0 0 2024-03-24T00:48:56.818 INFO:tasks.workunit.client.1.smithi088.stdout:3/567: creat d3/d6/d58/fb6 x:0 0 0 2024-03-24T00:48:56.820 INFO:tasks.workunit.client.0.smithi038.stdout:0/472: creat d0/d12/d60/d28/f9b x:0 0 0 2024-03-24T00:48:56.820 INFO:tasks.workunit.client.0.smithi038.stdout:0/473: chown d0/d12/c19 3061561 1 2024-03-24T00:48:56.820 INFO:tasks.workunit.client.0.smithi038.stdout:0/474: chown d0/d12/d60/d28/c4f 635085 1 2024-03-24T00:48:56.824 INFO:tasks.workunit.client.1.smithi088.stdout:6/603: rename db/c5f to db/d1d/d50/d90/d55/cba 0 2024-03-24T00:48:56.825 INFO:tasks.workunit.client.1.smithi088.stdout:2/572: sync 2024-03-24T00:48:56.825 INFO:tasks.workunit.client.0.smithi038.stdout:3/564: getdents d3/d9 0 2024-03-24T00:48:56.827 INFO:tasks.workunit.client.1.smithi088.stdout:5/610: mknod d1/d10/d81/cc6 0 2024-03-24T00:48:56.829 INFO:tasks.workunit.client.0.smithi038.stdout:9/613: creat d12/d41/fcc x:0 0 0 2024-03-24T00:48:56.829 INFO:tasks.workunit.client.0.smithi038.stdout:9/614: chown f6 112260 1 2024-03-24T00:48:56.832 INFO:tasks.workunit.client.1.smithi088.stdout:0/603: link d9/dc/de/d19/d44/d46/d65/d2a/d2e/f5f d9/dc/de/d19/d44/d46/d65/d2a/d2e/fca 0 2024-03-24T00:48:56.832 INFO:tasks.workunit.client.1.smithi088.stdout:0/604: chown d9/d26/d8f/da1/d62/d9b/db0/db4 186 1 2024-03-24T00:48:56.834 INFO:tasks.workunit.client.0.smithi038.stdout:8/547: mknod d0/d1d/d8c/cba 0 2024-03-24T00:48:56.834 INFO:tasks.workunit.client.0.smithi038.stdout:8/548: creat d0/d1d/d27/d6b/fbb x:0 0 0 2024-03-24T00:48:56.834 INFO:tasks.workunit.client.0.smithi038.stdout:8/549: readlink d0/d1d/d27/l9d 0 2024-03-24T00:48:56.834 INFO:tasks.workunit.client.0.smithi038.stdout:8/550: fsync d0/d1d/d20/f23 0 2024-03-24T00:48:56.834 INFO:tasks.workunit.client.0.smithi038.stdout:8/551: read - d0/d1d/d27/d6b/fbb zero size 2024-03-24T00:48:56.835 INFO:tasks.workunit.client.0.smithi038.stdout:4/524: getdents da/d12 0 2024-03-24T00:48:56.835 INFO:tasks.workunit.client.0.smithi038.stdout:4/525: creat da/d12/d29/d5b/d73/fb2 x:0 0 0 2024-03-24T00:48:56.836 INFO:tasks.workunit.client.1.smithi088.stdout:4/580: dwrite d5/d7/f44 [0,4194304] 0 2024-03-24T00:48:56.837 INFO:tasks.workunit.client.0.smithi038.stdout:0/475: creat d0/d12/d9a/f9c x:0 0 0 2024-03-24T00:48:56.837 INFO:tasks.workunit.client.0.smithi038.stdout:4/526: write da/d12/d37/dd/d15/d4c/f6a [1174496,18363] 0 2024-03-24T00:48:56.842 INFO:tasks.workunit.client.1.smithi088.stdout:3/568: mknod d3/de/cb7 0 2024-03-24T00:48:56.845 INFO:tasks.workunit.client.0.smithi038.stdout:8/552: link d0/d45/d66/da6/fae d0/d45/d66/d6d/fbc 0 2024-03-24T00:48:56.846 INFO:tasks.workunit.client.1.smithi088.stdout:8/607: mkdir d9/dc/d11/dd2 0 2024-03-24T00:48:56.846 INFO:tasks.workunit.client.1.smithi088.stdout:8/608: stat d9/dc/d11/d24/d43/f50 0 2024-03-24T00:48:56.848 INFO:tasks.workunit.client.0.smithi038.stdout:3/565: dread d3/d9/dc/df/f3a [0,4194304] 0 2024-03-24T00:48:56.848 INFO:tasks.workunit.client.0.smithi038.stdout:3/566: truncate d3/d9/dc/d51/f75 4919399 0 2024-03-24T00:48:56.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/527: dread da/d12/d24/d68/d6c/f79 [0,4194304] 0 2024-03-24T00:48:56.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/528: truncate da/d12/d37/dd/d15/d4c/f6a 1940787 0 2024-03-24T00:48:56.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/529: chown da/d12/d24/c2f 7791 1 2024-03-24T00:48:56.853 INFO:tasks.workunit.client.1.smithi088.stdout:6/604: creat db/d1d/d23/fbb x:0 0 0 2024-03-24T00:48:56.854 INFO:tasks.workunit.client.0.smithi038.stdout:3/567: write d3/d30/d5d/d6a/f76 [657858,112892] 0 2024-03-24T00:48:56.854 INFO:tasks.workunit.client.0.smithi038.stdout:3/568: truncate d3/d5a/fa5 549164 0 2024-03-24T00:48:56.854 INFO:tasks.workunit.client.1.smithi088.stdout:5/611: creat d1/d43/d4e/fc7 x:0 0 0 2024-03-24T00:48:56.857 INFO:tasks.workunit.client.0.smithi038.stdout:8/553: truncate d0/d1d/d20/f58 7507567 0 2024-03-24T00:48:56.859 INFO:tasks.workunit.client.1.smithi088.stdout:0/605: mkdir d9/d26/d8f/da1/d62/d9b/dcb 0 2024-03-24T00:48:56.859 INFO:tasks.workunit.client.1.smithi088.stdout:0/606: truncate d9/dc/de/d19/d44/d46/d65/d2a/f75 2282972 0 2024-03-24T00:48:56.862 INFO:tasks.workunit.client.0.smithi038.stdout:8/554: creat d0/d15/fbd x:0 0 0 2024-03-24T00:48:56.863 INFO:tasks.workunit.client.1.smithi088.stdout:9/616: getdents d1/d15/d80 0 2024-03-24T00:48:56.863 INFO:tasks.workunit.client.1.smithi088.stdout:9/617: creat d1/d15/d21/fc8 x:0 0 0 2024-03-24T00:48:56.867 INFO:tasks.workunit.client.1.smithi088.stdout:7/515: rename d3/f30 to d3/d31/f9e 0 2024-03-24T00:48:56.867 INFO:tasks.workunit.client.1.smithi088.stdout:6/605: unlink db/d1d/d23/l53 0 2024-03-24T00:48:56.868 INFO:tasks.workunit.client.1.smithi088.stdout:6/606: readlink db/d1d/d50/d90/d55/l82 0 2024-03-24T00:48:56.868 INFO:tasks.workunit.client.1.smithi088.stdout:6/607: write db/d1d/d50/d90/d56/f30 [2117656,28630] 0 2024-03-24T00:48:56.870 INFO:tasks.workunit.client.0.smithi038.stdout:8/555: unlink d0/d1d/d27/d4d/ca5 0 2024-03-24T00:48:56.879 INFO:tasks.workunit.client.1.smithi088.stdout:5/612: link d1/d19/fc0 d1/d43/d4e/d5b/fc8 0 2024-03-24T00:48:56.882 INFO:tasks.workunit.client.0.smithi038.stdout:9/615: fsync d12/d28/d3a/fca 0 2024-03-24T00:48:56.887 INFO:tasks.workunit.client.0.smithi038.stdout:7/516: sync 2024-03-24T00:48:56.890 INFO:tasks.workunit.client.0.smithi038.stdout:6/591: sync 2024-03-24T00:48:56.891 INFO:tasks.workunit.client.0.smithi038.stdout:5/534: sync 2024-03-24T00:48:56.891 INFO:tasks.workunit.client.0.smithi038.stdout:5/535: fdatasync d6/d60/f9a 0 2024-03-24T00:48:56.892 INFO:tasks.workunit.client.0.smithi038.stdout:8/556: dread d0/d45/d37/d5f/d68/f91 [0,4194304] 0 2024-03-24T00:48:56.894 INFO:tasks.workunit.client.1.smithi088.stdout:3/569: dwrite d3/d6/d27/d3c/f42 [4194304,4194304] 0 2024-03-24T00:48:56.895 INFO:tasks.workunit.client.0.smithi038.stdout:7/517: link d2/d2c/l42 d2/d9/da/dc/d8a/db8/lc3 0 2024-03-24T00:48:56.900 INFO:tasks.workunit.client.1.smithi088.stdout:4/581: dwrite d5/d17/f83 [0,4194304] 0 2024-03-24T00:48:56.900 INFO:tasks.workunit.client.0.smithi038.stdout:9/616: link d12/d1a/d5f/c65 d12/d28/ccd 0 2024-03-24T00:48:56.905 INFO:tasks.workunit.client.0.smithi038.stdout:6/592: mkdir d1/da/d1b/dbb 0 2024-03-24T00:48:56.907 INFO:tasks.workunit.client.1.smithi088.stdout:9/618: rmdir d1/d10/d77/d8a 39 2024-03-24T00:48:56.909 INFO:tasks.workunit.client.0.smithi038.stdout:8/557: getdents d0/d1d/d20/d3c/d9e/da3 0 2024-03-24T00:48:56.912 INFO:tasks.workunit.client.1.smithi088.stdout:1/590: dwrite d6/d23/daf/d95/d5b/d5d/f76 [0,4194304] 0 2024-03-24T00:48:56.912 INFO:tasks.workunit.client.1.smithi088.stdout:2/573: dread db/dd/d96/d4e/f64 [0,4194304] 0 2024-03-24T00:48:56.912 INFO:tasks.workunit.client.1.smithi088.stdout:2/574: fdatasync db/d21/f2e 0 2024-03-24T00:48:56.915 INFO:tasks.workunit.client.1.smithi088.stdout:3/570: truncate d3/d6/d27/d3c/f47 3605126 0 2024-03-24T00:48:56.915 INFO:tasks.workunit.client.1.smithi088.stdout:3/571: fsync d3/de/d6f/f75 0 2024-03-24T00:48:56.916 INFO:tasks.workunit.client.0.smithi038.stdout:7/518: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/cc4 0 2024-03-24T00:48:56.924 INFO:tasks.workunit.client.0.smithi038.stdout:9/617: creat d12/d28/d3a/fce x:0 0 0 2024-03-24T00:48:56.924 INFO:tasks.workunit.client.0.smithi038.stdout:9/618: chown d12/d28/d3a 110581 1 2024-03-24T00:48:56.924 INFO:tasks.workunit.client.0.smithi038.stdout:9/619: chown d12/d28/d3a/d26/d79/f92 3486 1 2024-03-24T00:48:56.925 INFO:tasks.workunit.client.1.smithi088.stdout:6/608: dwrite db/d1d/d23/d5c/f95 [0,4194304] 0 2024-03-24T00:48:56.925 INFO:tasks.workunit.client.1.smithi088.stdout:6/609: chown db/dd/l14 2024 1 2024-03-24T00:48:56.927 INFO:tasks.workunit.client.0.smithi038.stdout:6/593: creat d1/da/d1b/d25/fbc x:0 0 0 2024-03-24T00:48:56.927 INFO:tasks.workunit.client.0.smithi038.stdout:6/594: chown d1/da/d73/d75/f9f 0 1 2024-03-24T00:48:56.929 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:56 smithi088 ceph-mon[36610]: pgmap v34: 65 pgs: 65 active+clean; 2.2 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 225 MiB/s rd, 279 MiB/s wr, 398 op/s 2024-03-24T00:48:56.930 INFO:tasks.workunit.client.1.smithi088.stdout:4/582: link d5/d7/f14 d5/d7/fcc 0 2024-03-24T00:48:56.933 INFO:tasks.workunit.client.0.smithi038.stdout:3/569: dwrite d3/d9/dc/da6/db5/dad/fbd [0,4194304] 0 2024-03-24T00:48:56.933 INFO:tasks.workunit.client.0.smithi038.stdout:9/620: dread d12/d28/f70 [0,4194304] 0 2024-03-24T00:48:56.934 INFO:tasks.workunit.client.0.smithi038.stdout:3/570: write d3/d2a/f59 [1369616,105091] 0 2024-03-24T00:48:56.936 INFO:tasks.workunit.client.1.smithi088.stdout:0/607: dwrite d9/dc/de/d19/d44/fa4 [0,4194304] 0 2024-03-24T00:48:56.937 INFO:tasks.workunit.client.0.smithi038.stdout:8/558: symlink d0/d1d/d27/d4d/d81/lbe 0 2024-03-24T00:48:56.937 INFO:tasks.workunit.client.0.smithi038.stdout:8/559: chown d0/d45/d37/d5f/d68 4 1 2024-03-24T00:48:56.937 INFO:tasks.workunit.client.0.smithi038.stdout:8/560: write d0/d1d/d27/d4d/f7a [575967,95948] 0 2024-03-24T00:48:56.942 INFO:tasks.workunit.client.0.smithi038.stdout:7/519: mkdir d2/d9/da/dc/da8/dc5 0 2024-03-24T00:48:56.942 INFO:tasks.workunit.client.1.smithi088.stdout:8/609: rename d9/dc/d11/d41/d56 to d9/dc/d11/d24/d29/d5b/d8b/dd3 0 2024-03-24T00:48:56.942 INFO:tasks.workunit.client.1.smithi088.stdout:7/516: dwrite d3/f2c [0,4194304] 0 2024-03-24T00:48:56.944 INFO:tasks.workunit.client.0.smithi038.stdout:9/621: dread d12/f1e [0,4194304] 0 2024-03-24T00:48:56.945 INFO:tasks.workunit.client.0.smithi038.stdout:3/571: dread d3/d9/dc/da6/db5/d19/f42 [0,4194304] 0 2024-03-24T00:48:56.946 INFO:tasks.workunit.client.0.smithi038.stdout:0/476: rename d0/d12/d60 to d0/d12/d42/d9d 0 2024-03-24T00:48:56.946 INFO:tasks.workunit.client.0.smithi038.stdout:0/477: chown d0/d12/d42/d9d/l66 25 1 2024-03-24T00:48:56.946 INFO:tasks.workunit.client.0.smithi038.stdout:0/478: dread - d0/d12/d42/d9d/f90 zero size 2024-03-24T00:48:56.949 INFO:tasks.workunit.client.0.smithi038.stdout:6/595: symlink d1/da/d1b/dbb/lbd 0 2024-03-24T00:48:56.949 INFO:tasks.workunit.client.0.smithi038.stdout:6/596: truncate d1/da/d1b/d25/d49/d4e/d51/f5e 317969 0 2024-03-24T00:48:56.955 INFO:tasks.workunit.client.1.smithi088.stdout:8/610: dread d9/dc/d11/d80/db0/f90 [0,4194304] 0 2024-03-24T00:48:56.955 INFO:tasks.workunit.client.1.smithi088.stdout:8/611: chown d9/dc/d11/d24/d43/l72 53799910 1 2024-03-24T00:48:56.955 INFO:tasks.workunit.client.1.smithi088.stdout:8/612: creat d9/dc/d11/d41/d54/d5d/db3/fd4 x:0 0 0 2024-03-24T00:48:56.958 INFO:tasks.workunit.client.1.smithi088.stdout:9/619: dwrite d1/d10/d11/f5f [0,4194304] 0 2024-03-24T00:48:56.959 INFO:tasks.workunit.client.0.smithi038.stdout:5/536: dwrite d6/d19/d20/d42/d43/d4c/f74 [0,4194304] 0 2024-03-24T00:48:56.967 INFO:tasks.workunit.client.0.smithi038.stdout:7/520: unlink d2/d9/d16/l60 0 2024-03-24T00:48:56.969 INFO:tasks.workunit.client.0.smithi038.stdout:3/572: link d3/d5a/c92 d3/d30/d5d/d7c/dbc/cbe 0 2024-03-24T00:48:56.974 INFO:tasks.workunit.client.1.smithi088.stdout:0/608: creat d9/dc/de/d19/d44/d46/d65/d2a/fcc x:0 0 0 2024-03-24T00:48:56.975 INFO:tasks.workunit.client.0.smithi038.stdout:0/479: creat d0/d12/d42/d9d/f9e x:0 0 0 2024-03-24T00:48:56.977 INFO:tasks.workunit.client.0.smithi038.stdout:4/530: rename da/d12/d37/dd/f3a to da/d12/d37/dd/d15/fb3 0 2024-03-24T00:48:56.979 INFO:tasks.workunit.client.1.smithi088.stdout:0/609: dread d9/d26/d8f/fad [0,4194304] 0 2024-03-24T00:48:56.988 INFO:tasks.workunit.client.0.smithi038.stdout:5/537: write d6/d23/f32 [20929,102964] 0 2024-03-24T00:48:56.989 INFO:tasks.workunit.client.1.smithi088.stdout:7/517: rmdir d3/d32/d34/d9c 39 2024-03-24T00:48:56.989 INFO:tasks.workunit.client.0.smithi038.stdout:9/622: write d12/d28/d6c/f74 [2090818,12585] 0 2024-03-24T00:48:56.995 INFO:tasks.workunit.client.1.smithi088.stdout:5/613: rename d1/dd/f54 to d1/d19/d48/dc2/fc9 0 2024-03-24T00:48:56.995 INFO:tasks.workunit.client.1.smithi088.stdout:5/614: creat d1/d10/d1b/d2f/d57/fca x:0 0 0 2024-03-24T00:48:56.995 INFO:tasks.workunit.client.1.smithi088.stdout:5/615: dread - d1/dd/d6f/da1/d3f/f41 zero size 2024-03-24T00:48:56.995 INFO:tasks.workunit.client.1.smithi088.stdout:5/616: readlink d1/dd/d6f/da1/d3f/l86 0 2024-03-24T00:48:56.996 INFO:tasks.workunit.client.1.smithi088.stdout:5/617: read - d1/d10/d1b/d2f/d57/fa5 zero size 2024-03-24T00:48:57.000 INFO:tasks.workunit.client.1.smithi088.stdout:8/613: symlink d9/dc/d11/d24/d43/ld5 0 2024-03-24T00:48:57.014 INFO:tasks.workunit.client.1.smithi088.stdout:9/620: link d1/d22/d27/d39/f58 d1/d10/d29/db0/fc9 0 2024-03-24T00:48:57.015 INFO:tasks.workunit.client.0.smithi038.stdout:6/597: dwrite d1/d37/fa5 [0,4194304] 0 2024-03-24T00:48:57.019 INFO:tasks.workunit.client.0.smithi038.stdout:4/531: mkdir da/d12/d29/d5b/d73/db4 0 2024-03-24T00:48:57.020 INFO:tasks.workunit.client.0.smithi038.stdout:4/532: creat da/d27/fb5 x:0 0 0 2024-03-24T00:48:57.023 INFO:tasks.workunit.client.1.smithi088.stdout:6/610: rmdir db/dd/d61/d6d/d74 39 2024-03-24T00:48:57.023 INFO:tasks.workunit.client.1.smithi088.stdout:6/611: creat db/d8e/fbc x:0 0 0 2024-03-24T00:48:57.025 INFO:tasks.workunit.client.0.smithi038.stdout:7/521: rename d2/d9/da/d23/d2b/d5b/d72/d92/f58 to d2/d9/d16/fc6 0 2024-03-24T00:48:57.031 INFO:tasks.workunit.client.1.smithi088.stdout:0/610: dwrite d9/dc/de/d19/d44/f5b [0,4194304] 0 2024-03-24T00:48:57.031 INFO:tasks.workunit.client.1.smithi088.stdout:0/611: dread - d9/d26/d8f/da1/d62/f9d zero size 2024-03-24T00:48:57.031 INFO:tasks.workunit.client.1.smithi088.stdout:0/612: write d9/d26/d8f/da1/fc2 [668657,81922] 0 2024-03-24T00:48:57.034 INFO:tasks.workunit.client.1.smithi088.stdout:7/518: dread - d3/f8c zero size 2024-03-24T00:48:57.035 INFO:tasks.workunit.client.0.smithi038.stdout:9/623: symlink d12/d28/d3a/lcf 0 2024-03-24T00:48:57.038 INFO:tasks.workunit.client.0.smithi038.stdout:8/561: write d0/d15/f98 [3932067,10514] 0 2024-03-24T00:48:57.040 INFO:tasks.workunit.client.1.smithi088.stdout:1/591: rename d6/d6e/c74 to d6/d23/daf/d95/d5b/d5d/d61/d7d/da5/cb5 0 2024-03-24T00:48:57.041 INFO:tasks.workunit.client.0.smithi038.stdout:9/624: write d12/f42 [397714,123182] 0 2024-03-24T00:48:57.041 INFO:tasks.workunit.client.0.smithi038.stdout:9/625: stat d12/d1a/d5f/c8d 0 2024-03-24T00:48:57.046 INFO:tasks.workunit.client.0.smithi038.stdout:7/522: dread d2/d2c/f69 [0,4194304] 0 2024-03-24T00:48:57.046 INFO:tasks.workunit.client.0.smithi038.stdout:7/523: fdatasync d2/d9/da/dc/d4a/f67 0 2024-03-24T00:48:57.046 INFO:tasks.workunit.client.0.smithi038.stdout:7/524: write d2/f4 [1919411,14269] 0 2024-03-24T00:48:57.049 INFO:tasks.workunit.client.0.smithi038.stdout:6/598: mkdir d1/da/d1b/d25/d49/d4e/dbe 0 2024-03-24T00:48:57.053 INFO:tasks.workunit.client.1.smithi088.stdout:5/618: link d1/d10/d1b/d2f/f4a d1/d43/d4e/d5b/fcb 0 2024-03-24T00:48:57.059 INFO:tasks.workunit.client.0.smithi038.stdout:9/626: dread d12/d28/d3a/f6d [0,4194304] 0 2024-03-24T00:48:57.059 INFO:tasks.workunit.client.0.smithi038.stdout:9/627: truncate d12/d28/d6c/f74 5135144 0 2024-03-24T00:48:57.059 INFO:tasks.workunit.client.0.smithi038.stdout:9/628: dread - d12/d28/d3a/fce zero size 2024-03-24T00:48:57.059 INFO:tasks.workunit.client.0.smithi038.stdout:3/573: dwrite d3/d9/dc/df/f31 [0,4194304] 0 2024-03-24T00:48:57.064 INFO:tasks.workunit.client.0.smithi038.stdout:7/525: dread d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f97 [0,4194304] 0 2024-03-24T00:48:57.064 INFO:tasks.workunit.client.0.smithi038.stdout:7/526: write d2/d9/da/f73 [441322,91396] 0 2024-03-24T00:48:57.065 INFO:tasks.workunit.client.0.smithi038.stdout:7/527: write d2/d9/da/d23/d2b/d5b/d72/d92/d33/f96 [230481,70567] 0 2024-03-24T00:48:57.065 INFO:tasks.workunit.client.1.smithi088.stdout:8/614: creat d9/dc/d11/fd6 x:0 0 0 2024-03-24T00:48:57.065 INFO:tasks.workunit.client.1.smithi088.stdout:8/615: read - d9/dc/d11/d24/d29/d5b/fad zero size 2024-03-24T00:48:57.066 INFO:tasks.workunit.client.1.smithi088.stdout:1/592: dread d6/d23/daf/d95/d58/f51 [4194304,4194304] 0 2024-03-24T00:48:57.074 INFO:tasks.workunit.client.0.smithi038.stdout:6/599: unlink d1/da/f3b 0 2024-03-24T00:48:57.077 INFO:tasks.workunit.client.0.smithi038.stdout:3/574: creat d3/d30/d70/d90/d99/fbf x:0 0 0 2024-03-24T00:48:57.081 INFO:tasks.workunit.client.0.smithi038.stdout:9/629: write f6 [2363161,50366] 0 2024-03-24T00:48:57.081 INFO:tasks.workunit.client.0.smithi038.stdout:9/630: readlink d12/d28/l6b 0 2024-03-24T00:48:57.081 INFO:tasks.workunit.client.0.smithi038.stdout:9/631: truncate d12/d28/d3a/fce 1001425 0 2024-03-24T00:48:57.085 INFO:tasks.workunit.client.0.smithi038.stdout:5/538: dwrite d6/d19/d20/d42/d43/d68/f8b [8388608,4194304] 0 2024-03-24T00:48:57.087 INFO:tasks.workunit.client.0.smithi038.stdout:0/480: dwrite d0/f7 [0,4194304] 0 2024-03-24T00:48:57.092 INFO:tasks.workunit.client.1.smithi088.stdout:2/575: rename db/dd/l1e to db/dd/d96/d4e/lb7 0 2024-03-24T00:48:57.092 INFO:tasks.workunit.client.1.smithi088.stdout:2/576: chown db/d21/d35 105426058 1 2024-03-24T00:48:57.093 INFO:tasks.workunit.client.1.smithi088.stdout:9/621: dwrite d1/d10/d29/f45 [0,4194304] 0 2024-03-24T00:48:57.098 INFO:tasks.workunit.client.0.smithi038.stdout:7/528: creat d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fc7 x:0 0 0 2024-03-24T00:48:57.100 INFO:tasks.workunit.client.0.smithi038.stdout:5/539: symlink d6/d16/d21/d2d/da6/lb3 0 2024-03-24T00:48:57.100 INFO:tasks.workunit.client.0.smithi038.stdout:5/540: chown d6/d19/d20/d42/d43/d87 161362074 1 2024-03-24T00:48:57.101 INFO:tasks.workunit.client.0.smithi038.stdout:7/529: creat d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fc8 x:0 0 0 2024-03-24T00:48:57.106 INFO:tasks.workunit.client.0.smithi038.stdout:6/600: dread d1/da/d1b/d1d/f2c [0,4194304] 0 2024-03-24T00:48:57.106 INFO:tasks.workunit.client.0.smithi038.stdout:6/601: chown d1/da/d1b/d1d/c27 762275 1 2024-03-24T00:48:57.107 INFO:tasks.workunit.client.1.smithi088.stdout:1/593: dread d6/d23/f73 [4194304,4194304] 0 2024-03-24T00:48:57.107 INFO:tasks.workunit.client.1.smithi088.stdout:1/594: write d6/d23/daf/d95/d5b/d5d/d61/d7d/fad [534208,105938] 0 2024-03-24T00:48:57.107 INFO:tasks.workunit.client.1.smithi088.stdout:1/595: stat d6/d23/daf/d95/d5b/d5d/d61 0 2024-03-24T00:48:57.113 INFO:tasks.workunit.client.1.smithi088.stdout:1/596: dread d6/d27/d67/d88/f92 [0,4194304] 0 2024-03-24T00:48:57.120 INFO:tasks.workunit.client.1.smithi088.stdout:0/613: dwrite d9/f56 [4194304,4194304] 0 2024-03-24T00:48:57.120 INFO:tasks.workunit.client.1.smithi088.stdout:0/614: truncate d9/d26/d8f/da1/d62/d9b/fbb 344618 0 2024-03-24T00:48:57.120 INFO:tasks.workunit.client.1.smithi088.stdout:0/615: chown d9/dc/de/d19/d44/d46/d65/d2a/l33 758 1 2024-03-24T00:48:57.120 INFO:tasks.workunit.client.1.smithi088.stdout:0/616: readlink d9/dc/de/d19/d44/d46/d65/l25 0 2024-03-24T00:48:57.128 INFO:tasks.workunit.client.0.smithi038.stdout:8/562: dwrite d0/d45/d66/f97 [0,4194304] 0 2024-03-24T00:48:57.140 INFO:tasks.workunit.client.0.smithi038.stdout:7/530: dread d2/f1a [0,4194304] 0 2024-03-24T00:48:57.141 INFO:tasks.workunit.client.0.smithi038.stdout:7/531: readlink d2/d9/da/dc/d93/l9c 0 2024-03-24T00:48:57.141 INFO:tasks.workunit.client.0.smithi038.stdout:7/532: creat d2/d9/da/dc/d8a/fc9 x:0 0 0 2024-03-24T00:48:57.155 INFO:tasks.workunit.client.0.smithi038.stdout:0/481: dwrite d0/d2d/f2f [0,4194304] 0 2024-03-24T00:48:57.155 INFO:tasks.workunit.client.0.smithi038.stdout:0/482: fdatasync d0/d1b/d2c/f3b 0 2024-03-24T00:48:57.161 INFO:tasks.workunit.client.0.smithi038.stdout:4/533: dwrite da/d38/f61 [0,4194304] 0 2024-03-24T00:48:57.173 INFO:tasks.workunit.client.0.smithi038.stdout:3/575: dwrite d3/d9/dc/da6/db5/f54 [0,4194304] 0 2024-03-24T00:48:57.179 INFO:tasks.workunit.client.1.smithi088.stdout:0/617: dwrite d9/dc/de/d19/d44/d46/d65/fb5 [0,4194304] 0 2024-03-24T00:48:57.179 INFO:tasks.workunit.client.1.smithi088.stdout:0/618: stat d9/d26/d8f/da1/d62/d9b/lc0 0 2024-03-24T00:48:57.179 INFO:tasks.workunit.client.1.smithi088.stdout:0/619: stat d9/d26/d8f/da1/d62/d9b/dcb 0 2024-03-24T00:48:57.179 INFO:tasks.workunit.client.1.smithi088.stdout:0/620: write d9/dc/de/d7e/d91/f9f [609417,67046] 0 2024-03-24T00:48:57.190 INFO:tasks.workunit.client.0.smithi038.stdout:5/541: truncate d6/d16/d14/f8e 2103329 0 2024-03-24T00:48:57.190 INFO:tasks.workunit.client.0.smithi038.stdout:5/542: chown d6/d1e 769631 1 2024-03-24T00:48:57.193 INFO:tasks.workunit.client.1.smithi088.stdout:2/577: mknod db/dd/d96/cb8 0 2024-03-24T00:48:57.193 INFO:tasks.workunit.client.1.smithi088.stdout:2/578: chown db/d21/c5b 162568408 1 2024-03-24T00:48:57.201 INFO:tasks.workunit.client.0.smithi038.stdout:7/533: creat d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fca x:0 0 0 2024-03-24T00:48:57.209 INFO:tasks.workunit.client.0.smithi038.stdout:4/534: mkdir da/d12/d24/d5f/d8a/db6 0 2024-03-24T00:48:57.212 INFO:tasks.workunit.client.1.smithi088.stdout:9/622: link d1/d15/lb1 d1/d22/d27/d39/d98/lca 0 2024-03-24T00:48:57.212 INFO:tasks.workunit.client.1.smithi088.stdout:1/597: truncate d6/d23/daf/d95/d58/f78 2914002 0 2024-03-24T00:48:57.212 INFO:tasks.workunit.client.0.smithi038.stdout:3/576: mknod d3/d9/dc/da6/db5/d19/cc0 0 2024-03-24T00:48:57.213 INFO:tasks.workunit.client.0.smithi038.stdout:7/534: mkdir d2/d2c/d9a/dcb 0 2024-03-24T00:48:57.213 INFO:tasks.workunit.client.0.smithi038.stdout:4/535: creat da/d12/d29/d5b/d73/db4/fb7 x:0 0 0 2024-03-24T00:48:57.220 INFO:tasks.workunit.client.0.smithi038.stdout:6/602: dwrite d1/faa [0,4194304] 0 2024-03-24T00:48:57.234 INFO:tasks.workunit.client.0.smithi038.stdout:3/577: getdents d3/d9/dc/da6/db5/dad 0 2024-03-24T00:48:57.234 INFO:tasks.workunit.client.1.smithi088.stdout:0/621: symlink d9/d26/lcd 0 2024-03-24T00:48:57.236 INFO:tasks.workunit.client.0.smithi038.stdout:8/563: dwrite d0/d45/d37/d5f/db7/f4f [0,4194304] 0 2024-03-24T00:48:57.236 INFO:tasks.workunit.client.0.smithi038.stdout:8/564: chown d0/d1d/d20/f58 388565717 1 2024-03-24T00:48:57.238 INFO:tasks.workunit.client.0.smithi038.stdout:7/535: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/dcc 0 2024-03-24T00:48:57.241 INFO:tasks.workunit.client.1.smithi088.stdout:2/579: link db/dd/d96/d99/db2/c7e db/dc/cb9 0 2024-03-24T00:48:57.257 INFO:tasks.workunit.client.0.smithi038.stdout:5/543: dwrite d6/d19/d20/d42/d79/f9e [0,4194304] 0 2024-03-24T00:48:57.259 INFO:tasks.workunit.client.1.smithi088.stdout:3/572: rename d3/d1a/d1e/d76/dae/d4b/l92 to d3/d6/lb8 0 2024-03-24T00:48:57.259 INFO:tasks.workunit.client.1.smithi088.stdout:3/573: write d3/de/d6f/d85/f9a [902905,86556] 0 2024-03-24T00:48:57.259 INFO:tasks.workunit.client.1.smithi088.stdout:3/574: dread - d3/d6/d19/f77 zero size 2024-03-24T00:48:57.262 INFO:tasks.workunit.client.1.smithi088.stdout:9/623: rmdir d1/d22/d27/d39/d98/da3 39 2024-03-24T00:48:57.262 INFO:tasks.workunit.client.1.smithi088.stdout:9/624: chown d1/d15/d21/f82 0 1 2024-03-24T00:48:57.262 INFO:tasks.workunit.client.1.smithi088.stdout:9/625: chown d1/d15/l60 29 1 2024-03-24T00:48:57.262 INFO:tasks.workunit.client.1.smithi088.stdout:3/575: write d3/de/d6f/d85/d31/f72 [292300,101376] 0 2024-03-24T00:48:57.262 INFO:tasks.workunit.client.1.smithi088.stdout:3/576: fsync d3/d1a/d1e/f9b 0 2024-03-24T00:48:57.262 INFO:tasks.workunit.client.0.smithi038.stdout:0/483: dwrite d0/d12/d42/d9d/f90 [0,4194304] 0 2024-03-24T00:48:57.263 INFO:tasks.workunit.client.0.smithi038.stdout:0/484: readlink d0/d12/d15/d58/d64/l75 0 2024-03-24T00:48:57.263 INFO:tasks.workunit.client.0.smithi038.stdout:6/603: rename d1/da/d1b/d25/l78 to d1/da/lbf 0 2024-03-24T00:48:57.265 INFO:tasks.workunit.client.1.smithi088.stdout:1/598: symlink d6/lb6 0 2024-03-24T00:48:57.266 INFO:tasks.workunit.client.0.smithi038.stdout:3/578: mknod d3/d5a/cc1 0 2024-03-24T00:48:57.266 INFO:tasks.workunit.client.1.smithi088.stdout:2/580: dread db/f3c [0,4194304] 0 2024-03-24T00:48:57.268 INFO:tasks.workunit.client.1.smithi088.stdout:0/622: mkdir d9/dc/de/d7e/dce 0 2024-03-24T00:48:57.268 INFO:tasks.workunit.client.1.smithi088.stdout:0/623: chown d9/c2b 72 1 2024-03-24T00:48:57.272 INFO:tasks.workunit.client.0.smithi038.stdout:1/638: sync 2024-03-24T00:48:57.273 INFO:tasks.workunit.client.0.smithi038.stdout:2/515: sync 2024-03-24T00:48:57.273 INFO:tasks.workunit.client.1.smithi088.stdout:4/583: rename d5/d7/d15/d74/lbc to d5/d7/d15/lcd 0 2024-03-24T00:48:57.274 INFO:tasks.workunit.client.1.smithi088.stdout:4/584: chown d5/d16/d1a/d22/f68 4862241 1 2024-03-24T00:48:57.274 INFO:tasks.workunit.client.1.smithi088.stdout:4/585: chown d5/dac 398331 1 2024-03-24T00:48:57.275 INFO:tasks.workunit.client.1.smithi088.stdout:9/626: creat d1/d10/d11/fcb x:0 0 0 2024-03-24T00:48:57.277 INFO:tasks.workunit.client.0.smithi038.stdout:7/536: stat d2/d9/da/d23/c7a 0 2024-03-24T00:48:57.277 INFO:tasks.workunit.client.0.smithi038.stdout:7/537: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/d75/f78 0 2024-03-24T00:48:57.278 INFO:tasks.workunit.client.0.smithi038.stdout:3/579: dread d3/d9/dc/da6/db5/d19/f42 [0,4194304] 0 2024-03-24T00:48:57.279 INFO:tasks.workunit.client.1.smithi088.stdout:1/599: mknod d6/d23/daf/d95/d5b/d5d/d61/d7d/da5/cb7 0 2024-03-24T00:48:57.283 INFO:tasks.workunit.client.1.smithi088.stdout:0/624: unlink d9/dc/de/d7e/c8e 0 2024-03-24T00:48:57.286 INFO:tasks.workunit.client.0.smithi038.stdout:5/544: rmdir d6/d19/d20/d42/d43 39 2024-03-24T00:48:57.287 INFO:tasks.workunit.client.0.smithi038.stdout:4/536: dwrite da/d27/fb5 [0,4194304] 0 2024-03-24T00:48:57.287 INFO:tasks.workunit.client.1.smithi088.stdout:4/586: creat d5/d16/d1a/d22/dc6/fce x:0 0 0 2024-03-24T00:48:57.289 INFO:tasks.workunit.client.0.smithi038.stdout:8/565: dwrite d0/d1d/d27/d4d/fb2 [0,4194304] 0 2024-03-24T00:48:57.290 INFO:tasks.workunit.client.1.smithi088.stdout:9/627: creat d1/fcc x:0 0 0 2024-03-24T00:48:57.290 INFO:tasks.workunit.client.1.smithi088.stdout:9/628: write d1/d15/d80/da5/fb2 [1121626,73211] 0 2024-03-24T00:48:57.290 INFO:tasks.workunit.client.1.smithi088.stdout:9/629: fdatasync d1/d10/d29/f45 0 2024-03-24T00:48:57.290 INFO:tasks.workunit.client.1.smithi088.stdout:9/630: truncate d1/d22/d27/fbf 315174 0 2024-03-24T00:48:57.292 INFO:tasks.workunit.client.1.smithi088.stdout:1/600: mknod d6/d23/daf/d95/d3f/cb8 0 2024-03-24T00:48:57.293 INFO:tasks.workunit.client.0.smithi038.stdout:0/485: creat d0/d12/d42/d83/f9f x:0 0 0 2024-03-24T00:48:57.299 INFO:tasks.workunit.client.1.smithi088.stdout:0/625: truncate d9/f85 1578958 0 2024-03-24T00:48:57.301 INFO:tasks.workunit.client.1.smithi088.stdout:7/519: rename d3/c7d to d3/d32/d34/c9f 0 2024-03-24T00:48:57.301 INFO:tasks.workunit.client.1.smithi088.stdout:6/612: sync 2024-03-24T00:48:57.304 INFO:tasks.workunit.client.0.smithi038.stdout:1/639: creat da/dd/d4a/fd3 x:0 0 0 2024-03-24T00:48:57.309 INFO:tasks.workunit.client.1.smithi088.stdout:4/587: unlink d5/d17/l9f 0 2024-03-24T00:48:57.312 INFO:tasks.workunit.client.0.smithi038.stdout:2/516: rmdir d4/d27/d2f/d89 39 2024-03-24T00:48:57.314 INFO:tasks.workunit.client.1.smithi088.stdout:1/601: truncate d6/d27/d67/f6b 1980357 0 2024-03-24T00:48:57.315 INFO:tasks.workunit.client.0.smithi038.stdout:7/538: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d5d/dcd 0 2024-03-24T00:48:57.318 INFO:tasks.workunit.client.1.smithi088.stdout:0/626: truncate d9/dc/de/d19/f48 2813383 0 2024-03-24T00:48:57.319 INFO:tasks.workunit.client.0.smithi038.stdout:3/580: unlink d3/d2a/c5b 0 2024-03-24T00:48:57.322 INFO:tasks.workunit.client.1.smithi088.stdout:5/619: rename d1/d10/d1b/d2f/d57/d78/dc3 to d1/d19/d48/dc2/dcc 0 2024-03-24T00:48:57.323 INFO:tasks.workunit.client.1.smithi088.stdout:3/577: dwrite d3/de/d6f/f75 [0,4194304] 0 2024-03-24T00:48:57.324 INFO:tasks.workunit.client.1.smithi088.stdout:3/578: write d3/d1a/d1e/d76/f84 [832712,126495] 0 2024-03-24T00:48:57.326 INFO:tasks.workunit.client.1.smithi088.stdout:6/613: link db/d3d/d83/f9f db/fbd 0 2024-03-24T00:48:57.328 INFO:tasks.workunit.client.0.smithi038.stdout:5/545: symlink d6/d19/d20/d64/lb4 0 2024-03-24T00:48:57.334 INFO:tasks.workunit.client.1.smithi088.stdout:4/588: mknod d5/d13/d18/d24/d3d/ccf 0 2024-03-24T00:48:57.339 INFO:tasks.workunit.client.0.smithi038.stdout:4/537: link da/d12/d37/dd/d15/d6d/l7f da/d12/d37/dd/d15/d4c/d70/lb8 0 2024-03-24T00:48:57.342 INFO:tasks.workunit.client.1.smithi088.stdout:1/602: mkdir d6/d27/d67/d9b/db9 0 2024-03-24T00:48:57.354 INFO:tasks.workunit.client.1.smithi088.stdout:2/581: dwrite db/f20 [0,4194304] 0 2024-03-24T00:48:57.354 INFO:tasks.workunit.client.1.smithi088.stdout:2/582: truncate db/d21/f33 1176324 0 2024-03-24T00:48:57.354 INFO:tasks.workunit.client.1.smithi088.stdout:2/583: stat db/dd/fa5 0 2024-03-24T00:48:57.360 INFO:tasks.workunit.client.0.smithi038.stdout:1/640: symlink da/dd/d2c/d5a/d74/ld4 0 2024-03-24T00:48:57.365 INFO:tasks.workunit.client.1.smithi088.stdout:0/627: truncate d9/f55 3107696 0 2024-03-24T00:48:57.365 INFO:tasks.workunit.client.1.smithi088.stdout:0/628: chown d9/dc/de/d19/d44/d46/d65/d2a/d2e 187 1 2024-03-24T00:48:57.365 INFO:tasks.workunit.client.0.smithi038.stdout:8/566: dread d0/d15/f98 [4194304,4194304] 0 2024-03-24T00:48:57.365 INFO:tasks.workunit.client.0.smithi038.stdout:5/546: unlink d6/c15 0 2024-03-24T00:48:57.367 INFO:tasks.workunit.client.1.smithi088.stdout:7/520: dwrite d3/d31/f83 [0,4194304] 0 2024-03-24T00:48:57.378 INFO:tasks.workunit.client.0.smithi038.stdout:6/604: rename d1/da/d1b/da7 to d1/da/d1b/d2f/dc0 0 2024-03-24T00:48:57.378 INFO:tasks.workunit.client.0.smithi038.stdout:6/605: chown d1/da/d1b/d25/d49/d4e/dbe 8118016 1 2024-03-24T00:48:57.378 INFO:tasks.workunit.client.0.smithi038.stdout:6/606: fdatasync d1/da/d73/fab 0 2024-03-24T00:48:57.380 INFO:tasks.workunit.client.1.smithi088.stdout:4/589: read d5/d16/f29 [1582911,50112] 0 2024-03-24T00:48:57.380 INFO:tasks.workunit.client.1.smithi088.stdout:4/590: write d5/d13/fa7 [4615488,61045] 0 2024-03-24T00:48:57.393 INFO:tasks.workunit.client.0.smithi038.stdout:5/547: write d6/d19/d20/d42/d43/d68/f8b [8780087,56176] 0 2024-03-24T00:48:57.393 INFO:tasks.workunit.client.0.smithi038.stdout:3/581: mknod d3/cc2 0 2024-03-24T00:48:57.393 INFO:tasks.workunit.client.0.smithi038.stdout:6/607: truncate d1/da/d1b/d2f/f3d 1943127 0 2024-03-24T00:48:57.394 INFO:tasks.workunit.client.1.smithi088.stdout:5/620: creat d1/dd/d6f/d98/fcd x:0 0 0 2024-03-24T00:48:57.396 INFO:tasks.workunit.client.1.smithi088.stdout:6/614: dwrite db/dd/f41 [8388608,4194304] 0 2024-03-24T00:48:57.401 INFO:tasks.workunit.client.0.smithi038.stdout:2/517: rename d4/d18/d38/f45 to d4/d27/d54/d53/fba 0 2024-03-24T00:48:57.403 INFO:tasks.workunit.client.0.smithi038.stdout:0/486: dwrite d0/d12/d42/d9d/d28/f48 [0,4194304] 0 2024-03-24T00:48:57.407 INFO:tasks.workunit.client.1.smithi088.stdout:3/579: symlink d3/d1a/d1e/lb9 0 2024-03-24T00:48:57.407 INFO:tasks.workunit.client.0.smithi038.stdout:7/539: dwrite d2/d9/da/d23/d2b/fbb [0,4194304] 0 2024-03-24T00:48:57.418 INFO:tasks.workunit.client.1.smithi088.stdout:1/603: unlink d6/d23/daf/d95/d5b/c87 0 2024-03-24T00:48:57.423 INFO:tasks.workunit.client.1.smithi088.stdout:0/629: unlink d9/d26/d8f/da1/d62/lb9 0 2024-03-24T00:48:57.424 INFO:tasks.workunit.client.1.smithi088.stdout:0/630: readlink d9/dc/de/d7e/lbd 0 2024-03-24T00:48:57.427 INFO:tasks.workunit.client.0.smithi038.stdout:6/608: symlink d1/da/d1b/d25/d49/d4e/dbe/lc1 0 2024-03-24T00:48:57.427 INFO:tasks.workunit.client.0.smithi038.stdout:6/609: chown d1/da/d1b/d25/d49/d4e/d51/la6 86042648 1 2024-03-24T00:48:57.434 INFO:tasks.workunit.client.0.smithi038.stdout:2/518: stat d4/d27/d2f/d30/d3e/c4a 0 2024-03-24T00:48:57.436 INFO:tasks.workunit.client.0.smithi038.stdout:5/548: rename d6/c59 to d6/d19/d20/d33/cb5 0 2024-03-24T00:48:57.444 INFO:tasks.workunit.client.1.smithi088.stdout:7/521: write d3/f15 [5141931,129898] 0 2024-03-24T00:48:57.447 INFO:tasks.workunit.client.0.smithi038.stdout:0/487: creat d0/d12/d42/d9d/fa0 x:0 0 0 2024-03-24T00:48:57.452 INFO:tasks.workunit.client.0.smithi038.stdout:4/538: dwrite da/d12/d29/f98 [0,4194304] 0 2024-03-24T00:48:57.457 INFO:tasks.workunit.client.1.smithi088.stdout:6/615: dwrite db/dd/f2b [0,4194304] 0 2024-03-24T00:48:57.466 INFO:tasks.workunit.client.0.smithi038.stdout:6/610: dread d1/d37/fa5 [0,4194304] 0 2024-03-24T00:48:57.466 INFO:tasks.workunit.client.1.smithi088.stdout:8/616: rename d9/dc/d11/d41 to d9/dc/d11/d24/d29/d36/dc1/dc4/dd7 0 2024-03-24T00:48:57.468 INFO:tasks.workunit.client.0.smithi038.stdout:2/519: dread d4/d27/d2f/d30/d3e/f8b [0,4194304] 0 2024-03-24T00:48:57.468 INFO:tasks.workunit.client.0.smithi038.stdout:2/520: chown d4/f7a 7024 1 2024-03-24T00:48:57.469 INFO:tasks.workunit.client.0.smithi038.stdout:2/521: chown d4/d18/d26/d37/d69/d83/fb0 50 1 2024-03-24T00:48:57.469 INFO:tasks.workunit.client.0.smithi038.stdout:2/522: chown d4/d18/d26/f3d 794428467 1 2024-03-24T00:48:57.470 INFO:tasks.workunit.client.0.smithi038.stdout:7/540: creat d2/d9/da/dc/d8a/db8/fce x:0 0 0 2024-03-24T00:48:57.474 INFO:tasks.workunit.client.1.smithi088.stdout:5/621: mknod d1/cce 0 2024-03-24T00:48:57.474 INFO:tasks.workunit.client.1.smithi088.stdout:3/580: creat d3/de/fba x:0 0 0 2024-03-24T00:48:57.475 INFO:tasks.workunit.client.1.smithi088.stdout:5/622: write d1/d19/d48/fae [1040354,107381] 0 2024-03-24T00:48:57.475 INFO:tasks.workunit.client.1.smithi088.stdout:5/623: write d1/d10/d1b/d2f/d57/fa5 [461502,76331] 0 2024-03-24T00:48:57.475 INFO:tasks.workunit.client.1.smithi088.stdout:5/624: chown d1/d10/d1b/d2f/d57/f5f 0 1 2024-03-24T00:48:57.476 INFO:tasks.workunit.client.0.smithi038.stdout:5/549: dread d6/d19/d20/d42/d43/d4c/f74 [0,4194304] 0 2024-03-24T00:48:57.476 INFO:tasks.workunit.client.0.smithi038.stdout:5/550: chown d6 6 1 2024-03-24T00:48:57.477 INFO:tasks.workunit.client.1.smithi088.stdout:4/591: dwrite d5/d16/fb1 [0,4194304] 0 2024-03-24T00:48:57.477 INFO:tasks.workunit.client.1.smithi088.stdout:7/522: dread d3/f29 [0,4194304] 0 2024-03-24T00:48:57.481 INFO:tasks.workunit.client.0.smithi038.stdout:3/582: dread d3/d9/dc/d51/f52 [0,4194304] 0 2024-03-24T00:48:57.485 INFO:tasks.workunit.client.0.smithi038.stdout:1/641: dwrite da/f7e [0,4194304] 0 2024-03-24T00:48:57.486 INFO:tasks.workunit.client.1.smithi088.stdout:0/631: dwrite d9/d26/f38 [0,4194304] 0 2024-03-24T00:48:57.486 INFO:tasks.workunit.client.1.smithi088.stdout:0/632: truncate d9/dc/de/f6c 4199532 0 2024-03-24T00:48:57.489 INFO:tasks.workunit.client.0.smithi038.stdout:1/642: creat da/dd/d2c/db6/fd5 x:0 0 0 2024-03-24T00:48:57.491 INFO:tasks.workunit.client.0.smithi038.stdout:4/539: creat da/d12/d24/d5f/d8a/db6/fb9 x:0 0 0 2024-03-24T00:48:57.493 INFO:tasks.workunit.client.0.smithi038.stdout:4/540: readlink da/d12/d29/d5b/l8b 0 2024-03-24T00:48:57.493 INFO:tasks.workunit.client.0.smithi038.stdout:4/541: chown da/d12/f4e 8158458 1 2024-03-24T00:48:57.496 INFO:tasks.workunit.client.0.smithi038.stdout:4/542: creat da/d12/d29/d5b/d73/fba x:0 0 0 2024-03-24T00:48:57.497 INFO:tasks.workunit.client.0.smithi038.stdout:6/611: creat d1/da/d61/d69/fc2 x:0 0 0 2024-03-24T00:48:57.500 INFO:tasks.workunit.client.0.smithi038.stdout:8/567: dwrite d0/d15/f98 [0,4194304] 0 2024-03-24T00:48:57.501 INFO:tasks.workunit.client.0.smithi038.stdout:8/568: write d0/d45/f89 [450093,20098] 0 2024-03-24T00:48:57.502 INFO:tasks.workunit.client.1.smithi088.stdout:6/616: unlink db/d1d/d23/d3c/d43/f39 0 2024-03-24T00:48:57.509 INFO:tasks.workunit.client.1.smithi088.stdout:1/604: rename d6/d23/daf/d95/d5b/d5d/f76 to d6/fba 0 2024-03-24T00:48:57.509 INFO:tasks.workunit.client.1.smithi088.stdout:1/605: truncate d6/fab 573533 0 2024-03-24T00:48:57.509 INFO:tasks.workunit.client.1.smithi088.stdout:1/606: stat d6/c85 0 2024-03-24T00:48:57.510 INFO:tasks.workunit.client.0.smithi038.stdout:2/523: mknod d4/d18/da1/cbb 0 2024-03-24T00:48:57.510 INFO:tasks.workunit.client.0.smithi038.stdout:2/524: chown d4/d27/d2f/d30/d3e/d9b/f9c 0 1 2024-03-24T00:48:57.510 INFO:tasks.workunit.client.0.smithi038.stdout:5/551: dread d6/d1e/f1f [0,4194304] 0 2024-03-24T00:48:57.510 INFO:tasks.workunit.client.0.smithi038.stdout:5/552: fdatasync d6/d23/f3b 0 2024-03-24T00:48:57.510 INFO:tasks.workunit.client.0.smithi038.stdout:5/553: readlink d6/d16/l2e 0 2024-03-24T00:48:57.515 INFO:tasks.workunit.client.0.smithi038.stdout:7/541: creat d2/d2c/d99/fcf x:0 0 0 2024-03-24T00:48:57.515 INFO:tasks.workunit.client.0.smithi038.stdout:9/632: sync 2024-03-24T00:48:57.516 INFO:tasks.workunit.client.1.smithi088.stdout:7/523: symlink d3/d32/d34/la0 0 2024-03-24T00:48:57.519 INFO:tasks.workunit.client.1.smithi088.stdout:4/592: dread d5/d16/f57 [0,4194304] 0 2024-03-24T00:48:57.519 INFO:tasks.workunit.client.0.smithi038.stdout:1/643: read da/dd/d2c/d5a/d74/f75 [4170049,113715] 0 2024-03-24T00:48:57.520 INFO:tasks.workunit.client.0.smithi038.stdout:1/644: dread - da/dd/d7a/d82/d98/d14/faf zero size 2024-03-24T00:48:57.520 INFO:tasks.workunit.client.0.smithi038.stdout:1/645: chown da/lcf 15 1 2024-03-24T00:48:57.523 INFO:tasks.workunit.client.0.smithi038.stdout:8/569: dread d0/d45/d37/d5f/d68/f91 [0,4194304] 0 2024-03-24T00:48:57.524 INFO:tasks.workunit.client.0.smithi038.stdout:4/543: creat da/d12/d37/dd/d15/d6d/d8f/fbb x:0 0 0 2024-03-24T00:48:57.524 INFO:tasks.workunit.client.0.smithi038.stdout:4/544: write da/d12/d24/d68/f93 [78675,81011] 0 2024-03-24T00:48:57.532 INFO:tasks.workunit.client.0.smithi038.stdout:6/612: truncate d1/faa 1065668 0 2024-03-24T00:48:57.532 INFO:tasks.workunit.client.0.smithi038.stdout:2/525: unlink d4/d27/d54/l5d 0 2024-03-24T00:48:57.534 INFO:tasks.workunit.client.1.smithi088.stdout:0/633: creat d9/dc/de/fcf x:0 0 0 2024-03-24T00:48:57.538 INFO:tasks.workunit.client.0.smithi038.stdout:3/583: mkdir d3/dc3 0 2024-03-24T00:48:57.539 INFO:tasks.workunit.client.0.smithi038.stdout:3/584: stat d3/d9/dc/df/f6b 0 2024-03-24T00:48:57.539 INFO:tasks.workunit.client.0.smithi038.stdout:3/585: fsync d3/d9/d77/f7f 0 2024-03-24T00:48:57.539 INFO:tasks.workunit.client.0.smithi038.stdout:1/646: truncate da/dd/d7a/f91 2265363 0 2024-03-24T00:48:57.542 INFO:tasks.workunit.client.1.smithi088.stdout:6/617: unlink db/d1d/d50/d90/d56/f31 0 2024-03-24T00:48:57.543 INFO:tasks.workunit.client.0.smithi038.stdout:7/542: write d2/d9/da/d23/d2b/d5b/d72/d92/d98/f9f [600812,108228] 0 2024-03-24T00:48:57.545 INFO:tasks.workunit.client.0.smithi038.stdout:0/488: dwrite d0/d12/d42/f61 [4194304,4194304] 0 2024-03-24T00:48:57.546 INFO:tasks.workunit.client.1.smithi088.stdout:5/625: rename d1/dd/d6f/da1/d3f/d88/d8b/d94/cb3 to d1/d10/d79/dc5/ccf 0 2024-03-24T00:48:57.546 INFO:tasks.workunit.client.1.smithi088.stdout:5/626: dread - d1/d10/d1b/d2e/d46/f80 zero size 2024-03-24T00:48:57.547 INFO:tasks.workunit.client.0.smithi038.stdout:6/613: creat d1/da/d1b/d25/d49/d4e/dbe/fc3 x:0 0 0 2024-03-24T00:48:57.547 INFO:tasks.workunit.client.0.smithi038.stdout:2/526: symlink d4/d27/d2f/d30/d3e/d7c/lbc 0 2024-03-24T00:48:57.547 INFO:tasks.workunit.client.0.smithi038.stdout:2/527: dread - d4/d27/d2f/d89/f9a zero size 2024-03-24T00:48:57.550 INFO:tasks.workunit.client.1.smithi088.stdout:7/524: creat d3/d32/d34/d9c/d4b/d74/d79/fa1 x:0 0 0 2024-03-24T00:48:57.550 INFO:tasks.workunit.client.1.smithi088.stdout:0/634: dread d9/dc/de/f11 [0,4194304] 0 2024-03-24T00:48:57.550 INFO:tasks.workunit.client.1.smithi088.stdout:7/525: write d3/d32/d34/f96 [132516,88076] 0 2024-03-24T00:48:57.550 INFO:tasks.workunit.client.1.smithi088.stdout:7/526: chown d3/d32/d34/f72 59541 1 2024-03-24T00:48:57.550 INFO:tasks.workunit.client.1.smithi088.stdout:0/635: write d9/d26/d8f/da1/d63/fa7 [390677,24774] 0 2024-03-24T00:48:57.550 INFO:tasks.workunit.client.1.smithi088.stdout:0/636: chown d9/dc/f52 11340910 1 2024-03-24T00:48:57.551 INFO:tasks.workunit.client.0.smithi038.stdout:7/543: write d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f8f [53274,27060] 0 2024-03-24T00:48:57.551 INFO:tasks.workunit.client.0.smithi038.stdout:3/586: creat d3/d9/dc/da6/db5/dad/fc4 x:0 0 0 2024-03-24T00:48:57.551 INFO:tasks.workunit.client.0.smithi038.stdout:3/587: creat d3/d5a/fc5 x:0 0 0 2024-03-24T00:48:57.551 INFO:tasks.workunit.client.0.smithi038.stdout:3/588: write d3/d30/f33 [1265568,125793] 0 2024-03-24T00:48:57.552 INFO:tasks.workunit.client.1.smithi088.stdout:5/627: mkdir d1/d10/d1b/d2f/d57/d9e/dd0 0 2024-03-24T00:48:57.553 INFO:tasks.workunit.client.0.smithi038.stdout:0/489: symlink d0/d12/d15/la1 0 2024-03-24T00:48:57.554 INFO:tasks.workunit.client.0.smithi038.stdout:0/490: chown d0/d1b/f20 1 1 2024-03-24T00:48:57.554 INFO:tasks.workunit.client.0.smithi038.stdout:0/491: write d0/d12/d42/d9d/d28/f8a [868162,2299] 0 2024-03-24T00:48:57.554 INFO:tasks.workunit.client.1.smithi088.stdout:7/527: mkdir d3/d32/d76/da2 0 2024-03-24T00:48:57.557 INFO:tasks.workunit.client.1.smithi088.stdout:6/618: dread f8 [0,4194304] 0 2024-03-24T00:48:57.558 INFO:tasks.workunit.client.1.smithi088.stdout:0/637: creat d9/fd0 x:0 0 0 2024-03-24T00:48:57.559 INFO:tasks.workunit.client.0.smithi038.stdout:6/614: truncate d1/da/f67 167313 0 2024-03-24T00:48:57.561 INFO:tasks.workunit.client.0.smithi038.stdout:5/554: dread d6/f18 [0,4194304] 0 2024-03-24T00:48:57.562 INFO:tasks.workunit.client.0.smithi038.stdout:7/544: creat d2/d2c/d9a/fd0 x:0 0 0 2024-03-24T00:48:57.562 INFO:tasks.workunit.client.0.smithi038.stdout:7/545: stat d2/d9/da/dc/d93/l9c 0 2024-03-24T00:48:57.562 INFO:tasks.workunit.client.0.smithi038.stdout:7/546: read - d2/d9/da/d23/f5a zero size 2024-03-24T00:48:57.563 INFO:tasks.workunit.client.1.smithi088.stdout:7/528: symlink d3/d32/d34/d9c/d4b/d73/la3 0 2024-03-24T00:48:57.563 INFO:tasks.workunit.client.1.smithi088.stdout:7/529: chown d3/d32/d34/d9c/c50 105 1 2024-03-24T00:48:57.564 INFO:tasks.workunit.client.1.smithi088.stdout:7/530: chown d3/d31/f7e 566 1 2024-03-24T00:48:57.567 INFO:tasks.workunit.client.1.smithi088.stdout:8/617: dwrite d9/dc/d11/d24/f5e [0,4194304] 0 2024-03-24T00:48:57.570 INFO:tasks.workunit.client.1.smithi088.stdout:6/619: creat db/d1d/d50/d90/d55/fbe x:0 0 0 2024-03-24T00:48:57.572 INFO:tasks.workunit.client.0.smithi038.stdout:0/492: creat d0/d12/d92/fa2 x:0 0 0 2024-03-24T00:48:57.575 INFO:tasks.workunit.client.0.smithi038.stdout:7/547: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d98/cd1 0 2024-03-24T00:48:57.575 INFO:tasks.workunit.client.1.smithi088.stdout:0/638: symlink d9/d26/d8f/da1/d62/dc4/ld1 0 2024-03-24T00:48:57.575 INFO:tasks.workunit.client.1.smithi088.stdout:0/639: fsync d9/dc/de/d19/d44/d46/d65/f68 0 2024-03-24T00:48:57.575 INFO:tasks.workunit.client.1.smithi088.stdout:0/640: dread - d9/f69 zero size 2024-03-24T00:48:57.577 INFO:tasks.workunit.client.1.smithi088.stdout:7/531: symlink d3/d32/d34/d9c/d4b/d73/la4 0 2024-03-24T00:48:57.580 INFO:tasks.workunit.client.0.smithi038.stdout:9/633: write d12/d55/d56/f7c [196947,12307] 0 2024-03-24T00:48:57.581 INFO:tasks.workunit.client.1.smithi088.stdout:6/620: mkdir db/d1d/d23/d3c/d43/dbf 0 2024-03-24T00:48:57.583 INFO:tasks.workunit.client.1.smithi088.stdout:3/581: dwrite d3/d6/d27/d3c/f30 [0,4194304] 0 2024-03-24T00:48:57.585 INFO:tasks.workunit.client.0.smithi038.stdout:6/615: dread d1/da/d1b/d25/f44 [0,4194304] 0 2024-03-24T00:48:57.586 INFO:tasks.workunit.client.1.smithi088.stdout:0/641: mkdir d9/dc/de/d19/dd2 0 2024-03-24T00:48:57.586 INFO:tasks.workunit.client.1.smithi088.stdout:0/642: stat d9/d26/d8f/da1/d62/f6d 0 2024-03-24T00:48:57.587 INFO:tasks.workunit.client.1.smithi088.stdout:4/593: dwrite d5/d16/d37/f6f [0,4194304] 0 2024-03-24T00:48:57.588 INFO:tasks.workunit.client.1.smithi088.stdout:7/532: symlink d3/d32/d34/d9c/d4b/d74/d79/d7f/la5 0 2024-03-24T00:48:57.599 INFO:tasks.workunit.client.0.smithi038.stdout:9/634: link d12/d81/d84/d88/fa9 d12/d41/fd0 0 2024-03-24T00:48:57.600 INFO:tasks.workunit.client.0.smithi038.stdout:9/635: write d12/d28/d71/fbe [470384,68094] 0 2024-03-24T00:48:57.600 INFO:tasks.workunit.client.1.smithi088.stdout:3/582: truncate d3/d6/d27/d3c/f39 621670 0 2024-03-24T00:48:57.601 INFO:tasks.workunit.client.1.smithi088.stdout:6/621: write db/d1d/d50/d90/d56/f51 [2875859,19840] 0 2024-03-24T00:48:57.601 INFO:tasks.workunit.client.1.smithi088.stdout:6/622: chown db/d1d/d50/d90/d55/lb2 2 1 2024-03-24T00:48:57.601 INFO:tasks.workunit.client.1.smithi088.stdout:6/623: write db/d1d/d23/fa5 [196822,64403] 0 2024-03-24T00:48:57.605 INFO:tasks.workunit.client.1.smithi088.stdout:0/643: write d9/dc/de/d19/d44/f5b [3280249,128639] 0 2024-03-24T00:48:57.605 INFO:tasks.workunit.client.1.smithi088.stdout:0/644: dread - d9/d26/d8f/da1/fba zero size 2024-03-24T00:48:57.607 INFO:tasks.workunit.client.0.smithi038.stdout:6/616: creat d1/da/d1b/d2f/dc0/fc4 x:0 0 0 2024-03-24T00:48:57.612 INFO:tasks.workunit.client.0.smithi038.stdout:9/636: creat d12/d41/dc0/fd1 x:0 0 0 2024-03-24T00:48:57.612 INFO:tasks.workunit.client.1.smithi088.stdout:5/628: dread d1/d10/d1b/d2e/f91 [0,4194304] 0 2024-03-24T00:48:57.612 INFO:tasks.workunit.client.1.smithi088.stdout:5/629: creat d1/dd/d6f/d98/fd1 x:0 0 0 2024-03-24T00:48:57.613 INFO:tasks.workunit.client.0.smithi038.stdout:0/493: write d0/d12/d42/d9d/d28/f8a [484191,7680] 0 2024-03-24T00:48:57.615 INFO:tasks.workunit.client.0.smithi038.stdout:0/494: readlink d0/d12/d42/d9d/l38 0 2024-03-24T00:48:57.616 INFO:tasks.workunit.client.1.smithi088.stdout:1/607: getdents d6/d23/daf/d95/d5b/d5d 0 2024-03-24T00:48:57.617 INFO:tasks.workunit.client.0.smithi038.stdout:6/617: creat d1/d37/d84/fc5 x:0 0 0 2024-03-24T00:48:57.619 INFO:tasks.workunit.client.1.smithi088.stdout:4/594: creat d5/d17/fd0 x:0 0 0 2024-03-24T00:48:57.619 INFO:tasks.workunit.client.0.smithi038.stdout:7/548: dread d2/f63 [0,4194304] 0 2024-03-24T00:48:57.627 INFO:tasks.workunit.client.1.smithi088.stdout:7/533: symlink d3/d32/la6 0 2024-03-24T00:48:57.627 INFO:tasks.workunit.client.1.smithi088.stdout:7/534: stat d3/d32/d76 0 2024-03-24T00:48:57.634 INFO:tasks.workunit.client.1.smithi088.stdout:4/595: write d5/d7/d92/fad [3863555,124039] 0 2024-03-24T00:48:57.634 INFO:tasks.workunit.client.1.smithi088.stdout:4/596: fsync d5/f4b 0 2024-03-24T00:48:57.634 INFO:tasks.workunit.client.1.smithi088.stdout:4/597: chown d5/d7/dd/d88/cb9 0 1 2024-03-24T00:48:57.635 INFO:tasks.workunit.client.0.smithi038.stdout:0/495: symlink d0/d12/d92/la3 0 2024-03-24T00:48:57.635 INFO:tasks.workunit.client.0.smithi038.stdout:0/496: read - d0/d14/d84/f85 zero size 2024-03-24T00:48:57.638 INFO:tasks.workunit.client.1.smithi088.stdout:8/618: dwrite d9/dc/d11/d24/f71 [4194304,4194304] 0 2024-03-24T00:48:57.644 INFO:tasks.workunit.client.0.smithi038.stdout:3/589: dwrite d3/d9/dc/da6/db5/d19/f6c [0,4194304] 0 2024-03-24T00:48:57.646 INFO:tasks.workunit.client.1.smithi088.stdout:9/631: sync 2024-03-24T00:48:57.646 INFO:tasks.workunit.client.1.smithi088.stdout:9/632: write d1/d22/f49 [7334766,17432] 0 2024-03-24T00:48:57.646 INFO:tasks.workunit.client.1.smithi088.stdout:9/633: write d1/d10/d29/f2e [1484298,102853] 0 2024-03-24T00:48:57.648 INFO:tasks.workunit.client.1.smithi088.stdout:6/624: rename db/dd/f2b to db/dd/d61/d6d/d74/fc0 0 2024-03-24T00:48:57.649 INFO:tasks.workunit.client.0.smithi038.stdout:3/590: write d3/f78 [723,91267] 0 2024-03-24T00:48:57.649 INFO:tasks.workunit.client.0.smithi038.stdout:3/591: fdatasync d3/f6 0 2024-03-24T00:48:57.649 INFO:tasks.workunit.client.0.smithi038.stdout:3/592: chown d3/d30/d5d/f5e 31462251 1 2024-03-24T00:48:57.651 INFO:tasks.workunit.client.0.smithi038.stdout:6/618: creat d1/da/fc6 x:0 0 0 2024-03-24T00:48:57.653 INFO:tasks.workunit.client.0.smithi038.stdout:4/545: dwrite da/d12/d24/d5f/d8a/db6/fb9 [0,4194304] 0 2024-03-24T00:48:57.662 INFO:tasks.workunit.client.0.smithi038.stdout:9/637: rmdir d12/d28/d71 39 2024-03-24T00:48:57.662 INFO:tasks.workunit.client.0.smithi038.stdout:9/638: truncate d12/d1a/d80/fb4 94721 0 2024-03-24T00:48:57.668 INFO:tasks.workunit.client.1.smithi088.stdout:1/608: symlink d6/d27/d67/lbb 0 2024-03-24T00:48:57.668 INFO:tasks.workunit.client.1.smithi088.stdout:2/584: sync 2024-03-24T00:48:57.668 INFO:tasks.workunit.client.1.smithi088.stdout:2/585: creat db/dd/fba x:0 0 0 2024-03-24T00:48:57.674 INFO:tasks.workunit.client.1.smithi088.stdout:7/535: mkdir d3/d32/d34/d9c/d4b/da7 0 2024-03-24T00:48:57.687 INFO:tasks.workunit.client.0.smithi038.stdout:3/593: dread d3/d9/dc/da6/db5/dad/fbd [0,4194304] 0 2024-03-24T00:48:57.687 INFO:tasks.workunit.client.0.smithi038.stdout:3/594: chown d3/ce 0 1 2024-03-24T00:48:57.693 INFO:tasks.workunit.client.1.smithi088.stdout:0/645: dwrite d9/dc/de/d19/d44/f5b [0,4194304] 0 2024-03-24T00:48:57.693 INFO:tasks.workunit.client.1.smithi088.stdout:0/646: fsync d9/dc/de/d19/d44/d46/d65/f58 0 2024-03-24T00:48:57.697 INFO:tasks.workunit.client.1.smithi088.stdout:8/619: dread d9/f57 [0,4194304] 0 2024-03-24T00:48:57.700 INFO:tasks.workunit.client.0.smithi038.stdout:8/570: dwrite d0/d15/f98 [4194304,4194304] 0 2024-03-24T00:48:57.701 INFO:tasks.workunit.client.1.smithi088.stdout:4/598: link d5/d7/f14 d5/d13/d63/d78/fd1 0 2024-03-24T00:48:57.705 INFO:tasks.workunit.client.0.smithi038.stdout:6/619: rename d1/f10 to d1/da/d1b/d2f/dc0/fc7 0 2024-03-24T00:48:57.720 INFO:tasks.workunit.client.1.smithi088.stdout:3/583: mknod d3/d1a/d1e/d76/cbb 0 2024-03-24T00:48:57.728 INFO:tasks.workunit.client.1.smithi088.stdout:8/620: dread d9/f20 [0,4194304] 0 2024-03-24T00:48:57.730 INFO:tasks.workunit.client.1.smithi088.stdout:6/625: dwrite db/d8e/fbc [0,4194304] 0 2024-03-24T00:48:57.734 INFO:tasks.workunit.client.1.smithi088.stdout:5/630: rename d1/d19/f30 to d1/dd/d6f/da1/d3f/d88/d8b/fd2 0 2024-03-24T00:48:57.736 INFO:tasks.workunit.client.0.smithi038.stdout:1/647: dwrite da/dd/d7a/d82/d98/d34/fa5 [0,4194304] 0 2024-03-24T00:48:57.736 INFO:tasks.workunit.client.0.smithi038.stdout:1/648: chown da/dd/d4a/fc2 5 1 2024-03-24T00:48:57.738 INFO:tasks.workunit.client.0.smithi038.stdout:2/528: dwrite d4/d27/d2f/d30/d3e/d9b/fa7 [0,4194304] 0 2024-03-24T00:48:57.743 INFO:tasks.workunit.client.0.smithi038.stdout:9/639: symlink d12/d28/d71/d99/ld2 0 2024-03-24T00:48:57.743 INFO:tasks.workunit.client.0.smithi038.stdout:9/640: chown d12/d28/d3a/d26/c7b 992061 1 2024-03-24T00:48:57.756 INFO:tasks.workunit.client.1.smithi088.stdout:1/609: creat d6/d27/db0/fbc x:0 0 0 2024-03-24T00:48:57.758 INFO:tasks.workunit.client.1.smithi088.stdout:7/536: link d3/d32/l63 d3/d32/d34/d9c/d4b/d74/la8 0 2024-03-24T00:48:57.760 INFO:tasks.workunit.client.1.smithi088.stdout:0/647: dwrite d9/dc/de/d19/d44/f4a [0,4194304] 0 2024-03-24T00:48:57.760 INFO:tasks.workunit.client.1.smithi088.stdout:0/648: chown d9/l16 2716564 1 2024-03-24T00:48:57.760 INFO:tasks.workunit.client.1.smithi088.stdout:0/649: truncate d9/dc/de/d19/f7f 1597228 0 2024-03-24T00:48:57.761 INFO:tasks.workunit.client.1.smithi088.stdout:3/584: fsync d3/d1a/d1e/d76/f7c 0 2024-03-24T00:48:57.771 INFO:tasks.workunit.client.0.smithi038.stdout:3/595: getdents d3/d30/d5d/d7c/dbc 0 2024-03-24T00:48:57.771 INFO:tasks.workunit.client.0.smithi038.stdout:3/596: fdatasync d3/f3d 0 2024-03-24T00:48:57.779 INFO:tasks.workunit.client.1.smithi088.stdout:8/621: dwrite d9/dc/d11/d24/d29/f2c [4194304,4194304] 0 2024-03-24T00:48:57.780 INFO:tasks.workunit.client.0.smithi038.stdout:0/497: truncate d0/d12/d42/d9d/d28/f8a 830365 0 2024-03-24T00:48:57.780 INFO:tasks.workunit.client.0.smithi038.stdout:0/498: chown d0/d12/d15/c18 55636 1 2024-03-24T00:48:57.780 INFO:tasks.workunit.client.0.smithi038.stdout:0/499: readlink d0/d12/d15/d1a/l8c 0 2024-03-24T00:48:57.780 INFO:tasks.workunit.client.0.smithi038.stdout:0/500: chown d0/d12/d42/d9d/d28/l7b 143431202 1 2024-03-24T00:48:57.781 INFO:tasks.workunit.client.0.smithi038.stdout:0/501: creat d0/d14/d84/fa4 x:0 0 0 2024-03-24T00:48:57.785 INFO:tasks.workunit.client.0.smithi038.stdout:0/502: chown d0/d12/f89 0 1 2024-03-24T00:48:57.787 INFO:tasks.workunit.client.0.smithi038.stdout:6/620: mknod d1/da/d73/d75/cc8 0 2024-03-24T00:48:57.787 INFO:tasks.workunit.client.0.smithi038.stdout:6/621: dread - d1/da/d61/da2/db1/fb9 zero size 2024-03-24T00:48:57.787 INFO:tasks.workunit.client.0.smithi038.stdout:6/622: fsync d1/da/d1b/fa8 0 2024-03-24T00:48:57.793 INFO:tasks.workunit.client.0.smithi038.stdout:1/649: mknod da/cd6 0 2024-03-24T00:48:57.804 INFO:tasks.workunit.client.0.smithi038.stdout:2/529: link d4/d18/d38/d63/l65 d4/d27/d54/d67/lbd 0 2024-03-24T00:48:57.804 INFO:tasks.workunit.client.0.smithi038.stdout:2/530: chown d4/d18/f31 16 1 2024-03-24T00:48:57.804 INFO:tasks.workunit.client.0.smithi038.stdout:9/641: unlink d12/c91 0 2024-03-24T00:48:57.808 INFO:tasks.workunit.client.1.smithi088.stdout:6/626: dwrite db/d3d/d83/d72/f76 [0,4194304] 0 2024-03-24T00:48:57.808 INFO:tasks.workunit.client.1.smithi088.stdout:9/634: rename d1/d10/d11/c13 to d1/d10/ccd 0 2024-03-24T00:48:57.809 INFO:tasks.workunit.client.1.smithi088.stdout:8/622: dread d9/dc/d11/d24/f58 [0,4194304] 0 2024-03-24T00:48:57.809 INFO:tasks.workunit.client.1.smithi088.stdout:8/623: readlink d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/l5a 0 2024-03-24T00:48:57.810 INFO:tasks.workunit.client.1.smithi088.stdout:5/631: dwrite d1/d10/d1b/d2e/d46/fb0 [0,4194304] 0 2024-03-24T00:48:57.813 INFO:tasks.workunit.client.1.smithi088.stdout:1/610: rmdir d6/d23/daf/d95/d5b/d80 39 2024-03-24T00:48:57.813 INFO:tasks.workunit.client.0.smithi038.stdout:3/597: symlink d3/d30/d70/d90/lc6 0 2024-03-24T00:48:57.815 INFO:tasks.workunit.client.1.smithi088.stdout:2/586: mknod db/d21/cbb 0 2024-03-24T00:48:57.827 INFO:tasks.workunit.client.1.smithi088.stdout:0/650: symlink d9/dc/de/d19/d44/d46/d65/ld3 0 2024-03-24T00:48:57.829 INFO:tasks.workunit.client.0.smithi038.stdout:0/503: symlink d0/d12/d15/d1a/la5 0 2024-03-24T00:48:57.832 INFO:tasks.workunit.client.1.smithi088.stdout:5/632: dread d1/d19/f45 [0,4194304] 0 2024-03-24T00:48:57.834 INFO:tasks.workunit.client.0.smithi038.stdout:6/623: mknod d1/da/d61/da2/cc9 0 2024-03-24T00:48:57.835 INFO:tasks.workunit.client.1.smithi088.stdout:1/611: dread d6/d27/f8e [4194304,4194304] 0 2024-03-24T00:48:57.842 INFO:tasks.workunit.client.1.smithi088.stdout:3/585: link d3/de/d6f/d85/d49/c4f d3/d1a/d1e/d2f/cbc 0 2024-03-24T00:48:57.846 INFO:tasks.workunit.client.0.smithi038.stdout:1/650: mkdir da/d3c/d86/dd7 0 2024-03-24T00:48:57.854 INFO:tasks.workunit.client.0.smithi038.stdout:0/504: dread d0/d12/d42/d9d/d28/f39 [0,4194304] 0 2024-03-24T00:48:57.854 INFO:tasks.workunit.client.0.smithi038.stdout:9/642: mkdir d12/d41/d57/dd3 0 2024-03-24T00:48:57.854 INFO:tasks.workunit.client.0.smithi038.stdout:0/505: creat d0/d1b/fa6 x:0 0 0 2024-03-24T00:48:57.855 INFO:tasks.workunit.client.0.smithi038.stdout:9/643: truncate d12/d28/d6c/fbd 93768 0 2024-03-24T00:48:57.856 INFO:tasks.workunit.client.0.smithi038.stdout:5/555: dwrite d6/d19/d20/d42/f72 [0,4194304] 0 2024-03-24T00:48:57.858 INFO:tasks.workunit.client.0.smithi038.stdout:4/546: dwrite da/d12/d37/dd/f21 [0,4194304] 0 2024-03-24T00:48:57.861 INFO:tasks.workunit.client.0.smithi038.stdout:3/598: truncate f1 4945372 0 2024-03-24T00:48:57.861 INFO:tasks.workunit.client.0.smithi038.stdout:3/599: stat d3/f8c 0 2024-03-24T00:48:57.861 INFO:tasks.workunit.client.0.smithi038.stdout:3/600: stat d3/d9/dc/da6/fab 0 2024-03-24T00:48:57.866 INFO:tasks.workunit.client.1.smithi088.stdout:1/612: write d6/d23/f33 [3976059,58999] 0 2024-03-24T00:48:57.867 INFO:tasks.workunit.client.0.smithi038.stdout:6/624: mknod d1/d37/d68/cca 0 2024-03-24T00:48:57.868 INFO:tasks.workunit.client.0.smithi038.stdout:6/625: truncate d1/da/d1b/d1d/d3c/fb4 883039 0 2024-03-24T00:48:57.875 INFO:tasks.workunit.client.1.smithi088.stdout:9/635: creat d1/d10/d9b/dab/db8/fce x:0 0 0 2024-03-24T00:48:57.883 INFO:tasks.workunit.client.1.smithi088.stdout:7/537: dwrite d3/f20 [4194304,4194304] 0 2024-03-24T00:48:57.884 INFO:tasks.workunit.client.0.smithi038.stdout:2/531: dwrite f2 [0,4194304] 0 2024-03-24T00:48:57.886 INFO:tasks.workunit.client.0.smithi038.stdout:1/651: dread da/dd/d2c/d40/f53 [0,4194304] 0 2024-03-24T00:48:57.890 INFO:tasks.workunit.client.0.smithi038.stdout:8/571: dwrite d0/d15/d2c/f88 [0,4194304] 0 2024-03-24T00:48:57.894 INFO:tasks.workunit.client.0.smithi038.stdout:0/506: mkdir d0/d12/d15/d58/d64/da7 0 2024-03-24T00:48:57.895 INFO:tasks.workunit.client.0.smithi038.stdout:2/532: dread d4/d27/d54/d53/f64 [0,4194304] 0 2024-03-24T00:48:57.896 INFO:tasks.workunit.client.0.smithi038.stdout:9/644: creat d12/d81/d84/d88/fd4 x:0 0 0 2024-03-24T00:48:57.896 INFO:tasks.workunit.client.0.smithi038.stdout:9/645: chown fe 2396 1 2024-03-24T00:48:57.919 INFO:tasks.workunit.client.1.smithi088.stdout:8/624: mkdir d9/dc/d11/d24/d43/dd8 0 2024-03-24T00:48:57.921 INFO:tasks.workunit.client.1.smithi088.stdout:8/625: fdatasync d9/dc/f16 0 2024-03-24T00:48:57.922 INFO:tasks.workunit.client.0.smithi038.stdout:4/547: symlink da/d12/d29/d5b/d73/lbc 0 2024-03-24T00:48:57.923 INFO:tasks.workunit.client.1.smithi088.stdout:2/587: link db/d21/f74 db/dd/d96/d4e/d8f/fbc 0 2024-03-24T00:48:57.923 INFO:tasks.workunit.client.1.smithi088.stdout:2/588: chown db/d21/d35/d49 100 1 2024-03-24T00:48:57.927 INFO:tasks.workunit.client.0.smithi038.stdout:5/556: dwrite d6/d19/d20/d42/d43/d4c/f62 [0,4194304] 0 2024-03-24T00:48:57.927 INFO:tasks.workunit.client.0.smithi038.stdout:5/557: dread - d6/d19/d20/d33/d6f/faf zero size 2024-03-24T00:48:57.927 INFO:tasks.workunit.client.0.smithi038.stdout:6/626: dwrite d1/f6b [0,4194304] 0 2024-03-24T00:48:57.927 INFO:tasks.workunit.client.0.smithi038.stdout:3/601: unlink d3/d30/c67 0 2024-03-24T00:48:57.927 INFO:tasks.workunit.client.0.smithi038.stdout:3/602: creat d3/d9/dc/da6/fc7 x:0 0 0 2024-03-24T00:48:57.927 INFO:tasks.workunit.client.0.smithi038.stdout:3/603: truncate d3/d30/d5d/d6a/f8b 137841 0 2024-03-24T00:48:57.927 INFO:tasks.workunit.client.0.smithi038.stdout:3/604: readlink d3/d2a/l6d 0 2024-03-24T00:48:57.927 INFO:tasks.workunit.client.0.smithi038.stdout:3/605: chown d3/d30/l4d 7108 1 2024-03-24T00:48:57.928 INFO:tasks.workunit.client.1.smithi088.stdout:0/651: link d9/l16 d9/d26/d8f/da1/d62/d9b/dcb/ld4 0 2024-03-24T00:48:57.929 INFO:tasks.workunit.client.0.smithi038.stdout:1/652: symlink da/dd/d7a/d82/d98/d9e/ld8 0 2024-03-24T00:48:57.932 INFO:tasks.workunit.client.1.smithi088.stdout:5/633: mknod d1/cd3 0 2024-03-24T00:48:57.938 INFO:tasks.workunit.client.1.smithi088.stdout:1/613: symlink d6/d23/lbd 0 2024-03-24T00:48:57.941 INFO:tasks.workunit.client.0.smithi038.stdout:5/558: dread d6/d19/d20/d42/d43/d68/f8b [4194304,4194304] 0 2024-03-24T00:48:57.942 INFO:tasks.workunit.client.1.smithi088.stdout:9/636: rmdir d1/d10/d77/d8a 39 2024-03-24T00:48:57.942 INFO:tasks.workunit.client.1.smithi088.stdout:4/599: sync 2024-03-24T00:48:57.942 INFO:tasks.workunit.client.1.smithi088.stdout:9/637: write d1/d15/f68 [4302385,11913] 0 2024-03-24T00:48:57.942 INFO:tasks.workunit.client.0.smithi038.stdout:0/507: unlink d0/d12/d15/d1a/f7a 0 2024-03-24T00:48:57.942 INFO:tasks.workunit.client.0.smithi038.stdout:0/508: rename d0/d12/d9a to d0/d12/d9a/da8 22 2024-03-24T00:48:57.944 INFO:tasks.workunit.client.1.smithi088.stdout:7/538: chown d3/f35 420 1 2024-03-24T00:48:57.949 INFO:tasks.workunit.client.1.smithi088.stdout:8/626: mkdir d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/dd9 0 2024-03-24T00:48:57.950 INFO:tasks.workunit.client.0.smithi038.stdout:9/646: link d12/d1a/d5f/c65 d12/d28/da8/cd5 0 2024-03-24T00:48:57.952 INFO:tasks.workunit.client.0.smithi038.stdout:4/548: mkdir da/dbd 0 2024-03-24T00:48:57.954 INFO:tasks.workunit.client.0.smithi038.stdout:8/572: dwrite d0/d45/d66/d6d/fbc [0,4194304] 0 2024-03-24T00:48:57.954 INFO:tasks.workunit.client.0.smithi038.stdout:8/573: write d0/d45/f89 [1149003,69800] 0 2024-03-24T00:48:57.959 INFO:tasks.workunit.client.1.smithi088.stdout:5/634: rename d1/d10/d1b/d2e/d46/lac to d1/dd/d6f/da1/d3f/d88/d8f/d69/ld4 0 2024-03-24T00:48:57.965 INFO:tasks.workunit.client.1.smithi088.stdout:3/586: link d3/d1a/d1e/d2f/d45/f6b d3/d1a/d1e/d76/fbd 0 2024-03-24T00:48:57.966 INFO:tasks.workunit.client.1.smithi088.stdout:1/614: creat d6/d23/daf/d95/d58/fbe x:0 0 0 2024-03-24T00:48:57.967 INFO:tasks.workunit.client.1.smithi088.stdout:1/615: write d6/d23/daf/d95/d5b/fa8 [218655,78895] 0 2024-03-24T00:48:57.967 INFO:tasks.workunit.client.0.smithi038.stdout:6/627: truncate d1/da/d1b/d25/d49/d4e/d51/f5e 29710 0 2024-03-24T00:48:57.967 INFO:tasks.workunit.client.0.smithi038.stdout:6/628: creat d1/da/d1b/d2f/fcb x:0 0 0 2024-03-24T00:48:57.971 INFO:tasks.workunit.client.1.smithi088.stdout:4/600: symlink d5/d13/d18/d24/ld2 0 2024-03-24T00:48:57.971 INFO:tasks.workunit.client.1.smithi088.stdout:4/601: write d5/d13/d18/f1b [4362584,12268] 0 2024-03-24T00:48:57.971 INFO:tasks.workunit.client.0.smithi038.stdout:3/606: truncate d3/d30/f33 42289 0 2024-03-24T00:48:57.973 INFO:tasks.workunit.client.1.smithi088.stdout:9/638: mkdir d1/dcf 0 2024-03-24T00:48:57.973 INFO:tasks.workunit.client.0.smithi038.stdout:1/653: rmdir da/dd/d7a 39 2024-03-24T00:48:57.975 INFO:tasks.workunit.client.1.smithi088.stdout:8/627: stat d9/dc/d11/d24/d43/d4a/c5f 0 2024-03-24T00:48:57.975 INFO:tasks.workunit.client.1.smithi088.stdout:8/628: stat d9/dc/d11/d24/d43/l72 0 2024-03-24T00:48:57.977 INFO:tasks.workunit.client.0.smithi038.stdout:5/559: creat d6/d19/d20/d42/d79/fb6 x:0 0 0 2024-03-24T00:48:57.978 INFO:tasks.workunit.client.0.smithi038.stdout:0/509: truncate d0/d12/d42/d9d/d28/f8a 516867 0 2024-03-24T00:48:57.978 INFO:tasks.workunit.client.0.smithi038.stdout:0/510: creat d0/d12/d42/d44/fa9 x:0 0 0 2024-03-24T00:48:57.979 INFO:tasks.workunit.client.1.smithi088.stdout:6/627: dwrite db/d1d/d23/d3c/d43/f34 [0,4194304] 0 2024-03-24T00:48:57.979 INFO:tasks.workunit.client.0.smithi038.stdout:0/511: write d0/d2d/f6f [1867449,126102] 0 2024-03-24T00:48:57.982 INFO:tasks.workunit.client.0.smithi038.stdout:2/533: rmdir d4/d27/d2f/d30/d3e/d9b 39 2024-03-24T00:48:57.982 INFO:tasks.workunit.client.0.smithi038.stdout:9/647: creat d12/d1a/d5f/fd6 x:0 0 0 2024-03-24T00:48:57.984 INFO:tasks.workunit.client.0.smithi038.stdout:7/549: sync 2024-03-24T00:48:57.984 INFO:tasks.workunit.client.0.smithi038.stdout:7/550: fdatasync d2/d2c/f69 0 2024-03-24T00:48:57.984 INFO:tasks.workunit.client.0.smithi038.stdout:7/551: fsync d2/d9/da/d23/d2b/d5b/d72/d92/d33/f27 0 2024-03-24T00:48:57.984 INFO:tasks.workunit.client.0.smithi038.stdout:7/552: stat d2/d9/da/c4f 0 2024-03-24T00:48:57.985 INFO:tasks.workunit.client.1.smithi088.stdout:2/589: symlink db/d21/lbd 0 2024-03-24T00:48:57.985 INFO:tasks.workunit.client.1.smithi088.stdout:5/635: symlink d1/d10/d81/ld5 0 2024-03-24T00:48:57.988 INFO:tasks.workunit.client.1.smithi088.stdout:3/587: symlink d3/d1a/d1e/d2f/d45/lbe 0 2024-03-24T00:48:57.990 INFO:tasks.workunit.client.0.smithi038.stdout:4/549: creat da/fbe x:0 0 0 2024-03-24T00:48:57.990 INFO:tasks.workunit.client.0.smithi038.stdout:4/550: chown da/d12/d24/d5f/f97 942562127 1 2024-03-24T00:48:57.990 INFO:tasks.workunit.client.0.smithi038.stdout:4/551: readlink da/d12/d24/d68/d6c/l80 0 2024-03-24T00:48:57.990 INFO:tasks.workunit.client.0.smithi038.stdout:4/552: dread - da/d12/d24/d5f/d9a/f92 zero size 2024-03-24T00:48:57.993 INFO:tasks.workunit.client.1.smithi088.stdout:4/602: truncate d5/d13/fa7 4232205 0 2024-03-24T00:48:57.993 INFO:tasks.workunit.client.1.smithi088.stdout:4/603: dread - d5/d16/d37/fae zero size 2024-03-24T00:48:57.993 INFO:tasks.workunit.client.1.smithi088.stdout:4/604: fdatasync d5/d13/d18/d6e/d7d/fbb 0 2024-03-24T00:48:57.993 INFO:tasks.workunit.client.1.smithi088.stdout:4/605: write d5/d17/fd0 [944637,130980] 0 2024-03-24T00:48:57.998 INFO:tasks.workunit.client.1.smithi088.stdout:9/639: truncate d1/d22/d27/f8d 2790465 0 2024-03-24T00:48:57.998 INFO:tasks.workunit.client.1.smithi088.stdout:9/640: stat d1/d22/d27/f8d 0 2024-03-24T00:48:57.999 INFO:tasks.workunit.client.0.smithi038.stdout:2/534: dread d4/d18/d26/f2c [0,4194304] 0 2024-03-24T00:48:58.000 INFO:tasks.workunit.client.0.smithi038.stdout:8/574: truncate d0/d1d/d27/d4d/f7a 1174997 0 2024-03-24T00:48:58.001 INFO:tasks.workunit.client.1.smithi088.stdout:8/629: creat d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/fda x:0 0 0 2024-03-24T00:48:58.005 INFO:tasks.workunit.client.0.smithi038.stdout:3/607: unlink d3/d2a/fb6 0 2024-03-24T00:48:58.007 INFO:tasks.workunit.client.0.smithi038.stdout:1/654: mkdir da/dd/d7a/d82/d98/d34/dd9 0 2024-03-24T00:48:58.007 INFO:tasks.workunit.client.0.smithi038.stdout:1/655: stat da/dd/d7a 0 2024-03-24T00:48:58.007 INFO:tasks.workunit.client.0.smithi038.stdout:1/656: truncate da/dd/d7a/d82/d98/fba 1143304 0 2024-03-24T00:48:58.009 INFO:tasks.workunit.client.1.smithi088.stdout:6/628: rename db/d1d/d50/d90/cb3 to db/d1d/d23/d3c/d43/daf/cc1 0 2024-03-24T00:48:58.010 INFO:tasks.workunit.client.0.smithi038.stdout:5/560: getdents d6/d16/d21/d2d/da6 0 2024-03-24T00:48:58.020 INFO:tasks.workunit.client.1.smithi088.stdout:5/636: mknod d1/d10/d1b/d2f/d57/d9e/dba/dbb/cd6 0 2024-03-24T00:48:58.020 INFO:tasks.workunit.client.1.smithi088.stdout:2/590: truncate db/d21/d35/f79 3654333 0 2024-03-24T00:48:58.020 INFO:tasks.workunit.client.1.smithi088.stdout:5/637: chown d1/dd/d6f/da1/d3f/d88/d8f/d69 257823 1 2024-03-24T00:48:58.020 INFO:tasks.workunit.client.1.smithi088.stdout:5/638: write d1/fb [907325,72055] 0 2024-03-24T00:48:58.020 INFO:tasks.workunit.client.1.smithi088.stdout:0/652: dwrite d9/d26/f5a [0,4194304] 0 2024-03-24T00:48:58.021 INFO:tasks.workunit.client.0.smithi038.stdout:9/648: unlink d12/l13 0 2024-03-24T00:48:58.024 INFO:tasks.workunit.client.1.smithi088.stdout:9/641: write d1/d10/d29/f45 [508715,22824] 0 2024-03-24T00:48:58.024 INFO:tasks.workunit.client.0.smithi038.stdout:4/553: write da/d12/f3f [1369860,68768] 0 2024-03-24T00:48:58.026 INFO:tasks.workunit.client.1.smithi088.stdout:4/606: symlink d5/d13/d18/d24/db3/ld3 0 2024-03-24T00:48:58.026 INFO:tasks.workunit.client.1.smithi088.stdout:4/607: fsync d5/d7/d72/f71 0 2024-03-24T00:48:58.030 INFO:tasks.workunit.client.0.smithi038.stdout:0/512: dwrite d0/d12/d15/d58/d64/f7d [0,4194304] 0 2024-03-24T00:48:58.030 INFO:tasks.workunit.client.0.smithi038.stdout:7/553: truncate d2/d9/da/d23/d2b/d5b/d72/d92/f71 2253663 0 2024-03-24T00:48:58.030 INFO:tasks.workunit.client.0.smithi038.stdout:7/554: chown d2/d9/da/d23/d2b/d5b/d72/d92/f35 121 1 2024-03-24T00:48:58.030 INFO:tasks.workunit.client.0.smithi038.stdout:0/513: dread - d0/d12/f89 zero size 2024-03-24T00:48:58.033 INFO:tasks.workunit.client.1.smithi088.stdout:6/629: rename db/d1d/d23/d3c/f92 to db/d1d/d50/d90/d55/fc2 0 2024-03-24T00:48:58.036 INFO:tasks.workunit.client.1.smithi088.stdout:3/588: dread d3/de/d6f/d85/d49/f54 [4194304,4194304] 0 2024-03-24T00:48:58.037 INFO:tasks.workunit.client.0.smithi038.stdout:8/575: getdents d0/d1d/d27 0 2024-03-24T00:48:58.039 INFO:tasks.workunit.client.1.smithi088.stdout:1/616: dwrite d6/d23/daf/d95/d58/f81 [0,4194304] 0 2024-03-24T00:48:58.039 INFO:tasks.workunit.client.0.smithi038.stdout:6/629: getdents d1/d37/d68 0 2024-03-24T00:48:58.044 INFO:tasks.workunit.client.1.smithi088.stdout:2/591: mknod db/dd/d96/d4e/d8f/d9c/cbe 0 2024-03-24T00:48:58.045 INFO:tasks.workunit.client.1.smithi088.stdout:4/608: dread d5/d13/d18/f9e [0,4194304] 0 2024-03-24T00:48:58.045 INFO:tasks.workunit.client.1.smithi088.stdout:4/609: creat d5/d16/d94/fd4 x:0 0 0 2024-03-24T00:48:58.045 INFO:tasks.workunit.client.1.smithi088.stdout:4/610: truncate d5/d13/d18/d24/f9c 655163 0 2024-03-24T00:48:58.045 INFO:tasks.workunit.client.1.smithi088.stdout:5/639: unlink d1/dd/d6f/da1/f77 0 2024-03-24T00:48:58.046 INFO:tasks.workunit.client.1.smithi088.stdout:1/617: dread d6/d27/d67/d88/f92 [0,4194304] 0 2024-03-24T00:48:58.046 INFO:tasks.workunit.client.0.smithi038.stdout:1/657: mkdir da/dd/d22/d5c/dda 0 2024-03-24T00:48:58.046 INFO:tasks.workunit.client.0.smithi038.stdout:1/658: fdatasync da/dd/d2c/d5a/d74/f97 0 2024-03-24T00:48:58.047 INFO:tasks.workunit.client.1.smithi088.stdout:0/653: mknod d9/dc/de/d7e/d91/cd5 0 2024-03-24T00:48:58.047 INFO:tasks.workunit.client.1.smithi088.stdout:0/654: fdatasync d9/dc/de/fcf 0 2024-03-24T00:48:58.051 INFO:tasks.workunit.client.0.smithi038.stdout:4/554: truncate da/d12/d24/d68/d6c/f79 1176458 0 2024-03-24T00:48:58.051 INFO:tasks.workunit.client.0.smithi038.stdout:4/555: creat da/d27/fbf x:0 0 0 2024-03-24T00:48:58.051 INFO:tasks.workunit.client.0.smithi038.stdout:4/556: chown da/d12/d29/d5b/d73/db4 503494 1 2024-03-24T00:48:58.051 INFO:tasks.workunit.client.0.smithi038.stdout:4/557: write da/d12/d29/d5b/d73/fb2 [598592,51081] 0 2024-03-24T00:48:58.053 INFO:tasks.workunit.client.1.smithi088.stdout:8/630: write d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/f91 [4122883,35958] 0 2024-03-24T00:48:58.057 INFO:tasks.workunit.client.0.smithi038.stdout:0/514: link d0/d14/c41 d0/d12/d42/d5e/caa 0 2024-03-24T00:48:58.059 INFO:tasks.workunit.client.1.smithi088.stdout:6/630: symlink db/d8e/lc3 0 2024-03-24T00:48:58.059 INFO:tasks.workunit.client.1.smithi088.stdout:6/631: write db/d1d/d23/d3c/d43/f28 [4405879,48055] 0 2024-03-24T00:48:58.060 INFO:tasks.workunit.client.1.smithi088.stdout:6/632: stat db/d3d 0 2024-03-24T00:48:58.060 INFO:tasks.workunit.client.1.smithi088.stdout:6/633: fdatasync db/dd/f1e 0 2024-03-24T00:48:58.060 INFO:tasks.workunit.client.1.smithi088.stdout:0/655: write d9/dc/de/f6c [2537315,105236] 0 2024-03-24T00:48:58.060 INFO:tasks.workunit.client.1.smithi088.stdout:0/656: chown d9/dc/de/d19/d44/d46/d65/d2a/c71 377 1 2024-03-24T00:48:58.061 INFO:tasks.workunit.client.0.smithi038.stdout:7/555: link d2/d9/da/d23/f39 d2/d9/da/d23/d2b/d5b/d72/d92/d75/fd2 0 2024-03-24T00:48:58.063 INFO:tasks.workunit.client.0.smithi038.stdout:2/535: rmdir d4/d18/da1 39 2024-03-24T00:48:58.068 INFO:tasks.workunit.client.0.smithi038.stdout:8/576: truncate d0/d45/d66/f97 2791870 0 2024-03-24T00:48:58.070 INFO:tasks.workunit.client.0.smithi038.stdout:6/630: creat d1/da/d1b/d25/d49/fcc x:0 0 0 2024-03-24T00:48:58.070 INFO:tasks.workunit.client.0.smithi038.stdout:6/631: write d1/da/d61/d79/fb0 [687802,70271] 0 2024-03-24T00:48:58.070 INFO:tasks.workunit.client.0.smithi038.stdout:6/632: readlink d1/l9b 0 2024-03-24T00:48:58.073 INFO:tasks.workunit.client.0.smithi038.stdout:1/659: mkdir da/dd/d7a/d82/ddb 0 2024-03-24T00:48:58.073 INFO:tasks.workunit.client.1.smithi088.stdout:2/592: mkdir db/dd/d96/d4e/d8f/d9c/db0/dbf 0 2024-03-24T00:48:58.078 INFO:tasks.workunit.client.0.smithi038.stdout:2/536: dread d4/f24 [0,4194304] 0 2024-03-24T00:48:58.078 INFO:tasks.workunit.client.0.smithi038.stdout:2/537: write d4/d27/d54/d53/f6b [1030217,89749] 0 2024-03-24T00:48:58.078 INFO:tasks.workunit.client.0.smithi038.stdout:2/538: chown d4/d18/d5b/l74 10891 1 2024-03-24T00:48:58.078 INFO:tasks.workunit.client.0.smithi038.stdout:2/539: truncate d4/d27/d2f/d89/f9a 631492 0 2024-03-24T00:48:58.079 INFO:tasks.workunit.client.1.smithi088.stdout:4/611: rmdir d5/d8b 39 2024-03-24T00:48:58.079 INFO:tasks.workunit.client.0.smithi038.stdout:8/577: dread d0/d1d/d27/d4d/f7a [0,4194304] 0 2024-03-24T00:48:58.080 INFO:tasks.workunit.client.0.smithi038.stdout:4/558: creat da/d12/d24/fc0 x:0 0 0 2024-03-24T00:48:58.082 INFO:tasks.workunit.client.1.smithi088.stdout:5/640: rmdir d1/dd/d6f/da1/d3f 39 2024-03-24T00:48:58.105 INFO:tasks.workunit.client.0.smithi038.stdout:7/556: mknod d2/d9/da/dc/d93/cd3 0 2024-03-24T00:48:58.108 INFO:tasks.workunit.client.0.smithi038.stdout:6/633: rename d1/da/d1b/f91 to d1/da/d61/fcd 0 2024-03-24T00:48:58.121 INFO:tasks.workunit.client.0.smithi038.stdout:7/557: read d2/f1f [6711264,76645] 0 2024-03-24T00:48:58.121 INFO:tasks.workunit.client.0.smithi038.stdout:7/558: read d2/d9/da/d23/d2b/d5b/d72/d92/d5d/f7f [1639442,93559] 0 2024-03-24T00:48:58.121 INFO:tasks.workunit.client.0.smithi038.stdout:4/559: unlink da/d12/fa0 0 2024-03-24T00:48:58.122 INFO:tasks.workunit.client.1.smithi088.stdout:9/642: getdents d1/d22/d27 0 2024-03-24T00:48:58.123 INFO:tasks.workunit.client.0.smithi038.stdout:0/515: rmdir d0/d12/d42/d44 39 2024-03-24T00:48:58.123 INFO:tasks.workunit.client.0.smithi038.stdout:0/516: write d0/d1b/f87 [989428,102128] 0 2024-03-24T00:48:58.123 INFO:tasks.workunit.client.0.smithi038.stdout:0/517: read - d0/d12/d42/d9d/d28/f9b zero size 2024-03-24T00:48:58.123 INFO:tasks.workunit.client.1.smithi088.stdout:0/657: dwrite d9/dc/de/d19/d44/f5b [4194304,4194304] 0 2024-03-24T00:48:58.125 INFO:tasks.workunit.client.0.smithi038.stdout:1/660: getdents da/dd/d22/d5c 0 2024-03-24T00:48:58.126 INFO:tasks.workunit.client.1.smithi088.stdout:6/634: unlink db/d1d/d50/d90/d55/ca8 0 2024-03-24T00:48:58.126 INFO:tasks.workunit.client.1.smithi088.stdout:4/612: rmdir d5/d7 39 2024-03-24T00:48:58.127 INFO:tasks.workunit.client.1.smithi088.stdout:5/641: mknod d1/dd/d6f/da1/d3f/d88/cd7 0 2024-03-24T00:48:58.127 INFO:tasks.workunit.client.1.smithi088.stdout:5/642: write d1/dd/d6f/da1/d3f/d88/d8f/d69/f8c [17780,56295] 0 2024-03-24T00:48:58.129 INFO:tasks.workunit.client.0.smithi038.stdout:8/578: rmdir d0/d45/d66/da6 39 2024-03-24T00:48:58.130 INFO:tasks.workunit.client.1.smithi088.stdout:1/618: rmdir d6/d27/d67/d88 39 2024-03-24T00:48:58.130 INFO:tasks.workunit.client.1.smithi088.stdout:1/619: write d6/d23/daf/f50 [752658,58686] 0 2024-03-24T00:48:58.130 INFO:tasks.workunit.client.1.smithi088.stdout:1/620: fdatasync d6/d23/daf/d95/f39 0 2024-03-24T00:48:58.131 INFO:tasks.workunit.client.0.smithi038.stdout:7/559: creat d2/d9/da/dc/da8/dc5/fd4 x:0 0 0 2024-03-24T00:48:58.131 INFO:tasks.workunit.client.0.smithi038.stdout:7/560: readlink d2/d9/da/d23/d2b/d5b/l64 0 2024-03-24T00:48:58.131 INFO:tasks.workunit.client.0.smithi038.stdout:7/561: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/d33/f3c 0 2024-03-24T00:48:58.131 INFO:tasks.workunit.client.0.smithi038.stdout:7/562: creat d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fd5 x:0 0 0 2024-03-24T00:48:58.132 INFO:tasks.workunit.client.1.smithi088.stdout:9/643: link d1/d22/d27/d39/d98/lca d1/d10/d9b/dab/d7a/ld0 0 2024-03-24T00:48:58.133 INFO:tasks.workunit.client.0.smithi038.stdout:4/560: truncate da/d12/d24/d68/d6c/f79 674498 0 2024-03-24T00:48:58.134 INFO:tasks.workunit.client.0.smithi038.stdout:6/634: dread d1/da/f1a [0,4194304] 0 2024-03-24T00:48:58.136 INFO:tasks.workunit.client.0.smithi038.stdout:0/518: unlink d0/d12/d42/d5e/f67 0 2024-03-24T00:48:58.137 INFO:tasks.workunit.client.0.smithi038.stdout:1/661: mkdir da/ddc 0 2024-03-24T00:48:58.138 INFO:tasks.workunit.client.0.smithi038.stdout:9/649: dwrite d12/d28/f36 [0,4194304] 0 2024-03-24T00:48:58.138 INFO:tasks.workunit.client.0.smithi038.stdout:3/608: dwrite d3/f6 [4194304,4194304] 0 2024-03-24T00:48:58.139 INFO:tasks.workunit.client.0.smithi038.stdout:3/609: creat d3/d5a/fc8 x:0 0 0 2024-03-24T00:48:58.139 INFO:tasks.workunit.client.0.smithi038.stdout:3/610: dread - d3/d9/dc/da6/fab zero size 2024-03-24T00:48:58.140 INFO:tasks.workunit.client.1.smithi088.stdout:0/658: dread d9/dc/de/d19/d44/d46/d65/f68 [0,4194304] 0 2024-03-24T00:48:58.140 INFO:tasks.workunit.client.1.smithi088.stdout:4/613: link d5/d13/d63/d78/lc9 d5/d16/ld5 0 2024-03-24T00:48:58.144 INFO:tasks.workunit.client.1.smithi088.stdout:6/635: rename db/dd/f1e to db/dd/d61/fc4 0 2024-03-24T00:48:58.145 INFO:tasks.workunit.client.0.smithi038.stdout:8/579: unlink d0/d45/d37/d5f/db7/l1c 0 2024-03-24T00:48:58.151 INFO:tasks.workunit.client.0.smithi038.stdout:6/635: mknod d1/da/d1b/d25/cce 0 2024-03-24T00:48:58.151 INFO:tasks.workunit.client.0.smithi038.stdout:4/561: symlink da/d12/d24/d68/d6c/da6/lc1 0 2024-03-24T00:48:58.151 INFO:tasks.workunit.client.0.smithi038.stdout:0/519: getdents d0/d12/d15/d58/d64 0 2024-03-24T00:48:58.152 INFO:tasks.workunit.client.0.smithi038.stdout:0/520: dread - d0/f78 zero size 2024-03-24T00:48:58.152 INFO:tasks.workunit.client.0.smithi038.stdout:2/540: dwrite d4/d27/d54/d53/f7f [0,4194304] 0 2024-03-24T00:48:58.152 INFO:tasks.workunit.client.0.smithi038.stdout:2/541: write d4/d76/fb3 [43559,33376] 0 2024-03-24T00:48:58.152 INFO:tasks.workunit.client.1.smithi088.stdout:2/593: dwrite db/d21/d73/f83 [0,4194304] 0 2024-03-24T00:48:58.152 INFO:tasks.workunit.client.1.smithi088.stdout:2/594: fsync db/d21/d35/d49/f8c 0 2024-03-24T00:48:58.152 INFO:tasks.workunit.client.1.smithi088.stdout:2/595: truncate db/dd/f85 709381 0 2024-03-24T00:48:58.153 INFO:tasks.workunit.client.1.smithi088.stdout:8/631: dwrite d9/f2b [0,4194304] 0 2024-03-24T00:48:58.153 INFO:tasks.workunit.client.1.smithi088.stdout:1/621: symlink d6/d23/lbf 0 2024-03-24T00:48:58.153 INFO:tasks.workunit.client.0.smithi038.stdout:1/662: mkdir da/ddc/ddd 0 2024-03-24T00:48:58.156 INFO:tasks.workunit.client.1.smithi088.stdout:5/643: dread d1/d10/d81/f97 [0,4194304] 0 2024-03-24T00:48:58.156 INFO:tasks.workunit.client.1.smithi088.stdout:5/644: creat d1/dd/d6f/da1/d3f/d88/d8b/fd8 x:0 0 0 2024-03-24T00:48:58.161 INFO:tasks.workunit.client.1.smithi088.stdout:6/636: mkdir db/dc5 0 2024-03-24T00:48:58.164 INFO:tasks.workunit.client.1.smithi088.stdout:7/539: sync 2024-03-24T00:48:58.172 INFO:tasks.workunit.client.1.smithi088.stdout:4/614: write d5/d13/d63/d78/fd1 [126351,88810] 0 2024-03-24T00:48:58.175 INFO:tasks.workunit.client.0.smithi038.stdout:6/636: dread d1/f6 [0,4194304] 0 2024-03-24T00:48:58.176 INFO:tasks.workunit.client.0.smithi038.stdout:6/637: creat d1/d37/d84/fcf x:0 0 0 2024-03-24T00:48:58.176 INFO:tasks.workunit.client.0.smithi038.stdout:6/638: creat d1/da/d1b/d2f/fd0 x:0 0 0 2024-03-24T00:48:58.176 INFO:tasks.workunit.client.0.smithi038.stdout:6/639: readlink d1/da/d1b/d1d/l9a 0 2024-03-24T00:48:58.176 INFO:tasks.workunit.client.0.smithi038.stdout:6/640: dread - d1/da/d1b/fa1 zero size 2024-03-24T00:48:58.177 INFO:tasks.workunit.client.0.smithi038.stdout:3/611: rename d3/d9/dc/da6/db5/d19/f47 to d3/d9/d77/fc9 0 2024-03-24T00:48:58.183 INFO:tasks.workunit.client.0.smithi038.stdout:8/580: mknod d0/d15/cbf 0 2024-03-24T00:48:58.185 INFO:tasks.workunit.client.0.smithi038.stdout:4/562: symlink da/d12/d24/d68/d6c/da6/lc2 0 2024-03-24T00:48:58.189 INFO:tasks.workunit.client.0.smithi038.stdout:2/542: mknod d4/d55/cbe 0 2024-03-24T00:48:58.190 INFO:tasks.workunit.client.0.smithi038.stdout:1/663: rmdir da/dd/d4a/d8b 39 2024-03-24T00:48:58.192 INFO:tasks.workunit.client.1.smithi088.stdout:8/632: mknod d9/dc/d11/d24/d29/d5b/db9/cdb 0 2024-03-24T00:48:58.194 INFO:tasks.workunit.client.1.smithi088.stdout:1/622: creat d6/d23/daf/d95/d5b/d5d/d61/d7d/da5/fc0 x:0 0 0 2024-03-24T00:48:58.196 INFO:tasks.workunit.client.1.smithi088.stdout:5/645: symlink d1/d10/d1b/d2f/d57/d9e/ld9 0 2024-03-24T00:48:58.196 INFO:tasks.workunit.client.1.smithi088.stdout:5/646: chown d1/d10/d81/cc6 1760016 1 2024-03-24T00:48:58.196 INFO:tasks.workunit.client.1.smithi088.stdout:5/647: write d1/d10/d1b/d2f/d57/d9e/fb5 [508709,51433] 0 2024-03-24T00:48:58.197 INFO:tasks.workunit.client.0.smithi038.stdout:7/563: dwrite d2/d9/d16/fc6 [0,4194304] 0 2024-03-24T00:48:58.197 INFO:tasks.workunit.client.0.smithi038.stdout:7/564: fsync d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f8f 0 2024-03-24T00:48:58.197 INFO:tasks.workunit.client.0.smithi038.stdout:7/565: readlink d2/d9/d16/l1e 0 2024-03-24T00:48:58.197 INFO:tasks.workunit.client.1.smithi088.stdout:5/648: read d1/d19/d1e/f9b [218424,39713] 0 2024-03-24T00:48:58.198 INFO:tasks.workunit.client.1.smithi088.stdout:5/649: truncate d1/d19/fc0 671571 0 2024-03-24T00:48:58.198 INFO:tasks.workunit.client.1.smithi088.stdout:6/637: mknod db/d3d/d83/cc6 0 2024-03-24T00:48:58.198 INFO:tasks.workunit.client.0.smithi038.stdout:7/566: creat d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fd6 x:0 0 0 2024-03-24T00:48:58.198 INFO:tasks.workunit.client.1.smithi088.stdout:7/540: mknod d3/d32/d34/d7c/ca9 0 2024-03-24T00:48:58.202 INFO:tasks.workunit.client.1.smithi088.stdout:4/615: creat d5/d13/d63/d78/fd6 x:0 0 0 2024-03-24T00:48:58.203 INFO:tasks.workunit.client.0.smithi038.stdout:4/563: dread da/d12/d24/d68/f82 [0,4194304] 0 2024-03-24T00:48:58.204 INFO:tasks.workunit.client.0.smithi038.stdout:4/564: creat da/d12/d29/d5b/d73/db4/fc3 x:0 0 0 2024-03-24T00:48:58.204 INFO:tasks.workunit.client.0.smithi038.stdout:0/521: dwrite d0/d12/d42/f51 [4194304,4194304] 0 2024-03-24T00:48:58.205 INFO:tasks.workunit.client.0.smithi038.stdout:2/543: dread d4/d18/d38/d63/f7e [0,4194304] 0 2024-03-24T00:48:58.205 INFO:tasks.workunit.client.1.smithi088.stdout:2/596: dwrite db/f89 [0,4194304] 0 2024-03-24T00:48:58.206 INFO:tasks.workunit.client.1.smithi088.stdout:0/659: dwrite d9/dc/de/d19/d44/d46/d65/d2a/f92 [0,4194304] 0 2024-03-24T00:48:58.209 INFO:tasks.workunit.client.1.smithi088.stdout:1/623: mkdir d6/d6e/dc1 0 2024-03-24T00:48:58.209 INFO:tasks.workunit.client.1.smithi088.stdout:4/616: read f4 [56677,13911] 0 2024-03-24T00:48:58.212 INFO:tasks.workunit.client.0.smithi038.stdout:9/650: getdents d12/d28/d3a 0 2024-03-24T00:48:58.214 INFO:tasks.workunit.client.1.smithi088.stdout:5/650: getdents d1/dd/d6f/da1/d3f/d88/d8b/d94 0 2024-03-24T00:48:58.214 INFO:tasks.workunit.client.1.smithi088.stdout:8/633: dread d9/f63 [0,4194304] 0 2024-03-24T00:48:58.214 INFO:tasks.workunit.client.1.smithi088.stdout:6/638: link db/d1d/l22 db/d3d/d83/d72/lc7 0 2024-03-24T00:48:58.215 INFO:tasks.workunit.client.1.smithi088.stdout:6/639: readlink db/d1d/d23/d3c/d43/l32 0 2024-03-24T00:48:58.215 INFO:tasks.workunit.client.1.smithi088.stdout:6/640: write db/d1d/d23/fbb [198111,114932] 0 2024-03-24T00:48:58.222 INFO:tasks.workunit.client.1.smithi088.stdout:2/597: write db/dd/f2f [723402,7566] 0 2024-03-24T00:48:58.222 INFO:tasks.workunit.client.1.smithi088.stdout:2/598: chown db/d21/d35/d49/l51 4010854 1 2024-03-24T00:48:58.222 INFO:tasks.workunit.client.1.smithi088.stdout:2/599: fdatasync db/dd/f1f 0 2024-03-24T00:48:58.225 INFO:tasks.workunit.client.0.smithi038.stdout:4/565: dread da/d12/f5c [0,4194304] 0 2024-03-24T00:48:58.225 INFO:tasks.workunit.client.1.smithi088.stdout:0/660: dread d9/dc/de/f3e [0,4194304] 0 2024-03-24T00:48:58.225 INFO:tasks.workunit.client.1.smithi088.stdout:0/661: fsync d9/dc/de/f1d 0 2024-03-24T00:48:58.227 INFO:tasks.workunit.client.0.smithi038.stdout:6/641: creat d1/da/d61/fd1 x:0 0 0 2024-03-24T00:48:58.227 INFO:tasks.workunit.client.0.smithi038.stdout:6/642: chown d1/da/d1b/d2f/f7f 21 1 2024-03-24T00:48:58.227 INFO:tasks.workunit.client.0.smithi038.stdout:6/643: write d1/da/d1b/d25/d49/d72/fb7 [400484,13251] 0 2024-03-24T00:48:58.227 INFO:tasks.workunit.client.0.smithi038.stdout:6/644: write d1/da/f52 [778837,127698] 0 2024-03-24T00:48:58.229 INFO:tasks.workunit.client.0.smithi038.stdout:0/522: write d0/d2d/f2f [2864269,46056] 0 2024-03-24T00:48:58.229 INFO:tasks.workunit.client.0.smithi038.stdout:0/523: write d0/d12/d9a/f9c [332285,89053] 0 2024-03-24T00:48:58.235 INFO:tasks.workunit.client.1.smithi088.stdout:4/617: rename d5/d13/d18/d24/l39 to d5/d7/d15/d60/ld7 0 2024-03-24T00:48:58.235 INFO:tasks.workunit.client.1.smithi088.stdout:4/618: stat d5/d13/d18/c8c 0 2024-03-24T00:48:58.236 INFO:tasks.workunit.client.0.smithi038.stdout:3/612: link d3/d30/f34 d3/d30/d5d/fca 0 2024-03-24T00:48:58.236 INFO:tasks.workunit.client.0.smithi038.stdout:3/613: chown d3/d30/d70/d90/d99 461638 1 2024-03-24T00:48:58.241 INFO:tasks.workunit.client.1.smithi088.stdout:5/651: symlink d1/d19/d48/dc2/lda 0 2024-03-24T00:48:58.244 INFO:tasks.workunit.client.0.smithi038.stdout:8/581: symlink d0/d10/lc0 0 2024-03-24T00:48:58.245 INFO:tasks.workunit.client.0.smithi038.stdout:1/664: symlink da/d3c/d86/lde 0 2024-03-24T00:48:58.249 INFO:tasks.workunit.client.1.smithi088.stdout:2/600: mknod db/dd/d96/d4e/cc0 0 2024-03-24T00:48:58.251 INFO:tasks.workunit.client.1.smithi088.stdout:3/589: sync 2024-03-24T00:48:58.254 INFO:tasks.workunit.client.1.smithi088.stdout:4/619: dread d5/f56 [0,4194304] 0 2024-03-24T00:48:58.255 INFO:tasks.workunit.client.1.smithi088.stdout:4/620: fsync d5/d17/f2a 0 2024-03-24T00:48:58.255 INFO:tasks.workunit.client.1.smithi088.stdout:4/621: chown d5/d16/l8d 1 1 2024-03-24T00:48:58.257 INFO:tasks.workunit.client.0.smithi038.stdout:7/567: creat d2/d2c/fd7 x:0 0 0 2024-03-24T00:48:58.260 INFO:tasks.workunit.client.1.smithi088.stdout:5/652: truncate d1/d10/d1b/d2e/f91 3887545 0 2024-03-24T00:48:58.260 INFO:tasks.workunit.client.1.smithi088.stdout:5/653: truncate d1/d10/d1b/d2f/d57/fca 1045558 0 2024-03-24T00:48:58.267 INFO:tasks.workunit.client.1.smithi088.stdout:2/601: creat db/dd/d96/d4e/d8f/d9c/fc1 x:0 0 0 2024-03-24T00:48:58.268 INFO:tasks.workunit.client.1.smithi088.stdout:1/624: dwrite d6/f68 [0,4194304] 0 2024-03-24T00:48:58.270 INFO:tasks.workunit.client.1.smithi088.stdout:4/622: mkdir d5/d16/d94/da6/dd8 0 2024-03-24T00:48:58.271 INFO:tasks.workunit.client.0.smithi038.stdout:3/614: dread d3/d9/dc/da6/db5/d19/f42 [0,4194304] 0 2024-03-24T00:48:58.276 INFO:tasks.workunit.client.0.smithi038.stdout:2/544: symlink d4/d27/d2f/d30/d3e/lbf 0 2024-03-24T00:48:58.278 INFO:tasks.workunit.client.1.smithi088.stdout:5/654: mknod d1/d10/d4f/cdb 0 2024-03-24T00:48:58.285 INFO:tasks.workunit.client.0.smithi038.stdout:6/645: dwrite d1/da/d61/fcd [4194304,4194304] 0 2024-03-24T00:48:58.286 INFO:tasks.workunit.client.1.smithi088.stdout:2/602: link db/f55 db/dd/d96/d4e/d8f/dad/fc2 0 2024-03-24T00:48:58.289 INFO:tasks.workunit.client.1.smithi088.stdout:1/625: mkdir d6/dc2 0 2024-03-24T00:48:58.290 INFO:tasks.workunit.client.1.smithi088.stdout:1/626: fsync d6/d23/daf/d95/d58/fbe 0 2024-03-24T00:48:58.290 INFO:tasks.workunit.client.1.smithi088.stdout:1/627: chown d6/lb 13938 1 2024-03-24T00:48:58.290 INFO:tasks.workunit.client.1.smithi088.stdout:1/628: creat d6/d23/daf/d95/d5b/d5d/d61/d7d/da5/fc3 x:0 0 0 2024-03-24T00:48:58.300 INFO:tasks.workunit.client.0.smithi038.stdout:8/582: mknod d0/d45/db8/cc1 0 2024-03-24T00:48:58.300 INFO:tasks.workunit.client.0.smithi038.stdout:2/545: dread d4/d27/f2a [0,4194304] 0 2024-03-24T00:48:58.300 INFO:tasks.workunit.client.0.smithi038.stdout:2/546: chown d4/d27/d54/dab 42 1 2024-03-24T00:48:58.306 INFO:tasks.workunit.client.0.smithi038.stdout:1/665: creat da/dd/d7a/d82/d98/d34/fdf x:0 0 0 2024-03-24T00:48:58.345 INFO:tasks.workunit.client.0.smithi038.stdout:7/568: rmdir d2/d9/da/dc/da8 39 2024-03-24T00:48:58.345 INFO:tasks.workunit.client.1.smithi088.stdout:4/623: symlink d5/d13/d18/d24/d3d/ld9 0 2024-03-24T00:48:58.350 INFO:tasks.workunit.client.1.smithi088.stdout:2/603: mkdir db/dd/d96/d99/dc3 0 2024-03-24T00:48:58.351 INFO:tasks.workunit.client.0.smithi038.stdout:2/547: rmdir d4/d18/d26/d37/d69/d83 39 2024-03-24T00:48:58.351 INFO:tasks.workunit.client.0.smithi038.stdout:2/548: readlink d4/d18/d26/d37/l52 0 2024-03-24T00:48:58.354 INFO:tasks.workunit.client.0.smithi038.stdout:1/666: creat da/dd/d7a/d82/fe0 x:0 0 0 2024-03-24T00:48:58.355 INFO:tasks.workunit.client.1.smithi088.stdout:1/629: creat d6/dc2/fc4 x:0 0 0 2024-03-24T00:48:58.356 INFO:tasks.workunit.client.0.smithi038.stdout:7/569: rmdir d2/d9/da/d23/d2b/d5b 39 2024-03-24T00:48:58.360 INFO:tasks.workunit.client.1.smithi088.stdout:1/630: link d6/lb d6/d6e/dc1/lc5 0 2024-03-24T00:48:58.362 INFO:tasks.workunit.client.1.smithi088.stdout:5/655: rename d1/d10/d1b/d2e/d46 to d1/d19/d1e/ddc 0 2024-03-24T00:48:58.363 INFO:tasks.workunit.client.1.smithi088.stdout:5/656: stat d1/d10/d1b/d2f/d57/d9e/fab 0 2024-03-24T00:48:58.363 INFO:tasks.workunit.client.1.smithi088.stdout:5/657: write d1/d19/d48/f92 [1035260,107644] 0 2024-03-24T00:48:58.364 INFO:tasks.workunit.client.0.smithi038.stdout:6/646: rmdir d1/d37/d84 39 2024-03-24T00:48:58.364 INFO:tasks.workunit.client.0.smithi038.stdout:6/647: write d1/f93 [952113,39693] 0 2024-03-24T00:48:58.364 INFO:tasks.workunit.client.0.smithi038.stdout:6/648: dread - d1/da/d1b/d25/d49/fcc zero size 2024-03-24T00:48:58.366 INFO:tasks.workunit.client.1.smithi088.stdout:7/541: dwrite d3/d32/d34/d9c/d4b/f60 [0,4194304] 0 2024-03-24T00:48:58.367 INFO:tasks.workunit.client.0.smithi038.stdout:1/667: dread da/dd/d2c/d41/d60/fc5 [0,4194304] 0 2024-03-24T00:48:58.367 INFO:tasks.workunit.client.0.smithi038.stdout:1/668: truncate da/dd/f85 988838 0 2024-03-24T00:48:58.367 INFO:tasks.workunit.client.0.smithi038.stdout:7/570: rename d2/d2c/f4d to d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fd8 0 2024-03-24T00:48:58.367 INFO:tasks.workunit.client.0.smithi038.stdout:8/583: dread d0/d1d/d20/d3c/f47 [0,4194304] 0 2024-03-24T00:48:58.368 INFO:tasks.workunit.client.1.smithi088.stdout:1/631: creat d6/d23/daf/d95/d5b/d5d/d61/fc6 x:0 0 0 2024-03-24T00:48:58.368 INFO:tasks.workunit.client.0.smithi038.stdout:4/566: dwrite da/d12/d29/d5b/d73/fba [0,4194304] 0 2024-03-24T00:48:58.368 INFO:tasks.workunit.client.1.smithi088.stdout:5/658: link d1/d10/d1b/d2f/d57/fca d1/d10/d1b/d2f/d57/d9e/fdd 0 2024-03-24T00:48:58.371 INFO:tasks.workunit.client.0.smithi038.stdout:1/669: symlink da/dd/d2c/d40/le1 0 2024-03-24T00:48:58.372 INFO:tasks.workunit.client.0.smithi038.stdout:8/584: rename d0/d45/d37/d5f/db7/c4a to d0/d1d/d27/d4d/cc2 0 2024-03-24T00:48:58.378 INFO:tasks.workunit.client.0.smithi038.stdout:1/670: creat da/ddc/fe2 x:0 0 0 2024-03-24T00:48:58.383 INFO:tasks.workunit.client.0.smithi038.stdout:2/549: dread d4/f66 [0,4194304] 0 2024-03-24T00:48:58.383 INFO:tasks.workunit.client.0.smithi038.stdout:2/550: truncate d4/d27/d54/d53/fae 543147 0 2024-03-24T00:48:58.384 INFO:tasks.workunit.client.0.smithi038.stdout:8/585: mknod d0/d1d/d20/cc3 0 2024-03-24T00:48:58.384 INFO:tasks.workunit.client.1.smithi088.stdout:7/542: truncate d3/d32/f8e 578162 0 2024-03-24T00:48:58.385 INFO:tasks.workunit.client.1.smithi088.stdout:1/632: stat d6/d23/daf/d95/d5b/d5d/l77 0 2024-03-24T00:48:58.385 INFO:tasks.workunit.client.0.smithi038.stdout:9/651: dwrite d12/d28/d6c/fbd [0,4194304] 0 2024-03-24T00:48:58.385 INFO:tasks.workunit.client.1.smithi088.stdout:1/633: fsync d6/d23/daf/d95/d3f/d53/faa 0 2024-03-24T00:48:58.390 INFO:tasks.workunit.client.1.smithi088.stdout:5/659: dread d1/d19/d48/fae [0,4194304] 0 2024-03-24T00:48:58.390 INFO:tasks.workunit.client.0.smithi038.stdout:1/671: rename da/dd/d7a/d82/d98/d14/dc0/cca to da/dd/ce3 0 2024-03-24T00:48:58.390 INFO:tasks.workunit.client.0.smithi038.stdout:1/672: getdents da/dd/d22/d5c 0 2024-03-24T00:48:58.392 INFO:tasks.workunit.client.1.smithi088.stdout:5/660: read d1/d10/d1b/d2f/d57/fca [271357,68328] 0 2024-03-24T00:48:58.392 INFO:tasks.workunit.client.1.smithi088.stdout:5/661: dread - d1/d10/d1b/d2f/d57/f8d zero size 2024-03-24T00:48:58.392 INFO:tasks.workunit.client.1.smithi088.stdout:5/662: chown d1/d10/l4b 60253699 1 2024-03-24T00:48:58.396 INFO:tasks.workunit.client.1.smithi088.stdout:1/634: creat d6/d23/daf/d95/d5b/fc7 x:0 0 0 2024-03-24T00:48:58.398 INFO:tasks.workunit.client.0.smithi038.stdout:2/551: rmdir d4/d27/d2f/d30/d3e 39 2024-03-24T00:48:58.401 INFO:tasks.workunit.client.1.smithi088.stdout:1/635: mkdir d6/d23/daf/d95/d5b/d5d/d61/d7d/dc8 0 2024-03-24T00:48:58.407 INFO:tasks.workunit.client.0.smithi038.stdout:1/673: mknod da/dd/d7a/d82/ce4 0 2024-03-24T00:48:58.409 INFO:tasks.workunit.client.1.smithi088.stdout:9/644: sync 2024-03-24T00:48:58.409 INFO:tasks.workunit.client.1.smithi088.stdout:1/636: creat d6/d23/daf/d95/d3f/fc9 x:0 0 0 2024-03-24T00:48:58.413 INFO:tasks.workunit.client.0.smithi038.stdout:8/586: symlink d0/d1d/lc4 0 2024-03-24T00:48:58.413 INFO:tasks.workunit.client.0.smithi038.stdout:8/587: fdatasync d0/d1d/d27/d4d/daf/fb5 0 2024-03-24T00:48:58.422 INFO:tasks.workunit.client.0.smithi038.stdout:9/652: mkdir d12/d28/d6c/dbc/dd7 0 2024-03-24T00:48:58.422 INFO:tasks.workunit.client.0.smithi038.stdout:9/653: readlink d12/d28/d71/lac 0 2024-03-24T00:48:58.424 INFO:tasks.workunit.client.1.smithi088.stdout:9/645: creat d1/d10/d9b/fd1 x:0 0 0 2024-03-24T00:48:58.430 INFO:tasks.workunit.client.1.smithi088.stdout:2/604: dwrite db/dd/d96/d99/db2/d88/f97 [0,4194304] 0 2024-03-24T00:48:58.430 INFO:tasks.workunit.client.1.smithi088.stdout:2/605: readlink db/dd/d96/l52 0 2024-03-24T00:48:58.431 INFO:tasks.workunit.client.0.smithi038.stdout:1/674: rmdir da/ddc/ddd 0 2024-03-24T00:48:58.431 INFO:tasks.workunit.client.0.smithi038.stdout:1/675: fsync da/dd/d7a/d82/d98/d34/fdf 0 2024-03-24T00:48:58.435 INFO:tasks.workunit.client.0.smithi038.stdout:2/552: dread d4/d27/f47 [4194304,4194304] 0 2024-03-24T00:48:58.435 INFO:tasks.workunit.client.0.smithi038.stdout:2/553: readlink d4/d18/d26/d37/l52 0 2024-03-24T00:48:58.435 INFO:tasks.workunit.client.0.smithi038.stdout:2/554: read d4/d18/f28 [942402,67183] 0 2024-03-24T00:48:58.435 INFO:tasks.workunit.client.0.smithi038.stdout:4/567: dwrite da/d12/d37/dd/f2b [0,4194304] 0 2024-03-24T00:48:58.439 INFO:tasks.workunit.client.1.smithi088.stdout:1/637: mknod d6/d23/daf/d95/d5b/d5d/cca 0 2024-03-24T00:48:58.440 INFO:tasks.workunit.client.1.smithi088.stdout:1/638: fsync d6/f3e 0 2024-03-24T00:48:58.459 INFO:tasks.workunit.client.1.smithi088.stdout:5/663: dwrite d1/d19/d48/dc2/fc9 [0,4194304] 0 2024-03-24T00:48:58.459 INFO:tasks.workunit.client.1.smithi088.stdout:5/664: readlink d1/d10/d1b/l27 0 2024-03-24T00:48:58.459 INFO:tasks.workunit.client.0.smithi038.stdout:7/571: dread d2/d9/da/d23/d2b/d5b/d72/d92/f9e [0,4194304] 0 2024-03-24T00:48:58.460 INFO:tasks.workunit.client.0.smithi038.stdout:7/572: write d2/d9/da/d23/d2b/d5b/d72/d92/d5d/f7f [4835717,126163] 0 2024-03-24T00:48:58.460 INFO:tasks.workunit.client.0.smithi038.stdout:7/573: truncate d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fc7 703127 0 2024-03-24T00:48:58.460 INFO:tasks.workunit.client.0.smithi038.stdout:3/615: dwrite d3/d9/dc/d51/f7d [0,4194304] 0 2024-03-24T00:48:58.460 INFO:tasks.workunit.client.0.smithi038.stdout:3/616: chown d3/d9/dc/da6/db5/d19/d82/fba 137 1 2024-03-24T00:48:58.463 INFO:tasks.workunit.client.1.smithi088.stdout:9/646: rename d1/d15/la4 to d1/d10/d11/ld2 0 2024-03-24T00:48:58.463 INFO:tasks.workunit.client.1.smithi088.stdout:9/647: readlink d1/d10/d11/ld2 0 2024-03-24T00:48:58.463 INFO:tasks.workunit.client.1.smithi088.stdout:9/648: fsync d1/d10/d11/f5f 0 2024-03-24T00:48:58.465 INFO:tasks.workunit.client.1.smithi088.stdout:0/662: sync 2024-03-24T00:48:58.465 INFO:tasks.workunit.client.1.smithi088.stdout:0/663: chown d9/d26/d8f/da1/d62/d9b/db0 30 1 2024-03-24T00:48:58.465 INFO:tasks.workunit.client.1.smithi088.stdout:0/664: fsync d9/d26/d8f/da1/d62/f6a 0 2024-03-24T00:48:58.465 INFO:tasks.workunit.client.1.smithi088.stdout:6/641: sync 2024-03-24T00:48:58.465 INFO:tasks.workunit.client.1.smithi088.stdout:6/642: chown db/dd/d61/c69 531726935 1 2024-03-24T00:48:58.465 INFO:tasks.workunit.client.1.smithi088.stdout:2/606: dread db/d21/d35/f60 [0,4194304] 0 2024-03-24T00:48:58.465 INFO:tasks.workunit.client.1.smithi088.stdout:7/543: dwrite d3/d31/f38 [4194304,4194304] 0 2024-03-24T00:48:58.465 INFO:tasks.workunit.client.1.smithi088.stdout:2/607: getdents db/dd/d96/d99/db2/d88 0 2024-03-24T00:48:58.465 INFO:tasks.workunit.client.1.smithi088.stdout:7/544: write d3/d32/d34/d9c/d4b/f5e [2396736,97965] 0 2024-03-24T00:48:58.466 INFO:tasks.workunit.client.1.smithi088.stdout:7/545: creat d3/d32/d34/faa x:0 0 0 2024-03-24T00:48:58.471 INFO:tasks.workunit.client.0.smithi038.stdout:2/555: creat d4/d18/d38/fc0 x:0 0 0 2024-03-24T00:48:58.471 INFO:tasks.workunit.client.0.smithi038.stdout:2/556: chown d4/d27/d2f/d30/d3e/l4d 43 1 2024-03-24T00:48:58.473 INFO:tasks.workunit.client.1.smithi088.stdout:1/639: mkdir d6/d23/dcb 0 2024-03-24T00:48:58.473 INFO:tasks.workunit.client.1.smithi088.stdout:8/634: sync 2024-03-24T00:48:58.473 INFO:tasks.workunit.client.1.smithi088.stdout:3/590: sync 2024-03-24T00:48:58.477 INFO:tasks.workunit.client.1.smithi088.stdout:5/665: dread d1/d19/d1e/f47 [0,4194304] 0 2024-03-24T00:48:58.477 INFO:tasks.workunit.client.1.smithi088.stdout:5/666: read d1/d43/d4e/d5b/fcb [880461,40987] 0 2024-03-24T00:48:58.478 INFO:tasks.workunit.client.1.smithi088.stdout:0/665: mkdir d9/dc/de/d19/d84/dd6 0 2024-03-24T00:48:58.478 INFO:tasks.workunit.client.1.smithi088.stdout:0/666: truncate d9/dc/de/d19/d8d/f98 740242 0 2024-03-24T00:48:58.479 INFO:tasks.workunit.client.0.smithi038.stdout:7/574: mknod d2/d9/da/dc/cd9 0 2024-03-24T00:48:58.479 INFO:tasks.workunit.client.1.smithi088.stdout:2/608: symlink db/d21/d73/lc4 0 2024-03-24T00:48:58.484 INFO:tasks.workunit.client.0.smithi038.stdout:3/617: link d3/f29 d3/dc3/fcb 0 2024-03-24T00:48:58.489 INFO:tasks.workunit.client.0.smithi038.stdout:8/588: dwrite d0/d45/d66/d6d/fbc [0,4194304] 0 2024-03-24T00:48:58.490 INFO:tasks.workunit.client.1.smithi088.stdout:7/546: fsync d3/f1f 0 2024-03-24T00:48:58.495 INFO:tasks.workunit.client.0.smithi038.stdout:2/557: symlink d4/d55/d88/lc1 0 2024-03-24T00:48:58.499 INFO:tasks.workunit.client.0.smithi038.stdout:7/575: link d2/d9/da/dc/d8a/fa5 d2/d9/da/d23/d2b/d5b/d72/d92/d5d/fda 0 2024-03-24T00:48:58.499 INFO:tasks.workunit.client.0.smithi038.stdout:7/576: chown d2/d9/f54 1 1 2024-03-24T00:48:58.500 INFO:tasks.workunit.client.1.smithi088.stdout:8/635: dread d9/dc/d11/d24/d29/d5b/f34 [0,4194304] 0 2024-03-24T00:48:58.500 INFO:tasks.workunit.client.1.smithi088.stdout:8/636: getdents d9/dc/d11/dd2 0 2024-03-24T00:48:58.507 INFO:tasks.workunit.client.0.smithi038.stdout:4/568: dread da/f3b [0,4194304] 0 2024-03-24T00:48:58.508 INFO:tasks.workunit.client.1.smithi088.stdout:6/643: rename db/d1d/d23/d3c to db/dc5/dc8 0 2024-03-24T00:48:58.512 INFO:tasks.workunit.client.0.smithi038.stdout:8/589: truncate d0/d1d/d20/f58 3791369 0 2024-03-24T00:48:58.512 INFO:tasks.workunit.client.0.smithi038.stdout:2/558: symlink d4/d55/lc2 0 2024-03-24T00:48:58.516 INFO:tasks.workunit.client.1.smithi088.stdout:7/547: mkdir d3/d32/d34/d9c/d4b/d73/dab 0 2024-03-24T00:48:58.517 INFO:tasks.workunit.client.1.smithi088.stdout:3/591: dread d3/d6/d27/f8d [0,4194304] 0 2024-03-24T00:48:58.517 INFO:tasks.workunit.client.0.smithi038.stdout:1/676: dwrite da/dd/d22/d94/fa0 [0,4194304] 0 2024-03-24T00:48:58.517 INFO:tasks.workunit.client.0.smithi038.stdout:1/677: write da/dd/d7a/d82/d98/f15 [2801227,57829] 0 2024-03-24T00:48:58.517 INFO:tasks.workunit.client.0.smithi038.stdout:1/678: truncate da/dd/d22/d5c/f6e 986838 0 2024-03-24T00:48:58.518 INFO:tasks.workunit.client.1.smithi088.stdout:9/649: dwrite d1/d15/f68 [4194304,4194304] 0 2024-03-24T00:48:58.518 INFO:tasks.workunit.client.0.smithi038.stdout:5/561: sync 2024-03-24T00:48:58.518 INFO:tasks.workunit.client.0.smithi038.stdout:5/562: fsync d6/d19/d20/d42/f72 0 2024-03-24T00:48:58.518 INFO:tasks.workunit.client.0.smithi038.stdout:5/563: chown d6/d19/d20/d42/d43/d4c/d6d 350 1 2024-03-24T00:48:58.520 INFO:tasks.workunit.client.1.smithi088.stdout:8/637: truncate d9/dc/f44 3749743 0 2024-03-24T00:48:58.521 INFO:tasks.workunit.client.1.smithi088.stdout:0/667: dread d9/d26/f61 [4194304,4194304] 0 2024-03-24T00:48:58.522 INFO:tasks.workunit.client.0.smithi038.stdout:7/577: rmdir d2/d9/da/dc/da8 39 2024-03-24T00:48:58.526 INFO:tasks.workunit.client.1.smithi088.stdout:5/667: rename d1/d43/d4e/d5b/lad to d1/dd/d6f/d98/lde 0 2024-03-24T00:48:58.527 INFO:tasks.workunit.client.1.smithi088.stdout:5/668: write d1/d10/d1b/d2f/d93/fa9 [478685,63943] 0 2024-03-24T00:48:58.530 INFO:tasks.workunit.client.0.smithi038.stdout:4/569: creat da/d12/d24/d5f/d8a/fc4 x:0 0 0 2024-03-24T00:48:58.530 INFO:tasks.workunit.client.0.smithi038.stdout:4/570: readlink da/d12/d29/l51 0 2024-03-24T00:48:58.530 INFO:tasks.workunit.client.0.smithi038.stdout:9/654: dwrite d12/d28/f70 [0,4194304] 0 2024-03-24T00:48:58.532 INFO:tasks.workunit.client.1.smithi088.stdout:1/640: write d6/f68 [968883,63780] 0 2024-03-24T00:48:58.533 INFO:tasks.workunit.client.0.smithi038.stdout:8/590: dread d0/d1d/d20/d3c/f47 [0,4194304] 0 2024-03-24T00:48:58.538 INFO:tasks.workunit.client.1.smithi088.stdout:6/644: creat db/dd/d61/fc9 x:0 0 0 2024-03-24T00:48:58.538 INFO:tasks.workunit.client.1.smithi088.stdout:6/645: write db/dc5/dc8/d43/f34 [4256018,86131] 0 2024-03-24T00:48:58.539 INFO:tasks.workunit.client.0.smithi038.stdout:2/559: rename d4/fa to d4/d55/fc3 0 2024-03-24T00:48:58.542 INFO:tasks.workunit.client.0.smithi038.stdout:5/564: dread d6/d16/d67/f8f [0,4194304] 0 2024-03-24T00:48:58.542 INFO:tasks.workunit.client.0.smithi038.stdout:5/565: stat d6/d19/d20/d42/d43/d68/f8b 0 2024-03-24T00:48:58.542 INFO:tasks.workunit.client.1.smithi088.stdout:0/668: write d9/dc/de/d19/d44/f5b [5612140,81715] 0 2024-03-24T00:48:58.542 INFO:tasks.workunit.client.1.smithi088.stdout:7/548: creat d3/d32/d34/d9c/d4b/d74/fac x:0 0 0 2024-03-24T00:48:58.543 INFO:tasks.workunit.client.0.smithi038.stdout:8/591: dread d0/d15/f63 [0,4194304] 0 2024-03-24T00:48:58.545 INFO:tasks.workunit.client.0.smithi038.stdout:3/618: dwrite d3/d9/dc/da6/fab [0,4194304] 0 2024-03-24T00:48:58.545 INFO:tasks.workunit.client.0.smithi038.stdout:3/619: chown d3/d9/dc/da6/db5/d19/d82/f93 1 1 2024-03-24T00:48:58.548 INFO:tasks.workunit.client.0.smithi038.stdout:1/679: link da/dd/d4a/d8b/l9b da/ddc/le5 0 2024-03-24T00:48:58.549 INFO:tasks.workunit.client.1.smithi088.stdout:9/650: creat d1/d22/d27/fd3 x:0 0 0 2024-03-24T00:48:58.550 INFO:tasks.workunit.client.1.smithi088.stdout:2/609: dwrite db/d21/f44 [0,4194304] 0 2024-03-24T00:48:58.550 INFO:tasks.workunit.client.1.smithi088.stdout:3/592: dread d3/de/d6f/f7a [0,4194304] 0 2024-03-24T00:48:58.560 INFO:tasks.workunit.client.1.smithi088.stdout:5/669: rename d1/d19/d1e/ddc to d1/d10/d1b/d2e/d9f/ddf 0 2024-03-24T00:48:58.563 INFO:tasks.workunit.client.1.smithi088.stdout:4/624: sync 2024-03-24T00:48:58.563 INFO:tasks.workunit.client.1.smithi088.stdout:4/625: creat d5/d13/d18/d24/d3d/fda x:0 0 0 2024-03-24T00:48:58.565 INFO:tasks.workunit.client.0.smithi038.stdout:9/655: creat d12/d81/fd8 x:0 0 0 2024-03-24T00:48:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:7/578: rename d2/d9/da/dc/c84 to d2/d9/da/dc/d8a/db8/cdb 0 2024-03-24T00:48:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:7/579: chown d2/d9/da/d23/d2b/d5b/d72/d92/d98/f9f 3 1 2024-03-24T00:48:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:1/680: write da/dd/d7a/d82/d98/d34/f39 [3024076,21176] 0 2024-03-24T00:48:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:1/681: fdatasync da/f7e 0 2024-03-24T00:48:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:1/682: chown da/dd/d7a/d82/lb5 4 1 2024-03-24T00:48:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:1/683: write da/dd/d7a/d82/d98/d14/f4d [863061,22472] 0 2024-03-24T00:48:58.572 INFO:tasks.workunit.client.1.smithi088.stdout:1/641: truncate d6/d23/daf/d95/d5b/d5e/f66 935016 0 2024-03-24T00:48:58.576 INFO:tasks.workunit.client.0.smithi038.stdout:8/592: symlink d0/d45/db8/lc5 0 2024-03-24T00:48:58.579 INFO:tasks.workunit.client.1.smithi088.stdout:6/646: unlink db/d1d/d50/d90/d56/lab 0 2024-03-24T00:48:58.579 INFO:tasks.workunit.client.1.smithi088.stdout:6/647: stat db/dd/d61/d6d/d74/l81 0 2024-03-24T00:48:58.579 INFO:tasks.workunit.client.1.smithi088.stdout:6/648: fdatasync db/d3d/d83/d71/fb1 0 2024-03-24T00:48:58.579 INFO:tasks.workunit.client.1.smithi088.stdout:6/649: stat db/dd/d89 0 2024-03-24T00:48:58.589 INFO:tasks.workunit.client.1.smithi088.stdout:7/549: unlink d3/d32/d34/c57 0 2024-03-24T00:48:58.589 INFO:tasks.workunit.client.1.smithi088.stdout:0/669: mknod d9/dc/de/d7e/cd7 0 2024-03-24T00:48:58.590 INFO:tasks.workunit.client.1.smithi088.stdout:0/670: chown d9/dc/de/d19/d44/d46/f57 4571 1 2024-03-24T00:48:58.590 INFO:tasks.workunit.client.1.smithi088.stdout:0/671: readlink d9/dc/de/d19/d44/d46/d65/d2a/l5d 0 2024-03-24T00:48:58.590 INFO:tasks.workunit.client.1.smithi088.stdout:9/651: rmdir d1/d15/d80 39 2024-03-24T00:48:58.590 INFO:tasks.workunit.client.1.smithi088.stdout:0/672: read d9/dc/de/d19/d44/d46/d59/f70 [4594767,12692] 0 2024-03-24T00:48:58.591 INFO:tasks.workunit.client.0.smithi038.stdout:0/524: sync 2024-03-24T00:48:58.592 INFO:tasks.workunit.client.0.smithi038.stdout:0/525: creat d0/d12/d92/fab x:0 0 0 2024-03-24T00:48:58.592 INFO:tasks.workunit.client.0.smithi038.stdout:0/526: chown d0/d2d/f6f 11481 1 2024-03-24T00:48:58.592 INFO:tasks.workunit.client.0.smithi038.stdout:0/527: fsync d0/d14/f5c 0 2024-03-24T00:48:58.593 INFO:tasks.workunit.client.0.smithi038.stdout:0/528: read d0/d12/f68 [525232,66862] 0 2024-03-24T00:48:58.593 INFO:tasks.workunit.client.0.smithi038.stdout:0/529: chown d0/d12/d15/d1a/l1c 5 1 2024-03-24T00:48:58.593 INFO:tasks.workunit.client.0.smithi038.stdout:3/620: link d3/d9/dc/da6/db5/dad/fbd d3/d9/dc/da6/db5/fcc 0 2024-03-24T00:48:58.593 INFO:tasks.workunit.client.0.smithi038.stdout:0/530: write d0/d2d/f2f [4774152,63761] 0 2024-03-24T00:48:58.593 INFO:tasks.workunit.client.0.smithi038.stdout:3/621: creat d3/d30/d70/d90/d99/fcd x:0 0 0 2024-03-24T00:48:58.594 INFO:tasks.workunit.client.1.smithi088.stdout:2/610: truncate db/d21/f5f 680537 0 2024-03-24T00:48:58.594 INFO:tasks.workunit.client.1.smithi088.stdout:2/611: stat db/dd/d96/fa2 0 2024-03-24T00:48:58.597 INFO:tasks.workunit.client.0.smithi038.stdout:9/656: truncate d12/d28/f36 1579166 0 2024-03-24T00:48:58.598 INFO:tasks.workunit.client.0.smithi038.stdout:9/657: fdatasync d12/d28/d3a/f2b 0 2024-03-24T00:48:58.599 INFO:tasks.workunit.client.0.smithi038.stdout:9/658: write d12/d28/d71/f8b [916918,2587] 0 2024-03-24T00:48:58.600 INFO:tasks.workunit.client.1.smithi088.stdout:6/650: dread db/dd/d61/d6d/d74/d88/f86 [0,4194304] 0 2024-03-24T00:48:58.600 INFO:tasks.workunit.client.1.smithi088.stdout:6/651: read - db/dd/d61/fc9 zero size 2024-03-24T00:48:58.603 INFO:tasks.workunit.client.0.smithi038.stdout:4/571: rename da/d12/d29/l39 to da/d12/d37/lc5 0 2024-03-24T00:48:58.604 INFO:tasks.workunit.client.0.smithi038.stdout:4/572: fdatasync da/d12/d37/dd/d15/f17 0 2024-03-24T00:48:58.604 INFO:tasks.workunit.client.0.smithi038.stdout:4/573: dread - da/d12/f87 zero size 2024-03-24T00:48:58.604 INFO:tasks.workunit.client.0.smithi038.stdout:4/574: fsync da/d12/d29/d5b/d73/fba 0 2024-03-24T00:48:58.604 INFO:tasks.workunit.client.0.smithi038.stdout:4/575: truncate da/d12/d24/d5f/f75 243348 0 2024-03-24T00:48:58.604 INFO:tasks.workunit.client.0.smithi038.stdout:4/576: stat da/d12/d37/dd/f95 0 2024-03-24T00:48:58.604 INFO:tasks.workunit.client.0.smithi038.stdout:4/577: dread - da/d12/d29/d5b/d73/db4/fb7 zero size 2024-03-24T00:48:58.604 INFO:tasks.workunit.client.0.smithi038.stdout:4/578: readlink da/d12/l36 0 2024-03-24T00:48:58.604 INFO:tasks.workunit.client.0.smithi038.stdout:4/579: chown da/d38/f3c 1567 1 2024-03-24T00:48:58.610 INFO:tasks.workunit.client.1.smithi088.stdout:6/652: dread db/d1d/d50/d90/f4d [0,4194304] 0 2024-03-24T00:48:58.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:48:58 smithi038 ceph-mon[29271]: pgmap v35: 65 pgs: 65 active+clean; 2.2 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 152 MiB/s rd, 180 MiB/s wr, 253 op/s 2024-03-24T00:48:58.614 INFO:tasks.workunit.client.1.smithi088.stdout:2/612: dread db/f38 [0,4194304] 0 2024-03-24T00:48:58.615 INFO:tasks.workunit.client.0.smithi038.stdout:7/580: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d33/cdc 0 2024-03-24T00:48:58.615 INFO:tasks.workunit.client.0.smithi038.stdout:7/581: chown d2/d9/da/d23/d2b/d5b/d72/d92/c45 29 1 2024-03-24T00:48:58.621 INFO:tasks.workunit.client.1.smithi088.stdout:4/626: mkdir d5/d13/d18/d24/ddb 0 2024-03-24T00:48:58.621 INFO:tasks.workunit.client.0.smithi038.stdout:6/649: sync 2024-03-24T00:48:58.622 INFO:tasks.workunit.client.0.smithi038.stdout:6/650: fdatasync d1/da/d1b/d25/d49/fcc 0 2024-03-24T00:48:58.622 INFO:tasks.workunit.client.0.smithi038.stdout:5/566: rmdir d6/d1e 39 2024-03-24T00:48:58.622 INFO:tasks.workunit.client.0.smithi038.stdout:8/593: link d0/d45/d37/d5f/db7/c2b d0/d1d/d20/d3c/d9e/da3/cc6 0 2024-03-24T00:48:58.622 INFO:tasks.workunit.client.0.smithi038.stdout:0/531: write d0/d12/d15/f22 [4013656,67306] 0 2024-03-24T00:48:58.622 INFO:tasks.workunit.client.0.smithi038.stdout:0/532: creat d0/d1b/d2c/fac x:0 0 0 2024-03-24T00:48:58.622 INFO:tasks.workunit.client.0.smithi038.stdout:0/533: truncate d0/d14/d84/f96 30393 0 2024-03-24T00:48:58.622 INFO:tasks.workunit.client.0.smithi038.stdout:0/534: chown d0/c5b 2 1 2024-03-24T00:48:58.622 INFO:tasks.workunit.client.0.smithi038.stdout:0/535: dread - d0/d14/d84/f85 zero size 2024-03-24T00:48:58.626 INFO:tasks.workunit.client.0.smithi038.stdout:1/684: mkdir da/dd/d22/d5c/dda/de6 0 2024-03-24T00:48:58.633 INFO:tasks.workunit.client.1.smithi088.stdout:7/550: readlink d3/l11 0 2024-03-24T00:48:58.649 INFO:tasks.workunit.client.1.smithi088.stdout:1/642: dwrite d6/d23/daf/d95/d5b/d5e/f60 [0,4194304] 0 2024-03-24T00:48:58.653 INFO:tasks.workunit.client.0.smithi038.stdout:3/622: creat d3/d9/dc/d51/fce x:0 0 0 2024-03-24T00:48:58.667 INFO:tasks.workunit.client.1.smithi088.stdout:9/652: dwrite d1/d10/d29/db0/fc9 [0,4194304] 0 2024-03-24T00:48:58.681 INFO:tasks.workunit.client.0.smithi038.stdout:2/560: rename d4/d55/lb5 to d4/d18/d5b/lc4 0 2024-03-24T00:48:58.681 INFO:tasks.workunit.client.0.smithi038.stdout:5/567: mknod d6/d19/d20/d33/cb7 0 2024-03-24T00:48:58.691 INFO:tasks.workunit.client.1.smithi088.stdout:5/670: getdents d1/dd/d6f/d98 0 2024-03-24T00:48:58.691 INFO:tasks.workunit.client.1.smithi088.stdout:6/653: getdents db/d1d/d50 0 2024-03-24T00:48:58.691 INFO:tasks.workunit.client.1.smithi088.stdout:6/654: write db/d1d/d50/d90/d55/fbe [47399,90470] 0 2024-03-24T00:48:58.691 INFO:tasks.workunit.client.1.smithi088.stdout:6/655: chown db/dd/f37 152183 1 2024-03-24T00:48:58.691 INFO:tasks.workunit.client.1.smithi088.stdout:6/656: truncate db/dd/d61/d6d/d74/d88/f9c 186681 0 2024-03-24T00:48:58.691 INFO:tasks.workunit.client.1.smithi088.stdout:6/657: readlink db/d3d/la3 0 2024-03-24T00:48:58.691 INFO:tasks.workunit.client.1.smithi088.stdout:6/658: getdents db/dc5/dc8/d43/dbf 0 2024-03-24T00:48:58.692 INFO:tasks.workunit.client.0.smithi038.stdout:1/685: mknod da/dd/d2c/d41/d60/ce7 0 2024-03-24T00:48:58.692 INFO:tasks.workunit.client.0.smithi038.stdout:1/686: fsync da/d3c/f81 0 2024-03-24T00:48:58.698 INFO:tasks.workunit.client.0.smithi038.stdout:5/568: symlink d6/d19/d20/d42/d43/d50/d92/lb8 0 2024-03-24T00:48:58.698 INFO:tasks.workunit.client.0.smithi038.stdout:5/569: write d6/d19/d20/d42/d79/f9e [4798316,96943] 0 2024-03-24T00:48:58.699 INFO:tasks.workunit.client.1.smithi088.stdout:0/673: dwrite d9/dc/de/d19/f1a [0,4194304] 0 2024-03-24T00:48:58.699 INFO:tasks.workunit.client.1.smithi088.stdout:7/551: stat d3/c52 0 2024-03-24T00:48:58.701 INFO:tasks.workunit.client.0.smithi038.stdout:9/659: dwrite d12/d28/d3a/f45 [0,4194304] 0 2024-03-24T00:48:58.703 INFO:tasks.workunit.client.0.smithi038.stdout:6/651: rename d1/da/c3f to d1/da/d61/d79/cd2 0 2024-03-24T00:48:58.707 INFO:tasks.workunit.client.1.smithi088.stdout:5/671: mkdir d1/dd/d6f/da1/d8e/de0 0 2024-03-24T00:48:58.710 INFO:tasks.workunit.client.1.smithi088.stdout:7/552: dread d3/d32/d34/f3b [0,4194304] 0 2024-03-24T00:48:58.711 INFO:tasks.workunit.client.0.smithi038.stdout:9/660: creat d12/d28/d71/d99/fd9 x:0 0 0 2024-03-24T00:48:58.713 INFO:tasks.workunit.client.0.smithi038.stdout:6/652: mknod d1/da/d1b/d25/d49/d4e/cd3 0 2024-03-24T00:48:58.715 INFO:tasks.workunit.client.1.smithi088.stdout:2/613: dwrite db/d21/f74 [0,4194304] 0 2024-03-24T00:48:58.723 INFO:tasks.workunit.client.0.smithi038.stdout:9/661: rename d12/d41/d9d to d12/d28/d6c/dbc/dd7/dda 0 2024-03-24T00:48:58.723 INFO:tasks.workunit.client.0.smithi038.stdout:9/662: creat d12/d28/d71/d99/fdb x:0 0 0 2024-03-24T00:48:58.725 INFO:tasks.workunit.client.0.smithi038.stdout:4/580: dwrite da/d12/d24/f30 [0,4194304] 0 2024-03-24T00:48:58.734 INFO:tasks.workunit.client.1.smithi088.stdout:7/553: dread d3/d32/d34/f37 [8388608,4194304] 0 2024-03-24T00:48:58.734 INFO:tasks.workunit.client.1.smithi088.stdout:5/672: dread d1/d19/d1e/f47 [0,4194304] 0 2024-03-24T00:48:58.747 INFO:tasks.workunit.client.0.smithi038.stdout:0/536: dwrite d0/d12/d15/d1a/f98 [0,4194304] 0 2024-03-24T00:48:58.747 INFO:tasks.workunit.client.0.smithi038.stdout:9/663: dread d12/d41/fd0 [0,4194304] 0 2024-03-24T00:48:58.747 INFO:tasks.workunit.client.0.smithi038.stdout:0/537: dread - d0/d1b/fa6 zero size 2024-03-24T00:48:58.747 INFO:tasks.workunit.client.0.smithi038.stdout:4/581: getdents da/d12/d29 0 2024-03-24T00:48:58.750 INFO:tasks.workunit.client.1.smithi088.stdout:2/614: getdents db/dd/d96/d4e/d8f/dad 0 2024-03-24T00:48:58.750 INFO:tasks.workunit.client.1.smithi088.stdout:2/615: chown db/dd/d96/d99/db2/f6d 198985996 1 2024-03-24T00:48:58.751 INFO:tasks.workunit.client.1.smithi088.stdout:2/616: chown db/dd/d96/d4e/d8f/dad 397 1 2024-03-24T00:48:58.751 INFO:tasks.workunit.client.1.smithi088.stdout:2/617: creat db/d21/d73/fc5 x:0 0 0 2024-03-24T00:48:58.751 INFO:tasks.workunit.client.1.smithi088.stdout:1/643: dwrite d6/f8 [4194304,4194304] 0 2024-03-24T00:48:58.751 INFO:tasks.workunit.client.1.smithi088.stdout:1/644: chown d6/d23/daf/d95/d5b/fa7 83 1 2024-03-24T00:48:58.756 INFO:tasks.workunit.client.1.smithi088.stdout:7/554: getdents d3/d32/d34/d9c/d4b/d73 0 2024-03-24T00:48:58.761 INFO:tasks.workunit.client.0.smithi038.stdout:0/538: dread d0/d12/d9a/f9c [0,4194304] 0 2024-03-24T00:48:58.766 INFO:tasks.workunit.client.1.smithi088.stdout:5/673: creat d1/d10/d1b/d2e/fe1 x:0 0 0 2024-03-24T00:48:58.766 INFO:tasks.workunit.client.1.smithi088.stdout:5/674: chown d1/d10/d4f/cdb 996 1 2024-03-24T00:48:58.779 INFO:tasks.workunit.client.0.smithi038.stdout:7/582: dwrite d2/d9/d16/fc6 [0,4194304] 0 2024-03-24T00:48:58.780 INFO:tasks.workunit.client.0.smithi038.stdout:5/570: dwrite d6/d19/d20/d42/d43/d4c/f9c [0,4194304] 0 2024-03-24T00:48:58.780 INFO:tasks.workunit.client.0.smithi038.stdout:5/571: chown d6/d19/d20/d42/d43/d50/l51 0 1 2024-03-24T00:48:58.784 INFO:tasks.workunit.client.1.smithi088.stdout:1/645: unlink d6/d23/f73 0 2024-03-24T00:48:58.790 INFO:tasks.workunit.client.1.smithi088.stdout:0/674: dwrite d9/f45 [0,4194304] 0 2024-03-24T00:48:58.793 INFO:tasks.workunit.client.0.smithi038.stdout:3/623: dwrite d3/f49 [0,4194304] 0 2024-03-24T00:48:58.793 INFO:tasks.workunit.client.0.smithi038.stdout:3/624: chown d3/f25 262133 1 2024-03-24T00:48:58.795 INFO:tasks.workunit.client.1.smithi088.stdout:7/555: mknod d3/d31/cad 0 2024-03-24T00:48:58.795 INFO:tasks.workunit.client.0.smithi038.stdout:3/625: write d3/d9/dc/da6/db5/d19/d82/d89/fa3 [563642,3599] 0 2024-03-24T00:48:58.795 INFO:tasks.workunit.client.0.smithi038.stdout:9/664: getdents d12/d81/d84 0 2024-03-24T00:48:58.796 INFO:tasks.workunit.client.0.smithi038.stdout:8/594: dwrite d0/d45/d37/f87 [0,4194304] 0 2024-03-24T00:48:58.796 INFO:tasks.workunit.client.0.smithi038.stdout:4/582: link da/d12/d37/lc5 da/d12/d37/d83/lc6 0 2024-03-24T00:48:58.801 INFO:tasks.workunit.client.1.smithi088.stdout:9/653: dwrite d1/d10/fad [0,4194304] 0 2024-03-24T00:48:58.802 INFO:tasks.workunit.client.1.smithi088.stdout:5/675: mknod d1/d19/d48/dc2/dcc/ce2 0 2024-03-24T00:48:58.802 INFO:tasks.workunit.client.1.smithi088.stdout:5/676: truncate d1/dd/d6f/da1/d3f/d88/d8f/fbd 882199 0 2024-03-24T00:48:58.802 INFO:tasks.workunit.client.1.smithi088.stdout:5/677: chown d1/d10/d1b/d2e/f91 80125155 1 2024-03-24T00:48:58.808 INFO:tasks.workunit.client.0.smithi038.stdout:0/539: rename d0/d12/d42/d44/l7e to d0/d12/d42/d83/lad 0 2024-03-24T00:48:58.817 INFO:tasks.workunit.client.0.smithi038.stdout:1/687: dwrite da/dd/d2c/f63 [0,4194304] 0 2024-03-24T00:48:58.817 INFO:tasks.workunit.client.0.smithi038.stdout:1/688: write da/dd/d2c/d40/f5d [444890,66583] 0 2024-03-24T00:48:58.818 INFO:tasks.workunit.client.0.smithi038.stdout:1/689: chown da/d3c/l1d 1779938 1 2024-03-24T00:48:58.818 INFO:tasks.workunit.client.0.smithi038.stdout:1/690: fsync da/dd/d4a/d76/fad 0 2024-03-24T00:48:58.818 INFO:tasks.workunit.client.1.smithi088.stdout:1/646: write d6/f10 [1411373,107040] 0 2024-03-24T00:48:58.819 INFO:tasks.workunit.client.1.smithi088.stdout:2/618: dwrite db/d21/d35/f7d [0,4194304] 0 2024-03-24T00:48:58.821 INFO:tasks.workunit.client.0.smithi038.stdout:6/653: write d1/da/d1b/d25/d49/d4e/d51/f5e [459432,9104] 0 2024-03-24T00:48:58.821 INFO:tasks.workunit.client.0.smithi038.stdout:6/654: creat d1/da/d61/d69/fd4 x:0 0 0 2024-03-24T00:48:58.822 INFO:tasks.workunit.client.0.smithi038.stdout:6/655: creat d1/da/d1b/d1d/fd5 x:0 0 0 2024-03-24T00:48:58.827 INFO:tasks.workunit.client.1.smithi088.stdout:0/675: creat d9/d26/d8f/da1/d63/fd8 x:0 0 0 2024-03-24T00:48:58.827 INFO:tasks.workunit.client.1.smithi088.stdout:0/676: stat d9/d15/l1e 0 2024-03-24T00:48:58.827 INFO:tasks.workunit.client.1.smithi088.stdout:0/677: chown d9/d15 1335496 1 2024-03-24T00:48:58.829 INFO:tasks.workunit.client.0.smithi038.stdout:5/572: symlink d6/d16/lb9 0 2024-03-24T00:48:58.831 INFO:tasks.workunit.client.1.smithi088.stdout:7/556: creat d3/d32/d34/d9c/fae x:0 0 0 2024-03-24T00:48:58.837 INFO:tasks.workunit.client.1.smithi088.stdout:8/638: sync 2024-03-24T00:48:58.842 INFO:tasks.workunit.client.0.smithi038.stdout:3/626: rename d3/d9/dc/da6/db5/fcc to d3/d9/dc/da6/db5/fcf 0 2024-03-24T00:48:58.848 INFO:tasks.workunit.client.1.smithi088.stdout:0/678: read d9/dc/de/d19/d44/f4a [313733,17976] 0 2024-03-24T00:48:58.848 INFO:tasks.workunit.client.0.smithi038.stdout:8/595: symlink d0/d15/d2c/lc7 0 2024-03-24T00:48:58.848 INFO:tasks.workunit.client.0.smithi038.stdout:8/596: write d0/d1d/d27/d4d/f7b [875298,32890] 0 2024-03-24T00:48:58.848 INFO:tasks.workunit.client.0.smithi038.stdout:4/583: mknod da/d12/d29/d5b/cc7 0 2024-03-24T00:48:58.848 INFO:tasks.workunit.client.0.smithi038.stdout:4/584: chown da/d12/d37/dd/d15/d4c/d70/f9b 1 1 2024-03-24T00:48:58.850 INFO:tasks.workunit.client.1.smithi088.stdout:5/678: creat d1/dd/fe3 x:0 0 0 2024-03-24T00:48:58.850 INFO:tasks.workunit.client.0.smithi038.stdout:0/540: link d0/d14/c6c d0/d12/d42/d44/cae 0 2024-03-24T00:48:58.854 INFO:tasks.workunit.client.0.smithi038.stdout:9/665: rename d12/d28/d3a/f6d to d12/d1a/fdc 0 2024-03-24T00:48:58.857 INFO:tasks.workunit.client.0.smithi038.stdout:7/583: dwrite d2/d9/f54 [4194304,4194304] 0 2024-03-24T00:48:58.862 INFO:tasks.workunit.client.1.smithi088.stdout:1/647: mknod d6/d23/daf/d95/d5b/d5d/d61/d7d/dc8/ccc 0 2024-03-24T00:48:58.869 INFO:tasks.workunit.client.0.smithi038.stdout:1/691: mkdir da/dd/d22/d5c/de8 0 2024-03-24T00:48:58.871 INFO:tasks.workunit.client.1.smithi088.stdout:8/639: mkdir d9/dc/dce/ddc 0 2024-03-24T00:48:58.871 INFO:tasks.workunit.client.1.smithi088.stdout:8/640: dread - d9/dc/fd1 zero size 2024-03-24T00:48:58.872 INFO:tasks.workunit.client.1.smithi088.stdout:8/641: fsync d9/f57 0 2024-03-24T00:48:58.873 INFO:tasks.workunit.client.1.smithi088.stdout:5/679: dread d1/d19/fa6 [0,4194304] 0 2024-03-24T00:48:58.873 INFO:tasks.workunit.client.1.smithi088.stdout:5/680: write d1/dd/d6f/da1/d3f/d88/d8b/fd8 [531465,45496] 0 2024-03-24T00:48:58.874 INFO:tasks.workunit.client.1.smithi088.stdout:0/679: write d9/dc/de/f3e [812092,116346] 0 2024-03-24T00:48:58.875 INFO:tasks.workunit.client.0.smithi038.stdout:4/585: write da/d27/fb5 [2830394,117004] 0 2024-03-24T00:48:58.879 INFO:tasks.workunit.client.1.smithi088.stdout:6/659: dwrite db/d1d/d50/f77 [0,4194304] 0 2024-03-24T00:48:58.880 INFO:tasks.workunit.client.1.smithi088.stdout:9/654: rmdir d1/d15/d80/da5 39 2024-03-24T00:48:58.881 INFO:tasks.workunit.client.1.smithi088.stdout:5/681: dread d1/dd/d6f/da1/d3f/d88/d8f/d69/f85 [0,4194304] 0 2024-03-24T00:48:58.881 INFO:tasks.workunit.client.1.smithi088.stdout:5/682: readlink d1/d10/l40 0 2024-03-24T00:48:58.882 INFO:tasks.workunit.client.0.smithi038.stdout:3/627: dread d3/d9/fb [0,4194304] 0 2024-03-24T00:48:58.890 INFO:tasks.workunit.client.0.smithi038.stdout:6/656: mknod d1/da/d1b/d1d/d3c/cd6 0 2024-03-24T00:48:58.890 INFO:tasks.workunit.client.0.smithi038.stdout:6/657: chown d1/da/d1b/d2f 7777 1 2024-03-24T00:48:58.891 INFO:tasks.workunit.client.1.smithi088.stdout:1/648: mknod d6/d27/d67/d88/da4/ccd 0 2024-03-24T00:48:58.892 INFO:tasks.workunit.client.1.smithi088.stdout:3/593: sync 2024-03-24T00:48:58.892 INFO:tasks.workunit.client.1.smithi088.stdout:4/627: sync 2024-03-24T00:48:58.892 INFO:tasks.workunit.client.1.smithi088.stdout:4/628: chown d5/d16/c43 905 1 2024-03-24T00:48:58.892 INFO:tasks.workunit.client.1.smithi088.stdout:4/629: fsync d5/d13/d18/d24/d3d/fda 0 2024-03-24T00:48:58.892 INFO:tasks.workunit.client.1.smithi088.stdout:0/680: creat d9/d26/d8f/da1/d62/d9b/db0/db4/fd9 x:0 0 0 2024-03-24T00:48:58.892 INFO:tasks.workunit.client.1.smithi088.stdout:1/649: rename d6/d23 to d6/d23/daf/d95/d5b/d5e/dce 22 2024-03-24T00:48:58.892 INFO:tasks.workunit.client.1.smithi088.stdout:0/681: write d9/dc/de/fcf [276302,26539] 0 2024-03-24T00:48:58.892 INFO:tasks.workunit.client.0.smithi038.stdout:0/541: mkdir d0/daf 0 2024-03-24T00:48:58.893 INFO:tasks.workunit.client.0.smithi038.stdout:5/573: rmdir d6/d19/d20/d64/d6b/d77 39 2024-03-24T00:48:58.900 INFO:tasks.workunit.client.1.smithi088.stdout:6/660: link db/d1d/d23/d5c/c96 db/dc5/dc8/d43/daf/cca 0 2024-03-24T00:48:58.900 INFO:tasks.workunit.client.1.smithi088.stdout:6/661: chown db/dc5/dc8/l54 7674 1 2024-03-24T00:48:58.900 INFO:tasks.workunit.client.1.smithi088.stdout:6/662: fdatasync db/dd/f37 0 2024-03-24T00:48:58.900 INFO:tasks.workunit.client.1.smithi088.stdout:6/663: dread - db/dc5/dc8/d43/f9a zero size 2024-03-24T00:48:58.901 INFO:tasks.workunit.client.1.smithi088.stdout:7/557: write d3/f21 [443220,110606] 0 2024-03-24T00:48:58.902 INFO:tasks.workunit.client.1.smithi088.stdout:5/683: creat d1/d19/db9/fe4 x:0 0 0 2024-03-24T00:48:58.907 INFO:tasks.workunit.client.1.smithi088.stdout:3/594: link d3/d6/d27/d3c/f47 d3/d1a/d1e/d76/dae/d4b/fbf 0 2024-03-24T00:48:58.910 INFO:tasks.workunit.client.1.smithi088.stdout:3/595: write d3/d6/d19/f77 [371457,101405] 0 2024-03-24T00:48:58.910 INFO:tasks.workunit.client.1.smithi088.stdout:3/596: readlink d3/d6/d19/l9c 0 2024-03-24T00:48:58.913 INFO:tasks.workunit.client.1.smithi088.stdout:9/655: read d1/d10/d77/d8a/f99 [2927381,38232] 0 2024-03-24T00:48:58.913 INFO:tasks.workunit.client.1.smithi088.stdout:9/656: write d1/d22/d27/f97 [796633,105673] 0 2024-03-24T00:48:58.918 INFO:tasks.workunit.client.0.smithi038.stdout:3/628: truncate d3/f29 221221 0 2024-03-24T00:48:58.918 INFO:tasks.workunit.client.0.smithi038.stdout:3/629: chown d3/d9/dc/da6/db5/l8e 133835 1 2024-03-24T00:48:58.923 INFO:tasks.workunit.client.1.smithi088.stdout:4/630: rename d5/d7/d92/fad to d5/d16/d1a/d22/dc6/fdc 0 2024-03-24T00:48:58.927 INFO:tasks.workunit.client.0.smithi038.stdout:0/542: unlink d0/d12/d42/d9d/l50 0 2024-03-24T00:48:58.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:48:58 smithi088 ceph-mon[36610]: pgmap v35: 65 pgs: 65 active+clean; 2.2 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 152 MiB/s rd, 180 MiB/s wr, 253 op/s 2024-03-24T00:48:58.932 INFO:tasks.workunit.client.1.smithi088.stdout:1/650: creat d6/d27/d67/fcf x:0 0 0 2024-03-24T00:48:58.934 INFO:tasks.workunit.client.0.smithi038.stdout:5/574: symlink d6/d19/d20/d42/lba 0 2024-03-24T00:48:58.936 INFO:tasks.workunit.client.1.smithi088.stdout:0/682: symlink d9/dc/lda 0 2024-03-24T00:48:58.941 INFO:tasks.workunit.client.1.smithi088.stdout:7/558: truncate d3/d32/d34/d9c/d4b/f99 21867 0 2024-03-24T00:48:58.941 INFO:tasks.workunit.client.1.smithi088.stdout:5/684: mkdir d1/dd/d6f/da1/d3f/d88/d8b/de5 0 2024-03-24T00:48:58.941 INFO:tasks.workunit.client.1.smithi088.stdout:9/657: creat d1/d10/d77/dc0/fd4 x:0 0 0 2024-03-24T00:48:58.943 INFO:tasks.workunit.client.1.smithi088.stdout:1/651: creat d6/d23/dcb/fd0 x:0 0 0 2024-03-24T00:48:58.950 INFO:tasks.workunit.client.1.smithi088.stdout:7/559: symlink d3/d32/d76/laf 0 2024-03-24T00:48:58.950 INFO:tasks.workunit.client.1.smithi088.stdout:0/683: unlink d9/dc/de/d19/d44/d46/d59/f70 0 2024-03-24T00:48:58.950 INFO:tasks.workunit.client.1.smithi088.stdout:0/684: write d9/dc/de/f1d [630982,85959] 0 2024-03-24T00:48:58.950 INFO:tasks.workunit.client.1.smithi088.stdout:8/642: dwrite d9/dc/d11/d24/d29/d39/fc9 [0,4194304] 0 2024-03-24T00:48:58.950 INFO:tasks.workunit.client.1.smithi088.stdout:0/685: write d9/dc/de/d7e/f89 [831451,63375] 0 2024-03-24T00:48:58.951 INFO:tasks.workunit.client.0.smithi038.stdout:9/666: dwrite d12/d81/fd8 [0,4194304] 0 2024-03-24T00:48:58.952 INFO:tasks.workunit.client.1.smithi088.stdout:5/685: link d1/dd/fe3 d1/d10/d1b/d2f/d57/d9e/daa/fe6 0 2024-03-24T00:48:58.954 INFO:tasks.workunit.client.0.smithi038.stdout:7/584: dwrite d2/d2c/f80 [0,4194304] 0 2024-03-24T00:48:58.954 INFO:tasks.workunit.client.0.smithi038.stdout:8/597: dwrite d0/d45/d66/f40 [0,4194304] 0 2024-03-24T00:48:58.956 INFO:tasks.workunit.client.1.smithi088.stdout:4/631: dread f2 [0,4194304] 0 2024-03-24T00:48:58.956 INFO:tasks.workunit.client.1.smithi088.stdout:4/632: write d5/d17/f81 [461059,13099] 0 2024-03-24T00:48:58.960 INFO:tasks.workunit.client.0.smithi038.stdout:3/630: rename d3/d9/dc/df/c40 to d3/d30/d70/cd0 0 2024-03-24T00:48:58.964 INFO:tasks.workunit.client.1.smithi088.stdout:9/658: unlink d1/d22/d27/d39/f86 0 2024-03-24T00:48:58.964 INFO:tasks.workunit.client.1.smithi088.stdout:9/659: write d1/d10/d77/fbb [994594,65574] 0 2024-03-24T00:48:58.967 INFO:tasks.workunit.client.1.smithi088.stdout:6/664: dread db/dd/d61/fc4 [0,4194304] 0 2024-03-24T00:48:58.967 INFO:tasks.workunit.client.1.smithi088.stdout:2/619: dwrite db/dd/f28 [0,4194304] 0 2024-03-24T00:48:58.968 INFO:tasks.workunit.client.0.smithi038.stdout:5/575: dread d6/d1e/f1f [0,4194304] 0 2024-03-24T00:48:58.969 INFO:tasks.workunit.client.0.smithi038.stdout:5/576: write d6/d16/d3c/f93 [716634,23301] 0 2024-03-24T00:48:58.969 INFO:tasks.workunit.client.0.smithi038.stdout:0/543: mkdir d0/d12/d42/d5e/db0 0 2024-03-24T00:48:58.970 INFO:tasks.workunit.client.1.smithi088.stdout:1/652: creat d6/d23/daf/d95/d5b/d5d/d61/d7d/dc8/fd1 x:0 0 0 2024-03-24T00:48:58.979 INFO:tasks.workunit.client.1.smithi088.stdout:1/653: read d6/d23/daf/d95/d5b/d5e/f64 [338249,126170] 0 2024-03-24T00:48:58.980 INFO:tasks.workunit.client.0.smithi038.stdout:9/667: mkdir d12/d28/d9c/ddd 0 2024-03-24T00:48:58.983 INFO:tasks.workunit.client.0.smithi038.stdout:1/692: dwrite da/dd/d22/f55 [0,4194304] 0 2024-03-24T00:48:58.986 INFO:tasks.workunit.client.0.smithi038.stdout:7/585: truncate d2/d9/da/dc/d4a/f6d 3594691 0 2024-03-24T00:48:58.993 INFO:tasks.workunit.client.0.smithi038.stdout:5/577: read d6/d19/d20/d42/d43/d68/d6e/f7f [3734815,109078] 0 2024-03-24T00:48:58.993 INFO:tasks.workunit.client.0.smithi038.stdout:8/598: mkdir d0/d1d/d20/d3c/d9e/da3/dc8 0 2024-03-24T00:48:58.993 INFO:tasks.workunit.client.0.smithi038.stdout:8/599: chown d0/d15/fbd 5165684 1 2024-03-24T00:48:58.994 INFO:tasks.workunit.client.1.smithi088.stdout:4/633: symlink d5/dac/ldd 0 2024-03-24T00:48:58.998 INFO:tasks.workunit.client.1.smithi088.stdout:6/665: getdents db/dd/d61/d6d/d74 0 2024-03-24T00:48:58.999 INFO:tasks.workunit.client.0.smithi038.stdout:4/586: dwrite da/fa2 [0,4194304] 0 2024-03-24T00:48:59.001 INFO:tasks.workunit.client.0.smithi038.stdout:6/658: dwrite d1/da/d1b/d25/d49/d4e/d51/f9c [0,4194304] 0 2024-03-24T00:48:59.003 INFO:tasks.workunit.client.0.smithi038.stdout:0/544: rename d0/d12/d42/d44/fa9 to d0/d12/d42/d83/fb1 0 2024-03-24T00:48:59.003 INFO:tasks.workunit.client.0.smithi038.stdout:0/545: creat d0/d12/d92/fb2 x:0 0 0 2024-03-24T00:48:59.003 INFO:tasks.workunit.client.0.smithi038.stdout:0/546: chown d0/d12/d15/l91 112 1 2024-03-24T00:48:59.003 INFO:tasks.workunit.client.0.smithi038.stdout:3/631: mknod d3/d30/d5d/d7c/db0/cd1 0 2024-03-24T00:48:59.003 INFO:tasks.workunit.client.0.smithi038.stdout:3/632: stat d3/d9/dc/da6/db5/dad/fc4 0 2024-03-24T00:48:59.007 INFO:tasks.workunit.client.1.smithi088.stdout:2/620: unlink db/d21/c5b 0 2024-03-24T00:48:59.009 INFO:tasks.workunit.client.1.smithi088.stdout:9/660: dwrite d1/d10/d77/d8a/f99 [0,4194304] 0 2024-03-24T00:48:59.009 INFO:tasks.workunit.client.1.smithi088.stdout:7/560: rename d3/f1d to d3/d32/d34/d9c/d4b/d74/d79/fb0 0 2024-03-24T00:48:59.014 INFO:tasks.workunit.client.1.smithi088.stdout:3/597: dwrite d3/d1a/d1e/d76/f7c [0,4194304] 0 2024-03-24T00:48:59.014 INFO:tasks.workunit.client.1.smithi088.stdout:3/598: chown d3/de/d6f/fa9 322715 1 2024-03-24T00:48:59.014 INFO:tasks.workunit.client.1.smithi088.stdout:3/599: chown d3/de/f16 0 1 2024-03-24T00:48:59.014 INFO:tasks.workunit.client.1.smithi088.stdout:1/654: creat d6/d27/d67/d88/fd2 x:0 0 0 2024-03-24T00:48:59.014 INFO:tasks.workunit.client.1.smithi088.stdout:1/655: truncate d6/fb4 675178 0 2024-03-24T00:48:59.018 INFO:tasks.workunit.client.0.smithi038.stdout:8/600: write d0/d15/d2c/f88 [169344,83088] 0 2024-03-24T00:48:59.019 INFO:tasks.workunit.client.1.smithi088.stdout:6/666: mkdir db/d3d/dcb 0 2024-03-24T00:48:59.023 INFO:tasks.workunit.client.0.smithi038.stdout:7/586: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/cdd 0 2024-03-24T00:48:59.027 INFO:tasks.workunit.client.0.smithi038.stdout:5/578: mknod d6/d19/d20/d42/d43/d87/cbb 0 2024-03-24T00:48:59.031 INFO:tasks.workunit.client.1.smithi088.stdout:9/661: unlink d1/d10/d9b/dab/db8/c8e 0 2024-03-24T00:48:59.033 INFO:tasks.workunit.client.0.smithi038.stdout:0/547: dread d0/d12/d42/f61 [0,4194304] 0 2024-03-24T00:48:59.033 INFO:tasks.workunit.client.0.smithi038.stdout:1/693: dwrite da/dd/d7a/d82/fe0 [0,4194304] 0 2024-03-24T00:48:59.034 INFO:tasks.workunit.client.0.smithi038.stdout:0/548: chown d0/d12/d42/d83/lad 4529 1 2024-03-24T00:48:59.034 INFO:tasks.workunit.client.0.smithi038.stdout:0/549: readlink d0/d12/d42/d9d/l38 0 2024-03-24T00:48:59.038 INFO:tasks.workunit.client.0.smithi038.stdout:4/587: symlink da/d12/d29/lc8 0 2024-03-24T00:48:59.039 INFO:tasks.workunit.client.1.smithi088.stdout:0/686: rename d9/dc/l86 to d9/dc/de/d19/d44/d46/d65/d2a/ldb 0 2024-03-24T00:48:59.041 INFO:tasks.workunit.client.1.smithi088.stdout:3/600: symlink d3/d6/d27/d3c/lc0 0 2024-03-24T00:48:59.044 INFO:tasks.workunit.client.1.smithi088.stdout:8/643: dwrite d9/dc/d11/d24/d29/d5b/d6f/f83 [4194304,4194304] 0 2024-03-24T00:48:59.045 INFO:tasks.workunit.client.1.smithi088.stdout:8/644: creat d9/dc/d11/d24/d29/d5b/db9/fdd x:0 0 0 2024-03-24T00:48:59.045 INFO:tasks.workunit.client.1.smithi088.stdout:8/645: write d9/f32 [3128996,100098] 0 2024-03-24T00:48:59.045 INFO:tasks.workunit.client.1.smithi088.stdout:8/646: readlink d9/d6b/l9f 0 2024-03-24T00:48:59.045 INFO:tasks.workunit.client.1.smithi088.stdout:8/647: fsync d9/dc/d11/d24/d29/d36/f97 0 2024-03-24T00:48:59.053 INFO:tasks.workunit.client.0.smithi038.stdout:6/659: rename d1/da/d1b/d25/f8e to d1/da/d1b/d2f/fd7 0 2024-03-24T00:48:59.053 INFO:tasks.workunit.client.0.smithi038.stdout:6/660: chown d1/da/d1b/d1d/f4f 7596 1 2024-03-24T00:48:59.054 INFO:tasks.workunit.client.1.smithi088.stdout:9/662: symlink d1/d10/d11/ld5 0 2024-03-24T00:48:59.055 INFO:tasks.workunit.client.0.smithi038.stdout:9/668: dwrite d12/d28/d71/d99/fc9 [0,4194304] 0 2024-03-24T00:48:59.056 INFO:tasks.workunit.client.0.smithi038.stdout:9/669: chown d12/f42 21444 1 2024-03-24T00:48:59.056 INFO:tasks.workunit.client.0.smithi038.stdout:7/587: dread d2/d9/da/d23/d2b/d5b/d72/d92/d98/f9f [0,4194304] 0 2024-03-24T00:48:59.057 INFO:tasks.workunit.client.1.smithi088.stdout:4/634: dwrite d5/d16/d37/f4e [0,4194304] 0 2024-03-24T00:48:59.060 INFO:tasks.workunit.client.1.smithi088.stdout:5/686: rename d1/dd/d6f/da1/d3f/d88/d8f/fbd to d1/d10/d1b/d2f/fe7 0 2024-03-24T00:48:59.063 INFO:tasks.workunit.client.0.smithi038.stdout:3/633: mkdir d3/d9/dc/da6/db5/d19/dd2 0 2024-03-24T00:48:59.063 INFO:tasks.workunit.client.0.smithi038.stdout:3/634: creat d3/d9/dc/da6/db5/d19/d82/fd3 x:0 0 0 2024-03-24T00:48:59.063 INFO:tasks.workunit.client.1.smithi088.stdout:8/648: write d9/dc/d11/d24/d29/d36/f86 [2845891,47867] 0 2024-03-24T00:48:59.064 INFO:tasks.workunit.client.1.smithi088.stdout:8/649: fdatasync d9/dc/d11/d24/d29/d5b/d6f/f83 0 2024-03-24T00:48:59.064 INFO:tasks.workunit.client.1.smithi088.stdout:9/663: dread d1/f20 [0,4194304] 0 2024-03-24T00:48:59.064 INFO:tasks.workunit.client.1.smithi088.stdout:9/664: truncate d1/d10/d77/fb4 1010229 0 2024-03-24T00:48:59.065 INFO:tasks.workunit.client.1.smithi088.stdout:0/687: symlink d9/d26/d8f/da1/ldc 0 2024-03-24T00:48:59.066 INFO:tasks.workunit.client.0.smithi038.stdout:4/588: dread da/d12/d29/d5b/d73/fba [0,4194304] 0 2024-03-24T00:48:59.068 INFO:tasks.workunit.client.0.smithi038.stdout:8/601: rmdir d0/d45/d37/d5f/db7/d62 39 2024-03-24T00:48:59.068 INFO:tasks.workunit.client.0.smithi038.stdout:8/602: readlink d0/d1d/d27/l3f 0 2024-03-24T00:48:59.068 INFO:tasks.workunit.client.0.smithi038.stdout:8/603: chown d0/d10/lc0 67487955 1 2024-03-24T00:48:59.068 INFO:tasks.workunit.client.0.smithi038.stdout:8/604: write d0/d1d/f8a [300950,72032] 0 2024-03-24T00:48:59.069 INFO:tasks.workunit.client.1.smithi088.stdout:3/601: getdents d3/de/d6f 0 2024-03-24T00:48:59.069 INFO:tasks.workunit.client.1.smithi088.stdout:3/602: dread - d3/d6/d27/d3c/fa8 zero size 2024-03-24T00:48:59.070 INFO:tasks.workunit.client.0.smithi038.stdout:5/579: mknod d6/d19/d20/d64/d6b/d77/cbc 0 2024-03-24T00:48:59.082 INFO:tasks.workunit.client.0.smithi038.stdout:0/550: unlink d0/d1b/fa6 0 2024-03-24T00:48:59.082 INFO:tasks.workunit.client.0.smithi038.stdout:0/551: chown d0/d12/d42/d44/cae 6473 1 2024-03-24T00:48:59.082 INFO:tasks.workunit.client.0.smithi038.stdout:0/552: creat d0/d2d/fb3 x:0 0 0 2024-03-24T00:48:59.082 INFO:tasks.workunit.client.0.smithi038.stdout:0/553: fdatasync d0/d12/d15/d58/d64/f7d 0 2024-03-24T00:48:59.096 INFO:tasks.workunit.client.0.smithi038.stdout:9/670: mknod d12/d28/d3a/d8f/cde 0 2024-03-24T00:48:59.096 INFO:tasks.workunit.client.0.smithi038.stdout:9/671: write d12/d28/d3a/f3f [661273,21669] 0 2024-03-24T00:48:59.096 INFO:tasks.workunit.client.0.smithi038.stdout:7/588: link d2/d9/da/d23/lc0 d2/d9/da/d23/d2b/lde 0 2024-03-24T00:48:59.096 INFO:tasks.workunit.client.0.smithi038.stdout:7/589: fdatasync d2/d9/da/d23/d2b/d5b/d72/fbf 0 2024-03-24T00:48:59.105 INFO:tasks.workunit.client.0.smithi038.stdout:2/561: sync 2024-03-24T00:48:59.112 INFO:tasks.workunit.client.0.smithi038.stdout:5/580: symlink d6/d60/lbd 0 2024-03-24T00:48:59.124 INFO:tasks.workunit.client.0.smithi038.stdout:0/554: unlink d0/d12/d15/d1a/l1c 0 2024-03-24T00:48:59.124 INFO:tasks.workunit.client.0.smithi038.stdout:0/555: read d0/d12/f68 [489091,122085] 0 2024-03-24T00:48:59.125 INFO:tasks.workunit.client.0.smithi038.stdout:4/589: symlink da/d12/d37/dd/lc9 0 2024-03-24T00:48:59.125 INFO:tasks.workunit.client.1.smithi088.stdout:2/621: dwrite db/d21/f5f [0,4194304] 0 2024-03-24T00:48:59.131 INFO:tasks.workunit.client.0.smithi038.stdout:0/556: mkdir d0/d12/d42/d5e/db4 0 2024-03-24T00:48:59.131 INFO:tasks.workunit.client.0.smithi038.stdout:0/557: chown d0/c62 5903 1 2024-03-24T00:48:59.131 INFO:tasks.workunit.client.0.smithi038.stdout:0/558: truncate d0/d12/d42/d83/f9f 109005 0 2024-03-24T00:48:59.132 INFO:tasks.workunit.client.0.smithi038.stdout:3/635: dwrite d3/f8 [0,4194304] 0 2024-03-24T00:48:59.133 INFO:tasks.workunit.client.0.smithi038.stdout:7/590: dread d2/d9/da/d23/d2b/d5b/d72/d92/d33/f31 [0,4194304] 0 2024-03-24T00:48:59.133 INFO:tasks.workunit.client.0.smithi038.stdout:7/591: readlink d2/d9/da/d23/d2b/d5b/d72/d92/l25 0 2024-03-24T00:48:59.138 INFO:tasks.workunit.client.0.smithi038.stdout:3/636: mkdir d3/d9/d77/dd4 0 2024-03-24T00:48:59.140 INFO:tasks.workunit.client.0.smithi038.stdout:7/592: creat d2/fdf x:0 0 0 2024-03-24T00:48:59.140 INFO:tasks.workunit.client.0.smithi038.stdout:7/593: dread - d2/d2c/f32 zero size 2024-03-24T00:48:59.142 INFO:tasks.workunit.client.0.smithi038.stdout:1/694: write da/f7e [3775955,103946] 0 2024-03-24T00:48:59.149 INFO:tasks.workunit.client.1.smithi088.stdout:5/687: truncate d1/d10/d1b/d2f/d57/f71 176198 0 2024-03-24T00:48:59.153 INFO:tasks.workunit.client.1.smithi088.stdout:8/650: unlink d9/dc/d11/d24/d29/d39/cb8 0 2024-03-24T00:48:59.153 INFO:tasks.workunit.client.1.smithi088.stdout:8/651: chown d9/dc/d11/d24/d29/d39/d84/f6e 125 1 2024-03-24T00:48:59.153 INFO:tasks.workunit.client.1.smithi088.stdout:8/652: dread - d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/fd4 zero size 2024-03-24T00:48:59.154 INFO:tasks.workunit.client.1.smithi088.stdout:9/665: creat d1/d10/d9b/dab/db8/da7/fd6 x:0 0 0 2024-03-24T00:48:59.155 INFO:tasks.workunit.client.0.smithi038.stdout:3/637: symlink d3/d9/d77/ld5 0 2024-03-24T00:48:59.161 INFO:tasks.workunit.client.1.smithi088.stdout:3/603: mknod d3/de/d6f/d85/d49/d70/d96/cc1 0 2024-03-24T00:48:59.163 INFO:tasks.workunit.client.0.smithi038.stdout:2/562: write d4/d27/d2f/d30/d3e/d9b/fa7 [54094,61441] 0 2024-03-24T00:48:59.163 INFO:tasks.workunit.client.0.smithi038.stdout:2/563: stat d4/d55/lc2 0 2024-03-24T00:48:59.166 INFO:tasks.workunit.client.1.smithi088.stdout:4/635: rmdir d5/d13/d63/d78 39 2024-03-24T00:48:59.166 INFO:tasks.workunit.client.1.smithi088.stdout:4/636: chown d5/d16/d37/c9d 46243698 1 2024-03-24T00:48:59.177 INFO:tasks.workunit.client.0.smithi038.stdout:6/661: dwrite d1/da/d1b/d2f/f7f [0,4194304] 0 2024-03-24T00:48:59.178 INFO:tasks.workunit.client.0.smithi038.stdout:6/662: chown d1/da/d1b/fb3 291697 1 2024-03-24T00:48:59.178 INFO:tasks.workunit.client.0.smithi038.stdout:3/638: symlink d3/d9/dc/da6/db5/d19/d82/d89/ld6 0 2024-03-24T00:48:59.178 INFO:tasks.workunit.client.0.smithi038.stdout:3/639: truncate d3/d9/dc/df/f6b 362790 0 2024-03-24T00:48:59.178 INFO:tasks.workunit.client.0.smithi038.stdout:3/640: fdatasync d3/d30/d5d/f5e 0 2024-03-24T00:48:59.180 INFO:tasks.workunit.client.1.smithi088.stdout:3/604: write f2 [8159352,66549] 0 2024-03-24T00:48:59.181 INFO:tasks.workunit.client.1.smithi088.stdout:3/605: write d3/d1a/d1e/d76/dae/d4b/daf/fb2 [238882,38662] 0 2024-03-24T00:48:59.181 INFO:tasks.workunit.client.1.smithi088.stdout:3/606: truncate d3/f87 476381 0 2024-03-24T00:48:59.181 INFO:tasks.workunit.client.1.smithi088.stdout:3/607: write f2 [7179464,39350] 0 2024-03-24T00:48:59.181 INFO:tasks.workunit.client.1.smithi088.stdout:3/608: truncate d3/d6/d58/fb6 830176 0 2024-03-24T00:48:59.181 INFO:tasks.workunit.client.1.smithi088.stdout:3/609: read d3/d6/f2b [1112540,131058] 0 2024-03-24T00:48:59.181 INFO:tasks.workunit.client.1.smithi088.stdout:3/610: stat d3/d1a/d1e/d2f/d45 0 2024-03-24T00:48:59.181 INFO:tasks.workunit.client.1.smithi088.stdout:3/611: chown d3/de/d6f/d85/d31/f72 762 1 2024-03-24T00:48:59.182 INFO:tasks.workunit.client.0.smithi038.stdout:5/581: dread d6/d19/d20/d42/f72 [0,4194304] 0 2024-03-24T00:48:59.182 INFO:tasks.workunit.client.0.smithi038.stdout:3/641: fsync d3/d9/dc/da6/db5/d19/f46 0 2024-03-24T00:48:59.182 INFO:tasks.workunit.client.0.smithi038.stdout:8/605: dwrite d0/d45/d66/f97 [0,4194304] 0 2024-03-24T00:48:59.182 INFO:tasks.workunit.client.0.smithi038.stdout:8/606: write d0/d45/d37/d5f/db7/fb4 [365980,40030] 0 2024-03-24T00:48:59.187 INFO:tasks.workunit.client.0.smithi038.stdout:3/642: write d3/d2a/f59 [969632,15413] 0 2024-03-24T00:48:59.199 INFO:tasks.workunit.client.1.smithi088.stdout:2/622: mkdir db/dd/d96/d99/db2/d88/dc6 0 2024-03-24T00:48:59.204 INFO:tasks.workunit.client.0.smithi038.stdout:7/594: getdents d2/d9/d16 0 2024-03-24T00:48:59.204 INFO:tasks.workunit.client.0.smithi038.stdout:7/595: creat d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fe0 x:0 0 0 2024-03-24T00:48:59.204 INFO:tasks.workunit.client.0.smithi038.stdout:7/596: chown d2/d9/da/dc/d8a/db8/fce 22 1 2024-03-24T00:48:59.210 INFO:tasks.workunit.client.1.smithi088.stdout:7/561: rename d3/l13 to d3/d32/d34/d9c/d4b/d73/dab/lb1 0 2024-03-24T00:48:59.223 INFO:tasks.workunit.client.0.smithi038.stdout:0/559: dwrite d0/d12/d42/d44/f93 [0,4194304] 0 2024-03-24T00:48:59.223 INFO:tasks.workunit.client.0.smithi038.stdout:0/560: fsync d0/d1b/d2c/f86 0 2024-03-24T00:48:59.223 INFO:tasks.workunit.client.0.smithi038.stdout:0/561: chown d0/d12/d42/d44/l54 563 1 2024-03-24T00:48:59.223 INFO:tasks.workunit.client.0.smithi038.stdout:0/562: fsync d0/d1b/d2c/f3b 0 2024-03-24T00:48:59.228 INFO:tasks.workunit.client.0.smithi038.stdout:9/672: dwrite d12/d28/d3a/f2b [0,4194304] 0 2024-03-24T00:48:59.229 INFO:tasks.workunit.client.0.smithi038.stdout:9/673: stat d12/d28/d3a/d26/l2c 0 2024-03-24T00:48:59.237 INFO:tasks.workunit.client.1.smithi088.stdout:5/688: dwrite d1/d10/d1b/d2e/d9f/ddf/fb0 [0,4194304] 0 2024-03-24T00:48:59.238 INFO:tasks.workunit.client.0.smithi038.stdout:5/582: dread d6/d16/d14/f8e [0,4194304] 0 2024-03-24T00:48:59.238 INFO:tasks.workunit.client.0.smithi038.stdout:5/583: chown d6/d19/d20/d42/lba 7 1 2024-03-24T00:48:59.240 INFO:tasks.workunit.client.0.smithi038.stdout:3/643: rename d3/d9/f58 to d3/d5a/fd7 0 2024-03-24T00:48:59.244 INFO:tasks.workunit.client.1.smithi088.stdout:8/653: mkdir d9/dc/dce/ddc/dde 0 2024-03-24T00:48:59.244 INFO:tasks.workunit.client.1.smithi088.stdout:8/654: chown d9/dc/d11/f2f 4004 1 2024-03-24T00:48:59.244 INFO:tasks.workunit.client.0.smithi038.stdout:7/597: getdents d2/d9/da/d23/d2b/d5b/d72/d92/d75 0 2024-03-24T00:48:59.245 INFO:tasks.workunit.client.0.smithi038.stdout:7/598: creat d2/d9/da/dc/da8/dc5/fe1 x:0 0 0 2024-03-24T00:48:59.248 INFO:tasks.workunit.client.0.smithi038.stdout:4/590: dwrite da/d12/d29/d5b/d73/fba [4194304,4194304] 0 2024-03-24T00:48:59.257 INFO:tasks.workunit.client.1.smithi088.stdout:3/612: mkdir d3/d6/d19/d2a/dc2 0 2024-03-24T00:48:59.257 INFO:tasks.workunit.client.1.smithi088.stdout:3/613: write d3/d6/d27/d3c/fa8 [762376,13121] 0 2024-03-24T00:48:59.260 INFO:tasks.workunit.client.0.smithi038.stdout:0/563: creat d0/d14/fb5 x:0 0 0 2024-03-24T00:48:59.261 INFO:tasks.workunit.client.1.smithi088.stdout:2/623: symlink db/dd/d96/d99/db2/d88/lc7 0 2024-03-24T00:48:59.263 INFO:tasks.workunit.client.0.smithi038.stdout:9/674: dread d12/d1a/d80/f83 [0,4194304] 0 2024-03-24T00:48:59.264 INFO:tasks.workunit.client.0.smithi038.stdout:6/663: dwrite d1/da/d1b/d25/d49/d72/f76 [0,4194304] 0 2024-03-24T00:48:59.264 INFO:tasks.workunit.client.0.smithi038.stdout:6/664: dread - d1/da/d1b/fb3 zero size 2024-03-24T00:48:59.265 INFO:tasks.workunit.client.0.smithi038.stdout:2/564: dwrite d4/d18/d38/d63/f7e [0,4194304] 0 2024-03-24T00:48:59.273 INFO:tasks.workunit.client.1.smithi088.stdout:7/562: creat d3/d32/d34/d9c/d4b/d74/d79/d7f/fb2 x:0 0 0 2024-03-24T00:48:59.273 INFO:tasks.workunit.client.1.smithi088.stdout:1/656: rename d6/d23/daf/d95/d5b/d5d/d61/d7d/da5/fc3 to d6/d23/daf/d95/d5b/d5d/d61/d7d/fd3 0 2024-03-24T00:48:59.276 INFO:tasks.workunit.client.1.smithi088.stdout:6/667: sync 2024-03-24T00:48:59.277 INFO:tasks.workunit.client.1.smithi088.stdout:5/689: mknod d1/d10/d1b/d2f/d93/ce8 0 2024-03-24T00:48:59.277 INFO:tasks.workunit.client.1.smithi088.stdout:5/690: fdatasync d1/d10/d1b/d2f/d57/f5f 0 2024-03-24T00:48:59.280 INFO:tasks.workunit.client.0.smithi038.stdout:5/584: rename d6/d16/d3c/fa5 to d6/d1e/d97/fbe 0 2024-03-24T00:48:59.283 INFO:tasks.workunit.client.1.smithi088.stdout:8/655: rmdir d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/dd9 0 2024-03-24T00:48:59.284 INFO:tasks.workunit.client.0.smithi038.stdout:4/591: mkdir da/d12/d37/dd/d15/d6d/d8f/dca 0 2024-03-24T00:48:59.287 INFO:tasks.workunit.client.1.smithi088.stdout:6/668: dread db/d1d/f58 [0,4194304] 0 2024-03-24T00:48:59.288 INFO:tasks.workunit.client.0.smithi038.stdout:6/665: dread d1/d37/f90 [0,4194304] 0 2024-03-24T00:48:59.288 INFO:tasks.workunit.client.0.smithi038.stdout:9/675: mknod d12/d41/dc0/cdf 0 2024-03-24T00:48:59.298 INFO:tasks.workunit.client.0.smithi038.stdout:8/607: dwrite d0/d45/d37/d5f/d68/fa0 [0,4194304] 0 2024-03-24T00:48:59.299 INFO:tasks.workunit.client.0.smithi038.stdout:8/608: write d0/d1d/d27/d4d/daf/fb5 [2017936,44632] 0 2024-03-24T00:48:59.299 INFO:tasks.workunit.client.0.smithi038.stdout:8/609: creat d0/d1d/d27/d6b/fc9 x:0 0 0 2024-03-24T00:48:59.300 INFO:tasks.workunit.client.1.smithi088.stdout:9/666: rmdir d1/d10/d11 39 2024-03-24T00:48:59.300 INFO:tasks.workunit.client.1.smithi088.stdout:3/614: rmdir d3 39 2024-03-24T00:48:59.306 INFO:tasks.workunit.client.0.smithi038.stdout:8/610: dread d0/d1d/d20/f23 [0,4194304] 0 2024-03-24T00:48:59.307 INFO:tasks.workunit.client.1.smithi088.stdout:2/624: link db/d21/d35/d49/l72 db/dd/d96/d4e/lc8 0 2024-03-24T00:48:59.308 INFO:tasks.workunit.client.0.smithi038.stdout:2/565: mkdir d4/d27/d2f/dc5 0 2024-03-24T00:48:59.308 INFO:tasks.workunit.client.0.smithi038.stdout:2/566: chown d4/d27/d54/d67/cac 260682 1 2024-03-24T00:48:59.310 INFO:tasks.workunit.client.0.smithi038.stdout:2/567: write d4/f50 [164405,81921] 0 2024-03-24T00:48:59.310 INFO:tasks.workunit.client.0.smithi038.stdout:2/568: stat d4/d18/d26/d37/d69/d83/l84 0 2024-03-24T00:48:59.311 INFO:tasks.workunit.client.1.smithi088.stdout:0/688: rename d9/d26/d8f/da1/d62/d9b/dcb to d9/d26/d8f/dc7/ddd 0 2024-03-24T00:48:59.311 INFO:tasks.workunit.client.0.smithi038.stdout:5/585: symlink d6/d1e/d97/lbf 0 2024-03-24T00:48:59.313 INFO:tasks.workunit.client.1.smithi088.stdout:7/563: creat d3/d32/d34/d9c/d4b/d73/dab/fb3 x:0 0 0 2024-03-24T00:48:59.316 INFO:tasks.workunit.client.0.smithi038.stdout:7/599: rename d2/d9/da/dc/d4a/f67 to d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/fe2 0 2024-03-24T00:48:59.316 INFO:tasks.workunit.client.0.smithi038.stdout:7/600: creat d2/d9/da/dc/da8/dc5/fe3 x:0 0 0 2024-03-24T00:48:59.316 INFO:tasks.workunit.client.0.smithi038.stdout:7/601: chown d2/d9/da/c15 1 1 2024-03-24T00:48:59.318 INFO:tasks.workunit.client.0.smithi038.stdout:4/592: unlink da/d27/l55 0 2024-03-24T00:48:59.320 INFO:tasks.workunit.client.1.smithi088.stdout:5/691: unlink d1/d10/d1b/d2f/d57/d9e/fdd 0 2024-03-24T00:48:59.320 INFO:tasks.workunit.client.1.smithi088.stdout:5/692: chown d1/dd/d6f/da1/f50 72076210 1 2024-03-24T00:48:59.326 INFO:tasks.workunit.client.0.smithi038.stdout:6/666: symlink d1/da/d73/ld8 0 2024-03-24T00:48:59.326 INFO:tasks.workunit.client.0.smithi038.stdout:6/667: getdents d1/d37/d68 0 2024-03-24T00:48:59.326 INFO:tasks.workunit.client.0.smithi038.stdout:6/668: write d1/da/d1b/d2f/dc0/fad [822533,21775] 0 2024-03-24T00:48:59.326 INFO:tasks.workunit.client.0.smithi038.stdout:9/676: creat d12/d28/d3a/fe0 x:0 0 0 2024-03-24T00:48:59.334 INFO:tasks.workunit.client.1.smithi088.stdout:3/615: write d3/d6/d19/d2a/f71 [2087583,86976] 0 2024-03-24T00:48:59.339 INFO:tasks.workunit.client.1.smithi088.stdout:2/625: mkdir db/dd/dc9 0 2024-03-24T00:48:59.339 INFO:tasks.workunit.client.1.smithi088.stdout:2/626: chown db/dd/d96/d99/db2/d88/d8e/ca9 520 1 2024-03-24T00:48:59.345 INFO:tasks.workunit.client.1.smithi088.stdout:4/637: rename d5/d16/f57 to d5/d7/d15/fde 0 2024-03-24T00:48:59.346 INFO:tasks.workunit.client.0.smithi038.stdout:5/586: dread d6/d19/d20/d42/d43/d50/f81 [0,4194304] 0 2024-03-24T00:48:59.346 INFO:tasks.workunit.client.0.smithi038.stdout:6/669: write d1/da/d61/da2/faf [2512849,12124] 0 2024-03-24T00:48:59.347 INFO:tasks.workunit.client.0.smithi038.stdout:2/569: creat d4/d18/d38/d63/fc6 x:0 0 0 2024-03-24T00:48:59.351 INFO:tasks.workunit.client.1.smithi088.stdout:5/693: creat d1/d10/d1b/d2e/d9f/fe9 x:0 0 0 2024-03-24T00:48:59.354 INFO:tasks.workunit.client.0.smithi038.stdout:8/611: rename d0/d10/c5c to d0/d45/d66/cca 0 2024-03-24T00:48:59.354 INFO:tasks.workunit.client.0.smithi038.stdout:8/612: chown d0/d45/d37/d5f/db7/l71 0 1 2024-03-24T00:48:59.354 INFO:tasks.workunit.client.0.smithi038.stdout:8/613: readlink d0/d45/d37/d5f/db7/l71 0 2024-03-24T00:48:59.354 INFO:tasks.workunit.client.0.smithi038.stdout:8/614: readlink d0/d10/l21 0 2024-03-24T00:48:59.354 INFO:tasks.workunit.client.0.smithi038.stdout:8/615: chown d0/d1d/d8c/cba 3090368 1 2024-03-24T00:48:59.354 INFO:tasks.workunit.client.0.smithi038.stdout:8/616: getdents d0/d1d/d27 0 2024-03-24T00:48:59.357 INFO:tasks.workunit.client.1.smithi088.stdout:6/669: getdents db/d1d/d23/d5c 0 2024-03-24T00:48:59.357 INFO:tasks.workunit.client.1.smithi088.stdout:6/670: creat db/d1d/d23/fcc x:0 0 0 2024-03-24T00:48:59.358 INFO:tasks.workunit.client.0.smithi038.stdout:9/677: mkdir d12/d41/d57/de1 0 2024-03-24T00:48:59.360 INFO:tasks.workunit.client.1.smithi088.stdout:9/667: mknod d1/d10/d29/cd7 0 2024-03-24T00:48:59.362 INFO:tasks.workunit.client.1.smithi088.stdout:8/656: dwrite d9/f63 [0,4194304] 0 2024-03-24T00:48:59.364 INFO:tasks.workunit.client.1.smithi088.stdout:3/616: mkdir d3/d67/dc3 0 2024-03-24T00:48:59.364 INFO:tasks.workunit.client.0.smithi038.stdout:5/587: link d6/d19/d20/d42/lba d6/d19/d20/d42/d43/d68/d6e/lc0 0 2024-03-24T00:48:59.367 INFO:tasks.workunit.client.0.smithi038.stdout:7/602: rename d2/d9/fb0 to d2/d9/da/dc/d8a/db8/fe4 0 2024-03-24T00:48:59.368 INFO:tasks.workunit.client.1.smithi088.stdout:4/638: symlink d5/d7/d92/ldf 0 2024-03-24T00:48:59.371 INFO:tasks.workunit.client.1.smithi088.stdout:2/627: rename db/dd/f28 to db/dd/d96/d99/fca 0 2024-03-24T00:48:59.372 INFO:tasks.workunit.client.1.smithi088.stdout:1/657: dwrite d6/d23/daf/d95/d3f/f57 [0,4194304] 0 2024-03-24T00:48:59.373 INFO:tasks.workunit.client.0.smithi038.stdout:7/603: rename d2/d2c/f69 to d2/d9/da/dc/da8/dc5/fe5 0 2024-03-24T00:48:59.377 INFO:tasks.workunit.client.0.smithi038.stdout:7/604: link d2/d9/l88 d2/d2c/d9a/dcb/le6 0 2024-03-24T00:48:59.378 INFO:tasks.workunit.client.1.smithi088.stdout:5/694: truncate d1/d10/d1b/f70 5472837 0 2024-03-24T00:48:59.378 INFO:tasks.workunit.client.1.smithi088.stdout:5/695: write d1/d19/fc0 [1644421,121244] 0 2024-03-24T00:48:59.379 INFO:tasks.workunit.client.1.smithi088.stdout:5/696: write d1/d19/d48/fae [1770467,76395] 0 2024-03-24T00:48:59.383 INFO:tasks.workunit.client.1.smithi088.stdout:6/671: link db/dd/d89/lb9 db/d8e/lcd 0 2024-03-24T00:48:59.384 INFO:tasks.workunit.client.1.smithi088.stdout:9/668: mkdir d1/d22/d27/d39/d98/da3/dd8 0 2024-03-24T00:48:59.384 INFO:tasks.workunit.client.1.smithi088.stdout:9/669: chown d1/d10/d29/d85/f91 5555 1 2024-03-24T00:48:59.385 INFO:tasks.workunit.client.0.smithi038.stdout:7/605: mknod d2/d9/da/d9b/ce7 0 2024-03-24T00:48:59.385 INFO:tasks.workunit.client.0.smithi038.stdout:7/606: write d2/d9/da/d23/d2b/d5b/d72/d92/d5d/f7f [2174753,94892] 0 2024-03-24T00:48:59.385 INFO:tasks.workunit.client.1.smithi088.stdout:8/657: symlink d9/dc/d11/dd2/ldf 0 2024-03-24T00:48:59.386 INFO:tasks.workunit.client.1.smithi088.stdout:3/617: symlink d3/d1a/d1e/d76/dae/d4b/lc4 0 2024-03-24T00:48:59.387 INFO:tasks.workunit.client.1.smithi088.stdout:2/628: read db/dd/d96/d4e/f6e [3640205,40934] 0 2024-03-24T00:48:59.388 INFO:tasks.workunit.client.1.smithi088.stdout:4/639: rename d5/d16/d37/f4e to d5/d7/d15/d74/d95/fe0 0 2024-03-24T00:48:59.389 INFO:tasks.workunit.client.1.smithi088.stdout:4/640: readlink d5/d16/d94/lb0 0 2024-03-24T00:48:59.393 INFO:tasks.workunit.client.1.smithi088.stdout:1/658: creat d6/d23/daf/d95/d5b/d80/fd4 x:0 0 0 2024-03-24T00:48:59.393 INFO:tasks.workunit.client.0.smithi038.stdout:5/588: dread d6/d19/f1c [0,4194304] 0 2024-03-24T00:48:59.398 INFO:tasks.workunit.client.1.smithi088.stdout:5/697: mkdir d1/dd/d6f/da1/d3f/dea 0 2024-03-24T00:48:59.400 INFO:tasks.workunit.client.1.smithi088.stdout:6/672: dread db/d1d/d23/fbb [0,4194304] 0 2024-03-24T00:48:59.400 INFO:tasks.workunit.client.1.smithi088.stdout:6/673: write db/dd/d87/f98 [841831,33900] 0 2024-03-24T00:48:59.401 INFO:tasks.workunit.client.1.smithi088.stdout:6/674: chown db/dd/d61/d6d/d74/d88/da1 3 1 2024-03-24T00:48:59.407 INFO:tasks.workunit.client.1.smithi088.stdout:3/618: creat d3/d32/fc5 x:0 0 0 2024-03-24T00:48:59.410 INFO:tasks.workunit.client.1.smithi088.stdout:2/629: rmdir db/dd/d96/d4e/d8f/d9c/db0/dbf 0 2024-03-24T00:48:59.410 INFO:tasks.workunit.client.1.smithi088.stdout:2/630: stat db/dd/f1f 0 2024-03-24T00:48:59.414 INFO:tasks.workunit.client.0.smithi038.stdout:8/617: dread d0/d45/d66/da6/fae [0,4194304] 0 2024-03-24T00:48:59.414 INFO:tasks.workunit.client.0.smithi038.stdout:8/618: write d0/d45/d37/f48 [1652959,33993] 0 2024-03-24T00:48:59.416 INFO:tasks.workunit.client.0.smithi038.stdout:5/589: read d6/d19/d20/d42/d4b/f4f [263134,70146] 0 2024-03-24T00:48:59.416 INFO:tasks.workunit.client.0.smithi038.stdout:5/590: chown d6/d1e/f1f 2977 1 2024-03-24T00:48:59.417 INFO:tasks.workunit.client.1.smithi088.stdout:5/698: mkdir d1/d10/d1b/d2f/d8a/deb 0 2024-03-24T00:48:59.417 INFO:tasks.workunit.client.1.smithi088.stdout:9/670: rename d1/d10/d11/d23/fbc to d1/d10/d9b/dab/d7a/fd9 0 2024-03-24T00:48:59.420 INFO:tasks.workunit.client.0.smithi038.stdout:0/564: dwrite d0/d2d/fb3 [0,4194304] 0 2024-03-24T00:48:59.421 INFO:tasks.workunit.client.1.smithi088.stdout:3/619: symlink d3/d6/d58/lc6 0 2024-03-24T00:48:59.421 INFO:tasks.workunit.client.1.smithi088.stdout:3/620: write d3/d1a/d1e/f28 [4075745,109829] 0 2024-03-24T00:48:59.424 INFO:tasks.workunit.client.0.smithi038.stdout:6/670: dwrite d1/da/f1a [4194304,4194304] 0 2024-03-24T00:48:59.425 INFO:tasks.workunit.client.0.smithi038.stdout:8/619: mknod d0/d1d/d20/d3c/ccb 0 2024-03-24T00:48:59.427 INFO:tasks.workunit.client.0.smithi038.stdout:5/591: link d6/d19/d20/d42/d4b/f4f d6/d16/d14/fc1 0 2024-03-24T00:48:59.430 INFO:tasks.workunit.client.1.smithi088.stdout:4/641: symlink d5/d7/dd/le1 0 2024-03-24T00:48:59.431 INFO:tasks.workunit.client.1.smithi088.stdout:1/659: dread d6/d23/f29 [0,4194304] 0 2024-03-24T00:48:59.437 INFO:tasks.workunit.client.1.smithi088.stdout:9/671: mkdir d1/d22/d27/d39/d98/dda 0 2024-03-24T00:48:59.438 INFO:tasks.workunit.client.1.smithi088.stdout:9/672: truncate d1/d10/d9b/dab/db8/da7/fd6 795537 0 2024-03-24T00:48:59.438 INFO:tasks.workunit.client.1.smithi088.stdout:9/673: readlink d1/d22/d27/d39/lc4 0 2024-03-24T00:48:59.438 INFO:tasks.workunit.client.1.smithi088.stdout:9/674: chown d1/d22/d27/d39/f58 0 1 2024-03-24T00:48:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:9/678: dwrite d12/d28/d71/d99/fc9 [0,4194304] 0 2024-03-24T00:48:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:9/679: readlink d12/d28/d71/d99/ld2 0 2024-03-24T00:48:59.440 INFO:tasks.workunit.client.1.smithi088.stdout:3/621: mknod d3/de/d6f/d85/d49/d70/d96/cc7 0 2024-03-24T00:48:59.445 INFO:tasks.workunit.client.1.smithi088.stdout:2/631: mkdir db/d21/d35/dcb 0 2024-03-24T00:48:59.452 INFO:tasks.workunit.client.0.smithi038.stdout:2/570: dwrite d4/d27/f47 [0,4194304] 0 2024-03-24T00:48:59.454 INFO:tasks.workunit.client.1.smithi088.stdout:1/660: symlink d6/d23/daf/ld5 0 2024-03-24T00:48:59.457 INFO:tasks.workunit.client.1.smithi088.stdout:7/564: sync 2024-03-24T00:48:59.457 INFO:tasks.workunit.client.1.smithi088.stdout:7/565: truncate d3/d32/d34/d9c/fae 629224 0 2024-03-24T00:48:59.457 INFO:tasks.workunit.client.1.smithi088.stdout:0/689: sync 2024-03-24T00:48:59.457 INFO:tasks.workunit.client.1.smithi088.stdout:8/658: dwrite d9/dc/d11/d24/d29/d39/fc6 [4194304,4194304] 0 2024-03-24T00:48:59.458 INFO:tasks.workunit.client.0.smithi038.stdout:6/671: rename d1/da/d1b/d25/d49/d4e/d51/la6 to d1/d37/d68/ld9 0 2024-03-24T00:48:59.459 INFO:tasks.workunit.client.1.smithi088.stdout:9/675: symlink d1/d15/d21/d9c/ldb 0 2024-03-24T00:48:59.462 INFO:tasks.workunit.client.0.smithi038.stdout:8/620: truncate d0/d1d/d27/d4d/fb2 3229999 0 2024-03-24T00:48:59.462 INFO:tasks.workunit.client.0.smithi038.stdout:8/621: chown d0/d1d/d27/d4d/f7b 211 1 2024-03-24T00:48:59.462 INFO:tasks.workunit.client.0.smithi038.stdout:8/622: readlink d0/d45/db8/lc5 0 2024-03-24T00:48:59.466 INFO:tasks.workunit.client.0.smithi038.stdout:5/592: creat d6/d19/d20/d64/d6b/fc2 x:0 0 0 2024-03-24T00:48:59.468 INFO:tasks.workunit.client.1.smithi088.stdout:3/622: unlink d3/d1a/d1e/d2f/d45/db5/f82 0 2024-03-24T00:48:59.469 INFO:tasks.workunit.client.0.smithi038.stdout:6/672: dread d1/d37/f8b [0,4194304] 0 2024-03-24T00:48:59.469 INFO:tasks.workunit.client.0.smithi038.stdout:6/673: creat d1/da/d1b/d25/d49/fda x:0 0 0 2024-03-24T00:48:59.470 INFO:tasks.workunit.client.1.smithi088.stdout:1/661: rename d6/d23/daf/d95/d5b/d5d/d61/lae to d6/d27/ld6 0 2024-03-24T00:48:59.470 INFO:tasks.workunit.client.1.smithi088.stdout:1/662: dread - d6/d23/daf/d95/d5b/d5d/d61/d7d/dc8/fd1 zero size 2024-03-24T00:48:59.471 INFO:tasks.workunit.client.1.smithi088.stdout:6/675: dwrite db/dd/d61/d6d/d74/f8b [0,4194304] 0 2024-03-24T00:48:59.471 INFO:tasks.workunit.client.1.smithi088.stdout:6/676: fsync db/d1d/d50/d90/d55/d99/fa6 0 2024-03-24T00:48:59.471 INFO:tasks.workunit.client.1.smithi088.stdout:6/677: dread - db/dd/d61/d6d/d74/d88/f9b zero size 2024-03-24T00:48:59.471 INFO:tasks.workunit.client.0.smithi038.stdout:0/565: getdents d0/d1b 0 2024-03-24T00:48:59.474 INFO:tasks.workunit.client.1.smithi088.stdout:7/566: creat d3/d31/d56/fb4 x:0 0 0 2024-03-24T00:48:59.475 INFO:tasks.workunit.client.0.smithi038.stdout:2/571: mkdir d4/dc7 0 2024-03-24T00:48:59.476 INFO:tasks.workunit.client.1.smithi088.stdout:0/690: getdents d9/dc/de/d19/d44 0 2024-03-24T00:48:59.482 INFO:tasks.workunit.client.0.smithi038.stdout:8/623: mkdir d0/d45/d37/d5f/db7/dcc 0 2024-03-24T00:48:59.493 INFO:tasks.workunit.client.1.smithi088.stdout:3/623: symlink d3/de/d6f/d85/d31/lc8 0 2024-03-24T00:48:59.495 INFO:tasks.workunit.client.1.smithi088.stdout:4/642: dwrite d5/f46 [0,4194304] 0 2024-03-24T00:48:59.497 INFO:tasks.workunit.client.1.smithi088.stdout:6/678: dread db/f13 [4194304,4194304] 0 2024-03-24T00:48:59.497 INFO:tasks.workunit.client.1.smithi088.stdout:1/663: mknod d6/d23/daf/d95/d5b/d5d/cd7 0 2024-03-24T00:48:59.499 INFO:tasks.workunit.client.0.smithi038.stdout:2/572: unlink d4/d18/d26/d37/f43 0 2024-03-24T00:48:59.499 INFO:tasks.workunit.client.0.smithi038.stdout:2/573: truncate d4/d55/d88/f8a 528846 0 2024-03-24T00:48:59.499 INFO:tasks.workunit.client.0.smithi038.stdout:8/624: creat d0/d1d/d20/d79/fcd x:0 0 0 2024-03-24T00:48:59.499 INFO:tasks.workunit.client.0.smithi038.stdout:8/625: chown d0/d45 97 1 2024-03-24T00:48:59.500 INFO:tasks.workunit.client.0.smithi038.stdout:5/593: getdents d6/d16/d67 0 2024-03-24T00:48:59.503 INFO:tasks.workunit.client.0.smithi038.stdout:8/626: stat d0/d10/c18 0 2024-03-24T00:48:59.503 INFO:tasks.workunit.client.0.smithi038.stdout:8/627: creat d0/d1d/d27/d4d/fce x:0 0 0 2024-03-24T00:48:59.504 INFO:tasks.workunit.client.0.smithi038.stdout:8/628: creat d0/d15/fcf x:0 0 0 2024-03-24T00:48:59.504 INFO:tasks.workunit.client.0.smithi038.stdout:0/566: write d0/f5 [555344,42192] 0 2024-03-24T00:48:59.504 INFO:tasks.workunit.client.0.smithi038.stdout:0/567: readlink d0/d12/d42/d9d/l66 0 2024-03-24T00:48:59.507 INFO:tasks.workunit.client.1.smithi088.stdout:1/664: dread d6/d27/d67/d88/f92 [0,4194304] 0 2024-03-24T00:48:59.508 INFO:tasks.workunit.client.0.smithi038.stdout:5/594: rename d6/d19/d20/d42/d43/d50/fa7 to d6/d1e/d97/fc3 0 2024-03-24T00:48:59.509 INFO:tasks.workunit.client.1.smithi088.stdout:0/691: mkdir d9/dc/de/d19/d8d/dde 0 2024-03-24T00:48:59.509 INFO:tasks.workunit.client.1.smithi088.stdout:0/692: chown d9/dc/de/d19/d44/d46 0 1 2024-03-24T00:48:59.509 INFO:tasks.workunit.client.1.smithi088.stdout:0/693: truncate d9/d26/d8f/da1/fc2 851167 0 2024-03-24T00:48:59.512 INFO:tasks.workunit.client.1.smithi088.stdout:8/659: getdents d9/dc/d11/d24/d29/d36/d6a/d7b/d8f/dbc 0 2024-03-24T00:48:59.517 INFO:tasks.workunit.client.1.smithi088.stdout:5/699: dwrite d1/d10/f17 [0,4194304] 0 2024-03-24T00:48:59.518 INFO:tasks.workunit.client.1.smithi088.stdout:3/624: fdatasync d3/d6/d19/d2a/f71 0 2024-03-24T00:48:59.518 INFO:tasks.workunit.client.1.smithi088.stdout:3/625: stat d3/d1a/d1e/d2f/d45/db5/d6e 0 2024-03-24T00:48:59.519 INFO:tasks.workunit.client.0.smithi038.stdout:9/680: dwrite f4 [0,4194304] 0 2024-03-24T00:48:59.519 INFO:tasks.workunit.client.0.smithi038.stdout:9/681: stat d12/d1a/f49 0 2024-03-24T00:48:59.521 INFO:tasks.workunit.client.1.smithi088.stdout:2/632: dwrite db/d21/d73/f8d [0,4194304] 0 2024-03-24T00:48:59.526 INFO:tasks.workunit.client.0.smithi038.stdout:0/568: creat d0/d12/d15/d58/d64/da7/fb6 x:0 0 0 2024-03-24T00:48:59.526 INFO:tasks.workunit.client.1.smithi088.stdout:9/676: rename d1/d15/d21/d9c/ldb to d1/d10/d29/ldc 0 2024-03-24T00:48:59.533 INFO:tasks.workunit.client.0.smithi038.stdout:6/674: dwrite d1/da/d1b/d1d/f24 [0,4194304] 0 2024-03-24T00:48:59.533 INFO:tasks.workunit.client.0.smithi038.stdout:2/574: dread d4/f10 [0,4194304] 0 2024-03-24T00:48:59.533 INFO:tasks.workunit.client.0.smithi038.stdout:2/575: chown d4/d55/cbe 2841818 1 2024-03-24T00:48:59.535 INFO:tasks.workunit.client.1.smithi088.stdout:7/567: dread d3/fc [0,4194304] 0 2024-03-24T00:48:59.538 INFO:tasks.workunit.client.0.smithi038.stdout:9/682: unlink d12/d1a/fdc 0 2024-03-24T00:48:59.538 INFO:tasks.workunit.client.0.smithi038.stdout:9/683: readlink d12/d1a/d5f/lb6 0 2024-03-24T00:48:59.538 INFO:tasks.workunit.client.0.smithi038.stdout:9/684: dread - d12/d28/d6c/dbc/fc8 zero size 2024-03-24T00:48:59.538 INFO:tasks.workunit.client.0.smithi038.stdout:9/685: stat d12/d28/fc7 0 2024-03-24T00:48:59.538 INFO:tasks.workunit.client.0.smithi038.stdout:9/686: getdents d12/d28/d9c/ddd 0 2024-03-24T00:48:59.538 INFO:tasks.workunit.client.0.smithi038.stdout:9/687: write d12/f42 [879095,72696] 0 2024-03-24T00:48:59.538 INFO:tasks.workunit.client.0.smithi038.stdout:9/688: chown d12/d28/d9c/caa 401546744 1 2024-03-24T00:48:59.539 INFO:tasks.workunit.client.0.smithi038.stdout:0/569: symlink d0/d12/d42/d44/lb7 0 2024-03-24T00:48:59.540 INFO:tasks.workunit.client.1.smithi088.stdout:4/643: mkdir d5/d7/d15/d74/de2 0 2024-03-24T00:48:59.541 INFO:tasks.workunit.client.1.smithi088.stdout:4/644: write d5/d16/d94/fd4 [989990,100488] 0 2024-03-24T00:48:59.541 INFO:tasks.workunit.client.1.smithi088.stdout:4/645: write d5/f56 [1337015,9618] 0 2024-03-24T00:48:59.542 INFO:tasks.workunit.client.0.smithi038.stdout:5/595: write d6/d1e/f63 [4394972,81871] 0 2024-03-24T00:48:59.544 INFO:tasks.workunit.client.1.smithi088.stdout:8/660: getdents d9/dc/d11/d24/d29/d36 0 2024-03-24T00:48:59.544 INFO:tasks.workunit.client.1.smithi088.stdout:8/661: creat d9/dc/d11/d24/d29/d5b/db9/da5/fe0 x:0 0 0 2024-03-24T00:48:59.544 INFO:tasks.workunit.client.0.smithi038.stdout:6/675: rmdir d1/da/d73/d75 39 2024-03-24T00:48:59.545 INFO:tasks.workunit.client.0.smithi038.stdout:2/576: symlink d4/dc7/lc8 0 2024-03-24T00:48:59.545 INFO:tasks.workunit.client.1.smithi088.stdout:1/665: truncate d6/d27/f1d 335786 0 2024-03-24T00:48:59.545 INFO:tasks.workunit.client.0.smithi038.stdout:9/689: link d12/d28/d71/f8b d12/d28/d6c/dbc/dd7/dda/fe2 0 2024-03-24T00:48:59.548 INFO:tasks.workunit.client.1.smithi088.stdout:3/626: mknod d3/d1a/d1e/d2f/d45/db5/cc9 0 2024-03-24T00:48:59.550 INFO:tasks.workunit.client.1.smithi088.stdout:3/627: write d3/d6/d58/fb6 [157678,44506] 0 2024-03-24T00:48:59.551 INFO:tasks.workunit.client.0.smithi038.stdout:6/676: rename d1/da/d1b/d2f/f62 to d1/da/d61/d69/fdb 0 2024-03-24T00:48:59.553 INFO:tasks.workunit.client.1.smithi088.stdout:2/633: rename db/dd/d96/d4e/f64 to db/dd/d96/d4e/fcc 0 2024-03-24T00:48:59.555 INFO:tasks.workunit.client.0.smithi038.stdout:2/577: stat d4/d18/d5b/l74 0 2024-03-24T00:48:59.557 INFO:tasks.workunit.client.1.smithi088.stdout:7/568: mkdir d3/d31/db5 0 2024-03-24T00:48:59.557 INFO:tasks.workunit.client.1.smithi088.stdout:7/569: chown d3/d32/d34/d9c/d4b/d73/f92 24 1 2024-03-24T00:48:59.557 INFO:tasks.workunit.client.1.smithi088.stdout:6/679: dwrite db/d1d/d50/d90/d56/f30 [0,4194304] 0 2024-03-24T00:48:59.557 INFO:tasks.workunit.client.1.smithi088.stdout:6/680: fdatasync db/d1d/d23/d5c/f78 0 2024-03-24T00:48:59.557 INFO:tasks.workunit.client.1.smithi088.stdout:6/681: fsync db/d8e/fbc 0 2024-03-24T00:48:59.562 INFO:tasks.workunit.client.0.smithi038.stdout:8/629: dwrite d0/d45/d37/f87 [0,4194304] 0 2024-03-24T00:48:59.564 INFO:tasks.workunit.client.0.smithi038.stdout:6/677: dread d1/da/d1b/d2f/dc0/fad [0,4194304] 0 2024-03-24T00:48:59.564 INFO:tasks.workunit.client.0.smithi038.stdout:0/570: read d0/d12/d42/d9d/f90 [1524318,36350] 0 2024-03-24T00:48:59.569 INFO:tasks.workunit.client.0.smithi038.stdout:2/578: truncate d4/d18/d26/f3d 375662 0 2024-03-24T00:48:59.569 INFO:tasks.workunit.client.0.smithi038.stdout:9/690: creat d12/fe3 x:0 0 0 2024-03-24T00:48:59.569 INFO:tasks.workunit.client.0.smithi038.stdout:5/596: dread d6/f9 [0,4194304] 0 2024-03-24T00:48:59.569 INFO:tasks.workunit.client.0.smithi038.stdout:5/597: readlink d6/d19/d20/d42/lba 0 2024-03-24T00:48:59.576 INFO:tasks.workunit.client.1.smithi088.stdout:8/662: mknod d9/dc/d11/d24/ce1 0 2024-03-24T00:48:59.577 INFO:tasks.workunit.client.1.smithi088.stdout:6/682: dread db/dd/d61/d6d/f91 [0,4194304] 0 2024-03-24T00:48:59.585 INFO:tasks.workunit.client.1.smithi088.stdout:9/677: dwrite d1/d10/d11/d23/fa0 [0,4194304] 0 2024-03-24T00:48:59.586 INFO:tasks.workunit.client.1.smithi088.stdout:9/678: write d1/d10/d9b/dab/db8/fc2 [1558214,116720] 0 2024-03-24T00:48:59.587 INFO:tasks.workunit.client.0.smithi038.stdout:2/579: write d4/f7a [735588,36813] 0 2024-03-24T00:48:59.588 INFO:tasks.workunit.client.0.smithi038.stdout:0/571: mkdir d0/d14/d88/db8 0 2024-03-24T00:48:59.588 INFO:tasks.workunit.client.1.smithi088.stdout:8/663: dread d9/d6b/f6c [0,4194304] 0 2024-03-24T00:48:59.590 INFO:tasks.workunit.client.1.smithi088.stdout:5/700: getdents d1/dd/d6f/da1/d3f/d88/d8b 0 2024-03-24T00:48:59.590 INFO:tasks.workunit.client.1.smithi088.stdout:5/701: write d1/f3b [3050890,57737] 0 2024-03-24T00:48:59.596 INFO:tasks.workunit.client.0.smithi038.stdout:5/598: mknod d6/d19/d20/d42/d43/d68/cc4 0 2024-03-24T00:48:59.597 INFO:tasks.workunit.client.1.smithi088.stdout:2/634: creat db/dd/d96/d4e/d8f/d9c/db0/fcd x:0 0 0 2024-03-24T00:48:59.598 INFO:tasks.workunit.client.1.smithi088.stdout:2/635: chown db/dd/d96/d4e/d8f 18777 1 2024-03-24T00:48:59.598 INFO:tasks.workunit.client.1.smithi088.stdout:2/636: read db/dd/d96/d4e/fcc [1180997,111417] 0 2024-03-24T00:48:59.598 INFO:tasks.workunit.client.1.smithi088.stdout:2/637: fdatasync f1 0 2024-03-24T00:48:59.603 INFO:tasks.workunit.client.1.smithi088.stdout:7/570: creat d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/fb6 x:0 0 0 2024-03-24T00:48:59.604 INFO:tasks.workunit.client.0.smithi038.stdout:2/580: symlink d4/d18/lc9 0 2024-03-24T00:48:59.606 INFO:tasks.workunit.client.0.smithi038.stdout:5/599: dread d6/d16/d21/f5f [0,4194304] 0 2024-03-24T00:48:59.609 INFO:tasks.workunit.client.0.smithi038.stdout:9/691: rmdir d12/d28/d71/d99 39 2024-03-24T00:48:59.611 INFO:tasks.workunit.client.0.smithi038.stdout:2/581: mkdir d4/dca 0 2024-03-24T00:48:59.621 INFO:tasks.workunit.client.0.smithi038.stdout:9/692: rmdir d12/d81/d84/d88 39 2024-03-24T00:48:59.622 INFO:tasks.workunit.client.1.smithi088.stdout:6/683: symlink db/d1d/d50/d90/d55/lce 0 2024-03-24T00:48:59.622 INFO:tasks.workunit.client.1.smithi088.stdout:1/666: rmdir d6/dc2 39 2024-03-24T00:48:59.625 INFO:tasks.workunit.client.0.smithi038.stdout:9/693: unlink d12/d28/d71/cb2 0 2024-03-24T00:48:59.625 INFO:tasks.workunit.client.0.smithi038.stdout:9/694: write d12/d41/fd0 [622500,89844] 0 2024-03-24T00:48:59.628 INFO:tasks.workunit.client.1.smithi088.stdout:8/664: creat d9/dc/d11/d24/d29/d36/dc1/dc4/fe2 x:0 0 0 2024-03-24T00:48:59.628 INFO:tasks.workunit.client.1.smithi088.stdout:8/665: dread - d9/dc/fc3 zero size 2024-03-24T00:48:59.633 INFO:tasks.workunit.client.0.smithi038.stdout:9/695: unlink d12/d28/d3a/c60 0 2024-03-24T00:48:59.633 INFO:tasks.workunit.client.0.smithi038.stdout:9/696: write d12/d1a/d5f/fd6 [768609,94823] 0 2024-03-24T00:48:59.635 INFO:tasks.workunit.client.1.smithi088.stdout:2/638: creat db/d21/d35/dcb/fce x:0 0 0 2024-03-24T00:48:59.635 INFO:tasks.workunit.client.1.smithi088.stdout:7/571: write d3/f24 [1004518,53465] 0 2024-03-24T00:48:59.638 INFO:tasks.workunit.client.1.smithi088.stdout:2/639: creat db/dd/fcf x:0 0 0 2024-03-24T00:48:59.638 INFO:tasks.workunit.client.1.smithi088.stdout:2/640: read - db/dd/d96/d99/db2/fb4 zero size 2024-03-24T00:48:59.638 INFO:tasks.workunit.client.1.smithi088.stdout:6/684: creat db/dd/d89/fcf x:0 0 0 2024-03-24T00:48:59.640 INFO:tasks.workunit.client.1.smithi088.stdout:1/667: dread f2 [0,4194304] 0 2024-03-24T00:48:59.640 INFO:tasks.workunit.client.1.smithi088.stdout:1/668: fdatasync d6/d23/daf/d95/d5b/d5e/f66 0 2024-03-24T00:48:59.643 INFO:tasks.workunit.client.1.smithi088.stdout:1/669: creat d6/d23/daf/d95/d5b/d80/fd8 x:0 0 0 2024-03-24T00:48:59.643 INFO:tasks.workunit.client.1.smithi088.stdout:1/670: write d6/d23/daf/d95/d5b/d5d/d61/f7e [2109239,100289] 0 2024-03-24T00:48:59.643 INFO:tasks.workunit.client.1.smithi088.stdout:9/679: rmdir d1/d22/d27 39 2024-03-24T00:48:59.643 INFO:tasks.workunit.client.1.smithi088.stdout:8/666: symlink d9/dc/d11/dd2/le3 0 2024-03-24T00:48:59.643 INFO:tasks.workunit.client.1.smithi088.stdout:5/702: truncate d1/d10/d1b/d2f/d57/f71 392849 0 2024-03-24T00:48:59.644 INFO:tasks.workunit.client.0.smithi038.stdout:0/572: dread d0/d12/d42/d44/f93 [0,4194304] 0 2024-03-24T00:48:59.644 INFO:tasks.workunit.client.1.smithi088.stdout:3/628: rename d3/d1a/d1e/d76 to d3/d6/dca 0 2024-03-24T00:48:59.644 INFO:tasks.workunit.client.1.smithi088.stdout:3/629: stat d3/d67 0 2024-03-24T00:48:59.652 INFO:tasks.workunit.client.0.smithi038.stdout:8/630: dwrite d0/d45/d37/d5f/d68/f91 [0,4194304] 0 2024-03-24T00:48:59.652 INFO:tasks.workunit.client.0.smithi038.stdout:6/678: dwrite d1/da/d61/d69/fdb [0,4194304] 0 2024-03-24T00:48:59.658 INFO:tasks.workunit.client.1.smithi088.stdout:9/680: link d1/f7 d1/d15/fdd 0 2024-03-24T00:48:59.658 INFO:tasks.workunit.client.1.smithi088.stdout:9/681: chown d1/d10/d11/d23/fa0 3042 1 2024-03-24T00:48:59.663 INFO:tasks.workunit.client.1.smithi088.stdout:5/703: mknod d1/d10/d1b/d2f/d57/d9e/cec 0 2024-03-24T00:48:59.666 INFO:tasks.workunit.client.0.smithi038.stdout:8/631: read d0/d45/d66/d6d/fbc [1903725,33621] 0 2024-03-24T00:48:59.666 INFO:tasks.workunit.client.0.smithi038.stdout:8/632: chown d0/d1d/d27/d4d/f75 10 1 2024-03-24T00:48:59.667 INFO:tasks.workunit.client.0.smithi038.stdout:2/582: dread d4/d27/f91 [0,4194304] 0 2024-03-24T00:48:59.667 INFO:tasks.workunit.client.0.smithi038.stdout:2/583: stat d4/d18/d38/f5a 0 2024-03-24T00:48:59.668 INFO:tasks.workunit.client.0.smithi038.stdout:5/600: dwrite d6/d23/f53 [8388608,4194304] 0 2024-03-24T00:48:59.676 INFO:tasks.workunit.client.0.smithi038.stdout:6/679: mknod d1/da/d61/da2/cdc 0 2024-03-24T00:48:59.676 INFO:tasks.workunit.client.0.smithi038.stdout:6/680: chown d1/da/d1b/d2f/dc0/fc7 275 1 2024-03-24T00:48:59.676 INFO:tasks.workunit.client.0.smithi038.stdout:6/681: creat d1/da/d1b/d25/d49/d72/fdd x:0 0 0 2024-03-24T00:48:59.685 INFO:tasks.workunit.client.0.smithi038.stdout:2/584: dread d4/d27/f33 [0,4194304] 0 2024-03-24T00:48:59.694 INFO:tasks.workunit.client.0.smithi038.stdout:8/633: mknod d0/d1d/d20/d3c/d9e/da3/cd0 0 2024-03-24T00:48:59.697 INFO:tasks.workunit.client.0.smithi038.stdout:5/601: mknod d6/d1e/d97/db0/cc5 0 2024-03-24T00:48:59.698 INFO:tasks.workunit.client.1.smithi088.stdout:7/572: dwrite d3/d32/d34/d9c/d4b/d73/f93 [0,4194304] 0 2024-03-24T00:48:59.699 INFO:tasks.workunit.client.0.smithi038.stdout:6/682: getdents d1/d37/d84 0 2024-03-24T00:48:59.699 INFO:tasks.workunit.client.0.smithi038.stdout:6/683: dread - d1/d37/d84/fc5 zero size 2024-03-24T00:48:59.699 INFO:tasks.workunit.client.0.smithi038.stdout:6/684: creat d1/da/d61/d69/fde x:0 0 0 2024-03-24T00:48:59.699 INFO:tasks.workunit.client.0.smithi038.stdout:6/685: readlink d1/l41 0 2024-03-24T00:48:59.701 INFO:tasks.workunit.client.1.smithi088.stdout:0/694: sync 2024-03-24T00:48:59.702 INFO:tasks.workunit.client.1.smithi088.stdout:1/671: dwrite d6/fab [0,4194304] 0 2024-03-24T00:48:59.702 INFO:tasks.workunit.client.1.smithi088.stdout:1/672: chown d6/d6e/l9e 180 1 2024-03-24T00:48:59.702 INFO:tasks.workunit.client.1.smithi088.stdout:1/673: chown d6/d27/d67/d88 40 1 2024-03-24T00:48:59.702 INFO:tasks.workunit.client.1.smithi088.stdout:1/674: stat d6/d23/daf/d95/d58/f48 0 2024-03-24T00:48:59.702 INFO:tasks.workunit.client.1.smithi088.stdout:1/675: truncate d6/d23/daf/d95/d5b/d5d/d61/d7d/dc8/fd1 989490 0 2024-03-24T00:48:59.703 INFO:tasks.workunit.client.0.smithi038.stdout:8/634: symlink d0/d45/db8/ld1 0 2024-03-24T00:48:59.710 INFO:tasks.workunit.client.0.smithi038.stdout:9/697: dwrite d12/d28/d71/fbe [0,4194304] 0 2024-03-24T00:48:59.713 INFO:tasks.workunit.client.1.smithi088.stdout:8/667: dwrite d9/dc/d11/d24/d29/d39/fc6 [4194304,4194304] 0 2024-03-24T00:48:59.714 INFO:tasks.workunit.client.1.smithi088.stdout:0/695: creat d9/dc/de/d19/d44/d46/d65/d6f/fdf x:0 0 0 2024-03-24T00:48:59.715 INFO:tasks.workunit.client.1.smithi088.stdout:1/676: link d6/d23/c4e d6/d23/daf/d95/d5b/d93/cd9 0 2024-03-24T00:48:59.715 INFO:tasks.workunit.client.1.smithi088.stdout:4/646: sync 2024-03-24T00:48:59.716 INFO:tasks.workunit.client.1.smithi088.stdout:3/630: write d3/d6/d27/d3c/f47 [3700963,45081] 0 2024-03-24T00:48:59.723 INFO:tasks.workunit.client.1.smithi088.stdout:0/696: mkdir d9/dc/de0 0 2024-03-24T00:48:59.728 INFO:tasks.workunit.client.1.smithi088.stdout:1/677: mknod d6/d23/daf/d95/d5b/cda 0 2024-03-24T00:48:59.729 INFO:tasks.workunit.client.1.smithi088.stdout:4/647: creat d5/d17/d91/fe3 x:0 0 0 2024-03-24T00:48:59.731 INFO:tasks.workunit.client.1.smithi088.stdout:3/631: creat d3/de/d6f/d85/fcb x:0 0 0 2024-03-24T00:48:59.735 INFO:tasks.workunit.client.1.smithi088.stdout:0/697: creat d9/dc/de/d7e/dce/fe1 x:0 0 0 2024-03-24T00:48:59.739 INFO:tasks.workunit.client.0.smithi038.stdout:8/635: dread d0/d45/d66/f97 [0,4194304] 0 2024-03-24T00:48:59.741 INFO:tasks.workunit.client.1.smithi088.stdout:1/678: link d6/d23/daf/f50 d6/d23/daf/d95/d5b/d5d/d61/fdb 0 2024-03-24T00:48:59.741 INFO:tasks.workunit.client.1.smithi088.stdout:1/679: creat d6/d27/d67/d9b/fdc x:0 0 0 2024-03-24T00:48:59.741 INFO:tasks.workunit.client.1.smithi088.stdout:1/680: chown d6/d23/daf/d95/d5b/d80/c96 82330722 1 2024-03-24T00:48:59.764 INFO:tasks.workunit.client.1.smithi088.stdout:9/682: dwrite d1/d10/d11/d23/fa0 [0,4194304] 0 2024-03-24T00:48:59.764 INFO:tasks.workunit.client.1.smithi088.stdout:9/683: dread - d1/d10/d29/d85/f91 zero size 2024-03-24T00:48:59.783 INFO:tasks.workunit.client.1.smithi088.stdout:2/641: dwrite db/f4f [0,4194304] 0 2024-03-24T00:48:59.784 INFO:tasks.workunit.client.0.smithi038.stdout:8/636: dread d0/d1d/d20/f58 [0,4194304] 0 2024-03-24T00:48:59.818 INFO:tasks.workunit.client.1.smithi088.stdout:2/642: mknod db/dd/d96/d4e/cd0 0 2024-03-24T00:48:59.819 INFO:tasks.workunit.client.1.smithi088.stdout:7/573: dwrite d3/d32/d34/d9c/d4b/f99 [0,4194304] 0 2024-03-24T00:48:59.819 INFO:tasks.workunit.client.1.smithi088.stdout:7/574: creat d3/d32/d34/d9c/d4b/d74/d79/d7f/fb7 x:0 0 0 2024-03-24T00:48:59.820 INFO:tasks.workunit.client.1.smithi088.stdout:7/575: chown d3/d32/d34/d9c/d4b/f4c 0 1 2024-03-24T00:48:59.820 INFO:tasks.workunit.client.1.smithi088.stdout:7/576: rename d3/d32/d34 to d3/d32/d34/d9c/d4b/d73/db8 22 2024-03-24T00:48:59.826 INFO:tasks.workunit.client.1.smithi088.stdout:2/643: unlink db/f46 0 2024-03-24T00:48:59.830 INFO:tasks.workunit.client.0.smithi038.stdout:9/698: dwrite d12/d28/d3a/f3f [0,4194304] 0 2024-03-24T00:48:59.830 INFO:tasks.workunit.client.0.smithi038.stdout:9/699: chown d12/d28/d71/d99/fdb 0 1 2024-03-24T00:48:59.831 INFO:tasks.workunit.client.1.smithi088.stdout:8/668: dwrite d9/dc/d11/d24/d29/f51 [0,4194304] 0 2024-03-24T00:48:59.831 INFO:tasks.workunit.client.1.smithi088.stdout:0/698: write d9/dc/de/d19/d44/d46/d65/d2a/f75 [2542449,15637] 0 2024-03-24T00:48:59.832 INFO:tasks.workunit.client.1.smithi088.stdout:7/577: truncate d3/f29 165767 0 2024-03-24T00:48:59.835 INFO:tasks.workunit.client.1.smithi088.stdout:1/681: dwrite d6/d23/daf/d95/f22 [0,4194304] 0 2024-03-24T00:48:59.836 INFO:tasks.workunit.client.0.smithi038.stdout:9/700: rename d12/d28/fc7 to d12/d28/d6c/dbc/dd7/fe4 0 2024-03-24T00:48:59.837 INFO:tasks.workunit.client.0.smithi038.stdout:5/602: dwrite d6/f18 [4194304,4194304] 0 2024-03-24T00:48:59.841 INFO:tasks.workunit.client.1.smithi088.stdout:8/669: rmdir d9/dc/d11/d24/d29 39 2024-03-24T00:48:59.841 INFO:tasks.workunit.client.1.smithi088.stdout:0/699: mkdir d9/dc/de/d19/d44/de2 0 2024-03-24T00:48:59.843 INFO:tasks.workunit.client.1.smithi088.stdout:7/578: symlink d3/d31/d56/lb9 0 2024-03-24T00:48:59.845 INFO:tasks.workunit.client.0.smithi038.stdout:9/701: mknod d12/d28/d71/ce5 0 2024-03-24T00:48:59.853 INFO:tasks.workunit.client.1.smithi088.stdout:1/682: creat d6/d23/daf/d95/fdd x:0 0 0 2024-03-24T00:48:59.853 INFO:tasks.workunit.client.1.smithi088.stdout:1/683: chown d6/d23/daf/d95/d5b/d5d/d61/d7d/da5/cb7 110 1 2024-03-24T00:48:59.854 INFO:tasks.workunit.client.0.smithi038.stdout:9/702: mknod d12/d28/d71/ce6 0 2024-03-24T00:48:59.854 INFO:tasks.workunit.client.0.smithi038.stdout:5/603: creat d6/d19/d20/d42/d43/d84/fc6 x:0 0 0 2024-03-24T00:48:59.855 INFO:tasks.workunit.client.1.smithi088.stdout:8/670: mknod d9/d6b/ce4 0 2024-03-24T00:48:59.856 INFO:tasks.workunit.client.1.smithi088.stdout:9/684: dwrite d1/d22/d72/f79 [0,4194304] 0 2024-03-24T00:48:59.856 INFO:tasks.workunit.client.0.smithi038.stdout:6/686: dwrite d1/da/d61/d79/fb0 [0,4194304] 0 2024-03-24T00:48:59.856 INFO:tasks.workunit.client.0.smithi038.stdout:6/687: dread - d1/da/d61/da2/db1/fb9 zero size 2024-03-24T00:48:59.857 INFO:tasks.workunit.client.0.smithi038.stdout:8/637: dwrite d0/d15/f63 [0,4194304] 0 2024-03-24T00:48:59.858 INFO:tasks.workunit.client.1.smithi088.stdout:0/700: mknod d9/dc/de/d19/d44/de2/ce3 0 2024-03-24T00:48:59.858 INFO:tasks.workunit.client.1.smithi088.stdout:3/632: dwrite d3/f87 [0,4194304] 0 2024-03-24T00:48:59.864 INFO:tasks.workunit.client.1.smithi088.stdout:7/579: rename d3/d31/d56/fb4 to d3/d32/fba 0 2024-03-24T00:48:59.869 INFO:tasks.workunit.client.1.smithi088.stdout:9/685: symlink d1/d10/d77/dc0/lde 0 2024-03-24T00:48:59.870 INFO:tasks.workunit.client.1.smithi088.stdout:2/644: dwrite db/d21/d35/f7d [0,4194304] 0 2024-03-24T00:48:59.874 INFO:tasks.workunit.client.1.smithi088.stdout:6/685: sync 2024-03-24T00:48:59.874 INFO:tasks.workunit.client.1.smithi088.stdout:6/686: write db/d1d/d23/fbb [1275823,57330] 0 2024-03-24T00:48:59.877 INFO:tasks.workunit.client.0.smithi038.stdout:6/688: mkdir d1/da/d1b/d25/d49/d4e/ddf 0 2024-03-24T00:48:59.877 INFO:tasks.workunit.client.0.smithi038.stdout:6/689: dread - d1/da/d61/fd1 zero size 2024-03-24T00:48:59.880 INFO:tasks.workunit.client.0.smithi038.stdout:8/638: mknod d0/d45/d37/d5f/db7/dcc/cd2 0 2024-03-24T00:48:59.881 INFO:tasks.workunit.client.1.smithi088.stdout:7/580: write d3/ff [670045,113049] 0 2024-03-24T00:48:59.881 INFO:tasks.workunit.client.1.smithi088.stdout:7/581: write d3/d32/d34/d9c/d4b/d74/d79/f97 [649486,57049] 0 2024-03-24T00:48:59.882 INFO:tasks.workunit.client.1.smithi088.stdout:9/686: creat d1/d15/d84/fdf x:0 0 0 2024-03-24T00:48:59.883 INFO:tasks.workunit.client.0.smithi038.stdout:8/639: creat d0/d10/fd3 x:0 0 0 2024-03-24T00:48:59.883 INFO:tasks.workunit.client.1.smithi088.stdout:1/684: dread d6/d27/d67/f6a [0,4194304] 0 2024-03-24T00:48:59.885 INFO:tasks.workunit.client.1.smithi088.stdout:2/645: symlink db/dd/d96/d99/dc3/ld1 0 2024-03-24T00:48:59.885 INFO:tasks.workunit.client.1.smithi088.stdout:2/646: read - db/dd/d96/f53 zero size 2024-03-24T00:48:59.885 INFO:tasks.workunit.client.1.smithi088.stdout:2/647: read db/d21/d35/f3d [1654914,84634] 0 2024-03-24T00:48:59.888 INFO:tasks.workunit.client.1.smithi088.stdout:0/701: write d9/dc/f49 [202409,50004] 0 2024-03-24T00:48:59.889 INFO:tasks.workunit.client.1.smithi088.stdout:1/685: link d6/d27/fb3 d6/d23/daf/d95/d5b/d5d/d61/d7d/fde 0 2024-03-24T00:48:59.894 INFO:tasks.workunit.client.0.smithi038.stdout:8/640: creat d0/d1d/d20/d3c/d9e/da3/fd4 x:0 0 0 2024-03-24T00:48:59.894 INFO:tasks.workunit.client.1.smithi088.stdout:8/671: dread d9/dc/f52 [0,4194304] 0 2024-03-24T00:48:59.897 INFO:tasks.workunit.client.1.smithi088.stdout:2/648: unlink db/dd/d96/d99/db2/f6d 0 2024-03-24T00:48:59.900 INFO:tasks.workunit.client.1.smithi088.stdout:6/687: dread db/dd/d61/d6d/f91 [0,4194304] 0 2024-03-24T00:48:59.906 INFO:tasks.workunit.client.1.smithi088.stdout:9/687: read d1/d10/f34 [253499,108558] 0 2024-03-24T00:48:59.906 INFO:tasks.workunit.client.1.smithi088.stdout:7/582: rename d3/d32/d34/d9c/d4b/d74/fac to d3/fbb 0 2024-03-24T00:48:59.907 INFO:tasks.workunit.client.1.smithi088.stdout:1/686: creat d6/d23/daf/fdf x:0 0 0 2024-03-24T00:48:59.907 INFO:tasks.workunit.client.1.smithi088.stdout:7/583: fsync d3/d32/f43 0 2024-03-24T00:48:59.907 INFO:tasks.workunit.client.0.smithi038.stdout:6/690: dread d1/d37/fa5 [0,4194304] 0 2024-03-24T00:48:59.908 INFO:tasks.workunit.client.0.smithi038.stdout:6/691: write d1/da/d1b/fb8 [278521,44437] 0 2024-03-24T00:48:59.908 INFO:tasks.workunit.client.1.smithi088.stdout:7/584: write d3/d31/f7e [302160,75927] 0 2024-03-24T00:48:59.908 INFO:tasks.workunit.client.1.smithi088.stdout:7/585: write d3/f35 [3824781,110119] 0 2024-03-24T00:48:59.908 INFO:tasks.workunit.client.1.smithi088.stdout:7/586: creat d3/d32/d34/d9c/d4b/d74/d79/fbc x:0 0 0 2024-03-24T00:48:59.922 INFO:tasks.workunit.client.0.smithi038.stdout:8/641: read d0/d45/d37/d5f/db7/f4f [4104217,121098] 0 2024-03-24T00:48:59.927 INFO:tasks.workunit.client.0.smithi038.stdout:9/703: dwrite d12/d28/da8/fb1 [0,4194304] 0 2024-03-24T00:48:59.929 INFO:tasks.workunit.client.1.smithi088.stdout:6/688: dread db/d3d/d83/d72/f76 [0,4194304] 0 2024-03-24T00:48:59.929 INFO:tasks.workunit.client.1.smithi088.stdout:6/689: fsync db/dc5/dc8/d43/f28 0 2024-03-24T00:48:59.929 INFO:tasks.workunit.client.1.smithi088.stdout:1/687: read d6/f8 [6591634,70690] 0 2024-03-24T00:48:59.930 INFO:tasks.workunit.client.1.smithi088.stdout:1/688: write d6/d23/daf/d95/d5b/d80/fd4 [264535,55794] 0 2024-03-24T00:48:59.932 INFO:tasks.workunit.client.1.smithi088.stdout:2/649: dread db/f38 [0,4194304] 0 2024-03-24T00:48:59.932 INFO:tasks.workunit.client.1.smithi088.stdout:2/650: truncate db/d21/d35/f60 4491899 0 2024-03-24T00:48:59.942 INFO:tasks.workunit.client.1.smithi088.stdout:2/651: dread db/d21/d35/d49/f71 [0,4194304] 0 2024-03-24T00:48:59.946 INFO:tasks.workunit.client.0.smithi038.stdout:9/704: dread d12/d28/d71/d99/fc9 [0,4194304] 0 2024-03-24T00:48:59.965 INFO:tasks.workunit.client.0.smithi038.stdout:9/705: dread fc [0,4194304] 0 2024-03-24T00:48:59.965 INFO:tasks.workunit.client.0.smithi038.stdout:9/706: creat d12/d28/d6c/dbc/fe7 x:0 0 0 2024-03-24T00:48:59.984 INFO:tasks.workunit.client.1.smithi088.stdout:8/672: mkdir d9/dc/d11/d24/d29/d39/de5 0 2024-03-24T00:48:59.985 INFO:tasks.workunit.client.1.smithi088.stdout:3/633: write d3/f87 [4200396,36969] 0 2024-03-24T00:48:59.986 INFO:tasks.workunit.client.0.smithi038.stdout:6/692: rename d1/d37/l5c to d1/da/d1b/d2f/le0 0 2024-03-24T00:48:59.988 INFO:tasks.workunit.client.1.smithi088.stdout:0/702: rename d9/dc/de/d19/d44/d46/d65/d2a/d2e/fca to d9/dc/de0/fe4 0 2024-03-24T00:48:59.988 INFO:tasks.workunit.client.1.smithi088.stdout:0/703: chown d9/f1c 3853 1 2024-03-24T00:48:59.988 INFO:tasks.workunit.client.0.smithi038.stdout:8/642: mknod d0/d45/d37/d5f/db7/d62/cd5 0 2024-03-24T00:48:59.989 INFO:tasks.workunit.client.0.smithi038.stdout:8/643: read d0/d45/d66/f93 [475773,127824] 0 2024-03-24T00:48:59.990 INFO:tasks.workunit.client.1.smithi088.stdout:4/648: sync 2024-03-24T00:48:59.990 INFO:tasks.workunit.client.1.smithi088.stdout:5/704: sync 2024-03-24T00:48:59.990 INFO:tasks.workunit.client.1.smithi088.stdout:5/705: read - d1/d19/db9/fe4 zero size 2024-03-24T00:48:59.990 INFO:tasks.workunit.client.1.smithi088.stdout:5/706: readlink d1/d10/d1b/lb4 0 2024-03-24T00:48:59.991 INFO:tasks.workunit.client.1.smithi088.stdout:4/649: write d5/d17/f1d [2733987,105575] 0 2024-03-24T00:48:59.991 INFO:tasks.workunit.client.1.smithi088.stdout:7/587: mknod d3/d32/d34/d9c/d4b/d74/d79/d7f/cbd 0 2024-03-24T00:48:59.992 INFO:tasks.workunit.client.0.smithi038.stdout:9/707: dread d12/d28/f4c [0,4194304] 0 2024-03-24T00:48:59.992 INFO:tasks.workunit.client.0.smithi038.stdout:9/708: getdents d12/d81/d84 0 2024-03-24T00:48:59.995 INFO:tasks.workunit.client.1.smithi088.stdout:6/690: link db/dc5/dc8/d43/f28 db/d1d/d50/fd0 0 2024-03-24T00:49:00.000 INFO:tasks.workunit.client.1.smithi088.stdout:1/689: mknod d6/d23/daf/d95/d5b/d5d/d61/d7d/ce0 0 2024-03-24T00:49:00.000 INFO:tasks.workunit.client.1.smithi088.stdout:2/652: mknod db/d21/d35/dcb/cd2 0 2024-03-24T00:49:00.001 INFO:tasks.workunit.client.1.smithi088.stdout:9/688: mkdir d1/d10/d77/d8a/de0 0 2024-03-24T00:49:00.001 INFO:tasks.workunit.client.1.smithi088.stdout:5/707: mkdir d1/d10/d1b/d2f/d57/d9e/db1/ded 0 2024-03-24T00:49:00.002 INFO:tasks.workunit.client.0.smithi038.stdout:6/693: mknod d1/ce1 0 2024-03-24T00:49:00.002 INFO:tasks.workunit.client.0.smithi038.stdout:6/694: creat d1/da/d1b/d25/d49/d72/fe2 x:0 0 0 2024-03-24T00:49:00.004 INFO:tasks.workunit.client.0.smithi038.stdout:8/644: rename d0/d1d/d20/d3c/d9e/da3/caa to d0/d1d/d20/d3c/d9e/da3/cd6 0 2024-03-24T00:49:00.005 INFO:tasks.workunit.client.0.smithi038.stdout:9/709: mknod d12/d28/d9c/ddd/ce8 0 2024-03-24T00:49:00.005 INFO:tasks.workunit.client.0.smithi038.stdout:9/710: chown d12/d28 374387909 1 2024-03-24T00:49:00.008 INFO:tasks.workunit.client.1.smithi088.stdout:4/650: mkdir d5/d7/de4 0 2024-03-24T00:49:00.008 INFO:tasks.workunit.client.1.smithi088.stdout:6/691: truncate db/d1d/d50/d90/f5a 3004027 0 2024-03-24T00:49:00.008 INFO:tasks.workunit.client.1.smithi088.stdout:6/692: chown db/dd/d89 1235 1 2024-03-24T00:49:00.008 INFO:tasks.workunit.client.1.smithi088.stdout:6/693: dread - db/dc5/dc8/d43/fa2 zero size 2024-03-24T00:49:00.008 INFO:tasks.workunit.client.1.smithi088.stdout:1/690: mkdir d6/d23/daf/d95/d5b/d5d/d61/d7d/de1 0 2024-03-24T00:49:00.008 INFO:tasks.workunit.client.1.smithi088.stdout:1/691: write d6/d23/dcb/fd0 [624061,124214] 0 2024-03-24T00:49:00.010 INFO:tasks.workunit.client.0.smithi038.stdout:6/695: creat d1/da/d1b/d1d/d3c/fe3 x:0 0 0 2024-03-24T00:49:00.013 INFO:tasks.workunit.client.1.smithi088.stdout:2/653: link db/dd/d96/d4e/f81 db/dd/d96/d4e/fd3 0 2024-03-24T00:49:00.015 INFO:tasks.workunit.client.0.smithi038.stdout:8/645: truncate d0/d15/f63 1503096 0 2024-03-24T00:49:00.017 INFO:tasks.workunit.client.0.smithi038.stdout:9/711: truncate d12/d41/f58 944622 0 2024-03-24T00:49:00.023 INFO:tasks.workunit.client.0.smithi038.stdout:9/712: creat d12/d28/d6c/dbc/dd7/fe9 x:0 0 0 2024-03-24T00:49:00.023 INFO:tasks.workunit.client.0.smithi038.stdout:9/713: chown d12/d28/d3a/f4e 38 1 2024-03-24T00:49:00.027 INFO:tasks.workunit.client.0.smithi038.stdout:9/714: symlink d12/d28/d6c/dbc/lea 0 2024-03-24T00:49:00.028 INFO:tasks.workunit.client.0.smithi038.stdout:4/593: sync 2024-03-24T00:49:00.028 INFO:tasks.workunit.client.0.smithi038.stdout:7/607: sync 2024-03-24T00:49:00.028 INFO:tasks.workunit.client.0.smithi038.stdout:1/695: sync 2024-03-24T00:49:00.028 INFO:tasks.workunit.client.0.smithi038.stdout:1/696: readlink da/dd/d2c/d41/d60/la1 0 2024-03-24T00:49:00.028 INFO:tasks.workunit.client.0.smithi038.stdout:7/608: fsync d2/f4 0 2024-03-24T00:49:00.028 INFO:tasks.workunit.client.0.smithi038.stdout:3/644: sync 2024-03-24T00:49:00.032 INFO:tasks.workunit.client.0.smithi038.stdout:9/715: mkdir d12/d28/d6c/dbc/deb 0 2024-03-24T00:49:00.033 INFO:tasks.workunit.client.0.smithi038.stdout:4/594: unlink da/d27/fbf 0 2024-03-24T00:49:00.036 INFO:tasks.workunit.client.1.smithi088.stdout:1/692: dread d6/d23/daf/d95/d5b/d5d/fa9 [0,4194304] 0 2024-03-24T00:49:00.036 INFO:tasks.workunit.client.1.smithi088.stdout:1/693: chown d6/d23/daf/d95/d3f/c55 10 1 2024-03-24T00:49:00.040 INFO:tasks.workunit.client.0.smithi038.stdout:1/697: mkdir da/dd/d2c/d40/de9 0 2024-03-24T00:49:00.041 INFO:tasks.workunit.client.1.smithi088.stdout:2/654: write db/dd/f2f [3973395,48653] 0 2024-03-24T00:49:00.041 INFO:tasks.workunit.client.1.smithi088.stdout:7/588: dread d3/d31/f70 [0,4194304] 0 2024-03-24T00:49:00.041 INFO:tasks.workunit.client.1.smithi088.stdout:7/589: chown d3/d32/d34/d9c/c4d 205 1 2024-03-24T00:49:00.041 INFO:tasks.workunit.client.1.smithi088.stdout:2/655: chown db/dd/d96/l7b 12 1 2024-03-24T00:49:00.041 INFO:tasks.workunit.client.1.smithi088.stdout:2/656: chown db/dd/d96/l98 0 1 2024-03-24T00:49:00.042 INFO:tasks.workunit.client.1.smithi088.stdout:2/657: write db/d21/f44 [624383,119047] 0 2024-03-24T00:49:00.047 INFO:tasks.workunit.client.1.smithi088.stdout:1/694: rename d6/d27/c44 to d6/d23/daf/d95/d5b/ce2 0 2024-03-24T00:49:00.047 INFO:tasks.workunit.client.1.smithi088.stdout:1/695: write d6/d23/daf/d95/fdd [297276,113571] 0 2024-03-24T00:49:00.050 INFO:tasks.workunit.client.1.smithi088.stdout:4/651: dread d5/d13/d18/d24/d3d/f77 [0,4194304] 0 2024-03-24T00:49:00.050 INFO:tasks.workunit.client.1.smithi088.stdout:4/652: fdatasync d5/d13/d18/d24/f30 0 2024-03-24T00:49:00.053 INFO:tasks.workunit.client.0.smithi038.stdout:3/645: rmdir d3/d9/dc 39 2024-03-24T00:49:00.059 INFO:tasks.workunit.client.1.smithi088.stdout:8/673: write d9/dc/d11/d24/f5e [1009337,121243] 0 2024-03-24T00:49:00.063 INFO:tasks.workunit.client.1.smithi088.stdout:0/704: dread d9/f55 [0,4194304] 0 2024-03-24T00:49:00.067 INFO:tasks.workunit.client.0.smithi038.stdout:4/595: symlink da/d12/d37/d83/dac/lcb 0 2024-03-24T00:49:00.067 INFO:tasks.workunit.client.0.smithi038.stdout:4/596: chown da/d12/d37/dd/l20 5189290 1 2024-03-24T00:49:00.070 INFO:tasks.workunit.client.1.smithi088.stdout:8/674: write d9/dc/d11/d24/d29/d36/f88 [202559,38789] 0 2024-03-24T00:49:00.070 INFO:tasks.workunit.client.1.smithi088.stdout:8/675: readlink d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/lb5 0 2024-03-24T00:49:00.070 INFO:tasks.workunit.client.1.smithi088.stdout:8/676: chown d9/dc/d11/l1d 47895771 1 2024-03-24T00:49:00.072 INFO:tasks.workunit.client.1.smithi088.stdout:1/696: mkdir d6/d27/db0/de3 0 2024-03-24T00:49:00.073 INFO:tasks.workunit.client.1.smithi088.stdout:1/697: chown d6/d23/daf/d95/d5b/d5d/d61/d7d/fd3 3266 1 2024-03-24T00:49:00.078 INFO:tasks.workunit.client.1.smithi088.stdout:4/653: chown d5/d7/dd/c54 5534 1 2024-03-24T00:49:00.078 INFO:tasks.workunit.client.1.smithi088.stdout:4/654: write d5/d13/f86 [502194,61468] 0 2024-03-24T00:49:00.078 INFO:tasks.workunit.client.0.smithi038.stdout:1/698: mknod da/dd/d2c/db6/dc4/cea 0 2024-03-24T00:49:00.082 INFO:tasks.workunit.client.0.smithi038.stdout:7/609: rmdir d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab 39 2024-03-24T00:49:00.085 INFO:tasks.workunit.client.0.smithi038.stdout:1/699: read da/dd/d4a/f6b [329263,69298] 0 2024-03-24T00:49:00.085 INFO:tasks.workunit.client.0.smithi038.stdout:3/646: mknod d3/d2a/cd8 0 2024-03-24T00:49:00.091 INFO:tasks.workunit.client.1.smithi088.stdout:9/689: dwrite d1/d10/d29/d6d/f57 [0,4194304] 0 2024-03-24T00:49:00.093 INFO:tasks.workunit.client.1.smithi088.stdout:7/590: rmdir d3/d31/d56 39 2024-03-24T00:49:00.098 INFO:tasks.workunit.client.0.smithi038.stdout:3/647: dread d3/d5a/fa5 [0,4194304] 0 2024-03-24T00:49:00.098 INFO:tasks.workunit.client.0.smithi038.stdout:3/648: stat d3/f3d 0 2024-03-24T00:49:00.101 INFO:tasks.workunit.client.0.smithi038.stdout:1/700: dread da/dd/d2c/d41/d60/fc5 [0,4194304] 0 2024-03-24T00:49:00.101 INFO:tasks.workunit.client.0.smithi038.stdout:1/701: write da/dd/d4a/d76/f36 [1493399,63822] 0 2024-03-24T00:49:00.105 INFO:tasks.workunit.client.0.smithi038.stdout:4/597: symlink da/d12/d24/d5f/lcc 0 2024-03-24T00:49:00.105 INFO:tasks.workunit.client.1.smithi088.stdout:7/591: dread d3/f24 [0,4194304] 0 2024-03-24T00:49:00.105 INFO:tasks.workunit.client.1.smithi088.stdout:7/592: fdatasync d3/d31/f9e 0 2024-03-24T00:49:00.108 INFO:tasks.workunit.client.1.smithi088.stdout:5/708: dwrite d1/d10/d1b/d9d/fa3 [0,4194304] 0 2024-03-24T00:49:00.108 INFO:tasks.workunit.client.1.smithi088.stdout:5/709: fsync d1/d10/d1b/d2f/fe7 0 2024-03-24T00:49:00.108 INFO:tasks.workunit.client.1.smithi088.stdout:3/634: dwrite d3/d6/d27/f8d [4194304,4194304] 0 2024-03-24T00:49:00.112 INFO:tasks.workunit.client.0.smithi038.stdout:7/610: rename d2/d9/da/d23/d2b/d5b/d72/d92/d5d/cba to d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/ce8 0 2024-03-24T00:49:00.116 INFO:tasks.workunit.client.1.smithi088.stdout:0/705: getdents d9/dc/de/d19/d44 0 2024-03-24T00:49:00.117 INFO:tasks.workunit.client.0.smithi038.stdout:3/649: rmdir d3/d9/d77 39 2024-03-24T00:49:00.119 INFO:tasks.workunit.client.1.smithi088.stdout:4/655: dread f2 [0,4194304] 0 2024-03-24T00:49:00.120 INFO:tasks.workunit.client.0.smithi038.stdout:9/716: dwrite d12/d55/f9b [4194304,4194304] 0 2024-03-24T00:49:00.120 INFO:tasks.workunit.client.0.smithi038.stdout:9/717: stat d12/d28/d3a/fe0 0 2024-03-24T00:49:00.120 INFO:tasks.workunit.client.0.smithi038.stdout:9/718: chown d12/d28/d3a/d26/c7b 36 1 2024-03-24T00:49:00.121 INFO:tasks.workunit.client.1.smithi088.stdout:6/694: dwrite db/dd/d61/fc9 [0,4194304] 0 2024-03-24T00:49:00.121 INFO:tasks.workunit.client.1.smithi088.stdout:6/695: getdents db/d1d/d23/d5c 0 2024-03-24T00:49:00.121 INFO:tasks.workunit.client.1.smithi088.stdout:6/696: getdents db/dc5/dc8/d43/dbf 0 2024-03-24T00:49:00.122 INFO:tasks.workunit.client.0.smithi038.stdout:8/646: dwrite d0/d1d/d20/d79/fcd [0,4194304] 0 2024-03-24T00:49:00.123 INFO:tasks.workunit.client.1.smithi088.stdout:2/658: dwrite db/dd/d96/d4e/d8f/d9c/db0/fcd [0,4194304] 0 2024-03-24T00:49:00.123 INFO:tasks.workunit.client.1.smithi088.stdout:2/659: chown db/dd 582 1 2024-03-24T00:49:00.123 INFO:tasks.workunit.client.1.smithi088.stdout:2/660: write db/d21/d35/f79 [2535449,5649] 0 2024-03-24T00:49:00.124 INFO:tasks.workunit.client.0.smithi038.stdout:6/696: dwrite d1/da/d1b/d25/d49/d4e/f95 [0,4194304] 0 2024-03-24T00:49:00.124 INFO:tasks.workunit.client.0.smithi038.stdout:4/598: rmdir da/d12/d24/d68/d6c/da6 39 2024-03-24T00:49:00.124 INFO:tasks.workunit.client.0.smithi038.stdout:6/697: write d1/da/d1b/fa8 [584147,41203] 0 2024-03-24T00:49:00.125 INFO:tasks.workunit.client.1.smithi088.stdout:1/698: symlink d6/d23/daf/d95/d5b/d5d/d61/d7d/le4 0 2024-03-24T00:49:00.130 INFO:tasks.workunit.client.0.smithi038.stdout:1/702: rename da/dd/d7a/f91 to da/dd/d2c/d41/d60/feb 0 2024-03-24T00:49:00.130 INFO:tasks.workunit.client.0.smithi038.stdout:1/703: fdatasync da/dd/d7a/d82/d98/fba 0 2024-03-24T00:49:00.131 INFO:tasks.workunit.client.0.smithi038.stdout:4/599: dread da/d12/d24/d5f/d8a/da4/faa [0,4194304] 0 2024-03-24T00:49:00.131 INFO:tasks.workunit.client.0.smithi038.stdout:4/600: write da/d12/d29/d5b/d73/db4/fb7 [655538,126634] 0 2024-03-24T00:49:00.132 INFO:tasks.workunit.client.1.smithi088.stdout:9/690: creat d1/d22/d27/d39/d98/da3/fe1 x:0 0 0 2024-03-24T00:49:00.137 INFO:tasks.workunit.client.0.smithi038.stdout:9/719: mkdir d12/d81/d84/dec 0 2024-03-24T00:49:00.142 INFO:tasks.workunit.client.0.smithi038.stdout:9/720: dread d12/d41/d57/f89 [4194304,4194304] 0 2024-03-24T00:49:00.151 INFO:tasks.workunit.client.0.smithi038.stdout:8/647: truncate d0/d45/d66/da6/fae 3363770 0 2024-03-24T00:49:00.153 INFO:tasks.workunit.client.1.smithi088.stdout:5/710: creat d1/d43/d4e/d5b/d66/fee x:0 0 0 2024-03-24T00:49:00.153 INFO:tasks.workunit.client.0.smithi038.stdout:9/721: dread d12/f35 [0,4194304] 0 2024-03-24T00:49:00.156 INFO:tasks.workunit.client.1.smithi088.stdout:7/593: dread d3/f28 [0,4194304] 0 2024-03-24T00:49:00.156 INFO:tasks.workunit.client.1.smithi088.stdout:3/635: mkdir d3/de/d6f/dcc 0 2024-03-24T00:49:00.158 INFO:tasks.workunit.client.1.smithi088.stdout:0/706: mknod d9/d15/ce5 0 2024-03-24T00:49:00.165 INFO:tasks.workunit.client.1.smithi088.stdout:4/656: mkdir d5/d13/d18/d24/de5 0 2024-03-24T00:49:00.168 INFO:tasks.workunit.client.1.smithi088.stdout:6/697: creat db/dc5/dc8/fd1 x:0 0 0 2024-03-24T00:49:00.172 INFO:tasks.workunit.client.1.smithi088.stdout:5/711: dread d1/dd/d6f/da1/d3f/d88/d8f/d69/f8c [0,4194304] 0 2024-03-24T00:49:00.174 INFO:tasks.workunit.client.0.smithi038.stdout:6/698: rmdir d1/da/d1b/d25/d49/d4e 39 2024-03-24T00:49:00.175 INFO:tasks.workunit.client.0.smithi038.stdout:9/722: write d12/d55/f67 [2812805,101172] 0 2024-03-24T00:49:00.175 INFO:tasks.workunit.client.0.smithi038.stdout:6/699: dread - d1/da/d1b/d25/d49/fcc zero size 2024-03-24T00:49:00.175 INFO:tasks.workunit.client.0.smithi038.stdout:6/700: write d1/da/d1b/f26 [2930482,77515] 0 2024-03-24T00:49:00.175 INFO:tasks.workunit.client.0.smithi038.stdout:6/701: chown d1/da/d1b/d25/d49/l6f 1956959 1 2024-03-24T00:49:00.178 INFO:tasks.workunit.client.1.smithi088.stdout:8/677: symlink d9/dc/d11/le6 0 2024-03-24T00:49:00.181 INFO:tasks.workunit.client.0.smithi038.stdout:1/704: mknod da/dd/cec 0 2024-03-24T00:49:00.181 INFO:tasks.workunit.client.0.smithi038.stdout:1/705: write da/f35 [2343146,4014] 0 2024-03-24T00:49:00.181 INFO:tasks.workunit.client.0.smithi038.stdout:1/706: write da/dd/d7a/d82/d98/d14/f2e [852544,80230] 0 2024-03-24T00:49:00.181 INFO:tasks.workunit.client.0.smithi038.stdout:1/707: fsync f4 0 2024-03-24T00:49:00.183 INFO:tasks.workunit.client.1.smithi088.stdout:5/712: dread d1/d19/f6c [4194304,4194304] 0 2024-03-24T00:49:00.183 INFO:tasks.workunit.client.1.smithi088.stdout:5/713: chown d1/d10/d1b/la8 122 1 2024-03-24T00:49:00.184 INFO:tasks.workunit.client.0.smithi038.stdout:2/585: sync 2024-03-24T00:49:00.184 INFO:tasks.workunit.client.0.smithi038.stdout:2/586: readlink d4/d18/d26/d37/d69/d83/l98 0 2024-03-24T00:49:00.184 INFO:tasks.workunit.client.0.smithi038.stdout:2/587: readlink d4/d18/lc9 0 2024-03-24T00:49:00.184 INFO:tasks.workunit.client.0.smithi038.stdout:0/573: sync 2024-03-24T00:49:00.184 INFO:tasks.workunit.client.0.smithi038.stdout:5/604: sync 2024-03-24T00:49:00.191 INFO:tasks.workunit.client.0.smithi038.stdout:4/601: rename da/d12/d37/dd/d15/d6d/d8f/c96 to da/d12/d24/d68/d6c/d7c/ccd 0 2024-03-24T00:49:00.191 INFO:tasks.workunit.client.0.smithi038.stdout:4/602: dread - da/d38/f63 zero size 2024-03-24T00:49:00.191 INFO:tasks.workunit.client.0.smithi038.stdout:4/603: fsync da/d12/d29/fb1 0 2024-03-24T00:49:00.194 INFO:tasks.workunit.client.1.smithi088.stdout:2/661: mkdir db/dd4 0 2024-03-24T00:49:00.195 INFO:tasks.workunit.client.0.smithi038.stdout:8/648: unlink d0/d45/d66/d6d/f9b 0 2024-03-24T00:49:00.195 INFO:tasks.workunit.client.0.smithi038.stdout:8/649: creat d0/d1d/d27/d4d/fd7 x:0 0 0 2024-03-24T00:49:00.197 INFO:tasks.workunit.client.1.smithi088.stdout:1/699: stat d6/d23/daf/d95/d5b/d93/cd9 0 2024-03-24T00:49:00.197 INFO:tasks.workunit.client.1.smithi088.stdout:1/700: write d6/d23/daf/fdf [675865,77141] 0 2024-03-24T00:49:00.198 INFO:tasks.workunit.client.1.smithi088.stdout:5/714: dread d1/d43/d4e/d5b/fc8 [0,4194304] 0 2024-03-24T00:49:00.201 INFO:tasks.workunit.client.1.smithi088.stdout:3/636: creat d3/d1a/d1e/d2f/fcd x:0 0 0 2024-03-24T00:49:00.201 INFO:tasks.workunit.client.1.smithi088.stdout:3/637: creat d3/d32/fce x:0 0 0 2024-03-24T00:49:00.202 INFO:tasks.workunit.client.0.smithi038.stdout:9/723: mknod d12/d28/d6c/dbc/deb/ced 0 2024-03-24T00:49:00.202 INFO:tasks.workunit.client.0.smithi038.stdout:6/702: symlink d1/d37/le4 0 2024-03-24T00:49:00.202 INFO:tasks.workunit.client.0.smithi038.stdout:6/703: dread - d1/da/d61/da2/fac zero size 2024-03-24T00:49:00.204 INFO:tasks.workunit.client.1.smithi088.stdout:8/678: dread d9/dc/d11/d24/d29/d5b/d8b/f96 [0,4194304] 0 2024-03-24T00:49:00.204 INFO:tasks.workunit.client.1.smithi088.stdout:0/707: rmdir d9/dc/de/d19/d8d/dde 0 2024-03-24T00:49:00.208 INFO:tasks.workunit.client.1.smithi088.stdout:9/691: getdents d1/d22 0 2024-03-24T00:49:00.211 INFO:tasks.workunit.client.1.smithi088.stdout:1/701: symlink d6/d23/daf/d95/d5b/d5d/d61/le5 0 2024-03-24T00:49:00.212 INFO:tasks.workunit.client.0.smithi038.stdout:2/588: symlink d4/d55/lcb 0 2024-03-24T00:49:00.213 INFO:tasks.workunit.client.0.smithi038.stdout:2/589: fdatasync d4/d27/f39 0 2024-03-24T00:49:00.218 INFO:tasks.workunit.client.1.smithi088.stdout:7/594: rename d3/fd to d3/d32/d34/d9c/d4b/d73/fbe 0 2024-03-24T00:49:00.218 INFO:tasks.workunit.client.1.smithi088.stdout:9/692: read d1/d10/d77/fb4 [369428,61170] 0 2024-03-24T00:49:00.220 INFO:tasks.workunit.client.0.smithi038.stdout:5/605: mknod d6/d19/d20/cc7 0 2024-03-24T00:49:00.220 INFO:tasks.workunit.client.0.smithi038.stdout:3/650: dwrite d3/d9/dc/da6/db5/f53 [0,4194304] 0 2024-03-24T00:49:00.221 INFO:tasks.workunit.client.0.smithi038.stdout:3/651: fdatasync d3/d5a/fa5 0 2024-03-24T00:49:00.221 INFO:tasks.workunit.client.0.smithi038.stdout:1/708: mknod da/dd/d4a/d8b/ced 0 2024-03-24T00:49:00.221 INFO:tasks.workunit.client.0.smithi038.stdout:0/574: rename d0/d12/d42/l76 to d0/d14/d88/db8/lb9 0 2024-03-24T00:49:00.222 INFO:tasks.workunit.client.1.smithi088.stdout:5/715: symlink d1/d10/d1b/d2f/d57/d78/lef 0 2024-03-24T00:49:00.222 INFO:tasks.workunit.client.1.smithi088.stdout:5/716: chown d1/d10/d1b/d2f 107558 1 2024-03-24T00:49:00.224 INFO:tasks.workunit.client.1.smithi088.stdout:6/698: dread db/d1d/d50/d90/d56/f51 [0,4194304] 0 2024-03-24T00:49:00.224 INFO:tasks.workunit.client.1.smithi088.stdout:6/699: chown db/d1d/d23/fa5 502339187 1 2024-03-24T00:49:00.233 INFO:tasks.workunit.client.1.smithi088.stdout:3/638: symlink d3/d6/d27/lcf 0 2024-03-24T00:49:00.233 INFO:tasks.workunit.client.1.smithi088.stdout:8/679: write d9/dc/d11/d24/d29/d39/f93 [2432116,149] 0 2024-03-24T00:49:00.233 INFO:tasks.workunit.client.1.smithi088.stdout:3/639: fdatasync d3/d6/d58/fb6 0 2024-03-24T00:49:00.233 INFO:tasks.workunit.client.1.smithi088.stdout:8/680: write d9/dc/d11/d24/f2a [4282831,61397] 0 2024-03-24T00:49:00.239 INFO:tasks.workunit.client.0.smithi038.stdout:8/650: write d0/f32 [1365647,106018] 0 2024-03-24T00:49:00.239 INFO:tasks.workunit.client.0.smithi038.stdout:8/651: truncate d0/d15/f63 1925281 0 2024-03-24T00:49:00.239 INFO:tasks.workunit.client.0.smithi038.stdout:6/704: link d1/d37/d84/fa4 d1/d37/d84/fe5 0 2024-03-24T00:49:00.240 INFO:tasks.workunit.client.1.smithi088.stdout:0/708: unlink d9/d26/d8f/da1/d62/l73 0 2024-03-24T00:49:00.243 INFO:tasks.workunit.client.1.smithi088.stdout:4/657: dwrite d5/d16/f2e [0,4194304] 0 2024-03-24T00:49:00.246 INFO:tasks.workunit.client.1.smithi088.stdout:1/702: mknod d6/d23/daf/ce6 0 2024-03-24T00:49:00.246 INFO:tasks.workunit.client.1.smithi088.stdout:9/693: creat d1/d10/d29/fe2 x:0 0 0 2024-03-24T00:49:00.246 INFO:tasks.workunit.client.1.smithi088.stdout:9/694: fdatasync d1/d15/fdd 0 2024-03-24T00:49:00.250 INFO:tasks.workunit.client.1.smithi088.stdout:5/717: symlink d1/d10/d1b/d2f/d57/d9e/dba/lf0 0 2024-03-24T00:49:00.255 INFO:tasks.workunit.client.1.smithi088.stdout:6/700: creat db/d1d/d50/d90/d55/d99/fd2 x:0 0 0 2024-03-24T00:49:00.256 INFO:tasks.workunit.client.0.smithi038.stdout:1/709: dread da/f72 [0,4194304] 0 2024-03-24T00:49:00.256 INFO:tasks.workunit.client.0.smithi038.stdout:3/652: creat d3/d9/d77/dd4/fd9 x:0 0 0 2024-03-24T00:49:00.257 INFO:tasks.workunit.client.0.smithi038.stdout:0/575: creat d0/d12/fba x:0 0 0 2024-03-24T00:49:00.257 INFO:tasks.workunit.client.1.smithi088.stdout:3/640: creat d3/d1a/fd0 x:0 0 0 2024-03-24T00:49:00.257 INFO:tasks.workunit.client.0.smithi038.stdout:9/724: link d12/d1a/d80/f83 d12/d81/fee 0 2024-03-24T00:49:00.259 INFO:tasks.workunit.client.1.smithi088.stdout:4/658: dread d5/d13/d18/d24/f9c [0,4194304] 0 2024-03-24T00:49:00.259 INFO:tasks.workunit.client.1.smithi088.stdout:4/659: creat d5/d13/d18/d24/d3d/fe6 x:0 0 0 2024-03-24T00:49:00.261 INFO:tasks.workunit.client.0.smithi038.stdout:5/606: read d6/d19/d20/d42/d43/d4c/f74 [1927734,67922] 0 2024-03-24T00:49:00.261 INFO:tasks.workunit.client.0.smithi038.stdout:5/607: chown d6/d16/d3c 18 1 2024-03-24T00:49:00.262 INFO:tasks.workunit.client.0.smithi038.stdout:6/705: mkdir d1/da/de6 0 2024-03-24T00:49:00.267 INFO:tasks.workunit.client.0.smithi038.stdout:1/710: creat da/dd/d7a/d82/ddb/fee x:0 0 0 2024-03-24T00:49:00.272 INFO:tasks.workunit.client.0.smithi038.stdout:3/653: mknod d3/d9/dc/da6/db5/d19/dd2/cda 0 2024-03-24T00:49:00.272 INFO:tasks.workunit.client.0.smithi038.stdout:4/604: dread da/d38/f61 [0,4194304] 0 2024-03-24T00:49:00.273 INFO:tasks.workunit.client.0.smithi038.stdout:0/576: mkdir d0/d12/d15/d58/d64/da7/dbb 0 2024-03-24T00:49:00.273 INFO:tasks.workunit.client.0.smithi038.stdout:0/577: read - d0/d1b/d2c/f6d zero size 2024-03-24T00:49:00.273 INFO:tasks.workunit.client.1.smithi088.stdout:8/681: mkdir d9/dc/d11/d24/d29/da9/de7 0 2024-03-24T00:49:00.273 INFO:tasks.workunit.client.1.smithi088.stdout:7/595: mkdir d3/dbf 0 2024-03-24T00:49:00.276 INFO:tasks.workunit.client.0.smithi038.stdout:8/652: rename d0/d1d/c5e to d0/d45/d37/d5f/db7/d62/cd8 0 2024-03-24T00:49:00.276 INFO:tasks.workunit.client.0.smithi038.stdout:8/653: chown d0/d1d/d27/d4d/fd7 3033215 1 2024-03-24T00:49:00.277 INFO:tasks.workunit.client.0.smithi038.stdout:6/706: link d1/da/d1b/d1d/d3c/l71 d1/da/le7 0 2024-03-24T00:49:00.277 INFO:tasks.workunit.client.0.smithi038.stdout:0/578: link d0/c5b d0/d12/cbc 0 2024-03-24T00:49:00.277 INFO:tasks.workunit.client.1.smithi088.stdout:4/660: rmdir d5/d7/d15 39 2024-03-24T00:49:00.278 INFO:tasks.workunit.client.0.smithi038.stdout:6/707: creat d1/fe8 x:0 0 0 2024-03-24T00:49:00.278 INFO:tasks.workunit.client.0.smithi038.stdout:1/711: mkdir da/d3c/d86/def 0 2024-03-24T00:49:00.278 INFO:tasks.workunit.client.0.smithi038.stdout:1/712: write da/f23 [2987206,21254] 0 2024-03-24T00:49:00.278 INFO:tasks.workunit.client.0.smithi038.stdout:9/725: dread d12/d41/fd0 [0,4194304] 0 2024-03-24T00:49:00.280 INFO:tasks.workunit.client.1.smithi088.stdout:8/682: truncate d9/dc/d11/d24/d29/d5b/f34 6480218 0 2024-03-24T00:49:00.280 INFO:tasks.workunit.client.1.smithi088.stdout:8/683: write d9/d6b/f6c [1780331,4403] 0 2024-03-24T00:49:00.280 INFO:tasks.workunit.client.1.smithi088.stdout:8/684: write d9/dc/d11/d24/f2a [593547,71088] 0 2024-03-24T00:49:00.281 INFO:tasks.workunit.client.0.smithi038.stdout:0/579: mknod d0/d12/d42/d83/cbd 0 2024-03-24T00:49:00.281 INFO:tasks.workunit.client.0.smithi038.stdout:0/580: creat d0/d2d/fbe x:0 0 0 2024-03-24T00:49:00.282 INFO:tasks.workunit.client.0.smithi038.stdout:0/581: chown d0/d12/d42/d9d/d28/l7b 3 1 2024-03-24T00:49:00.284 INFO:tasks.workunit.client.1.smithi088.stdout:7/596: mknod d3/d31/cc0 0 2024-03-24T00:49:00.284 INFO:tasks.workunit.client.1.smithi088.stdout:7/597: dread - d3/d31/d56/f7b zero size 2024-03-24T00:49:00.284 INFO:tasks.workunit.client.1.smithi088.stdout:7/598: read - d3/d32/d34/faa zero size 2024-03-24T00:49:00.286 INFO:tasks.workunit.client.0.smithi038.stdout:6/708: creat d1/da/d1b/d25/d49/d4e/ddf/fe9 x:0 0 0 2024-03-24T00:49:00.286 INFO:tasks.workunit.client.0.smithi038.stdout:6/709: fdatasync d1/da/f89 0 2024-03-24T00:49:00.286 INFO:tasks.workunit.client.0.smithi038.stdout:4/605: dread da/d12/d29/d5b/d73/fb2 [0,4194304] 0 2024-03-24T00:49:00.286 INFO:tasks.workunit.client.1.smithi088.stdout:5/718: truncate d1/d10/d1b/f70 1886726 0 2024-03-24T00:49:00.287 INFO:tasks.workunit.client.0.smithi038.stdout:5/608: write d6/d1e/f9f [165535,20920] 0 2024-03-24T00:49:00.291 INFO:tasks.workunit.client.0.smithi038.stdout:1/713: symlink da/lf0 0 2024-03-24T00:49:00.292 INFO:tasks.workunit.client.0.smithi038.stdout:1/714: fsync da/dd/d7a/d82/d98/f1f 0 2024-03-24T00:49:00.293 INFO:tasks.workunit.client.0.smithi038.stdout:0/582: rename d0/d1b/d2c/f6d to d0/d14/d88/db8/fbf 0 2024-03-24T00:49:00.293 INFO:tasks.workunit.client.0.smithi038.stdout:0/583: creat d0/d12/d15/d1a/fc0 x:0 0 0 2024-03-24T00:49:00.298 INFO:tasks.workunit.client.0.smithi038.stdout:5/609: mkdir d6/d19/d20/d42/d43/d68/dc8 0 2024-03-24T00:49:00.301 INFO:tasks.workunit.client.0.smithi038.stdout:1/715: mknod da/dd/d22/d5c/de8/cf1 0 2024-03-24T00:49:00.302 INFO:tasks.workunit.client.0.smithi038.stdout:4/606: mkdir da/d12/d24/d68/d6c/dce 0 2024-03-24T00:49:00.302 INFO:tasks.workunit.client.0.smithi038.stdout:4/607: fdatasync da/d38/f3c 0 2024-03-24T00:49:00.306 INFO:tasks.workunit.client.1.smithi088.stdout:5/719: mkdir d1/d10/d1b/d2f/d57/d9e/dba/df1 0 2024-03-24T00:49:00.306 INFO:tasks.workunit.client.1.smithi088.stdout:5/720: write d1/d10/d1b/d2e/d9f/ddf/f80 [876939,127362] 0 2024-03-24T00:49:00.306 INFO:tasks.workunit.client.1.smithi088.stdout:5/721: write d1/d19/d1e/f9b [329194,82661] 0 2024-03-24T00:49:00.311 INFO:tasks.workunit.client.1.smithi088.stdout:2/662: dwrite db/d21/d35/d49/fac [0,4194304] 0 2024-03-24T00:49:00.312 INFO:tasks.workunit.client.0.smithi038.stdout:3/654: dread d3/d9/d77/f85 [4194304,4194304] 0 2024-03-24T00:49:00.314 INFO:tasks.workunit.client.1.smithi088.stdout:7/599: write d3/f27 [673659,99319] 0 2024-03-24T00:49:00.316 INFO:tasks.workunit.client.1.smithi088.stdout:1/703: read d6/d27/f3a [189764,118350] 0 2024-03-24T00:49:00.319 INFO:tasks.workunit.client.1.smithi088.stdout:5/722: rename d1/dd/d6f/da1/d3f/d88/d8b/fd2 to d1/d10/d1b/d2f/d57/d9e/dba/ff2 0 2024-03-24T00:49:00.319 INFO:tasks.workunit.client.0.smithi038.stdout:8/654: dread d0/d45/d37/f61 [4194304,4194304] 0 2024-03-24T00:49:00.323 INFO:tasks.workunit.client.0.smithi038.stdout:4/608: mkdir da/d12/d37/d83/dac/dcf 0 2024-03-24T00:49:00.324 INFO:tasks.workunit.client.0.smithi038.stdout:1/716: link da/d3c/l1a da/dd/d2c/d41/d60/dae/lf2 0 2024-03-24T00:49:00.325 INFO:tasks.workunit.client.1.smithi088.stdout:1/704: mkdir d6/d23/dcb/de7 0 2024-03-24T00:49:00.325 INFO:tasks.workunit.client.1.smithi088.stdout:1/705: write d6/d23/daf/d95/d5b/d5d/d61/d7d/da5/fc0 [17240,78483] 0 2024-03-24T00:49:00.325 INFO:tasks.workunit.client.1.smithi088.stdout:1/706: read - d6/d6e/f79 zero size 2024-03-24T00:49:00.325 INFO:tasks.workunit.client.1.smithi088.stdout:1/707: chown d6/d23/daf/d95/d5b/d5d/d61/d7d/dc8 621443 1 2024-03-24T00:49:00.332 INFO:tasks.workunit.client.0.smithi038.stdout:2/590: dwrite d4/d27/d2f/d89/f9a [0,4194304] 0 2024-03-24T00:49:00.332 INFO:tasks.workunit.client.0.smithi038.stdout:3/655: mknod d3/d30/d5d/cdb 0 2024-03-24T00:49:00.333 INFO:tasks.workunit.client.1.smithi088.stdout:5/723: rmdir d1/dd/d6f/d98 39 2024-03-24T00:49:00.339 INFO:tasks.workunit.client.0.smithi038.stdout:5/610: symlink d6/lc9 0 2024-03-24T00:49:00.341 INFO:tasks.workunit.client.1.smithi088.stdout:9/695: dwrite d1/d10/d29/f2e [0,4194304] 0 2024-03-24T00:49:00.341 INFO:tasks.workunit.client.1.smithi088.stdout:9/696: chown d1/d10/d29/f2e 4856762 1 2024-03-24T00:49:00.354 INFO:tasks.workunit.client.0.smithi038.stdout:4/609: mknod da/d12/d24/d5f/d8a/da4/cd0 0 2024-03-24T00:49:00.370 INFO:tasks.workunit.client.0.smithi038.stdout:9/726: dwrite d12/d28/d71/d99/fc6 [0,4194304] 0 2024-03-24T00:49:00.397 INFO:tasks.workunit.client.1.smithi088.stdout:6/701: dwrite db/dd/d61/d6d/d74/fc0 [0,4194304] 0 2024-03-24T00:49:00.398 INFO:tasks.workunit.client.0.smithi038.stdout:2/591: symlink d4/d18/d26/d37/lcc 0 2024-03-24T00:49:00.399 INFO:tasks.workunit.client.1.smithi088.stdout:0/709: write d9/d26/d8f/da1/d62/d9b/db0/fb2 [126877,15500] 0 2024-03-24T00:49:00.403 INFO:tasks.workunit.client.1.smithi088.stdout:1/708: dwrite d6/d23/daf/d95/fdd [0,4194304] 0 2024-03-24T00:49:00.404 INFO:tasks.workunit.client.0.smithi038.stdout:8/655: dwrite d0/d1d/d27/d4d/f7b [0,4194304] 0 2024-03-24T00:49:00.405 INFO:tasks.workunit.client.0.smithi038.stdout:1/717: dwrite da/dd/d7a/d82/d98/d9e/fb0 [0,4194304] 0 2024-03-24T00:49:00.405 INFO:tasks.workunit.client.0.smithi038.stdout:1/718: readlink da/dd/d7a/d82/d98/d14/lc6 0 2024-03-24T00:49:00.405 INFO:tasks.workunit.client.0.smithi038.stdout:3/656: mknod d3/d9/d77/cdc 0 2024-03-24T00:49:00.405 INFO:tasks.workunit.client.0.smithi038.stdout:3/657: write d3/d30/d70/d90/d99/fbf [869665,97531] 0 2024-03-24T00:49:00.405 INFO:tasks.workunit.client.0.smithi038.stdout:3/658: readlink d3/d2a/la7 0 2024-03-24T00:49:00.405 INFO:tasks.workunit.client.0.smithi038.stdout:8/656: readlink d0/d45/d37/d5f/db7/l6e 0 2024-03-24T00:49:00.407 INFO:tasks.workunit.client.0.smithi038.stdout:6/710: dwrite d1/da/d61/da2/fac [0,4194304] 0 2024-03-24T00:49:00.407 INFO:tasks.workunit.client.0.smithi038.stdout:6/711: write d1/da/f52 [1788062,29420] 0 2024-03-24T00:49:00.440 INFO:tasks.workunit.client.0.smithi038.stdout:2/592: rmdir d4/dca 0 2024-03-24T00:49:00.441 INFO:tasks.workunit.client.0.smithi038.stdout:5/611: dwrite d6/d16/d14/f8e [0,4194304] 0 2024-03-24T00:49:00.447 INFO:tasks.workunit.client.0.smithi038.stdout:8/657: creat d0/d45/d37/d5f/db7/d62/fd9 x:0 0 0 2024-03-24T00:49:00.454 INFO:tasks.workunit.client.0.smithi038.stdout:4/610: getdents da/d38 0 2024-03-24T00:49:00.454 INFO:tasks.workunit.client.0.smithi038.stdout:4/611: dread - da/d12/d37/dd/d15/d4c/d70/fa5 zero size 2024-03-24T00:49:00.455 INFO:tasks.workunit.client.0.smithi038.stdout:0/584: write d0/d14/d84/f96 [349036,7562] 0 2024-03-24T00:49:00.455 INFO:tasks.workunit.client.0.smithi038.stdout:0/585: stat d0/d12/d42/d5e/db4 0 2024-03-24T00:49:00.461 INFO:tasks.workunit.client.0.smithi038.stdout:6/712: symlink d1/da/d1b/d1d/lea 0 2024-03-24T00:49:00.461 INFO:tasks.workunit.client.0.smithi038.stdout:6/713: fsync d1/da/d61/fd1 0 2024-03-24T00:49:00.462 INFO:tasks.workunit.client.0.smithi038.stdout:6/714: fdatasync d1/da/d1b/fb3 0 2024-03-24T00:49:00.462 INFO:tasks.workunit.client.0.smithi038.stdout:6/715: readlink d1/da/d61/d69/l9d 0 2024-03-24T00:49:00.464 INFO:tasks.workunit.client.1.smithi088.stdout:6/702: mknod db/d3d/d83/d71/cd3 0 2024-03-24T00:49:00.468 INFO:tasks.workunit.client.0.smithi038.stdout:9/727: getdents d12/d28/d3a 0 2024-03-24T00:49:00.470 INFO:tasks.workunit.client.1.smithi088.stdout:0/710: mkdir d9/dc/de/d7e/de6 0 2024-03-24T00:49:00.474 INFO:tasks.workunit.client.1.smithi088.stdout:9/697: rmdir d1/d15/d21/d9c 39 2024-03-24T00:49:00.477 INFO:tasks.workunit.client.0.smithi038.stdout:5/612: creat d6/d60/fca x:0 0 0 2024-03-24T00:49:00.480 INFO:tasks.workunit.client.0.smithi038.stdout:8/658: mkdir d0/d1d/d27/d6b/da4/dda 0 2024-03-24T00:49:00.483 INFO:tasks.workunit.client.0.smithi038.stdout:4/612: mkdir da/d12/d24/d5f/d8a/db6/dd1 0 2024-03-24T00:49:00.487 INFO:tasks.workunit.client.0.smithi038.stdout:0/586: symlink d0/d12/d15/d58/lc1 0 2024-03-24T00:49:00.487 INFO:tasks.workunit.client.0.smithi038.stdout:0/587: read - d0/d12/d42/d83/fb1 zero size 2024-03-24T00:49:00.489 INFO:tasks.workunit.client.0.smithi038.stdout:9/728: truncate fe 336499 0 2024-03-24T00:49:00.489 INFO:tasks.workunit.client.0.smithi038.stdout:9/729: write d12/d81/d84/f8e [4301577,2962] 0 2024-03-24T00:49:00.490 INFO:tasks.workunit.client.0.smithi038.stdout:7/611: sync 2024-03-24T00:49:00.490 INFO:tasks.workunit.client.0.smithi038.stdout:7/612: write d2/d9/da/dc/da8/dc5/fe1 [367291,15069] 0 2024-03-24T00:49:00.490 INFO:tasks.workunit.client.0.smithi038.stdout:7/613: truncate d2/d9/da/d23/d2b/f5f 72888 0 2024-03-24T00:49:00.490 INFO:tasks.workunit.client.0.smithi038.stdout:7/614: chown d2/fb9 78 1 2024-03-24T00:49:00.491 INFO:tasks.workunit.client.1.smithi088.stdout:5/724: write d1/d10/d1b/d2f/d57/f71 [1335987,30682] 0 2024-03-24T00:49:00.492 INFO:tasks.workunit.client.0.smithi038.stdout:6/716: dread d1/da/d1b/f26 [0,4194304] 0 2024-03-24T00:49:00.492 INFO:tasks.workunit.client.0.smithi038.stdout:6/717: fsync d1/f13 0 2024-03-24T00:49:00.495 INFO:tasks.workunit.client.1.smithi088.stdout:6/703: symlink db/d1d/ld4 0 2024-03-24T00:49:00.498 INFO:tasks.workunit.client.0.smithi038.stdout:5/613: creat d6/d1e/fcb x:0 0 0 2024-03-24T00:49:00.498 INFO:tasks.workunit.client.0.smithi038.stdout:5/614: creat d6/d1e/fcc x:0 0 0 2024-03-24T00:49:00.498 INFO:tasks.workunit.client.0.smithi038.stdout:5/615: read - d6/d19/d20/d64/d6b/fc2 zero size 2024-03-24T00:49:00.499 INFO:tasks.workunit.client.1.smithi088.stdout:0/711: link d9/ca9 d9/d26/d8f/da1/d62/dc4/ce7 0 2024-03-24T00:49:00.503 INFO:tasks.workunit.client.0.smithi038.stdout:8/659: rename d0/d45/d66/d6d/l86 to d0/d1d/d27/d4d/d81/ldb 0 2024-03-24T00:49:00.503 INFO:tasks.workunit.client.0.smithi038.stdout:8/660: fdatasync d0/d1d/f8a 0 2024-03-24T00:49:00.503 INFO:tasks.workunit.client.0.smithi038.stdout:8/661: fsync d0/d15/f54 0 2024-03-24T00:49:00.504 INFO:tasks.workunit.client.0.smithi038.stdout:4/613: creat da/d12/d24/d68/d6c/fd2 x:0 0 0 2024-03-24T00:49:00.505 INFO:tasks.workunit.client.0.smithi038.stdout:6/718: dread d1/faa [0,4194304] 0 2024-03-24T00:49:00.506 INFO:tasks.workunit.client.0.smithi038.stdout:6/719: write d1/da/d1b/d2f/fd0 [834857,121033] 0 2024-03-24T00:49:00.506 INFO:tasks.workunit.client.0.smithi038.stdout:6/720: creat d1/da/feb x:0 0 0 2024-03-24T00:49:00.507 INFO:tasks.workunit.client.0.smithi038.stdout:0/588: symlink d0/d12/d42/lc2 0 2024-03-24T00:49:00.508 INFO:tasks.workunit.client.0.smithi038.stdout:0/589: write d0/d14/d84/f96 [795497,24636] 0 2024-03-24T00:49:00.511 INFO:tasks.workunit.client.0.smithi038.stdout:9/730: fdatasync d12/d81/fa5 0 2024-03-24T00:49:00.516 INFO:tasks.workunit.client.0.smithi038.stdout:5/616: creat d6/d19/d20/d70/fcd x:0 0 0 2024-03-24T00:49:00.516 INFO:tasks.workunit.client.0.smithi038.stdout:5/617: chown d6/d60/fca 209757130 1 2024-03-24T00:49:00.521 INFO:tasks.workunit.client.1.smithi088.stdout:1/709: rmdir d6/d23/daf/d95 39 2024-03-24T00:49:00.522 INFO:tasks.workunit.client.1.smithi088.stdout:6/704: stat db/dc5/dc8/d43/c36 0 2024-03-24T00:49:00.522 INFO:tasks.workunit.client.1.smithi088.stdout:6/705: read - db/d3d/d83/f9f zero size 2024-03-24T00:49:00.522 INFO:tasks.workunit.client.1.smithi088.stdout:6/706: truncate db/d1d/d50/d90/d55/fbe 660299 0 2024-03-24T00:49:00.522 INFO:tasks.workunit.client.0.smithi038.stdout:0/590: dread d0/d1b/f87 [0,4194304] 0 2024-03-24T00:49:00.522 INFO:tasks.workunit.client.0.smithi038.stdout:4/614: dread da/d12/d37/dd/d15/f17 [0,4194304] 0 2024-03-24T00:49:00.523 INFO:tasks.workunit.client.0.smithi038.stdout:4/615: readlink da/l13 0 2024-03-24T00:49:00.523 INFO:tasks.workunit.client.0.smithi038.stdout:4/616: fsync da/d12/d37/dd/d15/d4c/f53 0 2024-03-24T00:49:00.523 INFO:tasks.workunit.client.0.smithi038.stdout:6/721: rename d1/da/d1b/d2f/c3a to d1/da/d73/d75/cec 0 2024-03-24T00:49:00.523 INFO:tasks.workunit.client.0.smithi038.stdout:6/722: fdatasync d1/da/d61/d69/f6e 0 2024-03-24T00:49:00.524 INFO:tasks.workunit.client.0.smithi038.stdout:2/593: dwrite d4/d18/d26/f85 [0,4194304] 0 2024-03-24T00:49:00.525 INFO:tasks.workunit.client.1.smithi088.stdout:5/725: dread d1/d10/d1b/d9d/fa3 [0,4194304] 0 2024-03-24T00:49:00.526 INFO:tasks.workunit.client.1.smithi088.stdout:1/710: rename d6/d23/c42 to d6/d23/daf/d95/d5b/d5e/ce8 0 2024-03-24T00:49:00.527 INFO:tasks.workunit.client.0.smithi038.stdout:9/731: readlink d12/d55/d56/la4 0 2024-03-24T00:49:00.527 INFO:tasks.workunit.client.0.smithi038.stdout:9/732: chown f4 30767500 1 2024-03-24T00:49:00.527 INFO:tasks.workunit.client.1.smithi088.stdout:0/712: dread d9/dc/de/d19/d44/f4a [0,4194304] 0 2024-03-24T00:49:00.528 INFO:tasks.workunit.client.1.smithi088.stdout:0/713: dread - d9/dc/de/d7e/d91/fc1 zero size 2024-03-24T00:49:00.528 INFO:tasks.workunit.client.1.smithi088.stdout:6/707: rmdir db/dd/d61/d6d/d74/d88/da1 39 2024-03-24T00:49:00.530 INFO:tasks.workunit.client.0.smithi038.stdout:7/615: truncate d2/d9/da/dc/d4a/f6d 4504996 0 2024-03-24T00:49:00.530 INFO:tasks.workunit.client.0.smithi038.stdout:7/616: truncate d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fa0 344766 0 2024-03-24T00:49:00.533 INFO:tasks.workunit.client.0.smithi038.stdout:5/618: getdents d6/d19/d20/d42/d43 0 2024-03-24T00:49:00.535 INFO:tasks.workunit.client.1.smithi088.stdout:1/711: unlink d6/d23/daf/d95/d5b/d5e/f60 0 2024-03-24T00:49:00.537 INFO:tasks.workunit.client.0.smithi038.stdout:8/662: rmdir d0/d1d/d27/d6b 39 2024-03-24T00:49:00.537 INFO:tasks.workunit.client.1.smithi088.stdout:6/708: creat db/dc5/dc8/d4b/fd5 x:0 0 0 2024-03-24T00:49:00.538 INFO:tasks.workunit.client.1.smithi088.stdout:8/685: sync 2024-03-24T00:49:00.538 INFO:tasks.workunit.client.1.smithi088.stdout:3/641: sync 2024-03-24T00:49:00.538 INFO:tasks.workunit.client.1.smithi088.stdout:3/642: fdatasync d3/d6/d27/d3c/f59 0 2024-03-24T00:49:00.538 INFO:tasks.workunit.client.1.smithi088.stdout:4/661: sync 2024-03-24T00:49:00.539 INFO:tasks.workunit.client.1.smithi088.stdout:7/600: sync 2024-03-24T00:49:00.539 INFO:tasks.workunit.client.1.smithi088.stdout:2/663: sync 2024-03-24T00:49:00.539 INFO:tasks.workunit.client.1.smithi088.stdout:0/714: symlink d9/dc/de/d7e/d91/le8 0 2024-03-24T00:49:00.539 INFO:tasks.workunit.client.1.smithi088.stdout:2/664: dread - db/d21/d73/fc5 zero size 2024-03-24T00:49:00.540 INFO:tasks.workunit.client.0.smithi038.stdout:0/591: link d0/d2d/c2e d0/d12/cc3 0 2024-03-24T00:49:00.543 INFO:tasks.workunit.client.0.smithi038.stdout:4/617: link da/d12/d37/dd/d15/f77 da/d38/fd3 0 2024-03-24T00:49:00.546 INFO:tasks.workunit.client.0.smithi038.stdout:6/723: mknod d1/da/d1b/d25/d49/d4e/dbe/ced 0 2024-03-24T00:49:00.546 INFO:tasks.workunit.client.0.smithi038.stdout:6/724: creat d1/da/d61/fee x:0 0 0 2024-03-24T00:49:00.546 INFO:tasks.workunit.client.0.smithi038.stdout:6/725: dread - d1/da/d61/fee zero size 2024-03-24T00:49:00.546 INFO:tasks.workunit.client.0.smithi038.stdout:6/726: creat d1/da/d61/da2/fef x:0 0 0 2024-03-24T00:49:00.547 INFO:tasks.workunit.client.1.smithi088.stdout:6/709: rename db/d1d/d50/d90/d55/l97 to db/dd/d61/d6d/ld6 0 2024-03-24T00:49:00.549 INFO:tasks.workunit.client.1.smithi088.stdout:0/715: getdents d9/d15 0 2024-03-24T00:49:00.550 INFO:tasks.workunit.client.1.smithi088.stdout:0/716: write d9/dc/de/d19/d44/f6e [884667,20402] 0 2024-03-24T00:49:00.550 INFO:tasks.workunit.client.1.smithi088.stdout:0/717: write d9/d26/d8f/da1/d63/f8a [503093,26879] 0 2024-03-24T00:49:00.555 INFO:tasks.workunit.client.1.smithi088.stdout:8/686: rename d9/dc/d11/d24/d29/d5b/d6f/fa7 to d9/dc/dce/ddc/fe8 0 2024-03-24T00:49:00.555 INFO:tasks.workunit.client.1.smithi088.stdout:8/687: read - d9/dc/d11/d24/d29/d36/d6a/d7b/fa0 zero size 2024-03-24T00:49:00.556 INFO:tasks.workunit.client.0.smithi038.stdout:2/594: unlink d4/d18/d26/d37/lcc 0 2024-03-24T00:49:00.561 INFO:tasks.workunit.client.1.smithi088.stdout:1/712: dread d6/d23/daf/d95/d5b/f75 [0,4194304] 0 2024-03-24T00:49:00.564 INFO:tasks.workunit.client.0.smithi038.stdout:9/733: creat d12/d28/d3a/fef x:0 0 0 2024-03-24T00:49:00.575 INFO:tasks.workunit.client.1.smithi088.stdout:7/601: dread d3/d31/f7a [0,4194304] 0 2024-03-24T00:49:00.576 INFO:tasks.workunit.client.0.smithi038.stdout:7/617: rename d2/d9/da/d9b/cbc to d2/d9/da/d23/ce9 0 2024-03-24T00:49:00.576 INFO:tasks.workunit.client.0.smithi038.stdout:6/727: read d1/da/d1b/d25/d49/d4e/d51/f9c [2833475,96973] 0 2024-03-24T00:49:00.576 INFO:tasks.workunit.client.0.smithi038.stdout:7/618: chown d2/d9/da/d23/d2b/c91 2 1 2024-03-24T00:49:00.577 INFO:tasks.workunit.client.1.smithi088.stdout:0/718: getdents d9/dc/de/d7e 0 2024-03-24T00:49:00.578 INFO:tasks.workunit.client.1.smithi088.stdout:1/713: dread d6/fba [0,4194304] 0 2024-03-24T00:49:00.584 INFO:tasks.workunit.client.1.smithi088.stdout:7/602: mknod d3/d32/d34/d9c/d4b/d73/dab/cc1 0 2024-03-24T00:49:00.586 INFO:tasks.workunit.client.0.smithi038.stdout:0/592: symlink d0/d1b/d2c/lc4 0 2024-03-24T00:49:00.588 INFO:tasks.workunit.client.1.smithi088.stdout:0/719: getdents d9/d26/d8f/da1/d62/d9b 0 2024-03-24T00:49:00.591 INFO:tasks.workunit.client.1.smithi088.stdout:1/714: getdents d6/d23/daf/d95/d5b/d80 0 2024-03-24T00:49:00.595 INFO:tasks.workunit.client.1.smithi088.stdout:9/698: dwrite d1/d10/f3e [0,4194304] 0 2024-03-24T00:49:00.605 INFO:tasks.workunit.client.0.smithi038.stdout:4/618: link da/d12/d29/d5b/d73/db4/fb7 da/d12/d24/d5f/fd4 0 2024-03-24T00:49:00.608 INFO:tasks.workunit.client.1.smithi088.stdout:6/710: dwrite db/dc5/dc8/d4b/f6e [0,4194304] 0 2024-03-24T00:49:00.608 INFO:tasks.workunit.client.1.smithi088.stdout:6/711: write db/d1d/d23/fbb [1534879,114560] 0 2024-03-24T00:49:00.609 INFO:tasks.workunit.client.0.smithi038.stdout:2/595: getdents d4/d18/da1 0 2024-03-24T00:49:00.611 INFO:tasks.workunit.client.0.smithi038.stdout:9/734: getdents d12/d28/d6c 0 2024-03-24T00:49:00.611 INFO:tasks.workunit.client.0.smithi038.stdout:9/735: fdatasync d12/f42 0 2024-03-24T00:49:00.613 INFO:tasks.workunit.client.1.smithi088.stdout:7/603: rmdir d3/d31 39 2024-03-24T00:49:00.613 INFO:tasks.workunit.client.0.smithi038.stdout:6/728: creat d1/da/d1b/d25/d49/d4e/ff0 x:0 0 0 2024-03-24T00:49:00.614 INFO:tasks.workunit.client.1.smithi088.stdout:3/643: dwrite d3/d6/d27/d3c/f30 [4194304,4194304] 0 2024-03-24T00:49:00.615 INFO:tasks.workunit.client.0.smithi038.stdout:8/663: dwrite d0/d1d/d27/d4d/f7a [0,4194304] 0 2024-03-24T00:49:00.615 INFO:tasks.workunit.client.0.smithi038.stdout:7/619: symlink d2/d2c/d90/lea 0 2024-03-24T00:49:00.615 INFO:tasks.workunit.client.0.smithi038.stdout:7/620: fdatasync d2/d9/da/f47 0 2024-03-24T00:49:00.615 INFO:tasks.workunit.client.0.smithi038.stdout:7/621: truncate d2/d9/da/f73 1226754 0 2024-03-24T00:49:00.616 INFO:tasks.workunit.client.0.smithi038.stdout:7/622: chown d2/d2c 2808 1 2024-03-24T00:49:00.616 INFO:tasks.workunit.client.0.smithi038.stdout:7/623: fsync d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/fe2 0 2024-03-24T00:49:00.616 INFO:tasks.workunit.client.0.smithi038.stdout:0/593: rename d0/d12/d42/d83/f9f to d0/d14/d84/fc5 0 2024-03-24T00:49:00.619 INFO:tasks.workunit.client.1.smithi088.stdout:4/662: dwrite d5/d7/d72/f73 [4194304,4194304] 0 2024-03-24T00:49:00.619 INFO:tasks.workunit.client.1.smithi088.stdout:4/663: write d5/d13/d18/d6e/d7d/f7e [2857398,115544] 0 2024-03-24T00:49:00.619 INFO:tasks.workunit.client.1.smithi088.stdout:6/712: symlink db/d1d/d50/d90/ld7 0 2024-03-24T00:49:00.619 INFO:tasks.workunit.client.1.smithi088.stdout:8/688: dwrite d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/f91 [0,4194304] 0 2024-03-24T00:49:00.620 INFO:tasks.workunit.client.1.smithi088.stdout:9/699: getdents d1/d10/d9b 0 2024-03-24T00:49:00.624 INFO:tasks.workunit.client.1.smithi088.stdout:7/604: rename d3/d32/fba to d3/d32/d34/fc2 0 2024-03-24T00:49:00.624 INFO:tasks.workunit.client.1.smithi088.stdout:7/605: chown d3/d31/f7a 0 1 2024-03-24T00:49:00.624 INFO:tasks.workunit.client.1.smithi088.stdout:7/606: chown d3/d32/d34/la0 19 1 2024-03-24T00:49:00.624 INFO:tasks.workunit.client.1.smithi088.stdout:7/607: chown d3/d32/c59 3 1 2024-03-24T00:49:00.625 INFO:tasks.workunit.client.0.smithi038.stdout:9/736: mkdir d12/d1a/d5f/df0 0 2024-03-24T00:49:00.625 INFO:tasks.workunit.client.0.smithi038.stdout:9/737: write d12/d81/d84/fa1 [664605,109469] 0 2024-03-24T00:49:00.625 INFO:tasks.workunit.client.0.smithi038.stdout:4/619: write da/fa2 [382111,46786] 0 2024-03-24T00:49:00.625 INFO:tasks.workunit.client.0.smithi038.stdout:4/620: dread - da/d12/d37/dd/f8d zero size 2024-03-24T00:49:00.627 INFO:tasks.workunit.client.0.smithi038.stdout:6/729: symlink d1/da/d61/d69/lf1 0 2024-03-24T00:49:00.630 INFO:tasks.workunit.client.1.smithi088.stdout:6/713: mknod db/dd/d87/d9e/cd8 0 2024-03-24T00:49:00.632 INFO:tasks.workunit.client.0.smithi038.stdout:8/664: symlink d0/d1d/d20/d3c/d9e/da3/dc8/ldc 0 2024-03-24T00:49:00.632 INFO:tasks.workunit.client.0.smithi038.stdout:8/665: chown d0/d45/d37/d5f/db7/c2b 18 1 2024-03-24T00:49:00.634 INFO:tasks.workunit.client.1.smithi088.stdout:3/644: link d3/d6/dca/dae/d4b/l6a d3/ld1 0 2024-03-24T00:49:00.635 INFO:tasks.workunit.client.0.smithi038.stdout:7/624: mkdir d2/d2c/d9a/deb 0 2024-03-24T00:49:00.635 INFO:tasks.workunit.client.0.smithi038.stdout:7/625: write d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fe0 [742580,125831] 0 2024-03-24T00:49:00.636 INFO:tasks.workunit.client.0.smithi038.stdout:7/626: readlink d2/d9/da/d23/d2b/d5b/d72/d92/d75/l8c 0 2024-03-24T00:49:00.638 INFO:tasks.workunit.client.1.smithi088.stdout:6/714: link db/d8e/fbc db/dd/fd9 0 2024-03-24T00:49:00.639 INFO:tasks.workunit.client.1.smithi088.stdout:8/689: read d9/dc/d11/f2f [963613,128791] 0 2024-03-24T00:49:00.639 INFO:tasks.workunit.client.0.smithi038.stdout:0/594: mknod d0/d12/d42/d9d/d28/cc6 0 2024-03-24T00:49:00.639 INFO:tasks.workunit.client.0.smithi038.stdout:0/595: truncate d0/d12/d15/d58/d64/da7/fb6 209476 0 2024-03-24T00:49:00.640 INFO:tasks.workunit.client.0.smithi038.stdout:0/596: write d0/d12/d42/d9d/d28/f39 [4888902,116202] 0 2024-03-24T00:49:00.640 INFO:tasks.workunit.client.0.smithi038.stdout:0/597: stat d0/d2d/f65 0 2024-03-24T00:49:00.640 INFO:tasks.workunit.client.0.smithi038.stdout:0/598: creat d0/d12/d42/d9d/fc7 x:0 0 0 2024-03-24T00:49:00.642 INFO:tasks.workunit.client.1.smithi088.stdout:6/715: creat db/dd/d61/d6d/d74/d88/fda x:0 0 0 2024-03-24T00:49:00.642 INFO:tasks.workunit.client.1.smithi088.stdout:6/716: creat db/dd/d61/fdb x:0 0 0 2024-03-24T00:49:00.642 INFO:tasks.workunit.client.1.smithi088.stdout:6/717: rename db/dd to db/dd/d61/d6d/d74/d88/ddc 22 2024-03-24T00:49:00.642 INFO:tasks.workunit.client.1.smithi088.stdout:9/700: read d1/d15/f68 [2885596,70070] 0 2024-03-24T00:49:00.642 INFO:tasks.workunit.client.1.smithi088.stdout:7/608: dread d3/d32/d34/d9c/d4b/d74/d79/f97 [0,4194304] 0 2024-03-24T00:49:00.643 INFO:tasks.workunit.client.0.smithi038.stdout:6/730: mknod d1/da/d1b/d25/d49/d4e/cf2 0 2024-03-24T00:49:00.643 INFO:tasks.workunit.client.0.smithi038.stdout:8/666: creat d0/d1d/d20/fdd x:0 0 0 2024-03-24T00:49:00.647 INFO:tasks.workunit.client.0.smithi038.stdout:9/738: rename d12/d28/d3a/c37 to d12/d81/cf1 0 2024-03-24T00:49:00.647 INFO:tasks.workunit.client.0.smithi038.stdout:9/739: chown d12/d28/d3a/d26/lc5 17 1 2024-03-24T00:49:00.647 INFO:tasks.workunit.client.0.smithi038.stdout:9/740: readlink d12/d1a/d5f/lc2 0 2024-03-24T00:49:00.653 INFO:tasks.workunit.client.0.smithi038.stdout:4/621: dread da/fa2 [0,4194304] 0 2024-03-24T00:49:00.654 INFO:tasks.workunit.client.1.smithi088.stdout:9/701: getdents d1/d10/d77 0 2024-03-24T00:49:00.654 INFO:tasks.workunit.client.1.smithi088.stdout:1/715: dwrite d6/d23/daf/d95/d58/f81 [0,4194304] 0 2024-03-24T00:49:00.655 INFO:tasks.workunit.client.1.smithi088.stdout:3/645: write d3/d6/dca/dae/d4b/fbf [3309532,61074] 0 2024-03-24T00:49:00.656 INFO:tasks.workunit.client.0.smithi038.stdout:0/599: mknod d0/d12/d15/d1a/cc8 0 2024-03-24T00:49:00.657 INFO:tasks.workunit.client.1.smithi088.stdout:7/609: mkdir d3/d32/d34/d9c/d4b/d73/dab/dc3 0 2024-03-24T00:49:00.658 INFO:tasks.workunit.client.1.smithi088.stdout:3/646: read d3/de/d6f/f7a [531519,112581] 0 2024-03-24T00:49:00.658 INFO:tasks.workunit.client.1.smithi088.stdout:7/610: creat d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/fc4 x:0 0 0 2024-03-24T00:49:00.658 INFO:tasks.workunit.client.1.smithi088.stdout:7/611: fsync d3/d32/d34/d9c/d4b/f60 0 2024-03-24T00:49:00.659 INFO:tasks.workunit.client.0.smithi038.stdout:8/667: link d0/d1d/d20/d3c/ccb d0/d1d/d27/d4d/cde 0 2024-03-24T00:49:00.664 INFO:tasks.workunit.client.0.smithi038.stdout:7/627: rename d2/d2c/f62 to d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/fec 0 2024-03-24T00:49:00.664 INFO:tasks.workunit.client.0.smithi038.stdout:7/628: fsync d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f66 0 2024-03-24T00:49:00.665 INFO:tasks.workunit.client.1.smithi088.stdout:6/718: rmdir db/d1d/d50/d90/d55 39 2024-03-24T00:49:00.669 INFO:tasks.workunit.client.0.smithi038.stdout:9/741: mkdir d12/d28/d3a/d26/df2 0 2024-03-24T00:49:00.671 INFO:tasks.workunit.client.1.smithi088.stdout:3/647: dread d3/f97 [0,4194304] 0 2024-03-24T00:49:00.671 INFO:tasks.workunit.client.0.smithi038.stdout:4/622: creat da/d12/d24/d68/d6c/da6/fd5 x:0 0 0 2024-03-24T00:49:00.675 INFO:tasks.workunit.client.0.smithi038.stdout:0/600: symlink d0/lc9 0 2024-03-24T00:49:00.677 INFO:tasks.workunit.client.1.smithi088.stdout:7/612: mkdir d3/d32/d34/d9c/d4b/da7/dc5 0 2024-03-24T00:49:00.678 INFO:tasks.workunit.client.1.smithi088.stdout:9/702: dread d1/d10/d11/f5f [0,4194304] 0 2024-03-24T00:49:00.679 INFO:tasks.workunit.client.0.smithi038.stdout:8/668: mkdir d0/d10/ddf 0 2024-03-24T00:49:00.679 INFO:tasks.workunit.client.1.smithi088.stdout:6/719: stat db/d3d/d83/d72/lc7 0 2024-03-24T00:49:00.680 INFO:tasks.workunit.client.1.smithi088.stdout:0/720: dwrite d9/f85 [0,4194304] 0 2024-03-24T00:49:00.681 INFO:tasks.workunit.client.1.smithi088.stdout:4/664: dwrite d5/f9 [0,4194304] 0 2024-03-24T00:49:00.681 INFO:tasks.workunit.client.1.smithi088.stdout:3/648: dread d3/d6/dca/dae/d4b/f4c [0,4194304] 0 2024-03-24T00:49:00.682 INFO:tasks.workunit.client.0.smithi038.stdout:6/731: rename d1/da/d1b/d25/d49/d4e/cd3 to d1/da/d1b/d1d/d3c/cf3 0 2024-03-24T00:49:00.686 INFO:tasks.workunit.client.1.smithi088.stdout:6/720: symlink db/dc5/dc8/d43/d5b/ldd 0 2024-03-24T00:49:00.692 INFO:tasks.workunit.client.0.smithi038.stdout:9/742: unlink d12/d55/c61 0 2024-03-24T00:49:00.696 INFO:tasks.workunit.client.1.smithi088.stdout:0/721: symlink d9/d15/le9 0 2024-03-24T00:49:00.698 INFO:tasks.workunit.client.1.smithi088.stdout:7/613: dread d3/d32/d34/d9c/d4b/f4c [0,4194304] 0 2024-03-24T00:49:00.698 INFO:tasks.workunit.client.1.smithi088.stdout:4/665: mkdir d5/d16/d37/de7 0 2024-03-24T00:49:00.698 INFO:tasks.workunit.client.1.smithi088.stdout:4/666: write d5/d16/d1a/d22/dc6/fdc [4213040,56628] 0 2024-03-24T00:49:00.700 INFO:tasks.workunit.client.1.smithi088.stdout:3/649: creat d3/de/d6f/d89/fd2 x:0 0 0 2024-03-24T00:49:00.700 INFO:tasks.workunit.client.1.smithi088.stdout:3/650: write d3/d6/dca/dae/d4b/f4c [736998,103803] 0 2024-03-24T00:49:00.704 INFO:tasks.workunit.client.1.smithi088.stdout:9/703: read d1/d10/d11/d23/f70 [3805633,126205] 0 2024-03-24T00:49:00.704 INFO:tasks.workunit.client.1.smithi088.stdout:9/704: fdatasync d1/d22/d27/fbf 0 2024-03-24T00:49:00.704 INFO:tasks.workunit.client.1.smithi088.stdout:9/705: readlink d1/d10/l42 0 2024-03-24T00:49:00.704 INFO:tasks.workunit.client.1.smithi088.stdout:6/721: rename db/d3d/d83/cc6 to db/dd/d61/d6d/d74/cde 0 2024-03-24T00:49:00.704 INFO:tasks.workunit.client.1.smithi088.stdout:6/722: read - db/dd/d61/d6d/d74/d88/fda zero size 2024-03-24T00:49:00.708 INFO:tasks.workunit.client.1.smithi088.stdout:0/722: rmdir d9/dc/de/d19/d84 39 2024-03-24T00:49:00.711 INFO:tasks.workunit.client.1.smithi088.stdout:7/614: creat d3/d32/d34/d7c/fc6 x:0 0 0 2024-03-24T00:49:00.715 INFO:tasks.workunit.client.1.smithi088.stdout:8/690: dwrite d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/f9a [4194304,4194304] 0 2024-03-24T00:49:00.716 INFO:tasks.workunit.client.1.smithi088.stdout:8/691: truncate d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/fb2 230825 0 2024-03-24T00:49:00.716 INFO:tasks.workunit.client.0.smithi038.stdout:0/601: creat d0/daf/fca x:0 0 0 2024-03-24T00:49:00.719 INFO:tasks.workunit.client.0.smithi038.stdout:8/669: creat d0/d45/d66/da6/fe0 x:0 0 0 2024-03-24T00:49:00.720 INFO:tasks.workunit.client.0.smithi038.stdout:8/670: write d0/d45/d37/d5f/d68/f91 [5100360,93212] 0 2024-03-24T00:49:00.720 INFO:tasks.workunit.client.0.smithi038.stdout:8/671: fsync d0/d15/fcf 0 2024-03-24T00:49:00.720 INFO:tasks.workunit.client.0.smithi038.stdout:8/672: stat d0/d1d/d27/d6b 0 2024-03-24T00:49:00.720 INFO:tasks.workunit.client.0.smithi038.stdout:8/673: creat d0/d45/d66/d6d/fe1 x:0 0 0 2024-03-24T00:49:00.722 INFO:tasks.workunit.client.1.smithi088.stdout:8/692: read d9/dc/d11/d24/d29/d5b/d8b/dd3/f66 [112799,63500] 0 2024-03-24T00:49:00.724 INFO:tasks.workunit.client.0.smithi038.stdout:4/623: dwrite da/d12/d37/dd/d15/d6d/f88 [0,4194304] 0 2024-03-24T00:49:00.725 INFO:tasks.workunit.client.0.smithi038.stdout:6/732: truncate d1/da/d1b/d1d/f2c 1277451 0 2024-03-24T00:49:00.725 INFO:tasks.workunit.client.1.smithi088.stdout:0/723: dread d9/f55 [0,4194304] 0 2024-03-24T00:49:00.725 INFO:tasks.workunit.client.1.smithi088.stdout:0/724: dread - d9/dc/de/d19/f51 zero size 2024-03-24T00:49:00.728 INFO:tasks.workunit.client.0.smithi038.stdout:0/602: rename d0/d12/d15/d1a/f99 to d0/d12/d42/d5e/db4/fcb 0 2024-03-24T00:49:00.728 INFO:tasks.workunit.client.0.smithi038.stdout:0/603: fsync d0/d14/d88/f97 0 2024-03-24T00:49:00.728 INFO:tasks.workunit.client.0.smithi038.stdout:0/604: truncate d0/f69 1349970 0 2024-03-24T00:49:00.728 INFO:tasks.workunit.client.0.smithi038.stdout:0/605: creat d0/d12/d15/d1a/fcc x:0 0 0 2024-03-24T00:49:00.732 INFO:tasks.workunit.client.1.smithi088.stdout:3/651: rename d3/de/f29 to d3/de/d6f/dcc/fd3 0 2024-03-24T00:49:00.735 INFO:tasks.workunit.client.1.smithi088.stdout:6/723: mkdir db/dd/d87/ddf 0 2024-03-24T00:49:00.735 INFO:tasks.workunit.client.1.smithi088.stdout:6/724: creat db/dd/d61/fe0 x:0 0 0 2024-03-24T00:49:00.743 INFO:tasks.workunit.client.0.smithi038.stdout:7/629: write d2/d9/da/dc/d4a/f6d [3395130,130435] 0 2024-03-24T00:49:00.744 INFO:tasks.workunit.client.0.smithi038.stdout:4/624: symlink da/d12/d24/d5f/d8a/db6/dd1/ld6 0 2024-03-24T00:49:00.755 INFO:tasks.workunit.client.0.smithi038.stdout:9/743: dwrite d12/d28/da8/fbf [0,4194304] 0 2024-03-24T00:49:00.755 INFO:tasks.workunit.client.0.smithi038.stdout:8/674: rename d0/d1d/d20/d3c/c7e to d0/d45/ce2 0 2024-03-24T00:49:00.755 INFO:tasks.workunit.client.0.smithi038.stdout:8/675: write d0/d15/d2c/f88 [352520,50796] 0 2024-03-24T00:49:00.770 INFO:tasks.workunit.client.0.smithi038.stdout:7/630: getdents d2/d9/da/d23/d2b/d5b/d72/d92 0 2024-03-24T00:49:00.772 INFO:tasks.workunit.client.1.smithi088.stdout:4/667: mkdir d5/d7/dd/de8 0 2024-03-24T00:49:00.774 INFO:tasks.workunit.client.1.smithi088.stdout:0/725: symlink d9/d26/d8f/dc7/lea 0 2024-03-24T00:49:00.780 INFO:tasks.workunit.client.1.smithi088.stdout:9/706: dwrite d1/d10/d11/f17 [0,4194304] 0 2024-03-24T00:49:00.783 INFO:tasks.workunit.client.1.smithi088.stdout:3/652: truncate d3/de/d6f/d85/d49/f8e 4151825 0 2024-03-24T00:49:00.783 INFO:tasks.workunit.client.1.smithi088.stdout:3/653: stat d3/d1a/d1e/d2f/d45/db5/d6e/la5 0 2024-03-24T00:49:00.783 INFO:tasks.workunit.client.1.smithi088.stdout:3/654: chown d3/d67/l83 494523 1 2024-03-24T00:49:00.787 INFO:tasks.workunit.client.0.smithi038.stdout:6/733: dread d1/da/d61/da2/faf [0,4194304] 0 2024-03-24T00:49:00.789 INFO:tasks.workunit.client.1.smithi088.stdout:5/726: sync 2024-03-24T00:49:00.793 INFO:tasks.workunit.client.1.smithi088.stdout:6/725: creat db/dd/d61/d6d/fe1 x:0 0 0 2024-03-24T00:49:00.794 INFO:tasks.workunit.client.1.smithi088.stdout:6/726: fdatasync db/d8e/f94 0 2024-03-24T00:49:00.798 INFO:tasks.workunit.client.0.smithi038.stdout:9/744: unlink d12/d28/c59 0 2024-03-24T00:49:00.799 INFO:tasks.workunit.client.1.smithi088.stdout:4/668: creat d5/d7/d15/d74/fe9 x:0 0 0 2024-03-24T00:49:00.799 INFO:tasks.workunit.client.1.smithi088.stdout:0/726: rmdir d9/d15 39 2024-03-24T00:49:00.800 INFO:tasks.workunit.client.0.smithi038.stdout:4/625: dwrite da/d12/d24/d68/f93 [0,4194304] 0 2024-03-24T00:49:00.801 INFO:tasks.workunit.client.0.smithi038.stdout:0/606: getdents d0/d12/d42/d5e 0 2024-03-24T00:49:00.803 INFO:tasks.workunit.client.1.smithi088.stdout:7/615: rename d3/f17 to d3/d31/fc7 0 2024-03-24T00:49:00.803 INFO:tasks.workunit.client.1.smithi088.stdout:7/616: stat d3/d32/d34/d7c/ca9 0 2024-03-24T00:49:00.804 INFO:tasks.workunit.client.0.smithi038.stdout:7/631: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d5d/ced 0 2024-03-24T00:49:00.806 INFO:tasks.workunit.client.0.smithi038.stdout:6/734: truncate d1/da/d1b/d25/d49/d4e/f66 2364142 0 2024-03-24T00:49:00.810 INFO:tasks.workunit.client.1.smithi088.stdout:9/707: symlink d1/d22/d27/d39/d98/da3/dd8/le3 0 2024-03-24T00:49:00.811 INFO:tasks.workunit.client.1.smithi088.stdout:9/708: write d1/d10/d29/f45 [1381996,123733] 0 2024-03-24T00:49:00.811 INFO:tasks.workunit.client.0.smithi038.stdout:8/676: mknod d0/d1d/ce3 0 2024-03-24T00:49:00.811 INFO:tasks.workunit.client.0.smithi038.stdout:2/596: sync 2024-03-24T00:49:00.812 INFO:tasks.workunit.client.0.smithi038.stdout:5/619: sync 2024-03-24T00:49:00.812 INFO:tasks.workunit.client.0.smithi038.stdout:3/659: sync 2024-03-24T00:49:00.812 INFO:tasks.workunit.client.0.smithi038.stdout:1/719: sync 2024-03-24T00:49:00.813 INFO:tasks.workunit.client.1.smithi088.stdout:7/617: dread d3/f15 [4194304,4194304] 0 2024-03-24T00:49:00.815 INFO:tasks.workunit.client.1.smithi088.stdout:5/727: unlink d1/d10/d81/cc1 0 2024-03-24T00:49:00.815 INFO:tasks.workunit.client.1.smithi088.stdout:6/727: read db/f19 [2879919,10053] 0 2024-03-24T00:49:00.816 INFO:tasks.workunit.client.0.smithi038.stdout:4/626: creat da/d12/d24/d5f/fd7 x:0 0 0 2024-03-24T00:49:00.816 INFO:tasks.workunit.client.0.smithi038.stdout:0/607: mkdir d0/d12/d15/d58/d64/da7/dbb/dcd 0 2024-03-24T00:49:00.819 INFO:tasks.workunit.client.1.smithi088.stdout:4/669: creat d5/d7/d72/fea x:0 0 0 2024-03-24T00:49:00.819 INFO:tasks.workunit.client.1.smithi088.stdout:4/670: write d5/d7/d72/f71 [915270,102307] 0 2024-03-24T00:49:00.819 INFO:tasks.workunit.client.1.smithi088.stdout:4/671: chown d5/d16/d1a/d22 87 1 2024-03-24T00:49:00.819 INFO:tasks.workunit.client.1.smithi088.stdout:4/672: write d5/f56 [3012885,6400] 0 2024-03-24T00:49:00.821 INFO:tasks.workunit.client.0.smithi038.stdout:7/632: unlink d2/ce 0 2024-03-24T00:49:00.821 INFO:tasks.workunit.client.0.smithi038.stdout:2/597: rename d4/l17 to d4/d27/d54/lcd 0 2024-03-24T00:49:00.832 INFO:tasks.workunit.client.1.smithi088.stdout:8/693: rename d9/f3e to d9/dc/dce/ddc/fe9 0 2024-03-24T00:49:00.832 INFO:tasks.workunit.client.0.smithi038.stdout:5/620: getdents d6/d19/d20/d64/d6b 0 2024-03-24T00:49:00.832 INFO:tasks.workunit.client.0.smithi038.stdout:3/660: unlink d3/f78 0 2024-03-24T00:49:00.837 INFO:tasks.workunit.client.0.smithi038.stdout:1/720: symlink da/dd/d2c/d40/lf3 0 2024-03-24T00:49:00.838 INFO:tasks.workunit.client.0.smithi038.stdout:3/661: dread d3/d9/fb3 [0,4194304] 0 2024-03-24T00:49:00.838 INFO:tasks.workunit.client.0.smithi038.stdout:3/662: readlink d3/d9/dc/l87 0 2024-03-24T00:49:00.838 INFO:tasks.workunit.client.1.smithi088.stdout:1/716: sync 2024-03-24T00:49:00.838 INFO:tasks.workunit.client.1.smithi088.stdout:2/665: sync 2024-03-24T00:49:00.838 INFO:tasks.workunit.client.1.smithi088.stdout:1/717: chown d6/d23/daf/d95/d5b/d5e/f64 125066 1 2024-03-24T00:49:00.839 INFO:tasks.workunit.client.1.smithi088.stdout:7/618: dread d3/fc [0,4194304] 0 2024-03-24T00:49:00.840 INFO:tasks.workunit.client.0.smithi038.stdout:8/677: dread d0/d1d/d20/f60 [0,4194304] 0 2024-03-24T00:49:00.840 INFO:tasks.workunit.client.0.smithi038.stdout:4/627: mkdir da/d12/d29/d5b/d73/db4/dd8 0 2024-03-24T00:49:00.840 INFO:tasks.workunit.client.0.smithi038.stdout:4/628: stat da/d12/d29/d5b/d73/db4/fb7 0 2024-03-24T00:49:00.843 INFO:tasks.workunit.client.1.smithi088.stdout:6/728: mknod db/dd/d61/d6d/d74/d88/da1/ce2 0 2024-03-24T00:49:00.844 INFO:tasks.workunit.client.1.smithi088.stdout:6/729: chown db/c29 19 1 2024-03-24T00:49:00.844 INFO:tasks.workunit.client.1.smithi088.stdout:6/730: read - db/dd/d61/d6d/d74/d88/f9b zero size 2024-03-24T00:49:00.846 INFO:tasks.workunit.client.1.smithi088.stdout:5/728: creat d1/d10/ff3 x:0 0 0 2024-03-24T00:49:00.846 INFO:tasks.workunit.client.1.smithi088.stdout:5/729: truncate d1/dd/d6f/d98/fd1 796496 0 2024-03-24T00:49:00.846 INFO:tasks.workunit.client.1.smithi088.stdout:5/730: chown d1/d19/d48/dc2 26400267 1 2024-03-24T00:49:00.848 INFO:tasks.workunit.client.0.smithi038.stdout:7/633: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/cee 0 2024-03-24T00:49:00.848 INFO:tasks.workunit.client.0.smithi038.stdout:7/634: chown d2/f63 6003 1 2024-03-24T00:49:00.849 INFO:tasks.workunit.client.0.smithi038.stdout:0/608: rename d0/c77 to d0/d12/d92/cce 0 2024-03-24T00:49:00.855 INFO:tasks.workunit.client.0.smithi038.stdout:5/621: mkdir d6/d19/d20/d42/d43/d68/dce 0 2024-03-24T00:49:00.855 INFO:tasks.workunit.client.0.smithi038.stdout:5/622: fdatasync d6/d19/d20/d42/d43/d84/fc6 0 2024-03-24T00:49:00.859 INFO:tasks.workunit.client.1.smithi088.stdout:4/673: mknod d5/d13/d63/ceb 0 2024-03-24T00:49:00.859 INFO:tasks.workunit.client.1.smithi088.stdout:4/674: write d5/d13/f86 [1252122,121908] 0 2024-03-24T00:49:00.864 INFO:tasks.workunit.client.1.smithi088.stdout:2/666: write db/f38 [1488238,39171] 0 2024-03-24T00:49:00.865 INFO:tasks.workunit.client.1.smithi088.stdout:7/619: dread d3/f21 [8388608,4194304] 0 2024-03-24T00:49:00.866 INFO:tasks.workunit.client.1.smithi088.stdout:7/620: chown d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/c8d 29466630 1 2024-03-24T00:49:00.867 INFO:tasks.workunit.client.1.smithi088.stdout:9/709: rename d1/d10/d11/ca8 to d1/db5/ce4 0 2024-03-24T00:49:00.867 INFO:tasks.workunit.client.1.smithi088.stdout:9/710: truncate d1/d22/d27/f8d 3541100 0 2024-03-24T00:49:00.867 INFO:tasks.workunit.client.1.smithi088.stdout:4/675: read d5/d13/d18/d24/f90 [212935,83877] 0 2024-03-24T00:49:00.867 INFO:tasks.workunit.client.1.smithi088.stdout:0/727: dwrite d9/d26/d8f/da1/d62/d9b/db0/db4/fc8 [0,4194304] 0 2024-03-24T00:49:00.869 INFO:tasks.workunit.client.1.smithi088.stdout:5/731: dread d1/d10/d1b/d2e/d9f/ddf/f80 [0,4194304] 0 2024-03-24T00:49:00.872 INFO:tasks.workunit.client.0.smithi038.stdout:1/721: truncate da/f23 4096750 0 2024-03-24T00:49:00.872 INFO:tasks.workunit.client.0.smithi038.stdout:3/663: creat d3/d9/dc/df/fdd x:0 0 0 2024-03-24T00:49:00.872 INFO:tasks.workunit.client.0.smithi038.stdout:1/722: chown da/dd/d2c 3351610 1 2024-03-24T00:49:00.872 INFO:tasks.workunit.client.0.smithi038.stdout:1/723: write da/dd/d7a/d82/d98/f9a [2062316,21155] 0 2024-03-24T00:49:00.874 INFO:tasks.workunit.client.0.smithi038.stdout:1/724: chown da/d3c/d86/lde 191203 1 2024-03-24T00:49:00.874 INFO:tasks.workunit.client.0.smithi038.stdout:1/725: read da/dd/d22/f71 [185862,58817] 0 2024-03-24T00:49:00.874 INFO:tasks.workunit.client.1.smithi088.stdout:8/694: symlink d9/dc/d11/d24/d29/d5b/db9/da5/lea 0 2024-03-24T00:49:00.875 INFO:tasks.workunit.client.0.smithi038.stdout:7/635: mkdir d2/d9/da/d23/def 0 2024-03-24T00:49:00.878 INFO:tasks.workunit.client.0.smithi038.stdout:7/636: readlink d2/d9/d16/l1e 0 2024-03-24T00:49:00.885 INFO:tasks.workunit.client.0.smithi038.stdout:9/745: dwrite d12/d81/f96 [0,4194304] 0 2024-03-24T00:49:00.886 INFO:tasks.workunit.client.0.smithi038.stdout:9/746: write d12/d41/dc0/fd1 [143478,115147] 0 2024-03-24T00:49:00.889 INFO:tasks.workunit.client.0.smithi038.stdout:4/629: dread da/d12/d37/dd/d15/fb3 [0,4194304] 0 2024-03-24T00:49:00.889 INFO:tasks.workunit.client.0.smithi038.stdout:4/630: truncate da/d38/fd3 323640 0 2024-03-24T00:49:00.889 INFO:tasks.workunit.client.0.smithi038.stdout:4/631: stat da/d12/f3f 0 2024-03-24T00:49:00.889 INFO:tasks.workunit.client.0.smithi038.stdout:4/632: creat da/d12/d37/d83/dac/fd9 x:0 0 0 2024-03-24T00:49:00.891 INFO:tasks.workunit.client.0.smithi038.stdout:1/726: write da/dd/d7a/d82/d98/d14/f2e [700318,54038] 0 2024-03-24T00:49:00.891 INFO:tasks.workunit.client.0.smithi038.stdout:1/727: dread - da/ddc/fe2 zero size 2024-03-24T00:49:00.891 INFO:tasks.workunit.client.0.smithi038.stdout:1/728: truncate da/dd/d7a/d82/d98/d14/f47 3882629 0 2024-03-24T00:49:00.894 INFO:tasks.workunit.client.1.smithi088.stdout:1/718: rename d6/d23/daf/d95/d5b/d5d/d61/d7d/de1 to d6/d23/daf/d95/d5b/d5d/d61/d7d/dc8/de9 0 2024-03-24T00:49:00.896 INFO:tasks.workunit.client.1.smithi088.stdout:9/711: symlink d1/le5 0 2024-03-24T00:49:00.897 INFO:tasks.workunit.client.1.smithi088.stdout:9/712: write d1/f8 [9128816,27822] 0 2024-03-24T00:49:00.900 INFO:tasks.workunit.client.1.smithi088.stdout:4/676: mknod d5/d17/cec 0 2024-03-24T00:49:00.903 INFO:tasks.workunit.client.0.smithi038.stdout:8/678: write d0/d45/d37/d5f/d68/fa0 [1733766,77956] 0 2024-03-24T00:49:00.903 INFO:tasks.workunit.client.0.smithi038.stdout:8/679: readlink d0/d1d/lc4 0 2024-03-24T00:49:00.905 INFO:tasks.workunit.client.1.smithi088.stdout:5/732: truncate d1/d19/d1e/f47 2406588 0 2024-03-24T00:49:00.905 INFO:tasks.workunit.client.1.smithi088.stdout:5/733: write d1/d10/d1b/d2e/fe1 [803267,66910] 0 2024-03-24T00:49:00.906 INFO:tasks.workunit.client.0.smithi038.stdout:3/664: dread d3/d9/d77/fc9 [0,4194304] 0 2024-03-24T00:49:00.908 INFO:tasks.workunit.client.0.smithi038.stdout:4/633: rename da/d12/d24/d68/d6c/fd2 to da/d12/d24/d5f/fda 0 2024-03-24T00:49:00.909 INFO:tasks.workunit.client.1.smithi088.stdout:8/695: mkdir d9/dc/d11/d24/d29/d39/d84/deb 0 2024-03-24T00:49:00.912 INFO:tasks.workunit.client.0.smithi038.stdout:1/729: creat da/ff4 x:0 0 0 2024-03-24T00:49:00.914 INFO:tasks.workunit.client.0.smithi038.stdout:1/730: write da/dd/d7a/d82/d98/d14/faf [891115,13767] 0 2024-03-24T00:49:00.914 INFO:tasks.workunit.client.0.smithi038.stdout:2/598: dwrite d4/d18/d38/fc0 [0,4194304] 0 2024-03-24T00:49:00.917 INFO:tasks.workunit.client.1.smithi088.stdout:6/731: dwrite db/dc5/dc8/d43/f34 [0,4194304] 0 2024-03-24T00:49:00.917 INFO:tasks.workunit.client.1.smithi088.stdout:2/667: rename db/dc/c40 to db/dd/d96/d99/db2/d88/dc6/cd5 0 2024-03-24T00:49:00.917 INFO:tasks.workunit.client.1.smithi088.stdout:6/732: truncate db/d1d/d50/d90/d55/fbe 1046220 0 2024-03-24T00:49:00.917 INFO:tasks.workunit.client.1.smithi088.stdout:0/728: dread d9/dc/de/d19/d44/f6e [0,4194304] 0 2024-03-24T00:49:00.917 INFO:tasks.workunit.client.1.smithi088.stdout:6/733: creat db/dd/d61/d6d/fe3 x:0 0 0 2024-03-24T00:49:00.917 INFO:tasks.workunit.client.1.smithi088.stdout:6/734: getdents db/d3d/dcb 0 2024-03-24T00:49:00.917 INFO:tasks.workunit.client.1.smithi088.stdout:6/735: write db/d1d/f62 [237310,92397] 0 2024-03-24T00:49:00.918 INFO:tasks.workunit.client.1.smithi088.stdout:8/696: write d9/d6b/f6c [1191395,7114] 0 2024-03-24T00:49:00.919 INFO:tasks.workunit.client.1.smithi088.stdout:8/697: fsync d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/fda 0 2024-03-24T00:49:00.922 INFO:tasks.workunit.client.1.smithi088.stdout:9/713: mkdir d1/d22/d27/d39/d98/de6 0 2024-03-24T00:49:00.924 INFO:tasks.workunit.client.0.smithi038.stdout:8/680: read d0/d1d/d27/d4d/fb2 [3026005,110074] 0 2024-03-24T00:49:00.925 INFO:tasks.workunit.client.1.smithi088.stdout:8/698: dread d9/dc/d11/d24/d29/d36/f88 [0,4194304] 0 2024-03-24T00:49:00.926 INFO:tasks.workunit.client.0.smithi038.stdout:3/665: creat d3/d30/d5d/fde x:0 0 0 2024-03-24T00:49:00.930 INFO:tasks.workunit.client.0.smithi038.stdout:4/634: symlink da/ldb 0 2024-03-24T00:49:00.930 INFO:tasks.workunit.client.0.smithi038.stdout:1/731: getdents da/dd/d7a/d82/d98 0 2024-03-24T00:49:00.931 INFO:tasks.workunit.client.0.smithi038.stdout:1/732: creat da/dd/d22/dab/ff5 x:0 0 0 2024-03-24T00:49:00.931 INFO:tasks.workunit.client.1.smithi088.stdout:4/677: mknod d5/d17/d91/ced 0 2024-03-24T00:49:00.933 INFO:tasks.workunit.client.0.smithi038.stdout:2/599: creat d4/d27/d54/d67/fce x:0 0 0 2024-03-24T00:49:00.939 INFO:tasks.workunit.client.1.smithi088.stdout:6/736: dread db/d8e/fbc [0,4194304] 0 2024-03-24T00:49:00.941 INFO:tasks.workunit.client.0.smithi038.stdout:3/666: link d3/d30/d70/d90/lc6 d3/d9/dc/da6/ldf 0 2024-03-24T00:49:00.941 INFO:tasks.workunit.client.0.smithi038.stdout:3/667: creat d3/d9/dc/df/fe0 x:0 0 0 2024-03-24T00:49:00.944 INFO:tasks.workunit.client.0.smithi038.stdout:4/635: mknod da/d27/cdc 0 2024-03-24T00:49:00.944 INFO:tasks.workunit.client.0.smithi038.stdout:9/747: dread d12/d55/d56/f7c [0,4194304] 0 2024-03-24T00:49:00.945 INFO:tasks.workunit.client.0.smithi038.stdout:9/748: readlink d12/d41/d57/l64 0 2024-03-24T00:49:00.945 INFO:tasks.workunit.client.0.smithi038.stdout:9/749: chown d12/d1a/f3c 6761605 1 2024-03-24T00:49:00.945 INFO:tasks.workunit.client.1.smithi088.stdout:8/699: dread d9/dc/f52 [0,4194304] 0 2024-03-24T00:49:00.945 INFO:tasks.workunit.client.0.smithi038.stdout:5/623: dwrite d6/d19/d20/d70/fcd [0,4194304] 0 2024-03-24T00:49:00.948 INFO:tasks.workunit.client.0.smithi038.stdout:1/733: creat da/dd/d2c/d40/de9/ff6 x:0 0 0 2024-03-24T00:49:00.951 INFO:tasks.workunit.client.0.smithi038.stdout:2/600: rmdir d4/d18/da1 39 2024-03-24T00:49:00.959 INFO:tasks.workunit.client.1.smithi088.stdout:0/729: mknod d9/dc/de/d19/d84/ceb 0 2024-03-24T00:49:00.961 INFO:tasks.workunit.client.0.smithi038.stdout:4/636: symlink da/d12/d37/dd/d15/d6d/d8f/ldd 0 2024-03-24T00:49:00.961 INFO:tasks.workunit.client.0.smithi038.stdout:9/750: rmdir d12/d55 39 2024-03-24T00:49:00.962 INFO:tasks.workunit.client.1.smithi088.stdout:5/734: dread d1/d10/d81/f97 [0,4194304] 0 2024-03-24T00:49:00.965 INFO:tasks.workunit.client.1.smithi088.stdout:1/719: dwrite f0 [4194304,4194304] 0 2024-03-24T00:49:00.966 INFO:tasks.workunit.client.1.smithi088.stdout:2/668: dread db/d21/d35/d49/f8c [0,4194304] 0 2024-03-24T00:49:00.966 INFO:tasks.workunit.client.1.smithi088.stdout:2/669: chown db/dd/d96/d99/dc3/ld1 421527650 1 2024-03-24T00:49:00.970 INFO:tasks.workunit.client.1.smithi088.stdout:4/678: link d5/d13/d18/d24/l58 d5/d7/d15/d60/lee 0 2024-03-24T00:49:00.970 INFO:tasks.workunit.client.0.smithi038.stdout:4/637: write da/d12/d37/dd/d15/f17 [4449793,46527] 0 2024-03-24T00:49:00.970 INFO:tasks.workunit.client.0.smithi038.stdout:4/638: chown da/d12/d24/d5f/d8a/db6/dd1/ld6 21854 1 2024-03-24T00:49:00.971 INFO:tasks.workunit.client.0.smithi038.stdout:5/624: getdents d6/d19/d20/d64/d6b/d77 0 2024-03-24T00:49:00.979 INFO:tasks.workunit.client.1.smithi088.stdout:5/735: rmdir d1/d10/d79 39 2024-03-24T00:49:00.980 INFO:tasks.workunit.client.1.smithi088.stdout:5/736: dread - d1/d10/d1b/d2f/d57/f5f zero size 2024-03-24T00:49:00.981 INFO:tasks.workunit.client.0.smithi038.stdout:9/751: creat d12/d28/d3a/d26/d79/ff3 x:0 0 0 2024-03-24T00:49:00.984 INFO:tasks.workunit.client.1.smithi088.stdout:0/730: write d9/dc/de/d19/d44/d46/d65/f35 [5759139,63915] 0 2024-03-24T00:49:00.984 INFO:tasks.workunit.client.1.smithi088.stdout:0/731: stat d9/dc/de/d19/d44/d46/d65 0 2024-03-24T00:49:00.984 INFO:tasks.workunit.client.1.smithi088.stdout:0/732: fsync d9/dc/de/d19/f93 0 2024-03-24T00:49:00.984 INFO:tasks.workunit.client.1.smithi088.stdout:0/733: chown d9/d26/d8f/da1/d62/d9b/db0 224 1 2024-03-24T00:49:00.990 INFO:tasks.workunit.client.1.smithi088.stdout:2/670: unlink db/dd/d96/d4e/f81 0 2024-03-24T00:49:00.990 INFO:tasks.workunit.client.1.smithi088.stdout:1/720: creat d6/d23/dcb/de7/fea x:0 0 0 2024-03-24T00:49:00.990 INFO:tasks.workunit.client.1.smithi088.stdout:2/671: fsync db/d21/d35/f7d 0 2024-03-24T00:49:00.990 INFO:tasks.workunit.client.0.smithi038.stdout:4/639: rename da/d12/d37/f54 to da/d12/d24/d68/d6c/dce/fde 0 2024-03-24T00:49:00.994 INFO:tasks.workunit.client.0.smithi038.stdout:9/752: link d12/d28/f70 d12/d41/ff4 0 2024-03-24T00:49:00.994 INFO:tasks.workunit.client.0.smithi038.stdout:7/637: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d75/f78 [4194304,4194304] 0 2024-03-24T00:49:00.995 INFO:tasks.workunit.client.1.smithi088.stdout:4/679: dread d5/d7/d15/d74/d95/fe0 [0,4194304] 0 2024-03-24T00:49:01.002 INFO:tasks.workunit.client.1.smithi088.stdout:5/737: unlink d1/dd/d6f/da1/d3f/f41 0 2024-03-24T00:49:01.003 INFO:tasks.workunit.client.1.smithi088.stdout:7/621: rename d3/l1e to d3/d32/d34/d9c/d4b/d74/lc8 0 2024-03-24T00:49:01.003 INFO:tasks.workunit.client.1.smithi088.stdout:5/738: write d1/d43/d4e/d5b/fcb [1939835,51739] 0 2024-03-24T00:49:01.003 INFO:tasks.workunit.client.1.smithi088.stdout:5/739: chown d1/d10/d1b 0 1 2024-03-24T00:49:01.007 INFO:tasks.workunit.client.0.smithi038.stdout:7/638: mkdir d2/d9/da/d23/def/df0 0 2024-03-24T00:49:01.007 INFO:tasks.workunit.client.0.smithi038.stdout:7/639: readlink d2/d9/d16/l1e 0 2024-03-24T00:49:01.009 INFO:tasks.workunit.client.1.smithi088.stdout:2/672: unlink db/dd/d96/f7c 0 2024-03-24T00:49:01.009 INFO:tasks.workunit.client.1.smithi088.stdout:2/673: stat db/dd/d96/d4e/d8f/fbc 0 2024-03-24T00:49:01.016 INFO:tasks.workunit.client.1.smithi088.stdout:9/714: rename d1/d22/d72/l73 to d1/d10/d29/d85/le7 0 2024-03-24T00:49:01.017 INFO:tasks.workunit.client.0.smithi038.stdout:3/668: dwrite d3/d30/d70/f7b [0,4194304] 0 2024-03-24T00:49:01.020 INFO:tasks.workunit.client.0.smithi038.stdout:7/640: getdents d2/d9/da/dc/d4a 0 2024-03-24T00:49:01.020 INFO:tasks.workunit.client.0.smithi038.stdout:7/641: write d2/fb9 [8628,108293] 0 2024-03-24T00:49:01.020 INFO:tasks.workunit.client.0.smithi038.stdout:7/642: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fca 0 2024-03-24T00:49:01.022 INFO:tasks.workunit.client.1.smithi088.stdout:5/740: unlink d1/d10/d81/f97 0 2024-03-24T00:49:01.023 INFO:tasks.workunit.client.1.smithi088.stdout:5/741: stat d1/d19/l2d 0 2024-03-24T00:49:01.024 INFO:tasks.workunit.client.0.smithi038.stdout:8/681: dwrite d0/d15/f94 [0,4194304] 0 2024-03-24T00:49:01.026 INFO:tasks.workunit.client.1.smithi088.stdout:8/700: dwrite d9/dc/d11/d24/f2a [0,4194304] 0 2024-03-24T00:49:01.026 INFO:tasks.workunit.client.1.smithi088.stdout:1/721: dread d6/d27/f8e [0,4194304] 0 2024-03-24T00:49:01.026 INFO:tasks.workunit.client.1.smithi088.stdout:7/622: write d3/f28 [3356237,1468] 0 2024-03-24T00:49:01.029 INFO:tasks.workunit.client.1.smithi088.stdout:2/674: dread db/d21/d35/f79 [0,4194304] 0 2024-03-24T00:49:01.030 INFO:tasks.workunit.client.1.smithi088.stdout:3/655: sync 2024-03-24T00:49:01.031 INFO:tasks.workunit.client.0.smithi038.stdout:3/669: creat d3/d30/d5d/d7c/db0/fe1 x:0 0 0 2024-03-24T00:49:01.034 INFO:tasks.workunit.client.0.smithi038.stdout:7/643: creat d2/d9/da/dc/d93/ff1 x:0 0 0 2024-03-24T00:49:01.040 INFO:tasks.workunit.client.1.smithi088.stdout:0/734: rmdir d9/d26/d8f 39 2024-03-24T00:49:01.044 INFO:tasks.workunit.client.1.smithi088.stdout:9/715: symlink d1/d22/d27/d39/d98/da3/le8 0 2024-03-24T00:49:01.045 INFO:tasks.workunit.client.0.smithi038.stdout:8/682: truncate d0/d1d/f22 2595241 0 2024-03-24T00:49:01.045 INFO:tasks.workunit.client.0.smithi038.stdout:8/683: write d0/d1d/d27/d4d/f75 [5010200,117916] 0 2024-03-24T00:49:01.048 INFO:tasks.workunit.client.1.smithi088.stdout:7/623: dread d3/f2b [0,4194304] 0 2024-03-24T00:49:01.051 INFO:tasks.workunit.client.1.smithi088.stdout:8/701: dread d9/dc/d11/d24/d29/d39/fc9 [0,4194304] 0 2024-03-24T00:49:01.051 INFO:tasks.workunit.client.1.smithi088.stdout:1/722: creat d6/d23/daf/d95/d5b/feb x:0 0 0 2024-03-24T00:49:01.052 INFO:tasks.workunit.client.0.smithi038.stdout:1/734: dwrite da/dd/d2c/d5a/d74/f75 [0,4194304] 0 2024-03-24T00:49:01.052 INFO:tasks.workunit.client.0.smithi038.stdout:1/735: read - da/dd/d2c/d40/de9/ff6 zero size 2024-03-24T00:49:01.063 INFO:tasks.workunit.client.0.smithi038.stdout:3/670: mknod d3/d30/d5d/d7c/db0/ce2 0 2024-03-24T00:49:01.064 INFO:tasks.workunit.client.1.smithi088.stdout:2/675: creat db/dd/d96/d99/db2/d88/fd6 x:0 0 0 2024-03-24T00:49:01.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/684: mknod d0/d1d/d27/ce4 0 2024-03-24T00:49:01.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/685: chown d0/d10 210509232 1 2024-03-24T00:49:01.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/686: getdents d0/d10/ddf 0 2024-03-24T00:49:01.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/687: stat d0/d10/fab 0 2024-03-24T00:49:01.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/688: creat d0/d45/fe5 x:0 0 0 2024-03-24T00:49:01.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/689: dread - d0/d1d/d27/d4d/fce zero size 2024-03-24T00:49:01.074 INFO:tasks.workunit.client.0.smithi038.stdout:4/640: dread da/d12/d24/d68/f93 [0,4194304] 0 2024-03-24T00:49:01.074 INFO:tasks.workunit.client.0.smithi038.stdout:4/641: stat da/d38/f63 0 2024-03-24T00:49:01.074 INFO:tasks.workunit.client.0.smithi038.stdout:4/642: stat da/d12/d29/d5b/cc7 0 2024-03-24T00:49:01.074 INFO:tasks.workunit.client.0.smithi038.stdout:4/643: chown da/d38/f4a 5189368 1 2024-03-24T00:49:01.075 INFO:tasks.workunit.client.1.smithi088.stdout:0/735: creat d9/dc/de0/fec x:0 0 0 2024-03-24T00:49:01.093 INFO:tasks.workunit.client.1.smithi088.stdout:0/736: dread d9/dc/de0/fe4 [0,4194304] 0 2024-03-24T00:49:01.102 INFO:tasks.workunit.client.0.smithi038.stdout:4/644: write da/d12/d37/dd/f2b [4104872,93457] 0 2024-03-24T00:49:01.102 INFO:tasks.workunit.client.0.smithi038.stdout:4/645: write da/d12/d24/d5f/fda [828244,92558] 0 2024-03-24T00:49:01.102 INFO:tasks.workunit.client.0.smithi038.stdout:4/646: chown da/d12/d24/d68 92626 1 2024-03-24T00:49:01.103 INFO:tasks.workunit.client.1.smithi088.stdout:3/656: read f1 [1771727,64547] 0 2024-03-24T00:49:01.103 INFO:tasks.workunit.client.1.smithi088.stdout:3/657: truncate d3/d1a/d1e/d2f/fcd 367292 0 2024-03-24T00:49:01.103 INFO:tasks.workunit.client.1.smithi088.stdout:9/716: dread d1/d10/d2c/f7d [0,4194304] 0 2024-03-24T00:49:01.103 INFO:tasks.workunit.client.1.smithi088.stdout:8/702: mknod d9/dc/d11/d24/d29/d5b/db9/da5/cec 0 2024-03-24T00:49:01.105 INFO:tasks.workunit.client.0.smithi038.stdout:2/601: dwrite d4/d27/f94 [0,4194304] 0 2024-03-24T00:49:01.108 INFO:tasks.workunit.client.1.smithi088.stdout:4/680: dwrite d5/d17/f83 [0,4194304] 0 2024-03-24T00:49:01.108 INFO:tasks.workunit.client.1.smithi088.stdout:2/676: creat db/dd/d96/d4e/d8f/dad/fd7 x:0 0 0 2024-03-24T00:49:01.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:00 smithi038 ceph-mon[29271]: pgmap v36: 65 pgs: 65 active+clean; 2.2 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 152 MiB/s rd, 180 MiB/s wr, 253 op/s 2024-03-24T00:49:01.116 INFO:tasks.workunit.client.1.smithi088.stdout:0/737: dread d9/dc/de/fcf [0,4194304] 0 2024-03-24T00:49:01.117 INFO:tasks.workunit.client.1.smithi088.stdout:4/681: dread d5/d7/fe [0,4194304] 0 2024-03-24T00:49:01.122 INFO:tasks.workunit.client.0.smithi038.stdout:7/644: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/f70 [0,4194304] 0 2024-03-24T00:49:01.124 INFO:tasks.workunit.client.0.smithi038.stdout:9/753: dwrite d12/d41/f6a [0,4194304] 0 2024-03-24T00:49:01.124 INFO:tasks.workunit.client.0.smithi038.stdout:9/754: truncate d12/d81/d84/fa1 1583306 0 2024-03-24T00:49:01.124 INFO:tasks.workunit.client.0.smithi038.stdout:8/690: unlink d0/d1d/f8a 0 2024-03-24T00:49:01.124 INFO:tasks.workunit.client.0.smithi038.stdout:8/691: write d0/d1d/d27/d4d/fce [697765,28403] 0 2024-03-24T00:49:01.127 INFO:tasks.workunit.client.1.smithi088.stdout:8/703: creat d9/dc/d11/d24/d29/d5b/d8b/dd3/fed x:0 0 0 2024-03-24T00:49:01.128 INFO:tasks.workunit.client.1.smithi088.stdout:7/624: dread d3/f75 [0,4194304] 0 2024-03-24T00:49:01.136 INFO:tasks.workunit.client.1.smithi088.stdout:0/738: creat d9/d26/d8f/dc7/fed x:0 0 0 2024-03-24T00:49:01.137 INFO:tasks.workunit.client.1.smithi088.stdout:5/742: dwrite d1/d19/d1e/f42 [0,4194304] 0 2024-03-24T00:49:01.137 INFO:tasks.workunit.client.1.smithi088.stdout:5/743: fsync d1/d10/d1b/d2f/d57/f71 0 2024-03-24T00:49:01.138 INFO:tasks.workunit.client.1.smithi088.stdout:5/744: getdents d1/dd/d6f/d98/da0 0 2024-03-24T00:49:01.138 INFO:tasks.workunit.client.1.smithi088.stdout:5/745: fdatasync d1/d19/d48/dc2/fc9 0 2024-03-24T00:49:01.138 INFO:tasks.workunit.client.0.smithi038.stdout:1/736: dwrite da/dd/d2c/f63 [0,4194304] 0 2024-03-24T00:49:01.139 INFO:tasks.workunit.client.1.smithi088.stdout:1/723: dwrite d6/d23/daf/d95/d5b/d5d/d61/f7e [0,4194304] 0 2024-03-24T00:49:01.139 INFO:tasks.workunit.client.1.smithi088.stdout:1/724: fdatasync d6/f2a 0 2024-03-24T00:49:01.141 INFO:tasks.workunit.client.0.smithi038.stdout:3/671: getdents d3/d30/d5d/d6a 0 2024-03-24T00:49:01.141 INFO:tasks.workunit.client.0.smithi038.stdout:3/672: chown d3/d30/f7e 3 1 2024-03-24T00:49:01.142 INFO:tasks.workunit.client.0.smithi038.stdout:3/673: dread - d3/d9/dc/df/fe0 zero size 2024-03-24T00:49:01.142 INFO:tasks.workunit.client.0.smithi038.stdout:3/674: creat d3/d9/dc/da6/db5/dad/fe3 x:0 0 0 2024-03-24T00:49:01.144 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:00 smithi088 ceph-mon[36610]: pgmap v36: 65 pgs: 65 active+clean; 2.2 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 152 MiB/s rd, 180 MiB/s wr, 253 op/s 2024-03-24T00:49:01.144 INFO:tasks.workunit.client.1.smithi088.stdout:4/682: rename d5/d16/l49 to d5/d16/d1a/d22/dc6/lef 0 2024-03-24T00:49:01.157 INFO:tasks.workunit.client.1.smithi088.stdout:3/658: rmdir d3/de 39 2024-03-24T00:49:01.160 INFO:tasks.workunit.client.1.smithi088.stdout:8/704: mkdir d9/dc/d11/d24/d43/d4a/dee 0 2024-03-24T00:49:01.160 INFO:tasks.workunit.client.1.smithi088.stdout:8/705: chown d9/d6b/ce4 1566707 1 2024-03-24T00:49:01.163 INFO:tasks.workunit.client.1.smithi088.stdout:7/625: chown d3/c25 941920 1 2024-03-24T00:49:01.168 INFO:tasks.workunit.client.1.smithi088.stdout:0/739: rmdir d9/dc/de/d19/d44/d46/d65/d2a/d2e 39 2024-03-24T00:49:01.173 INFO:tasks.workunit.client.1.smithi088.stdout:9/717: dwrite d1/d10/d9b/dab/db8/da7/fd6 [0,4194304] 0 2024-03-24T00:49:01.179 INFO:tasks.workunit.client.0.smithi038.stdout:8/692: dwrite d0/d10/f92 [0,4194304] 0 2024-03-24T00:49:01.182 INFO:tasks.workunit.client.1.smithi088.stdout:3/659: write d3/de/d6f/fa9 [3584667,25390] 0 2024-03-24T00:49:01.182 INFO:tasks.workunit.client.1.smithi088.stdout:8/706: dread d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/f60 [0,4194304] 0 2024-03-24T00:49:01.189 INFO:tasks.workunit.client.0.smithi038.stdout:9/755: mkdir d12/d28/d9c/ddd/df5 0 2024-03-24T00:49:01.190 INFO:tasks.workunit.client.1.smithi088.stdout:5/746: creat d1/dd/d6f/da1/ff4 x:0 0 0 2024-03-24T00:49:01.190 INFO:tasks.workunit.client.1.smithi088.stdout:1/725: rmdir d6/d23/daf/d95/d5b/d93 39 2024-03-24T00:49:01.193 INFO:tasks.workunit.client.0.smithi038.stdout:4/647: symlink da/d12/d37/dd/d15/d6d/ldf 0 2024-03-24T00:49:01.193 INFO:tasks.workunit.client.0.smithi038.stdout:4/648: creat da/d12/d24/d5f/d8a/fe0 x:0 0 0 2024-03-24T00:49:01.198 INFO:tasks.workunit.client.1.smithi088.stdout:2/677: rename db/dd/d96/d4e/d8f/fbc to db/d21/d35/d49/fd8 0 2024-03-24T00:49:01.198 INFO:tasks.workunit.client.1.smithi088.stdout:2/678: fsync db/dd/d96/d4e/d8f/d9c/fc1 0 2024-03-24T00:49:01.199 INFO:tasks.workunit.client.1.smithi088.stdout:0/740: dread d9/dc/de0/fe4 [0,4194304] 0 2024-03-24T00:49:01.205 INFO:tasks.workunit.client.1.smithi088.stdout:4/683: mkdir d5/d13/d18/d24/d3d/df0 0 2024-03-24T00:49:01.205 INFO:tasks.workunit.client.1.smithi088.stdout:7/626: creat d3/d32/d34/d9c/d4b/da7/fc9 x:0 0 0 2024-03-24T00:49:01.206 INFO:tasks.workunit.client.1.smithi088.stdout:7/627: creat d3/d32/d34/d9c/d4b/d74/d79/d7f/fca x:0 0 0 2024-03-24T00:49:01.206 INFO:tasks.workunit.client.1.smithi088.stdout:7/628: write d3/d32/d34/fc2 [410660,117397] 0 2024-03-24T00:49:01.207 INFO:tasks.workunit.client.0.smithi038.stdout:3/675: unlink d3/d9/dc/df/f6b 0 2024-03-24T00:49:01.214 INFO:tasks.workunit.client.1.smithi088.stdout:9/718: symlink d1/d15/d80/da5/le9 0 2024-03-24T00:49:01.214 INFO:tasks.workunit.client.1.smithi088.stdout:9/719: stat d1/d10/d29/d85/f91 0 2024-03-24T00:49:01.214 INFO:tasks.workunit.client.1.smithi088.stdout:3/660: write d3/d1a/d1e/d2f/d45/f5d [3403592,92531] 0 2024-03-24T00:49:01.216 INFO:tasks.workunit.client.1.smithi088.stdout:1/726: creat d6/d27/db0/de3/fec x:0 0 0 2024-03-24T00:49:01.217 INFO:tasks.workunit.client.0.smithi038.stdout:2/602: rmdir d4/d27/d54 39 2024-03-24T00:49:01.218 INFO:tasks.workunit.client.1.smithi088.stdout:5/747: dread d1/d19/f45 [0,4194304] 0 2024-03-24T00:49:01.221 INFO:tasks.workunit.client.0.smithi038.stdout:1/737: dwrite da/dd/d2c/d5a/d74/f88 [0,4194304] 0 2024-03-24T00:49:01.221 INFO:tasks.workunit.client.0.smithi038.stdout:7/645: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d33/f3c [0,4194304] 0 2024-03-24T00:49:01.221 INFO:tasks.workunit.client.0.smithi038.stdout:7/646: chown d2/d9/da/f47 0 1 2024-03-24T00:49:01.221 INFO:tasks.workunit.client.0.smithi038.stdout:7/647: creat d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/ff2 x:0 0 0 2024-03-24T00:49:01.230 INFO:tasks.workunit.client.1.smithi088.stdout:2/679: link db/dd/d96/fa2 db/dd/d96/d4e/d8f/d9c/db0/fd9 0 2024-03-24T00:49:01.230 INFO:tasks.workunit.client.1.smithi088.stdout:2/680: fdatasync db/dd/d96/d99/db2/d88/fd6 0 2024-03-24T00:49:01.231 INFO:tasks.workunit.client.1.smithi088.stdout:4/684: mkdir d5/d13/d63/d78/df1 0 2024-03-24T00:49:01.235 INFO:tasks.workunit.client.1.smithi088.stdout:9/720: write d1/f4 [4035806,110603] 0 2024-03-24T00:49:01.236 INFO:tasks.workunit.client.1.smithi088.stdout:8/707: dwrite d9/dc/d11/d24/d29/d36/f88 [0,4194304] 0 2024-03-24T00:49:01.236 INFO:tasks.workunit.client.1.smithi088.stdout:8/708: chown d9/l3b 4 1 2024-03-24T00:49:01.237 INFO:tasks.workunit.client.0.smithi038.stdout:8/693: symlink d0/d10/le6 0 2024-03-24T00:49:01.243 INFO:tasks.workunit.client.0.smithi038.stdout:9/756: link d12/d28/d3a/d26/f86 d12/d81/d84/dec/ff6 0 2024-03-24T00:49:01.246 INFO:tasks.workunit.client.1.smithi088.stdout:6/737: sync 2024-03-24T00:49:01.248 INFO:tasks.workunit.client.1.smithi088.stdout:3/661: creat d3/d1a/d1e/fd4 x:0 0 0 2024-03-24T00:49:01.252 INFO:tasks.workunit.client.1.smithi088.stdout:1/727: mknod d6/d27/d67/d88/da4/ced 0 2024-03-24T00:49:01.260 INFO:tasks.workunit.client.1.smithi088.stdout:2/681: mknod db/dd/d96/d99/db2/d88/cda 0 2024-03-24T00:49:01.260 INFO:tasks.workunit.client.1.smithi088.stdout:2/682: stat db/d21/d35/f7d 0 2024-03-24T00:49:01.261 INFO:tasks.workunit.client.1.smithi088.stdout:0/741: rmdir d9/dc/de0 39 2024-03-24T00:49:01.261 INFO:tasks.workunit.client.0.smithi038.stdout:4/649: rename da/d27/l74 to da/d38/le1 0 2024-03-24T00:49:01.262 INFO:tasks.workunit.client.0.smithi038.stdout:3/676: mknod d3/d30/d70/d90/ce4 0 2024-03-24T00:49:01.262 INFO:tasks.workunit.client.0.smithi038.stdout:3/677: write d3/d9/dc/da6/db5/dad/fc4 [304728,18648] 0 2024-03-24T00:49:01.264 INFO:tasks.workunit.client.0.smithi038.stdout:2/603: unlink d4/d18/d26/d37/d69/fa0 0 2024-03-24T00:49:01.264 INFO:tasks.workunit.client.0.smithi038.stdout:2/604: chown d4/d76/fb3 17154825 1 2024-03-24T00:49:01.269 INFO:tasks.workunit.client.1.smithi088.stdout:9/721: mknod d1/d15/d21/d9c/cea 0 2024-03-24T00:49:01.273 INFO:tasks.workunit.client.1.smithi088.stdout:7/629: dwrite d3/f35 [0,4194304] 0 2024-03-24T00:49:01.273 INFO:tasks.workunit.client.1.smithi088.stdout:7/630: stat d3/d32/d34/d9c/d4b/d74/d79/fa1 0 2024-03-24T00:49:01.276 INFO:tasks.workunit.client.1.smithi088.stdout:1/728: fsync d6/d27/db0/de3/fec 0 2024-03-24T00:49:01.289 INFO:tasks.workunit.client.0.smithi038.stdout:1/738: creat da/dd/d22/d5c/dda/ff7 x:0 0 0 2024-03-24T00:49:01.297 INFO:tasks.workunit.client.0.smithi038.stdout:7/648: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fc8 [0,4194304] 0 2024-03-24T00:49:01.299 INFO:tasks.workunit.client.1.smithi088.stdout:3/662: dwrite d3/d32/fad [0,4194304] 0 2024-03-24T00:49:01.299 INFO:tasks.workunit.client.1.smithi088.stdout:3/663: dread - d3/d67/f9d zero size 2024-03-24T00:49:01.301 INFO:tasks.workunit.client.1.smithi088.stdout:0/742: dread d9/f80 [0,4194304] 0 2024-03-24T00:49:01.301 INFO:tasks.workunit.client.1.smithi088.stdout:4/685: symlink d5/d7/d72/lf2 0 2024-03-24T00:49:01.301 INFO:tasks.workunit.client.1.smithi088.stdout:7/631: symlink d3/d32/d34/d9c/d4b/da7/dc5/lcb 0 2024-03-24T00:49:01.303 INFO:tasks.workunit.client.1.smithi088.stdout:1/729: unlink d6/d23/daf/d95/d5b/d5e/la2 0 2024-03-24T00:49:01.304 INFO:tasks.workunit.client.0.smithi038.stdout:9/757: creat d12/d28/d9c/ddd/df5/ff7 x:0 0 0 2024-03-24T00:49:01.306 INFO:tasks.workunit.client.1.smithi088.stdout:8/709: getdents d9/dc/d11/d24/d29/d39/d84 0 2024-03-24T00:49:01.306 INFO:tasks.workunit.client.1.smithi088.stdout:9/722: dread d1/d22/f43 [0,4194304] 0 2024-03-24T00:49:01.309 INFO:tasks.workunit.client.0.smithi038.stdout:8/694: rename d0/d45/l67 to d0/d1d/d8c/le7 0 2024-03-24T00:49:01.309 INFO:tasks.workunit.client.0.smithi038.stdout:8/695: write d0/d1d/d27/d4d/fce [1450758,23529] 0 2024-03-24T00:49:01.325 INFO:tasks.workunit.client.0.smithi038.stdout:1/739: mkdir da/dd/d22/d5c/dda/de6/df8 0 2024-03-24T00:49:01.346 INFO:tasks.workunit.client.1.smithi088.stdout:5/748: dwrite d1/dd/f13 [0,4194304] 0 2024-03-24T00:49:01.350 INFO:tasks.workunit.client.0.smithi038.stdout:6/735: sync 2024-03-24T00:49:01.350 INFO:tasks.workunit.client.0.smithi038.stdout:6/736: write d1/da/d1b/d1d/fd5 [660790,54178] 0 2024-03-24T00:49:01.353 INFO:tasks.workunit.client.0.smithi038.stdout:7/649: symlink d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/dcc/lf3 0 2024-03-24T00:49:01.357 INFO:tasks.workunit.client.0.smithi038.stdout:3/678: rename d3/d9/dc/da6/ldf to d3/d30/d5d/d7c/le5 0 2024-03-24T00:49:01.359 INFO:tasks.workunit.client.0.smithi038.stdout:5/625: sync 2024-03-24T00:49:01.361 INFO:tasks.workunit.client.1.smithi088.stdout:4/686: rmdir d5/d13/d18/d24/db3 39 2024-03-24T00:49:01.361 INFO:tasks.workunit.client.1.smithi088.stdout:8/710: write d9/dc/d11/d24/d29/d36/f88 [5030981,71987] 0 2024-03-24T00:49:01.361 INFO:tasks.workunit.client.1.smithi088.stdout:4/687: write d5/d13/d18/f79 [1019416,46567] 0 2024-03-24T00:49:01.361 INFO:tasks.workunit.client.1.smithi088.stdout:4/688: truncate d5/d13/d18/f19 829356 0 2024-03-24T00:49:01.369 INFO:tasks.workunit.client.1.smithi088.stdout:3/664: dread d3/de/f17 [4194304,4194304] 0 2024-03-24T00:49:01.386 INFO:tasks.workunit.client.0.smithi038.stdout:2/605: dwrite d4/d55/fc3 [4194304,4194304] 0 2024-03-24T00:49:01.396 INFO:tasks.workunit.client.1.smithi088.stdout:6/738: dwrite db/d1d/d50/d90/f4d [0,4194304] 0 2024-03-24T00:49:01.396 INFO:tasks.workunit.client.1.smithi088.stdout:6/739: chown db/d1d/d23/d5c/c5d 209797536 1 2024-03-24T00:49:01.400 INFO:tasks.workunit.client.1.smithi088.stdout:0/743: dwrite d9/dc/f49 [0,4194304] 0 2024-03-24T00:49:01.401 INFO:tasks.workunit.client.1.smithi088.stdout:0/744: chown d9/d15/l77 63544623 1 2024-03-24T00:49:01.401 INFO:tasks.workunit.client.1.smithi088.stdout:0/745: truncate d9/d26/d8f/dc7/fed 988388 0 2024-03-24T00:49:01.404 INFO:tasks.workunit.client.0.smithi038.stdout:9/758: dwrite d12/d81/f96 [4194304,4194304] 0 2024-03-24T00:49:01.406 INFO:tasks.workunit.client.0.smithi038.stdout:8/696: dwrite d0/f76 [0,4194304] 0 2024-03-24T00:49:01.418 INFO:tasks.workunit.client.0.smithi038.stdout:4/650: dwrite da/d12/d37/dd/f21 [0,4194304] 0 2024-03-24T00:49:01.419 INFO:tasks.workunit.client.1.smithi088.stdout:7/632: dwrite d3/d31/d56/f5b [0,4194304] 0 2024-03-24T00:49:01.419 INFO:tasks.workunit.client.1.smithi088.stdout:7/633: write d3/d32/d34/d9c/d4b/d74/d79/d7f/fb2 [809213,26208] 0 2024-03-24T00:49:01.420 INFO:tasks.workunit.client.0.smithi038.stdout:2/606: dread d4/f19 [0,4194304] 0 2024-03-24T00:49:01.429 INFO:tasks.workunit.client.1.smithi088.stdout:5/749: dwrite d1/dd/f13 [0,4194304] 0 2024-03-24T00:49:01.439 INFO:tasks.workunit.client.0.smithi038.stdout:1/740: dwrite da/dd/d2c/d5a/f92 [0,4194304] 0 2024-03-24T00:49:01.441 INFO:tasks.workunit.client.1.smithi088.stdout:1/730: dwrite f2 [0,4194304] 0 2024-03-24T00:49:01.445 INFO:tasks.workunit.client.0.smithi038.stdout:7/650: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fc7 [0,4194304] 0 2024-03-24T00:49:01.465 INFO:tasks.workunit.client.0.smithi038.stdout:6/737: rmdir d1/da/d1b 39 2024-03-24T00:49:01.465 INFO:tasks.workunit.client.0.smithi038.stdout:6/738: readlink d1/da/d1b/l45 0 2024-03-24T00:49:01.466 INFO:tasks.workunit.client.0.smithi038.stdout:6/739: stat d1/da/d1b/d25/f2a 0 2024-03-24T00:49:01.466 INFO:tasks.workunit.client.0.smithi038.stdout:6/740: stat d1/da/d61/da2/cdc 0 2024-03-24T00:49:01.466 INFO:tasks.workunit.client.0.smithi038.stdout:0/609: sync 2024-03-24T00:49:01.469 INFO:tasks.workunit.client.1.smithi088.stdout:9/723: mknod d1/d10/d29/d85/ceb 0 2024-03-24T00:49:01.477 INFO:tasks.workunit.client.1.smithi088.stdout:8/711: link d9/dc/d11/d24/d29/d5b/db9/fc5 d9/dc/d11/d80/db0/fef 0 2024-03-24T00:49:01.479 INFO:tasks.workunit.client.0.smithi038.stdout:6/741: write d1/da/d1b/d1d/d3c/f86 [2741267,126338] 0 2024-03-24T00:49:01.479 INFO:tasks.workunit.client.0.smithi038.stdout:6/742: truncate d1/fe8 747240 0 2024-03-24T00:49:01.480 INFO:tasks.workunit.client.0.smithi038.stdout:6/743: chown d1/da/d1b/d25/d49/d4e/c7c 447942 1 2024-03-24T00:49:01.495 INFO:tasks.workunit.client.1.smithi088.stdout:8/712: read d9/dc/d11/d24/f58 [2864279,96194] 0 2024-03-24T00:49:01.497 INFO:tasks.workunit.client.1.smithi088.stdout:3/665: mkdir d3/d6/d19/d2a/dc2/dd5 0 2024-03-24T00:49:01.497 INFO:tasks.workunit.client.1.smithi088.stdout:3/666: truncate d3/de/d6f/d89/fa0 218068 0 2024-03-24T00:49:01.497 INFO:tasks.workunit.client.1.smithi088.stdout:3/667: readlink d3/d6/d27/d3c/d80/l86 0 2024-03-24T00:49:01.497 INFO:tasks.workunit.client.1.smithi088.stdout:3/668: chown d3/d6/lb8 94193 1 2024-03-24T00:49:01.503 INFO:tasks.workunit.client.1.smithi088.stdout:3/669: dread d3/de/d6f/d89/fa0 [0,4194304] 0 2024-03-24T00:49:01.515 INFO:tasks.workunit.client.1.smithi088.stdout:4/689: symlink d5/d7/d15/lf3 0 2024-03-24T00:49:01.525 INFO:tasks.workunit.client.1.smithi088.stdout:3/670: dread d3/d6/d27/d3c/faa [0,4194304] 0 2024-03-24T00:49:01.525 INFO:tasks.workunit.client.1.smithi088.stdout:3/671: chown d3/de/c10 14549415 1 2024-03-24T00:49:01.529 INFO:tasks.workunit.client.0.smithi038.stdout:8/697: truncate d0/d45/d37/d5f/d68/f91 2597406 0 2024-03-24T00:49:01.529 INFO:tasks.workunit.client.0.smithi038.stdout:8/698: creat d0/d45/d66/da6/fe8 x:0 0 0 2024-03-24T00:49:01.532 INFO:tasks.workunit.client.1.smithi088.stdout:4/690: dread d5/d13/d18/d24/d3d/f55 [0,4194304] 0 2024-03-24T00:49:01.534 INFO:tasks.workunit.client.1.smithi088.stdout:6/740: creat db/fe4 x:0 0 0 2024-03-24T00:49:01.541 INFO:tasks.workunit.client.1.smithi088.stdout:0/746: creat d9/dc/de/d7e/d91/fee x:0 0 0 2024-03-24T00:49:01.541 INFO:tasks.workunit.client.1.smithi088.stdout:3/672: dread d3/de/d6f/d85/d49/f54 [4194304,4194304] 0 2024-03-24T00:49:01.571 INFO:tasks.workunit.client.0.smithi038.stdout:4/651: rmdir da/d12/d37/dd/d15/d6d/d8f/dca 0 2024-03-24T00:49:01.597 INFO:tasks.workunit.client.0.smithi038.stdout:2/607: dread d4/d55/d88/f8a [0,4194304] 0 2024-03-24T00:49:01.613 INFO:tasks.workunit.client.0.smithi038.stdout:8/699: dwrite d0/d1d/d27/d4d/f75 [0,4194304] 0 2024-03-24T00:49:01.621 INFO:tasks.workunit.client.1.smithi088.stdout:7/634: truncate d3/f20 5978912 0 2024-03-24T00:49:01.621 INFO:tasks.workunit.client.1.smithi088.stdout:7/635: fsync d3/d32/d34/d9c/d4b/f6c 0 2024-03-24T00:49:01.622 INFO:tasks.workunit.client.1.smithi088.stdout:7/636: chown d3/d31/f7a 1 1 2024-03-24T00:49:01.652 INFO:tasks.workunit.client.1.smithi088.stdout:7/637: dread d3/f2c [0,4194304] 0 2024-03-24T00:49:01.663 INFO:tasks.workunit.client.0.smithi038.stdout:2/608: dwrite d4/d18/d38/d63/f7e [0,4194304] 0 2024-03-24T00:49:01.663 INFO:tasks.workunit.client.0.smithi038.stdout:2/609: stat d4/f66 0 2024-03-24T00:49:01.689 INFO:tasks.workunit.client.1.smithi088.stdout:1/731: unlink d6/d23/f33 0 2024-03-24T00:49:01.690 INFO:tasks.workunit.client.1.smithi088.stdout:1/732: write d6/d27/db0/de3/fec [562689,120153] 0 2024-03-24T00:49:01.692 INFO:tasks.workunit.client.0.smithi038.stdout:5/626: creat d6/d19/d20/fcf x:0 0 0 2024-03-24T00:49:01.692 INFO:tasks.workunit.client.0.smithi038.stdout:5/627: read - d6/d1e/fcc zero size 2024-03-24T00:49:01.704 INFO:tasks.workunit.client.0.smithi038.stdout:6/744: getdents d1 0 2024-03-24T00:49:01.704 INFO:tasks.workunit.client.0.smithi038.stdout:6/745: chown d1/da/d61/d79/fa3 1 1 2024-03-24T00:49:01.712 INFO:tasks.workunit.client.1.smithi088.stdout:9/724: creat d1/d10/d29/fec x:0 0 0 2024-03-24T00:49:01.712 INFO:tasks.workunit.client.1.smithi088.stdout:9/725: readlink d1/d10/d9b/dab/d7a/ld0 0 2024-03-24T00:49:01.713 INFO:tasks.workunit.client.1.smithi088.stdout:9/726: dread - d1/d10/d77/dc0/fd4 zero size 2024-03-24T00:49:01.713 INFO:tasks.workunit.client.1.smithi088.stdout:9/727: fdatasync d1/ff 0 2024-03-24T00:49:01.718 INFO:tasks.workunit.client.1.smithi088.stdout:8/713: link d9/dc/d11/d24/d29/d73/f76 d9/dc/d11/d24/d29/d36/d6a/d7b/d8f/ff0 0 2024-03-24T00:49:01.721 INFO:tasks.workunit.client.1.smithi088.stdout:4/691: mkdir d5/d13/d63/df4 0 2024-03-24T00:49:01.721 INFO:tasks.workunit.client.0.smithi038.stdout:4/652: stat da/d12/d37/dd/d15/d6d/l7f 0 2024-03-24T00:49:01.722 INFO:tasks.workunit.client.0.smithi038.stdout:4/653: write da/d12/d37/dd/d15/f17 [5180609,115232] 0 2024-03-24T00:49:01.723 INFO:tasks.workunit.client.1.smithi088.stdout:6/741: mkdir db/dd/d61/d6d/d74/de5 0 2024-03-24T00:49:01.725 INFO:tasks.workunit.client.0.smithi038.stdout:8/700: symlink d0/d1d/d20/d3c/d9e/da3/dc8/le9 0 2024-03-24T00:49:01.727 INFO:tasks.workunit.client.1.smithi088.stdout:0/747: symlink d9/dc/lef 0 2024-03-24T00:49:01.728 INFO:tasks.workunit.client.1.smithi088.stdout:0/748: fsync d9/dc/de/d19/f51 0 2024-03-24T00:49:01.728 INFO:tasks.workunit.client.1.smithi088.stdout:0/749: write d9/dc/de/d19/f51 [873098,77772] 0 2024-03-24T00:49:01.728 INFO:tasks.workunit.client.1.smithi088.stdout:0/750: readlink d9/l16 0 2024-03-24T00:49:01.728 INFO:tasks.workunit.client.0.smithi038.stdout:7/651: truncate d2/d2c/f80 2780178 0 2024-03-24T00:49:01.729 INFO:tasks.workunit.client.1.smithi088.stdout:3/673: mkdir d3/d6/d19/d2a/dc2/dd5/dd6 0 2024-03-24T00:49:01.731 INFO:tasks.workunit.client.1.smithi088.stdout:7/638: link d3/d31/f83 d3/d31/fcc 0 2024-03-24T00:49:01.735 INFO:tasks.workunit.client.1.smithi088.stdout:1/733: truncate d6/d27/f8e 8361124 0 2024-03-24T00:49:01.735 INFO:tasks.workunit.client.1.smithi088.stdout:9/728: rename d1/d22/d27/d39/c6b to d1/d10/d11/d23/ced 0 2024-03-24T00:49:01.735 INFO:tasks.workunit.client.0.smithi038.stdout:2/610: creat d4/d18/fcf x:0 0 0 2024-03-24T00:49:01.737 INFO:tasks.workunit.client.0.smithi038.stdout:5/628: rmdir d6/d19/d20/d64 39 2024-03-24T00:49:01.738 INFO:tasks.workunit.client.0.smithi038.stdout:5/629: fsync d6/d1e/fcc 0 2024-03-24T00:49:01.738 INFO:tasks.workunit.client.0.smithi038.stdout:5/630: chown d6/d16/d14 110 1 2024-03-24T00:49:01.740 INFO:tasks.workunit.client.1.smithi088.stdout:8/714: symlink d9/dc/d11/d24/d43/lf1 0 2024-03-24T00:49:01.740 INFO:tasks.workunit.client.0.smithi038.stdout:0/610: getdents d0/d1b/d2c 0 2024-03-24T00:49:01.741 INFO:tasks.workunit.client.1.smithi088.stdout:0/751: creat d9/dc/de/d19/d44/ff0 x:0 0 0 2024-03-24T00:49:01.743 INFO:tasks.workunit.client.0.smithi038.stdout:6/746: creat d1/d37/ff4 x:0 0 0 2024-03-24T00:49:01.744 INFO:tasks.workunit.client.0.smithi038.stdout:4/654: getdents da/d12/d24/d68/d6c 0 2024-03-24T00:49:01.749 INFO:tasks.workunit.client.1.smithi088.stdout:7/639: dread d3/d32/d34/d9c/d4b/d74/d79/f97 [0,4194304] 0 2024-03-24T00:49:01.756 INFO:tasks.workunit.client.0.smithi038.stdout:7/652: truncate d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fc8 2209559 0 2024-03-24T00:49:01.756 INFO:tasks.workunit.client.0.smithi038.stdout:7/653: write d2/d9/da/d23/d2b/d5b/d72/d92/f9e [4359307,8318] 0 2024-03-24T00:49:01.762 INFO:tasks.workunit.client.0.smithi038.stdout:2/611: creat d4/d18/d26/d37/d69/d83/fd0 x:0 0 0 2024-03-24T00:49:01.762 INFO:tasks.workunit.client.0.smithi038.stdout:2/612: creat d4/da3/fd1 x:0 0 0 2024-03-24T00:49:01.764 INFO:tasks.workunit.client.1.smithi088.stdout:9/729: dread d1/d22/d27/f8d [0,4194304] 0 2024-03-24T00:49:01.764 INFO:tasks.workunit.client.1.smithi088.stdout:9/730: fsync d1/f8 0 2024-03-24T00:49:01.765 INFO:tasks.workunit.client.1.smithi088.stdout:9/731: dread - d1/d15/d21/fc8 zero size 2024-03-24T00:49:01.768 INFO:tasks.workunit.client.1.smithi088.stdout:8/715: mknod d9/dc/d11/d24/d43/d4a/dee/cf2 0 2024-03-24T00:49:01.774 INFO:tasks.workunit.client.1.smithi088.stdout:0/752: mknod d9/dc/de/d19/d44/d46/d65/cf1 0 2024-03-24T00:49:01.774 INFO:tasks.workunit.client.1.smithi088.stdout:0/753: write d9/dc/de/d19/d44/f4a [4409350,12705] 0 2024-03-24T00:49:01.775 INFO:tasks.workunit.client.0.smithi038.stdout:6/747: mkdir d1/d37/d84/df5 0 2024-03-24T00:49:01.778 INFO:tasks.workunit.client.0.smithi038.stdout:0/611: link d0/d2d/fbe d0/d14/fcf 0 2024-03-24T00:49:01.778 INFO:tasks.workunit.client.0.smithi038.stdout:0/612: write d0/d12/d15/d58/d64/f7d [4255878,30195] 0 2024-03-24T00:49:01.783 INFO:tasks.workunit.client.0.smithi038.stdout:7/654: mknod d2/d9/da/d23/d2b/d5b/d72/cf4 0 2024-03-24T00:49:01.785 INFO:tasks.workunit.client.1.smithi088.stdout:4/692: dwrite d5/d13/d18/d24/f53 [0,4194304] 0 2024-03-24T00:49:01.785 INFO:tasks.workunit.client.1.smithi088.stdout:4/693: fsync d5/d16/d1a/d22/f68 0 2024-03-24T00:49:01.786 INFO:tasks.workunit.client.0.smithi038.stdout:2/613: dread d4/d27/f39 [0,4194304] 0 2024-03-24T00:49:01.786 INFO:tasks.workunit.client.1.smithi088.stdout:4/694: write d5/d16/d37/fae [598898,130420] 0 2024-03-24T00:49:01.790 INFO:tasks.workunit.client.1.smithi088.stdout:9/732: symlink d1/d22/d27/d39/d98/da3/lee 0 2024-03-24T00:49:01.790 INFO:tasks.workunit.client.0.smithi038.stdout:0/613: symlink d0/d12/d42/ld0 0 2024-03-24T00:49:01.790 INFO:tasks.workunit.client.0.smithi038.stdout:0/614: fdatasync d0/d12/d42/d9d/d28/f8a 0 2024-03-24T00:49:01.793 INFO:tasks.workunit.client.1.smithi088.stdout:0/754: creat d9/dc/de/d19/d44/d46/ff2 x:0 0 0 2024-03-24T00:49:01.794 INFO:tasks.workunit.client.0.smithi038.stdout:4/655: dwrite da/d12/d37/dd/d15/d4c/d70/fa5 [0,4194304] 0 2024-03-24T00:49:01.794 INFO:tasks.workunit.client.0.smithi038.stdout:6/748: dread d1/da/d1b/d1d/d3c/f86 [0,4194304] 0 2024-03-24T00:49:01.797 INFO:tasks.workunit.client.0.smithi038.stdout:2/614: mknod d4/da3/cd2 0 2024-03-24T00:49:01.797 INFO:tasks.workunit.client.0.smithi038.stdout:2/615: creat d4/d18/d26/d37/d69/d83/fd3 x:0 0 0 2024-03-24T00:49:01.797 INFO:tasks.workunit.client.0.smithi038.stdout:2/616: truncate d4/d27/d54/d67/fce 599724 0 2024-03-24T00:49:01.799 INFO:tasks.workunit.client.1.smithi088.stdout:7/640: unlink d3/f2a 0 2024-03-24T00:49:01.800 INFO:tasks.workunit.client.0.smithi038.stdout:2/617: creat d4/d18/d26/d37/d69/d83/fd4 x:0 0 0 2024-03-24T00:49:01.801 INFO:tasks.workunit.client.1.smithi088.stdout:9/733: dread d1/d10/d77/fbb [0,4194304] 0 2024-03-24T00:49:01.801 INFO:tasks.workunit.client.0.smithi038.stdout:3/679: rename d3/c63 to d3/d9/d77/dd4/ce6 0 2024-03-24T00:49:01.802 INFO:tasks.workunit.client.0.smithi038.stdout:7/655: mknod d2/d2c/d90/cf5 0 2024-03-24T00:49:01.825 INFO:tasks.workunit.client.1.smithi088.stdout:8/716: truncate f1 1674550 0 2024-03-24T00:49:01.830 INFO:tasks.workunit.client.0.smithi038.stdout:9/759: rename d12/d28/da8/fbf to d12/d28/d3a/d8f/ff8 0 2024-03-24T00:49:01.836 INFO:tasks.workunit.client.1.smithi088.stdout:6/742: rename db/dc5/dc8/l45 to db/d1d/d23/d5c/le6 0 2024-03-24T00:49:01.840 INFO:tasks.workunit.client.1.smithi088.stdout:8/717: creat d9/dc/d11/d80/ff3 x:0 0 0 2024-03-24T00:49:01.841 INFO:tasks.workunit.client.0.smithi038.stdout:8/701: rename d0/d10/ddf to d0/d1d/d27/d4d/daf/dea 0 2024-03-24T00:49:01.843 INFO:tasks.workunit.client.1.smithi088.stdout:9/734: mkdir d1/d10/d29/def 0 2024-03-24T00:49:01.843 INFO:tasks.workunit.client.1.smithi088.stdout:9/735: chown d1/d10/d29/fc5 776 1 2024-03-24T00:49:01.849 INFO:tasks.workunit.client.1.smithi088.stdout:4/695: rename d5/f4b to d5/d13/d18/d24/ff5 0 2024-03-24T00:49:01.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/656: dread da/d12/f4e [0,4194304] 0 2024-03-24T00:49:01.866 INFO:tasks.workunit.client.0.smithi038.stdout:0/615: rename d0/d12/d42/d44/c80 to d0/d12/d15/d58/d64/da7/dbb/dcd/cd1 0 2024-03-24T00:49:01.866 INFO:tasks.workunit.client.0.smithi038.stdout:8/702: truncate d0/d45/d66/f40 3493311 0 2024-03-24T00:49:01.867 INFO:tasks.workunit.client.1.smithi088.stdout:8/718: creat d9/dc/d11/d24/d43/dd8/ff4 x:0 0 0 2024-03-24T00:49:01.867 INFO:tasks.workunit.client.1.smithi088.stdout:8/719: write d9/dc/d11/d24/d29/d5b/d8b/dd3/fed [981064,92878] 0 2024-03-24T00:49:01.870 INFO:tasks.workunit.client.1.smithi088.stdout:9/736: mkdir d1/d22/d27/df0 0 2024-03-24T00:49:01.873 INFO:tasks.workunit.client.0.smithi038.stdout:8/703: rename d0/d1d/d27/d6b/c8f to d0/d45/d37/ceb 0 2024-03-24T00:49:01.879 INFO:tasks.workunit.client.1.smithi088.stdout:7/641: rename d3/f1f to d3/d32/d76/d8f/fcd 0 2024-03-24T00:49:01.884 INFO:tasks.workunit.client.1.smithi088.stdout:8/720: mkdir d9/d6b/df5 0 2024-03-24T00:49:01.891 INFO:tasks.workunit.client.1.smithi088.stdout:9/737: creat d1/d10/d29/def/ff1 x:0 0 0 2024-03-24T00:49:01.891 INFO:tasks.workunit.client.1.smithi088.stdout:4/696: rename d5/d17/f2a to d5/d13/d18/d24/ddb/ff6 0 2024-03-24T00:49:01.891 INFO:tasks.workunit.client.1.smithi088.stdout:4/697: chown d5/d16/d37/de7 13936331 1 2024-03-24T00:49:01.891 INFO:tasks.workunit.client.1.smithi088.stdout:7/642: creat d3/d32/d34/d9c/d4b/da7/dc5/fce x:0 0 0 2024-03-24T00:49:01.894 INFO:tasks.workunit.client.1.smithi088.stdout:6/743: creat db/dd/d61/d6d/d74/d88/fe7 x:0 0 0 2024-03-24T00:49:01.895 INFO:tasks.workunit.client.1.smithi088.stdout:6/744: stat db/dd/d89/dad 0 2024-03-24T00:49:01.897 INFO:tasks.workunit.client.1.smithi088.stdout:9/738: unlink d1/d10/fad 0 2024-03-24T00:49:01.897 INFO:tasks.workunit.client.1.smithi088.stdout:9/739: readlink d1/d10/l7c 0 2024-03-24T00:49:01.897 INFO:tasks.workunit.client.1.smithi088.stdout:4/698: truncate d5/f46 298689 0 2024-03-24T00:49:01.898 INFO:tasks.workunit.client.0.smithi038.stdout:7/656: dwrite d2/d9/da/d23/f39 [0,4194304] 0 2024-03-24T00:49:01.901 INFO:tasks.workunit.client.0.smithi038.stdout:2/618: dwrite d4/d27/f91 [0,4194304] 0 2024-03-24T00:49:01.901 INFO:tasks.workunit.client.0.smithi038.stdout:2/619: fdatasync d4/d18/da1/fa4 0 2024-03-24T00:49:01.907 INFO:tasks.workunit.client.0.smithi038.stdout:7/657: fdatasync d2/d2c/f32 0 2024-03-24T00:49:01.907 INFO:tasks.workunit.client.0.smithi038.stdout:6/749: dwrite d1/da/d1b/d1d/f24 [0,4194304] 0 2024-03-24T00:49:01.907 INFO:tasks.workunit.client.0.smithi038.stdout:6/750: dread - d1/da/d1b/d25/fbc zero size 2024-03-24T00:49:01.907 INFO:tasks.workunit.client.0.smithi038.stdout:6/751: chown d1/da/d1b/d25/d49/d4e/ddf/fe9 64021293 1 2024-03-24T00:49:01.907 INFO:tasks.workunit.client.1.smithi088.stdout:9/740: rename d1/d22/d27/f97 to d1/d10/d2c/ff2 0 2024-03-24T00:49:01.907 INFO:tasks.workunit.client.1.smithi088.stdout:9/741: chown d1/d10/d11/d23/c95 0 1 2024-03-24T00:49:01.908 INFO:tasks.workunit.client.0.smithi038.stdout:2/620: symlink d4/d27/d54/d67/ld5 0 2024-03-24T00:49:01.910 INFO:tasks.workunit.client.0.smithi038.stdout:9/760: dwrite d12/d28/d3a/d26/f4a [0,4194304] 0 2024-03-24T00:49:01.911 INFO:tasks.workunit.client.0.smithi038.stdout:6/752: rename d1/da/d1b/f6a to d1/da/d61/ff6 0 2024-03-24T00:49:01.911 INFO:tasks.workunit.client.1.smithi088.stdout:9/742: mkdir d1/d15/d21/df3 0 2024-03-24T00:49:01.914 INFO:tasks.workunit.client.1.smithi088.stdout:8/721: dread d9/f63 [0,4194304] 0 2024-03-24T00:49:01.919 INFO:tasks.workunit.client.1.smithi088.stdout:8/722: unlink d9/dc/d11/d24/d29/da9/ccc 0 2024-03-24T00:49:01.922 INFO:tasks.workunit.client.0.smithi038.stdout:9/761: mkdir d12/d1a/d5f/df9 0 2024-03-24T00:49:01.922 INFO:tasks.workunit.client.0.smithi038.stdout:6/753: symlink d1/da/d1b/d25/d49/d72/lf7 0 2024-03-24T00:49:01.922 INFO:tasks.workunit.client.0.smithi038.stdout:9/762: readlink d12/d28/d6c/lab 0 2024-03-24T00:49:01.922 INFO:tasks.workunit.client.0.smithi038.stdout:7/658: dread d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fe0 [0,4194304] 0 2024-03-24T00:49:01.923 INFO:tasks.workunit.client.1.smithi088.stdout:8/723: creat d9/dc/d11/d24/d29/da9/de7/ff6 x:0 0 0 2024-03-24T00:49:01.932 INFO:tasks.workunit.client.0.smithi038.stdout:3/680: dwrite d3/d9/d77/fc9 [0,4194304] 0 2024-03-24T00:49:01.932 INFO:tasks.workunit.client.0.smithi038.stdout:0/616: dwrite d0/d12/d42/d9d/d28/f39 [4194304,4194304] 0 2024-03-24T00:49:01.933 INFO:tasks.workunit.client.0.smithi038.stdout:2/621: read d4/d27/d2f/d30/f8d [2870157,87004] 0 2024-03-24T00:49:01.936 INFO:tasks.workunit.client.0.smithi038.stdout:9/763: unlink d12/d28/d71/d99/fc9 0 2024-03-24T00:49:01.937 INFO:tasks.workunit.client.0.smithi038.stdout:4/657: dwrite da/d12/d37/dd/d15/d6d/f4b [0,4194304] 0 2024-03-24T00:49:01.937 INFO:tasks.workunit.client.1.smithi088.stdout:7/643: rmdir d3/d32/d34/d9c/d4b 39 2024-03-24T00:49:01.940 INFO:tasks.workunit.client.0.smithi038.stdout:6/754: creat d1/ff8 x:0 0 0 2024-03-24T00:49:01.941 INFO:tasks.workunit.client.0.smithi038.stdout:3/681: symlink d3/d30/d5d/d6a/le7 0 2024-03-24T00:49:01.942 INFO:tasks.workunit.client.1.smithi088.stdout:7/644: mknod d3/d32/d34/d9c/d4b/ccf 0 2024-03-24T00:49:01.942 INFO:tasks.workunit.client.1.smithi088.stdout:7/645: fsync d3/d32/d34/f96 0 2024-03-24T00:49:01.942 INFO:tasks.workunit.client.1.smithi088.stdout:7/646: fsync d3/d32/d34/d9c/d4b/f6c 0 2024-03-24T00:49:01.942 INFO:tasks.workunit.client.1.smithi088.stdout:7/647: chown d3/d31/d56/l65 2 1 2024-03-24T00:49:01.944 INFO:tasks.workunit.client.0.smithi038.stdout:2/622: symlink d4/d27/d2f/ld6 0 2024-03-24T00:49:01.948 INFO:tasks.workunit.client.0.smithi038.stdout:9/764: creat d12/d28/d9c/ddd/ffa x:0 0 0 2024-03-24T00:49:01.950 INFO:tasks.workunit.client.1.smithi088.stdout:4/699: dwrite d5/d16/f2e [0,4194304] 0 2024-03-24T00:49:01.950 INFO:tasks.workunit.client.1.smithi088.stdout:4/700: chown d5/f46 3949 1 2024-03-24T00:49:01.951 INFO:tasks.workunit.client.0.smithi038.stdout:3/682: link d3/d2a/la7 d3/d5a/le8 0 2024-03-24T00:49:01.953 INFO:tasks.workunit.client.1.smithi088.stdout:4/701: mknod d5/d13/d18/d24/de5/cf7 0 2024-03-24T00:49:01.958 INFO:tasks.workunit.client.0.smithi038.stdout:2/623: creat d4/d27/d54/fd7 x:0 0 0 2024-03-24T00:49:01.958 INFO:tasks.workunit.client.0.smithi038.stdout:3/683: link d3/d9/dc/da6/db5/d19/d82/c88 d3/d2a/ce9 0 2024-03-24T00:49:01.958 INFO:tasks.workunit.client.0.smithi038.stdout:3/684: dread - d3/d30/fac zero size 2024-03-24T00:49:01.961 INFO:tasks.workunit.client.0.smithi038.stdout:4/658: dread da/d12/d37/dd/d15/d6d/f4b [0,4194304] 0 2024-03-24T00:49:01.961 INFO:tasks.workunit.client.0.smithi038.stdout:1/741: sync 2024-03-24T00:49:01.962 INFO:tasks.workunit.client.1.smithi088.stdout:7/648: dread d3/d32/d34/f37 [4194304,4194304] 0 2024-03-24T00:49:01.962 INFO:tasks.workunit.client.1.smithi088.stdout:7/649: readlink d3/d32/l53 0 2024-03-24T00:49:01.963 INFO:tasks.workunit.client.1.smithi088.stdout:7/650: read - d3/d32/d34/d9c/d4b/d74/d79/d7f/fb7 zero size 2024-03-24T00:49:01.965 INFO:tasks.workunit.client.0.smithi038.stdout:1/742: write da/dd/d7a/d82/d98/d9e/fa7 [282911,26002] 0 2024-03-24T00:49:01.970 INFO:tasks.workunit.client.1.smithi088.stdout:7/651: mknod d3/d31/cd0 0 2024-03-24T00:49:01.970 INFO:tasks.workunit.client.0.smithi038.stdout:4/659: dread da/d12/d24/d5f/fd4 [0,4194304] 0 2024-03-24T00:49:01.970 INFO:tasks.workunit.client.0.smithi038.stdout:1/743: creat da/dd/d7a/d82/d98/d14/ff9 x:0 0 0 2024-03-24T00:49:01.973 INFO:tasks.workunit.client.1.smithi088.stdout:5/750: mknod d1/d10/d1b/d2f/d57/cf5 0 2024-03-24T00:49:01.974 INFO:tasks.workunit.client.1.smithi088.stdout:7/652: mkdir d3/d32/d34/d9c/d4b/d73/dd1 0 2024-03-24T00:49:01.976 INFO:tasks.workunit.client.0.smithi038.stdout:4/660: mknod da/d12/d29/d5b/ce2 0 2024-03-24T00:49:01.976 INFO:tasks.workunit.client.0.smithi038.stdout:4/661: write da/d12/d24/d68/d6c/dce/fde [683376,98157] 0 2024-03-24T00:49:01.978 INFO:tasks.workunit.client.1.smithi088.stdout:5/751: dread d1/d10/d1b/d2f/fe7 [0,4194304] 0 2024-03-24T00:49:01.980 INFO:tasks.workunit.client.0.smithi038.stdout:1/744: truncate da/f21 1919293 0 2024-03-24T00:49:01.980 INFO:tasks.workunit.client.0.smithi038.stdout:1/745: creat da/dd/d2c/d40/de9/ffa x:0 0 0 2024-03-24T00:49:01.987 INFO:tasks.workunit.client.0.smithi038.stdout:8/704: getdents d0/d10 0 2024-03-24T00:49:01.988 INFO:tasks.workunit.client.1.smithi088.stdout:6/745: dwrite db/d8e/fbc [0,4194304] 0 2024-03-24T00:49:01.992 INFO:tasks.workunit.client.0.smithi038.stdout:1/746: mkdir da/dd/d2c/db6/dfb 0 2024-03-24T00:49:01.992 INFO:tasks.workunit.client.0.smithi038.stdout:1/747: dread - da/dd/d22/d94/fce zero size 2024-03-24T00:49:01.997 INFO:tasks.workunit.client.1.smithi088.stdout:7/653: dread d3/d32/d34/d9c/d4b/f6c [0,4194304] 0 2024-03-24T00:49:02.001 INFO:tasks.workunit.client.0.smithi038.stdout:1/748: symlink da/dd/d2c/d41/lfc 0 2024-03-24T00:49:02.001 INFO:tasks.workunit.client.0.smithi038.stdout:1/749: readlink da/dd/d2c/d5a/l9f 0 2024-03-24T00:49:02.002 INFO:tasks.workunit.client.0.smithi038.stdout:1/750: fdatasync da/f56 0 2024-03-24T00:49:02.002 INFO:tasks.workunit.client.0.smithi038.stdout:1/751: dread - da/dd/d2c/d41/d60/dae/fc8 zero size 2024-03-24T00:49:02.009 INFO:tasks.workunit.client.1.smithi088.stdout:6/746: rmdir db/dc5/dc8/d43/d5b 39 2024-03-24T00:49:02.009 INFO:tasks.workunit.client.1.smithi088.stdout:6/747: dread - db/dd/d61/d6d/fe1 zero size 2024-03-24T00:49:02.009 INFO:tasks.workunit.client.1.smithi088.stdout:6/748: chown db/d1d/d50/d90/d56/c4f 1601081 1 2024-03-24T00:49:02.014 INFO:tasks.workunit.client.0.smithi038.stdout:1/752: creat da/dd/d2c/d41/ffd x:0 0 0 2024-03-24T00:49:02.025 INFO:tasks.workunit.client.1.smithi088.stdout:6/749: write db/d3d/d83/d72/f76 [4158962,85183] 0 2024-03-24T00:49:02.025 INFO:tasks.workunit.client.1.smithi088.stdout:6/750: chown db/d1d/d23/c8c 2540 1 2024-03-24T00:49:02.031 INFO:tasks.workunit.client.1.smithi088.stdout:6/751: creat db/dd/d61/d6d/d74/d88/da1/fe8 x:0 0 0 2024-03-24T00:49:02.035 INFO:tasks.workunit.client.1.smithi088.stdout:6/752: creat db/dd/d87/ddf/fe9 x:0 0 0 2024-03-24T00:49:02.044 INFO:tasks.workunit.client.1.smithi088.stdout:6/753: creat db/d3d/d83/fea x:0 0 0 2024-03-24T00:49:02.044 INFO:tasks.workunit.client.1.smithi088.stdout:9/743: dwrite d1/d22/d72/faf [0,4194304] 0 2024-03-24T00:49:02.045 INFO:tasks.workunit.client.0.smithi038.stdout:9/765: dwrite d12/d28/d3a/d26/f4a [0,4194304] 0 2024-03-24T00:49:02.045 INFO:tasks.workunit.client.0.smithi038.stdout:9/766: readlink d12/d1a/d5f/lc4 0 2024-03-24T00:49:02.045 INFO:tasks.workunit.client.0.smithi038.stdout:9/767: write d12/d28/d3a/fe0 [162305,24065] 0 2024-03-24T00:49:02.051 INFO:tasks.workunit.client.0.smithi038.stdout:9/768: getdents d12/d28/d9c 0 2024-03-24T00:49:02.066 INFO:tasks.workunit.client.1.smithi088.stdout:2/683: sync 2024-03-24T00:49:02.067 INFO:tasks.workunit.client.0.smithi038.stdout:4/662: dwrite da/d12/d24/d5f/d9a/f92 [0,4194304] 0 2024-03-24T00:49:02.067 INFO:tasks.workunit.client.0.smithi038.stdout:4/663: dread - da/d12/d24/d5f/d8a/fc4 zero size 2024-03-24T00:49:02.067 INFO:tasks.workunit.client.0.smithi038.stdout:2/624: dwrite d4/d18/d26/f2c [0,4194304] 0 2024-03-24T00:49:02.070 INFO:tasks.workunit.client.1.smithi088.stdout:7/654: dwrite d3/d32/d34/d9c/d4b/d73/f93 [0,4194304] 0 2024-03-24T00:49:02.070 INFO:tasks.workunit.client.1.smithi088.stdout:7/655: write d3/d32/d34/d9c/d4b/d74/d79/d7f/fb7 [779716,19731] 0 2024-03-24T00:49:02.072 INFO:tasks.workunit.client.1.smithi088.stdout:3/674: sync 2024-03-24T00:49:02.072 INFO:tasks.workunit.client.1.smithi088.stdout:1/734: sync 2024-03-24T00:49:02.072 INFO:tasks.workunit.client.1.smithi088.stdout:3/675: fdatasync d3/d1a/d1e/d2f/f36 0 2024-03-24T00:49:02.081 INFO:tasks.workunit.client.1.smithi088.stdout:7/656: creat d3/d32/d34/fd2 x:0 0 0 2024-03-24T00:49:02.082 INFO:tasks.workunit.client.0.smithi038.stdout:8/705: dwrite d0/d15/fbd [0,4194304] 0 2024-03-24T00:49:02.082 INFO:tasks.workunit.client.0.smithi038.stdout:2/625: dread d4/d27/d54/d53/f64 [0,4194304] 0 2024-03-24T00:49:02.085 INFO:tasks.workunit.client.0.smithi038.stdout:4/664: mkdir da/d12/d29/d5b/d73/db4/de3 0 2024-03-24T00:49:02.091 INFO:tasks.workunit.client.1.smithi088.stdout:9/744: dread d1/d10/d29/d6d/f57 [0,4194304] 0 2024-03-24T00:49:02.091 INFO:tasks.workunit.client.1.smithi088.stdout:9/745: chown d1/d10/l42 966 1 2024-03-24T00:49:02.092 INFO:tasks.workunit.client.1.smithi088.stdout:3/676: mkdir d3/d67/dc3/dd7 0 2024-03-24T00:49:02.095 INFO:tasks.workunit.client.1.smithi088.stdout:5/752: dwrite d1/dd/d6f/da1/d3f/d88/d8f/d69/f82 [0,4194304] 0 2024-03-24T00:49:02.098 INFO:tasks.workunit.client.1.smithi088.stdout:2/684: unlink db/dd/d96/d4e/c68 0 2024-03-24T00:49:02.098 INFO:tasks.workunit.client.1.smithi088.stdout:2/685: getdents db/dd/d96/d99/db2/d88/d8e 0 2024-03-24T00:49:02.098 INFO:tasks.workunit.client.1.smithi088.stdout:7/657: mkdir d3/d32/d34/d9c/d4b/d74/dd3 0 2024-03-24T00:49:02.098 INFO:tasks.workunit.client.1.smithi088.stdout:7/658: readlink d3/d32/la6 0 2024-03-24T00:49:02.099 INFO:tasks.workunit.client.1.smithi088.stdout:7/659: write d3/fbb [310423,84270] 0 2024-03-24T00:49:02.099 INFO:tasks.workunit.client.1.smithi088.stdout:7/660: stat d3/d32/d34/d9c/d4b/f4c 0 2024-03-24T00:49:02.100 INFO:tasks.workunit.client.0.smithi038.stdout:8/706: creat d0/d1d/d27/d4d/fec x:0 0 0 2024-03-24T00:49:02.102 INFO:tasks.workunit.client.1.smithi088.stdout:1/735: getdents d6/d27/db0/de3 0 2024-03-24T00:49:02.103 INFO:tasks.workunit.client.0.smithi038.stdout:9/769: dwrite d12/d81/d84/d88/fd4 [0,4194304] 0 2024-03-24T00:49:02.103 INFO:tasks.workunit.client.0.smithi038.stdout:9/770: read d12/f42 [357589,13129] 0 2024-03-24T00:49:02.109 INFO:tasks.workunit.client.0.smithi038.stdout:2/626: rename d4/d27/f33 to d4/d18/d26/d37/fd8 0 2024-03-24T00:49:02.109 INFO:tasks.workunit.client.1.smithi088.stdout:0/755: sync 2024-03-24T00:49:02.111 INFO:tasks.workunit.client.1.smithi088.stdout:3/677: symlink d3/de/d6f/ld8 0 2024-03-24T00:49:02.113 INFO:tasks.workunit.client.1.smithi088.stdout:6/754: dwrite db/dd/d61/d6d/d74/d88/da1/fe8 [0,4194304] 0 2024-03-24T00:49:02.114 INFO:tasks.workunit.client.1.smithi088.stdout:6/755: truncate db/d1d/d50/d90/f8f 1302476 0 2024-03-24T00:49:02.114 INFO:tasks.workunit.client.0.smithi038.stdout:8/707: truncate d0/d45/f89 1068910 0 2024-03-24T00:49:02.119 INFO:tasks.workunit.client.0.smithi038.stdout:8/708: write d0/d1d/d27/d4d/fce [407484,127146] 0 2024-03-24T00:49:02.119 INFO:tasks.workunit.client.0.smithi038.stdout:8/709: dread - d0/d1d/d20/fdd zero size 2024-03-24T00:49:02.129 INFO:tasks.workunit.client.1.smithi088.stdout:6/756: read db/dd/d61/d6d/d74/fc0 [3281010,34122] 0 2024-03-24T00:49:02.129 INFO:tasks.workunit.client.1.smithi088.stdout:2/686: getdents db/dd/d96/d99/db2/d88/dc6 0 2024-03-24T00:49:02.130 INFO:tasks.workunit.client.1.smithi088.stdout:2/687: dread - db/dd/fcf zero size 2024-03-24T00:49:02.130 INFO:tasks.workunit.client.0.smithi038.stdout:9/771: link d12/d41/dc0/fd1 d12/d41/d57/de1/ffb 0 2024-03-24T00:49:02.130 INFO:tasks.workunit.client.1.smithi088.stdout:8/724: dwrite d9/dc/f44 [0,4194304] 0 2024-03-24T00:49:02.132 INFO:tasks.workunit.client.1.smithi088.stdout:7/661: link d3/d31/f83 d3/d32/d76/d8f/fd4 0 2024-03-24T00:49:02.134 INFO:tasks.workunit.client.0.smithi038.stdout:2/627: getdents d4/d27/d2f 0 2024-03-24T00:49:02.134 INFO:tasks.workunit.client.0.smithi038.stdout:2/628: creat d4/d27/d2f/d30/d3e/d7c/fd9 x:0 0 0 2024-03-24T00:49:02.136 INFO:tasks.workunit.client.1.smithi088.stdout:1/736: truncate d6/d23/daf/d95/d5b/d5d/fa9 2279218 0 2024-03-24T00:49:02.137 INFO:tasks.workunit.client.0.smithi038.stdout:8/710: creat d0/d1d/d27/d4d/daf/fed x:0 0 0 2024-03-24T00:49:02.138 INFO:tasks.workunit.client.0.smithi038.stdout:8/711: write d0/d45/d37/d5f/d68/fa0 [36279,87699] 0 2024-03-24T00:49:02.141 INFO:tasks.workunit.client.0.smithi038.stdout:0/617: dwrite d0/f69 [0,4194304] 0 2024-03-24T00:49:02.141 INFO:tasks.workunit.client.0.smithi038.stdout:0/618: chown d0/c5b 710614 1 2024-03-24T00:49:02.141 INFO:tasks.workunit.client.0.smithi038.stdout:0/619: creat d0/d14/d84/fd2 x:0 0 0 2024-03-24T00:49:02.143 INFO:tasks.workunit.client.1.smithi088.stdout:0/756: mknod d9/d26/d8f/da1/d62/d9b/db0/cf3 0 2024-03-24T00:49:02.143 INFO:tasks.workunit.client.1.smithi088.stdout:0/757: chown d9/dc/de/d19/d84/ca2 7114 1 2024-03-24T00:49:02.158 INFO:tasks.workunit.client.1.smithi088.stdout:6/757: mkdir db/d3d/d83/deb 0 2024-03-24T00:49:02.158 INFO:tasks.workunit.client.1.smithi088.stdout:6/758: chown db/dc5/dc8/d43/daf/cc1 155 1 2024-03-24T00:49:02.158 INFO:tasks.workunit.client.1.smithi088.stdout:6/759: chown db/d3d/d83/c6b 888 1 2024-03-24T00:49:02.168 INFO:tasks.workunit.client.0.smithi038.stdout:2/629: mknod d4/d27/d54/dab/cda 0 2024-03-24T00:49:02.168 INFO:tasks.workunit.client.0.smithi038.stdout:8/712: link d0/d45/c9c d0/d1d/d27/d6b/da4/dda/cee 0 2024-03-24T00:49:02.177 INFO:tasks.workunit.client.1.smithi088.stdout:8/725: creat d9/dc/d11/d24/d29/d36/ff7 x:0 0 0 2024-03-24T00:49:02.177 INFO:tasks.workunit.client.1.smithi088.stdout:7/662: rmdir d3/d31 39 2024-03-24T00:49:02.177 INFO:tasks.workunit.client.1.smithi088.stdout:7/663: fdatasync d3/d32/d34/d9c/d4b/da7/fc9 0 2024-03-24T00:49:02.178 INFO:tasks.workunit.client.1.smithi088.stdout:9/746: dwrite d1/d10/d29/d6d/f57 [0,4194304] 0 2024-03-24T00:49:02.179 INFO:tasks.workunit.client.1.smithi088.stdout:9/747: write d1/d10/d29/fe2 [588471,109320] 0 2024-03-24T00:49:02.179 INFO:tasks.workunit.client.1.smithi088.stdout:5/753: dwrite d1/d43/d4e/fc7 [0,4194304] 0 2024-03-24T00:49:02.185 INFO:tasks.workunit.client.0.smithi038.stdout:0/620: link d0/d12/c19 d0/d12/d42/d5e/db0/cd3 0 2024-03-24T00:49:02.193 INFO:tasks.workunit.client.0.smithi038.stdout:8/713: link d0/d15/fbd d0/d45/d37/d5f/db7/dcc/fef 0 2024-03-24T00:49:02.204 INFO:tasks.workunit.client.1.smithi088.stdout:1/737: mknod d6/d23/daf/d95/d5b/d80/cee 0 2024-03-24T00:49:02.206 INFO:tasks.workunit.client.1.smithi088.stdout:0/758: rmdir d9/dc/de/d19/d44/d46/d65/d2a/d2e 39 2024-03-24T00:49:02.206 INFO:tasks.workunit.client.1.smithi088.stdout:0/759: write d9/dc/de/d19/d44/d46/d65/f68 [5583629,25404] 0 2024-03-24T00:49:02.208 INFO:tasks.workunit.client.1.smithi088.stdout:3/678: rmdir d3/d6/d58 39 2024-03-24T00:49:02.212 INFO:tasks.workunit.client.0.smithi038.stdout:0/621: dread d0/f7 [0,4194304] 0 2024-03-24T00:49:02.216 INFO:tasks.workunit.client.0.smithi038.stdout:8/714: mknod d0/d1d/d27/d4d/daf/cf0 0 2024-03-24T00:49:02.216 INFO:tasks.workunit.client.0.smithi038.stdout:3/685: dwrite d3/d30/d5d/d6a/f8b [0,4194304] 0 2024-03-24T00:49:02.217 INFO:tasks.workunit.client.0.smithi038.stdout:4/665: dwrite da/d12/d37/dd/f21 [4194304,4194304] 0 2024-03-24T00:49:02.217 INFO:tasks.workunit.client.0.smithi038.stdout:4/666: chown da/d12/d24/d68/d6c/da6/lc2 6228 1 2024-03-24T00:49:02.218 INFO:tasks.workunit.client.0.smithi038.stdout:4/667: fdatasync da/d12/d24/d5f/fda 0 2024-03-24T00:49:02.218 INFO:tasks.workunit.client.1.smithi088.stdout:2/688: dwrite db/d21/d35/f79 [0,4194304] 0 2024-03-24T00:49:02.218 INFO:tasks.workunit.client.1.smithi088.stdout:3/679: dread d3/de/d6f/d85/f15 [0,4194304] 0 2024-03-24T00:49:02.230 INFO:tasks.workunit.client.1.smithi088.stdout:3/680: dread d3/d1a/d1e/d2f/d45/f6b [0,4194304] 0 2024-03-24T00:49:02.230 INFO:tasks.workunit.client.0.smithi038.stdout:2/630: rename d4/d27/d2f/d30/d3e to d4/d27/ddb 0 2024-03-24T00:49:02.234 INFO:tasks.workunit.client.1.smithi088.stdout:8/726: truncate d9/d6b/f6c 853781 0 2024-03-24T00:49:02.234 INFO:tasks.workunit.client.1.smithi088.stdout:8/727: fdatasync d9/dc/d11/d24/f5e 0 2024-03-24T00:49:02.250 INFO:tasks.workunit.client.1.smithi088.stdout:7/664: mknod d3/d32/d76/da2/cd5 0 2024-03-24T00:49:02.250 INFO:tasks.workunit.client.1.smithi088.stdout:7/665: write d3/d32/d34/d9c/d4b/d74/d79/d7f/fb2 [1445913,53260] 0 2024-03-24T00:49:02.251 INFO:tasks.workunit.client.1.smithi088.stdout:2/689: dread db/d21/d35/f7d [0,4194304] 0 2024-03-24T00:49:02.257 INFO:tasks.workunit.client.0.smithi038.stdout:9/772: dwrite d12/d55/fb5 [4194304,4194304] 0 2024-03-24T00:49:02.257 INFO:tasks.workunit.client.1.smithi088.stdout:9/748: rename d1/d10/d9b/dab/d7a/ld0 to d1/d10/d11/lf4 0 2024-03-24T00:49:02.258 INFO:tasks.workunit.client.1.smithi088.stdout:4/702: sync 2024-03-24T00:49:02.258 INFO:tasks.workunit.client.0.smithi038.stdout:9/773: truncate d12/d28/d3a/d26/f86 5218012 0 2024-03-24T00:49:02.268 INFO:tasks.workunit.client.1.smithi088.stdout:3/681: write d3/de/f16 [5861784,122714] 0 2024-03-24T00:49:02.272 INFO:tasks.workunit.client.0.smithi038.stdout:9/774: dread d12/d81/d84/fa1 [0,4194304] 0 2024-03-24T00:49:02.272 INFO:tasks.workunit.client.1.smithi088.stdout:8/728: dwrite d9/dc/dce/ddc/fe9 [4194304,4194304] 0 2024-03-24T00:49:02.281 INFO:tasks.workunit.client.1.smithi088.stdout:4/703: dread d5/d13/d18/d24/d3d/f42 [0,4194304] 0 2024-03-24T00:49:02.282 INFO:tasks.workunit.client.1.smithi088.stdout:5/754: link d1/d19/d1e/l53 d1/dd/d6f/lf6 0 2024-03-24T00:49:02.287 INFO:tasks.workunit.client.0.smithi038.stdout:0/622: getdents d0/d14/d88/db8 0 2024-03-24T00:49:02.288 INFO:tasks.workunit.client.1.smithi088.stdout:1/738: rmdir d6/d23/daf/d95/d5b/d93 39 2024-03-24T00:49:02.300 INFO:tasks.workunit.client.0.smithi038.stdout:3/686: getdents d3/d30 0 2024-03-24T00:49:02.303 INFO:tasks.workunit.client.1.smithi088.stdout:7/666: mkdir d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/dd6 0 2024-03-24T00:49:02.303 INFO:tasks.workunit.client.1.smithi088.stdout:2/690: symlink db/d21/d35/dcb/ldb 0 2024-03-24T00:49:02.303 INFO:tasks.workunit.client.1.smithi088.stdout:2/691: stat db/dc/l23 0 2024-03-24T00:49:02.303 INFO:tasks.workunit.client.1.smithi088.stdout:2/692: truncate db/dd/d96/d99/db2/fb4 782778 0 2024-03-24T00:49:02.303 INFO:tasks.workunit.client.1.smithi088.stdout:2/693: dread - db/dd/d96/d99/db2/f65 zero size 2024-03-24T00:49:02.304 INFO:tasks.workunit.client.0.smithi038.stdout:2/631: rename d4/d55/d88/fb9 to d4/d27/d8f/d9e/fdc 0 2024-03-24T00:49:02.304 INFO:tasks.workunit.client.1.smithi088.stdout:4/704: dread f4 [0,4194304] 0 2024-03-24T00:49:02.304 INFO:tasks.workunit.client.1.smithi088.stdout:4/705: readlink d5/d16/d1a/d22/dc6/lef 0 2024-03-24T00:49:02.307 INFO:tasks.workunit.client.1.smithi088.stdout:0/760: rename d9/d26/d8f/cbe to d9/dc/de/d19/cf4 0 2024-03-24T00:49:02.310 INFO:tasks.workunit.client.1.smithi088.stdout:3/682: rename d3/d1a/d1e/d2f/d45/db5/cc9 to d3/d1a/d1e/d2f/d45/db5/cd9 0 2024-03-24T00:49:02.315 INFO:tasks.workunit.client.0.smithi038.stdout:9/775: mkdir d12/d81/d84/dec/dfc 0 2024-03-24T00:49:02.323 INFO:tasks.workunit.client.0.smithi038.stdout:3/687: dread d3/f25 [0,4194304] 0 2024-03-24T00:49:02.323 INFO:tasks.workunit.client.0.smithi038.stdout:3/688: dread - d3/d30/fac zero size 2024-03-24T00:49:02.323 INFO:tasks.workunit.client.1.smithi088.stdout:6/760: rmdir db/dc5/dc8/d43/dbf 0 2024-03-24T00:49:02.327 INFO:tasks.workunit.client.1.smithi088.stdout:2/694: mkdir db/dd/d96/d99/db2/ddc 0 2024-03-24T00:49:02.332 INFO:tasks.workunit.client.1.smithi088.stdout:4/706: symlink d5/d13/d18/lf8 0 2024-03-24T00:49:02.336 INFO:tasks.workunit.client.0.smithi038.stdout:3/689: truncate d3/d9/d77/f7f 14381 0 2024-03-24T00:49:02.338 INFO:tasks.workunit.client.1.smithi088.stdout:6/761: dread db/d1d/d50/d90/f8f [0,4194304] 0 2024-03-24T00:49:02.339 INFO:tasks.workunit.client.1.smithi088.stdout:1/739: rename d6/d23/daf/d95/d5b/d5d/cca to d6/d27/cef 0 2024-03-24T00:49:02.344 INFO:tasks.workunit.client.0.smithi038.stdout:8/715: dwrite d0/d45/d37/d5f/db7/dcc/fef [0,4194304] 0 2024-03-24T00:49:02.344 INFO:tasks.workunit.client.0.smithi038.stdout:9/776: rmdir d12/d1a/d80 39 2024-03-24T00:49:02.347 INFO:tasks.workunit.client.1.smithi088.stdout:3/683: creat d3/de/d6f/fda x:0 0 0 2024-03-24T00:49:02.361 INFO:tasks.workunit.client.1.smithi088.stdout:4/707: read d5/d16/f7b [1878813,95139] 0 2024-03-24T00:49:02.365 INFO:tasks.workunit.client.1.smithi088.stdout:2/695: unlink db/dd/d96/d99/db2/d88/dc6/cd5 0 2024-03-24T00:49:02.368 INFO:tasks.workunit.client.1.smithi088.stdout:9/749: dwrite d1/d10/f1e [4194304,4194304] 0 2024-03-24T00:49:02.369 INFO:tasks.workunit.client.1.smithi088.stdout:0/761: dwrite d9/d26/d8f/dc7/fed [0,4194304] 0 2024-03-24T00:49:02.374 INFO:tasks.workunit.client.0.smithi038.stdout:0/623: dwrite d0/d12/d42/d9d/d28/f8a [0,4194304] 0 2024-03-24T00:49:02.374 INFO:tasks.workunit.client.0.smithi038.stdout:0/624: dread - d0/d12/d92/fab zero size 2024-03-24T00:49:02.388 INFO:tasks.workunit.client.1.smithi088.stdout:1/740: write d6/d27/d67/fa3 [1565771,44078] 0 2024-03-24T00:49:02.391 INFO:tasks.workunit.client.1.smithi088.stdout:5/755: dwrite d1/dd/fe3 [0,4194304] 0 2024-03-24T00:49:02.391 INFO:tasks.workunit.client.1.smithi088.stdout:2/696: mknod db/dd/dc9/cdd 0 2024-03-24T00:49:02.391 INFO:tasks.workunit.client.1.smithi088.stdout:2/697: fsync db/dd/d96/d99/db2/d88/fd6 0 2024-03-24T00:49:02.392 INFO:tasks.workunit.client.1.smithi088.stdout:2/698: creat db/d21/d35/d49/fde x:0 0 0 2024-03-24T00:49:02.396 INFO:tasks.workunit.client.1.smithi088.stdout:7/667: dwrite d3/d31/f70 [0,4194304] 0 2024-03-24T00:49:02.396 INFO:tasks.workunit.client.1.smithi088.stdout:6/762: dread db/d1d/d50/d90/d56/f51 [0,4194304] 0 2024-03-24T00:49:02.400 INFO:tasks.workunit.client.1.smithi088.stdout:9/750: rename d1/d10/d29/f45 to d1/db5/ff5 0 2024-03-24T00:49:02.404 INFO:tasks.workunit.client.1.smithi088.stdout:0/762: mkdir d9/dc/de/d7e/de6/df5 0 2024-03-24T00:49:02.407 INFO:tasks.workunit.client.1.smithi088.stdout:5/756: symlink d1/d19/db9/lf7 0 2024-03-24T00:49:02.408 INFO:tasks.workunit.client.1.smithi088.stdout:5/757: write d1/d10/d1b/d2f/d57/fa5 [1188231,92518] 0 2024-03-24T00:49:02.408 INFO:tasks.workunit.client.1.smithi088.stdout:5/758: chown d1/dd/d6f/da1/f50 596 1 2024-03-24T00:49:02.408 INFO:tasks.workunit.client.1.smithi088.stdout:5/759: fsync d1/d19/d1e/f47 0 2024-03-24T00:49:02.418 INFO:tasks.workunit.client.1.smithi088.stdout:7/668: creat d3/d32/d34/d9c/d4b/d73/dd1/fd7 x:0 0 0 2024-03-24T00:49:02.418 INFO:tasks.workunit.client.1.smithi088.stdout:6/763: symlink db/d1d/d50/d90/d56/lec 0 2024-03-24T00:49:02.419 INFO:tasks.workunit.client.1.smithi088.stdout:6/764: chown db/d1d/d50/d90/d55/d99/fd2 158 1 2024-03-24T00:49:02.419 INFO:tasks.workunit.client.1.smithi088.stdout:0/763: mknod d9/d26/d8f/dc7/ddd/cf6 0 2024-03-24T00:49:02.420 INFO:tasks.workunit.client.1.smithi088.stdout:5/760: mkdir d1/d43/d4e/df8 0 2024-03-24T00:49:02.424 INFO:tasks.workunit.client.1.smithi088.stdout:6/765: rename db/d1d/d23/c8c to db/d1d/d23/ced 0 2024-03-24T00:49:02.425 INFO:tasks.workunit.client.0.smithi038.stdout:8/716: dread d0/d15/f54 [0,4194304] 0 2024-03-24T00:49:02.426 INFO:tasks.workunit.client.1.smithi088.stdout:0/764: mkdir d9/dc/df7 0 2024-03-24T00:49:02.428 INFO:tasks.workunit.client.0.smithi038.stdout:8/717: mknod d0/d45/d37/d5f/db7/cf1 0 2024-03-24T00:49:02.428 INFO:tasks.workunit.client.0.smithi038.stdout:8/718: fsync d0/d10/fd3 0 2024-03-24T00:49:02.439 INFO:tasks.workunit.client.0.smithi038.stdout:9/777: dwrite d12/d41/d57/f89 [4194304,4194304] 0 2024-03-24T00:49:02.439 INFO:tasks.workunit.client.0.smithi038.stdout:9/778: write d12/d28/d3a/d26/f3d [818684,15025] 0 2024-03-24T00:49:02.439 INFO:tasks.workunit.client.0.smithi038.stdout:9/779: chown d12/d41/fd0 473704 1 2024-03-24T00:49:02.443 INFO:tasks.workunit.client.0.smithi038.stdout:9/780: mknod d12/d28/d3a/d8f/cfd 0 2024-03-24T00:49:02.446 INFO:tasks.workunit.client.1.smithi088.stdout:2/699: write db/d21/d73/f83 [1656493,88061] 0 2024-03-24T00:49:02.449 INFO:tasks.workunit.client.0.smithi038.stdout:9/781: unlink d12/d28/d71/f8b 0 2024-03-24T00:49:02.450 INFO:tasks.workunit.client.0.smithi038.stdout:9/782: write d12/d28/d3a/f3f [5013987,47488] 0 2024-03-24T00:49:02.450 INFO:tasks.workunit.client.0.smithi038.stdout:9/783: read - d12/d41/d57/f98 zero size 2024-03-24T00:49:02.454 INFO:tasks.workunit.client.0.smithi038.stdout:3/690: dwrite d3/d9/d77/f85 [4194304,4194304] 0 2024-03-24T00:49:02.461 INFO:tasks.workunit.client.1.smithi088.stdout:4/708: dwrite d5/d16/f7b [0,4194304] 0 2024-03-24T00:49:02.464 INFO:tasks.workunit.client.0.smithi038.stdout:9/784: rename d12/d28/d3a/d26/d79 to d12/d41/d57/dd3/dfe 0 2024-03-24T00:49:02.465 INFO:tasks.workunit.client.1.smithi088.stdout:1/741: dwrite d6/d23/daf/d95/d58/f48 [0,4194304] 0 2024-03-24T00:49:02.465 INFO:tasks.workunit.client.1.smithi088.stdout:1/742: stat d6/d23/daf/d95/d5b/d5d/d61/fdb 0 2024-03-24T00:49:02.465 INFO:tasks.workunit.client.0.smithi038.stdout:0/625: dwrite d0/d2d/f65 [0,4194304] 0 2024-03-24T00:49:02.469 INFO:tasks.workunit.client.1.smithi088.stdout:3/684: dwrite f2 [4194304,4194304] 0 2024-03-24T00:49:02.477 INFO:tasks.workunit.client.1.smithi088.stdout:9/751: dwrite d1/d15/d21/fc8 [0,4194304] 0 2024-03-24T00:49:02.477 INFO:tasks.workunit.client.1.smithi088.stdout:9/752: dread - d1/d22/d27/fbd zero size 2024-03-24T00:49:02.478 INFO:tasks.workunit.client.1.smithi088.stdout:5/761: dwrite d1/d19/d48/fae [0,4194304] 0 2024-03-24T00:49:02.479 INFO:tasks.workunit.client.1.smithi088.stdout:5/762: fdatasync d1/d10/ff3 0 2024-03-24T00:49:02.479 INFO:tasks.workunit.client.1.smithi088.stdout:5/763: fsync d1/d19/d48/dc2/fc9 0 2024-03-24T00:49:02.487 INFO:tasks.workunit.client.1.smithi088.stdout:4/709: dread d5/d7/d15/fde [0,4194304] 0 2024-03-24T00:49:02.488 INFO:tasks.workunit.client.0.smithi038.stdout:0/626: dread d0/f2 [0,4194304] 0 2024-03-24T00:49:02.488 INFO:tasks.workunit.client.0.smithi038.stdout:0/627: chown d0/d12/d42/d5e/caa 5 1 2024-03-24T00:49:02.490 INFO:tasks.workunit.client.1.smithi088.stdout:1/743: getdents d6/d23/daf/d95/d5b 0 2024-03-24T00:49:02.492 INFO:tasks.workunit.client.1.smithi088.stdout:7/669: rename d3/d32/d34/d9c/d4b/d73/dd1 to d3/d32/d34/d9c/d4b/d73/dab/dd8 0 2024-03-24T00:49:02.492 INFO:tasks.workunit.client.1.smithi088.stdout:7/670: fsync d3/fc 0 2024-03-24T00:49:02.502 INFO:tasks.workunit.client.1.smithi088.stdout:9/753: creat d1/d10/d11/d23/ff6 x:0 0 0 2024-03-24T00:49:02.513 INFO:tasks.workunit.client.1.smithi088.stdout:0/765: dwrite d9/d26/f5a [4194304,4194304] 0 2024-03-24T00:49:02.513 INFO:tasks.workunit.client.1.smithi088.stdout:7/671: dread d3/d32/f42 [0,4194304] 0 2024-03-24T00:49:02.514 INFO:tasks.workunit.client.0.smithi038.stdout:9/785: write d12/d81/fd8 [404032,62870] 0 2024-03-24T00:49:02.514 INFO:tasks.workunit.client.0.smithi038.stdout:9/786: stat d12/d28/d71/ce5 0 2024-03-24T00:49:02.514 INFO:tasks.workunit.client.0.smithi038.stdout:9/787: write d12/fe3 [4466,44918] 0 2024-03-24T00:49:02.516 INFO:tasks.workunit.client.0.smithi038.stdout:1/753: sync 2024-03-24T00:49:02.516 INFO:tasks.workunit.client.0.smithi038.stdout:7/659: sync 2024-03-24T00:49:02.516 INFO:tasks.workunit.client.0.smithi038.stdout:1/754: creat da/d3c/d86/ffe x:0 0 0 2024-03-24T00:49:02.518 INFO:tasks.workunit.client.1.smithi088.stdout:5/764: rmdir d1/d43 39 2024-03-24T00:49:02.519 INFO:tasks.workunit.client.1.smithi088.stdout:1/744: unlink d6/d23/daf/d95/d5b/l82 0 2024-03-24T00:49:02.521 INFO:tasks.workunit.client.0.smithi038.stdout:6/755: sync 2024-03-24T00:49:02.521 INFO:tasks.workunit.client.0.smithi038.stdout:5/631: sync 2024-03-24T00:49:02.521 INFO:tasks.workunit.client.0.smithi038.stdout:5/632: fsync d6/d19/d20/d42/d79/fb6 0 2024-03-24T00:49:02.521 INFO:tasks.workunit.client.0.smithi038.stdout:5/633: write d6/d19/d20/d42/d43/fa4 [443898,58571] 0 2024-03-24T00:49:02.523 INFO:tasks.workunit.client.0.smithi038.stdout:0/628: read d0/d12/d42/d44/f93 [3448593,31732] 0 2024-03-24T00:49:02.525 INFO:tasks.workunit.client.1.smithi088.stdout:6/766: rename db/dc5/dc8/d43/f28 to db/dc5/dc8/fee 0 2024-03-24T00:49:02.526 INFO:tasks.workunit.client.0.smithi038.stdout:8/719: dwrite d0/d45/d37/f48 [0,4194304] 0 2024-03-24T00:49:02.530 INFO:tasks.workunit.client.1.smithi088.stdout:8/729: sync 2024-03-24T00:49:02.530 INFO:tasks.workunit.client.1.smithi088.stdout:3/685: dwrite d3/d6/dca/f84 [0,4194304] 0 2024-03-24T00:49:02.531 INFO:tasks.workunit.client.1.smithi088.stdout:2/700: dwrite db/d21/f33 [0,4194304] 0 2024-03-24T00:49:02.531 INFO:tasks.workunit.client.1.smithi088.stdout:2/701: fdatasync db/d21/d35/d49/fd8 0 2024-03-24T00:49:02.532 INFO:tasks.workunit.client.0.smithi038.stdout:1/755: truncate da/dd/d7a/d82/d98/f54 4074484 0 2024-03-24T00:49:02.534 INFO:tasks.workunit.client.1.smithi088.stdout:0/766: unlink d9/d26/d8f/da1/fba 0 2024-03-24T00:49:02.540 INFO:tasks.workunit.client.0.smithi038.stdout:6/756: unlink d1/f7 0 2024-03-24T00:49:02.540 INFO:tasks.workunit.client.0.smithi038.stdout:6/757: chown d1/da/d1b/d2f/dc0/fc4 131292412 1 2024-03-24T00:49:02.548 INFO:tasks.workunit.client.0.smithi038.stdout:0/629: truncate d0/d2d/f8d 2508173 0 2024-03-24T00:49:02.548 INFO:tasks.workunit.client.0.smithi038.stdout:0/630: write d0/d2d/f6f [2585611,42123] 0 2024-03-24T00:49:02.550 INFO:tasks.workunit.client.1.smithi088.stdout:6/767: truncate db/dd/d61/fc9 1935283 0 2024-03-24T00:49:02.552 INFO:tasks.workunit.client.0.smithi038.stdout:9/788: rename d12/d55/d56/l7f to d12/d28/d6c/lff 0 2024-03-24T00:49:02.552 INFO:tasks.workunit.client.1.smithi088.stdout:5/765: dread d1/d10/f17 [0,4194304] 0 2024-03-24T00:49:02.554 INFO:tasks.workunit.client.0.smithi038.stdout:8/720: mkdir d0/d45/d37/d5f/db7/d62/df2 0 2024-03-24T00:49:02.558 INFO:tasks.workunit.client.0.smithi038.stdout:1/756: dread da/f23 [0,4194304] 0 2024-03-24T00:49:02.558 INFO:tasks.workunit.client.0.smithi038.stdout:0/631: truncate d0/f7 426493 0 2024-03-24T00:49:02.559 INFO:tasks.workunit.client.1.smithi088.stdout:3/686: mkdir d3/d6/d19/ddb 0 2024-03-24T00:49:02.561 INFO:tasks.workunit.client.1.smithi088.stdout:4/710: dwrite d5/d16/d1a/d22/dc6/fdc [0,4194304] 0 2024-03-24T00:49:02.561 INFO:tasks.workunit.client.1.smithi088.stdout:4/711: stat d5/d16 0 2024-03-24T00:49:02.561 INFO:tasks.workunit.client.1.smithi088.stdout:2/702: link db/dc/l50 db/d21/d35/d49/ldf 0 2024-03-24T00:49:02.564 INFO:tasks.workunit.client.1.smithi088.stdout:7/672: dwrite d3/d31/d56/f7b [0,4194304] 0 2024-03-24T00:49:02.565 INFO:tasks.workunit.client.0.smithi038.stdout:7/660: rename d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fd8 to d2/d9/da/d23/d2b/ff6 0 2024-03-24T00:49:02.566 INFO:tasks.workunit.client.1.smithi088.stdout:0/767: symlink d9/dc/de/d19/d44/d46/d65/d2a/d2e/lf8 0 2024-03-24T00:49:02.569 INFO:tasks.workunit.client.0.smithi038.stdout:1/757: creat da/dd/d4a/d8b/fff x:0 0 0 2024-03-24T00:49:02.569 INFO:tasks.workunit.client.0.smithi038.stdout:1/758: dread - da/ddc/fe2 zero size 2024-03-24T00:49:02.574 INFO:tasks.workunit.client.0.smithi038.stdout:0/632: mkdir d0/d12/d15/d58/d64/da7/dbb/dd4 0 2024-03-24T00:49:02.580 INFO:tasks.workunit.client.0.smithi038.stdout:8/721: symlink d0/d1d/lf3 0 2024-03-24T00:49:02.585 INFO:tasks.workunit.client.1.smithi088.stdout:0/768: dread d9/dc/de/d19/d44/d46/d65/fb5 [0,4194304] 0 2024-03-24T00:49:02.585 INFO:tasks.workunit.client.1.smithi088.stdout:0/769: chown d9/dc/de/d7e/d91/cc5 4391142 1 2024-03-24T00:49:02.586 INFO:tasks.workunit.client.0.smithi038.stdout:1/759: symlink da/dd/d7a/l100 0 2024-03-24T00:49:02.586 INFO:tasks.workunit.client.0.smithi038.stdout:1/760: dread - da/d3c/d86/fbd zero size 2024-03-24T00:49:02.592 INFO:tasks.workunit.client.0.smithi038.stdout:0/633: creat d0/d12/d15/d58/d64/da7/dbb/dcd/fd5 x:0 0 0 2024-03-24T00:49:02.597 INFO:tasks.workunit.client.0.smithi038.stdout:5/634: dwrite d6/d19/d20/d42/d43/d50/f81 [0,4194304] 0 2024-03-24T00:49:02.597 INFO:tasks.workunit.client.0.smithi038.stdout:5/635: read d6/d19/d20/d42/d43/d4c/f74 [151131,12221] 0 2024-03-24T00:49:02.605 INFO:tasks.workunit.client.1.smithi088.stdout:1/745: dwrite d6/fab [0,4194304] 0 2024-03-24T00:49:02.605 INFO:tasks.workunit.client.1.smithi088.stdout:1/746: truncate d6/d6e/f79 841827 0 2024-03-24T00:49:02.613 INFO:tasks.workunit.client.0.smithi038.stdout:0/634: rmdir d0/d12/d15/d58/d64/da7/dbb 39 2024-03-24T00:49:02.613 INFO:tasks.workunit.client.0.smithi038.stdout:0/635: truncate d0/d12/d15/d58/d64/da7/dbb/dcd/fd5 877301 0 2024-03-24T00:49:02.616 INFO:tasks.workunit.client.0.smithi038.stdout:5/636: mknod d6/d1e/d97/db0/cd0 0 2024-03-24T00:49:02.617 INFO:tasks.workunit.client.1.smithi088.stdout:8/730: rmdir d9/dc 39 2024-03-24T00:49:02.617 INFO:tasks.workunit.client.1.smithi088.stdout:8/731: stat d9/dc/d11/d24/d29/l69 0 2024-03-24T00:49:02.618 INFO:tasks.workunit.client.0.smithi038.stdout:8/722: mknod d0/d1d/cf4 0 2024-03-24T00:49:02.619 INFO:tasks.workunit.client.1.smithi088.stdout:6/768: creat db/d1d/d50/fef x:0 0 0 2024-03-24T00:49:02.619 INFO:tasks.workunit.client.1.smithi088.stdout:6/769: dread - db/dd/d61/fdb zero size 2024-03-24T00:49:02.619 INFO:tasks.workunit.client.0.smithi038.stdout:8/723: fdatasync d0/d15/f98 0 2024-03-24T00:49:02.623 INFO:tasks.workunit.client.1.smithi088.stdout:2/703: dwrite db/f87 [0,4194304] 0 2024-03-24T00:49:02.628 INFO:tasks.workunit.client.0.smithi038.stdout:6/758: dwrite d1/da/d1b/fa1 [0,4194304] 0 2024-03-24T00:49:02.628 INFO:tasks.workunit.client.0.smithi038.stdout:6/759: chown d1/da/d1b/d25/d49/d4e/d51 341 1 2024-03-24T00:49:02.628 INFO:tasks.workunit.client.0.smithi038.stdout:6/760: chown d1/c28 13759916 1 2024-03-24T00:49:02.628 INFO:tasks.workunit.client.0.smithi038.stdout:6/761: write d1/ff8 [601699,90401] 0 2024-03-24T00:49:02.628 INFO:tasks.workunit.client.0.smithi038.stdout:6/762: write d1/da/d1b/fb3 [279766,33107] 0 2024-03-24T00:49:02.634 INFO:tasks.workunit.client.0.smithi038.stdout:1/761: rename da/dd/d22/d5c to da/dd/d2c/d5a/d101 0 2024-03-24T00:49:02.635 INFO:tasks.workunit.client.0.smithi038.stdout:1/762: creat da/dd/d7a/d82/d98/d9e/f102 x:0 0 0 2024-03-24T00:49:02.635 INFO:tasks.workunit.client.1.smithi088.stdout:5/766: rmdir d1/dd/d6f/da1/d3f/dea 0 2024-03-24T00:49:02.635 INFO:tasks.workunit.client.1.smithi088.stdout:5/767: chown d1/dd/d6f/da1/d3f/d88/d8b/fbf 460002 1 2024-03-24T00:49:02.638 INFO:tasks.workunit.client.0.smithi038.stdout:8/724: write d0/f76 [1762672,105807] 0 2024-03-24T00:49:02.639 INFO:tasks.workunit.client.0.smithi038.stdout:5/637: mkdir d6/d1e/d97/dd1 0 2024-03-24T00:49:02.639 INFO:tasks.workunit.client.0.smithi038.stdout:5/638: dread - d6/d19/d20/d42/d43/d4c/f9d zero size 2024-03-24T00:49:02.641 INFO:tasks.workunit.client.1.smithi088.stdout:6/770: read db/dc5/dc8/d43/f34 [1486154,35650] 0 2024-03-24T00:49:02.641 INFO:tasks.workunit.client.1.smithi088.stdout:6/771: chown db/d1d/d50/d90/d56/l79 184021 1 2024-03-24T00:49:02.641 INFO:tasks.workunit.client.1.smithi088.stdout:6/772: chown db/d8e/fbc 20308307 1 2024-03-24T00:49:02.641 INFO:tasks.workunit.client.1.smithi088.stdout:6/773: chown db/dd/l25 88 1 2024-03-24T00:49:02.644 INFO:tasks.workunit.client.0.smithi038.stdout:5/639: write d6/d19/d20/d42/d43/fa4 [60512,103309] 0 2024-03-24T00:49:02.648 INFO:tasks.workunit.client.1.smithi088.stdout:3/687: creat d3/d6/dca/fdc x:0 0 0 2024-03-24T00:49:02.648 INFO:tasks.workunit.client.1.smithi088.stdout:3/688: chown d3/de/d6f/d89 254 1 2024-03-24T00:49:02.648 INFO:tasks.workunit.client.1.smithi088.stdout:3/689: readlink d3/de/d6f/ld8 0 2024-03-24T00:49:02.648 INFO:tasks.workunit.client.1.smithi088.stdout:3/690: fsync d3/de/d6f/f7a 0 2024-03-24T00:49:02.650 INFO:tasks.workunit.client.0.smithi038.stdout:5/640: write d6/d16/d67/f8f [1014540,31850] 0 2024-03-24T00:49:02.652 INFO:tasks.workunit.client.1.smithi088.stdout:4/712: symlink d5/d8b/lf9 0 2024-03-24T00:49:02.652 INFO:tasks.workunit.client.1.smithi088.stdout:6/774: dread db/d1d/d50/d90/f8f [0,4194304] 0 2024-03-24T00:49:02.652 INFO:tasks.workunit.client.1.smithi088.stdout:6/775: write db/d8e/f94 [997454,3146] 0 2024-03-24T00:49:02.654 INFO:tasks.workunit.client.1.smithi088.stdout:7/673: unlink d3/d31/d56/f5b 0 2024-03-24T00:49:02.655 INFO:tasks.workunit.client.1.smithi088.stdout:7/674: write d3/d32/d76/f87 [758756,94962] 0 2024-03-24T00:49:02.657 INFO:tasks.workunit.client.1.smithi088.stdout:0/770: unlink d9/dc/de/d19/d44/d46/d65/d2a/lc6 0 2024-03-24T00:49:02.658 INFO:tasks.workunit.client.1.smithi088.stdout:0/771: fsync d9/dc/de/d19/d44/ff0 0 2024-03-24T00:49:02.664 INFO:tasks.workunit.client.0.smithi038.stdout:6/763: truncate d1/da/f1a 1169089 0 2024-03-24T00:49:02.664 INFO:tasks.workunit.client.0.smithi038.stdout:6/764: chown d1/d37/d68/c7e 5053 1 2024-03-24T00:49:02.665 INFO:tasks.workunit.client.1.smithi088.stdout:2/704: dwrite db/d21/f5f [0,4194304] 0 2024-03-24T00:49:02.669 INFO:tasks.workunit.client.1.smithi088.stdout:8/732: mknod d9/d6b/df5/cf8 0 2024-03-24T00:49:02.678 INFO:tasks.workunit.client.0.smithi038.stdout:1/763: mkdir da/dd/d7a/d82/d98/d14/dc0/d103 0 2024-03-24T00:49:02.678 INFO:tasks.workunit.client.0.smithi038.stdout:1/764: readlink da/dd/d2c/d41/lfc 0 2024-03-24T00:49:02.685 INFO:tasks.workunit.client.0.smithi038.stdout:7/661: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d5d/f9d [0,4194304] 0 2024-03-24T00:49:02.690 INFO:tasks.workunit.client.0.smithi038.stdout:5/641: creat d6/d16/d21/d2d/fd2 x:0 0 0 2024-03-24T00:49:02.690 INFO:tasks.workunit.client.0.smithi038.stdout:5/642: read d6/f26 [1872500,118367] 0 2024-03-24T00:49:02.691 INFO:tasks.workunit.client.0.smithi038.stdout:6/765: mknod d1/da/d1b/d25/d49/d4e/cf9 0 2024-03-24T00:49:02.693 INFO:tasks.workunit.client.1.smithi088.stdout:5/768: creat d1/d10/d1b/d2f/d57/d9e/db1/ded/ff9 x:0 0 0 2024-03-24T00:49:02.693 INFO:tasks.workunit.client.1.smithi088.stdout:5/769: fsync d1/d10/d1b/d2e/fe1 0 2024-03-24T00:49:02.693 INFO:tasks.workunit.client.1.smithi088.stdout:5/770: creat d1/d19/ffa x:0 0 0 2024-03-24T00:49:02.695 INFO:tasks.workunit.client.0.smithi038.stdout:8/725: mknod d0/d1d/cf5 0 2024-03-24T00:49:02.700 INFO:tasks.workunit.client.1.smithi088.stdout:7/675: dwrite d3/d32/d34/d9c/d4b/f5e [0,4194304] 0 2024-03-24T00:49:02.704 INFO:tasks.workunit.client.1.smithi088.stdout:3/691: mknod d3/de/d6f/dcc/cdd 0 2024-03-24T00:49:02.704 INFO:tasks.workunit.client.0.smithi038.stdout:1/765: dread da/f58 [0,4194304] 0 2024-03-24T00:49:02.707 INFO:tasks.workunit.client.0.smithi038.stdout:9/789: dwrite d12/d81/fd8 [0,4194304] 0 2024-03-24T00:49:02.708 INFO:tasks.workunit.client.1.smithi088.stdout:6/776: rename db/dd/d89 to db/dd/d61/d6d/d74/df0 0 2024-03-24T00:49:02.715 INFO:tasks.workunit.client.1.smithi088.stdout:1/747: symlink d6/d23/daf/d95/d5b/lf0 0 2024-03-24T00:49:02.715 INFO:tasks.workunit.client.1.smithi088.stdout:3/692: dread d3/d6/f3b [0,4194304] 0 2024-03-24T00:49:02.716 INFO:tasks.workunit.client.1.smithi088.stdout:4/713: dwrite d5/d13/d18/f32 [0,4194304] 0 2024-03-24T00:49:02.716 INFO:tasks.workunit.client.1.smithi088.stdout:4/714: chown d5/d16/d1a/f50 0 1 2024-03-24T00:49:02.717 INFO:tasks.workunit.client.1.smithi088.stdout:4/715: creat d5/d16/d94/da6/ffa x:0 0 0 2024-03-24T00:49:02.717 INFO:tasks.workunit.client.1.smithi088.stdout:4/716: chown d5/d13/d18/d24/d3d/f77 1751 1 2024-03-24T00:49:02.717 INFO:tasks.workunit.client.1.smithi088.stdout:9/754: sync 2024-03-24T00:49:02.717 INFO:tasks.workunit.client.1.smithi088.stdout:9/755: getdents d1/dcf 0 2024-03-24T00:49:02.721 INFO:tasks.workunit.client.1.smithi088.stdout:8/733: link d9/dc/d11/d24/d29/d5b/d8b/dd3/fbd d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/ff9 0 2024-03-24T00:49:02.722 INFO:tasks.workunit.client.0.smithi038.stdout:4/668: sync 2024-03-24T00:49:02.732 INFO:tasks.workunit.client.0.smithi038.stdout:7/662: mknod d2/d9/da/d23/d2b/d5b/cf7 0 2024-03-24T00:49:02.732 INFO:tasks.workunit.client.0.smithi038.stdout:7/663: creat d2/d9/da/dc/d8a/ff8 x:0 0 0 2024-03-24T00:49:02.734 INFO:tasks.workunit.client.1.smithi088.stdout:5/771: creat d1/ffb x:0 0 0 2024-03-24T00:49:02.734 INFO:tasks.workunit.client.0.smithi038.stdout:4/669: dread da/d12/d24/d68/d6c/d7c/f90 [0,4194304] 0 2024-03-24T00:49:02.734 INFO:tasks.workunit.client.1.smithi088.stdout:7/676: dwrite d3/fbb [0,4194304] 0 2024-03-24T00:49:02.735 INFO:tasks.workunit.client.1.smithi088.stdout:7/677: readlink d3/d32/la6 0 2024-03-24T00:49:02.737 INFO:tasks.workunit.client.0.smithi038.stdout:6/766: link d1/da/d1b/d25/d49/d72/fe2 d1/d37/d84/ffa 0 2024-03-24T00:49:02.741 INFO:tasks.workunit.client.1.smithi088.stdout:0/772: rename d9/d15/l1e to d9/dc/de/d19/d44/d46/d65/d2a/d2e/lf9 0 2024-03-24T00:49:02.743 INFO:tasks.workunit.client.1.smithi088.stdout:7/678: dread d3/d31/f3c [0,4194304] 0 2024-03-24T00:49:02.745 INFO:tasks.workunit.client.1.smithi088.stdout:6/777: unlink db/dc5/dc8/d43/daf/cca 0 2024-03-24T00:49:02.745 INFO:tasks.workunit.client.0.smithi038.stdout:5/643: dwrite d6/d23/f53 [4194304,4194304] 0 2024-03-24T00:49:02.751 INFO:tasks.workunit.client.0.smithi038.stdout:8/726: symlink d0/d45/d37/d5f/db7/d62/df2/lf6 0 2024-03-24T00:49:02.751 INFO:tasks.workunit.client.1.smithi088.stdout:2/705: getdents db/dc 0 2024-03-24T00:49:02.751 INFO:tasks.workunit.client.0.smithi038.stdout:1/766: mkdir da/dd/d2c/d40/de9/d104 0 2024-03-24T00:49:02.754 INFO:tasks.workunit.client.0.smithi038.stdout:9/790: rmdir d12/d1a 39 2024-03-24T00:49:02.756 INFO:tasks.workunit.client.1.smithi088.stdout:1/748: fsync d6/d23/daf/d95/d5b/feb 0 2024-03-24T00:49:02.759 INFO:tasks.workunit.client.0.smithi038.stdout:2/632: sync 2024-03-24T00:49:02.769 INFO:tasks.workunit.client.0.smithi038.stdout:3/691: sync 2024-03-24T00:49:02.769 INFO:tasks.workunit.client.0.smithi038.stdout:4/670: mknod da/d12/d24/d68/d6c/ce4 0 2024-03-24T00:49:02.775 INFO:tasks.workunit.client.0.smithi038.stdout:8/727: creat d0/d45/d37/d5f/db7/dcc/ff7 x:0 0 0 2024-03-24T00:49:02.775 INFO:tasks.workunit.client.0.smithi038.stdout:8/728: write d0/d1d/d27/d6b/fc9 [867718,98291] 0 2024-03-24T00:49:02.775 INFO:tasks.workunit.client.0.smithi038.stdout:1/767: creat da/dd/d2c/d41/d60/f105 x:0 0 0 2024-03-24T00:49:02.776 INFO:tasks.workunit.client.1.smithi088.stdout:5/772: truncate d1/d10/d1b/d2e/fe1 103331 0 2024-03-24T00:49:02.780 INFO:tasks.workunit.client.0.smithi038.stdout:3/692: creat d3/d9/dc/da6/db5/d19/d82/fea x:0 0 0 2024-03-24T00:49:02.780 INFO:tasks.workunit.client.0.smithi038.stdout:3/693: dread - d3/d30/fac zero size 2024-03-24T00:49:02.783 INFO:tasks.workunit.client.0.smithi038.stdout:4/671: truncate da/d12/d29/d5b/d73/fba 4760853 0 2024-03-24T00:49:02.786 INFO:tasks.workunit.client.0.smithi038.stdout:8/729: rename d0/d10/le6 to d0/d1d/d8c/lf8 0 2024-03-24T00:49:02.787 INFO:tasks.workunit.client.0.smithi038.stdout:5/644: dread d6/d19/d20/d42/f91 [0,4194304] 0 2024-03-24T00:49:02.787 INFO:tasks.workunit.client.0.smithi038.stdout:5/645: read d6/d19/d20/d33/d6f/f7a [3427552,124154] 0 2024-03-24T00:49:02.787 INFO:tasks.workunit.client.0.smithi038.stdout:5/646: fdatasync d6/d1e/d97/fbe 0 2024-03-24T00:49:02.787 INFO:tasks.workunit.client.0.smithi038.stdout:5/647: dread - d6/d1e/d97/fbe zero size 2024-03-24T00:49:02.787 INFO:tasks.workunit.client.0.smithi038.stdout:5/648: creat d6/d19/d20/d33/d6f/fd3 x:0 0 0 2024-03-24T00:49:02.790 INFO:tasks.workunit.client.1.smithi088.stdout:6/778: rename db/dd/d61/fe0 to db/d3d/ff1 0 2024-03-24T00:49:02.790 INFO:tasks.workunit.client.1.smithi088.stdout:4/717: symlink d5/d7/d72/lfb 0 2024-03-24T00:49:02.790 INFO:tasks.workunit.client.1.smithi088.stdout:5/773: truncate d1/d10/d1b/d2f/f7e 1093131 0 2024-03-24T00:49:02.794 INFO:tasks.workunit.client.1.smithi088.stdout:1/749: rmdir d6/d23/daf/d95/d5b/d5e/d90 39 2024-03-24T00:49:02.794 INFO:tasks.workunit.client.0.smithi038.stdout:1/768: truncate da/dd/d2c/d5a/f92 2947993 0 2024-03-24T00:49:02.794 INFO:tasks.workunit.client.0.smithi038.stdout:1/769: creat da/dd/d7a/d82/ddb/f106 x:0 0 0 2024-03-24T00:49:02.801 INFO:tasks.workunit.client.1.smithi088.stdout:6/779: rename db/dd/d87/d9e to db/d1d/d23/d5c/df2 0 2024-03-24T00:49:02.801 INFO:tasks.workunit.client.1.smithi088.stdout:5/774: mknod d1/d10/d1b/d2f/d57/d9e/db1/cfc 0 2024-03-24T00:49:02.801 INFO:tasks.workunit.client.1.smithi088.stdout:5/775: dread - d1/dd/d6f/d98/fcd zero size 2024-03-24T00:49:02.801 INFO:tasks.workunit.client.1.smithi088.stdout:5/776: truncate d1/dd/fe3 4305720 0 2024-03-24T00:49:02.802 INFO:tasks.workunit.client.1.smithi088.stdout:5/777: creat d1/d19/db9/ffd x:0 0 0 2024-03-24T00:49:02.808 INFO:tasks.workunit.client.0.smithi038.stdout:4/672: mknod da/d12/d37/dd/ce5 0 2024-03-24T00:49:02.815 INFO:tasks.workunit.client.0.smithi038.stdout:1/770: truncate da/dd/d7a/d82/d98/d14/f4d 538805 0 2024-03-24T00:49:02.815 INFO:tasks.workunit.client.0.smithi038.stdout:1/771: dread - da/dd/d7a/d82/d98/d34/fdf zero size 2024-03-24T00:49:02.819 INFO:tasks.workunit.client.1.smithi088.stdout:3/693: dwrite d3/de/d6f/dcc/fd3 [0,4194304] 0 2024-03-24T00:49:02.831 INFO:tasks.workunit.client.0.smithi038.stdout:6/767: dread d1/da/d1b/d2f/dc0/fc7 [0,4194304] 0 2024-03-24T00:49:02.831 INFO:tasks.workunit.client.0.smithi038.stdout:6/768: write d1/da/d1b/d25/d49/d4e/d51/f9c [2391735,57328] 0 2024-03-24T00:49:02.839 INFO:tasks.workunit.client.1.smithi088.stdout:4/718: rename f4 to d5/d13/d18/d24/d3d/df0/ffc 0 2024-03-24T00:49:02.839 INFO:tasks.workunit.client.1.smithi088.stdout:4/719: stat d5/d16/d1a/l67 0 2024-03-24T00:49:02.839 INFO:tasks.workunit.client.1.smithi088.stdout:4/720: stat d5/d17/d91 0 2024-03-24T00:49:02.840 INFO:tasks.workunit.client.1.smithi088.stdout:4/721: creat d5/d16/d94/ffd x:0 0 0 2024-03-24T00:49:02.840 INFO:tasks.workunit.client.1.smithi088.stdout:4/722: readlink d5/d13/l93 0 2024-03-24T00:49:02.840 INFO:tasks.workunit.client.1.smithi088.stdout:4/723: creat d5/d17/d91/ffe x:0 0 0 2024-03-24T00:49:02.843 INFO:tasks.workunit.client.0.smithi038.stdout:3/694: dread d3/d9/dc/d51/f52 [0,4194304] 0 2024-03-24T00:49:02.843 INFO:tasks.workunit.client.0.smithi038.stdout:2/633: dwrite d4/f7 [0,4194304] 0 2024-03-24T00:49:02.845 INFO:tasks.workunit.client.1.smithi088.stdout:2/706: write db/dd/d96/d4e/d8f/d9c/db0/fcd [4101899,108554] 0 2024-03-24T00:49:02.846 INFO:tasks.workunit.client.1.smithi088.stdout:1/750: creat d6/d23/daf/d95/d5b/d5d/d61/ff1 x:0 0 0 2024-03-24T00:49:02.846 INFO:tasks.workunit.client.1.smithi088.stdout:7/679: dread d3/d31/f7a [0,4194304] 0 2024-03-24T00:49:02.846 INFO:tasks.workunit.client.1.smithi088.stdout:8/734: dwrite d9/dc/d11/d24/d29/d36/f4e [0,4194304] 0 2024-03-24T00:49:02.852 INFO:tasks.workunit.client.1.smithi088.stdout:4/724: dread d5/d7/dd/f65 [0,4194304] 0 2024-03-24T00:49:02.853 INFO:tasks.workunit.client.0.smithi038.stdout:8/730: dread d0/d45/d37/d5f/db7/f4f [0,4194304] 0 2024-03-24T00:49:02.853 INFO:tasks.workunit.client.0.smithi038.stdout:8/731: readlink d0/d45/db8/ld1 0 2024-03-24T00:49:02.857 INFO:tasks.workunit.client.1.smithi088.stdout:7/680: dread d3/d32/d34/d9c/f61 [0,4194304] 0 2024-03-24T00:49:02.858 INFO:tasks.workunit.client.1.smithi088.stdout:6/780: unlink db/dd/d87/ddf/fe9 0 2024-03-24T00:49:02.868 INFO:tasks.workunit.client.0.smithi038.stdout:6/769: read d1/f1f [1558088,125381] 0 2024-03-24T00:49:02.868 INFO:tasks.workunit.client.1.smithi088.stdout:4/725: write d5/d16/f2e [1174692,50693] 0 2024-03-24T00:49:02.868 INFO:tasks.workunit.client.1.smithi088.stdout:4/726: chown d5/d16/d1a/f1e 2500 1 2024-03-24T00:49:02.870 INFO:tasks.workunit.client.0.smithi038.stdout:5/649: dwrite d6/d19/f1c [4194304,4194304] 0 2024-03-24T00:49:02.872 INFO:tasks.workunit.client.0.smithi038.stdout:7/664: dwrite d2/d9/da/f36 [0,4194304] 0 2024-03-24T00:49:02.873 INFO:tasks.workunit.client.1.smithi088.stdout:6/781: write db/d1d/d50/f6f [2685419,74583] 0 2024-03-24T00:49:02.873 INFO:tasks.workunit.client.1.smithi088.stdout:0/773: dwrite d9/d26/d8f/da1/d63/f8a [0,4194304] 0 2024-03-24T00:49:02.873 INFO:tasks.workunit.client.1.smithi088.stdout:6/782: fsync db/f13 0 2024-03-24T00:49:02.875 INFO:tasks.workunit.client.0.smithi038.stdout:9/791: write fe [880539,30144] 0 2024-03-24T00:49:02.875 INFO:tasks.workunit.client.0.smithi038.stdout:9/792: creat d12/d41/dc0/f100 x:0 0 0 2024-03-24T00:49:02.881 INFO:tasks.workunit.client.1.smithi088.stdout:3/694: unlink d3/d67/l83 0 2024-03-24T00:49:02.884 INFO:tasks.workunit.client.0.smithi038.stdout:3/695: mkdir d3/d30/d5d/deb 0 2024-03-24T00:49:02.889 INFO:tasks.workunit.client.0.smithi038.stdout:2/634: creat d4/fdd x:0 0 0 2024-03-24T00:49:02.898 INFO:tasks.workunit.client.0.smithi038.stdout:9/793: write d12/d1a/d80/f83 [2476227,111713] 0 2024-03-24T00:49:02.899 INFO:tasks.workunit.client.0.smithi038.stdout:9/794: fsync d12/d28/d9c/ddd/ffa 0 2024-03-24T00:49:02.902 INFO:tasks.workunit.client.0.smithi038.stdout:6/770: symlink d1/da/d61/da2/lfb 0 2024-03-24T00:49:02.908 INFO:tasks.workunit.client.1.smithi088.stdout:2/707: creat db/fe0 x:0 0 0 2024-03-24T00:49:02.908 INFO:tasks.workunit.client.1.smithi088.stdout:2/708: write db/d21/d35/f79 [4411402,103429] 0 2024-03-24T00:49:02.908 INFO:tasks.workunit.client.1.smithi088.stdout:2/709: readlink db/dd/d96/l58 0 2024-03-24T00:49:02.909 INFO:tasks.workunit.client.1.smithi088.stdout:2/710: fsync db/f87 0 2024-03-24T00:49:02.909 INFO:tasks.workunit.client.1.smithi088.stdout:5/778: write d1/d10/d1b/d2f/f7e [955154,49762] 0 2024-03-24T00:49:02.912 INFO:tasks.workunit.client.0.smithi038.stdout:4/673: dwrite da/d12/d24/d5f/d8a/db6/fb9 [0,4194304] 0 2024-03-24T00:49:02.919 INFO:tasks.workunit.client.0.smithi038.stdout:8/732: dread d0/d1d/f22 [0,4194304] 0 2024-03-24T00:49:02.922 INFO:tasks.workunit.client.0.smithi038.stdout:5/650: link d6/d19/d20/d33/d6f/f88 d6/d19/d20/d42/d43/d68/dc8/fd4 0 2024-03-24T00:49:02.925 INFO:tasks.workunit.client.1.smithi088.stdout:1/751: unlink d6/lb 0 2024-03-24T00:49:02.927 INFO:tasks.workunit.client.1.smithi088.stdout:6/783: dwrite db/d1d/d23/fcc [0,4194304] 0 2024-03-24T00:49:02.928 INFO:tasks.workunit.client.1.smithi088.stdout:4/727: dread d5/fb5 [0,4194304] 0 2024-03-24T00:49:02.928 INFO:tasks.workunit.client.1.smithi088.stdout:4/728: chown d5/d16/c43 396 1 2024-03-24T00:49:02.928 INFO:tasks.workunit.client.1.smithi088.stdout:4/729: chown d5/d7/dd/lc4 1032 1 2024-03-24T00:49:02.929 INFO:tasks.workunit.client.1.smithi088.stdout:4/730: write d5/d16/f29 [3689170,62500] 0 2024-03-24T00:49:02.929 INFO:tasks.workunit.client.1.smithi088.stdout:2/711: dread db/dd/f17 [0,4194304] 0 2024-03-24T00:49:02.930 INFO:tasks.workunit.client.1.smithi088.stdout:2/712: stat db/dd/d96/d99/db2/c6a 0 2024-03-24T00:49:02.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:02 smithi088 ceph-mon[36610]: pgmap v37: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 224 MiB/s rd, 264 MiB/s wr, 386 op/s 2024-03-24T00:49:02.931 INFO:tasks.workunit.client.1.smithi088.stdout:7/681: symlink d3/d32/d34/d9c/d4b/d73/ld9 0 2024-03-24T00:49:02.931 INFO:tasks.workunit.client.1.smithi088.stdout:7/682: write d3/d32/d34/d9c/d4b/d74/d79/d7f/fb2 [2057527,24142] 0 2024-03-24T00:49:02.931 INFO:tasks.workunit.client.1.smithi088.stdout:7/683: write d3/d32/d34/d9c/d4b/f4c [4476810,119805] 0 2024-03-24T00:49:02.949 INFO:tasks.workunit.client.0.smithi038.stdout:3/696: rmdir d3/d9/dc/da6/db5 39 2024-03-24T00:49:02.950 INFO:tasks.workunit.client.0.smithi038.stdout:7/665: creat d2/d9/da/d9b/ff9 x:0 0 0 2024-03-24T00:49:02.953 INFO:tasks.workunit.client.0.smithi038.stdout:8/733: dread d0/d45/d37/f61 [0,4194304] 0 2024-03-24T00:49:02.956 INFO:tasks.workunit.client.0.smithi038.stdout:9/795: truncate d12/d55/fb5 8353620 0 2024-03-24T00:49:02.956 INFO:tasks.workunit.client.0.smithi038.stdout:9/796: fdatasync d12/d28/d3a/fce 0 2024-03-24T00:49:02.958 INFO:tasks.workunit.client.1.smithi088.stdout:0/774: link d9/c4c d9/dc/df7/cfa 0 2024-03-24T00:49:02.958 INFO:tasks.workunit.client.1.smithi088.stdout:0/775: fsync d9/dc/de/f5c 0 2024-03-24T00:49:02.965 INFO:tasks.workunit.client.1.smithi088.stdout:3/695: mknod d3/d1a/cde 0 2024-03-24T00:49:02.965 INFO:tasks.workunit.client.1.smithi088.stdout:3/696: fdatasync d3/de/d6f/f7a 0 2024-03-24T00:49:02.978 INFO:tasks.workunit.client.0.smithi038.stdout:4/674: creat da/d12/d37/fe6 x:0 0 0 2024-03-24T00:49:02.983 INFO:tasks.workunit.client.1.smithi088.stdout:5/779: creat d1/d10/d1b/d2f/d57/d78/ffe x:0 0 0 2024-03-24T00:49:02.983 INFO:tasks.workunit.client.1.smithi088.stdout:5/780: readlink d1/dd/l22 0 2024-03-24T00:49:02.984 INFO:tasks.workunit.client.1.smithi088.stdout:5/781: write d1/d10/d1b/d2f/d57/d9e/db1/ded/ff9 [535417,80124] 0 2024-03-24T00:49:02.984 INFO:tasks.workunit.client.1.smithi088.stdout:5/782: truncate d1/d10/f52 5222510 0 2024-03-24T00:49:02.984 INFO:tasks.workunit.client.1.smithi088.stdout:5/783: readlink d1/d10/d1b/d2f/l67 0 2024-03-24T00:49:02.984 INFO:tasks.workunit.client.0.smithi038.stdout:6/771: dwrite d1/da/d1b/d1d/f2c [0,4194304] 0 2024-03-24T00:49:02.984 INFO:tasks.workunit.client.0.smithi038.stdout:6/772: stat d1/da/d1b/d1d/f24 0 2024-03-24T00:49:02.985 INFO:tasks.workunit.client.0.smithi038.stdout:6/773: creat d1/da/d61/da2/db1/ffc x:0 0 0 2024-03-24T00:49:02.985 INFO:tasks.workunit.client.0.smithi038.stdout:6/774: chown d1/da/d61/d69/lf1 371419777 1 2024-03-24T00:49:03.000 INFO:tasks.workunit.client.0.smithi038.stdout:5/651: creat d6/d19/d20/d42/fd5 x:0 0 0 2024-03-24T00:49:03.004 INFO:tasks.workunit.client.1.smithi088.stdout:5/784: dread d1/d19/d1e/f76 [0,4194304] 0 2024-03-24T00:49:03.004 INFO:tasks.workunit.client.1.smithi088.stdout:5/785: fsync d1/d19/db9/ffd 0 2024-03-24T00:49:03.004 INFO:tasks.workunit.client.1.smithi088.stdout:5/786: truncate d1/d43/d4e/d5b/f63 1265970 0 2024-03-24T00:49:03.009 INFO:tasks.workunit.client.0.smithi038.stdout:3/697: unlink d3/d9/fb3 0 2024-03-24T00:49:03.011 INFO:tasks.workunit.client.1.smithi088.stdout:2/713: dwrite db/f89 [0,4194304] 0 2024-03-24T00:49:03.012 INFO:tasks.workunit.client.0.smithi038.stdout:1/772: dwrite da/f21 [0,4194304] 0 2024-03-24T00:49:03.015 INFO:tasks.workunit.client.0.smithi038.stdout:7/666: mknod d2/d9/da/d23/d2b/cfa 0 2024-03-24T00:49:03.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.017+0000 7f211ae00700 1 -- 172.21.15.38:0/3120871740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2114103e90 msgr2=0x7f2114106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:03.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.017+0000 7f211ae00700 1 --2- 172.21.15.38:0/3120871740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2114103e90 0x7f2114106280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009910 tx=0x7f20fc009c20 comp rx=0 tx=0).stop 2024-03-24T00:49:03.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.017+0000 7f211ae00700 1 -- 172.21.15.38:0/3120871740 shutdown_connections 2024-03-24T00:49:03.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.017+0000 7f211ae00700 1 --2- 172.21.15.38:0/3120871740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2114103e90 0x7f2114106280 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009910 tx=0x7f20fc009c20 comp rx=0 tx=0).stop 2024-03-24T00:49:03.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.017+0000 7f211ae00700 1 --2- 172.21.15.38:0/3120871740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114101560 0x7f2114103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:03.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.017+0000 7f211ae00700 1 -- 172.21.15.38:0/3120871740 >> 172.21.15.38:0/3120871740 conn(0x7f21140faf70 msgr2=0x7f21140fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:03.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.021+0000 7f211ae00700 1 -- 172.21.15.38:0/3120871740 shutdown_connections 2024-03-24T00:49:03.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.021+0000 7f211ae00700 1 -- 172.21.15.38:0/3120871740 wait complete. 2024-03-24T00:49:03.023 INFO:tasks.workunit.client.0.smithi038.stdout:1/773: dread da/dd/d2c/d40/f5d [0,4194304] 0 2024-03-24T00:49:03.023 INFO:tasks.workunit.client.0.smithi038.stdout:8/734: link d0/d45/d37/d5f/db7/d62/cd8 d0/d1d/d20/d3c/d9e/da3/cf9 0 2024-03-24T00:49:03.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.022+0000 7f211ae00700 1 Processor -- start 2024-03-24T00:49:03.024 INFO:tasks.workunit.client.1.smithi088.stdout:4/731: truncate d5/d7/d72/f73 3860532 0 2024-03-24T00:49:03.024 INFO:tasks.workunit.client.1.smithi088.stdout:4/732: chown d5/d8b 3 1 2024-03-24T00:49:03.026 INFO:tasks.workunit.client.0.smithi038.stdout:2/635: getdents d4 0 2024-03-24T00:49:03.028 INFO:tasks.workunit.client.0.smithi038.stdout:2/636: chown d4/d18/d26/f2c 77586966 1 2024-03-24T00:49:03.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.024+0000 7f211ae00700 1 -- start start 2024-03-24T00:49:03.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.024+0000 7f211ae00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2114101560 0x7f2114198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:03.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.024+0000 7f211ae00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114198e40 0x7f211419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:03.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.024+0000 7f211ae00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2114199350 con 0x7f2114101560 2024-03-24T00:49:03.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.024+0000 7f211ae00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21141994c0 con 0x7f2114198e40 2024-03-24T00:49:03.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.025+0000 7f2113400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114198e40 0x7f211419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:03.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.025+0000 7f2113e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2114101560 0x7f2114198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:03.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.025+0000 7f2113400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114198e40 0x7f211419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:50164/0 (socket says 172.21.15.38:50164) 2024-03-24T00:49:03.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.026+0000 7f2113400700 1 -- 172.21.15.38:0/784385592 learned_addr learned my addr 172.21.15.38:0/784385592 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:49:03.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.028+0000 7f2113400700 1 -- 172.21.15.38:0/784385592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2114101560 msgr2=0x7f2114198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:03.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.028+0000 7f2113400700 1 --2- 172.21.15.38:0/784385592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2114101560 0x7f2114198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:03.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.028+0000 7f2113400700 1 -- 172.21.15.38:0/784385592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc0095f0 con 0x7f2114198e40 2024-03-24T00:49:03.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.029+0000 7f2113400700 1 --2- 172.21.15.38:0/784385592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114198e40 0x7f211419de10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f20fc000c00 tx=0x7f20fc00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:03.031 INFO:tasks.workunit.client.0.smithi038.stdout:9/797: symlink d12/d1a/l101 0 2024-03-24T00:49:03.032 INFO:tasks.workunit.client.1.smithi088.stdout:9/756: sync 2024-03-24T00:49:03.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.030+0000 7f2110c00700 1 -- 172.21.15.38:0/784385592 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20fc01d070 con 0x7f2114198e40 2024-03-24T00:49:03.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.030+0000 7f2110c00700 1 -- 172.21.15.38:0/784385592 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20fc005dc0 con 0x7f2114198e40 2024-03-24T00:49:03.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.030+0000 7f2110c00700 1 -- 172.21.15.38:0/784385592 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20fc00fe30 con 0x7f2114198e40 2024-03-24T00:49:03.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.030+0000 7f211ae00700 1 -- 172.21.15.38:0/784385592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211419e350 con 0x7f2114198e40 2024-03-24T00:49:03.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.030+0000 7f211ae00700 1 -- 172.21.15.38:0/784385592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211418eeb0 con 0x7f2114198e40 2024-03-24T00:49:03.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.032+0000 7f211ae00700 1 -- 172.21.15.38:0/784385592 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21140fcb40 con 0x7f2114198e40 2024-03-24T00:49:03.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.035+0000 7f2110c00700 1 -- 172.21.15.38:0/784385592 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) v1 ==== 91370+0+0 (secure 0 0 0) 0x7f20fc00f5f0 con 0x7f2114198e40 2024-03-24T00:49:03.037 INFO:tasks.workunit.client.0.smithi038.stdout:9/798: fdatasync fe 0 2024-03-24T00:49:03.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.037+0000 7f2110c00700 1 --2- 172.21.15.38:0/784385592 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f20f806d6d0 0x7f20f806fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:03.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.037+0000 7f2110c00700 1 -- 172.21.15.38:0/784385592 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f20fc08e070 con 0x7f2114198e40 2024-03-24T00:49:03.042 INFO:tasks.workunit.client.0.smithi038.stdout:4/675: mkdir da/d12/d24/d68/d6c/da6/de7 0 2024-03-24T00:49:03.042 INFO:tasks.workunit.client.1.smithi088.stdout:0/776: creat d9/dc/de/d19/d8d/ffb x:0 0 0 2024-03-24T00:49:03.042 INFO:tasks.workunit.client.1.smithi088.stdout:0/777: getdents d9/dc/de/d19/d8d/d97 0 2024-03-24T00:49:03.043 INFO:tasks.workunit.client.0.smithi038.stdout:6/775: mkdir d1/da/d1b/dfd 0 2024-03-24T00:49:03.043 INFO:tasks.workunit.client.0.smithi038.stdout:6/776: dread - d1/da/d61/d69/fd4 zero size 2024-03-24T00:49:03.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.043+0000 7f2113e00700 1 --2- 172.21.15.38:0/784385592 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f20f806d6d0 0x7f20f806fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:03.044 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:02 smithi038 ceph-mon[29271]: pgmap v37: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 224 MiB/s rd, 264 MiB/s wr, 386 op/s 2024-03-24T00:49:03.044 INFO:tasks.workunit.client.1.smithi088.stdout:3/697: mkdir d3/de/d6f/d89/ddf 0 2024-03-24T00:49:03.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.045+0000 7f2113e00700 1 --2- 172.21.15.38:0/784385592 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f20f806d6d0 0x7f20f806fb90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f210c005950 tx=0x7f210c00b410 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:03.047 INFO:tasks.workunit.client.0.smithi038.stdout:3/698: mknod d3/dc3/cec 0 2024-03-24T00:49:03.050 INFO:tasks.workunit.client.0.smithi038.stdout:0/636: sync 2024-03-24T00:49:03.058 INFO:tasks.workunit.client.0.smithi038.stdout:7/667: creat d2/d9/da/d23/d2b/ffb x:0 0 0 2024-03-24T00:49:03.059 INFO:tasks.workunit.client.1.smithi088.stdout:1/752: getdents d6/d23/dcb 0 2024-03-24T00:49:03.059 INFO:tasks.workunit.client.1.smithi088.stdout:1/753: chown d6/l62 247044 1 2024-03-24T00:49:03.059 INFO:tasks.workunit.client.1.smithi088.stdout:1/754: creat d6/d27/d67/d9b/ff2 x:0 0 0 2024-03-24T00:49:03.059 INFO:tasks.workunit.client.1.smithi088.stdout:1/755: chown d6/d23/l3b 2365 1 2024-03-24T00:49:03.059 INFO:tasks.workunit.client.1.smithi088.stdout:1/756: readlink d6/d23/daf/d95/d3f/d53/lb2 0 2024-03-24T00:49:03.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.054+0000 7f2110c00700 1 -- 172.21.15.38:0/784385592 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f20fc05b480 con 0x7f2114198e40 2024-03-24T00:49:03.063 INFO:tasks.workunit.client.1.smithi088.stdout:6/784: write db/d1d/d23/d5c/f67 [1978453,25036] 0 2024-03-24T00:49:03.063 INFO:tasks.workunit.client.1.smithi088.stdout:6/785: write db/dd/d61/d6d/fe1 [846538,80430] 0 2024-03-24T00:49:03.070 INFO:tasks.workunit.client.0.smithi038.stdout:2/637: rename d4/d27/d2f/d30/l60 to d4/d27/d54/d67/lde 0 2024-03-24T00:49:03.070 INFO:tasks.workunit.client.0.smithi038.stdout:2/638: dread - d4/d27/ddb/d9b/f9c zero size 2024-03-24T00:49:03.080 INFO:tasks.workunit.client.0.smithi038.stdout:9/799: link d12/d81/d84/d88/fa9 d12/d41/d57/dd3/dfe/f102 0 2024-03-24T00:49:03.080 INFO:tasks.workunit.client.0.smithi038.stdout:9/800: dread - d12/d41/d57/f98 zero size 2024-03-24T00:49:03.087 INFO:tasks.workunit.client.0.smithi038.stdout:4/676: creat da/d12/d37/d83/dac/dcf/fe8 x:0 0 0 2024-03-24T00:49:03.087 INFO:tasks.workunit.client.0.smithi038.stdout:4/677: fdatasync da/d12/d37/dd/f8d 0 2024-03-24T00:49:03.094 INFO:tasks.workunit.client.0.smithi038.stdout:3/699: mkdir d3/d30/ded 0 2024-03-24T00:49:03.094 INFO:tasks.workunit.client.0.smithi038.stdout:7/668: mkdir d2/d9/d16/dfc 0 2024-03-24T00:49:03.097 INFO:tasks.workunit.client.0.smithi038.stdout:8/735: rmdir d0/d45/d37/d5f/db7/dcc 39 2024-03-24T00:49:03.097 INFO:tasks.workunit.client.0.smithi038.stdout:8/736: fdatasync d0/d45/d37/d5f/db7/f9a 0 2024-03-24T00:49:03.099 INFO:tasks.workunit.client.1.smithi088.stdout:4/733: mkdir d5/d16/d37/dff 0 2024-03-24T00:49:03.099 INFO:tasks.workunit.client.1.smithi088.stdout:4/734: fsync d5/d13/d63/d78/fd1 0 2024-03-24T00:49:03.105 INFO:tasks.workunit.client.0.smithi038.stdout:2/639: mknod d4/d27/d54/d67/cdf 0 2024-03-24T00:49:03.107 INFO:tasks.workunit.client.1.smithi088.stdout:8/735: sync 2024-03-24T00:49:03.109 INFO:tasks.workunit.client.0.smithi038.stdout:9/801: dread d12/d28/d3a/fca [0,4194304] 0 2024-03-24T00:49:03.109 INFO:tasks.workunit.client.0.smithi038.stdout:0/637: read d0/d12/d42/d9d/d28/f8a [3131844,127986] 0 2024-03-24T00:49:03.109 INFO:tasks.workunit.client.0.smithi038.stdout:6/777: rename d1/da/d61/da2/db1/fb9 to d1/da/d73/ffe 0 2024-03-24T00:49:03.110 INFO:tasks.workunit.client.1.smithi088.stdout:0/778: symlink d9/dc/de/d19/d84/dd6/lfc 0 2024-03-24T00:49:03.110 INFO:tasks.workunit.client.0.smithi038.stdout:3/700: mknod d3/d30/d5d/d7c/cee 0 2024-03-24T00:49:03.111 INFO:tasks.workunit.client.0.smithi038.stdout:3/701: fdatasync d3/d2a/f59 0 2024-03-24T00:49:03.111 INFO:tasks.workunit.client.1.smithi088.stdout:3/698: mkdir d3/de/d6f/d85/de0 0 2024-03-24T00:49:03.112 INFO:tasks.workunit.client.1.smithi088.stdout:3/699: write d3/de/fba [877478,33906] 0 2024-03-24T00:49:03.113 INFO:tasks.workunit.client.0.smithi038.stdout:7/669: creat d2/d9/da/d23/d2b/d5b/d72/d92/d75/ffd x:0 0 0 2024-03-24T00:49:03.115 INFO:tasks.workunit.client.1.smithi088.stdout:6/786: symlink db/d3d/d83/d71/lf3 0 2024-03-24T00:49:03.115 INFO:tasks.workunit.client.1.smithi088.stdout:6/787: chown db/f13 0 1 2024-03-24T00:49:03.115 INFO:tasks.workunit.client.1.smithi088.stdout:5/787: dwrite d1/d10/f17 [0,4194304] 0 2024-03-24T00:49:03.115 INFO:tasks.workunit.client.0.smithi038.stdout:2/640: mkdir d4/d18/d26/d37/de0 0 2024-03-24T00:49:03.118 INFO:tasks.workunit.client.1.smithi088.stdout:9/757: rmdir d1/d10/d11/d23 39 2024-03-24T00:49:03.118 INFO:tasks.workunit.client.1.smithi088.stdout:4/735: dread d5/d13/d18/d24/f90 [0,4194304] 0 2024-03-24T00:49:03.120 INFO:tasks.workunit.client.0.smithi038.stdout:3/702: write d3/d9/dc/d51/f52 [925656,38549] 0 2024-03-24T00:49:03.120 INFO:tasks.workunit.client.0.smithi038.stdout:3/703: chown d3/d9/f16 3 1 2024-03-24T00:49:03.125 INFO:tasks.workunit.client.0.smithi038.stdout:0/638: getdents d0/d12/d42/d5e/db0 0 2024-03-24T00:49:03.125 INFO:tasks.workunit.client.0.smithi038.stdout:0/639: creat d0/d12/d92/fd6 x:0 0 0 2024-03-24T00:49:03.128 INFO:tasks.workunit.client.1.smithi088.stdout:8/736: creat d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/ffa x:0 0 0 2024-03-24T00:49:03.128 INFO:tasks.workunit.client.1.smithi088.stdout:0/779: creat d9/dc/de/d19/ffd x:0 0 0 2024-03-24T00:49:03.128 INFO:tasks.workunit.client.1.smithi088.stdout:0/780: chown d9/dc/de/d19/d44/d46/d65/d6f 5208953 1 2024-03-24T00:49:03.128 INFO:tasks.workunit.client.1.smithi088.stdout:4/736: symlink d5/d13/d63/d78/df1/l100 0 2024-03-24T00:49:03.128 INFO:tasks.workunit.client.1.smithi088.stdout:3/700: unlink d3/d67/la1 0 2024-03-24T00:49:03.130 INFO:tasks.workunit.client.1.smithi088.stdout:0/781: rename d9/d26/d8f/da1/d62/d9b/db0/db4/fc8 to d9/ffe 0 2024-03-24T00:49:03.132 INFO:tasks.workunit.client.1.smithi088.stdout:4/737: symlink d5/d17/d91/l101 0 2024-03-24T00:49:03.132 INFO:tasks.workunit.client.1.smithi088.stdout:4/738: chown d5/d16/d1a/d22/l5f 1320125 1 2024-03-24T00:49:03.133 INFO:tasks.workunit.client.0.smithi038.stdout:6/778: symlink d1/da/d73/lff 0 2024-03-24T00:49:03.134 INFO:tasks.workunit.client.1.smithi088.stdout:3/701: link f1 d3/d67/dc3/dd7/fe1 0 2024-03-24T00:49:03.135 INFO:tasks.workunit.client.1.smithi088.stdout:0/782: creat d9/dc/de/d19/d84/fff x:0 0 0 2024-03-24T00:49:03.144 INFO:tasks.workunit.client.1.smithi088.stdout:4/739: rename d5/fc to d5/d7/d72/f102 0 2024-03-24T00:49:03.144 INFO:tasks.workunit.client.1.smithi088.stdout:4/740: readlink d5/d13/l93 0 2024-03-24T00:49:03.146 INFO:tasks.workunit.client.0.smithi038.stdout:0/640: truncate d0/fa 2198224 0 2024-03-24T00:49:03.147 INFO:tasks.workunit.client.1.smithi088.stdout:4/741: mknod d5/d16/d94/c103 0 2024-03-24T00:49:03.153 INFO:tasks.workunit.client.0.smithi038.stdout:8/737: rename d0/d1d/d20/d3c/d9e/da3 to d0/d45/d37/d5f/db7/d62/dfa 0 2024-03-24T00:49:03.153 INFO:tasks.workunit.client.1.smithi088.stdout:4/742: getdents d5/d13/d18/d24/d3d/df0 0 2024-03-24T00:49:03.154 INFO:tasks.workunit.client.0.smithi038.stdout:8/738: symlink d0/d15/d2c/lfb 0 2024-03-24T00:49:03.159 INFO:tasks.workunit.client.1.smithi088.stdout:7/684: sync 2024-03-24T00:49:03.160 INFO:tasks.workunit.client.1.smithi088.stdout:7/685: write d3/d32/d34/fc2 [1324955,47586] 0 2024-03-24T00:49:03.160 INFO:tasks.workunit.client.0.smithi038.stdout:2/641: read d4/f7 [3238074,71605] 0 2024-03-24T00:49:03.163 INFO:tasks.workunit.client.1.smithi088.stdout:7/686: creat d3/d32/d34/d9c/d4b/d74/d79/fda x:0 0 0 2024-03-24T00:49:03.163 INFO:tasks.workunit.client.0.smithi038.stdout:2/642: stat d4/d27/d2f/d89/f9f 0 2024-03-24T00:49:03.167 INFO:tasks.workunit.client.1.smithi088.stdout:7/687: getdents d3/d32/d34 0 2024-03-24T00:49:03.167 INFO:tasks.workunit.client.0.smithi038.stdout:7/670: dread d2/d9/da/d23/fa3 [0,4194304] 0 2024-03-24T00:49:03.171 INFO:tasks.workunit.client.1.smithi088.stdout:7/688: truncate d3/fc 3142144 0 2024-03-24T00:49:03.171 INFO:tasks.workunit.client.1.smithi088.stdout:7/689: read d3/d32/d34/d9c/d4b/d73/fbe [2882801,125358] 0 2024-03-24T00:49:03.171 INFO:tasks.workunit.client.1.smithi088.stdout:7/690: chown d3/d32/d34/d9c/d4b/d74/d79/d7f/fca 2066 1 2024-03-24T00:49:03.201 INFO:tasks.workunit.client.0.smithi038.stdout:5/652: dwrite d6/d19/d20/fcf [0,4194304] 0 2024-03-24T00:49:03.207 INFO:tasks.workunit.client.1.smithi088.stdout:1/757: dwrite d6/d23/daf/d95/d5b/d5d/d61/fdb [0,4194304] 0 2024-03-24T00:49:03.210 INFO:tasks.workunit.client.1.smithi088.stdout:4/743: dread d5/d13/d18/f19 [0,4194304] 0 2024-03-24T00:49:03.231 INFO:tasks.workunit.client.1.smithi088.stdout:3/702: dread d3/d6/dca/dae/d4b/fbf [0,4194304] 0 2024-03-24T00:49:03.231 INFO:tasks.workunit.client.1.smithi088.stdout:3/703: chown d3/de/c20 2673364 1 2024-03-24T00:49:03.231 INFO:tasks.workunit.client.1.smithi088.stdout:3/704: write d3/de/d6f/d85/d31/f94 [209220,83853] 0 2024-03-24T00:49:03.232 INFO:tasks.workunit.client.0.smithi038.stdout:1/774: dwrite da/dd/d4a/d76/f36 [0,4194304] 0 2024-03-24T00:49:03.232 INFO:tasks.workunit.client.0.smithi038.stdout:0/641: dread d0/d12/d42/f61 [4194304,4194304] 0 2024-03-24T00:49:03.232 INFO:tasks.workunit.client.0.smithi038.stdout:5/653: creat d6/d16/fd6 x:0 0 0 2024-03-24T00:49:03.242 INFO:tasks.workunit.client.0.smithi038.stdout:5/654: getdents d6/d19/d20/d42/d43/d68/d6e 0 2024-03-24T00:49:03.247 INFO:tasks.workunit.client.1.smithi088.stdout:1/758: symlink d6/d23/daf/lf3 0 2024-03-24T00:49:03.247 INFO:tasks.workunit.client.1.smithi088.stdout:4/744: creat d5/d7/d72/f104 x:0 0 0 2024-03-24T00:49:03.247 INFO:tasks.workunit.client.1.smithi088.stdout:3/705: mknod d3/d1a/d1e/ce2 0 2024-03-24T00:49:03.247 INFO:tasks.workunit.client.1.smithi088.stdout:3/706: fsync d3/de/f16 0 2024-03-24T00:49:03.247 INFO:tasks.workunit.client.1.smithi088.stdout:3/707: write d3/d1a/d1e/d2f/f36 [2139272,47870] 0 2024-03-24T00:49:03.247 INFO:tasks.workunit.client.1.smithi088.stdout:3/708: stat d3/d32/l51 0 2024-03-24T00:49:03.255 INFO:tasks.workunit.client.0.smithi038.stdout:4/678: dwrite da/d38/f3c [0,4194304] 0 2024-03-24T00:49:03.259 INFO:tasks.workunit.client.1.smithi088.stdout:9/758: dwrite d1/d22/d27/fbf [0,4194304] 0 2024-03-24T00:49:03.264 INFO:tasks.workunit.client.0.smithi038.stdout:4/679: creat da/d12/d24/d68/d6c/fe9 x:0 0 0 2024-03-24T00:49:03.268 INFO:tasks.workunit.client.0.smithi038.stdout:6/779: dwrite d1/f1f [0,4194304] 0 2024-03-24T00:49:03.270 INFO:tasks.workunit.client.1.smithi088.stdout:5/788: dwrite d1/d10/d1b/d2f/d57/f5f [0,4194304] 0 2024-03-24T00:49:03.274 INFO:tasks.workunit.client.1.smithi088.stdout:8/737: dwrite d9/f32 [0,4194304] 0 2024-03-24T00:49:03.279 INFO:tasks.workunit.client.1.smithi088.stdout:1/759: mknod d6/d23/daf/d95/d58/cf4 0 2024-03-24T00:49:03.279 INFO:tasks.workunit.client.1.smithi088.stdout:4/745: symlink d5/d13/d18/d24/d3d/l105 0 2024-03-24T00:49:03.279 INFO:tasks.workunit.client.1.smithi088.stdout:4/746: write d5/d7/d15/d74/fe9 [562646,34494] 0 2024-03-24T00:49:03.280 INFO:tasks.workunit.client.1.smithi088.stdout:4/747: read d5/d13/d18/f19 [540602,117249] 0 2024-03-24T00:49:03.282 INFO:tasks.workunit.client.1.smithi088.stdout:0/783: dwrite d9/fd0 [0,4194304] 0 2024-03-24T00:49:03.296 INFO:tasks.workunit.client.0.smithi038.stdout:6/780: dread d1/da/d1b/d25/d49/d72/fb7 [0,4194304] 0 2024-03-24T00:49:03.297 INFO:tasks.workunit.client.0.smithi038.stdout:6/781: creat d1/d37/f100 x:0 0 0 2024-03-24T00:49:03.302 INFO:tasks.workunit.client.0.smithi038.stdout:7/671: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/fe2 [0,4194304] 0 2024-03-24T00:49:03.303 INFO:tasks.workunit.client.0.smithi038.stdout:7/672: dread - d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fd5 zero size 2024-03-24T00:49:03.308 INFO:tasks.workunit.client.0.smithi038.stdout:4/680: write da/d12/d24/f30 [2028735,53998] 0 2024-03-24T00:49:03.309 INFO:tasks.workunit.client.1.smithi088.stdout:3/709: dread d3/de/d6f/d85/d31/f94 [0,4194304] 0 2024-03-24T00:49:03.309 INFO:tasks.workunit.client.1.smithi088.stdout:3/710: readlink d3/d6/d27/d3c/l26 0 2024-03-24T00:49:03.309 INFO:tasks.workunit.client.1.smithi088.stdout:3/711: readlink d3/d6/d27/l64 0 2024-03-24T00:49:03.314 INFO:tasks.workunit.client.0.smithi038.stdout:2/643: dwrite d4/d27/ddb/f58 [0,4194304] 0 2024-03-24T00:49:03.318 INFO:tasks.workunit.client.1.smithi088.stdout:9/759: dwrite d1/d10/f1e [8388608,4194304] 0 2024-03-24T00:49:03.318 INFO:tasks.workunit.client.1.smithi088.stdout:9/760: chown d1/d15 721 1 2024-03-24T00:49:03.334 INFO:tasks.workunit.client.1.smithi088.stdout:5/789: creat d1/d19/d48/dc2/dcc/fff x:0 0 0 2024-03-24T00:49:03.337 INFO:tasks.workunit.client.0.smithi038.stdout:9/802: dwrite d12/d55/f67 [0,4194304] 0 2024-03-24T00:49:03.338 INFO:tasks.workunit.client.1.smithi088.stdout:7/691: dwrite d3/d32/d34/d9c/d4b/d74/f95 [0,4194304] 0 2024-03-24T00:49:03.339 INFO:tasks.workunit.client.0.smithi038.stdout:2/644: dread d4/d18/f2e [0,4194304] 0 2024-03-24T00:49:03.339 INFO:tasks.workunit.client.0.smithi038.stdout:2/645: write d4/d27/d8f/d9e/fdc [65651,65099] 0 2024-03-24T00:49:03.340 INFO:tasks.workunit.client.0.smithi038.stdout:2/646: truncate d4/d27/d54/d53/f64 1151148 0 2024-03-24T00:49:03.340 INFO:tasks.workunit.client.0.smithi038.stdout:2/647: readlink d4/d18/l1a 0 2024-03-24T00:49:03.340 INFO:tasks.workunit.client.0.smithi038.stdout:2/648: chown d4/d18/d5b/caa 25654231 1 2024-03-24T00:49:03.340 INFO:tasks.workunit.client.0.smithi038.stdout:0/642: dwrite d0/d14/d84/fc5 [0,4194304] 0 2024-03-24T00:49:03.341 INFO:tasks.workunit.client.0.smithi038.stdout:0/643: truncate d0/d12/f68 1438571 0 2024-03-24T00:49:03.341 INFO:tasks.workunit.client.0.smithi038.stdout:2/649: chown d4/d76/fb1 2408277 1 2024-03-24T00:49:03.348 INFO:tasks.workunit.client.0.smithi038.stdout:2/650: read d4/f24 [223459,68706] 0 2024-03-24T00:49:03.348 INFO:tasks.workunit.client.0.smithi038.stdout:2/651: readlink d4/d27/d54/d67/lbd 0 2024-03-24T00:49:03.348 INFO:tasks.workunit.client.0.smithi038.stdout:2/652: fdatasync d4/d18/da1/fa4 0 2024-03-24T00:49:03.348 INFO:tasks.workunit.client.0.smithi038.stdout:2/653: dread - d4/d18/d26/f62 zero size 2024-03-24T00:49:03.349 INFO:tasks.workunit.client.1.smithi088.stdout:8/738: mknod d9/dc/d11/d24/d29/da9/cfb 0 2024-03-24T00:49:03.349 INFO:tasks.workunit.client.1.smithi088.stdout:8/739: chown d9/dc/d11/d80/db0/f90 1 1 2024-03-24T00:49:03.350 INFO:tasks.workunit.client.0.smithi038.stdout:2/654: creat d4/d18/fe1 x:0 0 0 2024-03-24T00:49:03.357 INFO:tasks.workunit.client.0.smithi038.stdout:5/655: dwrite d6/d16/d21/f5f [0,4194304] 0 2024-03-24T00:49:03.374 INFO:tasks.workunit.client.0.smithi038.stdout:9/803: truncate d12/d28/da8/fb1 1101685 0 2024-03-24T00:49:03.375 INFO:tasks.workunit.client.0.smithi038.stdout:9/804: stat d12/d41/fcc 0 2024-03-24T00:49:03.378 INFO:tasks.workunit.client.1.smithi088.stdout:0/784: creat d9/d26/d8f/da1/d62/d9b/f100 x:0 0 0 2024-03-24T00:49:03.379 INFO:tasks.workunit.client.0.smithi038.stdout:8/739: rename d0/d1d/d20/d3c to d0/d1d/d27/d6b/dfc 0 2024-03-24T00:49:03.395 INFO:tasks.workunit.client.1.smithi088.stdout:3/712: truncate d3/d6/d27/d3c/f30 6965749 0 2024-03-24T00:49:03.395 INFO:tasks.workunit.client.1.smithi088.stdout:3/713: write d3/de/d6f/d85/f9a [1294115,117223] 0 2024-03-24T00:49:03.396 INFO:tasks.workunit.client.0.smithi038.stdout:0/644: mkdir d0/dd7 0 2024-03-24T00:49:03.397 INFO:tasks.workunit.client.0.smithi038.stdout:0/645: chown d0/l9 1 1 2024-03-24T00:49:03.397 INFO:tasks.workunit.client.0.smithi038.stdout:0/646: fdatasync d0/d14/d84/fa4 0 2024-03-24T00:49:03.402 INFO:tasks.workunit.client.1.smithi088.stdout:1/760: write d6/d23/daf/d95/d5b/f75 [2500387,1539] 0 2024-03-24T00:49:03.403 INFO:tasks.workunit.client.1.smithi088.stdout:9/761: symlink d1/d10/d2c/lf7 0 2024-03-24T00:49:03.403 INFO:tasks.workunit.client.1.smithi088.stdout:9/762: creat d1/d10/d29/def/ff8 x:0 0 0 2024-03-24T00:49:03.404 INFO:tasks.workunit.client.1.smithi088.stdout:1/761: write d6/d23/daf/d95/d5b/d5d/d61/d7d/dc8/fd1 [66291,69376] 0 2024-03-24T00:49:03.406 INFO:tasks.workunit.client.1.smithi088.stdout:2/714: sync 2024-03-24T00:49:03.406 INFO:tasks.workunit.client.1.smithi088.stdout:5/790: mknod d1/dd/d6f/d98/da0/c100 0 2024-03-24T00:49:03.406 INFO:tasks.workunit.client.1.smithi088.stdout:5/791: creat d1/d19/db9/f101 x:0 0 0 2024-03-24T00:49:03.414 INFO:tasks.workunit.client.0.smithi038.stdout:7/673: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe 0 2024-03-24T00:49:03.415 INFO:tasks.workunit.client.0.smithi038.stdout:4/681: dwrite da/d12/d37/dd/d15/d4c/f6a [0,4194304] 0 2024-03-24T00:49:03.416 INFO:tasks.workunit.client.0.smithi038.stdout:9/805: dread d12/d41/ff4 [0,4194304] 0 2024-03-24T00:49:03.416 INFO:tasks.workunit.client.0.smithi038.stdout:9/806: getdents d12/d1a/d5f/df9 0 2024-03-24T00:49:03.420 INFO:tasks.workunit.client.0.smithi038.stdout:8/740: symlink d0/d45/lfd 0 2024-03-24T00:49:03.424 INFO:tasks.workunit.client.0.smithi038.stdout:5/656: dwrite d6/d1e/d97/fbe [0,4194304] 0 2024-03-24T00:49:03.424 INFO:tasks.workunit.client.0.smithi038.stdout:1/775: rename da/dd/d7a/d82/d98/fba to da/dd/d22/dab/f107 0 2024-03-24T00:49:03.424 INFO:tasks.workunit.client.0.smithi038.stdout:1/776: truncate da/f35 3088211 0 2024-03-24T00:49:03.424 INFO:tasks.workunit.client.0.smithi038.stdout:1/777: chown da/dd/d7a/d82/d98/f1f 424310305 1 2024-03-24T00:49:03.427 INFO:tasks.workunit.client.0.smithi038.stdout:0/647: symlink d0/ld8 0 2024-03-24T00:49:03.437 INFO:tasks.workunit.client.0.smithi038.stdout:2/655: dwrite d4/f66 [0,4194304] 0 2024-03-24T00:49:03.437 INFO:tasks.workunit.client.0.smithi038.stdout:2/656: fdatasync d4/f7a 0 2024-03-24T00:49:03.439 INFO:tasks.workunit.client.1.smithi088.stdout:7/692: truncate d3/f51 503365 0 2024-03-24T00:49:03.441 INFO:tasks.workunit.client.1.smithi088.stdout:8/740: rmdir d9/dc/d11/d24/d29/d36/d6a/d7b/d8f 39 2024-03-24T00:49:03.441 INFO:tasks.workunit.client.1.smithi088.stdout:8/741: readlink d9/dc/d11/d24/d29/da9/lb1 0 2024-03-24T00:49:03.445 INFO:tasks.workunit.client.1.smithi088.stdout:4/748: mkdir d5/d7/d72/d106 0 2024-03-24T00:49:03.450 INFO:tasks.workunit.client.1.smithi088.stdout:0/785: mknod d9/c101 0 2024-03-24T00:49:03.453 INFO:tasks.workunit.client.1.smithi088.stdout:3/714: unlink d3/d1a/d1e/d2f/d45/db5/cd9 0 2024-03-24T00:49:03.462 INFO:tasks.workunit.client.1.smithi088.stdout:9/763: mknod d1/d15/d80/cf9 0 2024-03-24T00:49:03.464 INFO:tasks.workunit.client.1.smithi088.stdout:1/762: creat d6/d27/d67/d9b/db9/ff5 x:0 0 0 2024-03-24T00:49:03.471 INFO:tasks.workunit.client.1.smithi088.stdout:6/788: sync 2024-03-24T00:49:03.471 INFO:tasks.workunit.client.1.smithi088.stdout:6/789: read db/dd/f10 [302412,102297] 0 2024-03-24T00:49:03.472 INFO:tasks.workunit.client.0.smithi038.stdout:9/807: mknod d12/d81/d84/c103 0 2024-03-24T00:49:03.474 INFO:tasks.workunit.client.1.smithi088.stdout:5/792: mknod d1/dd/d6f/da1/c102 0 2024-03-24T00:49:03.478 INFO:tasks.workunit.client.1.smithi088.stdout:7/693: rename d3/d32/f8e to d3/d32/d34/d9c/d4b/d73/fdb 0 2024-03-24T00:49:03.482 INFO:tasks.workunit.client.1.smithi088.stdout:8/742: unlink d9/dc/d11/d24/f71 0 2024-03-24T00:49:03.483 INFO:tasks.workunit.client.1.smithi088.stdout:4/749: mkdir d5/d7/de4/d107 0 2024-03-24T00:49:03.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.483+0000 7f211ae00700 1 -- 172.21.15.38:0/784385592 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21140611f0 con 0x7f20f806d6d0 2024-03-24T00:49:03.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.485+0000 7f2110c00700 1 -- 172.21.15.38:0/784385592 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f21140611f0 con 0x7f20f806d6d0 2024-03-24T00:49:03.487 INFO:tasks.workunit.client.0.smithi038.stdout:5/657: symlink d6/d1e/d97/dd1/ld7 0 2024-03-24T00:49:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.488+0000 7f211ae00700 1 -- 172.21.15.38:0/784385592 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f20f806d6d0 msgr2=0x7f20f806fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:03.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.488+0000 7f211ae00700 1 --2- 172.21.15.38:0/784385592 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f20f806d6d0 0x7f20f806fb90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f210c005950 tx=0x7f210c00b410 comp rx=0 tx=0).stop 2024-03-24T00:49:03.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.488+0000 7f211ae00700 1 -- 172.21.15.38:0/784385592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114198e40 msgr2=0x7f211419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:03.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.488+0000 7f211ae00700 1 --2- 172.21.15.38:0/784385592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114198e40 0x7f211419de10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f20fc000c00 tx=0x7f20fc00bd70 comp rx=0 tx=0).stop 2024-03-24T00:49:03.491 INFO:tasks.workunit.client.0.smithi038.stdout:1/778: dwrite da/dd/d2c/d41/d60/fc5 [0,4194304] 0 2024-03-24T00:49:03.491 INFO:tasks.workunit.client.0.smithi038.stdout:6/782: rename d1/da/l50 to d1/d37/d84/l101 0 2024-03-24T00:49:03.492 INFO:tasks.workunit.client.1.smithi088.stdout:0/786: mknod d9/dc/de/d19/d44/de2/c102 0 2024-03-24T00:49:03.492 INFO:tasks.workunit.client.1.smithi088.stdout:3/715: creat d3/de/d6f/d85/d49/d70/d96/fe3 x:0 0 0 2024-03-24T00:49:03.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.491+0000 7f211ae00700 1 -- 172.21.15.38:0/784385592 shutdown_connections 2024-03-24T00:49:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.491+0000 7f211ae00700 1 --2- 172.21.15.38:0/784385592 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f20f806d6d0 0x7f20f806fb90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.491+0000 7f211ae00700 1 --2- 172.21.15.38:0/784385592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2114101560 0x7f2114198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.491+0000 7f211ae00700 1 --2- 172.21.15.38:0/784385592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114198e40 0x7f211419de10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.491+0000 7f211ae00700 1 -- 172.21.15.38:0/784385592 >> 172.21.15.38:0/784385592 conn(0x7f21140faf70 msgr2=0x7f2114101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.492+0000 7f211ae00700 1 -- 172.21.15.38:0/784385592 shutdown_connections 2024-03-24T00:49:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.492+0000 7f211ae00700 1 -- 172.21.15.38:0/784385592 wait complete. 2024-03-24T00:49:03.494 INFO:tasks.workunit.client.0.smithi038.stdout:0/648: unlink d0/d1b/c45 0 2024-03-24T00:49:03.495 INFO:tasks.workunit.client.1.smithi088.stdout:2/715: mknod db/d21/ce1 0 2024-03-24T00:49:03.497 INFO:tasks.workunit.client.1.smithi088.stdout:6/790: mknod db/dd/d61/cf4 0 2024-03-24T00:49:03.497 INFO:tasks.workunit.client.1.smithi088.stdout:6/791: read f8 [829832,41362] 0 2024-03-24T00:49:03.498 INFO:tasks.workunit.client.1.smithi088.stdout:6/792: readlink db/d8e/lcd 0 2024-03-24T00:49:03.500 INFO:tasks.workunit.client.0.smithi038.stdout:3/704: sync 2024-03-24T00:49:03.507 INFO:tasks.workunit.client.1.smithi088.stdout:7/694: unlink d3/d32/d34/d9c/d4b/d73/f92 0 2024-03-24T00:49:03.511 INFO:tasks.workunit.client.0.smithi038.stdout:5/658: write d6/d16/f2a [2320491,2812] 0 2024-03-24T00:49:03.511 INFO:tasks.workunit.client.0.smithi038.stdout:5/659: fsync d6/d19/d20/d42/f72 0 2024-03-24T00:49:03.511 INFO:tasks.workunit.client.0.smithi038.stdout:4/682: symlink da/d12/d37/dd/lea 0 2024-03-24T00:49:03.511 INFO:tasks.workunit.client.0.smithi038.stdout:9/808: unlink d12/d41/ff4 0 2024-03-24T00:49:03.512 INFO:tasks.workunit.client.0.smithi038.stdout:1/779: dread da/dd/d22/f25 [0,4194304] 0 2024-03-24T00:49:03.512 INFO:tasks.workunit.client.0.smithi038.stdout:1/780: truncate da/dd/d7a/d82/fb4 970254 0 2024-03-24T00:49:03.512 INFO:tasks.workunit.client.1.smithi088.stdout:0/787: write d9/d26/f5a [4957413,21849] 0 2024-03-24T00:49:03.514 INFO:tasks.workunit.client.0.smithi038.stdout:7/674: dwrite d2/d2c/f74 [0,4194304] 0 2024-03-24T00:49:03.520 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:49:03.520 INFO:tasks.workunit.client.1.smithi088.stdout:8/743: mknod d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/cfc 0 2024-03-24T00:49:03.520 INFO:tasks.workunit.client.1.smithi088.stdout:8/744: readlink d9/dc/d11/d24/d29/d39/d84/lbb 0 2024-03-24T00:49:03.523 INFO:tasks.workunit.client.1.smithi088.stdout:3/716: write d3/d67/dc3/dd7/fe1 [4201248,94205] 0 2024-03-24T00:49:03.524 INFO:tasks.workunit.client.1.smithi088.stdout:4/750: link d5/d7/d15/d74/d95/f99 d5/d7/d72/d106/f108 0 2024-03-24T00:49:03.524 INFO:tasks.workunit.client.1.smithi088.stdout:4/751: chown d5/d17/lc2 310719 1 2024-03-24T00:49:03.524 INFO:tasks.workunit.client.1.smithi088.stdout:2/716: symlink db/dd/d96/d4e/d8f/d9c/db0/le2 0 2024-03-24T00:49:03.527 INFO:tasks.workunit.client.1.smithi088.stdout:6/793: link db/d1d/d50/d90/d56/c4a db/dd/cf5 0 2024-03-24T00:49:03.529 INFO:tasks.workunit.client.0.smithi038.stdout:6/783: mkdir d1/d37/d68/d102 0 2024-03-24T00:49:03.534 INFO:tasks.workunit.client.1.smithi088.stdout:5/793: rename d1/d19/db9/f101 to d1/d10/d1b/d2f/f103 0 2024-03-24T00:49:03.536 INFO:tasks.workunit.client.0.smithi038.stdout:2/657: rename d4/d27/d2f/d89 to d4/d27/d54/d53/de2 0 2024-03-24T00:49:03.536 INFO:tasks.workunit.client.0.smithi038.stdout:0/649: rmdir d0/d12/d42/d9d/d28 39 2024-03-24T00:49:03.537 INFO:tasks.workunit.client.1.smithi088.stdout:7/695: unlink d3/d31/l62 0 2024-03-24T00:49:03.537 INFO:tasks.workunit.client.1.smithi088.stdout:7/696: chown d3/d31/fcc 1934 1 2024-03-24T00:49:03.537 INFO:tasks.workunit.client.0.smithi038.stdout:3/705: rmdir d3/d9/dc/da6/db5/d19/d82/d89/da8 39 2024-03-24T00:49:03.537 INFO:tasks.workunit.client.0.smithi038.stdout:3/706: truncate d3/f8c 1102703 0 2024-03-24T00:49:03.539 INFO:tasks.workunit.client.1.smithi088.stdout:0/788: unlink d9/d26/d8f/da1/d62/dc4/ld1 0 2024-03-24T00:49:03.539 INFO:tasks.workunit.client.1.smithi088.stdout:0/789: stat d9/dc/lda 0 2024-03-24T00:49:03.539 INFO:tasks.workunit.client.1.smithi088.stdout:0/790: fdatasync d9/dc/de/d19/d44/f5b 0 2024-03-24T00:49:03.541 INFO:tasks.workunit.client.1.smithi088.stdout:9/764: dwrite d1/d22/d27/fbf [4194304,4194304] 0 2024-03-24T00:49:03.541 INFO:tasks.workunit.client.1.smithi088.stdout:8/745: mknod d9/dc/d11/d24/d29/d36/d6a/cfd 0 2024-03-24T00:49:03.541 INFO:tasks.workunit.client.1.smithi088.stdout:8/746: write d9/dc/d11/d24/d29/d5b/d8b/dd3/fed [1843516,27605] 0 2024-03-24T00:49:03.542 INFO:tasks.workunit.client.0.smithi038.stdout:4/683: mknod da/d12/d24/d5f/d8a/db6/dd1/ceb 0 2024-03-24T00:49:03.548 INFO:tasks.workunit.client.0.smithi038.stdout:4/684: readlink da/d12/d37/dd/d15/d6d/ldf 0 2024-03-24T00:49:03.548 INFO:tasks.workunit.client.1.smithi088.stdout:4/752: dread d5/d16/fb1 [0,4194304] 0 2024-03-24T00:49:03.548 INFO:tasks.workunit.client.1.smithi088.stdout:1/763: dwrite d6/fba [0,4194304] 0 2024-03-24T00:49:03.549 INFO:tasks.workunit.client.0.smithi038.stdout:5/660: write d6/d19/f1c [6167988,128038] 0 2024-03-24T00:49:03.552 INFO:tasks.workunit.client.1.smithi088.stdout:2/717: mknod db/dd/d96/d99/db2/d88/ce3 0 2024-03-24T00:49:03.552 INFO:tasks.workunit.client.1.smithi088.stdout:6/794: creat db/d3d/d83/deb/ff6 x:0 0 0 2024-03-24T00:49:03.553 INFO:tasks.workunit.client.0.smithi038.stdout:2/658: rmdir d4/d27/ddb/d7c 39 2024-03-24T00:49:03.553 INFO:tasks.workunit.client.0.smithi038.stdout:2/659: write d4/f19 [1025386,4603] 0 2024-03-24T00:49:03.557 INFO:tasks.workunit.client.0.smithi038.stdout:0/650: mkdir d0/daf/dd9 0 2024-03-24T00:49:03.557 INFO:tasks.workunit.client.1.smithi088.stdout:5/794: mknod d1/d10/d79/dbc/c104 0 2024-03-24T00:49:03.557 INFO:tasks.workunit.client.1.smithi088.stdout:7/697: mkdir d3/d32/d34/d9c/d4b/ddc 0 2024-03-24T00:49:03.557 INFO:tasks.workunit.client.1.smithi088.stdout:5/795: chown d1/d19/d1e/l4c 3627 1 2024-03-24T00:49:03.557 INFO:tasks.workunit.client.1.smithi088.stdout:5/796: fdatasync d1/d10/d1b/d2f/d57/f8d 0 2024-03-24T00:49:03.558 INFO:tasks.workunit.client.0.smithi038.stdout:3/707: mkdir d3/d9/def 0 2024-03-24T00:49:03.563 INFO:tasks.workunit.client.0.smithi038.stdout:9/809: truncate d12/d28/d6c/f74 2309790 0 2024-03-24T00:49:03.567 INFO:tasks.workunit.client.0.smithi038.stdout:6/784: write d1/da/d1b/fa1 [3548045,128019] 0 2024-03-24T00:49:03.567 INFO:tasks.workunit.client.0.smithi038.stdout:6/785: dread - d1/da/d61/da2/fef zero size 2024-03-24T00:49:03.568 INFO:tasks.workunit.client.0.smithi038.stdout:4/685: dread da/d12/d24/d68/f82 [0,4194304] 0 2024-03-24T00:49:03.568 INFO:tasks.workunit.client.1.smithi088.stdout:9/765: symlink d1/d10/d29/def/lfa 0 2024-03-24T00:49:03.569 INFO:tasks.workunit.client.1.smithi088.stdout:9/766: dread - d1/d15/d80/fb3 zero size 2024-03-24T00:49:03.571 INFO:tasks.workunit.client.0.smithi038.stdout:7/675: read d2/d2c/f74 [3708591,64914] 0 2024-03-24T00:49:03.573 INFO:tasks.workunit.client.0.smithi038.stdout:0/651: mknod d0/d14/d88/cda 0 2024-03-24T00:49:03.584 INFO:tasks.workunit.client.0.smithi038.stdout:3/708: mkdir d3/d9/dc/da6/db5/d19/dd2/df0 0 2024-03-24T00:49:03.585 INFO:tasks.workunit.client.1.smithi088.stdout:3/717: rmdir d3/de/d6f 39 2024-03-24T00:49:03.585 INFO:tasks.workunit.client.1.smithi088.stdout:3/718: truncate d3/d6/d27/d3c/f47 4485297 0 2024-03-24T00:49:03.585 INFO:tasks.workunit.client.1.smithi088.stdout:3/719: creat d3/d1a/fe4 x:0 0 0 2024-03-24T00:49:03.585 INFO:tasks.workunit.client.1.smithi088.stdout:3/720: fdatasync d3/d6/d27/d3c/fa8 0 2024-03-24T00:49:03.585 INFO:tasks.workunit.client.1.smithi088.stdout:3/721: getdents d3/d6/d19/ddb 0 2024-03-24T00:49:03.590 INFO:tasks.workunit.client.1.smithi088.stdout:4/753: creat d5/d16/d37/de7/f109 x:0 0 0 2024-03-24T00:49:03.594 INFO:tasks.workunit.client.0.smithi038.stdout:2/660: dread d4/d27/f2a [0,4194304] 0 2024-03-24T00:49:03.595 INFO:tasks.workunit.client.0.smithi038.stdout:9/810: dread d12/d28/d71/fbe [0,4194304] 0 2024-03-24T00:49:03.595 INFO:tasks.workunit.client.0.smithi038.stdout:8/741: dwrite d0/d45/d37/d5f/db7/d62/fad [0,4194304] 0 2024-03-24T00:49:03.599 INFO:tasks.workunit.client.0.smithi038.stdout:6/786: mknod d1/da/d61/d79/c103 0 2024-03-24T00:49:03.599 INFO:tasks.workunit.client.0.smithi038.stdout:6/787: fdatasync d1/d37/d84/fc5 0 2024-03-24T00:49:03.602 INFO:tasks.workunit.client.1.smithi088.stdout:6/795: mkdir db/dc5/dc8/d43/df7 0 2024-03-24T00:49:03.602 INFO:tasks.workunit.client.1.smithi088.stdout:6/796: truncate db/dd/d61/d6d/d74/d88/f9b 575145 0 2024-03-24T00:49:03.602 INFO:tasks.workunit.client.1.smithi088.stdout:6/797: chown db/d1d/d23/d5c/c96 1 1 2024-03-24T00:49:03.602 INFO:tasks.workunit.client.1.smithi088.stdout:6/798: chown db/dc5/dc8/d43 593427718 1 2024-03-24T00:49:03.613 INFO:tasks.workunit.client.1.smithi088.stdout:3/722: fsync d3/de/d6f/d85/f9a 0 2024-03-24T00:49:03.617 INFO:tasks.workunit.client.1.smithi088.stdout:7/698: stat d3/d32/l53 0 2024-03-24T00:49:03.618 INFO:tasks.workunit.client.0.smithi038.stdout:1/781: rename da/dd/d2c/db6 to da/dd/d2c/d5a/d101/de8/d108 0 2024-03-24T00:49:03.618 INFO:tasks.workunit.client.0.smithi038.stdout:1/782: truncate da/dd/d22/dab/fbc 133331 0 2024-03-24T00:49:03.618 INFO:tasks.workunit.client.0.smithi038.stdout:1/783: fsync da/dd/d7a/d82/d98/d14/faf 0 2024-03-24T00:49:03.620 INFO:tasks.workunit.client.1.smithi088.stdout:4/754: dread f2 [4194304,4194304] 0 2024-03-24T00:49:03.625 INFO:tasks.workunit.client.0.smithi038.stdout:0/652: dread d0/d1b/f20 [0,4194304] 0 2024-03-24T00:49:03.625 INFO:tasks.workunit.client.0.smithi038.stdout:7/676: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/cff 0 2024-03-24T00:49:03.629 INFO:tasks.workunit.client.1.smithi088.stdout:4/755: dread d5/d16/d37/f5a [0,4194304] 0 2024-03-24T00:49:03.630 INFO:tasks.workunit.client.1.smithi088.stdout:0/791: dwrite d9/dc/de/d19/fbf [0,4194304] 0 2024-03-24T00:49:03.633 INFO:tasks.workunit.client.0.smithi038.stdout:0/653: creat d0/d12/d92/fdb x:0 0 0 2024-03-24T00:49:03.639 INFO:tasks.workunit.client.1.smithi088.stdout:3/723: write d3/d32/fad [1828714,97306] 0 2024-03-24T00:49:03.643 INFO:tasks.workunit.client.1.smithi088.stdout:9/767: symlink d1/lfb 0 2024-03-24T00:49:03.643 INFO:tasks.workunit.client.1.smithi088.stdout:9/768: fsync d1/d22/f51 0 2024-03-24T00:49:03.643 INFO:tasks.workunit.client.1.smithi088.stdout:9/769: write d1/d10/d2c/f75 [1755096,50274] 0 2024-03-24T00:49:03.647 INFO:tasks.workunit.client.0.smithi038.stdout:3/709: mknod d3/d9/dc/da6/cf1 0 2024-03-24T00:49:03.648 INFO:tasks.workunit.client.0.smithi038.stdout:5/661: dwrite d6/d1e/f63 [0,4194304] 0 2024-03-24T00:49:03.650 INFO:tasks.workunit.client.1.smithi088.stdout:8/747: dwrite d9/d6b/f6c [0,4194304] 0 2024-03-24T00:49:03.654 INFO:tasks.workunit.client.1.smithi088.stdout:9/770: write d1/d10/d11/fc3 [38451,1993] 0 2024-03-24T00:49:03.655 INFO:tasks.workunit.client.0.smithi038.stdout:4/686: dwrite da/f7d [0,4194304] 0 2024-03-24T00:49:03.655 INFO:tasks.workunit.client.1.smithi088.stdout:5/797: dwrite d1/d19/d1e/f6d [4194304,4194304] 0 2024-03-24T00:49:03.655 INFO:tasks.workunit.client.1.smithi088.stdout:5/798: chown d1/cce 3903 1 2024-03-24T00:49:03.657 INFO:tasks.workunit.client.0.smithi038.stdout:2/661: unlink d4/d27/d2f/l51 0 2024-03-24T00:49:03.669 INFO:tasks.workunit.client.0.smithi038.stdout:0/654: write d0/d14/d84/fc5 [4083664,93567] 0 2024-03-24T00:49:03.670 INFO:tasks.workunit.client.1.smithi088.stdout:6/799: truncate db/dd/d61/d6d/fe1 893706 0 2024-03-24T00:49:03.673 INFO:tasks.workunit.client.0.smithi038.stdout:9/811: symlink d12/d41/d57/de1/l104 0 2024-03-24T00:49:03.673 INFO:tasks.workunit.client.0.smithi038.stdout:9/812: stat fc 0 2024-03-24T00:49:03.677 INFO:tasks.workunit.client.0.smithi038.stdout:5/662: write d6/d19/d20/d42/f72 [1987947,78923] 0 2024-03-24T00:49:03.680 INFO:tasks.workunit.client.0.smithi038.stdout:5/663: read d6/d16/d3c/f93 [392465,126712] 0 2024-03-24T00:49:03.680 INFO:tasks.workunit.client.0.smithi038.stdout:5/664: dread - d6/d19/d20/d33/d6f/fd3 zero size 2024-03-24T00:49:03.680 INFO:tasks.workunit.client.0.smithi038.stdout:5/665: write d6/d19/d20/d42/d43/d84/fc6 [642294,35748] 0 2024-03-24T00:49:03.681 INFO:tasks.workunit.client.1.smithi088.stdout:1/764: dwrite d6/d27/f3a [0,4194304] 0 2024-03-24T00:49:03.687 INFO:tasks.workunit.client.1.smithi088.stdout:7/699: rename d3/d32/d34/d9c/d4b/d73/fdb to d3/d32/d34/d9c/d4b/d74/d79/d7f/fdd 0 2024-03-24T00:49:03.689 INFO:tasks.workunit.client.0.smithi038.stdout:8/742: link d0/d1d/d8c/lf8 d0/d1d/d27/d6b/dfc/d9e/lfe 0 2024-03-24T00:49:03.694 INFO:tasks.workunit.client.0.smithi038.stdout:6/788: rename d1/da/d1b/d1d/f2c to d1/da/d1b/d2f/dc0/f104 0 2024-03-24T00:49:03.701 INFO:tasks.workunit.client.0.smithi038.stdout:7/677: symlink d2/d9/da/l100 0 2024-03-24T00:49:03.701 INFO:tasks.workunit.client.0.smithi038.stdout:7/678: readlink d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/dcc/lf3 0 2024-03-24T00:49:03.703 INFO:tasks.workunit.client.0.smithi038.stdout:1/784: dwrite da/dd/d2c/d41/f65 [0,4194304] 0 2024-03-24T00:49:03.703 INFO:tasks.workunit.client.0.smithi038.stdout:1/785: chown da/dd/f85 0 1 2024-03-24T00:49:03.703 INFO:tasks.workunit.client.0.smithi038.stdout:1/786: creat da/dd/d7a/d82/f109 x:0 0 0 2024-03-24T00:49:03.703 INFO:tasks.workunit.client.0.smithi038.stdout:1/787: read - da/dd/d7a/d82/d98/d9e/f102 zero size 2024-03-24T00:49:03.703 INFO:tasks.workunit.client.0.smithi038.stdout:1/788: stat da/dd/d4a/c4e 0 2024-03-24T00:49:03.703 INFO:tasks.workunit.client.0.smithi038.stdout:1/789: fsync da/dd/d2c/d5a/d74/f64 0 2024-03-24T00:49:03.706 INFO:tasks.workunit.client.1.smithi088.stdout:8/748: dwrite d9/dc/d11/d24/f2a [0,4194304] 0 2024-03-24T00:49:03.717 INFO:tasks.workunit.client.1.smithi088.stdout:3/724: mknod d3/d1a/d1e/d2f/ce5 0 2024-03-24T00:49:03.718 INFO:tasks.workunit.client.0.smithi038.stdout:3/710: creat d3/d9/dc/da6/db5/d19/dd2/df0/ff2 x:0 0 0 2024-03-24T00:49:03.718 INFO:tasks.workunit.client.0.smithi038.stdout:3/711: chown d3/d30/f7a 1231933401 1 2024-03-24T00:49:03.719 INFO:tasks.workunit.client.0.smithi038.stdout:2/662: creat d4/d55/fe3 x:0 0 0 2024-03-24T00:49:03.725 INFO:tasks.workunit.client.1.smithi088.stdout:8/749: read d9/dc/d11/d24/d29/d39/fc9 [1076374,94422] 0 2024-03-24T00:49:03.725 INFO:tasks.workunit.client.1.smithi088.stdout:8/750: fdatasync d9/f32 0 2024-03-24T00:49:03.726 INFO:tasks.workunit.client.1.smithi088.stdout:3/725: write d3/de/d6f/d85/f9a [305194,25182] 0 2024-03-24T00:49:03.726 INFO:tasks.workunit.client.1.smithi088.stdout:3/726: fsync d3/d6/f2b 0 2024-03-24T00:49:03.733 INFO:tasks.workunit.client.0.smithi038.stdout:7/679: dread d2/d9/da/d23/d2b/d5b/d72/d92/d5d/f7f [0,4194304] 0 2024-03-24T00:49:03.734 INFO:tasks.workunit.client.0.smithi038.stdout:4/687: dwrite da/d38/f4a [0,4194304] 0 2024-03-24T00:49:03.737 INFO:tasks.workunit.client.0.smithi038.stdout:0/655: symlink d0/d12/d15/d58/d64/da7/dbb/dd4/ldc 0 2024-03-24T00:49:03.747 INFO:tasks.workunit.client.0.smithi038.stdout:5/666: symlink d6/d19/d20/d42/d43/d68/d6e/ld8 0 2024-03-24T00:49:03.750 INFO:tasks.workunit.client.1.smithi088.stdout:3/727: dread d3/de/f17 [0,4194304] 0 2024-03-24T00:49:03.750 INFO:tasks.workunit.client.1.smithi088.stdout:3/728: fsync d3/d1a/d1e/f28 0 2024-03-24T00:49:03.751 INFO:tasks.workunit.client.1.smithi088.stdout:3/729: readlink d3/d6/d27/lcf 0 2024-03-24T00:49:03.751 INFO:tasks.workunit.client.1.smithi088.stdout:5/799: mkdir d1/dd/d6f/da1/d3f/d88/d8f/d69/d105 0 2024-03-24T00:49:03.758 INFO:tasks.workunit.client.0.smithi038.stdout:6/789: symlink d1/d37/d68/l105 0 2024-03-24T00:49:03.758 INFO:tasks.workunit.client.0.smithi038.stdout:6/790: creat d1/da/d61/f106 x:0 0 0 2024-03-24T00:49:03.763 INFO:tasks.workunit.client.0.smithi038.stdout:8/743: dwrite d0/d45/d37/d5f/d68/f85 [0,4194304] 0 2024-03-24T00:49:03.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.768+0000 7f8d02a00700 1 -- 172.21.15.38:0/231530034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cfc103ed0 msgr2=0x7f8cfc1062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:03.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.768+0000 7f8d02a00700 1 --2- 172.21.15.38:0/231530034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cfc103ed0 0x7f8cfc1062c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009910 tx=0x7f8ce8009c20 comp rx=0 tx=0).stop 2024-03-24T00:49:03.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.769+0000 7f8d02a00700 1 -- 172.21.15.38:0/231530034 shutdown_connections 2024-03-24T00:49:03.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.769+0000 7f8d02a00700 1 --2- 172.21.15.38:0/231530034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cfc103ed0 0x7f8cfc1062c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:03.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.769+0000 7f8d02a00700 1 --2- 172.21.15.38:0/231530034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc1015a0 0x7f8cfc103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:03.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.769+0000 7f8d02a00700 1 -- 172.21.15.38:0/231530034 >> 172.21.15.38:0/231530034 conn(0x7f8cfc0faf70 msgr2=0x7f8cfc0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:03.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.770+0000 7f8d02a00700 1 -- 172.21.15.38:0/231530034 shutdown_connections 2024-03-24T00:49:03.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.770+0000 7f8d02a00700 1 -- 172.21.15.38:0/231530034 wait complete. 2024-03-24T00:49:03.772 INFO:tasks.workunit.client.0.smithi038.stdout:2/663: rmdir d4/d27/d54/d53/de2 39 2024-03-24T00:49:03.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.773+0000 7f8d02a00700 1 Processor -- start 2024-03-24T00:49:03.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.773+0000 7f8d02a00700 1 -- start start 2024-03-24T00:49:03.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.774+0000 7f8d02a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cfc1015a0 0x7f8cfc1942c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:03.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.774+0000 7f8d02a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc103ed0 0x7f8cfc194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:03.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.774+0000 7f8d02a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc194e20 con 0x7f8cfc103ed0 2024-03-24T00:49:03.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.774+0000 7f8d02a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc194f60 con 0x7f8cfc1015a0 2024-03-24T00:49:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.774+0000 7f8cfbe00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cfc1015a0 0x7f8cfc1942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.774+0000 7f8cfb400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc103ed0 0x7f8cfc194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.775+0000 7f8cfb400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc103ed0 0x7f8cfc194800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34708/0 (socket says 172.21.15.38:34708) 2024-03-24T00:49:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.775+0000 7f8cfb400700 1 -- 172.21.15.38:0/3064859370 learned_addr learned my addr 172.21.15.38:0/3064859370 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:49:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.775+0000 7f8cfb400700 1 -- 172.21.15.38:0/3064859370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cfc1015a0 msgr2=0x7f8cfc1942c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.775+0000 7f8cfb400700 1 --2- 172.21.15.38:0/3064859370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cfc1015a0 0x7f8cfc1942c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:03.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.775+0000 7f8cfb400700 1 -- 172.21.15.38:0/3064859370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce80095f0 con 0x7f8cfc103ed0 2024-03-24T00:49:03.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.776+0000 7f8cfb400700 1 --2- 172.21.15.38:0/3064859370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc103ed0 0x7f8cfc194800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8005100 tx=0x7f8ce800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:03.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.776+0000 7f8cf8c00700 1 -- 172.21.15.38:0/3064859370 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ce801d070 con 0x7f8cfc103ed0 2024-03-24T00:49:03.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.776+0000 7f8cf8c00700 1 -- 172.21.15.38:0/3064859370 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ce800f460 con 0x7f8cfc103ed0 2024-03-24T00:49:03.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.776+0000 7f8cf8c00700 1 -- 172.21.15.38:0/3064859370 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ce8005720 con 0x7f8cfc103ed0 2024-03-24T00:49:03.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.778+0000 7f8d02a00700 1 -- 172.21.15.38:0/3064859370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc199960 con 0x7f8cfc103ed0 2024-03-24T00:49:03.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.778+0000 7f8d02a00700 1 -- 172.21.15.38:0/3064859370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc199e20 con 0x7f8cfc103ed0 2024-03-24T00:49:03.779 INFO:tasks.workunit.client.0.smithi038.stdout:7/680: unlink d2/d9/da/d23/d2b/c91 0 2024-03-24T00:49:03.781 INFO:tasks.workunit.client.1.smithi088.stdout:6/800: mknod db/d3d/d83/deb/cf8 0 2024-03-24T00:49:03.781 INFO:tasks.workunit.client.1.smithi088.stdout:6/801: creat db/dd/d61/d6d/ff9 x:0 0 0 2024-03-24T00:49:03.781 INFO:tasks.workunit.client.1.smithi088.stdout:6/802: write f8 [859178,109380] 0 2024-03-24T00:49:03.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.780+0000 7f8cf8c00700 1 -- 172.21.15.38:0/3064859370 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 25) v1 ==== 91370+0+0 (secure 0 0 0) 0x7f8ce800f5d0 con 0x7f8cfc103ed0 2024-03-24T00:49:03.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.781+0000 7f8d02a00700 1 -- 172.21.15.38:0/3064859370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc04ea80 con 0x7f8cfc103ed0 2024-03-24T00:49:03.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.781+0000 7f8cf8c00700 1 --2- 172.21.15.38:0/3064859370 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f8ce0071a10 0x7f8ce0073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:03.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.782+0000 7f8cf8c00700 1 -- 172.21.15.38:0/3064859370 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f8ce808dd40 con 0x7f8cfc103ed0 2024-03-24T00:49:03.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.782+0000 7f8cfbe00700 1 --2- 172.21.15.38:0/3064859370 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f8ce0071a10 0x7f8ce0073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:03.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.784+0000 7f8cfbe00700 1 --2- 172.21.15.38:0/3064859370 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f8ce0071a10 0x7f8ce0073ed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8cec007c60 tx=0x7f8cec0073d0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:03.788 INFO:tasks.workunit.client.0.smithi038.stdout:4/688: symlink da/d12/d24/d5f/d8a/db6/dd1/lec 0 2024-03-24T00:49:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:03.788+0000 7f8cf8c00700 1 -- 172.21.15.38:0/3064859370 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f8ce805b170 con 0x7f8cfc103ed0 2024-03-24T00:49:03.791 INFO:tasks.workunit.client.1.smithi088.stdout:1/765: creat d6/d23/daf/d95/d3f/d53/ff6 x:0 0 0 2024-03-24T00:49:03.792 INFO:tasks.workunit.client.0.smithi038.stdout:0/656: creat d0/d12/d42/fdd x:0 0 0 2024-03-24T00:49:03.792 INFO:tasks.workunit.client.0.smithi038.stdout:0/657: fdatasync d0/d12/d42/d9d/f9e 0 2024-03-24T00:49:03.792 INFO:tasks.workunit.client.0.smithi038.stdout:0/658: truncate d0/d12/d42/d9d/fc7 749154 0 2024-03-24T00:49:03.794 INFO:tasks.workunit.client.1.smithi088.stdout:7/700: rename d3/d32/d34/d7c/c8a to d3/d32/d34/d9c/d4b/da7/cde 0 2024-03-24T00:49:03.796 INFO:tasks.workunit.client.1.smithi088.stdout:7/701: read d3/d32/d34/f72 [278813,45076] 0 2024-03-24T00:49:03.799 INFO:tasks.workunit.client.1.smithi088.stdout:0/792: getdents d9/dc/de/d19 0 2024-03-24T00:49:03.799 INFO:tasks.workunit.client.1.smithi088.stdout:0/793: stat d9/dc/de/d19/d44/d46/d65/d2a/d2e/lf8 0 2024-03-24T00:49:03.806 INFO:tasks.workunit.client.0.smithi038.stdout:9/813: link d12/d28/d9c/ddd/ffa d12/d81/f105 0 2024-03-24T00:49:03.807 INFO:tasks.workunit.client.0.smithi038.stdout:9/814: fsync d12/f1e 0 2024-03-24T00:49:03.807 INFO:tasks.workunit.client.0.smithi038.stdout:9/815: chown d12/d81/d84/d88/fa9 65 1 2024-03-24T00:49:03.810 INFO:tasks.workunit.client.1.smithi088.stdout:8/751: creat d9/dc/d11/d24/d29/d36/dc1/dc4/ffe x:0 0 0 2024-03-24T00:49:03.817 INFO:tasks.workunit.client.0.smithi038.stdout:5/667: unlink d6/d19/l94 0 2024-03-24T00:49:03.825 INFO:tasks.workunit.client.1.smithi088.stdout:3/730: creat d3/d6/d58/fe6 x:0 0 0 2024-03-24T00:49:03.828 INFO:tasks.workunit.client.0.smithi038.stdout:6/791: rename d1/da/le7 to d1/da/d1b/d25/d49/d4e/l107 0 2024-03-24T00:49:03.831 INFO:tasks.workunit.client.1.smithi088.stdout:3/731: write d3/f97 [539127,95935] 0 2024-03-24T00:49:03.832 INFO:tasks.workunit.client.1.smithi088.stdout:5/800: mkdir d1/dd/d6f/da1/d3f/d106 0 2024-03-24T00:49:03.835 INFO:tasks.workunit.client.1.smithi088.stdout:2/718: sync 2024-03-24T00:49:03.836 INFO:tasks.workunit.client.0.smithi038.stdout:8/744: creat d0/d45/d37/d5f/db7/dcc/fff x:0 0 0 2024-03-24T00:49:03.836 INFO:tasks.workunit.client.0.smithi038.stdout:8/745: write d0/d45/d37/d5f/db7/dcc/ff7 [542036,75548] 0 2024-03-24T00:49:03.838 INFO:tasks.workunit.client.1.smithi088.stdout:5/801: read d1/d10/d1b/d2f/fe7 [843174,114873] 0 2024-03-24T00:49:03.838 INFO:tasks.workunit.client.1.smithi088.stdout:5/802: truncate d1/fb 1629022 0 2024-03-24T00:49:03.844 INFO:tasks.workunit.client.0.smithi038.stdout:6/792: dread d1/da/f1a [0,4194304] 0 2024-03-24T00:49:03.847 INFO:tasks.workunit.client.0.smithi038.stdout:3/712: rmdir d3/d2a 39 2024-03-24T00:49:03.848 INFO:tasks.workunit.client.1.smithi088.stdout:9/771: sync 2024-03-24T00:49:03.848 INFO:tasks.workunit.client.1.smithi088.stdout:4/756: sync 2024-03-24T00:49:03.848 INFO:tasks.workunit.client.1.smithi088.stdout:9/772: chown d1/d10/d11/fcb 403 1 2024-03-24T00:49:03.848 INFO:tasks.workunit.client.1.smithi088.stdout:4/757: read - d5/d13/d18/d24/d3d/fb7 zero size 2024-03-24T00:49:03.849 INFO:tasks.workunit.client.1.smithi088.stdout:4/758: creat d5/d16/d37/de7/f10a x:0 0 0 2024-03-24T00:49:03.849 INFO:tasks.workunit.client.1.smithi088.stdout:4/759: stat d5/d16/d37/lca 0 2024-03-24T00:49:03.851 INFO:tasks.workunit.client.1.smithi088.stdout:7/702: rename d3/f15 to d3/d32/d34/d9c/d4b/fdf 0 2024-03-24T00:49:03.851 INFO:tasks.workunit.client.1.smithi088.stdout:7/703: fdatasync d3/d31/f7a 0 2024-03-24T00:49:03.858 INFO:tasks.workunit.client.0.smithi038.stdout:5/668: dwrite d6/d19/d20/d42/fd5 [0,4194304] 0 2024-03-24T00:49:03.858 INFO:tasks.workunit.client.0.smithi038.stdout:2/664: link d4/d18/d26/d37/d69/d83/l84 d4/d27/d54/d53/de2/le4 0 2024-03-24T00:49:03.860 INFO:tasks.workunit.client.1.smithi088.stdout:0/794: truncate d9/dc/de/d19/d44/d46/d65/f35 2778201 0 2024-03-24T00:49:03.861 INFO:tasks.workunit.client.0.smithi038.stdout:7/681: getdents d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d 0 2024-03-24T00:49:03.863 INFO:tasks.workunit.client.1.smithi088.stdout:3/732: mknod d3/d6/d19/d2a/ce7 0 2024-03-24T00:49:03.865 INFO:tasks.workunit.client.0.smithi038.stdout:0/659: symlink d0/d12/d15/d58/lde 0 2024-03-24T00:49:03.865 INFO:tasks.workunit.client.0.smithi038.stdout:0/660: truncate d0/d14/d84/f85 992369 0 2024-03-24T00:49:03.866 INFO:tasks.workunit.client.1.smithi088.stdout:2/719: rmdir db/dc 39 2024-03-24T00:49:03.867 INFO:tasks.workunit.client.1.smithi088.stdout:5/803: mkdir d1/d10/d1b/d2f/d8a/deb/d107 0 2024-03-24T00:49:03.868 INFO:tasks.workunit.client.1.smithi088.stdout:5/804: truncate d1/d10/d1b/d2f/f7a 962455 0 2024-03-24T00:49:03.869 INFO:tasks.workunit.client.1.smithi088.stdout:6/803: rmdir db/dd/d61/d6d/d74/d88/da1 39 2024-03-24T00:49:03.869 INFO:tasks.workunit.client.1.smithi088.stdout:6/804: write db/dc5/dc8/d43/f34 [2380573,31572] 0 2024-03-24T00:49:03.870 INFO:tasks.workunit.client.0.smithi038.stdout:0/661: creat d0/d12/d15/d1a/fdf x:0 0 0 2024-03-24T00:49:03.870 INFO:tasks.workunit.client.0.smithi038.stdout:6/793: dread d1/da/d1b/d25/d49/d4e/d51/f9c [0,4194304] 0 2024-03-24T00:49:03.870 INFO:tasks.workunit.client.0.smithi038.stdout:9/816: write d12/d28/d6c/dbc/dd7/dda/fe2 [1909998,73800] 0 2024-03-24T00:49:03.871 INFO:tasks.workunit.client.0.smithi038.stdout:9/817: chown d12/d1a/d80/l87 220747 1 2024-03-24T00:49:03.874 INFO:tasks.workunit.client.0.smithi038.stdout:8/746: mkdir d0/d1d/d27/d6b/dfc/d100 0 2024-03-24T00:49:03.875 INFO:tasks.workunit.client.0.smithi038.stdout:8/747: write d0/d45/d37/d5f/d68/f91 [3162781,90368] 0 2024-03-24T00:49:03.880 INFO:tasks.workunit.client.1.smithi088.stdout:1/766: symlink d6/d23/daf/d95/d5b/lf7 0 2024-03-24T00:49:03.880 INFO:tasks.workunit.client.0.smithi038.stdout:3/713: symlink d3/d9/def/lf3 0 2024-03-24T00:49:03.886 INFO:tasks.workunit.client.1.smithi088.stdout:7/704: symlink d3/dbf/le0 0 2024-03-24T00:49:03.888 INFO:tasks.workunit.client.1.smithi088.stdout:0/795: dread d9/f45 [0,4194304] 0 2024-03-24T00:49:03.891 INFO:tasks.workunit.client.0.smithi038.stdout:6/794: dread d1/da/d61/fcd [4194304,4194304] 0 2024-03-24T00:49:03.892 INFO:tasks.workunit.client.1.smithi088.stdout:3/733: truncate d3/d32/fad 2725090 0 2024-03-24T00:49:03.896 INFO:tasks.workunit.client.0.smithi038.stdout:2/665: symlink d4/d27/ddb/d9b/le5 0 2024-03-24T00:49:03.896 INFO:tasks.workunit.client.0.smithi038.stdout:2/666: write d4/d27/d8f/d9e/fa2 [543227,102371] 0 2024-03-24T00:49:03.896 INFO:tasks.workunit.client.0.smithi038.stdout:2/667: dread - d4/fdd zero size 2024-03-24T00:49:03.900 INFO:tasks.workunit.client.1.smithi088.stdout:5/805: mknod d1/dd/d6f/da1/d3f/c108 0 2024-03-24T00:49:03.902 INFO:tasks.workunit.client.0.smithi038.stdout:4/689: dwrite da/d12/d24/d68/d6c/f79 [0,4194304] 0 2024-03-24T00:49:03.904 INFO:tasks.workunit.client.1.smithi088.stdout:6/805: creat db/d1d/d50/d90/ffa x:0 0 0 2024-03-24T00:49:03.904 INFO:tasks.workunit.client.1.smithi088.stdout:6/806: chown db/d1d/d50/d90/d55/lb2 53 1 2024-03-24T00:49:03.904 INFO:tasks.workunit.client.1.smithi088.stdout:6/807: write db/dd/d61/d6d/d74/d88/f9c [764497,61298] 0 2024-03-24T00:49:03.907 INFO:tasks.workunit.client.0.smithi038.stdout:3/714: symlink d3/d30/d70/d90/lf4 0 2024-03-24T00:49:03.907 INFO:tasks.workunit.client.0.smithi038.stdout:3/715: dread - d3/d30/fac zero size 2024-03-24T00:49:03.907 INFO:tasks.workunit.client.0.smithi038.stdout:3/716: chown d3/d9/dc/da6/db5/dad 14107951 1 2024-03-24T00:49:03.908 INFO:tasks.workunit.client.1.smithi088.stdout:1/767: rename d6/d23/daf/d95/d58/f51 to d6/d27/db0/ff8 0 2024-03-24T00:49:03.909 INFO:tasks.workunit.client.0.smithi038.stdout:6/795: creat d1/d37/d84/f108 x:0 0 0 2024-03-24T00:49:03.914 INFO:tasks.workunit.client.1.smithi088.stdout:3/734: link d3/d6/d27/d3c/c74 d3/de/ce8 0 2024-03-24T00:49:03.914 INFO:tasks.workunit.client.1.smithi088.stdout:2/720: unlink db/dd/d96/d4e/fd3 0 2024-03-24T00:49:03.914 INFO:tasks.workunit.client.1.smithi088.stdout:2/721: chown db/c14 42 1 2024-03-24T00:49:03.917 INFO:tasks.workunit.client.1.smithi088.stdout:5/806: symlink d1/dd/d6f/da1/d3f/d88/d8f/d69/l109 0 2024-03-24T00:49:03.918 INFO:tasks.workunit.client.0.smithi038.stdout:5/669: rename d6/d16/d3c to d6/d19/dd9 0 2024-03-24T00:49:03.921 INFO:tasks.workunit.client.0.smithi038.stdout:3/717: truncate d3/d9/dc/df/f27 4015717 0 2024-03-24T00:49:03.921 INFO:tasks.workunit.client.0.smithi038.stdout:6/796: rmdir d1/da/d1b/d25/d49/d4e 39 2024-03-24T00:49:03.921 INFO:tasks.workunit.client.0.smithi038.stdout:6/797: read d1/da/d1b/d25/f2a [251938,33407] 0 2024-03-24T00:49:03.921 INFO:tasks.workunit.client.0.smithi038.stdout:6/798: write d1/d37/d84/f108 [190059,34074] 0 2024-03-24T00:49:03.921 INFO:tasks.workunit.client.0.smithi038.stdout:6/799: write d1/da/d1b/d25/d49/d72/fe2 [461803,107434] 0 2024-03-24T00:49:03.921 INFO:tasks.workunit.client.0.smithi038.stdout:6/800: chown d1/da/d61/da2/db1 3 1 2024-03-24T00:49:03.922 INFO:tasks.workunit.client.1.smithi088.stdout:3/735: rename d3/de/d6f/d89 to d3/d6/d27/de9 0 2024-03-24T00:49:03.922 INFO:tasks.workunit.client.1.smithi088.stdout:3/736: chown d3/d1a/d1e/d2f/d45/db5/d6e 15995 1 2024-03-24T00:49:03.923 INFO:tasks.workunit.client.1.smithi088.stdout:5/807: creat d1/d10/d1b/d2f/d57/d9e/dd0/f10a x:0 0 0 2024-03-24T00:49:03.932 INFO:tasks.workunit.client.1.smithi088.stdout:3/737: truncate d3/de/d6f/f75 612470 0 2024-03-24T00:49:03.932 INFO:tasks.workunit.client.1.smithi088.stdout:3/738: stat d3/f95 0 2024-03-24T00:49:03.932 INFO:tasks.workunit.client.1.smithi088.stdout:3/739: fdatasync d3/d1a/d1e/f28 0 2024-03-24T00:49:03.932 INFO:tasks.workunit.client.1.smithi088.stdout:1/768: rename d6/d23/daf/d95/d5b/fa8 to d6/d23/daf/d95/d5b/d80/ff9 0 2024-03-24T00:49:03.933 INFO:tasks.workunit.client.0.smithi038.stdout:5/670: unlink d6/d19/d20/c80 0 2024-03-24T00:49:03.936 INFO:tasks.workunit.client.0.smithi038.stdout:9/818: rename d12/d41/d57/dd3/dfe/ff3 to d12/d28/d6c/dbc/dd7/f106 0 2024-03-24T00:49:03.936 INFO:tasks.workunit.client.0.smithi038.stdout:9/819: read - d12/d28/d6c/dbc/fc8 zero size 2024-03-24T00:49:03.940 INFO:tasks.workunit.client.0.smithi038.stdout:5/671: mknod d6/d16/d67/cda 0 2024-03-24T00:49:03.943 INFO:tasks.workunit.client.0.smithi038.stdout:8/748: dread d0/d45/d66/f93 [0,4194304] 0 2024-03-24T00:49:03.950 INFO:tasks.workunit.client.0.smithi038.stdout:5/672: creat d6/d19/d20/d33/fdb x:0 0 0 2024-03-24T00:49:03.951 INFO:tasks.workunit.client.0.smithi038.stdout:5/673: truncate d6/d16/d21/d2d/fd2 409628 0 2024-03-24T00:49:03.951 INFO:tasks.workunit.client.1.smithi088.stdout:3/740: mkdir d3/d6/d19/dea 0 2024-03-24T00:49:03.951 INFO:tasks.workunit.client.1.smithi088.stdout:3/741: chown d3/d1a/d1e/c2c 0 1 2024-03-24T00:49:03.955 INFO:tasks.workunit.client.0.smithi038.stdout:8/749: rename d0/d1d/d27/d4d/d81/la9 to d0/d45/d66/da6/l101 0 2024-03-24T00:49:03.955 INFO:tasks.workunit.client.0.smithi038.stdout:5/674: mkdir d6/d16/d67/ddc 0 2024-03-24T00:49:03.958 INFO:tasks.workunit.client.1.smithi088.stdout:3/742: mknod d3/de/d6f/dcc/ceb 0 2024-03-24T00:49:03.959 INFO:tasks.workunit.client.0.smithi038.stdout:8/750: getdents d0/d1d/d27 0 2024-03-24T00:49:03.979 INFO:tasks.workunit.client.1.smithi088.stdout:3/743: truncate d3/d6/d27/d3c/f59 4389372 0 2024-03-24T00:49:04.013 INFO:tasks.workunit.client.1.smithi088.stdout:6/808: dwrite db/d1d/d23/d5c/f78 [0,4194304] 0 2024-03-24T00:49:04.021 INFO:tasks.workunit.client.1.smithi088.stdout:6/809: truncate db/dd/d61/fc4 4336145 0 2024-03-24T00:49:04.024 INFO:tasks.workunit.client.0.smithi038.stdout:0/662: dwrite d0/d14/d88/f97 [0,4194304] 0 2024-03-24T00:49:04.040 INFO:tasks.workunit.client.0.smithi038.stdout:4/690: dread da/d12/d37/dd/d15/f64 [0,4194304] 0 2024-03-24T00:49:04.043 INFO:tasks.workunit.client.0.smithi038.stdout:0/663: dread d0/f5 [0,4194304] 0 2024-03-24T00:49:04.047 INFO:tasks.workunit.client.0.smithi038.stdout:0/664: mknod d0/d14/ce0 0 2024-03-24T00:49:04.047 INFO:tasks.workunit.client.0.smithi038.stdout:0/665: chown d0/d12/d15/d58/lde 28404 1 2024-03-24T00:49:04.065 INFO:tasks.workunit.client.1.smithi088.stdout:4/760: dwrite d5/d16/d37/de7/f10a [0,4194304] 0 2024-03-24T00:49:04.067 INFO:tasks.workunit.client.0.smithi038.stdout:7/682: write d2/d9/da/d23/d2b/f5f [540908,81598] 0 2024-03-24T00:49:04.068 INFO:tasks.workunit.client.1.smithi088.stdout:2/722: dwrite db/dd/d96/f53 [0,4194304] 0 2024-03-24T00:49:04.068 INFO:tasks.workunit.client.1.smithi088.stdout:2/723: stat db/dc/ca4 0 2024-03-24T00:49:04.069 INFO:tasks.workunit.client.1.smithi088.stdout:8/752: dwrite d9/dc/f52 [0,4194304] 0 2024-03-24T00:49:04.075 INFO:tasks.workunit.client.1.smithi088.stdout:6/810: read db/dc5/dc8/d4b/f6e [683959,801] 0 2024-03-24T00:49:04.075 INFO:tasks.workunit.client.1.smithi088.stdout:6/811: chown db/dd/d61/d6d/d74/d88/fda 17236 1 2024-03-24T00:49:04.075 INFO:tasks.workunit.client.1.smithi088.stdout:0/796: dwrite d9/dc/de/d19/d44/d46/f57 [0,4194304] 0 2024-03-24T00:49:04.076 INFO:tasks.workunit.client.0.smithi038.stdout:9/820: dwrite d12/d28/d3a/f3f [0,4194304] 0 2024-03-24T00:49:04.076 INFO:tasks.workunit.client.0.smithi038.stdout:0/666: dread d0/d2d/f65 [0,4194304] 0 2024-03-24T00:49:04.077 INFO:tasks.workunit.client.1.smithi088.stdout:7/705: dwrite d3/d32/d76/f81 [0,4194304] 0 2024-03-24T00:49:04.079 INFO:tasks.workunit.client.0.smithi038.stdout:6/801: dread d1/f6 [0,4194304] 0 2024-03-24T00:49:04.080 INFO:tasks.workunit.client.0.smithi038.stdout:3/718: dread d3/d30/d5d/f95 [0,4194304] 0 2024-03-24T00:49:04.080 INFO:tasks.workunit.client.0.smithi038.stdout:7/683: creat d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/f101 x:0 0 0 2024-03-24T00:49:04.080 INFO:tasks.workunit.client.0.smithi038.stdout:6/802: write d1/da/d61/fee [98407,106470] 0 2024-03-24T00:49:04.082 INFO:tasks.workunit.client.1.smithi088.stdout:4/761: rmdir d5/d7/de4 39 2024-03-24T00:49:04.082 INFO:tasks.workunit.client.0.smithi038.stdout:5/675: dwrite d6/d19/d20/d42/d43/d84/fc6 [0,4194304] 0 2024-03-24T00:49:04.085 INFO:tasks.workunit.client.1.smithi088.stdout:3/744: dwrite d3/d6/d19/d2a/f88 [0,4194304] 0 2024-03-24T00:49:04.088 INFO:tasks.workunit.client.0.smithi038.stdout:6/803: dread d1/f93 [0,4194304] 0 2024-03-24T00:49:04.090 INFO:tasks.workunit.client.0.smithi038.stdout:5/676: dread d6/d19/d20/d33/d6f/f88 [0,4194304] 0 2024-03-24T00:49:04.092 INFO:tasks.workunit.client.0.smithi038.stdout:5/677: write d6/d19/d20/d42/d43/d68/dc8/fd4 [549139,106208] 0 2024-03-24T00:49:04.092 INFO:tasks.workunit.client.0.smithi038.stdout:5/678: chown d6/d16/d21/d2d/l40 2314 1 2024-03-24T00:49:04.092 INFO:tasks.workunit.client.0.smithi038.stdout:9/821: rmdir d12/d55/d56 39 2024-03-24T00:49:04.092 INFO:tasks.workunit.client.1.smithi088.stdout:1/769: getdents d6/d23/daf/d95/d58 0 2024-03-24T00:49:04.097 INFO:tasks.workunit.client.0.smithi038.stdout:8/751: dwrite d0/d45/d37/d5f/d68/fa0 [4194304,4194304] 0 2024-03-24T00:49:04.097 INFO:tasks.workunit.client.0.smithi038.stdout:8/752: fsync d0/d1d/d20/f23 0 2024-03-24T00:49:04.097 INFO:tasks.workunit.client.0.smithi038.stdout:8/753: truncate d0/d1d/d27/d4d/daf/fed 319186 0 2024-03-24T00:49:04.100 INFO:tasks.workunit.client.0.smithi038.stdout:0/667: link d0/d12/d92/la3 d0/d12/d15/le1 0 2024-03-24T00:49:04.104 INFO:tasks.workunit.client.1.smithi088.stdout:2/724: unlink db/dd/d96/f4b 0 2024-03-24T00:49:04.108 INFO:tasks.workunit.client.0.smithi038.stdout:7/684: unlink d2/d9/da/dc/cd9 0 2024-03-24T00:49:04.112 INFO:tasks.workunit.client.1.smithi088.stdout:6/812: mkdir db/dd/d61/dfb 0 2024-03-24T00:49:04.112 INFO:tasks.workunit.client.1.smithi088.stdout:6/813: chown db/dd/cf5 267681955 1 2024-03-24T00:49:04.112 INFO:tasks.workunit.client.1.smithi088.stdout:6/814: truncate db/dd/d61/d6d/d74/d88/fe7 559296 0 2024-03-24T00:49:04.113 INFO:tasks.workunit.client.0.smithi038.stdout:3/719: creat d3/d9/dc/da6/db5/d19/da4/ff5 x:0 0 0 2024-03-24T00:49:04.114 INFO:tasks.workunit.client.0.smithi038.stdout:9/822: dread d12/d28/d3a/f4e [0,4194304] 0 2024-03-24T00:49:04.119 INFO:tasks.workunit.client.1.smithi088.stdout:0/797: unlink d9/f80 0 2024-03-24T00:49:04.123 INFO:tasks.workunit.client.1.smithi088.stdout:5/808: rename d1/d10/d1b to d1/d10/db6/d10b 0 2024-03-24T00:49:04.123 INFO:tasks.workunit.client.1.smithi088.stdout:5/809: fsync d1/d10/db6/d10b/d2f/d57/d9e/daa/fe6 0 2024-03-24T00:49:04.124 INFO:tasks.workunit.client.0.smithi038.stdout:4/691: dwrite da/d12/d29/d5b/d73/fba [4194304,4194304] 0 2024-03-24T00:49:04.129 INFO:tasks.workunit.client.0.smithi038.stdout:6/804: dwrite d1/d37/d84/fcf [0,4194304] 0 2024-03-24T00:49:04.129 INFO:tasks.workunit.client.0.smithi038.stdout:6/805: getdents d1/da/de6 0 2024-03-24T00:49:04.135 INFO:tasks.workunit.client.1.smithi088.stdout:5/810: dread d1/dd/d6f/d98/fd1 [0,4194304] 0 2024-03-24T00:49:04.135 INFO:tasks.workunit.client.1.smithi088.stdout:3/745: dread d3/d1a/d1e/d2f/fcd [0,4194304] 0 2024-03-24T00:49:04.144 INFO:tasks.workunit.client.0.smithi038.stdout:6/806: dread d1/da/d61/d69/f6e [0,4194304] 0 2024-03-24T00:49:04.147 INFO:tasks.workunit.client.0.smithi038.stdout:5/679: creat d6/d19/d20/d42/d43/d50/fdd x:0 0 0 2024-03-24T00:49:04.147 INFO:tasks.workunit.client.0.smithi038.stdout:5/680: fdatasync d6/d19/d20/d42/d43/d50/fdd 0 2024-03-24T00:49:04.150 INFO:tasks.workunit.client.1.smithi088.stdout:7/706: mknod d3/d32/d76/ce1 0 2024-03-24T00:49:04.156 INFO:tasks.workunit.client.1.smithi088.stdout:8/753: dwrite d9/dc/fd1 [0,4194304] 0 2024-03-24T00:49:04.157 INFO:tasks.workunit.client.1.smithi088.stdout:8/754: readlink d9/dc/l49 0 2024-03-24T00:49:04.170 INFO:tasks.workunit.client.0.smithi038.stdout:8/754: mkdir d0/d1d/d20/d79/d102 0 2024-03-24T00:49:04.174 INFO:tasks.workunit.client.1.smithi088.stdout:2/725: dwrite db/dd/d96/d4e/d8f/d9c/db0/fd9 [0,4194304] 0 2024-03-24T00:49:04.178 INFO:tasks.workunit.client.1.smithi088.stdout:1/770: mkdir d6/d27/d67/d88/dfa 0 2024-03-24T00:49:04.183 INFO:tasks.workunit.client.1.smithi088.stdout:6/815: symlink db/d1d/d50/d90/d55/lfc 0 2024-03-24T00:49:04.186 INFO:tasks.workunit.client.1.smithi088.stdout:0/798: mkdir d9/dc/de/d19/dd2/d103 0 2024-03-24T00:49:04.191 INFO:tasks.workunit.client.1.smithi088.stdout:7/707: dwrite d3/d31/f9e [0,4194304] 0 2024-03-24T00:49:04.193 INFO:tasks.workunit.client.1.smithi088.stdout:3/746: creat d3/d6/d19/d2a/dc2/dd5/fec x:0 0 0 2024-03-24T00:49:04.200 INFO:tasks.workunit.client.1.smithi088.stdout:9/773: sync 2024-03-24T00:49:04.203 INFO:tasks.workunit.client.1.smithi088.stdout:5/811: dwrite d1/dd/f13 [4194304,4194304] 0 2024-03-24T00:49:04.205 INFO:tasks.workunit.client.1.smithi088.stdout:2/726: dread db/d21/d35/f7d [0,4194304] 0 2024-03-24T00:49:04.205 INFO:tasks.workunit.client.1.smithi088.stdout:2/727: stat db/d21/d35/d49/fac 0 2024-03-24T00:49:04.205 INFO:tasks.workunit.client.1.smithi088.stdout:2/728: chown db/dd/d96/d4e/d8f/dad/fd7 5203 1 2024-03-24T00:49:04.205 INFO:tasks.workunit.client.1.smithi088.stdout:2/729: dread - db/d21/d35/dcb/fce zero size 2024-03-24T00:49:04.206 INFO:tasks.workunit.client.1.smithi088.stdout:8/755: creat d9/dc/d11/d24/fff x:0 0 0 2024-03-24T00:49:04.206 INFO:tasks.workunit.client.1.smithi088.stdout:8/756: fdatasync d9/dc/d11/d24/f5e 0 2024-03-24T00:49:04.208 INFO:tasks.workunit.client.1.smithi088.stdout:1/771: write d6/d23/daf/d95/d5b/d5d/d61/f7e [2821004,55085] 0 2024-03-24T00:49:04.209 INFO:tasks.workunit.client.1.smithi088.stdout:4/762: mkdir d5/d7/dd/d10b 0 2024-03-24T00:49:04.221 INFO:tasks.workunit.client.1.smithi088.stdout:6/816: symlink db/d3d/dcb/lfd 0 2024-03-24T00:49:04.221 INFO:tasks.workunit.client.1.smithi088.stdout:6/817: write db/d3d/ff1 [394210,75876] 0 2024-03-24T00:49:04.227 INFO:tasks.workunit.client.1.smithi088.stdout:0/799: creat d9/dc/f104 x:0 0 0 2024-03-24T00:49:04.230 INFO:tasks.workunit.client.0.smithi038.stdout:4/692: dwrite da/d12/d24/d5f/d8a/fc4 [0,4194304] 0 2024-03-24T00:49:04.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.231+0000 7f8d02a00700 1 -- 172.21.15.38:0/3064859370 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cfc105df0 con 0x7f8ce0071a10 2024-03-24T00:49:04.234 INFO:tasks.workunit.client.0.smithi038.stdout:0/668: mknod d0/d14/d84/ce2 0 2024-03-24T00:49:04.234 INFO:tasks.workunit.client.0.smithi038.stdout:0/669: creat d0/d12/d42/d83/fe3 x:0 0 0 2024-03-24T00:49:04.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.234+0000 7f8cf8c00700 1 -- 172.21.15.38:0/3064859370 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f8cfc105df0 con 0x7f8ce0071a10 2024-03-24T00:49:04.239 INFO:tasks.workunit.client.1.smithi088.stdout:7/708: rename d3/d32/d34/d9c/c6b to d3/d31/ce2 0 2024-03-24T00:49:04.239 INFO:tasks.workunit.client.1.smithi088.stdout:7/709: chown d3/d32/d34/d9c/d4b/d74/lc8 167 1 2024-03-24T00:49:04.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.239+0000 7f8d02a00700 1 -- 172.21.15.38:0/3064859370 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f8ce0071a10 msgr2=0x7f8ce0073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:04.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.239+0000 7f8d02a00700 1 --2- 172.21.15.38:0/3064859370 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f8ce0071a10 0x7f8ce0073ed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8cec007c60 tx=0x7f8cec0073d0 comp rx=0 tx=0).stop 2024-03-24T00:49:04.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.239+0000 7f8d02a00700 1 -- 172.21.15.38:0/3064859370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc103ed0 msgr2=0x7f8cfc194800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:04.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.239+0000 7f8d02a00700 1 --2- 172.21.15.38:0/3064859370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc103ed0 0x7f8cfc194800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8005100 tx=0x7f8ce800bd70 comp rx=0 tx=0).stop 2024-03-24T00:49:04.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.239+0000 7f8d02a00700 1 -- 172.21.15.38:0/3064859370 shutdown_connections 2024-03-24T00:49:04.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.240+0000 7f8d02a00700 1 --2- 172.21.15.38:0/3064859370 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f8ce0071a10 0x7f8ce0073ed0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:04.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.240+0000 7f8d02a00700 1 --2- 172.21.15.38:0/3064859370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cfc1015a0 0x7f8cfc1942c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:04.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.240+0000 7f8d02a00700 1 --2- 172.21.15.38:0/3064859370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cfc103ed0 0x7f8cfc194800 secure :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8005100 tx=0x7f8ce800bd70 comp rx=0 tx=0).stop 2024-03-24T00:49:04.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.240+0000 7f8d02a00700 1 -- 172.21.15.38:0/3064859370 >> 172.21.15.38:0/3064859370 conn(0x7f8cfc0faf70 msgr2=0x7f8cfc0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:04.242 INFO:tasks.workunit.client.0.smithi038.stdout:7/685: link d2/l26 d2/d9/da/dc/l102 0 2024-03-24T00:49:04.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.241+0000 7f8d02a00700 1 -- 172.21.15.38:0/3064859370 shutdown_connections 2024-03-24T00:49:04.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.241+0000 7f8d02a00700 1 -- 172.21.15.38:0/3064859370 wait complete. 2024-03-24T00:49:04.246 INFO:tasks.workunit.client.1.smithi088.stdout:3/747: unlink d3/d1a/d1e/d2f/d45/l7d 0 2024-03-24T00:49:04.246 INFO:tasks.workunit.client.1.smithi088.stdout:3/748: readlink d3/d6/d27/lcf 0 2024-03-24T00:49:04.246 INFO:tasks.workunit.client.1.smithi088.stdout:3/749: chown d3/d6/d27/d3c/c74 2092401 1 2024-03-24T00:49:04.248 INFO:tasks.workunit.client.0.smithi038.stdout:3/720: symlink d3/d9/def/lf6 0 2024-03-24T00:49:04.256 INFO:tasks.workunit.client.1.smithi088.stdout:5/812: mkdir d1/d10/d4f/d10c 0 2024-03-24T00:49:04.257 INFO:tasks.workunit.client.1.smithi088.stdout:5/813: truncate d1/d10/db6/d10b/d2f/d57/d9e/fa2 555423 0 2024-03-24T00:49:04.261 INFO:tasks.workunit.client.0.smithi038.stdout:9/823: mknod d12/d28/d3a/d8f/c107 0 2024-03-24T00:49:04.261 INFO:tasks.workunit.client.0.smithi038.stdout:9/824: write d12/d28/d6c/dbc/dd7/f106 [204735,86290] 0 2024-03-24T00:49:04.262 INFO:tasks.workunit.client.1.smithi088.stdout:2/730: creat db/dd/d96/d4e/d8f/fe4 x:0 0 0 2024-03-24T00:49:04.262 INFO:tasks.workunit.client.1.smithi088.stdout:2/731: truncate db/faa 902747 0 2024-03-24T00:49:04.262 INFO:tasks.workunit.client.1.smithi088.stdout:2/732: chown db/f18 30 1 2024-03-24T00:49:04.268 INFO:tasks.workunit.client.1.smithi088.stdout:8/757: mkdir d9/dc/dce/ddc/d100 0 2024-03-24T00:49:04.271 INFO:tasks.workunit.client.1.smithi088.stdout:1/772: symlink d6/d23/daf/d95/d5b/d5d/d61/d7d/dc8/de9/lfb 0 2024-03-24T00:49:04.273 INFO:tasks.workunit.client.1.smithi088.stdout:6/818: creat db/dc5/dc8/d43/df7/ffe x:0 0 0 2024-03-24T00:49:04.275 INFO:tasks.workunit.client.1.smithi088.stdout:7/710: rmdir d3/d32/d34/d9c/d4b/d74/dd3 0 2024-03-24T00:49:04.276 INFO:tasks.workunit.client.1.smithi088.stdout:0/800: dread d9/dc/de/d19/d44/d46/d65/d2a/f75 [0,4194304] 0 2024-03-24T00:49:04.278 INFO:tasks.workunit.client.0.smithi038.stdout:6/807: rename d1/da/d1b/d1d/c27 to d1/da/d61/d69/c109 0 2024-03-24T00:49:04.281 INFO:tasks.workunit.client.1.smithi088.stdout:9/774: dwrite d1/d10/d9b/dab/db8/f7b [4194304,4194304] 0 2024-03-24T00:49:04.281 INFO:tasks.workunit.client.1.smithi088.stdout:9/775: chown d1/d22/d72/faf 733622618 1 2024-03-24T00:49:04.281 INFO:tasks.workunit.client.1.smithi088.stdout:9/776: chown d1/d22/f51 840 1 2024-03-24T00:49:04.281 INFO:tasks.workunit.client.0.smithi038.stdout:5/681: unlink d6/d19/l57 0 2024-03-24T00:49:04.282 INFO:tasks.workunit.client.0.smithi038.stdout:5/682: write d6/f26 [820409,45534] 0 2024-03-24T00:49:04.282 INFO:tasks.workunit.client.0.smithi038.stdout:5/683: dread - d6/d19/d20/d33/fdb zero size 2024-03-24T00:49:04.282 INFO:tasks.workunit.client.1.smithi088.stdout:4/763: dwrite d5/d16/f29 [0,4194304] 0 2024-03-24T00:49:04.283 INFO:tasks.workunit.client.1.smithi088.stdout:5/814: mkdir d1/dd/d6f/da1/d3f/d88/d8f/d69/d10d 0 2024-03-24T00:49:04.283 INFO:tasks.workunit.client.1.smithi088.stdout:5/815: truncate d1/d19/d48/dc2/dcc/fff 891671 0 2024-03-24T00:49:04.284 INFO:tasks.workunit.client.1.smithi088.stdout:5/816: readlink d1/d10/l4b 0 2024-03-24T00:49:04.284 INFO:tasks.workunit.client.1.smithi088.stdout:2/733: unlink db/lb1 0 2024-03-24T00:49:04.284 INFO:tasks.workunit.client.1.smithi088.stdout:2/734: write db/dd/d96/d4e/d8f/d9c/fc1 [174691,68087] 0 2024-03-24T00:49:04.284 INFO:tasks.workunit.client.1.smithi088.stdout:2/735: truncate db/dd/d96/d4e/d8f/dad/fc2 4308104 0 2024-03-24T00:49:04.285 INFO:tasks.workunit.client.1.smithi088.stdout:2/736: write db/dd/d96/d4e/d8f/fa7 [356617,72398] 0 2024-03-24T00:49:04.286 INFO:tasks.workunit.client.0.smithi038.stdout:8/755: truncate d0/d15/d2c/f88 1679828 0 2024-03-24T00:49:04.288 INFO:tasks.workunit.client.1.smithi088.stdout:8/758: mkdir d9/dc/d11/d24/d29/d5b/d8b/d101 0 2024-03-24T00:49:04.294 INFO:tasks.workunit.client.1.smithi088.stdout:1/773: mkdir d6/d6e/dfc 0 2024-03-24T00:49:04.295 INFO:tasks.workunit.client.1.smithi088.stdout:6/819: symlink db/dd/d61/d6d/lff 0 2024-03-24T00:49:04.296 INFO:tasks.workunit.client.0.smithi038.stdout:4/693: unlink da/d12/d37/d83/dac/fd9 0 2024-03-24T00:49:04.302 INFO:tasks.workunit.client.1.smithi088.stdout:4/764: rename d5/d16/c43 to d5/c10c 0 2024-03-24T00:49:04.302 INFO:tasks.workunit.client.1.smithi088.stdout:0/801: write d9/f85 [3714180,49025] 0 2024-03-24T00:49:04.305 INFO:tasks.workunit.client.1.smithi088.stdout:1/774: dread d6/d27/d67/d88/f92 [0,4194304] 0 2024-03-24T00:49:04.305 INFO:tasks.workunit.client.0.smithi038.stdout:0/670: mkdir d0/d14/de4 0 2024-03-24T00:49:04.308 INFO:tasks.workunit.client.0.smithi038.stdout:3/721: creat d3/d9/dc/da6/db5/ff7 x:0 0 0 2024-03-24T00:49:04.308 INFO:tasks.workunit.client.0.smithi038.stdout:3/722: write d3/d9/dc/df/fb8 [720755,85208] 0 2024-03-24T00:49:04.308 INFO:tasks.workunit.client.0.smithi038.stdout:3/723: creat d3/d30/d5d/ff8 x:0 0 0 2024-03-24T00:49:04.309 INFO:tasks.workunit.client.0.smithi038.stdout:3/724: chown d3/d5a/fa5 257374 1 2024-03-24T00:49:04.309 INFO:tasks.workunit.client.0.smithi038.stdout:8/756: read d0/d1d/d20/d79/fcd [3302867,40542] 0 2024-03-24T00:49:04.309 INFO:tasks.workunit.client.1.smithi088.stdout:5/817: getdents d1/d10/db6/d10b/d2f/d57/d78 0 2024-03-24T00:49:04.311 INFO:tasks.workunit.client.0.smithi038.stdout:2/668: sync 2024-03-24T00:49:04.312 INFO:tasks.workunit.client.0.smithi038.stdout:1/790: sync 2024-03-24T00:49:04.312 INFO:tasks.workunit.client.0.smithi038.stdout:2/669: truncate d4/d18/d38/d63/f92 4496016 0 2024-03-24T00:49:04.313 INFO:tasks.workunit.client.0.smithi038.stdout:9/825: symlink d12/d28/d9c/ddd/l108 0 2024-03-24T00:49:04.314 INFO:tasks.workunit.client.1.smithi088.stdout:6/820: symlink db/d1d/d50/d90/d56/l100 0 2024-03-24T00:49:04.319 INFO:tasks.workunit.client.0.smithi038.stdout:6/808: symlink d1/da/d1b/l10a 0 2024-03-24T00:49:04.319 INFO:tasks.workunit.client.0.smithi038.stdout:6/809: creat d1/d37/d84/f10b x:0 0 0 2024-03-24T00:49:04.321 INFO:tasks.workunit.client.0.smithi038.stdout:1/791: read da/f56 [473653,125573] 0 2024-03-24T00:49:04.322 INFO:tasks.workunit.client.1.smithi088.stdout:3/750: dread d3/d6/dca/f93 [0,4194304] 0 2024-03-24T00:49:04.330 INFO:tasks.workunit.client.1.smithi088.stdout:4/765: dread d5/d16/f29 [0,4194304] 0 2024-03-24T00:49:04.331 INFO:tasks.workunit.client.1.smithi088.stdout:5/818: symlink d1/d10/d79/dbc/l10e 0 2024-03-24T00:49:04.332 INFO:tasks.workunit.client.0.smithi038.stdout:3/725: symlink d3/d30/d5d/deb/lf9 0 2024-03-24T00:49:04.332 INFO:tasks.workunit.client.0.smithi038.stdout:8/757: mknod d0/d1d/d27/d4d/daf/dea/c103 0 2024-03-24T00:49:04.335 INFO:tasks.workunit.client.1.smithi088.stdout:2/737: symlink db/d21/d35/le5 0 2024-03-24T00:49:04.335 INFO:tasks.workunit.client.1.smithi088.stdout:6/821: dread db/d1d/d23/d5c/f67 [0,4194304] 0 2024-03-24T00:49:04.335 INFO:tasks.workunit.client.0.smithi038.stdout:8/758: creat d0/d1d/d20/d79/f104 x:0 0 0 2024-03-24T00:49:04.336 INFO:tasks.workunit.client.0.smithi038.stdout:8/759: readlink d0/d15/d2c/lc7 0 2024-03-24T00:49:04.338 INFO:tasks.workunit.client.0.smithi038.stdout:5/684: mknod d6/d1e/d97/dd1/cde 0 2024-03-24T00:49:04.338 INFO:tasks.workunit.client.0.smithi038.stdout:5/685: stat d6/d16/d21/d2d 0 2024-03-24T00:49:04.339 INFO:tasks.workunit.client.0.smithi038.stdout:4/694: truncate da/d12/d24/d5f/d9a/f92 4098159 0 2024-03-24T00:49:04.340 INFO:tasks.workunit.client.1.smithi088.stdout:8/759: dread d9/dc/d11/d24/f5e [0,4194304] 0 2024-03-24T00:49:04.341 INFO:tasks.workunit.client.1.smithi088.stdout:4/766: symlink d5/d7/d15/d60/l10d 0 2024-03-24T00:49:04.341 INFO:tasks.workunit.client.1.smithi088.stdout:3/751: rename d3/d6/d27/lcf to d3/d6/d19/d2a/dc2/dd5/dd6/led 0 2024-03-24T00:49:04.343 INFO:tasks.workunit.client.0.smithi038.stdout:1/792: rename da/dd/d2c/d5a/d101/de8/d108/fd5 to da/dd/d2c/f10a 0 2024-03-24T00:49:04.346 INFO:tasks.workunit.client.1.smithi088.stdout:1/775: write d6/d23/daf/d95/d58/f7c [1405464,34426] 0 2024-03-24T00:49:04.346 INFO:tasks.workunit.client.1.smithi088.stdout:1/776: write d6/d23/dcb/de7/fea [319259,71555] 0 2024-03-24T00:49:04.353 INFO:tasks.workunit.client.1.smithi088.stdout:6/822: dread db/d1d/d50/d90/f4d [0,4194304] 0 2024-03-24T00:49:04.355 INFO:tasks.workunit.client.0.smithi038.stdout:7/686: dwrite d2/d9/da/f73 [0,4194304] 0 2024-03-24T00:49:04.356 INFO:tasks.workunit.client.0.smithi038.stdout:4/695: dread da/d12/d24/d68/d6c/f85 [0,4194304] 0 2024-03-24T00:49:04.361 INFO:tasks.workunit.client.0.smithi038.stdout:3/726: dread d3/d30/d5d/f95 [0,4194304] 0 2024-03-24T00:49:04.364 INFO:tasks.workunit.client.1.smithi088.stdout:4/767: creat d5/d7/de4/d107/f10e x:0 0 0 2024-03-24T00:49:04.364 INFO:tasks.workunit.client.0.smithi038.stdout:5/686: symlink d6/d60/ldf 0 2024-03-24T00:49:04.365 INFO:tasks.workunit.client.1.smithi088.stdout:2/738: write db/f55 [1756520,90247] 0 2024-03-24T00:49:04.367 INFO:tasks.workunit.client.1.smithi088.stdout:6/823: write db/d1d/d50/d90/f4d [292583,113012] 0 2024-03-24T00:49:04.368 INFO:tasks.workunit.client.1.smithi088.stdout:7/711: dwrite d3/d32/d34/fd2 [0,4194304] 0 2024-03-24T00:49:04.368 INFO:tasks.workunit.client.1.smithi088.stdout:0/802: dwrite d9/dc/de/d19/f51 [0,4194304] 0 2024-03-24T00:49:04.376 INFO:tasks.workunit.client.1.smithi088.stdout:1/777: creat d6/d6e/dfc/ffd x:0 0 0 2024-03-24T00:49:04.377 INFO:tasks.workunit.client.1.smithi088.stdout:1/778: write d6/d23/daf/d95/d3f/d53/ff6 [460985,91093] 0 2024-03-24T00:49:04.377 INFO:tasks.workunit.client.1.smithi088.stdout:1/779: dread - d6/d27/fb3 zero size 2024-03-24T00:49:04.377 INFO:tasks.workunit.client.1.smithi088.stdout:1/780: creat d6/d6e/ffe x:0 0 0 2024-03-24T00:49:04.381 INFO:tasks.workunit.client.0.smithi038.stdout:1/793: truncate da/dd/d7a/d82/d98/d34/f39 1335340 0 2024-03-24T00:49:04.381 INFO:tasks.workunit.client.0.smithi038.stdout:1/794: chown da/dd/d2c/d40/lf3 67158 1 2024-03-24T00:49:04.382 INFO:tasks.workunit.client.0.smithi038.stdout:1/795: write da/dd/d2c/f10a [1018925,99368] 0 2024-03-24T00:49:04.382 INFO:tasks.workunit.client.0.smithi038.stdout:1/796: fsync da/f23 0 2024-03-24T00:49:04.386 INFO:tasks.workunit.client.1.smithi088.stdout:6/824: read db/d1d/d23/d5c/f78 [993977,63764] 0 2024-03-24T00:49:04.388 INFO:tasks.workunit.client.1.smithi088.stdout:8/760: dread d9/dc/d11/f2f [0,4194304] 0 2024-03-24T00:49:04.389 INFO:tasks.workunit.client.1.smithi088.stdout:8/761: stat d9/dc/d11/d24/d29/d5b/f1e 0 2024-03-24T00:49:04.389 INFO:tasks.workunit.client.1.smithi088.stdout:2/739: truncate db/f3c 2256018 0 2024-03-24T00:49:04.395 INFO:tasks.workunit.client.0.smithi038.stdout:8/760: getdents d0/d1d/d27/d4d/daf/dea 0 2024-03-24T00:49:04.396 INFO:tasks.workunit.client.1.smithi088.stdout:4/768: dread d5/d13/d18/d24/d3d/f77 [0,4194304] 0 2024-03-24T00:49:04.396 INFO:tasks.workunit.client.1.smithi088.stdout:4/769: write d5/d7/fcc [3717461,127790] 0 2024-03-24T00:49:04.396 INFO:tasks.workunit.client.1.smithi088.stdout:4/770: stat d5/d16/d94/da6/fc7 0 2024-03-24T00:49:04.396 INFO:tasks.workunit.client.1.smithi088.stdout:4/771: stat d5/d13/d18/f79 0 2024-03-24T00:49:04.396 INFO:tasks.workunit.client.1.smithi088.stdout:4/772: fsync d5/d13/d18/f1b 0 2024-03-24T00:49:04.397 INFO:tasks.workunit.client.0.smithi038.stdout:4/696: creat da/dbd/fed x:0 0 0 2024-03-24T00:49:04.397 INFO:tasks.workunit.client.0.smithi038.stdout:4/697: chown da/d12/d24/d5f/lcc 1 1 2024-03-24T00:49:04.401 INFO:tasks.workunit.client.1.smithi088.stdout:5/819: dread d1/d10/db6/d10b/d9d/fa3 [0,4194304] 0 2024-03-24T00:49:04.405 INFO:tasks.workunit.client.1.smithi088.stdout:9/777: dwrite d1/d10/d29/d85/fac [8388608,4194304] 0 2024-03-24T00:49:04.405 INFO:tasks.workunit.client.1.smithi088.stdout:9/778: fsync d1/d15/d84/fdf 0 2024-03-24T00:49:04.405 INFO:tasks.workunit.client.1.smithi088.stdout:9/779: chown d1/d22/f93 1287402 1 2024-03-24T00:49:04.407 INFO:tasks.workunit.client.0.smithi038.stdout:3/727: rename d3/d5a/c69 to d3/d30/d5d/d6a/cfa 0 2024-03-24T00:49:04.407 INFO:tasks.workunit.client.0.smithi038.stdout:3/728: creat d3/d9/dc/da6/db5/d19/d82/ffb x:0 0 0 2024-03-24T00:49:04.410 INFO:tasks.workunit.client.1.smithi088.stdout:7/712: rename d3/d31/l3e to d3/d32/d34/d9c/d4b/d73/dab/dd8/le3 0 2024-03-24T00:49:04.412 INFO:tasks.workunit.client.1.smithi088.stdout:8/762: write d9/dc/d11/d24/d29/d36/f4e [3868683,40919] 0 2024-03-24T00:49:04.419 INFO:tasks.workunit.client.1.smithi088.stdout:0/803: mkdir d9/d26/d8f/da1/d62/d9b/db0/d105 0 2024-03-24T00:49:04.419 INFO:tasks.workunit.client.1.smithi088.stdout:0/804: write d9/dc/de/d7e/f8c [247861,121074] 0 2024-03-24T00:49:04.419 INFO:tasks.workunit.client.1.smithi088.stdout:0/805: chown d9/dc/de/d7e/lbd 454691926 1 2024-03-24T00:49:04.419 INFO:tasks.workunit.client.1.smithi088.stdout:0/806: truncate d9/dc/de/d19/d44/d46/d65/d2a/f75 3281187 0 2024-03-24T00:49:04.427 INFO:tasks.workunit.client.0.smithi038.stdout:4/698: mknod da/d12/d29/cee 0 2024-03-24T00:49:04.436 INFO:tasks.workunit.client.1.smithi088.stdout:6/825: dread db/dd/f41 [8388608,4194304] 0 2024-03-24T00:49:04.436 INFO:tasks.workunit.client.1.smithi088.stdout:6/826: creat db/dd/d87/ddf/f101 x:0 0 0 2024-03-24T00:49:04.438 INFO:tasks.workunit.client.0.smithi038.stdout:5/687: rename d6/d19/d20/d42/d43/fa4 to d6/d1e/d97/db0/fe0 0 2024-03-24T00:49:04.443 INFO:tasks.workunit.client.0.smithi038.stdout:4/699: stat da/d12/d37/l7b 0 2024-03-24T00:49:04.446 INFO:tasks.workunit.client.1.smithi088.stdout:5/820: write d1/d10/db6/d10b/d2e/d9f/ddf/fb0 [1343513,102289] 0 2024-03-24T00:49:04.450 INFO:tasks.workunit.client.1.smithi088.stdout:7/713: mkdir d3/d31/d56/de4 0 2024-03-24T00:49:04.450 INFO:tasks.workunit.client.1.smithi088.stdout:2/740: rename db/l63 to db/dd/le6 0 2024-03-24T00:49:04.451 INFO:tasks.workunit.client.1.smithi088.stdout:8/763: stat d9/dc/d11/d24/d29/d5b/db9/da5/lac 0 2024-03-24T00:49:04.455 INFO:tasks.workunit.client.1.smithi088.stdout:6/827: rename db/dd/d61/d6d/ff9 to db/d3d/f102 0 2024-03-24T00:49:04.456 INFO:tasks.workunit.client.1.smithi088.stdout:5/821: creat d1/dd/d6f/da1/d3f/d106/f10f x:0 0 0 2024-03-24T00:49:04.456 INFO:tasks.workunit.client.1.smithi088.stdout:2/741: mkdir db/dd/de7 0 2024-03-24T00:49:04.456 INFO:tasks.workunit.client.1.smithi088.stdout:2/742: chown db/d21/d73 937821821 1 2024-03-24T00:49:04.458 INFO:tasks.workunit.client.1.smithi088.stdout:7/714: creat d3/fe5 x:0 0 0 2024-03-24T00:49:04.459 INFO:tasks.workunit.client.1.smithi088.stdout:6/828: symlink db/dd/l103 0 2024-03-24T00:49:04.460 INFO:tasks.workunit.client.1.smithi088.stdout:5/822: creat d1/d10/db6/d10b/d2f/d57/d9e/daa/f110 x:0 0 0 2024-03-24T00:49:04.471 INFO:tasks.workunit.client.1.smithi088.stdout:2/743: rename db/dd/l77 to db/d21/d35/dcb/le8 0 2024-03-24T00:49:04.471 INFO:tasks.workunit.client.1.smithi088.stdout:7/715: mknod d3/d31/d56/ce6 0 2024-03-24T00:49:04.471 INFO:tasks.workunit.client.1.smithi088.stdout:5/823: mkdir d1/dd/d111 0 2024-03-24T00:49:04.471 INFO:tasks.workunit.client.1.smithi088.stdout:7/716: dread - d3/d32/d34/d9c/d4b/d74/d79/d7f/fca zero size 2024-03-24T00:49:04.471 INFO:tasks.workunit.client.1.smithi088.stdout:7/717: fsync d3/d32/d34/d7c/fc6 0 2024-03-24T00:49:04.472 INFO:tasks.workunit.client.1.smithi088.stdout:6/829: creat db/dd/f104 x:0 0 0 2024-03-24T00:49:04.472 INFO:tasks.workunit.client.1.smithi088.stdout:6/830: creat db/f105 x:0 0 0 2024-03-24T00:49:04.472 INFO:tasks.workunit.client.1.smithi088.stdout:7/718: write d3/d32/d34/d9c/fae [429453,56909] 0 2024-03-24T00:49:04.472 INFO:tasks.workunit.client.1.smithi088.stdout:7/719: stat d3/d32/d34/d9c/d4b/da7 0 2024-03-24T00:49:04.477 INFO:tasks.workunit.client.1.smithi088.stdout:1/781: dwrite d6/d23/daf/d95/d5b/d5d/d61/d7d/dc8/fd1 [0,4194304] 0 2024-03-24T00:49:04.481 INFO:tasks.workunit.client.1.smithi088.stdout:2/744: unlink db/d21/f5f 0 2024-03-24T00:49:04.481 INFO:tasks.workunit.client.1.smithi088.stdout:2/745: dread - db/d21/d35/f9e zero size 2024-03-24T00:49:04.482 INFO:tasks.workunit.client.1.smithi088.stdout:2/746: getdents db/dd4 0 2024-03-24T00:49:04.482 INFO:tasks.workunit.client.0.smithi038.stdout:8/761: read d0/d45/d37/f87 [1822585,86732] 0 2024-03-24T00:49:04.482 INFO:tasks.workunit.client.0.smithi038.stdout:8/762: chown d0/d1d/d27/d6b/dfc/f47 1166815966 1 2024-03-24T00:49:04.491 INFO:tasks.workunit.client.1.smithi088.stdout:5/824: creat d1/dd/f112 x:0 0 0 2024-03-24T00:49:04.493 INFO:tasks.workunit.client.0.smithi038.stdout:6/810: dwrite d1/da/d1b/d25/d49/d4e/ddf/fe9 [0,4194304] 0 2024-03-24T00:49:04.493 INFO:tasks.workunit.client.0.smithi038.stdout:6/811: stat d1/da/d1b/d25/d49/l74 0 2024-03-24T00:49:04.499 INFO:tasks.workunit.client.0.smithi038.stdout:8/763: unlink d0/d45/d66/da6/fe0 0 2024-03-24T00:49:04.500 INFO:tasks.workunit.client.1.smithi088.stdout:0/807: dwrite d9/dc/de/f3e [0,4194304] 0 2024-03-24T00:49:04.501 INFO:tasks.workunit.client.1.smithi088.stdout:0/808: chown d9/dc/de/d19/d44/de2 6 1 2024-03-24T00:49:04.504 INFO:tasks.workunit.client.1.smithi088.stdout:6/831: mkdir db/dd/d61/d6d/d74/d88/da1/d106 0 2024-03-24T00:49:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.509+0000 7f1200a00700 1 -- 172.21.15.38:0/1204805366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11fc102d00 msgr2=0x7f11fc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.509+0000 7f1200a00700 1 --2- 172.21.15.38:0/1204805366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11fc102d00 0x7f11fc103120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009910 tx=0x7f11ec009c20 comp rx=0 tx=0).stop 2024-03-24T00:49:04.511 INFO:tasks.workunit.client.1.smithi088.stdout:7/720: unlink d3/d31/f9b 0 2024-03-24T00:49:04.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.510+0000 7f1200a00700 1 -- 172.21.15.38:0/1204805366 shutdown_connections 2024-03-24T00:49:04.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.510+0000 7f1200a00700 1 --2- 172.21.15.38:0/1204805366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11fc103f00 0x7f11fc104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:04.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.510+0000 7f1200a00700 1 --2- 172.21.15.38:0/1204805366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11fc102d00 0x7f11fc103120 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:04.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.510+0000 7f1200a00700 1 -- 172.21.15.38:0/1204805366 >> 172.21.15.38:0/1204805366 conn(0x7f11fc0fe2a0 msgr2=0x7f11fc1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.513+0000 7f1200a00700 1 -- 172.21.15.38:0/1204805366 shutdown_connections 2024-03-24T00:49:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.513+0000 7f1200a00700 1 -- 172.21.15.38:0/1204805366 wait complete. 2024-03-24T00:49:04.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.514+0000 7f1200a00700 1 Processor -- start 2024-03-24T00:49:04.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.515+0000 7f1200a00700 1 -- start start 2024-03-24T00:49:04.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.515+0000 7f1200a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11fc102d00 0x7f11fc19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:04.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.515+0000 7f1200a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11fc103f00 0x7f11fc19d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:04.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.515+0000 7f1200a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc19d5f0 con 0x7f11fc103f00 2024-03-24T00:49:04.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.515+0000 7f1200a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc19d730 con 0x7f11fc102d00 2024-03-24T00:49:04.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.516+0000 7f11f9e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11fc102d00 0x7f11fc19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:04.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.516+0000 7f11f9e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11fc102d00 0x7f11fc19cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:50188/0 (socket says 172.21.15.38:50188) 2024-03-24T00:49:04.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.516+0000 7f11f9e00700 1 -- 172.21.15.38:0/435255665 learned_addr learned my addr 172.21.15.38:0/435255665 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:49:04.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.517+0000 7f11f9e00700 1 -- 172.21.15.38:0/435255665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11fc103f00 msgr2=0x7f11fc19d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:04.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.517+0000 7f11f9e00700 1 --2- 172.21.15.38:0/435255665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11fc103f00 0x7f11fc19d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:04.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.517+0000 7f11f9e00700 1 -- 172.21.15.38:0/435255665 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec0095f0 con 0x7f11fc102d00 2024-03-24T00:49:04.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.517+0000 7f11f9e00700 1 --2- 172.21.15.38:0/435255665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11fc102d00 0x7f11fc19cb20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00bd70 tx=0x7f11ec00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:04.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.518+0000 7f11f2a00700 1 -- 172.21.15.38:0/435255665 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11ec01d070 con 0x7f11fc102d00 2024-03-24T00:49:04.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.518+0000 7f11f2a00700 1 -- 172.21.15.38:0/435255665 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11ec00f460 con 0x7f11fc102d00 2024-03-24T00:49:04.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.518+0000 7f1200a00700 1 -- 172.21.15.38:0/435255665 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc1a20f0 con 0x7f11fc102d00 2024-03-24T00:49:04.520 INFO:tasks.workunit.client.0.smithi038.stdout:7/687: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d33/f3c [0,4194304] 0 2024-03-24T00:49:04.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.518+0000 7f11f2a00700 1 -- 172.21.15.38:0/435255665 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11ec021600 con 0x7f11fc102d00 2024-03-24T00:49:04.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.518+0000 7f1200a00700 1 -- 172.21.15.38:0/435255665 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc1a2630 con 0x7f11fc102d00 2024-03-24T00:49:04.525 INFO:tasks.workunit.client.0.smithi038.stdout:3/729: dwrite d3/d9/d77/dd4/fd9 [0,4194304] 0 2024-03-24T00:49:04.525 INFO:tasks.workunit.client.0.smithi038.stdout:3/730: readlink d3/d9/dc/da6/db5/l74 0 2024-03-24T00:49:04.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.522+0000 7f1200a00700 1 -- 172.21.15.38:0/435255665 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc076990 con 0x7f11fc102d00 2024-03-24T00:49:04.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.528+0000 7f11f2a00700 1 -- 172.21.15.38:0/435255665 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) v1 ==== 91370+0+0 (secure 0 0 0) 0x7f11ec00fa50 con 0x7f11fc102d00 2024-03-24T00:49:04.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.533+0000 7f11f2a00700 1 --2- 172.21.15.38:0/435255665 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f11e006d600 0x7f11e006fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:04.534 INFO:tasks.workunit.client.1.smithi088.stdout:5/825: dread d1/d19/d1e/f42 [0,4194304] 0 2024-03-24T00:49:04.534 INFO:tasks.workunit.client.1.smithi088.stdout:5/826: truncate d1/d10/ff3 449782 0 2024-03-24T00:49:04.534 INFO:tasks.workunit.client.1.smithi088.stdout:5/827: dread - d1/d19/db9/ffd zero size 2024-03-24T00:49:04.535 INFO:tasks.workunit.client.0.smithi038.stdout:6/812: symlink d1/da/d61/da2/db1/l10c 0 2024-03-24T00:49:04.535 INFO:tasks.workunit.client.0.smithi038.stdout:6/813: dread - d1/d37/d68/f85 zero size 2024-03-24T00:49:04.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.534+0000 7f11f2a00700 1 -- 172.21.15.38:0/435255665 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f11ec08e2c0 con 0x7f11fc102d00 2024-03-24T00:49:04.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.534+0000 7f11f2a00700 1 -- 172.21.15.38:0/435255665 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f11ec08e6a0 con 0x7f11fc102d00 2024-03-24T00:49:04.536 INFO:tasks.workunit.client.1.smithi088.stdout:8/764: dwrite d9/dc/d11/d80/ff3 [0,4194304] 0 2024-03-24T00:49:04.536 INFO:tasks.workunit.client.1.smithi088.stdout:8/765: stat d9/dc/d11/d24/f58 0 2024-03-24T00:49:04.536 INFO:tasks.workunit.client.1.smithi088.stdout:8/766: chown d9/dc/d11/d24/d29/d36/dc1/dc4/fe2 58881 1 2024-03-24T00:49:04.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.536+0000 7f11f9400700 1 --2- 172.21.15.38:0/435255665 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f11e006d600 0x7f11e006fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:04.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.537+0000 7f11f9400700 1 --2- 172.21.15.38:0/435255665 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f11e006d600 0x7f11e006fac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f11e8005fd0 tx=0x7f11e8005d00 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:04.550 INFO:tasks.workunit.client.1.smithi088.stdout:1/782: truncate d6/d23/daf/fdf 751417 0 2024-03-24T00:49:04.550 INFO:tasks.workunit.client.0.smithi038.stdout:0/671: dwrite d0/d12/d42/d9d/d28/f8a [4194304,4194304] 0 2024-03-24T00:49:04.551 INFO:tasks.workunit.client.0.smithi038.stdout:0/672: chown d0/d14/f5c 3354067 1 2024-03-24T00:49:04.551 INFO:tasks.workunit.client.0.smithi038.stdout:8/764: dread d0/d45/d37/d5f/db7/dcc/fef [0,4194304] 0 2024-03-24T00:49:04.551 INFO:tasks.workunit.client.0.smithi038.stdout:8/765: fdatasync d0/f76 0 2024-03-24T00:49:04.551 INFO:tasks.workunit.client.0.smithi038.stdout:8/766: fdatasync d0/d45/fe5 0 2024-03-24T00:49:04.552 INFO:tasks.workunit.client.1.smithi088.stdout:7/721: read d3/d32/d76/f81 [968707,94010] 0 2024-03-24T00:49:04.553 INFO:tasks.workunit.client.0.smithi038.stdout:4/700: dwrite da/dbd/fed [0,4194304] 0 2024-03-24T00:49:04.553 INFO:tasks.workunit.client.0.smithi038.stdout:4/701: creat da/d12/d24/d68/d6c/d7c/fef x:0 0 0 2024-03-24T00:49:04.556 INFO:tasks.workunit.client.0.smithi038.stdout:1/797: write da/dd/f85 [70611,67147] 0 2024-03-24T00:49:04.569 INFO:tasks.workunit.client.1.smithi088.stdout:5/828: dwrite d1/d10/ff3 [0,4194304] 0 2024-03-24T00:49:04.569 INFO:tasks.workunit.client.1.smithi088.stdout:5/829: readlink d1/dd/l2a 0 2024-03-24T00:49:04.570 INFO:tasks.workunit.client.0.smithi038.stdout:6/814: symlink d1/da/de6/l10d 0 2024-03-24T00:49:04.576 INFO:tasks.workunit.client.0.smithi038.stdout:3/731: dread d3/d9/f16 [0,4194304] 0 2024-03-24T00:49:04.584 INFO:tasks.workunit.client.1.smithi088.stdout:6/832: symlink db/dc5/dc8/d43/df7/l107 0 2024-03-24T00:49:04.589 INFO:tasks.workunit.client.0.smithi038.stdout:8/767: symlink d0/d45/d37/d5f/db7/d62/l105 0 2024-03-24T00:49:04.590 INFO:tasks.workunit.client.0.smithi038.stdout:5/688: dwrite d6/d19/d20/d42/d4b/f4f [0,4194304] 0 2024-03-24T00:49:04.590 INFO:tasks.workunit.client.0.smithi038.stdout:5/689: stat d6/d19/d20/d33/cb7 0 2024-03-24T00:49:04.592 INFO:tasks.workunit.client.1.smithi088.stdout:8/767: dwrite d9/dc/f44 [0,4194304] 0 2024-03-24T00:49:04.615 INFO:tasks.workunit.client.0.smithi038.stdout:4/702: mkdir da/d12/d37/dd/d15/d6d/d8f/df0 0 2024-03-24T00:49:04.621 INFO:tasks.workunit.client.0.smithi038.stdout:1/798: unlink da/dd/d7a/d82/d98/d9e/ld8 0 2024-03-24T00:49:04.622 INFO:tasks.workunit.client.0.smithi038.stdout:1/799: read da/dd/d2c/d5a/d74/f64 [971,89126] 0 2024-03-24T00:49:04.627 INFO:tasks.workunit.client.0.smithi038.stdout:3/732: unlink d3/d30/d5d/lb1 0 2024-03-24T00:49:04.628 INFO:tasks.workunit.client.0.smithi038.stdout:3/733: write d3/d9/dc/df/f3a [9146721,57766] 0 2024-03-24T00:49:04.634 INFO:tasks.workunit.client.0.smithi038.stdout:5/690: getdents d6/d19/d20/d42/d43/d68/dc8 0 2024-03-24T00:49:04.636 INFO:tasks.workunit.client.0.smithi038.stdout:0/673: dwrite d0/f69 [0,4194304] 0 2024-03-24T00:49:04.648 INFO:tasks.workunit.client.1.smithi088.stdout:1/783: symlink d6/d23/daf/d95/d5b/d80/lff 0 2024-03-24T00:49:04.651 INFO:tasks.workunit.client.1.smithi088.stdout:7/722: creat d3/d31/d56/fe7 x:0 0 0 2024-03-24T00:49:04.657 INFO:tasks.workunit.client.0.smithi038.stdout:8/768: dwrite d0/d10/fab [0,4194304] 0 2024-03-24T00:49:04.662 INFO:tasks.workunit.client.0.smithi038.stdout:4/703: mkdir da/d12/d37/dd/d15/df1 0 2024-03-24T00:49:04.662 INFO:tasks.workunit.client.0.smithi038.stdout:0/674: write d0/d12/d42/d9d/d28/f8a [5489585,15261] 0 2024-03-24T00:49:04.664 INFO:tasks.workunit.client.1.smithi088.stdout:5/830: rename d1/d19/d1e/l53 to d1/d10/db6/d10b/l113 0 2024-03-24T00:49:04.664 INFO:tasks.workunit.client.0.smithi038.stdout:6/815: dwrite d1/da/d1b/d1d/d3c/fb4 [0,4194304] 0 2024-03-24T00:49:04.666 INFO:tasks.workunit.client.0.smithi038.stdout:6/816: getdents d1/da/d1b/dfd 0 2024-03-24T00:49:04.666 INFO:tasks.workunit.client.0.smithi038.stdout:1/800: getdents da/dd/d2c/d5a/d101/de8/d108/dc4 0 2024-03-24T00:49:04.667 INFO:tasks.workunit.client.1.smithi088.stdout:1/784: dread f0 [4194304,4194304] 0 2024-03-24T00:49:04.667 INFO:tasks.workunit.client.1.smithi088.stdout:6/833: creat db/dd/d61/dfb/f108 x:0 0 0 2024-03-24T00:49:04.668 INFO:tasks.workunit.client.0.smithi038.stdout:3/734: rmdir d3/d9/dc/df 39 2024-03-24T00:49:04.670 INFO:tasks.workunit.client.1.smithi088.stdout:7/723: rename d3/d32/d76/d8f/fcd to d3/d32/d34/d9c/d4b/d73/fe8 0 2024-03-24T00:49:04.671 INFO:tasks.workunit.client.1.smithi088.stdout:7/724: fsync d3/ff 0 2024-03-24T00:49:04.673 INFO:tasks.workunit.client.0.smithi038.stdout:5/691: rmdir d6/d19/d20/d42/d43/d68/dce 0 2024-03-24T00:49:04.673 INFO:tasks.workunit.client.0.smithi038.stdout:5/692: write d6/d19/d20/d42/d43/d4c/f74 [645990,70993] 0 2024-03-24T00:49:04.678 INFO:tasks.workunit.client.1.smithi088.stdout:5/831: mknod d1/d10/db6/d10b/d2f/d57/d9e/dba/c114 0 2024-03-24T00:49:04.678 INFO:tasks.workunit.client.1.smithi088.stdout:5/832: chown d1/dd/d6f/ca4 92868 1 2024-03-24T00:49:04.685 INFO:tasks.workunit.client.1.smithi088.stdout:6/834: getdents db/d3d/d83/d72 0 2024-03-24T00:49:04.685 INFO:tasks.workunit.client.1.smithi088.stdout:6/835: chown db/d1d/d50/d90/d56/f51 175841 1 2024-03-24T00:49:04.694 INFO:tasks.workunit.client.1.smithi088.stdout:7/725: symlink d3/d31/le9 0 2024-03-24T00:49:04.695 INFO:tasks.workunit.client.0.smithi038.stdout:8/769: truncate d0/d1d/d20/f60 3384428 0 2024-03-24T00:49:04.695 INFO:tasks.workunit.client.0.smithi038.stdout:8/770: chown d0/d1d/d20/f23 1012709 1 2024-03-24T00:49:04.695 INFO:tasks.workunit.client.0.smithi038.stdout:8/771: chown d0/d45/d37/d5f/db7/c2b 28829427 1 2024-03-24T00:49:04.695 INFO:tasks.workunit.client.0.smithi038.stdout:8/772: fdatasync d0/d1d/d20/f58 0 2024-03-24T00:49:04.696 INFO:tasks.workunit.client.0.smithi038.stdout:8/773: chown d0/d45/db8/ld1 237 1 2024-03-24T00:49:04.696 INFO:tasks.workunit.client.0.smithi038.stdout:0/675: dread d0/d12/d42/f61 [4194304,4194304] 0 2024-03-24T00:49:04.697 INFO:tasks.workunit.client.1.smithi088.stdout:5/833: mkdir d1/dd/d6f/da1/d3f/d106/d115 0 2024-03-24T00:49:04.698 INFO:tasks.workunit.client.1.smithi088.stdout:5/834: fdatasync d1/d10/db6/d10b/d2e/d9f/ddf/fb0 0 2024-03-24T00:49:04.698 INFO:tasks.workunit.client.1.smithi088.stdout:3/752: sync 2024-03-24T00:49:04.706 INFO:tasks.workunit.client.0.smithi038.stdout:4/704: mknod da/d12/d29/d5b/d73/cf2 0 2024-03-24T00:49:04.706 INFO:tasks.workunit.client.0.smithi038.stdout:4/705: chown da/ldb 94730152 1 2024-03-24T00:49:04.706 INFO:tasks.workunit.client.0.smithi038.stdout:4/706: creat da/d12/d29/d5b/d73/db4/ff3 x:0 0 0 2024-03-24T00:49:04.710 INFO:tasks.workunit.client.1.smithi088.stdout:6/836: creat db/dc5/dc8/d43/f109 x:0 0 0 2024-03-24T00:49:04.719 INFO:tasks.workunit.client.1.smithi088.stdout:1/785: dwrite d6/d27/d67/d88/fa1 [0,4194304] 0 2024-03-24T00:49:04.719 INFO:tasks.workunit.client.1.smithi088.stdout:7/726: mkdir d3/d32/d34/d9c/d4b/d74/d79/dea 0 2024-03-24T00:49:04.726 INFO:tasks.workunit.client.0.smithi038.stdout:6/817: dwrite d1/d37/d84/fe5 [0,4194304] 0 2024-03-24T00:49:04.726 INFO:tasks.workunit.client.0.smithi038.stdout:6/818: fdatasync d1/f93 0 2024-03-24T00:49:04.727 INFO:tasks.workunit.client.0.smithi038.stdout:6/819: stat d1/da/d1b/d1d/l9a 0 2024-03-24T00:49:04.727 INFO:tasks.workunit.client.0.smithi038.stdout:6/820: stat d1/da/d1b/d2f/fd0 0 2024-03-24T00:49:04.732 INFO:tasks.workunit.client.0.smithi038.stdout:8/774: dread d0/d10/f92 [0,4194304] 0 2024-03-24T00:49:04.740 INFO:tasks.workunit.client.1.smithi088.stdout:5/835: creat d1/d10/db6/d10b/d2f/d57/d9e/dba/f116 x:0 0 0 2024-03-24T00:49:04.740 INFO:tasks.workunit.client.1.smithi088.stdout:5/836: readlink d1/d10/db6/d10b/d2f/d57/d78/lef 0 2024-03-24T00:49:04.748 INFO:tasks.workunit.client.1.smithi088.stdout:2/747: dwrite db/dd/f85 [0,4194304] 0 2024-03-24T00:49:04.749 INFO:tasks.workunit.client.1.smithi088.stdout:2/748: chown db/f55 114920430 1 2024-03-24T00:49:04.755 INFO:tasks.workunit.client.1.smithi088.stdout:6/837: creat db/d1d/d50/d90/d55/d99/f10a x:0 0 0 2024-03-24T00:49:04.765 INFO:tasks.workunit.client.0.smithi038.stdout:6/821: write d1/f6b [2059570,82175] 0 2024-03-24T00:49:04.766 INFO:tasks.workunit.client.0.smithi038.stdout:3/735: dwrite d3/d30/d70/d90/d99/fbf [0,4194304] 0 2024-03-24T00:49:04.766 INFO:tasks.workunit.client.0.smithi038.stdout:3/736: fdatasync d3/d9/f3c 0 2024-03-24T00:49:04.766 INFO:tasks.workunit.client.0.smithi038.stdout:5/693: dwrite d6/d19/d20/d42/d43/d68/dc8/fd4 [0,4194304] 0 2024-03-24T00:49:04.768 INFO:tasks.workunit.client.1.smithi088.stdout:6/838: dread db/d1d/d23/fa5 [0,4194304] 0 2024-03-24T00:49:04.768 INFO:tasks.workunit.client.1.smithi088.stdout:3/753: dwrite d3/d6/d19/d2a/dc2/dd5/fec [0,4194304] 0 2024-03-24T00:49:04.776 INFO:tasks.workunit.client.0.smithi038.stdout:0/676: mknod d0/d12/d92/ce5 0 2024-03-24T00:49:04.778 INFO:tasks.workunit.client.1.smithi088.stdout:1/786: symlink d6/dc2/l100 0 2024-03-24T00:49:04.778 INFO:tasks.workunit.client.1.smithi088.stdout:1/787: truncate d6/d6e/dfc/ffd 334925 0 2024-03-24T00:49:04.783 INFO:tasks.workunit.client.1.smithi088.stdout:5/837: rmdir d1/d10/db6/d10b/d2f/d57 39 2024-03-24T00:49:04.784 INFO:tasks.workunit.client.0.smithi038.stdout:4/707: truncate da/d12/d24/d5f/d8a/fc4 3379981 0 2024-03-24T00:49:04.784 INFO:tasks.workunit.client.0.smithi038.stdout:4/708: creat da/d12/d24/d68/d6c/ff4 x:0 0 0 2024-03-24T00:49:04.784 INFO:tasks.workunit.client.0.smithi038.stdout:4/709: chown da/l13 1 1 2024-03-24T00:49:04.787 INFO:tasks.workunit.client.0.smithi038.stdout:8/775: mkdir d0/d45/d66/d6d/d106 0 2024-03-24T00:49:04.789 INFO:tasks.workunit.client.0.smithi038.stdout:1/801: rmdir da/dd/d7a/d82/d98/d14 39 2024-03-24T00:49:04.792 INFO:tasks.workunit.client.0.smithi038.stdout:3/737: dread d3/d9/dc/df/f1d [0,4194304] 0 2024-03-24T00:49:04.796 INFO:tasks.workunit.client.1.smithi088.stdout:4/773: sync 2024-03-24T00:49:04.796 INFO:tasks.workunit.client.1.smithi088.stdout:9/780: sync 2024-03-24T00:49:04.796 INFO:tasks.workunit.client.1.smithi088.stdout:4/774: stat d5/d13/d63/d78 0 2024-03-24T00:49:04.796 INFO:tasks.workunit.client.1.smithi088.stdout:9/781: chown d1/d15/d84/fdf 509 1 2024-03-24T00:49:04.797 INFO:tasks.workunit.client.1.smithi088.stdout:8/768: sync 2024-03-24T00:49:04.797 INFO:tasks.workunit.client.1.smithi088.stdout:0/809: sync 2024-03-24T00:49:04.798 INFO:tasks.workunit.client.1.smithi088.stdout:0/810: write d9/dc/de/d19/d44/d46/d65/d6f/f9a [866998,47785] 0 2024-03-24T00:49:04.800 INFO:tasks.workunit.client.1.smithi088.stdout:2/749: getdents db/dd/d96/d99/db2/d88 0 2024-03-24T00:49:04.811 INFO:tasks.workunit.client.0.smithi038.stdout:6/822: symlink d1/da/d61/da2/l10e 0 2024-03-24T00:49:04.814 INFO:tasks.workunit.client.1.smithi088.stdout:1/788: write d6/d23/daf/d95/d58/f48 [2689405,95672] 0 2024-03-24T00:49:04.818 INFO:tasks.workunit.client.1.smithi088.stdout:5/838: mkdir d1/d10/db6/d10b/d2f/d93/d117 0 2024-03-24T00:49:04.819 INFO:tasks.workunit.client.0.smithi038.stdout:1/802: read da/dd/d2c/d41/d60/fc5 [3320689,109804] 0 2024-03-24T00:49:04.819 INFO:tasks.workunit.client.0.smithi038.stdout:1/803: dread - da/dd/d2c/d41/d60/dae/fc8 zero size 2024-03-24T00:49:04.821 INFO:tasks.workunit.client.1.smithi088.stdout:4/775: mkdir d5/d13/d18/d6e/d10f 0 2024-03-24T00:49:04.825 INFO:tasks.workunit.client.1.smithi088.stdout:4/776: fsync d5/d7/d15/d74/fe9 0 2024-03-24T00:49:04.829 INFO:tasks.workunit.client.1.smithi088.stdout:9/782: symlink d1/d10/d77/lfc 0 2024-03-24T00:49:04.830 INFO:tasks.workunit.client.0.smithi038.stdout:2/670: sync 2024-03-24T00:49:04.830 INFO:tasks.workunit.client.0.smithi038.stdout:7/688: sync 2024-03-24T00:49:04.831 INFO:tasks.workunit.client.1.smithi088.stdout:7/727: sync 2024-03-24T00:49:04.833 INFO:tasks.workunit.client.0.smithi038.stdout:5/694: mknod d6/d1e/ce1 0 2024-03-24T00:49:04.833 INFO:tasks.workunit.client.0.smithi038.stdout:5/695: stat d6/d23/f53 0 2024-03-24T00:49:04.837 INFO:tasks.workunit.client.1.smithi088.stdout:8/769: mknod d9/dc/d11/d24/d29/da9/de7/c102 0 2024-03-24T00:49:04.847 INFO:tasks.workunit.client.1.smithi088.stdout:0/811: creat d9/dc/de/d19/f106 x:0 0 0 2024-03-24T00:49:04.847 INFO:tasks.workunit.client.1.smithi088.stdout:2/750: symlink db/dd/d96/d99/dc3/le9 0 2024-03-24T00:49:04.847 INFO:tasks.workunit.client.1.smithi088.stdout:2/751: read - db/d21/d35/dcb/fce zero size 2024-03-24T00:49:04.849 INFO:tasks.workunit.client.0.smithi038.stdout:0/677: rename d0/d12/d42/d5e/caa to d0/d12/d15/d58/d64/da7/dbb/dd4/ce6 0 2024-03-24T00:49:04.855 INFO:tasks.workunit.client.0.smithi038.stdout:8/776: dwrite d0/d45/d37/d5f/d68/f91 [0,4194304] 0 2024-03-24T00:49:04.860 INFO:tasks.workunit.client.0.smithi038.stdout:4/710: link da/d38/f61 da/d12/d37/dd/d15/d4c/d70/ff5 0 2024-03-24T00:49:04.863 INFO:tasks.workunit.client.1.smithi088.stdout:7/728: creat d3/d32/d34/d9c/d4b/d73/dab/dc3/feb x:0 0 0 2024-03-24T00:49:04.863 INFO:tasks.workunit.client.1.smithi088.stdout:7/729: creat d3/d32/d34/d9c/d4b/d73/dab/fec x:0 0 0 2024-03-24T00:49:04.863 INFO:tasks.workunit.client.0.smithi038.stdout:9/826: sync 2024-03-24T00:49:04.863 INFO:tasks.workunit.client.0.smithi038.stdout:9/827: stat d12/d28/d3a/f2b 0 2024-03-24T00:49:04.864 INFO:tasks.workunit.client.0.smithi038.stdout:9/828: dread - d12/d41/d57/dd3/dfe/f92 zero size 2024-03-24T00:49:04.864 INFO:tasks.workunit.client.0.smithi038.stdout:3/738: mkdir d3/d9/dc/da6/dfc 0 2024-03-24T00:49:04.866 INFO:tasks.workunit.client.1.smithi088.stdout:5/839: dread d1/d10/db6/d10b/d2f/d57/f5f [0,4194304] 0 2024-03-24T00:49:04.866 INFO:tasks.workunit.client.1.smithi088.stdout:5/840: fdatasync d1/d10/db6/d10b/d2f/d93/fa9 0 2024-03-24T00:49:04.867 INFO:tasks.workunit.client.1.smithi088.stdout:8/770: creat d9/dc/dce/ddc/d100/f103 x:0 0 0 2024-03-24T00:49:04.867 INFO:tasks.workunit.client.1.smithi088.stdout:8/771: chown d9/dc/d11/d24/d29/d36/d6a/l7f 54757 1 2024-03-24T00:49:04.867 INFO:tasks.workunit.client.1.smithi088.stdout:8/772: fsync d9/dc/d11/d24/d29/d36/dc1/dc4/fe2 0 2024-03-24T00:49:04.868 INFO:tasks.workunit.client.0.smithi038.stdout:6/823: symlink d1/da/d1b/l10f 0 2024-03-24T00:49:04.868 INFO:tasks.workunit.client.0.smithi038.stdout:2/671: symlink d4/d27/le6 0 2024-03-24T00:49:04.871 INFO:tasks.workunit.client.1.smithi088.stdout:0/812: symlink d9/dc/de/d19/d44/d46/d65/d2a/d2e/l107 0 2024-03-24T00:49:04.871 INFO:tasks.workunit.client.1.smithi088.stdout:0/813: write d9/dc/de/f14 [1409407,37720] 0 2024-03-24T00:49:04.871 INFO:tasks.workunit.client.1.smithi088.stdout:0/814: fsync d9/dc/de/f3e 0 2024-03-24T00:49:04.871 INFO:tasks.workunit.client.1.smithi088.stdout:0/815: fdatasync d9/dc/de/d19/d44/d46/ff2 0 2024-03-24T00:49:04.878 INFO:tasks.workunit.client.1.smithi088.stdout:8/773: creat d9/dc/d11/f104 x:0 0 0 2024-03-24T00:49:04.878 INFO:tasks.workunit.client.1.smithi088.stdout:8/774: chown d9/dc/d11/d24/d29/d36/dc1/dc4/fe2 6384 1 2024-03-24T00:49:04.878 INFO:tasks.workunit.client.1.smithi088.stdout:8/775: creat d9/d6b/f105 x:0 0 0 2024-03-24T00:49:04.879 INFO:tasks.workunit.client.0.smithi038.stdout:5/696: unlink d6/d1e/fcc 0 2024-03-24T00:49:04.879 INFO:tasks.workunit.client.0.smithi038.stdout:5/697: readlink d6/lc9 0 2024-03-24T00:49:04.879 INFO:tasks.workunit.client.0.smithi038.stdout:1/804: rename da/dd/d2c/c62 to da/dd/d2c/d40/de9/d104/c10b 0 2024-03-24T00:49:04.879 INFO:tasks.workunit.client.0.smithi038.stdout:1/805: chown da/dd/d2c/d5a/cd2 2986546 1 2024-03-24T00:49:04.879 INFO:tasks.workunit.client.0.smithi038.stdout:0/678: creat d0/d12/d15/d58/fe7 x:0 0 0 2024-03-24T00:49:04.886 INFO:tasks.workunit.client.1.smithi088.stdout:6/839: dwrite db/d3d/d83/d72/f76 [0,4194304] 0 2024-03-24T00:49:04.887 INFO:tasks.workunit.client.0.smithi038.stdout:8/777: unlink d0/cc 0 2024-03-24T00:49:04.887 INFO:tasks.workunit.client.0.smithi038.stdout:8/778: read - d0/d1d/d27/d6b/fa1 zero size 2024-03-24T00:49:04.889 INFO:tasks.workunit.client.0.smithi038.stdout:3/739: symlink d3/d9/dc/da6/db5/d19/dd2/df0/lfd 0 2024-03-24T00:49:04.890 INFO:tasks.workunit.client.1.smithi088.stdout:4/777: dwrite d5/d7/dd/f75 [0,4194304] 0 2024-03-24T00:49:04.890 INFO:tasks.workunit.client.1.smithi088.stdout:4/778: truncate d5/d13/f96 596520 0 2024-03-24T00:49:04.890 INFO:tasks.workunit.client.1.smithi088.stdout:3/754: dwrite f2 [4194304,4194304] 0 2024-03-24T00:49:04.894 INFO:tasks.workunit.client.0.smithi038.stdout:6/824: unlink d1/da/d1b/d1d/d3c/l47 0 2024-03-24T00:49:04.894 INFO:tasks.workunit.client.0.smithi038.stdout:2/672: link d4/d18/d26/d37/d69/d83/fb0 d4/d18/d5b/fe7 0 2024-03-24T00:49:04.894 INFO:tasks.workunit.client.0.smithi038.stdout:6/825: fdatasync d1/da/d1b/d2f/fd0 0 2024-03-24T00:49:04.894 INFO:tasks.workunit.client.0.smithi038.stdout:2/673: readlink d4/d18/l1a 0 2024-03-24T00:49:04.894 INFO:tasks.workunit.client.0.smithi038.stdout:6/826: chown d1/da/d61/d69 146 1 2024-03-24T00:49:04.898 INFO:tasks.workunit.client.0.smithi038.stdout:5/698: dread d6/d16/d21/d2d/fd2 [0,4194304] 0 2024-03-24T00:49:04.902 INFO:tasks.workunit.client.0.smithi038.stdout:5/699: creat d6/d19/d20/d42/d79/fe2 x:0 0 0 2024-03-24T00:49:04.903 INFO:tasks.workunit.client.0.smithi038.stdout:7/689: symlink d2/d2c/l103 0 2024-03-24T00:49:04.903 INFO:tasks.workunit.client.0.smithi038.stdout:7/690: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f8f 0 2024-03-24T00:49:04.905 INFO:tasks.workunit.client.1.smithi088.stdout:1/789: dwrite d6/d27/d67/d88/fa1 [0,4194304] 0 2024-03-24T00:49:04.906 INFO:tasks.workunit.client.1.smithi088.stdout:1/790: dread - d6/d23/daf/d95/d5b/d5d/d61/d7d/fd3 zero size 2024-03-24T00:49:04.914 INFO:tasks.workunit.client.1.smithi088.stdout:6/840: dread db/dd/f41 [0,4194304] 0 2024-03-24T00:49:04.923 INFO:tasks.workunit.client.0.smithi038.stdout:0/679: unlink d0/d12/d15/c18 0 2024-03-24T00:49:04.923 INFO:tasks.workunit.client.0.smithi038.stdout:5/700: dread d6/d16/d67/f8f [0,4194304] 0 2024-03-24T00:49:04.923 INFO:tasks.workunit.client.0.smithi038.stdout:5/701: getdents d6/d19/d20/d64/d6b/d77 0 2024-03-24T00:49:04.933 INFO:tasks.workunit.client.0.smithi038.stdout:3/740: symlink d3/d9/dc/da6/db5/lfe 0 2024-03-24T00:49:04.943 INFO:tasks.workunit.client.0.smithi038.stdout:2/674: write d4/d27/ddb/f61 [2471323,1791] 0 2024-03-24T00:49:04.943 INFO:tasks.workunit.client.0.smithi038.stdout:2/675: chown d4/d18/c20 52008582 1 2024-03-24T00:49:04.944 INFO:tasks.workunit.client.0.smithi038.stdout:1/806: dread da/dd/d7a/d82/fb4 [0,4194304] 0 2024-03-24T00:49:04.946 INFO:tasks.workunit.client.0.smithi038.stdout:6/827: creat d1/da/d1b/d25/d49/d4e/f110 x:0 0 0 2024-03-24T00:49:04.947 INFO:tasks.workunit.client.1.smithi088.stdout:7/730: dwrite d3/d31/f3c [0,4194304] 0 2024-03-24T00:49:04.947 INFO:tasks.workunit.client.1.smithi088.stdout:7/731: dread - d3/d32/d34/d9c/d4b/d73/dab/fb3 zero size 2024-03-24T00:49:04.947 INFO:tasks.workunit.client.1.smithi088.stdout:7/732: dread - d3/d32/d34/d9c/d4b/d74/f82 zero size 2024-03-24T00:49:04.947 INFO:tasks.workunit.client.1.smithi088.stdout:7/733: chown d3/d31/fc7 11284478 1 2024-03-24T00:49:04.947 INFO:tasks.workunit.client.1.smithi088.stdout:7/734: chown d3/d32/d34/d9c/d4b/fdf 27127812 1 2024-03-24T00:49:04.950 INFO:tasks.workunit.client.0.smithi038.stdout:9/829: dwrite d12/d28/d3a/d26/f3d [0,4194304] 0 2024-03-24T00:49:04.953 INFO:tasks.workunit.client.1.smithi088.stdout:2/752: rmdir db 39 2024-03-24T00:49:04.953 INFO:tasks.workunit.client.1.smithi088.stdout:2/753: chown db/d21/d35/d49/fde 5 1 2024-03-24T00:49:04.958 INFO:tasks.workunit.client.0.smithi038.stdout:0/680: write d0/d12/f3c [3184506,109656] 0 2024-03-24T00:49:04.959 INFO:tasks.workunit.client.1.smithi088.stdout:6/841: dread db/d1d/d23/fcc [0,4194304] 0 2024-03-24T00:49:04.959 INFO:tasks.workunit.client.1.smithi088.stdout:6/842: chown db/dd/d61/d6d/d74/cde 72913604 1 2024-03-24T00:49:04.959 INFO:tasks.workunit.client.1.smithi088.stdout:6/843: fsync db/dd/d61/d6d/d74/d88/f9c 0 2024-03-24T00:49:04.959 INFO:tasks.workunit.client.1.smithi088.stdout:6/844: chown db/d1d/d50/d90/d55/l82 209154210 1 2024-03-24T00:49:04.962 INFO:tasks.workunit.client.0.smithi038.stdout:5/702: symlink d6/d19/d20/d42/d43/d48/le3 0 2024-03-24T00:49:04.963 INFO:tasks.workunit.client.0.smithi038.stdout:7/691: creat d2/d9/da/d23/d2b/d5b/d72/d92/d5d/dcd/f104 x:0 0 0 2024-03-24T00:49:04.963 INFO:tasks.workunit.client.0.smithi038.stdout:3/741: creat d3/d30/fff x:0 0 0 2024-03-24T00:49:04.963 INFO:tasks.workunit.client.0.smithi038.stdout:5/703: creat d6/d19/d20/d33/d6f/fe4 x:0 0 0 2024-03-24T00:49:04.964 INFO:tasks.workunit.client.0.smithi038.stdout:4/711: dwrite da/d12/d24/d68/d6c/d7c/f90 [0,4194304] 0 2024-03-24T00:49:04.966 INFO:tasks.workunit.client.1.smithi088.stdout:4/779: creat d5/f110 x:0 0 0 2024-03-24T00:49:04.967 INFO:tasks.workunit.client.0.smithi038.stdout:2/676: mkdir d4/d27/ddb/de8 0 2024-03-24T00:49:04.967 INFO:tasks.workunit.client.0.smithi038.stdout:2/677: truncate d4/d18/f31 561837 0 2024-03-24T00:49:04.967 INFO:tasks.workunit.client.0.smithi038.stdout:2/678: fdatasync d4/d27/ddb/f8b 0 2024-03-24T00:49:04.967 INFO:tasks.workunit.client.0.smithi038.stdout:2/679: readlink d4/d55/lcb 0 2024-03-24T00:49:04.970 INFO:tasks.workunit.client.0.smithi038.stdout:1/807: write da/f16 [4512082,69415] 0 2024-03-24T00:49:04.970 INFO:tasks.workunit.client.0.smithi038.stdout:1/808: chown f5 843 1 2024-03-24T00:49:04.970 INFO:tasks.workunit.client.0.smithi038.stdout:1/809: chown da/dd/d2c/d40/c69 25569 1 2024-03-24T00:49:04.970 INFO:tasks.workunit.client.1.smithi088.stdout:9/783: dwrite d1/d10/d77/d8a/f94 [0,4194304] 0 2024-03-24T00:49:04.971 INFO:tasks.workunit.client.0.smithi038.stdout:0/681: creat d0/dd7/fe8 x:0 0 0 2024-03-24T00:49:04.980 INFO:tasks.workunit.client.0.smithi038.stdout:7/692: mkdir d2/d9/da/dc/d8a/d105 0 2024-03-24T00:49:04.980 INFO:tasks.workunit.client.0.smithi038.stdout:7/693: readlink d2/d9/da/dc/d8a/db8/lc3 0 2024-03-24T00:49:04.980 INFO:tasks.workunit.client.0.smithi038.stdout:7/694: fsync d2/d9/da/f1d 0 2024-03-24T00:49:04.981 INFO:tasks.workunit.client.0.smithi038.stdout:7/695: write d2/d9/da/dc/da8/dc5/fe5 [2363799,107437] 0 2024-03-24T00:49:04.981 INFO:tasks.workunit.client.0.smithi038.stdout:7/696: getdents d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d 0 2024-03-24T00:49:04.981 INFO:tasks.workunit.client.0.smithi038.stdout:7/697: chown d2/d9/da/d9b/ff9 805 1 2024-03-24T00:49:04.981 INFO:tasks.workunit.client.0.smithi038.stdout:7/698: chown d2/d9/da/d23/d2b/lde 61724973 1 2024-03-24T00:49:04.981 INFO:tasks.workunit.client.0.smithi038.stdout:8/779: dwrite d0/d1d/d27/f8e [0,4194304] 0 2024-03-24T00:49:04.982 INFO:tasks.workunit.client.1.smithi088.stdout:1/791: rename d6/d23/daf/d95/d58/f48 to d6/d23/daf/d95/d5b/d5d/d61/d7d/da5/f101 0 2024-03-24T00:49:04.983 INFO:tasks.workunit.client.1.smithi088.stdout:3/755: creat d3/d6/fee x:0 0 0 2024-03-24T00:49:04.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.981+0000 7f1200a00700 1 -- 172.21.15.38:0/435255665 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f11fc0611f0 con 0x7f11e006d600 2024-03-24T00:49:04.987 INFO:tasks.workunit.client.0.smithi038.stdout:5/704: mknod d6/d16/d21/d2d/da6/ce5 0 2024-03-24T00:49:04.989 INFO:tasks.workunit.client.1.smithi088.stdout:7/735: mknod d3/dbf/ced 0 2024-03-24T00:49:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.997+0000 7f11f2a00700 1 -- 172.21.15.38:0/435255665 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f11fc0611f0 con 0x7f11e006d600 2024-03-24T00:49:04.998 INFO:tasks.workunit.client.0.smithi038.stdout:6/828: dwrite d1/d37/d68/f85 [0,4194304] 0 2024-03-24T00:49:05.000 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:49:05.000 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (9m) 50s ago 11m 13.8M - 0.25.0 c8568f914cd2 9f1243fbcb3e 2024-03-24T00:49:05.000 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (11m) 50s ago 11m 7922k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:49:05.000 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (10m) 60s ago 10m 7730k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:49:05.000 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (11m) 50s ago 11m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (10m) 60s ago 10m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (9m) 50s ago 11m 74.5M - 9.4.7 954c08fa6188 e6aa268d69e8 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (4m) 50s ago 3m 78.9M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (4m) 50s ago 4m 133M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (4m) 60s ago 4m 20.5M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (3m) 60s ago 3m 19.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 starting - - - - 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (83s) 60s ago 10m 490M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b21fee18bc67 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (13m) 50s ago 13m 49.6M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (9m) 60s ago 9m 37.8M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (11m) 50s ago 11m 8006k - 1.5.0 0da6a335fe13 c34cc077c93f 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (10m) 60s ago 10m 7998k - 1.5.0 0da6a335fe13 189e840ee2f1 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (8m) 50s ago 8m 103M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (8m) 50s ago 8m 111M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:49:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (7m) 50s ago 7m 89.9M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:49:05.002 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (7m) 60s ago 7m 109M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:49:05.002 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (6m) 60s ago 6m 99.5M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:49:05.002 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (5m) 60s ago 5m 97.5M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:49:05.002 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 starting - - - - 2024-03-24T00:49:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.999+0000 7f1200a00700 1 -- 172.21.15.38:0/435255665 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f11e006d600 msgr2=0x7f11e006fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.999+0000 7f1200a00700 1 --2- 172.21.15.38:0/435255665 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f11e006d600 0x7f11e006fac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f11e8005fd0 tx=0x7f11e8005d00 comp rx=0 tx=0).stop 2024-03-24T00:49:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.999+0000 7f1200a00700 1 -- 172.21.15.38:0/435255665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11fc102d00 msgr2=0x7f11fc19cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:04.999+0000 7f1200a00700 1 --2- 172.21.15.38:0/435255665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11fc102d00 0x7f11fc19cb20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00bd70 tx=0x7f11ec00be50 comp rx=0 tx=0).stop 2024-03-24T00:49:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.000+0000 7f1200a00700 1 -- 172.21.15.38:0/435255665 shutdown_connections 2024-03-24T00:49:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.000+0000 7f1200a00700 1 --2- 172.21.15.38:0/435255665 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f11e006d600 0x7f11e006fac0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.000+0000 7f1200a00700 1 --2- 172.21.15.38:0/435255665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11fc102d00 0x7f11fc19cb20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.000+0000 7f1200a00700 1 --2- 172.21.15.38:0/435255665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11fc103f00 0x7f11fc19d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.000+0000 7f1200a00700 1 -- 172.21.15.38:0/435255665 >> 172.21.15.38:0/435255665 conn(0x7f11fc0fe2a0 msgr2=0x7f11fc107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.000+0000 7f1200a00700 1 -- 172.21.15.38:0/435255665 shutdown_connections 2024-03-24T00:49:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.000+0000 7f1200a00700 1 -- 172.21.15.38:0/435255665 wait complete. 2024-03-24T00:49:05.042 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:04 smithi038 ceph-mon[29271]: pgmap v38: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 144 MiB/s rd, 165 MiB/s wr, 252 op/s 2024-03-24T00:49:05.042 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:04 smithi038 ceph-mon[29271]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:05.061 INFO:tasks.workunit.client.0.smithi038.stdout:4/712: mknod da/d12/d29/d5b/d73/db4/cf6 0 2024-03-24T00:49:05.061 INFO:tasks.workunit.client.0.smithi038.stdout:4/713: truncate da/d12/f87 463717 0 2024-03-24T00:49:05.066 INFO:tasks.workunit.client.0.smithi038.stdout:1/810: mknod da/dd/d2c/d5a/d101/dda/de6/df8/c10c 0 2024-03-24T00:49:05.067 INFO:tasks.workunit.client.0.smithi038.stdout:1/811: creat da/dd/d7a/d82/ddb/f10d x:0 0 0 2024-03-24T00:49:05.069 INFO:tasks.workunit.client.0.smithi038.stdout:9/830: getdents d12/d28/d9c/ddd 0 2024-03-24T00:49:05.069 INFO:tasks.workunit.client.0.smithi038.stdout:9/831: fsync d12/d1a/d80/fb4 0 2024-03-24T00:49:05.069 INFO:tasks.workunit.client.0.smithi038.stdout:9/832: chown d12/d28/d6c/dbc/dd7/dda 414453294 1 2024-03-24T00:49:05.069 INFO:tasks.workunit.client.0.smithi038.stdout:9/833: creat d12/d41/d57/f109 x:0 0 0 2024-03-24T00:49:05.070 INFO:tasks.workunit.client.0.smithi038.stdout:9/834: truncate d12/d28/f36 2538126 0 2024-03-24T00:49:05.073 INFO:tasks.workunit.client.0.smithi038.stdout:0/682: mknod d0/d14/de4/ce9 0 2024-03-24T00:49:05.082 INFO:tasks.workunit.client.0.smithi038.stdout:8/780: mkdir d0/d45/d66/d107 0 2024-03-24T00:49:05.090 INFO:tasks.workunit.client.0.smithi038.stdout:5/705: unlink d6/d19/d20/d42/d79/f9e 0 2024-03-24T00:49:05.090 INFO:tasks.workunit.client.0.smithi038.stdout:0/683: read d0/d2d/fb3 [710099,5641] 0 2024-03-24T00:49:05.090 INFO:tasks.workunit.client.0.smithi038.stdout:0/684: stat d0/d12/d42/l81 0 2024-03-24T00:49:05.098 INFO:tasks.workunit.client.0.smithi038.stdout:8/781: write d0/d1d/d20/f60 [448527,128131] 0 2024-03-24T00:49:05.108 INFO:tasks.workunit.client.1.smithi088.stdout:0/816: rmdir d9/dc/de/d19 39 2024-03-24T00:49:05.111 INFO:tasks.workunit.client.1.smithi088.stdout:6/845: symlink db/d3d/dcb/l10b 0 2024-03-24T00:49:05.115 INFO:tasks.workunit.client.1.smithi088.stdout:4/780: getdents d5/d8b 0 2024-03-24T00:49:05.115 INFO:tasks.workunit.client.1.smithi088.stdout:4/781: fdatasync d5/d17/d91/fe3 0 2024-03-24T00:49:05.115 INFO:tasks.workunit.client.1.smithi088.stdout:4/782: fsync d5/d16/d94/da6/fc7 0 2024-03-24T00:49:05.124 INFO:tasks.workunit.client.0.smithi038.stdout:4/714: truncate da/d12/d24/d5f/fda 799459 0 2024-03-24T00:49:05.128 INFO:tasks.workunit.client.0.smithi038.stdout:5/706: dread d6/d19/d20/d42/f72 [0,4194304] 0 2024-03-24T00:49:05.130 INFO:tasks.workunit.client.0.smithi038.stdout:2/680: rmdir d4/da3 39 2024-03-24T00:49:05.141 INFO:tasks.workunit.client.1.smithi088.stdout:9/784: truncate d1/d15/fdd 304546 0 2024-03-24T00:49:05.144 INFO:tasks.workunit.client.1.smithi088.stdout:1/792: rename d6/d23/daf/d95/d3f/c55 to d6/d23/daf/d95/d5b/d5d/d61/c102 0 2024-03-24T00:49:05.149 INFO:tasks.workunit.client.1.smithi088.stdout:0/817: read d9/d26/d8f/da1/d63/fa7 [1385422,65298] 0 2024-03-24T00:49:05.152 INFO:tasks.workunit.client.1.smithi088.stdout:3/756: creat d3/de/d6f/d85/d49/d70/d96/fef x:0 0 0 2024-03-24T00:49:05.152 INFO:tasks.workunit.client.1.smithi088.stdout:3/757: chown d3/d1a/d1e/f9b 1 1 2024-03-24T00:49:05.153 INFO:tasks.workunit.client.0.smithi038.stdout:9/835: creat d12/d28/d71/d99/f10a x:0 0 0 2024-03-24T00:49:05.156 INFO:tasks.workunit.client.0.smithi038.stdout:1/812: dwrite da/dd/f1c [8388608,4194304] 0 2024-03-24T00:49:05.156 INFO:tasks.workunit.client.1.smithi088.stdout:4/783: dread d5/d17/f83 [0,4194304] 0 2024-03-24T00:49:05.156 INFO:tasks.workunit.client.1.smithi088.stdout:2/754: mkdir db/d21/dea 0 2024-03-24T00:49:05.159 INFO:tasks.workunit.client.1.smithi088.stdout:5/841: sync 2024-03-24T00:49:05.159 INFO:tasks.workunit.client.1.smithi088.stdout:8/776: sync 2024-03-24T00:49:05.160 INFO:tasks.workunit.client.1.smithi088.stdout:6/846: link db/dd/fd9 db/dd/f10c 0 2024-03-24T00:49:05.163 INFO:tasks.workunit.client.0.smithi038.stdout:8/782: symlink d0/d45/d37/d5f/db7/dcc/l108 0 2024-03-24T00:49:05.163 INFO:tasks.workunit.client.0.smithi038.stdout:8/783: chown d0/d1d/d27/d4d/cb0 0 1 2024-03-24T00:49:05.163 INFO:tasks.workunit.client.0.smithi038.stdout:8/784: chown d0/d1d/d27/d4d/f53 31130 1 2024-03-24T00:49:05.163 INFO:tasks.workunit.client.0.smithi038.stdout:8/785: readlink d0/d1d/d8c/lf8 0 2024-03-24T00:49:05.163 INFO:tasks.workunit.client.0.smithi038.stdout:8/786: stat d0/d15/c51 0 2024-03-24T00:49:05.165 INFO:tasks.workunit.client.1.smithi088.stdout:1/793: mkdir d6/d27/d103 0 2024-03-24T00:49:05.169 INFO:tasks.workunit.client.1.smithi088.stdout:7/736: dread d3/f51 [0,4194304] 0 2024-03-24T00:49:05.171 INFO:tasks.workunit.client.1.smithi088.stdout:0/818: getdents d9/d26 0 2024-03-24T00:49:05.171 INFO:tasks.workunit.client.1.smithi088.stdout:0/819: stat d9/dc/de/d19/d44/d46/d65/d2a/c4d 0 2024-03-24T00:49:05.171 INFO:tasks.workunit.client.1.smithi088.stdout:0/820: write d9/d26/d8f/da1/d63/f8a [5109592,68686] 0 2024-03-24T00:49:05.174 INFO:tasks.workunit.client.0.smithi038.stdout:6/829: rmdir d1/da/d1b/d1d/d3c 39 2024-03-24T00:49:05.174 INFO:tasks.workunit.client.0.smithi038.stdout:6/830: write d1/da/d61/d69/fc2 [993629,55611] 0 2024-03-24T00:49:05.177 INFO:tasks.workunit.client.1.smithi088.stdout:4/784: truncate d5/d7/dd/f75 2698449 0 2024-03-24T00:49:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:04 smithi088 ceph-mon[36610]: pgmap v38: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 144 MiB/s rd, 165 MiB/s wr, 252 op/s 2024-03-24T00:49:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:04 smithi088 ceph-mon[36610]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:05.184 INFO:tasks.workunit.client.0.smithi038.stdout:4/715: creat da/d12/ff7 x:0 0 0 2024-03-24T00:49:05.189 INFO:tasks.workunit.client.0.smithi038.stdout:8/787: dread d0/d1d/d27/d6b/dfc/f47 [0,4194304] 0 2024-03-24T00:49:05.189 INFO:tasks.workunit.client.0.smithi038.stdout:8/788: chown d0/d45/d37/d5f/db7/d62/dfa/cf9 247 1 2024-03-24T00:49:05.192 INFO:tasks.workunit.client.1.smithi088.stdout:2/755: rename db/faa to db/dd/d96/d99/feb 0 2024-03-24T00:49:05.196 INFO:tasks.workunit.client.0.smithi038.stdout:5/707: unlink d6/d16/d21/c47 0 2024-03-24T00:49:05.198 INFO:tasks.workunit.client.1.smithi088.stdout:5/842: mkdir d1/dd/d6f/da1/d8e/d118 0 2024-03-24T00:49:05.201 INFO:tasks.workunit.client.0.smithi038.stdout:2/681: symlink d4/d18/d26/d37/d69/le9 0 2024-03-24T00:49:05.205 INFO:tasks.workunit.client.1.smithi088.stdout:8/777: link d9/dc/d11/dd2/le3 d9/dc/d11/d24/d29/d36/l106 0 2024-03-24T00:49:05.215 INFO:tasks.workunit.client.1.smithi088.stdout:6/847: mkdir db/d8e/d10d 0 2024-03-24T00:49:05.215 INFO:tasks.workunit.client.1.smithi088.stdout:6/848: write db/d1d/d23/fa5 [1102049,29850] 0 2024-03-24T00:49:05.215 INFO:tasks.workunit.client.1.smithi088.stdout:6/849: readlink db/dd/l14 0 2024-03-24T00:49:05.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.214+0000 7f054ea00700 1 -- 172.21.15.38:0/355567068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0548101cc0 msgr2=0x7f0548104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:05.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.214+0000 7f054ea00700 1 --2- 172.21.15.38:0/355567068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0548101cc0 0x7f0548104150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0534009910 tx=0x7f0534009c20 comp rx=0 tx=0).stop 2024-03-24T00:49:05.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.214+0000 7f054ea00700 1 -- 172.21.15.38:0/355567068 shutdown_connections 2024-03-24T00:49:05.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.214+0000 7f054ea00700 1 --2- 172.21.15.38:0/355567068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0548101cc0 0x7f0548104150 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:05.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.214+0000 7f054ea00700 1 --2- 172.21.15.38:0/355567068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05480ff350 0x7f0548101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:05.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.214+0000 7f054ea00700 1 -- 172.21.15.38:0/355567068 >> 172.21.15.38:0/355567068 conn(0x7f05480faf70 msgr2=0x7f05480fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:05.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.214+0000 7f054ea00700 1 -- 172.21.15.38:0/355567068 shutdown_connections 2024-03-24T00:49:05.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.214+0000 7f054ea00700 1 -- 172.21.15.38:0/355567068 wait complete. 2024-03-24T00:49:05.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.216+0000 7f054ea00700 1 Processor -- start 2024-03-24T00:49:05.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.217+0000 7f054ea00700 1 -- start start 2024-03-24T00:49:05.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.218+0000 7f054ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05480ff350 0x7f054810cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:05.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.218+0000 7f054ea00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0548101cc0 0x7f054810d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:05.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.218+0000 7f054ea00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054810da80 con 0x7f05480ff350 2024-03-24T00:49:05.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.218+0000 7f054ea00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054810dbf0 con 0x7f0548101cc0 2024-03-24T00:49:05.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.219+0000 7f0547e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05480ff350 0x7f054810cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:05.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.219+0000 7f0547e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05480ff350 0x7f054810cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34748/0 (socket says 172.21.15.38:34748) 2024-03-24T00:49:05.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.219+0000 7f0547e00700 1 -- 172.21.15.38:0/3827082009 learned_addr learned my addr 172.21.15.38:0/3827082009 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:49:05.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.220+0000 7f0547400700 1 --2- 172.21.15.38:0/3827082009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0548101cc0 0x7f054810d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:05.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.221+0000 7f0547400700 1 -- 172.21.15.38:0/3827082009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05480ff350 msgr2=0x7f054810cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:05.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.221+0000 7f0547400700 1 --2- 172.21.15.38:0/3827082009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05480ff350 0x7f054810cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:05.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.222+0000 7f0547400700 1 -- 172.21.15.38:0/3827082009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340095f0 con 0x7f0548101cc0 2024-03-24T00:49:05.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.223+0000 7f0547400700 1 --2- 172.21.15.38:0/3827082009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0548101cc0 0x7f054810d4f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f05340051c0 tx=0x7f053400bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:05.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.223+0000 7f0544c00700 1 -- 172.21.15.38:0/3827082009 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f053401d070 con 0x7f0548101cc0 2024-03-24T00:49:05.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.224+0000 7f0544c00700 1 -- 172.21.15.38:0/3827082009 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f053400f460 con 0x7f0548101cc0 2024-03-24T00:49:05.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.224+0000 7f0544c00700 1 -- 172.21.15.38:0/3827082009 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0534021640 con 0x7f0548101cc0 2024-03-24T00:49:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.225+0000 7f054ea00700 1 -- 172.21.15.38:0/3827082009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05481aa8b0 con 0x7f0548101cc0 2024-03-24T00:49:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.225+0000 7f054ea00700 1 -- 172.21.15.38:0/3827082009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05481aadf0 con 0x7f0548101cc0 2024-03-24T00:49:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.226+0000 7f054ea00700 1 -- 172.21.15.38:0/3827082009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054810fc20 con 0x7f0548101cc0 2024-03-24T00:49:05.227 INFO:tasks.workunit.client.0.smithi038.stdout:9/836: creat d12/d81/f10b x:0 0 0 2024-03-24T00:49:05.231 INFO:tasks.workunit.client.1.smithi088.stdout:7/737: mknod d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/dd6/cee 0 2024-03-24T00:49:05.232 INFO:tasks.workunit.client.0.smithi038.stdout:8/789: dread d0/d45/d37/d5f/d68/f85 [0,4194304] 0 2024-03-24T00:49:05.232 INFO:tasks.workunit.client.0.smithi038.stdout:8/790: fdatasync d0/d45/d66/f97 0 2024-03-24T00:49:05.233 INFO:tasks.workunit.client.0.smithi038.stdout:8/791: chown d0/d15/fcf 566063586 1 2024-03-24T00:49:05.233 INFO:tasks.workunit.client.0.smithi038.stdout:8/792: read - d0/d1d/d27/d6b/fa1 zero size 2024-03-24T00:49:05.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.232+0000 7f0544c00700 1 -- 172.21.15.38:0/3827082009 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) v1 ==== 91370+0+0 (secure 0 0 0) 0x7f053400f5d0 con 0x7f0548101cc0 2024-03-24T00:49:05.233 INFO:tasks.workunit.client.0.smithi038.stdout:1/813: creat da/dd/d2c/d5a/d101/f10e x:0 0 0 2024-03-24T00:49:05.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.233+0000 7f0544c00700 1 --2- 172.21.15.38:0/3827082009 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f052c06d650 0x7f052c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:05.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.233+0000 7f0544c00700 1 -- 172.21.15.38:0/3827082009 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f053408e2e0 con 0x7f0548101cc0 2024-03-24T00:49:05.235 INFO:tasks.workunit.client.1.smithi088.stdout:7/738: dread - d3/d32/d34/d9c/d4b/d74/f82 zero size 2024-03-24T00:49:05.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.234+0000 7f0547e00700 1 --2- 172.21.15.38:0/3827082009 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f052c06d650 0x7f052c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:05.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.237+0000 7f0547e00700 1 --2- 172.21.15.38:0/3827082009 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f052c06d650 0x7f052c06fb10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f05481a67c0 tx=0x7f0538005f90 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:05.242 INFO:tasks.workunit.client.1.smithi088.stdout:0/821: mkdir d9/d26/d108 0 2024-03-24T00:49:05.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.243+0000 7f0544c00700 1 -- 172.21.15.38:0/3827082009 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f0534057bc0 con 0x7f0548101cc0 2024-03-24T00:49:05.246 INFO:tasks.workunit.client.1.smithi088.stdout:4/785: link d5/d7/d72/d106/f108 d5/d13/d18/f111 0 2024-03-24T00:49:05.249 INFO:tasks.workunit.client.1.smithi088.stdout:1/794: dwrite d6/d23/f29 [4194304,4194304] 0 2024-03-24T00:49:05.249 INFO:tasks.workunit.client.1.smithi088.stdout:1/795: readlink d6/d23/l72 0 2024-03-24T00:49:05.249 INFO:tasks.workunit.client.1.smithi088.stdout:1/796: write d6/f2a [2353833,20126] 0 2024-03-24T00:49:05.249 INFO:tasks.workunit.client.0.smithi038.stdout:2/682: write d4/d27/f91 [1789269,114782] 0 2024-03-24T00:49:05.251 INFO:tasks.workunit.client.1.smithi088.stdout:2/756: truncate db/f20 4747268 0 2024-03-24T00:49:05.254 INFO:tasks.workunit.client.1.smithi088.stdout:5/843: creat d1/d10/d79/dbc/f119 x:0 0 0 2024-03-24T00:49:05.254 INFO:tasks.workunit.client.0.smithi038.stdout:6/831: dwrite d1/d37/d84/f108 [0,4194304] 0 2024-03-24T00:49:05.258 INFO:tasks.workunit.client.1.smithi088.stdout:7/739: dread d3/d32/d34/d9c/d4b/d73/fe8 [0,4194304] 0 2024-03-24T00:49:05.261 INFO:tasks.workunit.client.1.smithi088.stdout:3/758: dread d3/d6/d19/d2a/d5f/f65 [0,4194304] 0 2024-03-24T00:49:05.263 INFO:tasks.workunit.client.1.smithi088.stdout:9/785: dwrite d1/d10/d11/d23/f70 [0,4194304] 0 2024-03-24T00:49:05.263 INFO:tasks.workunit.client.1.smithi088.stdout:9/786: chown d1/d10/c89 5525 1 2024-03-24T00:49:05.264 INFO:tasks.workunit.client.1.smithi088.stdout:8/778: symlink d9/dc/l107 0 2024-03-24T00:49:05.264 INFO:tasks.workunit.client.1.smithi088.stdout:8/779: dread - d9/dc/d11/d24/d29/da9/fab zero size 2024-03-24T00:49:05.264 INFO:tasks.workunit.client.1.smithi088.stdout:8/780: chown d9/dc/d11/d24/d43/dd8 115833 1 2024-03-24T00:49:05.264 INFO:tasks.workunit.client.1.smithi088.stdout:8/781: write d9/dc/d11/d24/fff [907758,110070] 0 2024-03-24T00:49:05.265 INFO:tasks.workunit.client.1.smithi088.stdout:8/782: truncate d9/dc/d11/d24/d29/d5b/d8b/f96 4257703 0 2024-03-24T00:49:05.265 INFO:tasks.workunit.client.1.smithi088.stdout:8/783: chown d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/c64 2025948 1 2024-03-24T00:49:05.268 INFO:tasks.workunit.client.1.smithi088.stdout:0/822: read d9/dc/de/d19/fbf [879514,24567] 0 2024-03-24T00:49:05.269 INFO:tasks.workunit.client.1.smithi088.stdout:6/850: truncate db/d1d/f58 59664 0 2024-03-24T00:49:05.269 INFO:tasks.workunit.client.1.smithi088.stdout:6/851: truncate db/d1d/d50/fef 334178 0 2024-03-24T00:49:05.271 INFO:tasks.workunit.client.0.smithi038.stdout:4/716: dwrite da/d12/d37/dd/d15/d4c/f6a [0,4194304] 0 2024-03-24T00:49:05.272 INFO:tasks.workunit.client.0.smithi038.stdout:5/708: mknod d6/d16/ce6 0 2024-03-24T00:49:05.272 INFO:tasks.workunit.client.0.smithi038.stdout:5/709: fdatasync d6/d19/d20/d42/f91 0 2024-03-24T00:49:05.275 INFO:tasks.workunit.client.1.smithi088.stdout:4/786: write d5/fb5 [1254065,98917] 0 2024-03-24T00:49:05.279 INFO:tasks.workunit.client.1.smithi088.stdout:7/740: dread d3/d32/f42 [0,4194304] 0 2024-03-24T00:49:05.283 INFO:tasks.workunit.client.0.smithi038.stdout:9/837: mkdir d12/d28/d3a/d8f/d10c 0 2024-03-24T00:49:05.283 INFO:tasks.workunit.client.0.smithi038.stdout:9/838: fdatasync d12/d28/da8/fb1 0 2024-03-24T00:49:05.284 INFO:tasks.workunit.client.1.smithi088.stdout:1/797: link d6/d27/la0 d6/d27/db0/l104 0 2024-03-24T00:49:05.284 INFO:tasks.workunit.client.1.smithi088.stdout:1/798: fsync d6/d23/daf/d95/d5b/d5d/d61/f7e 0 2024-03-24T00:49:05.285 INFO:tasks.workunit.client.0.smithi038.stdout:2/683: read d4/d18/d26/f85 [236675,58889] 0 2024-03-24T00:49:05.286 INFO:tasks.workunit.client.0.smithi038.stdout:2/684: creat d4/d18/fea x:0 0 0 2024-03-24T00:49:05.286 INFO:tasks.workunit.client.0.smithi038.stdout:2/685: stat d4/d27/d2f 0 2024-03-24T00:49:05.286 INFO:tasks.workunit.client.0.smithi038.stdout:8/793: mkdir d0/d1d/d27/d4d/d81/d109 0 2024-03-24T00:49:05.288 INFO:tasks.workunit.client.1.smithi088.stdout:2/757: chown db/dd/d96/d4e/f6e 8673 1 2024-03-24T00:49:05.294 INFO:tasks.workunit.client.0.smithi038.stdout:0/685: rename d0/d2d to d0/d12/d15/d58/d64/da7/dbb/dd4/dea 0 2024-03-24T00:49:05.295 INFO:tasks.workunit.client.1.smithi088.stdout:5/844: symlink d1/d10/db6/d10b/l11a 0 2024-03-24T00:49:05.295 INFO:tasks.workunit.client.1.smithi088.stdout:5/845: getdents d1/d43 0 2024-03-24T00:49:05.295 INFO:tasks.workunit.client.1.smithi088.stdout:5/846: stat d1/dd/l12 0 2024-03-24T00:49:05.295 INFO:tasks.workunit.client.1.smithi088.stdout:5/847: fsync d1/d19/f6c 0 2024-03-24T00:49:05.300 INFO:tasks.workunit.client.1.smithi088.stdout:2/758: dread db/d21/d35/f3d [0,4194304] 0 2024-03-24T00:49:05.301 INFO:tasks.workunit.client.1.smithi088.stdout:9/787: truncate d1/d10/d11/fc3 671077 0 2024-03-24T00:49:05.301 INFO:tasks.workunit.client.1.smithi088.stdout:9/788: readlink d1/d10/d11/ld2 0 2024-03-24T00:49:05.302 INFO:tasks.workunit.client.1.smithi088.stdout:9/789: write d1/d10/d11/d23/f8b [993646,7898] 0 2024-03-24T00:49:05.313 INFO:tasks.workunit.client.0.smithi038.stdout:6/832: truncate d1/da/d1b/f26 4117506 0 2024-03-24T00:49:05.315 INFO:tasks.workunit.client.1.smithi088.stdout:6/852: dread db/dd/d61/d6d/d74/d88/f9b [0,4194304] 0 2024-03-24T00:49:05.316 INFO:tasks.workunit.client.0.smithi038.stdout:4/717: mkdir da/d12/d24/d5f/df8 0 2024-03-24T00:49:05.319 INFO:tasks.workunit.client.1.smithi088.stdout:4/787: getdents d5/d13/d18/d6e 0 2024-03-24T00:49:05.319 INFO:tasks.workunit.client.0.smithi038.stdout:0/686: dread d0/d12/d15/d58/d64/f7d [0,4194304] 0 2024-03-24T00:49:05.322 INFO:tasks.workunit.client.0.smithi038.stdout:5/710: unlink d6/d19/d20/d64/d6b/cb1 0 2024-03-24T00:49:05.327 INFO:tasks.workunit.client.0.smithi038.stdout:9/839: symlink d12/d28/d6c/dbc/l10d 0 2024-03-24T00:49:05.328 INFO:tasks.workunit.client.1.smithi088.stdout:7/741: mknod d3/d32/d76/cef 0 2024-03-24T00:49:05.328 INFO:tasks.workunit.client.1.smithi088.stdout:7/742: write d3/d32/d34/d9c/d4b/d73/dab/dd8/fd7 [725279,7394] 0 2024-03-24T00:49:05.330 INFO:tasks.workunit.client.0.smithi038.stdout:7/699: sync 2024-03-24T00:49:05.330 INFO:tasks.workunit.client.0.smithi038.stdout:3/742: sync 2024-03-24T00:49:05.333 INFO:tasks.workunit.client.0.smithi038.stdout:2/686: unlink d4/d18/d26/d37/d69/f6c 0 2024-03-24T00:49:05.333 INFO:tasks.workunit.client.0.smithi038.stdout:2/687: creat d4/d27/d54/feb x:0 0 0 2024-03-24T00:49:05.335 INFO:tasks.workunit.client.0.smithi038.stdout:8/794: symlink d0/d1d/d27/d6b/dfc/d9e/l10a 0 2024-03-24T00:49:05.335 INFO:tasks.workunit.client.0.smithi038.stdout:8/795: fdatasync d0/d10/fd3 0 2024-03-24T00:49:05.337 INFO:tasks.workunit.client.0.smithi038.stdout:1/814: rename da/dd/d2c/d5a/d74/f88 to da/ddc/f10f 0 2024-03-24T00:49:05.337 INFO:tasks.workunit.client.0.smithi038.stdout:1/815: write da/dd/d7a/d82/d98/d9e/fb0 [4409206,56694] 0 2024-03-24T00:49:05.343 INFO:tasks.workunit.client.1.smithi088.stdout:1/799: dwrite d6/d23/dcb/de7/fea [0,4194304] 0 2024-03-24T00:49:05.343 INFO:tasks.workunit.client.1.smithi088.stdout:1/800: truncate d6/d27/fb3 827495 0 2024-03-24T00:49:05.346 INFO:tasks.workunit.client.1.smithi088.stdout:7/743: write d3/d31/f3c [1170536,39920] 0 2024-03-24T00:49:05.350 INFO:tasks.workunit.client.1.smithi088.stdout:0/823: dwrite d9/d26/d8f/da1/d62/d9b/db0/fb2 [0,4194304] 0 2024-03-24T00:49:05.351 INFO:tasks.workunit.client.0.smithi038.stdout:6/833: link d1/da/d1b/l53 d1/da/d1b/d25/d49/d72/l111 0 2024-03-24T00:49:05.353 INFO:tasks.workunit.client.0.smithi038.stdout:4/718: creat da/d12/d24/d5f/df8/ff9 x:0 0 0 2024-03-24T00:49:05.355 INFO:tasks.workunit.client.1.smithi088.stdout:5/848: symlink d1/dd/d6f/da1/d3f/l11b 0 2024-03-24T00:49:05.357 INFO:tasks.workunit.client.0.smithi038.stdout:5/711: unlink d6/d60/c71 0 2024-03-24T00:49:05.358 INFO:tasks.workunit.client.0.smithi038.stdout:5/712: chown d6/d19/d20/d70/c89 95103 1 2024-03-24T00:49:05.359 INFO:tasks.workunit.client.1.smithi088.stdout:7/744: dread d3/d32/d34/d9c/d4b/d73/fe8 [0,4194304] 0 2024-03-24T00:49:05.361 INFO:tasks.workunit.client.1.smithi088.stdout:2/759: creat db/dd4/fec x:0 0 0 2024-03-24T00:49:05.364 INFO:tasks.workunit.client.0.smithi038.stdout:9/840: rmdir d12/d55/d56 39 2024-03-24T00:49:05.370 INFO:tasks.workunit.client.1.smithi088.stdout:9/790: creat d1/d22/d27/d39/ffd x:0 0 0 2024-03-24T00:49:05.371 INFO:tasks.workunit.client.0.smithi038.stdout:7/700: stat d2/d9/da/d23/d2b/d5b/l64 0 2024-03-24T00:49:05.377 INFO:tasks.workunit.client.0.smithi038.stdout:3/743: link d3/d9/d77/f85 d3/d9/dc/da6/db5/d19/d82/d89/da8/f100 0 2024-03-24T00:49:05.381 INFO:tasks.workunit.client.1.smithi088.stdout:8/784: getdents d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d 0 2024-03-24T00:49:05.389 INFO:tasks.workunit.client.0.smithi038.stdout:2/688: mkdir d4/d27/d54/dec 0 2024-03-24T00:49:05.394 INFO:tasks.workunit.client.0.smithi038.stdout:1/816: dread da/dd/d22/f25 [0,4194304] 0 2024-03-24T00:49:05.394 INFO:tasks.workunit.client.0.smithi038.stdout:1/817: write da/dd/d22/f25 [3220618,76876] 0 2024-03-24T00:49:05.396 INFO:tasks.workunit.client.0.smithi038.stdout:8/796: unlink d0/d10/c25 0 2024-03-24T00:49:05.396 INFO:tasks.workunit.client.0.smithi038.stdout:8/797: fsync d0/d10/fab 0 2024-03-24T00:49:05.400 INFO:tasks.workunit.client.1.smithi088.stdout:4/788: unlink d5/d7/l7a 0 2024-03-24T00:49:05.404 INFO:tasks.workunit.client.0.smithi038.stdout:6/834: unlink d1/fe8 0 2024-03-24T00:49:05.404 INFO:tasks.workunit.client.0.smithi038.stdout:4/719: rename da/d12/d29/d5b/d73/db4/cf6 to da/d12/d29/d5b/d73/cfa 0 2024-03-24T00:49:05.409 INFO:tasks.workunit.client.0.smithi038.stdout:5/713: truncate d6/d19/d20/d42/d43/d68/d6e/f7f 2940861 0 2024-03-24T00:49:05.409 INFO:tasks.workunit.client.0.smithi038.stdout:5/714: stat d6/d16/d67 0 2024-03-24T00:49:05.409 INFO:tasks.workunit.client.0.smithi038.stdout:5/715: fsync d6/d16/d21/d2d/fd2 0 2024-03-24T00:49:05.410 INFO:tasks.workunit.client.0.smithi038.stdout:5/716: chown d6/lc9 2201388 1 2024-03-24T00:49:05.410 INFO:tasks.workunit.client.1.smithi088.stdout:8/785: write d9/f37 [2128049,27346] 0 2024-03-24T00:49:05.412 INFO:tasks.workunit.client.1.smithi088.stdout:6/853: write db/dd/d61/fc4 [4230489,41430] 0 2024-03-24T00:49:05.412 INFO:tasks.workunit.client.1.smithi088.stdout:6/854: read db/dd/d61/d6d/d74/fc0 [2728692,86474] 0 2024-03-24T00:49:05.414 INFO:tasks.workunit.client.0.smithi038.stdout:0/687: dwrite d0/f78 [0,4194304] 0 2024-03-24T00:49:05.414 INFO:tasks.workunit.client.0.smithi038.stdout:2/689: mknod d4/d27/d54/ced 0 2024-03-24T00:49:05.422 INFO:tasks.workunit.client.0.smithi038.stdout:8/798: creat d0/d45/d37/d5f/db7/f10b x:0 0 0 2024-03-24T00:49:05.422 INFO:tasks.workunit.client.0.smithi038.stdout:1/818: mknod da/dd/d2c/d41/d60/c110 0 2024-03-24T00:49:05.427 INFO:tasks.workunit.client.0.smithi038.stdout:3/744: rename d3/d9/dc/da6/cf1 to d3/d9/def/c101 0 2024-03-24T00:49:05.427 INFO:tasks.workunit.client.0.smithi038.stdout:3/745: chown d3/d30/ded 2 1 2024-03-24T00:49:05.433 INFO:tasks.workunit.client.1.smithi088.stdout:0/824: truncate d9/d26/d8f/da1/d62/d9b/fbb 276777 0 2024-03-24T00:49:05.434 INFO:tasks.workunit.client.0.smithi038.stdout:5/717: mkdir d6/d19/d20/d42/d43/d68/dc8/de7 0 2024-03-24T00:49:05.435 INFO:tasks.workunit.client.1.smithi088.stdout:5/849: creat d1/dd/d6f/da1/d3f/f11c x:0 0 0 2024-03-24T00:49:05.441 INFO:tasks.workunit.client.0.smithi038.stdout:9/841: dread d12/d55/f9b [0,4194304] 0 2024-03-24T00:49:05.442 INFO:tasks.workunit.client.0.smithi038.stdout:0/688: creat d0/d14/de4/feb x:0 0 0 2024-03-24T00:49:05.442 INFO:tasks.workunit.client.1.smithi088.stdout:7/745: mknod d3/d32/d34/cf0 0 2024-03-24T00:49:05.443 INFO:tasks.workunit.client.1.smithi088.stdout:2/760: creat db/dd/fed x:0 0 0 2024-03-24T00:49:05.443 INFO:tasks.workunit.client.1.smithi088.stdout:2/761: read db/d21/d35/f5a [676271,6846] 0 2024-03-24T00:49:05.446 INFO:tasks.workunit.client.0.smithi038.stdout:3/746: dread d3/d9/dc/da6/db5/f53 [0,4194304] 0 2024-03-24T00:49:05.446 INFO:tasks.workunit.client.1.smithi088.stdout:9/791: creat d1/d10/d9b/ffe x:0 0 0 2024-03-24T00:49:05.448 INFO:tasks.workunit.client.0.smithi038.stdout:1/819: mkdir da/d3c/d86/d111 0 2024-03-24T00:49:05.448 INFO:tasks.workunit.client.0.smithi038.stdout:1/820: chown da/dd/d2c/d40/c69 2 1 2024-03-24T00:49:05.455 INFO:tasks.workunit.client.1.smithi088.stdout:4/789: creat d5/d7/f112 x:0 0 0 2024-03-24T00:49:05.462 INFO:tasks.workunit.client.0.smithi038.stdout:6/835: rename d1/da/d1b/l42 to d1/da/d1b/dfd/l112 0 2024-03-24T00:49:05.466 INFO:tasks.workunit.client.0.smithi038.stdout:4/720: symlink da/d12/d37/dd/d15/d6d/lfb 0 2024-03-24T00:49:05.466 INFO:tasks.workunit.client.0.smithi038.stdout:4/721: write da/d12/d24/d5f/fd4 [1729273,80051] 0 2024-03-24T00:49:05.467 INFO:tasks.workunit.client.1.smithi088.stdout:1/801: link d6/fab d6/d23/daf/d95/d5b/f105 0 2024-03-24T00:49:05.467 INFO:tasks.workunit.client.1.smithi088.stdout:1/802: stat d6/d27/d67/d88 0 2024-03-24T00:49:05.473 INFO:tasks.workunit.client.1.smithi088.stdout:8/786: rename d9/dc/d11/dd2/ldf to d9/dc/d11/d24/d29/d5b/db9/l108 0 2024-03-24T00:49:05.479 INFO:tasks.workunit.client.0.smithi038.stdout:5/718: truncate d6/d16/d21/f5f 2440791 0 2024-03-24T00:49:05.479 INFO:tasks.workunit.client.0.smithi038.stdout:5/719: fsync d6/d19/d20/d42/f91 0 2024-03-24T00:49:05.485 INFO:tasks.workunit.client.1.smithi088.stdout:1/803: dread d6/d27/db0/de3/fec [0,4194304] 0 2024-03-24T00:49:05.487 INFO:tasks.workunit.client.0.smithi038.stdout:4/722: write da/d12/d24/d68/d6c/f79 [2845629,36032] 0 2024-03-24T00:49:05.489 INFO:tasks.workunit.client.1.smithi088.stdout:6/855: mknod db/d1d/d23/c10e 0 2024-03-24T00:49:05.492 INFO:tasks.workunit.client.1.smithi088.stdout:3/759: sync 2024-03-24T00:49:05.492 INFO:tasks.workunit.client.1.smithi088.stdout:3/760: write d3/d6/dca/fdc [422089,32194] 0 2024-03-24T00:49:05.493 INFO:tasks.workunit.client.0.smithi038.stdout:8/799: dwrite d0/d1d/d20/fdd [0,4194304] 0 2024-03-24T00:49:05.493 INFO:tasks.workunit.client.0.smithi038.stdout:7/701: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/f35 [0,4194304] 0 2024-03-24T00:49:05.493 INFO:tasks.workunit.client.0.smithi038.stdout:8/800: dread - d0/d45/d37/d5f/db7/f10b zero size 2024-03-24T00:49:05.498 INFO:tasks.workunit.client.0.smithi038.stdout:3/747: unlink d3/d9/dc/da6/db5/lb9 0 2024-03-24T00:49:05.505 INFO:tasks.workunit.client.1.smithi088.stdout:2/762: rename db/dd/d96/d4e/d8f/d9c/fc1 to db/dd/d96/d4e/d8f/d9c/fee 0 2024-03-24T00:49:05.507 INFO:tasks.workunit.client.1.smithi088.stdout:0/825: dwrite d9/dc/de/d19/f93 [0,4194304] 0 2024-03-24T00:49:05.511 INFO:tasks.workunit.client.0.smithi038.stdout:6/836: mkdir d1/d37/d84/df5/d113 0 2024-03-24T00:49:05.513 INFO:tasks.workunit.client.1.smithi088.stdout:4/790: mknod d5/d7/d15/d74/c113 0 2024-03-24T00:49:05.513 INFO:tasks.workunit.client.1.smithi088.stdout:6/856: dread db/d1d/d50/d90/d56/f51 [0,4194304] 0 2024-03-24T00:49:05.513 INFO:tasks.workunit.client.1.smithi088.stdout:6/857: truncate db/dd/d61/d6d/d74/d88/fda 284788 0 2024-03-24T00:49:05.519 INFO:tasks.workunit.client.0.smithi038.stdout:9/842: rename d12/d81/d84/c103 to d12/d28/d3a/d8f/d10c/c10e 0 2024-03-24T00:49:05.519 INFO:tasks.workunit.client.0.smithi038.stdout:9/843: fsync d12/d28/da8/fb1 0 2024-03-24T00:49:05.519 INFO:tasks.workunit.client.0.smithi038.stdout:9/844: fsync d12/d81/d84/d88/fd4 0 2024-03-24T00:49:05.519 INFO:tasks.workunit.client.0.smithi038.stdout:9/845: creat d12/d1a/d5f/f10f x:0 0 0 2024-03-24T00:49:05.519 INFO:tasks.workunit.client.0.smithi038.stdout:9/846: write d12/d41/d57/dd3/dfe/f92 [320623,25822] 0 2024-03-24T00:49:05.525 INFO:tasks.workunit.client.1.smithi088.stdout:8/787: symlink d9/dc/d11/d24/d43/d4a/dee/l109 0 2024-03-24T00:49:05.527 INFO:tasks.workunit.client.0.smithi038.stdout:5/720: mkdir d6/d19/d20/d64/d6b/de8 0 2024-03-24T00:49:05.531 INFO:tasks.workunit.client.1.smithi088.stdout:3/761: rename d3/d6/dca/f93 to d3/de/d6f/dcc/ff0 0 2024-03-24T00:49:05.536 INFO:tasks.workunit.client.1.smithi088.stdout:1/804: dwrite d6/d27/db0/fbc [0,4194304] 0 2024-03-24T00:49:05.538 INFO:tasks.workunit.client.0.smithi038.stdout:4/723: link da/d12/d24/d5f/f97 da/d12/d24/d68/d6c/da6/de7/ffc 0 2024-03-24T00:49:05.543 INFO:tasks.workunit.client.1.smithi088.stdout:2/763: symlink db/dd/d96/d99/dc3/lef 0 2024-03-24T00:49:05.544 INFO:tasks.workunit.client.1.smithi088.stdout:8/788: write d9/dc/d11/d24/d29/d5b/db9/da5/fb7 [3346884,58688] 0 2024-03-24T00:49:05.544 INFO:tasks.workunit.client.1.smithi088.stdout:8/789: fdatasync d9/d6b/f6c 0 2024-03-24T00:49:05.553 INFO:tasks.workunit.client.0.smithi038.stdout:7/702: mknod d2/d9/da/dc/d4a/c106 0 2024-03-24T00:49:05.556 INFO:tasks.workunit.client.0.smithi038.stdout:8/801: stat d0/d1d/d8c/lf8 0 2024-03-24T00:49:05.556 INFO:tasks.workunit.client.1.smithi088.stdout:9/792: link d1/d10/d77/d8a/cb9 d1/d10/cff 0 2024-03-24T00:49:05.557 INFO:tasks.workunit.client.1.smithi088.stdout:9/793: chown d1/d10/d2c/f75 0 1 2024-03-24T00:49:05.560 INFO:tasks.workunit.client.0.smithi038.stdout:0/689: dwrite d0/d12/f68 [0,4194304] 0 2024-03-24T00:49:05.561 INFO:tasks.workunit.client.1.smithi088.stdout:4/791: mkdir d5/d7/d15/d74/d114 0 2024-03-24T00:49:05.561 INFO:tasks.workunit.client.0.smithi038.stdout:3/748: getdents d3/d9/def 0 2024-03-24T00:49:05.564 INFO:tasks.workunit.client.1.smithi088.stdout:6/858: rename db/d1d/d50/d90/f4d to db/d1d/d50/d90/d55/f10f 0 2024-03-24T00:49:05.568 INFO:tasks.workunit.client.1.smithi088.stdout:3/762: mkdir d3/d6/d58/da4/df1 0 2024-03-24T00:49:05.576 INFO:tasks.workunit.client.0.smithi038.stdout:6/837: rename d1/c28 to d1/da/d73/c114 0 2024-03-24T00:49:05.578 INFO:tasks.workunit.client.1.smithi088.stdout:2/764: getdents db/d21/d73 0 2024-03-24T00:49:05.578 INFO:tasks.workunit.client.1.smithi088.stdout:2/765: stat db/dd/dc9 0 2024-03-24T00:49:05.580 INFO:tasks.workunit.client.0.smithi038.stdout:9/847: stat d12/d55/d56/c5e 0 2024-03-24T00:49:05.582 INFO:tasks.workunit.client.1.smithi088.stdout:1/805: write d6/d23/f29 [3456783,51360] 0 2024-03-24T00:49:05.582 INFO:tasks.workunit.client.1.smithi088.stdout:0/826: dwrite d9/dc/de/d19/d44/d46/d65/f58 [0,4194304] 0 2024-03-24T00:49:05.583 INFO:tasks.workunit.client.1.smithi088.stdout:0/827: chown d9/d26/d8f/da1/d62/d9b/db0 10 1 2024-03-24T00:49:05.585 INFO:tasks.workunit.client.1.smithi088.stdout:4/792: creat d5/d13/d18/d24/d3d/df0/f115 x:0 0 0 2024-03-24T00:49:05.588 INFO:tasks.workunit.client.1.smithi088.stdout:5/850: dwrite d1/dd/d6f/da1/d3f/d88/d8f/d69/f8c [0,4194304] 0 2024-03-24T00:49:05.589 INFO:tasks.workunit.client.0.smithi038.stdout:5/721: mknod d6/d19/d20/d42/d79/ce9 0 2024-03-24T00:49:05.595 INFO:tasks.workunit.client.1.smithi088.stdout:3/763: truncate d3/f87 1961793 0 2024-03-24T00:49:05.595 INFO:tasks.workunit.client.1.smithi088.stdout:2/766: creat db/dd/d96/d99/db2/ff0 x:0 0 0 2024-03-24T00:49:05.596 INFO:tasks.workunit.client.1.smithi088.stdout:3/764: write d3/d1a/d1e/f28 [3962506,98206] 0 2024-03-24T00:49:05.596 INFO:tasks.workunit.client.1.smithi088.stdout:3/765: dread - d3/de/d6f/d85/d49/f99 zero size 2024-03-24T00:49:05.604 INFO:tasks.workunit.client.0.smithi038.stdout:4/724: unlink da/d12/d37/d83/dac/dcf/fe8 0 2024-03-24T00:49:05.612 INFO:tasks.workunit.client.0.smithi038.stdout:7/703: symlink d2/d9/da/dc/l107 0 2024-03-24T00:49:05.617 INFO:tasks.workunit.client.1.smithi088.stdout:4/793: truncate d5/d16/d1a/f3e 1091462 0 2024-03-24T00:49:05.621 INFO:tasks.workunit.client.0.smithi038.stdout:8/802: unlink d0/d45/d37/d5f/db7/d62/dfa/cc6 0 2024-03-24T00:49:05.624 INFO:tasks.workunit.client.1.smithi088.stdout:8/790: dwrite d9/dc/d11/d24/d29/d36/d6a/d7b/fa0 [0,4194304] 0 2024-03-24T00:49:05.626 INFO:tasks.workunit.client.0.smithi038.stdout:9/848: dread d12/d81/d84/d88/fa9 [0,4194304] 0 2024-03-24T00:49:05.632 INFO:tasks.workunit.client.1.smithi088.stdout:5/851: creat d1/d10/db6/d10b/d2f/d57/d9e/f11d x:0 0 0 2024-03-24T00:49:05.632 INFO:tasks.workunit.client.0.smithi038.stdout:0/690: mknod d0/d12/d42/d9d/d28/cec 0 2024-03-24T00:49:05.632 INFO:tasks.workunit.client.0.smithi038.stdout:0/691: chown d0/d12/d15/d58/lc1 85120997 1 2024-03-24T00:49:05.632 INFO:tasks.workunit.client.0.smithi038.stdout:3/749: rmdir d3/d9/dc/da6/db5/d19 39 2024-03-24T00:49:05.635 INFO:tasks.workunit.client.0.smithi038.stdout:4/725: link da/d12/d37/dd/d15/d4c/f7a da/d12/d29/ffd 0 2024-03-24T00:49:05.638 INFO:tasks.workunit.client.0.smithi038.stdout:7/704: unlink d2/d2c/d90/cf5 0 2024-03-24T00:49:05.640 INFO:tasks.workunit.client.1.smithi088.stdout:3/766: rmdir d3/de/d6f/d85/d49/d70 39 2024-03-24T00:49:05.640 INFO:tasks.workunit.client.1.smithi088.stdout:8/791: write d9/dc/d11/d24/d43/d4a/f55 [2015932,9290] 0 2024-03-24T00:49:05.642 INFO:tasks.workunit.client.1.smithi088.stdout:0/828: dwrite d9/dc/de/d19/f7f [0,4194304] 0 2024-03-24T00:49:05.643 INFO:tasks.workunit.client.0.smithi038.stdout:8/803: truncate d0/d1d/d20/f60 1159512 0 2024-03-24T00:49:05.643 INFO:tasks.workunit.client.0.smithi038.stdout:8/804: stat d0/cf 0 2024-03-24T00:49:05.644 INFO:tasks.workunit.client.0.smithi038.stdout:6/838: dwrite d1/d37/d84/fcf [0,4194304] 0 2024-03-24T00:49:05.645 INFO:tasks.workunit.client.0.smithi038.stdout:0/692: creat d0/d14/d88/db8/fed x:0 0 0 2024-03-24T00:49:05.651 INFO:tasks.workunit.client.0.smithi038.stdout:3/750: mkdir d3/dc3/d102 0 2024-03-24T00:49:05.651 INFO:tasks.workunit.client.0.smithi038.stdout:4/726: mknod da/d12/d37/d83/dac/cfe 0 2024-03-24T00:49:05.656 INFO:tasks.workunit.client.1.smithi088.stdout:0/829: dread d9/dc/de/d19/d44/f4a [0,4194304] 0 2024-03-24T00:49:05.656 INFO:tasks.workunit.client.0.smithi038.stdout:7/705: chown d2/d2c/c86 98847 1 2024-03-24T00:49:05.657 INFO:tasks.workunit.client.0.smithi038.stdout:7/706: readlink d2/d9/da/l100 0 2024-03-24T00:49:05.657 INFO:tasks.workunit.client.0.smithi038.stdout:7/707: truncate d2/d9/da/dc/da8/dc5/fe1 446656 0 2024-03-24T00:49:05.657 INFO:tasks.workunit.client.0.smithi038.stdout:7/708: stat d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f66 0 2024-03-24T00:49:05.662 INFO:tasks.workunit.client.0.smithi038.stdout:7/709: read d2/d9/da/d23/d2b/d5b/d72/d92/d33/f96 [78988,97480] 0 2024-03-24T00:49:05.662 INFO:tasks.workunit.client.0.smithi038.stdout:8/805: mkdir d0/d1d/d27/d4d/daf/d10c 0 2024-03-24T00:49:05.662 INFO:tasks.workunit.client.0.smithi038.stdout:8/806: read d0/d45/d66/d6d/fbc [1684966,51432] 0 2024-03-24T00:49:05.663 INFO:tasks.workunit.client.0.smithi038.stdout:7/710: creat d2/d9/da/dc/d93/f108 x:0 0 0 2024-03-24T00:49:05.669 INFO:tasks.workunit.client.0.smithi038.stdout:5/722: dwrite d6/d16/d67/f8f [0,4194304] 0 2024-03-24T00:49:05.669 INFO:tasks.workunit.client.0.smithi038.stdout:5/723: readlink d6/d19/d20/d64/d6b/d77/lae 0 2024-03-24T00:49:05.669 INFO:tasks.workunit.client.0.smithi038.stdout:5/724: readlink d6/d19/d20/d42/d43/d68/d6e/lc0 0 2024-03-24T00:49:05.669 INFO:tasks.workunit.client.0.smithi038.stdout:5/725: fdatasync d6/f18 0 2024-03-24T00:49:05.669 INFO:tasks.workunit.client.0.smithi038.stdout:6/839: creat d1/d37/d84/df5/f115 x:0 0 0 2024-03-24T00:49:05.676 INFO:tasks.workunit.client.0.smithi038.stdout:0/693: rename d0/d12/d15/c5a to d0/d1b/cee 0 2024-03-24T00:49:05.676 INFO:tasks.workunit.client.0.smithi038.stdout:0/694: chown d0/ld8 752300 1 2024-03-24T00:49:05.677 INFO:tasks.workunit.client.0.smithi038.stdout:4/727: rmdir da/d12/d24/d68 39 2024-03-24T00:49:05.683 INFO:tasks.workunit.client.0.smithi038.stdout:6/840: dread d1/f93 [0,4194304] 0 2024-03-24T00:49:05.684 INFO:tasks.workunit.client.1.smithi088.stdout:4/794: write d5/d16/d1a/f3e [1756214,82778] 0 2024-03-24T00:49:05.684 INFO:tasks.workunit.client.0.smithi038.stdout:3/751: rmdir d3/d9/dc/da6 39 2024-03-24T00:49:05.684 INFO:tasks.workunit.client.0.smithi038.stdout:3/752: chown d3/dc3/fcb 321 1 2024-03-24T00:49:05.687 INFO:tasks.workunit.client.1.smithi088.stdout:6/859: rename db/dd/d61/dfb to db/dd/d61/d6d/d74/de5/d110 0 2024-03-24T00:49:05.691 INFO:tasks.workunit.client.1.smithi088.stdout:5/852: creat d1/d10/db6/d10b/d2f/d57/d9e/dd0/f11e x:0 0 0 2024-03-24T00:49:05.696 INFO:tasks.workunit.client.1.smithi088.stdout:3/767: creat d3/d6/d19/d2a/dc2/dd5/dd6/ff2 x:0 0 0 2024-03-24T00:49:05.698 INFO:tasks.workunit.client.1.smithi088.stdout:8/792: truncate d9/dc/d11/d24/d29/d5b/d8b/dd3/fbd 1235668 0 2024-03-24T00:49:05.710 INFO:tasks.workunit.client.1.smithi088.stdout:0/830: creat d9/dc/de/d19/d44/d46/d65/f109 x:0 0 0 2024-03-24T00:49:05.716 INFO:tasks.workunit.client.0.smithi038.stdout:5/726: stat d6/d19/d20/d42/lba 0 2024-03-24T00:49:05.718 INFO:tasks.workunit.client.0.smithi038.stdout:0/695: link d0/d12/d42/lc2 d0/d12/d42/d83/lef 0 2024-03-24T00:49:05.722 INFO:tasks.workunit.client.0.smithi038.stdout:8/807: rename d0/d15/cbf to d0/d45/c10d 0 2024-03-24T00:49:05.724 INFO:tasks.workunit.client.1.smithi088.stdout:1/806: rename d6/d23/daf/d95/d5b/d5d/d61/fc6 to d6/d23/daf/d95/d5b/d93/f106 0 2024-03-24T00:49:05.724 INFO:tasks.workunit.client.0.smithi038.stdout:4/728: mknod da/d12/cff 0 2024-03-24T00:49:05.733 INFO:tasks.workunit.client.1.smithi088.stdout:6/860: link db/dd/d61/d6d/d74/df0/fcf db/dd/d87/ddf/f111 0 2024-03-24T00:49:05.737 INFO:tasks.workunit.client.0.smithi038.stdout:6/841: mknod d1/da/d1b/d25/c116 0 2024-03-24T00:49:05.737 INFO:tasks.workunit.client.0.smithi038.stdout:6/842: readlink d1/da/d1b/d1d/d3c/l88 0 2024-03-24T00:49:05.737 INFO:tasks.workunit.client.0.smithi038.stdout:6/843: read - d1/da/d73/ffe zero size 2024-03-24T00:49:05.738 INFO:tasks.workunit.client.1.smithi088.stdout:5/853: mknod d1/d10/db6/d10b/d2f/d57/c11f 0 2024-03-24T00:49:05.740 INFO:tasks.workunit.client.0.smithi038.stdout:3/753: creat d3/d9/dc/da6/db5/d19/d82/d89/da8/f103 x:0 0 0 2024-03-24T00:49:05.746 INFO:tasks.workunit.client.0.smithi038.stdout:7/711: mkdir d2/d2c/d9a/d109 0 2024-03-24T00:49:05.747 INFO:tasks.workunit.client.1.smithi088.stdout:8/793: creat d9/dc/d11/d80/db0/f10a x:0 0 0 2024-03-24T00:49:05.757 INFO:tasks.workunit.client.1.smithi088.stdout:6/861: dread db/dd/f10c [0,4194304] 0 2024-03-24T00:49:05.757 INFO:tasks.workunit.client.1.smithi088.stdout:1/807: mknod d6/d23/dcb/c107 0 2024-03-24T00:49:05.758 INFO:tasks.workunit.client.1.smithi088.stdout:1/808: fdatasync d6/dc2/fc4 0 2024-03-24T00:49:05.760 INFO:tasks.workunit.client.1.smithi088.stdout:4/795: dwrite d5/d13/d18/d24/d3d/f77 [0,4194304] 0 2024-03-24T00:49:05.766 INFO:tasks.workunit.client.0.smithi038.stdout:1/821: sync 2024-03-24T00:49:05.767 INFO:tasks.workunit.client.0.smithi038.stdout:2/690: sync 2024-03-24T00:49:05.767 INFO:tasks.workunit.client.0.smithi038.stdout:2/691: readlink d4/d27/le6 0 2024-03-24T00:49:05.767 INFO:tasks.workunit.client.0.smithi038.stdout:8/808: symlink d0/d45/d37/d5f/db7/d62/l10e 0 2024-03-24T00:49:05.772 INFO:tasks.workunit.client.0.smithi038.stdout:3/754: creat d3/d30/d5d/d7c/dbc/f104 x:0 0 0 2024-03-24T00:49:05.772 INFO:tasks.workunit.client.0.smithi038.stdout:0/696: creat d0/d12/d42/d5e/ff0 x:0 0 0 2024-03-24T00:49:05.778 INFO:tasks.workunit.client.0.smithi038.stdout:0/697: read d0/d1b/f20 [4217886,129498] 0 2024-03-24T00:49:05.780 INFO:tasks.workunit.client.1.smithi088.stdout:7/746: sync 2024-03-24T00:49:05.782 INFO:tasks.workunit.client.0.smithi038.stdout:1/822: unlink da/dd/d7a/d82/d98/f1f 0 2024-03-24T00:49:05.786 INFO:tasks.workunit.client.1.smithi088.stdout:5/854: link d1/dd/d6f/da1/d3f/d88/d8f/d69/f85 d1/d10/db6/d10b/d2f/d57/d9e/dd0/f120 0 2024-03-24T00:49:05.791 INFO:tasks.workunit.client.1.smithi088.stdout:2/767: sync 2024-03-24T00:49:05.791 INFO:tasks.workunit.client.1.smithi088.stdout:9/794: sync 2024-03-24T00:49:05.794 INFO:tasks.workunit.client.1.smithi088.stdout:8/794: rename d9/dc/d11/d24/d29/d5b/d6f/f83 to d9/dc/d11/d24/d29/d39/d84/f10b 0 2024-03-24T00:49:05.794 INFO:tasks.workunit.client.1.smithi088.stdout:8/795: chown d9/dc/d11/d24/d29/d5b/d6f/l7e 59221 1 2024-03-24T00:49:05.794 INFO:tasks.workunit.client.0.smithi038.stdout:2/692: mknod d4/d27/d54/d53/cee 0 2024-03-24T00:49:05.802 INFO:tasks.workunit.client.1.smithi088.stdout:3/768: dwrite d3/d1a/fe4 [0,4194304] 0 2024-03-24T00:49:05.803 INFO:tasks.workunit.client.0.smithi038.stdout:3/755: unlink d3/d30/d5d/d7c/dbc/cbe 0 2024-03-24T00:49:05.804 INFO:tasks.workunit.client.0.smithi038.stdout:3/756: stat d3/d9/dc/da6/db5/d19/l61 0 2024-03-24T00:49:05.807 INFO:tasks.workunit.client.1.smithi088.stdout:4/796: dread d5/d16/f29 [0,4194304] 0 2024-03-24T00:49:05.807 INFO:tasks.workunit.client.1.smithi088.stdout:4/797: dread - d5/d16/d94/da6/ffa zero size 2024-03-24T00:49:05.807 INFO:tasks.workunit.client.0.smithi038.stdout:0/698: mkdir d0/d1b/df1 0 2024-03-24T00:49:05.807 INFO:tasks.workunit.client.0.smithi038.stdout:0/699: read - d0/d12/d42/fdd zero size 2024-03-24T00:49:05.808 INFO:tasks.workunit.client.0.smithi038.stdout:0/700: creat d0/d14/d88/db8/ff2 x:0 0 0 2024-03-24T00:49:05.808 INFO:tasks.workunit.client.0.smithi038.stdout:0/701: dread - d0/d12/d15/d1a/fdf zero size 2024-03-24T00:49:05.808 INFO:tasks.workunit.client.0.smithi038.stdout:0/702: getdents d0/daf/dd9 0 2024-03-24T00:49:05.808 INFO:tasks.workunit.client.1.smithi088.stdout:1/809: truncate d6/d23/f34 1156573 0 2024-03-24T00:49:05.812 INFO:tasks.workunit.client.0.smithi038.stdout:6/844: write d1/d37/d84/fcf [335713,49498] 0 2024-03-24T00:49:05.812 INFO:tasks.workunit.client.0.smithi038.stdout:6/845: write d1/f5 [4620894,112902] 0 2024-03-24T00:49:05.815 INFO:tasks.workunit.client.1.smithi088.stdout:0/831: dwrite d9/dc/de/d7e/f89 [0,4194304] 0 2024-03-24T00:49:05.815 INFO:tasks.workunit.client.1.smithi088.stdout:0/832: readlink d9/dc/de/l2d 0 2024-03-24T00:49:05.815 INFO:tasks.workunit.client.0.smithi038.stdout:9/849: dwrite d12/d28/da8/fb1 [0,4194304] 0 2024-03-24T00:49:05.816 INFO:tasks.workunit.client.0.smithi038.stdout:4/729: dwrite da/d12/d24/d5f/d8a/fe0 [0,4194304] 0 2024-03-24T00:49:05.817 INFO:tasks.workunit.client.0.smithi038.stdout:2/693: dread d4/d27/d2f/d30/f8d [0,4194304] 0 2024-03-24T00:49:05.822 INFO:tasks.workunit.client.1.smithi088.stdout:2/768: symlink db/dd/d96/lf1 0 2024-03-24T00:49:05.823 INFO:tasks.workunit.client.1.smithi088.stdout:6/862: dwrite db/d3d/d83/f9f [0,4194304] 0 2024-03-24T00:49:05.823 INFO:tasks.workunit.client.1.smithi088.stdout:6/863: stat db/dc5/dc8/d4b/c63 0 2024-03-24T00:49:05.823 INFO:tasks.workunit.client.1.smithi088.stdout:6/864: truncate db/dd/d61/fdb 838144 0 2024-03-24T00:49:05.824 INFO:tasks.workunit.client.0.smithi038.stdout:6/846: rename d1/da/d1b/d25/d49/d72/l7a to d1/da/d1b/l117 0 2024-03-24T00:49:05.824 INFO:tasks.workunit.client.0.smithi038.stdout:3/757: mkdir d3/d9/dc/da6/db5/d19/d82/d89/da8/d105 0 2024-03-24T00:49:05.839 INFO:tasks.workunit.client.1.smithi088.stdout:9/795: truncate d1/d10/d29/fa6 602734 0 2024-03-24T00:49:05.839 INFO:tasks.workunit.client.1.smithi088.stdout:9/796: fsync d1/d10/f34 0 2024-03-24T00:49:05.843 INFO:tasks.workunit.client.1.smithi088.stdout:5/855: rename d1/dd/d6f/da1/d3f/d88/d8f/d69/ld4 to d1/d10/db6/d10b/l121 0 2024-03-24T00:49:05.845 INFO:tasks.workunit.client.1.smithi088.stdout:8/796: getdents d9/dc/dce 0 2024-03-24T00:49:05.845 INFO:tasks.workunit.client.0.smithi038.stdout:3/758: dread d3/d9/dc/f81 [0,4194304] 0 2024-03-24T00:49:05.849 INFO:tasks.workunit.client.0.smithi038.stdout:7/712: dwrite d2/d2c/f74 [0,4194304] 0 2024-03-24T00:49:05.851 INFO:tasks.workunit.client.1.smithi088.stdout:9/797: dread d1/d10/d2c/f75 [0,4194304] 0 2024-03-24T00:49:05.851 INFO:tasks.workunit.client.1.smithi088.stdout:9/798: fdatasync d1/d10/d9b/dab/db8/da7/fd6 0 2024-03-24T00:49:05.854 INFO:tasks.workunit.client.0.smithi038.stdout:9/850: mknod d12/d28/d71/d99/c110 0 2024-03-24T00:49:05.854 INFO:tasks.workunit.client.0.smithi038.stdout:9/851: fdatasync d12/d41/fd0 0 2024-03-24T00:49:05.859 INFO:tasks.workunit.client.0.smithi038.stdout:4/730: creat da/d12/d24/d68/f100 x:0 0 0 2024-03-24T00:49:05.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:05 smithi038 ceph-mon[29271]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:05.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:05 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:05.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:05 smithi038 ceph-mon[29271]: Standby manager daemon smithi038.huhioz restarted 2024-03-24T00:49:05.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:05 smithi038 ceph-mon[29271]: Standby manager daemon smithi038.huhioz started 2024-03-24T00:49:05.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:05 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:05.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:05 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.huhioz/crt"}]: dispatch 2024-03-24T00:49:05.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:05 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:49:05.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:05 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.huhioz/key"}]: dispatch 2024-03-24T00:49:05.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:05 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:49:05.861 INFO:tasks.workunit.client.1.smithi088.stdout:3/769: mknod d3/d1a/cf3 0 2024-03-24T00:49:05.861 INFO:tasks.workunit.client.1.smithi088.stdout:3/770: chown d3/f87 41 1 2024-03-24T00:49:05.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.860+0000 7f054ea00700 1 -- 172.21.15.38:0/3827082009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f05481ab7a0 con 0x7f0548101cc0 2024-03-24T00:49:05.862 INFO:tasks.workunit.client.0.smithi038.stdout:5/727: dwrite d6/d19/d20/d70/da3/fab [4194304,4194304] 0 2024-03-24T00:49:05.867 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:49:05.867 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:49:05.867 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:49:05.867 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:49:05.867 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:49:05.867 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 1, 2024-03-24T00:49:05.867 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-24T00:49:05.867 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:49:05.867 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:49:05.868 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:49:05.868 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:49:05.868 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:49:05.868 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:49:05.868 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:49:05.868 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:49:05.868 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 13, 2024-03-24T00:49:05.868 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-24T00:49:05.868 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:49:05.868 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:49:05.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.862+0000 7f0544c00700 1 -- 172.21.15.38:0/3827082009 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f0534057bc0 con 0x7f0548101cc0 2024-03-24T00:49:05.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.866+0000 7f054ea00700 1 -- 172.21.15.38:0/3827082009 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f052c06d650 msgr2=0x7f052c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:05.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.866+0000 7f054ea00700 1 --2- 172.21.15.38:0/3827082009 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f052c06d650 0x7f052c06fb10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f05481a67c0 tx=0x7f0538005f90 comp rx=0 tx=0).stop 2024-03-24T00:49:05.869 INFO:tasks.workunit.client.0.smithi038.stdout:2/694: rmdir d4/d27/d54/d53/de2 39 2024-03-24T00:49:05.869 INFO:tasks.workunit.client.0.smithi038.stdout:2/695: chown d4/d27/d54/d53/fba 1043321 1 2024-03-24T00:49:05.869 INFO:tasks.workunit.client.0.smithi038.stdout:2/696: chown d4/f7 0 1 2024-03-24T00:49:05.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.866+0000 7f054ea00700 1 -- 172.21.15.38:0/3827082009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0548101cc0 msgr2=0x7f054810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:05.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.866+0000 7f054ea00700 1 --2- 172.21.15.38:0/3827082009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0548101cc0 0x7f054810d4f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f05340051c0 tx=0x7f053400bd90 comp rx=0 tx=0).stop 2024-03-24T00:49:05.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.866+0000 7f054ea00700 1 -- 172.21.15.38:0/3827082009 shutdown_connections 2024-03-24T00:49:05.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.866+0000 7f054ea00700 1 --2- 172.21.15.38:0/3827082009 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7f052c06d650 0x7f052c06fb10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:05.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.866+0000 7f054ea00700 1 --2- 172.21.15.38:0/3827082009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05480ff350 0x7f054810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:05.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.866+0000 7f054ea00700 1 --2- 172.21.15.38:0/3827082009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0548101cc0 0x7f054810d4f0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f05340051c0 tx=0x7f053400bd90 comp rx=0 tx=0).stop 2024-03-24T00:49:05.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.866+0000 7f054ea00700 1 -- 172.21.15.38:0/3827082009 >> 172.21.15.38:0/3827082009 conn(0x7f05480faf70 msgr2=0x7f05480fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:05.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.868+0000 7f054ea00700 1 -- 172.21.15.38:0/3827082009 shutdown_connections 2024-03-24T00:49:05.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:05.868+0000 7f054ea00700 1 -- 172.21.15.38:0/3827082009 wait complete. 2024-03-24T00:49:05.873 INFO:tasks.workunit.client.0.smithi038.stdout:6/847: rename d1/da/f52 to d1/da/de6/f118 0 2024-03-24T00:49:05.878 INFO:tasks.workunit.client.1.smithi088.stdout:3/771: dread d3/d6/dca/dae/d4b/f4c [0,4194304] 0 2024-03-24T00:49:05.878 INFO:tasks.workunit.client.0.smithi038.stdout:3/759: truncate d3/f25 4055412 0 2024-03-24T00:49:05.878 INFO:tasks.workunit.client.0.smithi038.stdout:3/760: readlink d3/d9/def/lf3 0 2024-03-24T00:49:05.879 INFO:tasks.workunit.client.1.smithi088.stdout:4/798: mknod d5/d16/d1a/d22/dc6/c116 0 2024-03-24T00:49:05.880 INFO:tasks.workunit.client.1.smithi088.stdout:4/799: chown d5/d13/d63/fc8 142 1 2024-03-24T00:49:05.881 INFO:tasks.workunit.client.1.smithi088.stdout:4/800: creat d5/d13/d18/d24/d3d/df0/f117 x:0 0 0 2024-03-24T00:49:05.886 INFO:tasks.workunit.client.0.smithi038.stdout:5/728: link d6/d19/d20/d42/d43/d4c/c7b d6/d16/d21/cea 0 2024-03-24T00:49:05.887 INFO:tasks.workunit.client.1.smithi088.stdout:0/833: unlink d9/l18 0 2024-03-24T00:49:05.894 INFO:tasks.workunit.client.1.smithi088.stdout:7/747: getdents d3/d32/d34/d9c/d4b/d74/d79 0 2024-03-24T00:49:05.895 INFO:tasks.workunit.client.1.smithi088.stdout:7/748: fdatasync d3/d32/d34/faa 0 2024-03-24T00:49:05.897 INFO:tasks.workunit.client.0.smithi038.stdout:1/823: write da/dd/d7a/d82/d98/d34/f39 [407740,57709] 0 2024-03-24T00:49:05.898 INFO:tasks.workunit.client.1.smithi088.stdout:2/769: stat db/dd/d96/d4e/fcc 0 2024-03-24T00:49:05.898 INFO:tasks.workunit.client.1.smithi088.stdout:2/770: creat db/dd/d96/d99/db2/ff2 x:0 0 0 2024-03-24T00:49:05.900 INFO:tasks.workunit.client.1.smithi088.stdout:6/865: dwrite db/d1d/d50/d90/d55/f10f [0,4194304] 0 2024-03-24T00:49:05.919 INFO:tasks.workunit.client.0.smithi038.stdout:0/703: dwrite d0/d1b/f7c [0,4194304] 0 2024-03-24T00:49:05.919 INFO:tasks.workunit.client.0.smithi038.stdout:0/704: stat d0/c53 0 2024-03-24T00:49:05.919 INFO:tasks.workunit.client.0.smithi038.stdout:0/705: dread - d0/d12/d42/fdd zero size 2024-03-24T00:49:05.946 INFO:tasks.workunit.client.0.smithi038.stdout:3/761: symlink d3/d9/dc/l106 0 2024-03-24T00:49:05.946 INFO:tasks.workunit.client.0.smithi038.stdout:3/762: chown d3/d30/d5d/d7c/le5 1 1 2024-03-24T00:49:05.950 INFO:tasks.workunit.client.0.smithi038.stdout:2/697: dwrite d4/d18/d26/f4e [4194304,4194304] 0 2024-03-24T00:49:05.952 INFO:tasks.workunit.client.0.smithi038.stdout:4/731: mknod da/d12/d37/dd/c101 0 2024-03-24T00:49:05.953 INFO:tasks.workunit.client.0.smithi038.stdout:4/732: creat da/d12/d37/dd/d15/d6d/d8f/f102 x:0 0 0 2024-03-24T00:49:05.959 INFO:tasks.workunit.client.0.smithi038.stdout:7/713: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fc7 [0,4194304] 0 2024-03-24T00:49:05.960 INFO:tasks.workunit.client.0.smithi038.stdout:7/714: write d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f66 [439632,116461] 0 2024-03-24T00:49:05.961 INFO:tasks.workunit.client.0.smithi038.stdout:7/715: readlink d2/d9/da/dc/l107 0 2024-03-24T00:49:05.961 INFO:tasks.workunit.client.0.smithi038.stdout:8/809: dwrite d0/d45/d37/f48 [0,4194304] 0 2024-03-24T00:49:05.967 INFO:tasks.workunit.client.1.smithi088.stdout:5/856: creat d1/d43/d4e/f122 x:0 0 0 2024-03-24T00:49:05.972 INFO:tasks.workunit.client.0.smithi038.stdout:0/706: link d0/d14/de4/feb d0/d14/d88/db8/ff3 0 2024-03-24T00:49:05.976 INFO:tasks.workunit.client.1.smithi088.stdout:8/797: link d9/dc/c42 d9/dc/d11/d24/d29/d5b/d8b/dd3/c10c 0 2024-03-24T00:49:05.977 INFO:tasks.workunit.client.1.smithi088.stdout:8/798: truncate d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/fd4 930828 0 2024-03-24T00:49:05.977 INFO:tasks.workunit.client.1.smithi088.stdout:8/799: chown d9/dc/d11/d24/d29/d36/d6a/d7b/d8f 210 1 2024-03-24T00:49:05.981 INFO:tasks.workunit.client.0.smithi038.stdout:9/852: rename d12/d28/d3a to d12/d28/d9c/ddd/d111 0 2024-03-24T00:49:05.982 INFO:tasks.workunit.client.0.smithi038.stdout:9/853: chown d12/d41/d57/dd3/dfe/f92 1636505 1 2024-03-24T00:49:05.982 INFO:tasks.workunit.client.0.smithi038.stdout:9/854: chown d12/d28/f82 2 1 2024-03-24T00:49:05.982 INFO:tasks.workunit.client.0.smithi038.stdout:9/855: chown d12/d28/d6c/dbc/fc8 1012 1 2024-03-24T00:49:05.987 INFO:tasks.workunit.client.1.smithi088.stdout:9/799: symlink d1/d10/d77/d8a/de0/l100 0 2024-03-24T00:49:05.996 INFO:tasks.workunit.client.0.smithi038.stdout:5/729: dwrite d6/f18 [0,4194304] 0 2024-03-24T00:49:05.997 INFO:tasks.workunit.client.0.smithi038.stdout:5/730: creat d6/d60/feb x:0 0 0 2024-03-24T00:49:06.000 INFO:tasks.workunit.client.1.smithi088.stdout:4/801: truncate d5/d17/fd0 71260 0 2024-03-24T00:49:06.002 INFO:tasks.workunit.client.0.smithi038.stdout:1/824: dwrite da/dd/d7a/d82/d98/d9e/fb0 [0,4194304] 0 2024-03-24T00:49:06.008 INFO:tasks.workunit.client.0.smithi038.stdout:7/716: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/d10a 0 2024-03-24T00:49:06.008 INFO:tasks.workunit.client.0.smithi038.stdout:7/717: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/f35 0 2024-03-24T00:49:06.010 INFO:tasks.workunit.client.1.smithi088.stdout:7/749: truncate d3/d32/d34/fd2 3767181 0 2024-03-24T00:49:06.013 INFO:tasks.workunit.client.0.smithi038.stdout:0/707: truncate d0/d12/d42/d9d/d28/f48 3585956 0 2024-03-24T00:49:06.015 INFO:tasks.workunit.client.0.smithi038.stdout:5/731: dread d6/d19/d20/fcf [0,4194304] 0 2024-03-24T00:49:06.015 INFO:tasks.workunit.client.1.smithi088.stdout:2/771: mkdir db/dd/d96/d4e/d8f/dad/df3 0 2024-03-24T00:49:06.018 INFO:tasks.workunit.client.1.smithi088.stdout:6/866: getdents db/dd/d61/d6d/d74/df0 0 2024-03-24T00:49:06.026 INFO:tasks.workunit.client.0.smithi038.stdout:8/810: dwrite d0/d45/d37/d5f/db7/dcc/fef [0,4194304] 0 2024-03-24T00:49:06.038 INFO:tasks.workunit.client.0.smithi038.stdout:1/825: symlink da/dd/d4a/l112 0 2024-03-24T00:49:06.041 INFO:tasks.workunit.client.0.smithi038.stdout:0/708: unlink d0/dd7/fe8 0 2024-03-24T00:49:06.042 INFO:tasks.workunit.client.0.smithi038.stdout:5/732: write d6/d19/d20/d42/d79/fa0 [2285191,102745] 0 2024-03-24T00:49:06.042 INFO:tasks.workunit.client.0.smithi038.stdout:5/733: stat d6/d19/d20/d33 0 2024-03-24T00:49:06.050 INFO:tasks.workunit.client.1.smithi088.stdout:1/810: rename d6/d23/daf/d95 to d6/d6e/dfc/d108 0 2024-03-24T00:49:06.052 INFO:tasks.workunit.client.0.smithi038.stdout:9/856: mknod d12/d55/c112 0 2024-03-24T00:49:06.052 INFO:tasks.workunit.client.0.smithi038.stdout:9/857: truncate d12/d41/d57/f109 712090 0 2024-03-24T00:49:06.054 INFO:tasks.workunit.client.1.smithi088.stdout:9/800: mknod d1/d15/d80/da5/c101 0 2024-03-24T00:49:06.056 INFO:tasks.workunit.client.0.smithi038.stdout:8/811: creat d0/d45/d37/d5f/db7/d62/dfa/dc8/f10f x:0 0 0 2024-03-24T00:49:06.056 INFO:tasks.workunit.client.0.smithi038.stdout:2/698: dwrite d4/d27/d2f/d30/f8d [0,4194304] 0 2024-03-24T00:49:06.057 INFO:tasks.workunit.client.0.smithi038.stdout:2/699: fsync d4/d76/fb1 0 2024-03-24T00:49:06.057 INFO:tasks.workunit.client.0.smithi038.stdout:1/826: read da/dd/d2c/d41/f65 [1885990,13483] 0 2024-03-24T00:49:06.058 INFO:tasks.workunit.client.0.smithi038.stdout:1/827: creat da/dd/d2c/d40/de9/f113 x:0 0 0 2024-03-24T00:49:06.058 INFO:tasks.workunit.client.0.smithi038.stdout:1/828: dread - da/dd/d22/dab/ff5 zero size 2024-03-24T00:49:06.072 INFO:tasks.workunit.client.0.smithi038.stdout:7/718: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/d10b 0 2024-03-24T00:49:06.072 INFO:tasks.workunit.client.0.smithi038.stdout:4/733: dwrite da/d12/d37/dd/d15/d4c/d70/fa5 [0,4194304] 0 2024-03-24T00:49:06.075 INFO:tasks.workunit.client.0.smithi038.stdout:5/734: dread d6/d1e/f63 [4194304,4194304] 0 2024-03-24T00:49:06.075 INFO:tasks.workunit.client.0.smithi038.stdout:5/735: truncate d6/d19/d20/d70/da3/fb2 427112 0 2024-03-24T00:49:06.078 INFO:tasks.workunit.client.1.smithi088.stdout:5/857: dwrite d1/d19/f2c [0,4194304] 0 2024-03-24T00:49:06.078 INFO:tasks.workunit.client.1.smithi088.stdout:5/858: fsync d1/d19/fa6 0 2024-03-24T00:49:06.078 INFO:tasks.workunit.client.1.smithi088.stdout:5/859: write d1/d10/d79/dbc/f119 [547058,42213] 0 2024-03-24T00:49:06.080 INFO:tasks.workunit.client.1.smithi088.stdout:5/860: read d1/dd/d6f/d98/fd1 [23590,73776] 0 2024-03-24T00:49:06.080 INFO:tasks.workunit.client.1.smithi088.stdout:5/861: write d1/d19/d48/f92 [1710206,109999] 0 2024-03-24T00:49:06.081 INFO:tasks.workunit.client.0.smithi038.stdout:0/709: creat d0/d12/d15/d58/d64/da7/dbb/dd4/ff4 x:0 0 0 2024-03-24T00:49:06.081 INFO:tasks.workunit.client.0.smithi038.stdout:0/710: stat d0/d14/d84/fa4 0 2024-03-24T00:49:06.081 INFO:tasks.workunit.client.0.smithi038.stdout:9/858: mknod d12/d41/dc0/c113 0 2024-03-24T00:49:06.083 INFO:tasks.workunit.client.1.smithi088.stdout:4/802: mkdir d5/d13/d18/d118 0 2024-03-24T00:49:06.083 INFO:tasks.workunit.client.1.smithi088.stdout:4/803: write f2 [8449057,82621] 0 2024-03-24T00:49:06.084 INFO:tasks.workunit.client.0.smithi038.stdout:3/763: rename d3/d30/d5d/d7c/dbc to d3/d2a/d107 0 2024-03-24T00:49:06.086 INFO:tasks.workunit.client.1.smithi088.stdout:7/750: mknod d3/d32/d34/d9c/d4b/d74/d79/dea/cf1 0 2024-03-24T00:49:06.086 INFO:tasks.workunit.client.1.smithi088.stdout:7/751: chown d3/d32/d34/d7c/l88 540937809 1 2024-03-24T00:49:06.090 INFO:tasks.workunit.client.0.smithi038.stdout:2/700: creat d4/d27/d2f/fef x:0 0 0 2024-03-24T00:49:06.090 INFO:tasks.workunit.client.0.smithi038.stdout:2/701: chown d4/f2b 1483 1 2024-03-24T00:49:06.090 INFO:tasks.workunit.client.0.smithi038.stdout:2/702: write d4/d27/d54/d53/de2/f9f [303984,51964] 0 2024-03-24T00:49:06.090 INFO:tasks.workunit.client.0.smithi038.stdout:2/703: read - d4/d27/d54/d53/f95 zero size 2024-03-24T00:49:06.091 INFO:tasks.workunit.client.1.smithi088.stdout:2/772: creat db/dd/d96/d4e/d8f/dad/df3/ff4 x:0 0 0 2024-03-24T00:49:06.091 INFO:tasks.workunit.client.1.smithi088.stdout:2/773: creat db/dd/d96/d99/db2/ff5 x:0 0 0 2024-03-24T00:49:06.095 INFO:tasks.workunit.client.1.smithi088.stdout:6/867: mkdir db/d3d/dcb/d112 0 2024-03-24T00:49:06.104 INFO:tasks.workunit.client.0.smithi038.stdout:5/736: dread d6/d19/d20/d42/d43/d50/f81 [0,4194304] 0 2024-03-24T00:49:06.104 INFO:tasks.workunit.client.0.smithi038.stdout:5/737: stat d6/d19/d20/d42/d43 0 2024-03-24T00:49:06.110 INFO:tasks.workunit.client.1.smithi088.stdout:0/834: sync 2024-03-24T00:49:06.110 INFO:tasks.workunit.client.1.smithi088.stdout:4/804: dread d5/dac/fb8 [0,4194304] 0 2024-03-24T00:49:06.111 INFO:tasks.workunit.client.1.smithi088.stdout:4/805: stat d5/d7/d15/d74 0 2024-03-24T00:49:06.113 INFO:tasks.workunit.client.0.smithi038.stdout:7/719: chown d2/d2c/l49 8 1 2024-03-24T00:49:06.116 INFO:tasks.workunit.client.0.smithi038.stdout:7/720: chown d2/d9/da/d23/d2b/d5b/d72/d92/l25 10942101 1 2024-03-24T00:49:06.117 INFO:tasks.workunit.client.1.smithi088.stdout:3/772: rename d3/d6/d27/l8b to d3/d6/d27/de9/ddf/lf4 0 2024-03-24T00:49:06.117 INFO:tasks.workunit.client.1.smithi088.stdout:1/811: creat d6/d6e/dfc/d108/d5b/d5e/f109 x:0 0 0 2024-03-24T00:49:06.117 INFO:tasks.workunit.client.1.smithi088.stdout:1/812: write d6/d23/f29 [1726674,79273] 0 2024-03-24T00:49:06.117 INFO:tasks.workunit.client.1.smithi088.stdout:1/813: chown d6/d27/d67/d88/f92 3670 1 2024-03-24T00:49:06.118 INFO:tasks.workunit.client.0.smithi038.stdout:4/734: truncate da/f1e 1138847 0 2024-03-24T00:49:06.118 INFO:tasks.workunit.client.0.smithi038.stdout:4/735: chown da/d38/f76 6 1 2024-03-24T00:49:06.119 INFO:tasks.workunit.client.0.smithi038.stdout:4/736: creat da/d12/d24/d5f/df8/f103 x:0 0 0 2024-03-24T00:49:06.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.117+0000 7fd459000700 1 -- 172.21.15.38:0/2064175312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4540ff350 msgr2=0x7fd454101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:06.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.117+0000 7fd459000700 1 --2- 172.21.15.38:0/2064175312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4540ff350 0x7fd454101780 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009960 tx=0x7fd43c009c70 comp rx=0 tx=0).stop 2024-03-24T00:49:06.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.118+0000 7fd459000700 1 -- 172.21.15.38:0/2064175312 shutdown_connections 2024-03-24T00:49:06.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.118+0000 7fd459000700 1 --2- 172.21.15.38:0/2064175312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd454101cc0 0x7fd454104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:06.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.118+0000 7fd459000700 1 --2- 172.21.15.38:0/2064175312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4540ff350 0x7fd454101780 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:06.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.118+0000 7fd459000700 1 -- 172.21.15.38:0/2064175312 >> 172.21.15.38:0/2064175312 conn(0x7fd4540faf70 msgr2=0x7fd4540fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:06.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.118+0000 7fd459000700 1 -- 172.21.15.38:0/2064175312 shutdown_connections 2024-03-24T00:49:06.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.118+0000 7fd459000700 1 -- 172.21.15.38:0/2064175312 wait complete. 2024-03-24T00:49:06.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.120+0000 7fd459000700 1 Processor -- start 2024-03-24T00:49:06.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.120+0000 7fd459000700 1 -- start start 2024-03-24T00:49:06.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.120+0000 7fd459000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4540ff350 0x7fd45419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:06.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.120+0000 7fd459000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd454101cc0 0x7fd45419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:06.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.120+0000 7fd459000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45419d590 con 0x7fd4540ff350 2024-03-24T00:49:06.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.120+0000 7fd459000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45419d6d0 con 0x7fd454101cc0 2024-03-24T00:49:06.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.121+0000 7fd451e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4540ff350 0x7fd45419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:06.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.122+0000 7fd451e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4540ff350 0x7fd45419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34764/0 (socket says 172.21.15.38:34764) 2024-03-24T00:49:06.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.122+0000 7fd451e00700 1 -- 172.21.15.38:0/2533134338 learned_addr learned my addr 172.21.15.38:0/2533134338 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:49:06.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.122+0000 7fd451400700 1 --2- 172.21.15.38:0/2533134338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd454101cc0 0x7fd45419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:06.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.123+0000 7fd451e00700 1 -- 172.21.15.38:0/2533134338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd454101cc0 msgr2=0x7fd45419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:06.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.123+0000 7fd451e00700 1 --2- 172.21.15.38:0/2533134338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd454101cc0 0x7fd45419d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:06.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.123+0000 7fd451e00700 1 -- 172.21.15.38:0/2533134338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c0095f0 con 0x7fd4540ff350 2024-03-24T00:49:06.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.124+0000 7fd451e00700 1 --2- 172.21.15.38:0/2533134338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4540ff350 0x7fd45419cac0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd43c005250 tx=0x7fd43c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:06.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.124+0000 7fd44aa00700 1 -- 172.21.15.38:0/2533134338 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd43c01d070 con 0x7fd4540ff350 2024-03-24T00:49:06.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.125+0000 7fd459000700 1 -- 172.21.15.38:0/2533134338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd454105a30 con 0x7fd4540ff350 2024-03-24T00:49:06.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.125+0000 7fd459000700 1 -- 172.21.15.38:0/2533134338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd454105f70 con 0x7fd4540ff350 2024-03-24T00:49:06.127 INFO:tasks.workunit.client.1.smithi088.stdout:9/801: link d1/d22/d72/f9e d1/d10/d9b/dab/db8/f102 0 2024-03-24T00:49:06.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.126+0000 7fd44aa00700 1 -- 172.21.15.38:0/2533134338 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd43c00f460 con 0x7fd4540ff350 2024-03-24T00:49:06.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.126+0000 7fd44aa00700 1 -- 172.21.15.38:0/2533134338 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd43c021620 con 0x7fd4540ff350 2024-03-24T00:49:06.127 INFO:tasks.workunit.client.1.smithi088.stdout:9/802: write d1/d10/d9b/dab/d7a/fd9 [972965,75494] 0 2024-03-24T00:49:06.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.129+0000 7fd459000700 1 -- 172.21.15.38:0/2533134338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd430005320 con 0x7fd4540ff350 2024-03-24T00:49:06.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.129+0000 7fd44aa00700 1 -- 172.21.15.38:0/2533134338 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 25) v1 ==== 91370+0+0 (secure 0 0 0) 0x7fd43c021780 con 0x7fd4540ff350 2024-03-24T00:49:06.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.130+0000 7fd44aa00700 1 --2- 172.21.15.38:0/2533134338 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fd43806d620 0x7fd43806fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:06.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.130+0000 7fd44aa00700 1 -- 172.21.15.38:0/2533134338 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd43c064030 con 0x7fd4540ff350 2024-03-24T00:49:06.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.131+0000 7fd451400700 1 --2- 172.21.15.38:0/2533134338 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fd43806d620 0x7fd43806fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:06.135 INFO:tasks.workunit.client.0.smithi038.stdout:0/711: mknod d0/d1b/cf5 0 2024-03-24T00:49:06.135 INFO:tasks.workunit.client.0.smithi038.stdout:9/859: mknod d12/d28/d9c/ddd/d111/d8f/c114 0 2024-03-24T00:49:06.135 INFO:tasks.workunit.client.0.smithi038.stdout:9/860: chown d12/d41/f6a 1445 1 2024-03-24T00:49:06.135 INFO:tasks.workunit.client.0.smithi038.stdout:9/861: read - d12/d28/d9c/ddd/d111/d26/f5a zero size 2024-03-24T00:49:06.135 INFO:tasks.workunit.client.0.smithi038.stdout:9/862: dread - d12/d28/d9c/ddd/df5/ff7 zero size 2024-03-24T00:49:06.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.135+0000 7fd451400700 1 --2- 172.21.15.38:0/2533134338 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fd43806d620 0x7fd43806fae0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd444005ea0 tx=0x7fd444005e30 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:06.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.140+0000 7fd44aa00700 1 -- 172.21.15.38:0/2533134338 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fd43c059330 con 0x7fd4540ff350 2024-03-24T00:49:06.145 INFO:tasks.workunit.client.0.smithi038.stdout:3/764: truncate d3/d9/f16 3185959 0 2024-03-24T00:49:06.145 INFO:tasks.workunit.client.0.smithi038.stdout:3/765: read d3/d30/f33 [28066,5986] 0 2024-03-24T00:49:06.147 INFO:tasks.workunit.client.0.smithi038.stdout:1/829: dwrite da/dd/d2c/d5a/d74/f64 [0,4194304] 0 2024-03-24T00:49:06.153 INFO:tasks.workunit.client.1.smithi088.stdout:2/774: symlink db/dd/d96/d99/db2/lf6 0 2024-03-24T00:49:06.159 INFO:tasks.workunit.client.0.smithi038.stdout:8/812: rename d0/d45/d37/d5f/db7/f10b to d0/d1d/d27/d4d/daf/f110 0 2024-03-24T00:49:06.160 INFO:tasks.workunit.client.0.smithi038.stdout:8/813: creat d0/d45/d37/d5f/db7/dcc/f111 x:0 0 0 2024-03-24T00:49:06.160 INFO:tasks.workunit.client.0.smithi038.stdout:8/814: getdents d0/d1d/d27/d4d/d81/d109 0 2024-03-24T00:49:06.160 INFO:tasks.workunit.client.0.smithi038.stdout:8/815: truncate d0/d1d/d27/d4d/fce 1495036 0 2024-03-24T00:49:06.163 INFO:tasks.workunit.client.0.smithi038.stdout:9/863: dread d12/d28/d9c/ddd/d111/f2b [0,4194304] 0 2024-03-24T00:49:06.163 INFO:tasks.workunit.client.0.smithi038.stdout:9/864: chown d12/d1a/d5f/df0 5 1 2024-03-24T00:49:06.169 INFO:tasks.workunit.client.1.smithi088.stdout:0/835: truncate d9/dc/de/d19/d8d/f98 235841 0 2024-03-24T00:49:06.169 INFO:tasks.workunit.client.1.smithi088.stdout:0/836: write d9/dc/de/d19/f48 [2900025,123436] 0 2024-03-24T00:49:06.180 INFO:tasks.workunit.client.0.smithi038.stdout:5/738: symlink d6/d19/d20/d42/d43/d68/d6e/lec 0 2024-03-24T00:49:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:05 smithi088 ceph-mon[36610]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:05 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:05 smithi088 ceph-mon[36610]: Standby manager daemon smithi038.huhioz restarted 2024-03-24T00:49:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:05 smithi088 ceph-mon[36610]: Standby manager daemon smithi038.huhioz started 2024-03-24T00:49:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:05 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:05 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.huhioz/crt"}]: dispatch 2024-03-24T00:49:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:05 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:49:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:05 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.huhioz/key"}]: dispatch 2024-03-24T00:49:06.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:05 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:49:06.186 INFO:tasks.workunit.client.1.smithi088.stdout:4/806: dwrite d5/d17/d91/fe3 [0,4194304] 0 2024-03-24T00:49:06.191 INFO:tasks.workunit.client.0.smithi038.stdout:2/704: dwrite d4/d27/ddb/d7c/fd9 [0,4194304] 0 2024-03-24T00:49:06.192 INFO:tasks.workunit.client.1.smithi088.stdout:8/800: rename d9/d6b/f6c to d9/dc/d11/d24/d29/da9/f10d 0 2024-03-24T00:49:06.199 INFO:tasks.workunit.client.1.smithi088.stdout:3/773: symlink d3/de/d6f/dcc/lf5 0 2024-03-24T00:49:06.208 INFO:tasks.workunit.client.1.smithi088.stdout:6/868: dwrite db/dd/fd9 [4194304,4194304] 0 2024-03-24T00:49:06.208 INFO:tasks.workunit.client.1.smithi088.stdout:6/869: creat db/d3d/d83/deb/f113 x:0 0 0 2024-03-24T00:49:06.210 INFO:tasks.workunit.client.0.smithi038.stdout:0/712: dwrite d0/d12/d92/fd6 [0,4194304] 0 2024-03-24T00:49:06.210 INFO:tasks.workunit.client.0.smithi038.stdout:0/713: truncate d0/d12/d42/d83/fb1 475038 0 2024-03-24T00:49:06.210 INFO:tasks.workunit.client.0.smithi038.stdout:7/721: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/f71 [0,4194304] 0 2024-03-24T00:49:06.214 INFO:tasks.workunit.client.1.smithi088.stdout:1/814: rmdir d6/d6e/dfc/d108/d58 39 2024-03-24T00:49:06.214 INFO:tasks.workunit.client.1.smithi088.stdout:1/815: readlink d6/d23/l28 0 2024-03-24T00:49:06.223 INFO:tasks.workunit.client.1.smithi088.stdout:4/807: dwrite d5/d13/d18/d24/d3d/fda [0,4194304] 0 2024-03-24T00:49:06.239 INFO:tasks.workunit.client.1.smithi088.stdout:9/803: creat d1/d10/d9b/dab/f103 x:0 0 0 2024-03-24T00:49:06.239 INFO:tasks.workunit.client.1.smithi088.stdout:1/816: dread d6/d27/d67/f6a [0,4194304] 0 2024-03-24T00:49:06.241 INFO:tasks.workunit.client.1.smithi088.stdout:0/837: mknod d9/d26/d8f/da1/d62/c10a 0 2024-03-24T00:49:06.241 INFO:tasks.workunit.client.1.smithi088.stdout:0/838: creat d9/d26/d8f/da1/d62/d9b/f10b x:0 0 0 2024-03-24T00:49:06.252 INFO:tasks.workunit.client.0.smithi038.stdout:9/865: dwrite d12/d28/d71/d99/f10a [0,4194304] 0 2024-03-24T00:49:06.256 INFO:tasks.workunit.client.0.smithi038.stdout:3/766: link d3/d9/dc/df/fdd d3/d30/d5d/deb/f108 0 2024-03-24T00:49:06.259 INFO:tasks.workunit.client.1.smithi088.stdout:7/752: rename d3/d32/d34/d9c/d4b/d74/l91 to d3/d32/d34/d9c/d4b/d73/dab/dc3/lf2 0 2024-03-24T00:49:06.261 INFO:tasks.workunit.client.1.smithi088.stdout:3/774: rmdir d3/de/d6f/d85/d49 39 2024-03-24T00:49:06.261 INFO:tasks.workunit.client.1.smithi088.stdout:5/862: write d1/d10/f52 [3610261,10023] 0 2024-03-24T00:49:06.263 INFO:tasks.workunit.client.1.smithi088.stdout:6/870: link db/dd/d61/d6d/d74/d88/f86 db/d1d/f114 0 2024-03-24T00:49:06.276 INFO:tasks.workunit.client.0.smithi038.stdout:1/830: mkdir da/d3c/d114 0 2024-03-24T00:49:06.277 INFO:tasks.workunit.client.1.smithi088.stdout:1/817: dread d6/f17 [4194304,4194304] 0 2024-03-24T00:49:06.291 INFO:tasks.workunit.client.1.smithi088.stdout:4/808: rmdir d5/d7/d15/d60 39 2024-03-24T00:49:06.294 INFO:tasks.workunit.client.1.smithi088.stdout:2/775: dwrite db/dd/d96/d4e/fa0 [0,4194304] 0 2024-03-24T00:49:06.297 INFO:tasks.workunit.client.0.smithi038.stdout:8/816: symlink d0/d1d/d27/d6b/dfc/d9e/l112 0 2024-03-24T00:49:06.297 INFO:tasks.workunit.client.0.smithi038.stdout:8/817: fdatasync d0/d10/f92 0 2024-03-24T00:49:06.297 INFO:tasks.workunit.client.0.smithi038.stdout:8/818: dread - d0/d15/fcf zero size 2024-03-24T00:49:06.304 INFO:tasks.workunit.client.1.smithi088.stdout:8/801: rename d9/d6b/f105 to d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/f10e 0 2024-03-24T00:49:06.305 INFO:tasks.workunit.client.1.smithi088.stdout:0/839: dwrite d9/dc/de/d19/d44/d46/d65/d2a/f75 [0,4194304] 0 2024-03-24T00:49:06.305 INFO:tasks.workunit.client.1.smithi088.stdout:0/840: readlink d9/dc/de/d19/d8d/la5 0 2024-03-24T00:49:06.312 INFO:tasks.workunit.client.1.smithi088.stdout:5/863: link d1/d19/d1e/f76 d1/d10/db6/d10b/d2f/d8a/deb/d107/f123 0 2024-03-24T00:49:06.314 INFO:tasks.workunit.client.0.smithi038.stdout:0/714: dwrite d0/d14/d84/fd2 [0,4194304] 0 2024-03-24T00:49:06.314 INFO:tasks.workunit.client.0.smithi038.stdout:0/715: chown d0/d1b/cee 4 1 2024-03-24T00:49:06.315 INFO:tasks.workunit.client.0.smithi038.stdout:2/705: symlink d4/d27/d54/dab/lf0 0 2024-03-24T00:49:06.315 INFO:tasks.workunit.client.0.smithi038.stdout:2/706: chown d4/dc7 71 1 2024-03-24T00:49:06.317 INFO:tasks.workunit.client.1.smithi088.stdout:3/775: symlink d3/d6/d19/d2a/d5f/lf6 0 2024-03-24T00:49:06.324 INFO:tasks.workunit.client.1.smithi088.stdout:6/871: dwrite db/fbd [0,4194304] 0 2024-03-24T00:49:06.325 INFO:tasks.workunit.client.1.smithi088.stdout:6/872: chown db/d1d/d50/d90/d56 180639930 1 2024-03-24T00:49:06.325 INFO:tasks.workunit.client.1.smithi088.stdout:6/873: chown db/d1d/d23/d5c/df2/cd8 3635005 1 2024-03-24T00:49:06.327 INFO:tasks.workunit.client.1.smithi088.stdout:1/818: symlink d6/d6e/dfc/d108/d5b/d5d/d61/d7d/dc8/l10a 0 2024-03-24T00:49:06.329 INFO:tasks.workunit.client.1.smithi088.stdout:4/809: getdents d5/d7/d15 0 2024-03-24T00:49:06.333 INFO:tasks.workunit.client.0.smithi038.stdout:0/716: dread d0/d12/d42/d44/f93 [0,4194304] 0 2024-03-24T00:49:06.335 INFO:tasks.workunit.client.0.smithi038.stdout:4/737: getdents da/d12/d29/d5b/d73/db4 0 2024-03-24T00:49:06.337 INFO:tasks.workunit.client.1.smithi088.stdout:9/804: symlink d1/d10/d29/l104 0 2024-03-24T00:49:06.337 INFO:tasks.workunit.client.1.smithi088.stdout:9/805: readlink d1/le 0 2024-03-24T00:49:06.337 INFO:tasks.workunit.client.1.smithi088.stdout:9/806: chown d1/d10/d2c/f75 15 1 2024-03-24T00:49:06.337 INFO:tasks.workunit.client.1.smithi088.stdout:9/807: fdatasync d1/f8 0 2024-03-24T00:49:06.338 INFO:tasks.workunit.client.0.smithi038.stdout:5/739: write d6/d19/d20/d42/d43/d68/d6e/f7f [2004531,127081] 0 2024-03-24T00:49:06.339 INFO:tasks.workunit.client.0.smithi038.stdout:5/740: fsync d6/d19/d20/d33/d6f/f7a 0 2024-03-24T00:49:06.340 INFO:tasks.workunit.client.1.smithi088.stdout:2/776: rename db/dd/dc9/cdd to db/cf7 0 2024-03-24T00:49:06.344 INFO:tasks.workunit.client.1.smithi088.stdout:8/802: unlink d9/dc/d11/d24/d29/d39/d84/ca6 0 2024-03-24T00:49:06.344 INFO:tasks.workunit.client.1.smithi088.stdout:8/803: creat d9/d6b/f10f x:0 0 0 2024-03-24T00:49:06.346 INFO:tasks.workunit.client.1.smithi088.stdout:6/874: dread db/dd/d61/fdb [0,4194304] 0 2024-03-24T00:49:06.350 INFO:tasks.workunit.client.0.smithi038.stdout:9/866: link d12/d28/d9c/ddd/d111/d8f/c107 d12/d1a/d80/c115 0 2024-03-24T00:49:06.350 INFO:tasks.workunit.client.0.smithi038.stdout:9/867: dread - d12/d1a/d5f/f10f zero size 2024-03-24T00:49:06.351 INFO:tasks.workunit.client.1.smithi088.stdout:0/841: creat d9/d26/d108/f10c x:0 0 0 2024-03-24T00:49:06.354 INFO:tasks.workunit.client.0.smithi038.stdout:3/767: truncate d3/d9/dc/df/f1d 1907115 0 2024-03-24T00:49:06.355 INFO:tasks.workunit.client.0.smithi038.stdout:0/717: write d0/d1b/d2c/f86 [1256210,11681] 0 2024-03-24T00:49:06.357 INFO:tasks.workunit.client.0.smithi038.stdout:1/831: symlink da/dd/d4a/d76/l115 0 2024-03-24T00:49:06.357 INFO:tasks.workunit.client.0.smithi038.stdout:1/832: chown da/dd/d22/d94/fce 101966286 1 2024-03-24T00:49:06.358 INFO:tasks.workunit.client.1.smithi088.stdout:0/842: dread d9/d26/d8f/da1/d63/f8a [4194304,4194304] 0 2024-03-24T00:49:06.360 INFO:tasks.workunit.client.1.smithi088.stdout:5/864: unlink d1/dd/d6f/da1/d3f/d88/d8f/f5e 0 2024-03-24T00:49:06.360 INFO:tasks.workunit.client.1.smithi088.stdout:5/865: fdatasync d1/d10/db6/d10b/d2f/d57/d9e/f11d 0 2024-03-24T00:49:06.361 INFO:tasks.workunit.client.0.smithi038.stdout:8/819: symlink d0/d1d/d27/d4d/daf/l113 0 2024-03-24T00:49:06.361 INFO:tasks.workunit.client.0.smithi038.stdout:8/820: readlink d0/d1d/d8c/lf8 0 2024-03-24T00:49:06.362 INFO:tasks.workunit.client.0.smithi038.stdout:8/821: dread - d0/d1d/d27/d4d/fd7 zero size 2024-03-24T00:49:06.363 INFO:tasks.workunit.client.1.smithi088.stdout:3/776: link d3/d6/dca/f84 d3/de/d6f/d85/d90/ff7 0 2024-03-24T00:49:06.364 INFO:tasks.workunit.client.1.smithi088.stdout:7/753: dwrite d3/f2d [0,4194304] 0 2024-03-24T00:49:06.364 INFO:tasks.workunit.client.1.smithi088.stdout:7/754: write d3/d32/f48 [578917,3732] 0 2024-03-24T00:49:06.365 INFO:tasks.workunit.client.0.smithi038.stdout:6/848: sync 2024-03-24T00:49:06.400 INFO:tasks.workunit.client.0.smithi038.stdout:3/768: dread d3/d9/dc/da6/db5/d19/f6c [0,4194304] 0 2024-03-24T00:49:06.400 INFO:tasks.workunit.client.0.smithi038.stdout:3/769: write d3/d30/d5d/fde [83773,32560] 0 2024-03-24T00:49:06.400 INFO:tasks.workunit.client.0.smithi038.stdout:5/741: dwrite d6/d60/fca [0,4194304] 0 2024-03-24T00:49:06.407 INFO:tasks.workunit.client.1.smithi088.stdout:3/777: dwrite d3/d6/dca/dae/d4b/f4c [0,4194304] 0 2024-03-24T00:49:06.454 INFO:tasks.workunit.client.0.smithi038.stdout:2/707: creat d4/d18/da1/ff1 x:0 0 0 2024-03-24T00:49:06.456 INFO:tasks.workunit.client.1.smithi088.stdout:4/810: symlink d5/d7/d15/d74/d114/l119 0 2024-03-24T00:49:06.459 INFO:tasks.workunit.client.0.smithi038.stdout:4/738: symlink da/d12/d37/d83/dac/l104 0 2024-03-24T00:49:06.470 INFO:tasks.workunit.client.1.smithi088.stdout:2/777: creat db/dd/de7/ff8 x:0 0 0 2024-03-24T00:49:06.476 INFO:tasks.workunit.client.0.smithi038.stdout:9/868: truncate d12/d28/d9c/ddd/d111/d26/f86 1249313 0 2024-03-24T00:49:06.476 INFO:tasks.workunit.client.0.smithi038.stdout:9/869: chown d12/d28/d6c/dbc/l10d 504950 1 2024-03-24T00:49:06.476 INFO:tasks.workunit.client.0.smithi038.stdout:0/718: mknod d0/d12/d42/d83/cf6 0 2024-03-24T00:49:06.476 INFO:tasks.workunit.client.1.smithi088.stdout:3/778: dwrite d3/de/d6f/d85/d31/f72 [0,4194304] 0 2024-03-24T00:49:06.477 INFO:tasks.workunit.client.1.smithi088.stdout:3/779: dread - d3/f95 zero size 2024-03-24T00:49:06.477 INFO:tasks.workunit.client.1.smithi088.stdout:4/811: dread d5/d16/d37/de7/f10a [0,4194304] 0 2024-03-24T00:49:06.477 INFO:tasks.workunit.client.0.smithi038.stdout:2/708: write d4/d27/f94 [185494,39071] 0 2024-03-24T00:49:06.478 INFO:tasks.workunit.client.1.smithi088.stdout:8/804: creat d9/dc/d11/d24/d29/da9/f110 x:0 0 0 2024-03-24T00:49:06.482 INFO:tasks.workunit.client.0.smithi038.stdout:1/833: unlink da/dd/d2c/d5a/d74/cbf 0 2024-03-24T00:49:06.485 INFO:tasks.workunit.client.1.smithi088.stdout:6/875: mkdir db/dd/d115 0 2024-03-24T00:49:06.497 INFO:tasks.workunit.client.0.smithi038.stdout:6/849: symlink d1/da/d1b/d1d/l119 0 2024-03-24T00:49:06.502 INFO:tasks.workunit.client.1.smithi088.stdout:5/866: creat d1/d10/db6/d10b/d2e/d9f/ddf/f124 x:0 0 0 2024-03-24T00:49:06.508 INFO:tasks.workunit.client.1.smithi088.stdout:7/755: mkdir d3/d32/d76/df3 0 2024-03-24T00:49:06.509 INFO:tasks.workunit.client.1.smithi088.stdout:7/756: write d3/d32/d34/d7c/fc6 [237628,7002] 0 2024-03-24T00:49:06.526 INFO:tasks.workunit.client.1.smithi088.stdout:2/778: dwrite db/dd/d96/d4e/d8f/dad/fc2 [0,4194304] 0 2024-03-24T00:49:06.527 INFO:tasks.workunit.client.0.smithi038.stdout:6/850: dread d1/da/d61/fcd [4194304,4194304] 0 2024-03-24T00:49:06.527 INFO:tasks.workunit.client.0.smithi038.stdout:6/851: readlink d1/d37/d68/l105 0 2024-03-24T00:49:06.529 INFO:tasks.workunit.client.0.smithi038.stdout:6/852: getdents d1/d37/d84/df5/d113 0 2024-03-24T00:49:06.554 INFO:tasks.workunit.client.0.smithi038.stdout:3/770: creat d3/d9/dc/da6/dfc/f109 x:0 0 0 2024-03-24T00:49:06.555 INFO:tasks.workunit.client.0.smithi038.stdout:3/771: write d3/d9/dc/da6/db5/d19/d82/ffb [1039193,118916] 0 2024-03-24T00:49:06.555 INFO:tasks.workunit.client.0.smithi038.stdout:3/772: write d3/d30/d5d/d7c/db0/fe1 [130370,37523] 0 2024-03-24T00:49:06.556 INFO:tasks.workunit.client.0.smithi038.stdout:3/773: chown d3/d30/d5d/d7c/db0 28672 1 2024-03-24T00:49:06.556 INFO:tasks.workunit.client.0.smithi038.stdout:3/774: creat d3/d30/f10a x:0 0 0 2024-03-24T00:49:06.556 INFO:tasks.workunit.client.1.smithi088.stdout:1/819: creat d6/d6e/dfc/d108/d5b/d5d/d61/d7d/f10b x:0 0 0 2024-03-24T00:49:06.559 INFO:tasks.workunit.client.0.smithi038.stdout:5/742: symlink d6/d60/led 0 2024-03-24T00:49:06.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.558+0000 7fd459000700 1 -- 172.21.15.38:0/2533134338 --> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd430000bf0 con 0x7fd43806d620 2024-03-24T00:49:06.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.560+0000 7fd44aa00700 1 -- 172.21.15.38:0/2533134338 <== mgr.24351 v2:172.21.15.88:6828/832884416 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fd430000bf0 con 0x7fd43806d620 2024-03-24T00:49:06.561 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:49:06.561 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:49:06.561 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:49:06.561 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-24T00:49:06.561 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:49:06.561 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-03-24T00:49:06.561 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:49:06.561 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "2/2 daemons upgraded", 2024-03-24T00:49:06.561 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mgr daemons", 2024-03-24T00:49:06.562 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:49:06.562 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:49:06.562 INFO:tasks.workunit.client.1.smithi088.stdout:5/867: dwrite d1/d10/f17 [0,4194304] 0 2024-03-24T00:49:06.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.563+0000 7fd459000700 1 -- 172.21.15.38:0/2533134338 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fd43806d620 msgr2=0x7fd43806fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:06.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.563+0000 7fd459000700 1 --2- 172.21.15.38:0/2533134338 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fd43806d620 0x7fd43806fae0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd444005ea0 tx=0x7fd444005e30 comp rx=0 tx=0).stop 2024-03-24T00:49:06.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.563+0000 7fd459000700 1 -- 172.21.15.38:0/2533134338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4540ff350 msgr2=0x7fd45419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:06.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.563+0000 7fd459000700 1 --2- 172.21.15.38:0/2533134338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4540ff350 0x7fd45419cac0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd43c005250 tx=0x7fd43c00bf20 comp rx=0 tx=0).stop 2024-03-24T00:49:06.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.564+0000 7fd459000700 1 -- 172.21.15.38:0/2533134338 shutdown_connections 2024-03-24T00:49:06.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.564+0000 7fd459000700 1 --2- 172.21.15.38:0/2533134338 >> [v2:172.21.15.88:6828/832884416,v1:172.21.15.88:6829/832884416] conn(0x7fd43806d620 0x7fd43806fae0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:06.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.564+0000 7fd459000700 1 --2- 172.21.15.38:0/2533134338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4540ff350 0x7fd45419cac0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:06.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.564+0000 7fd459000700 1 --2- 172.21.15.38:0/2533134338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd454101cc0 0x7fd45419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:06.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.564+0000 7fd459000700 1 -- 172.21.15.38:0/2533134338 >> 172.21.15.38:0/2533134338 conn(0x7fd4540faf70 msgr2=0x7fd4540fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:06.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.566+0000 7fd459000700 1 -- 172.21.15.38:0/2533134338 shutdown_connections 2024-03-24T00:49:06.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:06.566+0000 7fd459000700 1 -- 172.21.15.38:0/2533134338 wait complete. 2024-03-24T00:49:06.601 INFO:tasks.workunit.client.0.smithi038.stdout:5/743: read d6/d19/d20/d42/d43/d84/fc6 [2468452,37623] 0 2024-03-24T00:49:06.601 INFO:tasks.workunit.client.0.smithi038.stdout:5/744: dread - d6/d19/da1/fac zero size 2024-03-24T00:49:06.618 INFO:tasks.workunit.client.1.smithi088.stdout:5/868: dwrite d1/d10/db6/d10b/d2f/d57/d9e/daa/fe6 [0,4194304] 0 2024-03-24T00:49:06.622 INFO:tasks.workunit.client.0.smithi038.stdout:3/775: dwrite d3/d30/fa9 [0,4194304] 0 2024-03-24T00:49:06.622 INFO:tasks.workunit.client.0.smithi038.stdout:3/776: write d3/d9/dc/da6/db5/d19/d82/fd3 [730585,97126] 0 2024-03-24T00:49:06.633 INFO:tasks.workunit.client.0.smithi038.stdout:5/745: dread d6/d1e/f63 [0,4194304] 0 2024-03-24T00:49:06.689 INFO:tasks.workunit.client.0.smithi038.stdout:5/746: dwrite d6/d16/d21/f5f [0,4194304] 0 2024-03-24T00:49:06.689 INFO:tasks.workunit.client.0.smithi038.stdout:5/747: write d6/d19/d20/d42/d43/d4c/f74 [1638693,115446] 0 2024-03-24T00:49:06.689 INFO:tasks.workunit.client.0.smithi038.stdout:5/748: chown d6/d19/d20/d42/d43/d68/dc8/fd4 1772841494 1 2024-03-24T00:49:06.729 INFO:tasks.workunit.client.0.smithi038.stdout:9/870: truncate d12/d28/d6c/fbd 84048 0 2024-03-24T00:49:06.733 INFO:tasks.workunit.client.0.smithi038.stdout:0/719: rmdir d0/d12/d42/d9d/d28 39 2024-03-24T00:49:06.733 INFO:tasks.workunit.client.0.smithi038.stdout:0/720: truncate d0/d12/d42/d5e/ff0 900806 0 2024-03-24T00:49:06.737 INFO:tasks.workunit.client.0.smithi038.stdout:2/709: mkdir d4/d18/d5b/df2 0 2024-03-24T00:49:06.739 INFO:tasks.workunit.client.0.smithi038.stdout:1/834: link da/dd/d4a/d76/l115 da/dd/d4a/d76/l116 0 2024-03-24T00:49:06.740 INFO:tasks.workunit.client.0.smithi038.stdout:1/835: write da/dd/d2c/d5a/f73 [559281,85571] 0 2024-03-24T00:49:06.742 INFO:tasks.workunit.client.0.smithi038.stdout:6/853: mknod d1/da/d1b/d1d/c11a 0 2024-03-24T00:49:06.743 INFO:tasks.workunit.client.0.smithi038.stdout:6/854: truncate d1/da/d61/d69/f6e 553736 0 2024-03-24T00:49:06.751 INFO:tasks.workunit.client.0.smithi038.stdout:0/721: write d0/d12/f68 [3129988,47409] 0 2024-03-24T00:49:06.752 INFO:tasks.workunit.client.0.smithi038.stdout:6/855: read d1/da/d1b/d2f/dc0/fad [131742,61706] 0 2024-03-24T00:49:06.753 INFO:tasks.workunit.client.0.smithi038.stdout:6/856: write d1/da/d1b/fb3 [563110,122738] 0 2024-03-24T00:49:06.753 INFO:tasks.workunit.client.0.smithi038.stdout:6/857: chown d1/da/d1b/d2f/c3e 208167 1 2024-03-24T00:49:06.753 INFO:tasks.workunit.client.0.smithi038.stdout:6/858: chown d1/da/d1b/fb8 25 1 2024-03-24T00:49:06.760 INFO:tasks.workunit.client.1.smithi088.stdout:4/812: link d5/d13/d18/f9e d5/d7/dd/d10b/f11a 0 2024-03-24T00:49:06.768 INFO:tasks.workunit.client.1.smithi088.stdout:3/780: link d3/d1a/d1e/d2f/d45/db5/d6e/l8c d3/d6/dca/dae/d4b/daf/lf8 0 2024-03-24T00:49:06.768 INFO:tasks.workunit.client.1.smithi088.stdout:3/781: readlink d3/d6/lb8 0 2024-03-24T00:49:06.773 INFO:tasks.workunit.client.1.smithi088.stdout:8/805: mknod d9/dc/d11/d24/d29/d5b/db9/da5/c111 0 2024-03-24T00:49:06.776 INFO:tasks.workunit.client.1.smithi088.stdout:6/876: rmdir db/d3d/d83 39 2024-03-24T00:49:06.778 INFO:tasks.workunit.client.0.smithi038.stdout:5/749: creat d6/d19/d20/d64/d6b/fee x:0 0 0 2024-03-24T00:49:06.780 INFO:tasks.workunit.client.1.smithi088.stdout:7/757: getdents d3/d32/d34/d9c/d4b/d74/d79/d7f 0 2024-03-24T00:49:06.782 INFO:tasks.workunit.client.1.smithi088.stdout:2/779: stat db/d21/d35/dcb/le8 0 2024-03-24T00:49:06.784 INFO:tasks.workunit.client.1.smithi088.stdout:1/820: symlink d6/d6e/dfc/d108/d5b/d80/l10c 0 2024-03-24T00:49:06.790 INFO:tasks.workunit.client.1.smithi088.stdout:4/813: mkdir d5/d13/d18/d6e/d11b 0 2024-03-24T00:49:06.791 INFO:tasks.workunit.client.1.smithi088.stdout:4/814: creat d5/d7/d92/f11c x:0 0 0 2024-03-24T00:49:06.795 INFO:tasks.workunit.client.1.smithi088.stdout:6/877: link db/fbd db/d3d/dcb/f116 0 2024-03-24T00:49:06.799 INFO:tasks.workunit.client.0.smithi038.stdout:1/836: getdents da/dd/d4a 0 2024-03-24T00:49:06.801 INFO:tasks.workunit.client.1.smithi088.stdout:1/821: stat d6/d6e/dfc/d108/d5b/d5d/d61/d7d/da5/cb5 0 2024-03-24T00:49:06.802 INFO:tasks.workunit.client.0.smithi038.stdout:0/722: mknod d0/d14/cf7 0 2024-03-24T00:49:06.802 INFO:tasks.workunit.client.0.smithi038.stdout:0/723: read d0/d12/d42/d9d/d28/f8a [8378772,5674] 0 2024-03-24T00:49:06.813 INFO:tasks.workunit.client.1.smithi088.stdout:7/758: write d3/d31/f6e [3138536,27963] 0 2024-03-24T00:49:06.816 INFO:tasks.workunit.client.0.smithi038.stdout:6/859: truncate d1/da/d1b/d1d/f24 2182360 0 2024-03-24T00:49:06.816 INFO:tasks.workunit.client.0.smithi038.stdout:6/860: creat d1/da/d61/d69/f11b x:0 0 0 2024-03-24T00:49:06.817 INFO:tasks.workunit.client.1.smithi088.stdout:6/878: mknod db/d3d/d83/d72/c117 0 2024-03-24T00:49:06.817 INFO:tasks.workunit.client.1.smithi088.stdout:6/879: chown db/dc5/dc8/fee 6604 1 2024-03-24T00:49:06.817 INFO:tasks.workunit.client.1.smithi088.stdout:6/880: creat db/d1d/d50/f118 x:0 0 0 2024-03-24T00:49:06.817 INFO:tasks.workunit.client.1.smithi088.stdout:6/881: chown db/d8e/d10d 3 1 2024-03-24T00:49:06.817 INFO:tasks.workunit.client.1.smithi088.stdout:6/882: chown db/dd/f41 14847 1 2024-03-24T00:49:06.820 INFO:tasks.workunit.client.1.smithi088.stdout:2/780: symlink db/d21/d35/lf9 0 2024-03-24T00:49:06.825 INFO:tasks.workunit.client.0.smithi038.stdout:5/750: dread d6/d1e/f55 [0,4194304] 0 2024-03-24T00:49:06.837 INFO:tasks.workunit.client.1.smithi088.stdout:4/815: rmdir d5/d16/d37 39 2024-03-24T00:49:06.837 INFO:tasks.workunit.client.1.smithi088.stdout:4/816: chown d5/d17/f1d 2073081 1 2024-03-24T00:49:06.837 INFO:tasks.workunit.client.1.smithi088.stdout:4/817: dread - d5/d13/fa5 zero size 2024-03-24T00:49:06.843 INFO:tasks.workunit.client.0.smithi038.stdout:1/837: mkdir da/dd/d2c/d40/de9/d117 0 2024-03-24T00:49:06.849 INFO:tasks.workunit.client.1.smithi088.stdout:2/781: write db/d21/d35/d49/fd8 [2989058,12440] 0 2024-03-24T00:49:06.849 INFO:tasks.workunit.client.1.smithi088.stdout:1/822: mkdir d6/d27/d10d 0 2024-03-24T00:49:06.855 INFO:tasks.workunit.client.1.smithi088.stdout:8/806: dread d9/dc/dce/ddc/fe9 [4194304,4194304] 0 2024-03-24T00:49:06.855 INFO:tasks.workunit.client.1.smithi088.stdout:8/807: chown d9/dc/d11/d24/d29/d5b/c75 9 1 2024-03-24T00:49:06.856 INFO:tasks.workunit.client.1.smithi088.stdout:8/808: write d9/dc/d11/d24/d29/d39/d84/faa [879562,79949] 0 2024-03-24T00:49:06.862 INFO:tasks.workunit.client.1.smithi088.stdout:6/883: write db/dd/d61/d6d/d74/d88/f9b [541821,57913] 0 2024-03-24T00:49:06.865 INFO:tasks.workunit.client.0.smithi038.stdout:1/838: creat da/d3c/d86/def/f118 x:0 0 0 2024-03-24T00:49:06.865 INFO:tasks.workunit.client.0.smithi038.stdout:0/724: dwrite d0/d1b/f20 [0,4194304] 0 2024-03-24T00:49:06.882 INFO:tasks.workunit.client.0.smithi038.stdout:8/822: mknod d0/d15/c114 0 2024-03-24T00:49:06.882 INFO:tasks.workunit.client.0.smithi038.stdout:0/725: symlink d0/d12/d15/d58/lf8 0 2024-03-24T00:49:06.882 INFO:tasks.workunit.client.0.smithi038.stdout:0/726: chown d0/d12/d42/d9d/d28/c5d 111 1 2024-03-24T00:49:06.893 INFO:tasks.workunit.client.0.smithi038.stdout:9/871: unlink d12/d28/d9c/ddd/d111/fef 0 2024-03-24T00:49:06.894 INFO:tasks.workunit.client.1.smithi088.stdout:1/823: getdents d6/d6e/dfc/d108/d58 0 2024-03-24T00:49:06.894 INFO:tasks.workunit.client.1.smithi088.stdout:8/809: symlink d9/dc/d11/d24/d29/d36/d6a/l112 0 2024-03-24T00:49:06.894 INFO:tasks.workunit.client.1.smithi088.stdout:8/810: fsync d9/dc/fc3 0 2024-03-24T00:49:06.895 INFO:tasks.workunit.client.1.smithi088.stdout:6/884: stat db/dd/d61/d6d/ld6 0 2024-03-24T00:49:06.906 INFO:tasks.workunit.client.0.smithi038.stdout:9/872: rename d12/d41/dc0/fd1 to d12/d28/d9c/ddd/d111/d8f/f116 0 2024-03-24T00:49:06.906 INFO:tasks.workunit.client.0.smithi038.stdout:9/873: fdatasync d12/d55/f9b 0 2024-03-24T00:49:06.916 INFO:tasks.workunit.client.0.smithi038.stdout:9/874: rename d12/d81/d84/f8e to d12/d1a/d5f/df9/f117 0 2024-03-24T00:49:06.922 INFO:tasks.workunit.client.1.smithi088.stdout:6/885: dread db/dc5/dc8/fee [0,4194304] 0 2024-03-24T00:49:06.923 INFO:tasks.workunit.client.0.smithi038.stdout:2/710: unlink d4/f10 0 2024-03-24T00:49:06.929 INFO:tasks.workunit.client.1.smithi088.stdout:2/782: dwrite db/dd/d96/d4e/d8f/dad/fd7 [0,4194304] 0 2024-03-24T00:49:06.930 INFO:tasks.workunit.client.0.smithi038.stdout:6/861: dwrite d1/da/d1b/d1d/fd5 [0,4194304] 0 2024-03-24T00:49:06.931 INFO:tasks.workunit.client.0.smithi038.stdout:6/862: truncate d1/da/d1b/d1d/f94 498411 0 2024-03-24T00:49:06.931 INFO:tasks.workunit.client.0.smithi038.stdout:0/727: dread d0/d12/d15/d58/d64/da7/dbb/dd4/dea/fb3 [0,4194304] 0 2024-03-24T00:49:06.933 INFO:tasks.workunit.client.1.smithi088.stdout:6/886: unlink db/d1d/d50/d90/d56/l100 0 2024-03-24T00:49:06.935 INFO:tasks.workunit.client.0.smithi038.stdout:1/839: dwrite da/dd/d4a/d76/f36 [0,4194304] 0 2024-03-24T00:49:06.938 INFO:tasks.workunit.client.0.smithi038.stdout:5/751: dwrite d6/d1e/f1f [0,4194304] 0 2024-03-24T00:49:06.939 INFO:tasks.workunit.client.0.smithi038.stdout:5/752: write d6/d1e/f55 [2136915,45895] 0 2024-03-24T00:49:06.939 INFO:tasks.workunit.client.0.smithi038.stdout:5/753: chown d6/d16/d21/f5f 922224899 1 2024-03-24T00:49:06.942 INFO:tasks.workunit.client.0.smithi038.stdout:6/863: symlink d1/da/d1b/d25/d49/d4e/dbe/l11c 0 2024-03-24T00:49:06.947 INFO:tasks.workunit.client.1.smithi088.stdout:7/759: dwrite d3/d32/d34/d9c/d4b/d74/d79/fb0 [0,4194304] 0 2024-03-24T00:49:06.950 INFO:tasks.workunit.client.0.smithi038.stdout:0/728: getdents d0/d12/d15/d58/d64/da7 0 2024-03-24T00:49:06.950 INFO:tasks.workunit.client.0.smithi038.stdout:0/729: chown d0/d1b/d2c/c4e 29 1 2024-03-24T00:49:06.954 INFO:tasks.workunit.client.1.smithi088.stdout:3/782: dwrite d3/d6/d27/de9/fd2 [0,4194304] 0 2024-03-24T00:49:06.960 INFO:tasks.workunit.client.1.smithi088.stdout:6/887: creat db/dd/d61/d6d/d74/d88/da1/f119 x:0 0 0 2024-03-24T00:49:06.965 INFO:tasks.workunit.client.0.smithi038.stdout:1/840: dread da/dd/d7a/d82/d98/d9e/fb0 [0,4194304] 0 2024-03-24T00:49:06.965 INFO:tasks.workunit.client.0.smithi038.stdout:1/841: write da/dd/d4a/d8b/fc7 [3459253,115158] 0 2024-03-24T00:49:06.965 INFO:tasks.workunit.client.0.smithi038.stdout:5/754: symlink d6/d19/da1/lef 0 2024-03-24T00:49:06.973 INFO:tasks.workunit.client.0.smithi038.stdout:8/823: dwrite d0/d45/d37/d5f/db7/d62/fad [4194304,4194304] 0 2024-03-24T00:49:06.978 INFO:tasks.workunit.client.0.smithi038.stdout:8/824: rmdir d0/d45/d37/d5f 39 2024-03-24T00:49:06.979 INFO:tasks.workunit.client.1.smithi088.stdout:1/824: dwrite d6/d6e/dfc/d108/d5b/d80/fd4 [0,4194304] 0 2024-03-24T00:49:06.979 INFO:tasks.workunit.client.0.smithi038.stdout:8/825: write d0/d1d/d27/d4d/daf/fed [1327687,73429] 0 2024-03-24T00:49:06.981 INFO:tasks.workunit.client.0.smithi038.stdout:5/755: mknod d6/cf0 0 2024-03-24T00:49:06.981 INFO:tasks.workunit.client.0.smithi038.stdout:5/756: chown d6/d19/d20/d42/d43/d50/d92 27 1 2024-03-24T00:49:06.981 INFO:tasks.workunit.client.0.smithi038.stdout:0/730: write d0/d1b/f20 [3174334,86146] 0 2024-03-24T00:49:06.981 INFO:tasks.workunit.client.0.smithi038.stdout:0/731: creat d0/d14/d84/ff9 x:0 0 0 2024-03-24T00:49:06.982 INFO:tasks.workunit.client.1.smithi088.stdout:7/760: dread d3/d31/d56/f7b [0,4194304] 0 2024-03-24T00:49:06.985 INFO:tasks.workunit.client.0.smithi038.stdout:2/711: dwrite d4/d18/f28 [0,4194304] 0 2024-03-24T00:49:06.990 INFO:tasks.workunit.client.1.smithi088.stdout:8/811: dwrite d9/dc/d11/d24/f58 [0,4194304] 0 2024-03-24T00:49:06.991 INFO:tasks.workunit.client.0.smithi038.stdout:5/757: symlink d6/d19/d20/d33/lf1 0 2024-03-24T00:49:06.991 INFO:tasks.workunit.client.0.smithi038.stdout:5/758: write d6/d16/f2a [2790993,5945] 0 2024-03-24T00:49:06.991 INFO:tasks.workunit.client.0.smithi038.stdout:1/842: read da/ddc/f10f [1897531,75092] 0 2024-03-24T00:49:07.006 INFO:tasks.workunit.client.1.smithi088.stdout:1/825: readlink d6/d6e/dfc/d108/d5b/l91 0 2024-03-24T00:49:07.009 INFO:tasks.workunit.client.0.smithi038.stdout:1/843: creat da/dd/d7a/d82/d98/d14/dc0/f119 x:0 0 0 2024-03-24T00:49:07.010 INFO:tasks.workunit.client.0.smithi038.stdout:0/732: dread d0/d12/d42/d9d/f90 [0,4194304] 0 2024-03-24T00:49:07.020 INFO:tasks.workunit.client.1.smithi088.stdout:9/808: rename d1/d10/l65 to d1/d10/d29/def/l105 0 2024-03-24T00:49:07.020 INFO:tasks.workunit.client.1.smithi088.stdout:9/809: chown d1/d22/d72/l88 63714291 1 2024-03-24T00:49:07.021 INFO:tasks.workunit.client.0.smithi038.stdout:1/844: mkdir da/dd/d7a/d82/ddb/d11a 0 2024-03-24T00:49:07.022 INFO:tasks.workunit.client.0.smithi038.stdout:1/845: dread - da/d3c/d86/fb3 zero size 2024-03-24T00:49:07.022 INFO:tasks.workunit.client.0.smithi038.stdout:1/846: chown da/d3c/l3b 1061 1 2024-03-24T00:49:07.025 INFO:tasks.workunit.client.1.smithi088.stdout:4/818: rmdir d5/dac 39 2024-03-24T00:49:07.028 INFO:tasks.workunit.client.0.smithi038.stdout:0/733: creat d0/d12/d15/d58/d64/ffa x:0 0 0 2024-03-24T00:49:07.029 INFO:tasks.workunit.client.1.smithi088.stdout:7/761: unlink d3/l64 0 2024-03-24T00:49:07.029 INFO:tasks.workunit.client.1.smithi088.stdout:7/762: chown d3/ff 6922872 1 2024-03-24T00:49:07.029 INFO:tasks.workunit.client.1.smithi088.stdout:7/763: readlink d3/d32/l49 0 2024-03-24T00:49:07.029 INFO:tasks.workunit.client.1.smithi088.stdout:7/764: chown d3/d31/cad 1 1 2024-03-24T00:49:07.051 INFO:tasks.workunit.client.1.smithi088.stdout:2/783: dwrite db/dd/d96/d99/db2/f65 [0,4194304] 0 2024-03-24T00:49:07.051 INFO:tasks.workunit.client.1.smithi088.stdout:2/784: read db/d21/d35/d49/f71 [737065,82067] 0 2024-03-24T00:49:07.063 INFO:tasks.workunit.client.1.smithi088.stdout:5/869: rename d1/d10/db6/d10b/d2f/d93/ce8 to d1/d43/d4e/df8/c125 0 2024-03-24T00:49:07.063 INFO:tasks.workunit.client.1.smithi088.stdout:5/870: write d1/d10/db6/d10b/d2f/d57/d9e/dba/ff2 [7239930,43700] 0 2024-03-24T00:49:07.063 INFO:tasks.workunit.client.1.smithi088.stdout:5/871: dread - d1/d10/db6/d10b/d2f/d57/d9e/f11d zero size 2024-03-24T00:49:07.063 INFO:tasks.workunit.client.1.smithi088.stdout:5/872: chown d1/cd3 7985 1 2024-03-24T00:49:07.067 INFO:tasks.workunit.client.1.smithi088.stdout:7/765: truncate d3/d32/d76/f87 736833 0 2024-03-24T00:49:07.067 INFO:tasks.workunit.client.1.smithi088.stdout:7/766: chown d3/d31/fc7 18300 1 2024-03-24T00:49:07.075 INFO:tasks.workunit.client.1.smithi088.stdout:0/843: link d9/dc/de/d7e/dce/fe1 d9/f10d 0 2024-03-24T00:49:07.075 INFO:tasks.workunit.client.1.smithi088.stdout:0/844: fsync d9/dc/de/d7e/d91/fee 0 2024-03-24T00:49:07.075 INFO:tasks.workunit.client.1.smithi088.stdout:2/785: mkdir db/dd/d96/d99/dfa 0 2024-03-24T00:49:07.078 INFO:tasks.workunit.client.0.smithi038.stdout:8/826: dwrite d0/d15/fbd [0,4194304] 0 2024-03-24T00:49:07.083 INFO:tasks.workunit.client.1.smithi088.stdout:9/810: dwrite d1/d22/f8c [0,4194304] 0 2024-03-24T00:49:07.083 INFO:tasks.workunit.client.1.smithi088.stdout:9/811: fsync d1/d22/d27/d39/d98/da3/fe1 0 2024-03-24T00:49:07.088 INFO:tasks.workunit.client.0.smithi038.stdout:5/759: dwrite d6/d19/d20/d70/fcd [0,4194304] 0 2024-03-24T00:49:07.089 INFO:tasks.workunit.client.1.smithi088.stdout:7/767: dread d3/d32/d34/d9c/d4b/f6c [0,4194304] 0 2024-03-24T00:49:07.089 INFO:tasks.workunit.client.1.smithi088.stdout:8/812: rename d9/dc/d11/d24/d29/d73/lc2 to d9/dc/d11/dd2/l113 0 2024-03-24T00:49:07.089 INFO:tasks.workunit.client.1.smithi088.stdout:0/845: rename d9/dc to d9/dc/de/d7e/d10e 22 2024-03-24T00:49:07.090 INFO:tasks.workunit.client.1.smithi088.stdout:1/826: dwrite d6/d6e/dfc/d108/d3f/d53/faa [0,4194304] 0 2024-03-24T00:49:07.091 INFO:tasks.workunit.client.1.smithi088.stdout:4/819: dwrite d5/d7/f112 [0,4194304] 0 2024-03-24T00:49:07.093 INFO:tasks.workunit.client.1.smithi088.stdout:5/873: link d1/dd/d6f/lf6 d1/d10/d4f/l126 0 2024-03-24T00:49:07.093 INFO:tasks.workunit.client.1.smithi088.stdout:2/786: mknod db/dd/cfb 0 2024-03-24T00:49:07.094 INFO:tasks.workunit.client.0.smithi038.stdout:6/864: rename d1/da/d61/fee to d1/da/d73/f11d 0 2024-03-24T00:49:07.095 INFO:tasks.workunit.client.0.smithi038.stdout:6/865: write d1/da/d1b/d2f/dc0/fc7 [5446391,8075] 0 2024-03-24T00:49:07.096 INFO:tasks.workunit.client.0.smithi038.stdout:7/722: sync 2024-03-24T00:49:07.096 INFO:tasks.workunit.client.0.smithi038.stdout:7/723: write d2/d9/da/dc/d4a/f6d [3240699,72450] 0 2024-03-24T00:49:07.096 INFO:tasks.workunit.client.0.smithi038.stdout:7/724: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/d5d/fda 0 2024-03-24T00:49:07.097 INFO:tasks.workunit.client.1.smithi088.stdout:8/813: truncate d9/dc/d11/d24/d29/f2c 2911089 0 2024-03-24T00:49:07.097 INFO:tasks.workunit.client.1.smithi088.stdout:8/814: write d9/dc/d11/d24/d29/d5b/db9/fc5 [427325,37813] 0 2024-03-24T00:49:07.097 INFO:tasks.workunit.client.1.smithi088.stdout:8/815: fdatasync d9/dc/d11/d24/d29/d39/f93 0 2024-03-24T00:49:07.101 INFO:tasks.workunit.client.0.smithi038.stdout:2/712: rename d4/d27/d54/d53/de2/caf to d4/d55/d88/cf3 0 2024-03-24T00:49:07.103 INFO:tasks.workunit.client.0.smithi038.stdout:5/760: write d6/d19/d20/d42/d43/d68/f8b [1816421,93565] 0 2024-03-24T00:49:07.103 INFO:tasks.workunit.client.0.smithi038.stdout:9/875: getdents d12/d28/d9c/ddd/d111/d8f 0 2024-03-24T00:49:07.104 INFO:tasks.workunit.client.0.smithi038.stdout:9/876: truncate d12/d41/d57/fa0 473456 0 2024-03-24T00:49:07.107 INFO:tasks.workunit.client.0.smithi038.stdout:7/725: mknod d2/d9/da/d23/def/df0/c10c 0 2024-03-24T00:49:07.107 INFO:tasks.workunit.client.0.smithi038.stdout:7/726: write d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fd5 [473380,72575] 0 2024-03-24T00:49:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:06 smithi038 ceph-mon[29271]: from='client.24423 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:06 smithi038 ceph-mon[29271]: pgmap v39: 65 pgs: 65 active+clean; 3.1 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 219 MiB/s rd, 246 MiB/s wr, 378 op/s 2024-03-24T00:49:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:06 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/3827082009' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:49:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:06 smithi038 ceph-mon[29271]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:06 smithi038 ceph-mon[29271]: mgrmap e26: smithi088.njctea(active, since 73s), standbys: smithi038.huhioz 2024-03-24T00:49:07.109 INFO:tasks.workunit.client.0.smithi038.stdout:8/827: rename d0/c5 to d0/d1d/d27/d4d/daf/dea/c115 0 2024-03-24T00:49:07.110 INFO:tasks.workunit.client.1.smithi088.stdout:1/827: dread d6/d6e/dfc/d108/f39 [0,4194304] 0 2024-03-24T00:49:07.110 INFO:tasks.workunit.client.1.smithi088.stdout:1/828: getdents d6/d27/d103 0 2024-03-24T00:49:07.111 INFO:tasks.workunit.client.1.smithi088.stdout:4/820: creat d5/d17/f11d x:0 0 0 2024-03-24T00:49:07.112 INFO:tasks.workunit.client.0.smithi038.stdout:3/777: sync 2024-03-24T00:49:07.112 INFO:tasks.workunit.client.0.smithi038.stdout:4/739: sync 2024-03-24T00:49:07.112 INFO:tasks.workunit.client.0.smithi038.stdout:3/778: fdatasync d3/d9/dc/da6/db5/d19/f42 0 2024-03-24T00:49:07.112 INFO:tasks.workunit.client.0.smithi038.stdout:5/761: symlink d6/d16/d21/lf2 0 2024-03-24T00:49:07.114 INFO:tasks.workunit.client.0.smithi038.stdout:3/779: write d3/d5a/f73 [158880,12923] 0 2024-03-24T00:49:07.114 INFO:tasks.workunit.client.1.smithi088.stdout:5/874: mknod d1/d10/c127 0 2024-03-24T00:49:07.114 INFO:tasks.workunit.client.1.smithi088.stdout:5/875: stat d1/d10/db6/d10b/d2f/d57/d9e/daa/f110 0 2024-03-24T00:49:07.114 INFO:tasks.workunit.client.1.smithi088.stdout:5/876: stat d1/dd/d6f/da1/d3f/d88/d8f/d69/f82 0 2024-03-24T00:49:07.115 INFO:tasks.workunit.client.1.smithi088.stdout:6/888: creat db/dd/d61/f11a x:0 0 0 2024-03-24T00:49:07.115 INFO:tasks.workunit.client.1.smithi088.stdout:6/889: write db/d1d/d50/d90/d55/d99/fa6 [797087,6796] 0 2024-03-24T00:49:07.116 INFO:tasks.workunit.client.1.smithi088.stdout:0/846: read d9/dc/de/d19/d44/d46/d65/d2a/f92 [126947,9701] 0 2024-03-24T00:49:07.117 INFO:tasks.workunit.client.1.smithi088.stdout:9/812: dread d1/d10/d2c/ff2 [0,4194304] 0 2024-03-24T00:49:07.120 INFO:tasks.workunit.client.0.smithi038.stdout:9/877: mkdir d12/d81/d84/dec/d118 0 2024-03-24T00:49:07.123 INFO:tasks.workunit.client.1.smithi088.stdout:2/787: creat db/dd/d96/d4e/d8f/d9c/db0/ffc x:0 0 0 2024-03-24T00:49:07.130 INFO:tasks.workunit.client.0.smithi038.stdout:4/740: write da/d12/d24/d5f/d8a/fc4 [974074,75113] 0 2024-03-24T00:49:07.130 INFO:tasks.workunit.client.0.smithi038.stdout:4/741: read - da/d12/d37/dd/f8d zero size 2024-03-24T00:49:07.130 INFO:tasks.workunit.client.0.smithi038.stdout:4/742: truncate da/d12/d24/d68/d6c/da6/fd5 55802 0 2024-03-24T00:49:07.130 INFO:tasks.workunit.client.0.smithi038.stdout:4/743: chown da/d12/d37/dd/d15/d6d/ldf 1168593 1 2024-03-24T00:49:07.130 INFO:tasks.workunit.client.0.smithi038.stdout:4/744: stat da/d12/d29/d5b/d73/fba 0 2024-03-24T00:49:07.131 INFO:tasks.workunit.client.1.smithi088.stdout:4/821: read d5/d7/d72/f73 [2787096,123201] 0 2024-03-24T00:49:07.136 INFO:tasks.workunit.client.0.smithi038.stdout:2/713: rename d4/d27/ddb/f6e to d4/dc7/ff4 0 2024-03-24T00:49:07.137 INFO:tasks.workunit.client.1.smithi088.stdout:1/829: dread d6/d6e/dfc/d108/d5b/d5d/d61/fdb [0,4194304] 0 2024-03-24T00:49:07.137 INFO:tasks.workunit.client.1.smithi088.stdout:1/830: chown d6/d6e/dfc/d108/d58/fbe 26 1 2024-03-24T00:49:07.138 INFO:tasks.workunit.client.1.smithi088.stdout:8/816: symlink d9/dc/dce/ddc/l114 0 2024-03-24T00:49:07.138 INFO:tasks.workunit.client.1.smithi088.stdout:7/768: dwrite d3/d32/d34/faa [0,4194304] 0 2024-03-24T00:49:07.138 INFO:tasks.workunit.client.1.smithi088.stdout:8/817: creat d9/dc/d11/d24/d29/da9/f115 x:0 0 0 2024-03-24T00:49:07.141 INFO:tasks.workunit.client.1.smithi088.stdout:5/877: creat d1/d19/d48/dc2/dcc/f128 x:0 0 0 2024-03-24T00:49:07.143 INFO:tasks.workunit.client.1.smithi088.stdout:6/890: symlink db/d3d/l11b 0 2024-03-24T00:49:07.145 INFO:tasks.workunit.client.0.smithi038.stdout:7/727: symlink d2/d9/da/d23/d2b/d5b/d72/d92/d75/l10d 0 2024-03-24T00:49:07.149 INFO:tasks.workunit.client.1.smithi088.stdout:9/813: creat d1/f106 x:0 0 0 2024-03-24T00:49:07.150 INFO:tasks.workunit.client.0.smithi038.stdout:4/745: unlink da/d12/d37/dd/d15/d6d/f4b 0 2024-03-24T00:49:07.151 INFO:tasks.workunit.client.0.smithi038.stdout:5/762: rename d6/d19/d20/d33/d6f/fd3 to d6/d19/d20/d70/ff3 0 2024-03-24T00:49:07.151 INFO:tasks.workunit.client.0.smithi038.stdout:5/763: write d6/d19/f1c [2390263,97264] 0 2024-03-24T00:49:07.151 INFO:tasks.workunit.client.0.smithi038.stdout:5/764: readlink d6/d19/da1/lef 0 2024-03-24T00:49:07.151 INFO:tasks.workunit.client.0.smithi038.stdout:5/765: dread - d6/d19/d20/d33/fdb zero size 2024-03-24T00:49:07.152 INFO:tasks.workunit.client.0.smithi038.stdout:5/766: write d6/d19/d20/d42/d43/d68/f8b [4626251,95248] 0 2024-03-24T00:49:07.152 INFO:tasks.workunit.client.0.smithi038.stdout:5/767: creat d6/d19/d20/d42/d43/d50/ff4 x:0 0 0 2024-03-24T00:49:07.158 INFO:tasks.workunit.client.1.smithi088.stdout:4/822: rename d5/d7/fe to d5/d13/d63/d78/f11e 0 2024-03-24T00:49:07.159 INFO:tasks.workunit.client.1.smithi088.stdout:4/823: fsync d5/d16/f7b 0 2024-03-24T00:49:07.164 INFO:tasks.workunit.client.0.smithi038.stdout:7/728: rename d2/d9/c4e to d2/d2c/d9a/deb/c10e 0 2024-03-24T00:49:07.164 INFO:tasks.workunit.client.0.smithi038.stdout:7/729: fdatasync d2/d9/da/f73 0 2024-03-24T00:49:07.166 INFO:tasks.workunit.client.0.smithi038.stdout:7/730: truncate d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/fb6 4350138 0 2024-03-24T00:49:07.167 INFO:tasks.workunit.client.0.smithi038.stdout:7/731: dread - d2/d9/da/d23/d2b/d5b/d72/d92/d33/f27 zero size 2024-03-24T00:49:07.167 INFO:tasks.workunit.client.1.smithi088.stdout:1/831: truncate d6/f3e 1228051 0 2024-03-24T00:49:07.169 INFO:tasks.workunit.client.1.smithi088.stdout:6/891: dread db/d1d/d50/f77 [0,4194304] 0 2024-03-24T00:49:07.171 INFO:tasks.workunit.client.0.smithi038.stdout:6/866: dwrite d1/da/feb [0,4194304] 0 2024-03-24T00:49:07.173 INFO:tasks.workunit.client.1.smithi088.stdout:8/818: read d9/dc/d11/d24/d29/d36/f86 [2947418,92253] 0 2024-03-24T00:49:07.173 INFO:tasks.workunit.client.0.smithi038.stdout:5/768: mkdir d6/d19/d20/d42/d43/d68/dc8/df5 0 2024-03-24T00:49:07.174 INFO:tasks.workunit.client.1.smithi088.stdout:0/847: dread d9/dc/f52 [0,4194304] 0 2024-03-24T00:49:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:06 smithi088 ceph-mon[36610]: from='client.24423 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:06 smithi088 ceph-mon[36610]: pgmap v39: 65 pgs: 65 active+clean; 3.1 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 219 MiB/s rd, 246 MiB/s wr, 378 op/s 2024-03-24T00:49:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:06 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/3827082009' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:49:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:06 smithi088 ceph-mon[36610]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:06 smithi088 ceph-mon[36610]: mgrmap e26: smithi088.njctea(active, since 73s), standbys: smithi038.huhioz 2024-03-24T00:49:07.190 INFO:tasks.workunit.client.0.smithi038.stdout:5/769: mkdir d6/d16/d67/df6 0 2024-03-24T00:49:07.190 INFO:tasks.workunit.client.0.smithi038.stdout:5/770: creat d6/d19/d20/d33/ff7 x:0 0 0 2024-03-24T00:49:07.194 INFO:tasks.workunit.client.1.smithi088.stdout:5/878: rmdir d1/d10/db6/d10b/d2f 39 2024-03-24T00:49:07.196 INFO:tasks.workunit.client.1.smithi088.stdout:7/769: dwrite d3/f2c [0,4194304] 0 2024-03-24T00:49:07.198 INFO:tasks.workunit.client.1.smithi088.stdout:7/770: write d3/d32/d34/d9c/d4b/d74/d79/d7f/fb2 [1376513,66025] 0 2024-03-24T00:49:07.200 INFO:tasks.workunit.client.1.smithi088.stdout:2/788: unlink db/dd/d96/d4e/lc8 0 2024-03-24T00:49:07.200 INFO:tasks.workunit.client.1.smithi088.stdout:2/789: fdatasync db/dd4/fec 0 2024-03-24T00:49:07.200 INFO:tasks.workunit.client.1.smithi088.stdout:2/790: read - db/dd/d96/d99/db2/d88/d8e/f9b zero size 2024-03-24T00:49:07.200 INFO:tasks.workunit.client.0.smithi038.stdout:2/714: dread d4/d27/f39 [0,4194304] 0 2024-03-24T00:49:07.201 INFO:tasks.workunit.client.0.smithi038.stdout:3/780: dwrite d3/d9/dc/da6/db5/d19/da4/ff5 [0,4194304] 0 2024-03-24T00:49:07.201 INFO:tasks.workunit.client.0.smithi038.stdout:3/781: fsync d3/d9/d77/f7f 0 2024-03-24T00:49:07.201 INFO:tasks.workunit.client.0.smithi038.stdout:3/782: chown d3/d9/dc/da6/db5/d19/dd2/df0 256050149 1 2024-03-24T00:49:07.203 INFO:tasks.workunit.client.1.smithi088.stdout:4/824: creat d5/d7/de4/f11f x:0 0 0 2024-03-24T00:49:07.207 INFO:tasks.workunit.client.0.smithi038.stdout:6/867: dread d1/da/d1b/d25/d49/d4e/d51/f9c [0,4194304] 0 2024-03-24T00:49:07.209 INFO:tasks.workunit.client.0.smithi038.stdout:4/746: dwrite da/d12/ff7 [0,4194304] 0 2024-03-24T00:49:07.209 INFO:tasks.workunit.client.0.smithi038.stdout:4/747: readlink da/d12/d37/d83/dac/lcb 0 2024-03-24T00:49:07.211 INFO:tasks.workunit.client.1.smithi088.stdout:1/832: mkdir d6/d27/db0/d10e 0 2024-03-24T00:49:07.211 INFO:tasks.workunit.client.0.smithi038.stdout:9/878: dwrite d12/d28/f4c [0,4194304] 0 2024-03-24T00:49:07.211 INFO:tasks.workunit.client.0.smithi038.stdout:9/879: fsync d12/d81/d84/dec/ff6 0 2024-03-24T00:49:07.214 INFO:tasks.workunit.client.1.smithi088.stdout:6/892: truncate db/fbd 1957080 0 2024-03-24T00:49:07.218 INFO:tasks.workunit.client.1.smithi088.stdout:8/819: symlink d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/l116 0 2024-03-24T00:49:07.221 INFO:tasks.workunit.client.1.smithi088.stdout:0/848: creat d9/d26/d8f/da1/d62/f10f x:0 0 0 2024-03-24T00:49:07.221 INFO:tasks.workunit.client.1.smithi088.stdout:0/849: chown d9/d26/d8f/da1/d62/f9d 509 1 2024-03-24T00:49:07.221 INFO:tasks.workunit.client.1.smithi088.stdout:0/850: fsync d9/dc/de/f3e 0 2024-03-24T00:49:07.222 INFO:tasks.workunit.client.0.smithi038.stdout:6/868: dread d1/da/d1b/d25/d49/d4e/f66 [0,4194304] 0 2024-03-24T00:49:07.223 INFO:tasks.workunit.client.0.smithi038.stdout:3/783: mknod d3/d2a/d107/c10b 0 2024-03-24T00:49:07.223 INFO:tasks.workunit.client.0.smithi038.stdout:3/784: chown d3/d30/d5d/deb 4599434 1 2024-03-24T00:49:07.223 INFO:tasks.workunit.client.0.smithi038.stdout:6/869: write d1/da/d61/d69/fd4 [461642,68726] 0 2024-03-24T00:49:07.223 INFO:tasks.workunit.client.0.smithi038.stdout:6/870: write d1/da/d1b/d1d/f4f [516649,68301] 0 2024-03-24T00:49:07.223 INFO:tasks.workunit.client.0.smithi038.stdout:6/871: dread - d1/d37/ff4 zero size 2024-03-24T00:49:07.223 INFO:tasks.workunit.client.0.smithi038.stdout:6/872: chown d1/da/d1b/d25/d49/l74 56063477 1 2024-03-24T00:49:07.226 INFO:tasks.workunit.client.1.smithi088.stdout:5/879: truncate d1/fa 7664655 0 2024-03-24T00:49:07.231 INFO:tasks.workunit.client.1.smithi088.stdout:5/880: write d1/d10/d79/dbc/f119 [332579,113821] 0 2024-03-24T00:49:07.231 INFO:tasks.workunit.client.1.smithi088.stdout:5/881: fdatasync d1/d10/db6/d10b/d2f/d57/d9e/dd0/f11e 0 2024-03-24T00:49:07.233 INFO:tasks.workunit.client.1.smithi088.stdout:9/814: rmdir d1/d15/d84 39 2024-03-24T00:49:07.233 INFO:tasks.workunit.client.1.smithi088.stdout:9/815: stat d1/d15/d80/ca1 0 2024-03-24T00:49:07.237 INFO:tasks.workunit.client.1.smithi088.stdout:7/771: readlink d3/l2e 0 2024-03-24T00:49:07.237 INFO:tasks.workunit.client.1.smithi088.stdout:7/772: fdatasync d3/d31/f83 0 2024-03-24T00:49:07.243 INFO:tasks.workunit.client.1.smithi088.stdout:2/791: mknod db/dd/d96/cfd 0 2024-03-24T00:49:07.251 INFO:tasks.workunit.client.1.smithi088.stdout:4/825: unlink d5/d7/d72/fea 0 2024-03-24T00:49:07.255 INFO:tasks.workunit.client.1.smithi088.stdout:1/833: creat d6/d27/d67/f10f x:0 0 0 2024-03-24T00:49:07.255 INFO:tasks.workunit.client.1.smithi088.stdout:1/834: chown d6/fb4 186 1 2024-03-24T00:49:07.258 INFO:tasks.workunit.client.1.smithi088.stdout:6/893: unlink db/dc5/dc8/d43/f9a 0 2024-03-24T00:49:07.263 INFO:tasks.workunit.client.1.smithi088.stdout:8/820: mknod d9/dc/d11/d24/d29/da9/d9c/c117 0 2024-03-24T00:49:07.267 INFO:tasks.workunit.client.1.smithi088.stdout:0/851: mknod d9/d26/d8f/da1/d62/c110 0 2024-03-24T00:49:07.274 INFO:tasks.workunit.client.0.smithi038.stdout:3/785: dwrite d3/d9/dc/da6/db5/d19/da4/ff5 [0,4194304] 0 2024-03-24T00:49:07.277 INFO:tasks.workunit.client.0.smithi038.stdout:9/880: mknod d12/d81/d84/c119 0 2024-03-24T00:49:07.277 INFO:tasks.workunit.client.0.smithi038.stdout:9/881: truncate d12/d28/d6c/dbc/fe7 336992 0 2024-03-24T00:49:07.277 INFO:tasks.workunit.client.0.smithi038.stdout:9/882: chown d12/d28/d71/d99/c110 233 1 2024-03-24T00:49:07.280 INFO:tasks.workunit.client.1.smithi088.stdout:5/882: symlink d1/dd/d6f/da1/d3f/d88/d8f/d69/l129 0 2024-03-24T00:49:07.289 INFO:tasks.workunit.client.0.smithi038.stdout:9/883: dread d12/d28/d9c/ddd/d111/d26/f3d [4194304,4194304] 0 2024-03-24T00:49:07.290 INFO:tasks.workunit.client.1.smithi088.stdout:9/816: dwrite d1/f106 [0,4194304] 0 2024-03-24T00:49:07.291 INFO:tasks.workunit.client.1.smithi088.stdout:2/792: rmdir db/d21/d35/dcb 39 2024-03-24T00:49:07.292 INFO:tasks.workunit.client.0.smithi038.stdout:9/884: mkdir d12/d28/d71/d11a 0 2024-03-24T00:49:07.293 INFO:tasks.workunit.client.0.smithi038.stdout:8/828: write d0/d1d/f22 [887028,98972] 0 2024-03-24T00:49:07.294 INFO:tasks.workunit.client.1.smithi088.stdout:0/852: write d9/ffe [2782464,109893] 0 2024-03-24T00:49:07.306 INFO:tasks.workunit.client.1.smithi088.stdout:6/894: dwrite db/d8e/f94 [0,4194304] 0 2024-03-24T00:49:07.307 INFO:tasks.workunit.client.0.smithi038.stdout:5/771: dwrite d6/d1e/d97/fc3 [0,4194304] 0 2024-03-24T00:49:07.307 INFO:tasks.workunit.client.0.smithi038.stdout:5/772: fsync d6/d16/d21/d2d/fd2 0 2024-03-24T00:49:07.307 INFO:tasks.workunit.client.0.smithi038.stdout:5/773: fsync d6/d60/f9a 0 2024-03-24T00:49:07.307 INFO:tasks.workunit.client.0.smithi038.stdout:5/774: chown d6/d19/d20/l30 91991 1 2024-03-24T00:49:07.307 INFO:tasks.workunit.client.0.smithi038.stdout:5/775: truncate d6/d19/d20/d64/d6b/fee 998763 0 2024-03-24T00:49:07.309 INFO:tasks.workunit.client.0.smithi038.stdout:9/885: creat d12/d81/d84/dec/dfc/f11b x:0 0 0 2024-03-24T00:49:07.309 INFO:tasks.workunit.client.0.smithi038.stdout:9/886: chown d12/d81/fee 12 1 2024-03-24T00:49:07.311 INFO:tasks.workunit.client.1.smithi088.stdout:4/826: creat d5/d13/d18/f120 x:0 0 0 2024-03-24T00:49:07.311 INFO:tasks.workunit.client.0.smithi038.stdout:8/829: mkdir d0/d1d/d20/d116 0 2024-03-24T00:49:07.314 INFO:tasks.workunit.client.1.smithi088.stdout:1/835: symlink d6/d27/d10d/l110 0 2024-03-24T00:49:07.314 INFO:tasks.workunit.client.1.smithi088.stdout:1/836: write d6/f2a [3363842,69108] 0 2024-03-24T00:49:07.315 INFO:tasks.workunit.client.1.smithi088.stdout:3/783: sync 2024-03-24T00:49:07.319 INFO:tasks.workunit.client.1.smithi088.stdout:8/821: mkdir d9/dc/d11/d24/d43/d4a/d118 0 2024-03-24T00:49:07.321 INFO:tasks.workunit.client.0.smithi038.stdout:5/776: symlink d6/d16/d21/d2d/lf8 0 2024-03-24T00:49:07.327 INFO:tasks.workunit.client.0.smithi038.stdout:9/887: symlink d12/d28/d9c/ddd/d111/d8f/l11c 0 2024-03-24T00:49:07.329 INFO:tasks.workunit.client.1.smithi088.stdout:5/883: symlink d1/dd/d6f/da1/d3f/d88/d8b/d94/l12a 0 2024-03-24T00:49:07.329 INFO:tasks.workunit.client.1.smithi088.stdout:5/884: write d1/d10/db6/d10b/d2f/d57/d9e/daa/f110 [397387,75846] 0 2024-03-24T00:49:07.332 INFO:tasks.workunit.client.0.smithi038.stdout:8/830: mkdir d0/d45/d66/d6d/d106/d117 0 2024-03-24T00:49:07.332 INFO:tasks.workunit.client.0.smithi038.stdout:8/831: stat d0/d1d/d27/d4d/daf/l113 0 2024-03-24T00:49:07.335 INFO:tasks.workunit.client.0.smithi038.stdout:5/777: link d6/d16/d21/cea d6/d19/d20/d64/d6b/de8/cf9 0 2024-03-24T00:49:07.338 INFO:tasks.workunit.client.1.smithi088.stdout:2/793: unlink db/dd/d96/d99/dc3/lef 0 2024-03-24T00:49:07.340 INFO:tasks.workunit.client.0.smithi038.stdout:9/888: creat d12/d28/d9c/ddd/d111/d8f/f11d x:0 0 0 2024-03-24T00:49:07.342 INFO:tasks.workunit.client.0.smithi038.stdout:2/715: dwrite d4/f66 [0,4194304] 0 2024-03-24T00:49:07.345 INFO:tasks.workunit.client.1.smithi088.stdout:0/853: mknod d9/dc/de/d19/d44/d46/c111 0 2024-03-24T00:49:07.349 INFO:tasks.workunit.client.0.smithi038.stdout:5/778: mknod d6/d19/d20/d42/d43/d68/dc8/df5/cfa 0 2024-03-24T00:49:07.353 INFO:tasks.workunit.client.1.smithi088.stdout:5/885: dread d1/d10/db6/d10b/d9d/fa3 [0,4194304] 0 2024-03-24T00:49:07.353 INFO:tasks.workunit.client.1.smithi088.stdout:5/886: truncate d1/dd/d6f/da1/d3f/f11c 640477 0 2024-03-24T00:49:07.354 INFO:tasks.workunit.client.0.smithi038.stdout:8/832: write d0/d1d/d27/d6b/dfc/f47 [3059848,7023] 0 2024-03-24T00:49:07.354 INFO:tasks.workunit.client.0.smithi038.stdout:8/833: chown d0/d45/c9c 0 1 2024-03-24T00:49:07.356 INFO:tasks.workunit.client.1.smithi088.stdout:4/827: mkdir d5/d7/d92/d121 0 2024-03-24T00:49:07.356 INFO:tasks.workunit.client.1.smithi088.stdout:4/828: creat d5/d13/d18/d24/d3d/df0/f122 x:0 0 0 2024-03-24T00:49:07.357 INFO:tasks.workunit.client.0.smithi038.stdout:2/716: mknod d4/d27/ddb/d9b/cf5 0 2024-03-24T00:49:07.359 INFO:tasks.workunit.client.0.smithi038.stdout:0/734: dwrite d0/d12/d15/d58/d64/da7/dbb/dd4/dea/f8d [0,4194304] 0 2024-03-24T00:49:07.359 INFO:tasks.workunit.client.0.smithi038.stdout:0/735: fsync d0/d12/fba 0 2024-03-24T00:49:07.359 INFO:tasks.workunit.client.0.smithi038.stdout:0/736: readlink d0/l10 0 2024-03-24T00:49:07.360 INFO:tasks.workunit.client.0.smithi038.stdout:0/737: chown d0/d12/d15/le1 232438 1 2024-03-24T00:49:07.360 INFO:tasks.workunit.client.0.smithi038.stdout:5/779: creat d6/d16/d14/ffb x:0 0 0 2024-03-24T00:49:07.360 INFO:tasks.workunit.client.0.smithi038.stdout:5/780: creat d6/d16/d14/ffc x:0 0 0 2024-03-24T00:49:07.361 INFO:tasks.workunit.client.1.smithi088.stdout:1/837: creat d6/d27/db0/de3/f111 x:0 0 0 2024-03-24T00:49:07.367 INFO:tasks.workunit.client.0.smithi038.stdout:8/834: link d0/d45/d37/d5f/db7/d62/dfa/cf9 d0/d45/d37/d5f/db7/dcc/c118 0 2024-03-24T00:49:07.369 INFO:tasks.workunit.client.1.smithi088.stdout:3/784: rmdir d3/de/d6f 39 2024-03-24T00:49:07.370 INFO:tasks.workunit.client.0.smithi038.stdout:2/717: getdents d4/d27/d2f/d30 0 2024-03-24T00:49:07.370 INFO:tasks.workunit.client.1.smithi088.stdout:9/817: dwrite d1/d10/d11/d23/fa0 [0,4194304] 0 2024-03-24T00:49:07.375 INFO:tasks.workunit.client.1.smithi088.stdout:0/854: rename d9/d26/d8f/da1/d62/la6 to d9/dc/de/d19/d8d/l112 0 2024-03-24T00:49:07.389 INFO:tasks.workunit.client.1.smithi088.stdout:5/887: write d1/d10/db6/d10b/d9d/fa3 [1730243,95782] 0 2024-03-24T00:49:07.389 INFO:tasks.workunit.client.1.smithi088.stdout:6/895: dwrite db/dd/f104 [0,4194304] 0 2024-03-24T00:49:07.393 INFO:tasks.workunit.client.1.smithi088.stdout:4/829: rmdir d5/d13/d18/d6e/d10f 0 2024-03-24T00:49:07.398 INFO:tasks.workunit.client.0.smithi038.stdout:9/889: dwrite d12/d28/d9c/ddd/d111/d26/fb7 [0,4194304] 0 2024-03-24T00:49:07.398 INFO:tasks.workunit.client.1.smithi088.stdout:8/822: dwrite d9/dc/d11/d24/d29/da9/de7/ff6 [0,4194304] 0 2024-03-24T00:49:07.402 INFO:tasks.workunit.client.1.smithi088.stdout:4/830: dread d5/f46 [0,4194304] 0 2024-03-24T00:49:07.402 INFO:tasks.workunit.client.1.smithi088.stdout:1/838: mkdir d6/d6e/dfc/d108/d3f/d112 0 2024-03-24T00:49:07.404 INFO:tasks.workunit.client.1.smithi088.stdout:3/785: creat d3/d6/d58/da4/ff9 x:0 0 0 2024-03-24T00:49:07.404 INFO:tasks.workunit.client.1.smithi088.stdout:3/786: chown d3/d1a/d1e/fd4 101605957 1 2024-03-24T00:49:07.405 INFO:tasks.workunit.client.1.smithi088.stdout:3/787: write d3/d1a/fd0 [942646,272] 0 2024-03-24T00:49:07.407 INFO:tasks.workunit.client.0.smithi038.stdout:0/738: rename d0/d12/d15/d58/d64/da7/dbb/dd4/dea/f65 to d0/d14/d84/ffb 0 2024-03-24T00:49:07.411 INFO:tasks.workunit.client.0.smithi038.stdout:3/786: dwrite d3/d30/d5d/f95 [0,4194304] 0 2024-03-24T00:49:07.411 INFO:tasks.workunit.client.0.smithi038.stdout:3/787: write d3/d2a/d107/f104 [367754,64621] 0 2024-03-24T00:49:07.412 INFO:tasks.workunit.client.1.smithi088.stdout:9/818: mkdir d1/db5/d107 0 2024-03-24T00:49:07.412 INFO:tasks.workunit.client.1.smithi088.stdout:9/819: dread - d1/d10/d9b/ffe zero size 2024-03-24T00:49:07.413 INFO:tasks.workunit.client.1.smithi088.stdout:9/820: chown d1/d15/f7f 6602 1 2024-03-24T00:49:07.413 INFO:tasks.workunit.client.1.smithi088.stdout:9/821: fdatasync d1/d10/d2c/ff2 0 2024-03-24T00:49:07.413 INFO:tasks.workunit.client.1.smithi088.stdout:9/822: fdatasync d1/d10/d29/d6d/f57 0 2024-03-24T00:49:07.414 INFO:tasks.workunit.client.1.smithi088.stdout:8/823: dread d9/dc/d11/d24/d29/f51 [4194304,4194304] 0 2024-03-24T00:49:07.414 INFO:tasks.workunit.client.1.smithi088.stdout:8/824: write d9/dc/d11/d24/d29/d5b/f53 [2208529,41284] 0 2024-03-24T00:49:07.416 INFO:tasks.workunit.client.0.smithi038.stdout:5/781: truncate d6/d19/d20/d42/d79/fa0 2589951 0 2024-03-24T00:49:07.420 INFO:tasks.workunit.client.0.smithi038.stdout:2/718: truncate d4/d27/d54/d53/f64 83966 0 2024-03-24T00:49:07.420 INFO:tasks.workunit.client.1.smithi088.stdout:2/794: symlink db/dd/d96/d4e/lfe 0 2024-03-24T00:49:07.421 INFO:tasks.workunit.client.1.smithi088.stdout:1/839: write d6/d6e/dfc/d108/d5b/f105 [425994,119368] 0 2024-03-24T00:49:07.422 INFO:tasks.workunit.client.1.smithi088.stdout:0/855: creat d9/dc/de/d19/d8d/d97/f113 x:0 0 0 2024-03-24T00:49:07.429 INFO:tasks.workunit.client.0.smithi038.stdout:9/890: symlink d12/d28/da8/l11e 0 2024-03-24T00:49:07.440 INFO:tasks.workunit.client.1.smithi088.stdout:5/888: creat d1/dd/d6f/d98/f12b x:0 0 0 2024-03-24T00:49:07.440 INFO:tasks.workunit.client.1.smithi088.stdout:5/889: chown d1/dd/d6f/d98/da0 107182 1 2024-03-24T00:49:07.441 INFO:tasks.workunit.client.1.smithi088.stdout:4/831: mkdir d5/d13/d123 0 2024-03-24T00:49:07.442 INFO:tasks.workunit.client.1.smithi088.stdout:4/832: fdatasync d5/d7/d72/f71 0 2024-03-24T00:49:07.442 INFO:tasks.workunit.client.1.smithi088.stdout:3/788: rmdir d3/d6/d19/d2a/dc2 39 2024-03-24T00:49:07.443 INFO:tasks.workunit.client.1.smithi088.stdout:3/789: creat d3/d32/ffa x:0 0 0 2024-03-24T00:49:07.446 INFO:tasks.workunit.client.1.smithi088.stdout:9/823: mknod d1/d15/d80/c108 0 2024-03-24T00:49:07.450 INFO:tasks.workunit.client.1.smithi088.stdout:6/896: dwrite db/f47 [0,4194304] 0 2024-03-24T00:49:07.450 INFO:tasks.workunit.client.1.smithi088.stdout:6/897: dread - db/dd/d61/d6d/fe3 zero size 2024-03-24T00:49:07.452 INFO:tasks.workunit.client.0.smithi038.stdout:9/891: write d12/d28/d71/fbe [486145,55163] 0 2024-03-24T00:49:07.453 INFO:tasks.workunit.client.0.smithi038.stdout:2/719: link d4/da3/fb7 d4/da3/ff6 0 2024-03-24T00:49:07.453 INFO:tasks.workunit.client.0.smithi038.stdout:2/720: write d4/d27/ddb/d7c/fb6 [330793,9289] 0 2024-03-24T00:49:07.456 INFO:tasks.workunit.client.0.smithi038.stdout:0/739: rmdir d0/daf/dd9 0 2024-03-24T00:49:07.460 INFO:tasks.workunit.client.1.smithi088.stdout:5/890: rename d1/dd/d6f/da1/f50 to d1/d10/db6/d10b/d2f/d57/d9e/daa/f12c 0 2024-03-24T00:49:07.462 INFO:tasks.workunit.client.0.smithi038.stdout:9/892: symlink d12/d28/d9c/ddd/d111/d8f/d10c/l11f 0 2024-03-24T00:49:07.465 INFO:tasks.workunit.client.1.smithi088.stdout:3/790: chown d3/c57 322769746 1 2024-03-24T00:49:07.470 INFO:tasks.workunit.client.0.smithi038.stdout:6/873: dwrite d1/da/d61/d69/fd4 [0,4194304] 0 2024-03-24T00:49:07.473 INFO:tasks.workunit.client.1.smithi088.stdout:9/824: creat d1/d10/d29/db0/f109 x:0 0 0 2024-03-24T00:49:07.477 INFO:tasks.workunit.client.0.smithi038.stdout:9/893: getdents d12/d55 0 2024-03-24T00:49:07.477 INFO:tasks.workunit.client.0.smithi038.stdout:2/721: truncate d4/f19 432251 0 2024-03-24T00:49:07.483 INFO:tasks.workunit.client.1.smithi088.stdout:6/898: creat db/dd/d87/f11c x:0 0 0 2024-03-24T00:49:07.483 INFO:tasks.workunit.client.1.smithi088.stdout:8/825: dwrite d9/dc/d11/d24/d29/d5b/f7d [0,4194304] 0 2024-03-24T00:49:07.486 INFO:tasks.workunit.client.0.smithi038.stdout:6/874: creat d1/d37/f11e x:0 0 0 2024-03-24T00:49:07.488 INFO:tasks.workunit.client.1.smithi088.stdout:5/891: creat d1/dd/d6f/da1/f12d x:0 0 0 2024-03-24T00:49:07.492 INFO:tasks.workunit.client.1.smithi088.stdout:3/791: symlink d3/d6/d27/d3c/lfb 0 2024-03-24T00:49:07.494 INFO:tasks.workunit.client.0.smithi038.stdout:6/875: symlink d1/da/d73/d75/l11f 0 2024-03-24T00:49:07.494 INFO:tasks.workunit.client.0.smithi038.stdout:6/876: write d1/da/d61/d69/f11b [1000428,129934] 0 2024-03-24T00:49:07.497 INFO:tasks.workunit.client.1.smithi088.stdout:0/856: dwrite d9/dc/de/f6c [4194304,4194304] 0 2024-03-24T00:49:07.497 INFO:tasks.workunit.client.1.smithi088.stdout:0/857: dread - d9/d26/d8f/da1/d62/d9b/f10b zero size 2024-03-24T00:49:07.499 INFO:tasks.workunit.client.1.smithi088.stdout:9/825: read d1/d10/d9b/dab/db8/f7b [6967142,71643] 0 2024-03-24T00:49:07.499 INFO:tasks.workunit.client.1.smithi088.stdout:9/826: chown d1/d10/d11/f17 10126 1 2024-03-24T00:49:07.499 INFO:tasks.workunit.client.1.smithi088.stdout:9/827: fsync d1/d22/d27/fd3 0 2024-03-24T00:49:07.499 INFO:tasks.workunit.client.1.smithi088.stdout:9/828: dread - d1/d22/f51 zero size 2024-03-24T00:49:07.500 INFO:tasks.workunit.client.0.smithi038.stdout:3/788: dwrite d3/d9/dc/da6/db5/d19/f28 [0,4194304] 0 2024-03-24T00:49:07.500 INFO:tasks.workunit.client.0.smithi038.stdout:3/789: fsync d3/d9/dc/da6/fc7 0 2024-03-24T00:49:07.500 INFO:tasks.workunit.client.0.smithi038.stdout:2/722: creat d4/d18/d26/d37/ff7 x:0 0 0 2024-03-24T00:49:07.501 INFO:tasks.workunit.client.0.smithi038.stdout:2/723: chown d4/d27/ddb/f78 0 1 2024-03-24T00:49:07.504 INFO:tasks.workunit.client.0.smithi038.stdout:0/740: rmdir d0/d12/d15/d58/d64/da7/dbb/dd4 39 2024-03-24T00:49:07.504 INFO:tasks.workunit.client.0.smithi038.stdout:0/741: chown d0/c5b 495 1 2024-03-24T00:49:07.505 INFO:tasks.workunit.client.1.smithi088.stdout:6/899: dread db/d1d/d50/f8d [0,4194304] 0 2024-03-24T00:49:07.508 INFO:tasks.workunit.client.0.smithi038.stdout:3/790: creat d3/d2a/f10c x:0 0 0 2024-03-24T00:49:07.516 INFO:tasks.workunit.client.1.smithi088.stdout:3/792: creat d3/d67/dc3/dd7/ffc x:0 0 0 2024-03-24T00:49:07.516 INFO:tasks.workunit.client.1.smithi088.stdout:3/793: chown d3/de/d6f/d85/d49/d70/d96/cc7 33581268 1 2024-03-24T00:49:07.516 INFO:tasks.workunit.client.1.smithi088.stdout:3/794: chown d3/d67/dc3/dd7 854867810 1 2024-03-24T00:49:07.516 INFO:tasks.workunit.client.1.smithi088.stdout:3/795: write d3/de/d6f/d85/d49/f54 [3695994,29252] 0 2024-03-24T00:49:07.516 INFO:tasks.workunit.client.1.smithi088.stdout:3/796: read - d3/d32/ffa zero size 2024-03-24T00:49:07.518 INFO:tasks.workunit.client.1.smithi088.stdout:0/858: getdents d9/dc/de 0 2024-03-24T00:49:07.521 INFO:tasks.workunit.client.1.smithi088.stdout:9/829: mkdir d1/d15/d21/df3/d10a 0 2024-03-24T00:49:07.523 INFO:tasks.workunit.client.0.smithi038.stdout:6/877: rmdir d1/d37/d84/df5 39 2024-03-24T00:49:07.523 INFO:tasks.workunit.client.0.smithi038.stdout:6/878: creat d1/da/d1b/d25/d49/d72/f120 x:0 0 0 2024-03-24T00:49:07.524 INFO:tasks.workunit.client.1.smithi088.stdout:6/900: mkdir db/dc5/dc8/d11d 0 2024-03-24T00:49:07.525 INFO:tasks.workunit.client.1.smithi088.stdout:6/901: write db/d1d/d50/d90/d55/fbe [870258,19762] 0 2024-03-24T00:49:07.525 INFO:tasks.workunit.client.1.smithi088.stdout:6/902: write db/dd/d61/d6d/d74/d88/fda [1200575,109207] 0 2024-03-24T00:49:07.525 INFO:tasks.workunit.client.0.smithi038.stdout:0/742: rename d0/d14/c55 to d0/d1b/df1/cfc 0 2024-03-24T00:49:07.525 INFO:tasks.workunit.client.0.smithi038.stdout:0/743: chown d0/d14/d88/db8/fbf 4 1 2024-03-24T00:49:07.526 INFO:tasks.workunit.client.1.smithi088.stdout:3/797: rename d3/d6/d27/d3c/c3a to d3/d6/d58/cfd 0 2024-03-24T00:49:07.528 INFO:tasks.workunit.client.0.smithi038.stdout:3/791: link d3/d2a/c84 d3/d9/dc/da6/db5/d19/c10d 0 2024-03-24T00:49:07.530 INFO:tasks.workunit.client.0.smithi038.stdout:0/744: write d0/d12/d15/d58/d64/da7/dbb/dd4/dea/f6f [1761346,11622] 0 2024-03-24T00:49:07.531 INFO:tasks.workunit.client.0.smithi038.stdout:9/894: dwrite d12/d81/fee [0,4194304] 0 2024-03-24T00:49:07.532 INFO:tasks.workunit.client.0.smithi038.stdout:2/724: rmdir d4/d18/da1 39 2024-03-24T00:49:07.539 INFO:tasks.workunit.client.0.smithi038.stdout:6/879: rename d1/da/d61/d69/f11b to d1/d37/d84/df5/f121 0 2024-03-24T00:49:07.539 INFO:tasks.workunit.client.0.smithi038.stdout:1/847: sync 2024-03-24T00:49:07.540 INFO:tasks.workunit.client.1.smithi088.stdout:0/859: unlink d9/dc/de/d19/d44/f50 0 2024-03-24T00:49:07.540 INFO:tasks.workunit.client.0.smithi038.stdout:0/745: rmdir d0/d14 39 2024-03-24T00:49:07.542 INFO:tasks.workunit.client.1.smithi088.stdout:9/830: mknod d1/d15/d21/d9c/c10b 0 2024-03-24T00:49:07.542 INFO:tasks.workunit.client.1.smithi088.stdout:9/831: truncate d1/d10/d9b/dab/f103 224645 0 2024-03-24T00:49:07.545 INFO:tasks.workunit.client.0.smithi038.stdout:9/895: link d12/d28/d9c/ddd/d111/d26/f86 d12/d28/d9c/ddd/d111/f120 0 2024-03-24T00:49:07.547 INFO:tasks.workunit.client.1.smithi088.stdout:0/860: creat d9/dc/de/d7e/de6/f114 x:0 0 0 2024-03-24T00:49:07.548 INFO:tasks.workunit.client.0.smithi038.stdout:3/792: dread d3/d9/d77/fc9 [0,4194304] 0 2024-03-24T00:49:07.548 INFO:tasks.workunit.client.0.smithi038.stdout:2/725: mkdir d4/d27/d2f/d30/df8 0 2024-03-24T00:49:07.549 INFO:tasks.workunit.client.1.smithi088.stdout:9/832: creat d1/d10/d29/f10c x:0 0 0 2024-03-24T00:49:07.551 INFO:tasks.workunit.client.0.smithi038.stdout:1/848: symlink da/dd/d2c/d5a/d101/dda/l11b 0 2024-03-24T00:49:07.554 INFO:tasks.workunit.client.0.smithi038.stdout:0/746: unlink d0/d14/d88/db8/ff3 0 2024-03-24T00:49:07.555 INFO:tasks.workunit.client.0.smithi038.stdout:7/732: sync 2024-03-24T00:49:07.555 INFO:tasks.workunit.client.0.smithi038.stdout:7/733: fdatasync d2/fdf 0 2024-03-24T00:49:07.556 INFO:tasks.workunit.client.0.smithi038.stdout:7/734: fsync d2/d9/da/d23/d2b/d5b/d72/d92/d5d/dcd/f104 0 2024-03-24T00:49:07.556 INFO:tasks.workunit.client.0.smithi038.stdout:7/735: chown d2/l26 30963 1 2024-03-24T00:49:07.556 INFO:tasks.workunit.client.1.smithi088.stdout:7/773: sync 2024-03-24T00:49:07.556 INFO:tasks.workunit.client.1.smithi088.stdout:7/774: fsync d3/d32/d34/d9c/f6f 0 2024-03-24T00:49:07.556 INFO:tasks.workunit.client.1.smithi088.stdout:7/775: truncate d3/d32/d34/d9c/d4b/f5e 4621687 0 2024-03-24T00:49:07.559 INFO:tasks.workunit.client.0.smithi038.stdout:3/793: truncate d3/d9/f2b 1578404 0 2024-03-24T00:49:07.559 INFO:tasks.workunit.client.0.smithi038.stdout:9/896: dread d12/d55/f67 [0,4194304] 0 2024-03-24T00:49:07.560 INFO:tasks.workunit.client.0.smithi038.stdout:9/897: dread - d12/d28/d9c/ddd/d111/d26/f5a zero size 2024-03-24T00:49:07.562 INFO:tasks.workunit.client.0.smithi038.stdout:2/726: symlink d4/d18/d26/d37/d69/da6/lf9 0 2024-03-24T00:49:07.562 INFO:tasks.workunit.client.0.smithi038.stdout:2/727: write d4/d27/d2f/f3c [531716,73106] 0 2024-03-24T00:49:07.563 INFO:tasks.workunit.client.0.smithi038.stdout:2/728: fdatasync d4/d27/ddb/d9b/fa7 0 2024-03-24T00:49:07.565 INFO:tasks.workunit.client.0.smithi038.stdout:7/736: creat d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/d10a/f10f x:0 0 0 2024-03-24T00:49:07.565 INFO:tasks.workunit.client.0.smithi038.stdout:7/737: fsync d2/d9/da/dc/d4a/f6d 0 2024-03-24T00:49:07.566 INFO:tasks.workunit.client.1.smithi088.stdout:6/903: write db/d3d/d83/d72/f76 [2472160,110111] 0 2024-03-24T00:49:07.569 INFO:tasks.workunit.client.0.smithi038.stdout:3/794: link d3/d9/dc/da6/db5/d19/dd2/cda d3/d30/d5d/d6a/c10e 0 2024-03-24T00:49:07.570 INFO:tasks.workunit.client.1.smithi088.stdout:0/861: dread d9/d26/d8f/da1/d62/d9b/fbb [0,4194304] 0 2024-03-24T00:49:07.574 INFO:tasks.workunit.client.1.smithi088.stdout:6/904: rename db/d1d/d50/d90/d55/lce to db/d1d/d23/l11e 0 2024-03-24T00:49:07.575 INFO:tasks.workunit.client.0.smithi038.stdout:9/898: symlink d12/d28/d9c/l121 0 2024-03-24T00:49:07.576 INFO:tasks.workunit.client.1.smithi088.stdout:7/776: read d3/fbb [1343796,75926] 0 2024-03-24T00:49:07.577 INFO:tasks.workunit.client.0.smithi038.stdout:2/729: symlink d4/d18/lfa 0 2024-03-24T00:49:07.581 INFO:tasks.workunit.client.0.smithi038.stdout:1/849: rmdir da/dd/d7a/d82/ddb 39 2024-03-24T00:49:07.582 INFO:tasks.workunit.client.0.smithi038.stdout:1/850: chown da/l4b 3 1 2024-03-24T00:49:07.582 INFO:tasks.workunit.client.1.smithi088.stdout:6/905: rename db/dd/d61/d6d/ld6 to db/d3d/dcb/l11f 0 2024-03-24T00:49:07.582 INFO:tasks.workunit.client.1.smithi088.stdout:3/798: dwrite d3/de/d6f/f7a [0,4194304] 0 2024-03-24T00:49:07.584 INFO:tasks.workunit.client.1.smithi088.stdout:9/833: dread d1/d10/d77/d8a/f71 [0,4194304] 0 2024-03-24T00:49:07.584 INFO:tasks.workunit.client.1.smithi088.stdout:9/834: creat d1/d10/d9b/dab/db8/da7/f10d x:0 0 0 2024-03-24T00:49:07.584 INFO:tasks.workunit.client.0.smithi038.stdout:7/738: creat d2/d9/da/d23/d2b/d5b/f110 x:0 0 0 2024-03-24T00:49:07.586 INFO:tasks.workunit.client.0.smithi038.stdout:4/748: sync 2024-03-24T00:49:07.588 INFO:tasks.workunit.client.1.smithi088.stdout:6/906: write db/d1d/d50/d90/f8f [153070,20453] 0 2024-03-24T00:49:07.588 INFO:tasks.workunit.client.1.smithi088.stdout:6/907: write db/d1d/d50/d90/d56/f51 [1178142,125352] 0 2024-03-24T00:49:07.588 INFO:tasks.workunit.client.0.smithi038.stdout:3/795: creat d3/d9/def/f10f x:0 0 0 2024-03-24T00:49:07.588 INFO:tasks.workunit.client.0.smithi038.stdout:3/796: creat d3/d30/d70/d90/d99/f110 x:0 0 0 2024-03-24T00:49:07.593 INFO:tasks.workunit.client.0.smithi038.stdout:2/730: mknod d4/d18/cfb 0 2024-03-24T00:49:07.594 INFO:tasks.workunit.client.1.smithi088.stdout:3/799: rmdir d3/de/d6f 39 2024-03-24T00:49:07.596 INFO:tasks.workunit.client.0.smithi038.stdout:1/851: creat da/dd/d2c/f11c x:0 0 0 2024-03-24T00:49:07.596 INFO:tasks.workunit.client.0.smithi038.stdout:1/852: write da/f72 [3137572,124452] 0 2024-03-24T00:49:07.603 INFO:tasks.workunit.client.0.smithi038.stdout:7/739: link d2/d9/da/dc/d4a/c106 d2/d9/da/d23/d2b/d5b/d72/d92/d98/c111 0 2024-03-24T00:49:07.603 INFO:tasks.workunit.client.0.smithi038.stdout:7/740: write d2/d9/da/d23/d2b/d5b/d72/d92/d98/fad [609134,50564] 0 2024-03-24T00:49:07.603 INFO:tasks.workunit.client.0.smithi038.stdout:7/741: chown d2/d9/fa7 16 1 2024-03-24T00:49:07.606 INFO:tasks.workunit.client.1.smithi088.stdout:3/800: creat d3/de/d6f/d85/d49/ffe x:0 0 0 2024-03-24T00:49:07.606 INFO:tasks.workunit.client.1.smithi088.stdout:6/908: dread db/d3d/d83/d72/f76 [0,4194304] 0 2024-03-24T00:49:07.607 INFO:tasks.workunit.client.1.smithi088.stdout:7/777: dread d3/d32/d76/d8f/fd4 [0,4194304] 0 2024-03-24T00:49:07.607 INFO:tasks.workunit.client.0.smithi038.stdout:4/749: truncate da/d27/f69 1543884 0 2024-03-24T00:49:07.612 INFO:tasks.workunit.client.1.smithi088.stdout:2/795: sync 2024-03-24T00:49:07.612 INFO:tasks.workunit.client.1.smithi088.stdout:1/840: sync 2024-03-24T00:49:07.612 INFO:tasks.workunit.client.1.smithi088.stdout:4/833: sync 2024-03-24T00:49:07.613 INFO:tasks.workunit.client.0.smithi038.stdout:9/899: dread d12/d55/d56/f7c [0,4194304] 0 2024-03-24T00:49:07.616 INFO:tasks.workunit.client.1.smithi088.stdout:3/801: rename d3/d6/f2b to d3/d6/d27/fff 0 2024-03-24T00:49:07.616 INFO:tasks.workunit.client.1.smithi088.stdout:3/802: stat d3/d6/d19/d2a/f53 0 2024-03-24T00:49:07.616 INFO:tasks.workunit.client.1.smithi088.stdout:3/803: truncate d3/de/d6f/fda 232825 0 2024-03-24T00:49:07.617 INFO:tasks.workunit.client.0.smithi038.stdout:0/747: dwrite d0/d14/d84/fa4 [0,4194304] 0 2024-03-24T00:49:07.622 INFO:tasks.workunit.client.1.smithi088.stdout:1/841: getdents d6/d27/d67/d88/da4 0 2024-03-24T00:49:07.711 INFO:tasks.workunit.client.0.smithi038.stdout:7/742: rename d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/db2 to d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/d112 0 2024-03-24T00:49:07.711 INFO:tasks.workunit.client.0.smithi038.stdout:4/750: unlink da/d12/d37/dd/d15/l4f 0 2024-03-24T00:49:07.711 INFO:tasks.workunit.client.0.smithi038.stdout:2/731: link d4/f1b d4/d27/d8f/ffc 0 2024-03-24T00:49:07.711 INFO:tasks.workunit.client.0.smithi038.stdout:2/732: stat d4/d27/d2f/ld6 0 2024-03-24T00:49:07.711 INFO:tasks.workunit.client.0.smithi038.stdout:1/853: dwrite da/f21 [4194304,4194304] 0 2024-03-24T00:49:07.711 INFO:tasks.workunit.client.0.smithi038.stdout:0/748: rename d0/f69 to d0/d14/de4/ffd 0 2024-03-24T00:49:07.711 INFO:tasks.workunit.client.0.smithi038.stdout:7/743: link d2/d2c/d9a/deb/c10e d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/c113 0 2024-03-24T00:49:07.711 INFO:tasks.workunit.client.0.smithi038.stdout:9/900: mkdir d12/d81/d122 0 2024-03-24T00:49:07.712 INFO:tasks.workunit.client.0.smithi038.stdout:9/901: getdents d12/d81/d122 0 2024-03-24T00:49:07.712 INFO:tasks.workunit.client.0.smithi038.stdout:9/902: chown d12/d28/d71/d99/ld2 181 1 2024-03-24T00:49:07.712 INFO:tasks.workunit.client.0.smithi038.stdout:1/854: write da/dd/d4a/d76/fad [800153,69574] 0 2024-03-24T00:49:07.712 INFO:tasks.workunit.client.0.smithi038.stdout:1/855: truncate da/dd/d2c/d5a/d74/f97 59467 0 2024-03-24T00:49:07.712 INFO:tasks.workunit.client.0.smithi038.stdout:1/856: stat da/dd/d2c/d5a/d101/ca8 0 2024-03-24T00:49:07.712 INFO:tasks.workunit.client.0.smithi038.stdout:2/733: symlink d4/d27/ddb/d9b/lfd 0 2024-03-24T00:49:07.713 INFO:tasks.workunit.client.1.smithi088.stdout:2/796: creat db/fff x:0 0 0 2024-03-24T00:49:07.713 INFO:tasks.workunit.client.1.smithi088.stdout:4/834: symlink d5/d7/d15/d74/d114/l124 0 2024-03-24T00:49:07.713 INFO:tasks.workunit.client.1.smithi088.stdout:7/778: rename d3/l2e to d3/d32/d34/d9c/d4b/d74/lf4 0 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:7/779: fdatasync d3/d32/d34/d9c/d4b/d74/f82 0 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:3/804: unlink d3/d6/d58/cb4 0 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:2/797: mkdir db/dd/dc9/d100 0 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:4/835: creat d5/d13/d18/d24/f125 x:0 0 0 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:4/836: chown d5/d7/d15/la4 1 1 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:7/780: mknod d3/d32/d34/cf5 0 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:0/862: dwrite d9/dc/de/d19/d44/f4a [0,4194304] 0 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:0/863: stat d9/dc/de/d19/d84/fff 0 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:0/864: readlink d9/d26/d8f/dc7/ddd/ld4 0 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:4/837: rename d5/d17/lc2 to d5/d16/d1a/l126 0 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:8/826: dwrite d9/dc/d11/d24/d29/f2c [0,4194304] 0 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:3/805: rmdir d3/d6/dca 39 2024-03-24T00:49:07.714 INFO:tasks.workunit.client.1.smithi088.stdout:6/909: dwrite db/dd/f37 [4194304,4194304] 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:4/838: link d5/d7/fcc d5/d7/de4/f127 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:7/781: creat d3/d32/d76/df3/ff6 x:0 0 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:4/839: readlink d5/d16/ld5 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:4/840: stat d5/d13/f12 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:8/827: getdents d9/dc/dce/ddc 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:3/806: mkdir d3/de/d6f/d85/de0/d100 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:9/835: dwrite d1/d22/f93 [0,4194304] 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:9/836: write d1/d22/f49 [4471290,73985] 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:5/892: dwrite d1/d10/db6/d10b/f70 [0,4194304] 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:6/910: mkdir db/dc5/dc8/d4b/d120 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:4/841: mknod d5/d7/d15/c128 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:1/842: dwrite d6/d6e/dfc/d108/d5b/d5d/d61/d7d/fad [0,4194304] 0 2024-03-24T00:49:07.715 INFO:tasks.workunit.client.1.smithi088.stdout:7/782: symlink d3/d32/d34/d9c/d4b/da7/dc5/lf7 0 2024-03-24T00:49:07.716 INFO:tasks.workunit.client.0.smithi038.stdout:9/903: rename d12/d1a/f3c to d12/d28/d71/d11a/f123 0 2024-03-24T00:49:07.719 INFO:tasks.workunit.client.1.smithi088.stdout:8/828: mkdir d9/dc/d11/d24/d119 0 2024-03-24T00:49:07.721 INFO:tasks.workunit.client.0.smithi038.stdout:1/857: mkdir da/dd/d7a/d82/d98/d14/d11d 0 2024-03-24T00:49:07.722 INFO:tasks.workunit.client.0.smithi038.stdout:1/858: fsync da/ddc/fe2 0 2024-03-24T00:49:07.722 INFO:tasks.workunit.client.0.smithi038.stdout:4/751: dwrite da/d12/d24/d5f/f97 [0,4194304] 0 2024-03-24T00:49:07.722 INFO:tasks.workunit.client.0.smithi038.stdout:4/752: fdatasync da/f1e 0 2024-03-24T00:49:07.722 INFO:tasks.workunit.client.0.smithi038.stdout:4/753: chown da/d12/d37/dd/d15/df1 23460019 1 2024-03-24T00:49:07.725 INFO:tasks.workunit.client.0.smithi038.stdout:2/734: truncate d4/d18/d38/d63/f7e 3257495 0 2024-03-24T00:49:07.727 INFO:tasks.workunit.client.1.smithi088.stdout:7/783: write d3/d32/d34/d9c/d4b/f6c [739347,61484] 0 2024-03-24T00:49:07.727 INFO:tasks.workunit.client.1.smithi088.stdout:0/865: dwrite d9/d26/d8f/da1/d63/f8a [4194304,4194304] 0 2024-03-24T00:49:07.733 INFO:tasks.workunit.client.0.smithi038.stdout:7/744: dread d2/d2c/f6c [0,4194304] 0 2024-03-24T00:49:07.744 INFO:tasks.workunit.client.0.smithi038.stdout:1/859: dread da/dd/d22/d94/fa0 [0,4194304] 0 2024-03-24T00:49:07.745 INFO:tasks.workunit.client.0.smithi038.stdout:1/860: write da/dd/d2c/d41/d60/dae/fc8 [838915,14285] 0 2024-03-24T00:49:07.745 INFO:tasks.workunit.client.0.smithi038.stdout:1/861: read da/f35 [24244,26832] 0 2024-03-24T00:49:07.746 INFO:tasks.workunit.client.0.smithi038.stdout:0/749: getdents d0/d1b 0 2024-03-24T00:49:07.748 INFO:tasks.workunit.client.1.smithi088.stdout:3/807: symlink d3/de/d6f/l101 0 2024-03-24T00:49:07.748 INFO:tasks.workunit.client.1.smithi088.stdout:3/808: stat d3/d1a/fe4 0 2024-03-24T00:49:07.749 INFO:tasks.workunit.client.1.smithi088.stdout:9/837: dwrite d1/d10/d77/dc0/fd4 [0,4194304] 0 2024-03-24T00:49:07.765 INFO:tasks.workunit.client.1.smithi088.stdout:5/893: mkdir d1/dd/d6f/da1/d3f/d106/d115/d12e 0 2024-03-24T00:49:07.765 INFO:tasks.workunit.client.1.smithi088.stdout:6/911: mkdir db/dd/d61/d6d/d74/d88/da1/d106/d121 0 2024-03-24T00:49:07.765 INFO:tasks.workunit.client.1.smithi088.stdout:5/894: truncate d1/d19/d1e/f47 2822156 0 2024-03-24T00:49:07.765 INFO:tasks.workunit.client.1.smithi088.stdout:5/895: chown d1/d10/db6/d10b/d2f/d57/fa5 1404217657 1 2024-03-24T00:49:07.766 INFO:tasks.workunit.client.1.smithi088.stdout:6/912: write db/d1d/d50/fd0 [487536,126308] 0 2024-03-24T00:49:07.767 INFO:tasks.workunit.client.1.smithi088.stdout:4/842: rename d5/d16/d1a/d22/f52 to d5/d7/d15/d74/d114/f129 0 2024-03-24T00:49:07.770 INFO:tasks.workunit.client.0.smithi038.stdout:7/745: unlink d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/fb6 0 2024-03-24T00:49:07.773 INFO:tasks.workunit.client.0.smithi038.stdout:0/750: mkdir d0/d12/d42/d83/dfe 0 2024-03-24T00:49:07.773 INFO:tasks.workunit.client.0.smithi038.stdout:0/751: stat d0/d12/d15/c24 0 2024-03-24T00:49:07.773 INFO:tasks.workunit.client.0.smithi038.stdout:0/752: truncate d0/d12/f89 609075 0 2024-03-24T00:49:07.773 INFO:tasks.workunit.client.0.smithi038.stdout:0/753: creat d0/d12/d15/d1a/fff x:0 0 0 2024-03-24T00:49:07.774 INFO:tasks.workunit.client.1.smithi088.stdout:1/843: symlink d6/d6e/dfc/d108/d3f/d112/l113 0 2024-03-24T00:49:07.775 INFO:tasks.workunit.client.0.smithi038.stdout:8/835: sync 2024-03-24T00:49:07.777 INFO:tasks.workunit.client.1.smithi088.stdout:8/829: creat d9/dc/dce/ddc/dde/f11a x:0 0 0 2024-03-24T00:49:07.779 INFO:tasks.workunit.client.0.smithi038.stdout:7/746: symlink d2/d9/da/d23/def/df0/l114 0 2024-03-24T00:49:07.779 INFO:tasks.workunit.client.0.smithi038.stdout:7/747: dread - d2/d9/da/dc/d93/ff1 zero size 2024-03-24T00:49:07.786 INFO:tasks.workunit.client.0.smithi038.stdout:7/748: creat d2/d2c/d90/f115 x:0 0 0 2024-03-24T00:49:07.787 INFO:tasks.workunit.client.0.smithi038.stdout:7/749: creat d2/d9/da/dc/d93/f116 x:0 0 0 2024-03-24T00:49:07.787 INFO:tasks.workunit.client.1.smithi088.stdout:3/809: mknod d3/de/d6f/dcc/c102 0 2024-03-24T00:49:07.787 INFO:tasks.workunit.client.1.smithi088.stdout:3/810: readlink d3/lac 0 2024-03-24T00:49:07.793 INFO:tasks.workunit.client.0.smithi038.stdout:7/750: rename d2/d9/da/d23/d2b/d5b/d72/d92/d5d/ca1 to d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/dcc/c117 0 2024-03-24T00:49:07.801 INFO:tasks.workunit.client.0.smithi038.stdout:7/751: rmdir d2/d9/d16 39 2024-03-24T00:49:07.801 INFO:tasks.workunit.client.0.smithi038.stdout:7/752: fdatasync d2/d9/da/f73 0 2024-03-24T00:49:07.801 INFO:tasks.workunit.client.0.smithi038.stdout:7/753: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/d33/f3c 0 2024-03-24T00:49:07.801 INFO:tasks.workunit.client.0.smithi038.stdout:7/754: write d2/d9/da/dc/d8a/db8/fce [182692,103504] 0 2024-03-24T00:49:07.803 INFO:tasks.workunit.client.1.smithi088.stdout:5/896: mknod d1/dd/d6f/da1/d3f/c12f 0 2024-03-24T00:49:07.803 INFO:tasks.workunit.client.1.smithi088.stdout:5/897: write d1/d10/db6/d10b/d2f/d57/d9e/dba/f116 [342615,12665] 0 2024-03-24T00:49:07.806 INFO:tasks.workunit.client.0.smithi038.stdout:7/755: link d2/d9/da/d23/c7a d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/c118 0 2024-03-24T00:49:07.809 INFO:tasks.workunit.client.1.smithi088.stdout:0/866: rename d9/dc/de/d19/d44/d46/d65/d2a/d2e to d9/dc/de/d7e/de6/df5/d115 0 2024-03-24T00:49:07.810 INFO:tasks.workunit.client.1.smithi088.stdout:4/843: truncate d5/d16/d37/f6f 2949275 0 2024-03-24T00:49:07.813 INFO:tasks.workunit.client.1.smithi088.stdout:1/844: mknod d6/c114 0 2024-03-24T00:49:07.815 INFO:tasks.workunit.client.1.smithi088.stdout:8/830: link d9/dc/d11/d24/d29/d36/d6a/d7b/d8f/dbc/fcf d9/dc/d11/d24/d29/d39/d84/f11b 0 2024-03-24T00:49:07.815 INFO:tasks.workunit.client.1.smithi088.stdout:7/784: getdents d3/d32/d76/df3 0 2024-03-24T00:49:07.819 INFO:tasks.workunit.client.1.smithi088.stdout:9/838: dwrite d1/d10/d11/f17 [4194304,4194304] 0 2024-03-24T00:49:07.823 INFO:tasks.workunit.client.1.smithi088.stdout:3/811: mknod d3/d1a/d1e/d2f/d45/db5/d6e/c103 0 2024-03-24T00:49:07.823 INFO:tasks.workunit.client.1.smithi088.stdout:5/898: creat d1/dd/d6f/da1/d3f/d88/d8f/f130 x:0 0 0 2024-03-24T00:49:07.823 INFO:tasks.workunit.client.1.smithi088.stdout:5/899: stat d1/d10/db6/d10b/d2f/d57/c11f 0 2024-03-24T00:49:07.823 INFO:tasks.workunit.client.1.smithi088.stdout:5/900: readlink d1/l9 0 2024-03-24T00:49:07.823 INFO:tasks.workunit.client.1.smithi088.stdout:5/901: chown d1/d19/fc0 58 1 2024-03-24T00:49:07.824 INFO:tasks.workunit.client.1.smithi088.stdout:6/913: rename db/d1d/d50/d90/d55/fbe to db/d3d/f122 0 2024-03-24T00:49:07.827 INFO:tasks.workunit.client.1.smithi088.stdout:4/844: mknod d5/d16/d94/da6/c12a 0 2024-03-24T00:49:07.827 INFO:tasks.workunit.client.1.smithi088.stdout:4/845: dread - d5/d7/de4/f11f zero size 2024-03-24T00:49:07.827 INFO:tasks.workunit.client.1.smithi088.stdout:4/846: chown d5/d7/de4/d107/f10e 7697942 1 2024-03-24T00:49:07.827 INFO:tasks.workunit.client.1.smithi088.stdout:4/847: fdatasync d5/d17/d91/ffe 0 2024-03-24T00:49:07.827 INFO:tasks.workunit.client.1.smithi088.stdout:4/848: fsync d5/d7/de4/d107/f10e 0 2024-03-24T00:49:07.829 INFO:tasks.workunit.client.1.smithi088.stdout:2/798: sync 2024-03-24T00:49:07.829 INFO:tasks.workunit.client.1.smithi088.stdout:1/845: mknod d6/d27/d67/c115 0 2024-03-24T00:49:07.829 INFO:tasks.workunit.client.1.smithi088.stdout:1/846: dread - d6/d6e/dfc/d108/d5b/feb zero size 2024-03-24T00:49:07.829 INFO:tasks.workunit.client.1.smithi088.stdout:1/847: chown d6/d6e/dfc/d108/d5b/d5d/d61/d7d/fd3 4720279 1 2024-03-24T00:49:07.829 INFO:tasks.workunit.client.1.smithi088.stdout:1/848: chown d6/d27/la0 93913985 1 2024-03-24T00:49:07.831 INFO:tasks.workunit.client.0.smithi038.stdout:7/756: dread d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/fe2 [0,4194304] 0 2024-03-24T00:49:07.833 INFO:tasks.workunit.client.1.smithi088.stdout:9/839: rmdir d1/d15/d80 39 2024-03-24T00:49:07.835 INFO:tasks.workunit.client.1.smithi088.stdout:5/902: mkdir d1/d19/d48/d84/d131 0 2024-03-24T00:49:07.837 INFO:tasks.workunit.client.1.smithi088.stdout:4/849: mkdir d5/d13/d63/d78/d12b 0 2024-03-24T00:49:07.837 INFO:tasks.workunit.client.1.smithi088.stdout:4/850: readlink d5/d16/d1a/d22/dc6/lef 0 2024-03-24T00:49:07.838 INFO:tasks.workunit.client.0.smithi038.stdout:1/862: dwrite da/dd/d7a/d82/d98/d9e/fb0 [0,4194304] 0 2024-03-24T00:49:07.838 INFO:tasks.workunit.client.0.smithi038.stdout:1/863: dread - da/dd/d2c/f11c zero size 2024-03-24T00:49:07.839 INFO:tasks.workunit.client.1.smithi088.stdout:2/799: rename db/dc/c3e to db/dc/c101 0 2024-03-24T00:49:07.840 INFO:tasks.workunit.client.1.smithi088.stdout:2/800: fdatasync db/dd/d96/fa2 0 2024-03-24T00:49:07.840 INFO:tasks.workunit.client.1.smithi088.stdout:2/801: chown db/d21/d35/d49 63101033 1 2024-03-24T00:49:07.840 INFO:tasks.workunit.client.1.smithi088.stdout:2/802: chown db/dd/d96/d99/db2/d88/cda 1 1 2024-03-24T00:49:07.845 INFO:tasks.workunit.client.1.smithi088.stdout:4/851: creat d5/d16/d37/de7/f12c x:0 0 0 2024-03-24T00:49:07.852 INFO:tasks.workunit.client.1.smithi088.stdout:9/840: mknod d1/d10/d9b/dab/c10e 0 2024-03-24T00:49:07.866 INFO:tasks.workunit.client.1.smithi088.stdout:3/812: write d3/de/d6f/d85/d90/ff7 [2806284,61406] 0 2024-03-24T00:49:07.883 INFO:tasks.workunit.client.0.smithi038.stdout:2/735: dwrite d4/f7a [0,4194304] 0 2024-03-24T00:49:07.883 INFO:tasks.workunit.client.0.smithi038.stdout:2/736: fdatasync d4/d27/d54/fd7 0 2024-03-24T00:49:07.884 INFO:tasks.workunit.client.1.smithi088.stdout:9/841: dread d1/d22/f93 [0,4194304] 0 2024-03-24T00:49:07.884 INFO:tasks.workunit.client.1.smithi088.stdout:9/842: creat d1/d10/d9b/dab/db8/da7/f10f x:0 0 0 2024-03-24T00:49:07.887 INFO:tasks.workunit.client.1.smithi088.stdout:2/803: write db/dd/d96/d4e/fa0 [2062894,11690] 0 2024-03-24T00:49:07.891 INFO:tasks.workunit.client.1.smithi088.stdout:2/804: creat db/dd/d96/d99/db2/f102 x:0 0 0 2024-03-24T00:49:07.891 INFO:tasks.workunit.client.1.smithi088.stdout:2/805: readlink db/d21/d35/d49/ldf 0 2024-03-24T00:49:07.891 INFO:tasks.workunit.client.1.smithi088.stdout:9/843: mknod d1/d10/d9b/c110 0 2024-03-24T00:49:07.893 INFO:tasks.workunit.client.0.smithi038.stdout:2/737: symlink d4/d18/lfe 0 2024-03-24T00:49:07.893 INFO:tasks.workunit.client.0.smithi038.stdout:2/738: dread - d4/d27/ddb/f78 zero size 2024-03-24T00:49:07.896 INFO:tasks.workunit.client.1.smithi088.stdout:9/844: link d1/d15/d21/f82 d1/d22/f111 0 2024-03-24T00:49:07.897 INFO:tasks.workunit.client.1.smithi088.stdout:9/845: getdents d1/d15/d84 0 2024-03-24T00:49:07.899 INFO:tasks.workunit.client.0.smithi038.stdout:2/739: write d4/da3/ff6 [338431,88340] 0 2024-03-24T00:49:07.900 INFO:tasks.workunit.client.1.smithi088.stdout:9/846: rename d1/d10/d11/c40 to d1/d15/c112 0 2024-03-24T00:49:07.904 INFO:tasks.workunit.client.0.smithi038.stdout:1/864: dwrite da/dd/d2c/d41/d60/dae/fc8 [0,4194304] 0 2024-03-24T00:49:07.905 INFO:tasks.workunit.client.1.smithi088.stdout:9/847: rmdir d1/d10/d11 39 2024-03-24T00:49:07.905 INFO:tasks.workunit.client.0.smithi038.stdout:2/740: rmdir d4/d27/d54/dec 0 2024-03-24T00:49:07.911 INFO:tasks.workunit.client.1.smithi088.stdout:1/849: dwrite d6/d6e/dfc/d108/d3f/f4b [0,4194304] 0 2024-03-24T00:49:07.911 INFO:tasks.workunit.client.0.smithi038.stdout:2/741: creat d4/d55/fff x:0 0 0 2024-03-24T00:49:07.917 INFO:tasks.workunit.client.1.smithi088.stdout:1/850: creat d6/d6e/dfc/d108/d5b/d93/f116 x:0 0 0 2024-03-24T00:49:07.917 INFO:tasks.workunit.client.1.smithi088.stdout:1/851: dread - d6/d6e/ffe zero size 2024-03-24T00:49:07.924 INFO:tasks.workunit.client.1.smithi088.stdout:1/852: mknod d6/d27/db0/d10e/c117 0 2024-03-24T00:49:07.924 INFO:tasks.workunit.client.1.smithi088.stdout:1/853: chown d6/d23/l28 239 1 2024-03-24T00:49:07.924 INFO:tasks.workunit.client.1.smithi088.stdout:8/831: dwrite d9/dc/d11/d24/d29/d5b/db9/fdd [0,4194304] 0 2024-03-24T00:49:07.924 INFO:tasks.workunit.client.1.smithi088.stdout:8/832: stat d9/dc/d11/le6 0 2024-03-24T00:49:07.936 INFO:tasks.workunit.client.0.smithi038.stdout:1/865: dread da/dd/d4a/d76/fad [0,4194304] 0 2024-03-24T00:49:07.942 INFO:tasks.workunit.client.1.smithi088.stdout:3/813: write d3/f87 [1951213,65123] 0 2024-03-24T00:49:07.943 INFO:tasks.workunit.client.1.smithi088.stdout:9/848: dread d1/db5/ff5 [0,4194304] 0 2024-03-24T00:49:07.943 INFO:tasks.workunit.client.1.smithi088.stdout:9/849: chown d1/d10/d9b/dab/db8/fce 6286985 1 2024-03-24T00:49:07.943 INFO:tasks.workunit.client.0.smithi038.stdout:1/866: rmdir da/dd/d2c/d40/de9 39 2024-03-24T00:49:07.946 INFO:tasks.workunit.client.0.smithi038.stdout:0/754: dwrite d0/d12/d42/d9d/d28/f8a [4194304,4194304] 0 2024-03-24T00:49:07.946 INFO:tasks.workunit.client.0.smithi038.stdout:0/755: truncate d0/d12/fba 267274 0 2024-03-24T00:49:07.953 INFO:tasks.workunit.client.0.smithi038.stdout:0/756: mkdir d0/d14/de4/d100 0 2024-03-24T00:49:07.953 INFO:tasks.workunit.client.1.smithi088.stdout:9/850: link d1/d10/l42 d1/d10/d77/d8a/de0/l113 0 2024-03-24T00:49:07.954 INFO:tasks.workunit.client.1.smithi088.stdout:9/851: fsync d1/d10/d11/fcb 0 2024-03-24T00:49:07.954 INFO:tasks.workunit.client.1.smithi088.stdout:8/833: dread d9/dc/d11/d24/d29/d39/fc6 [4194304,4194304] 0 2024-03-24T00:49:07.955 INFO:tasks.workunit.client.0.smithi038.stdout:0/757: mknod d0/d12/d42/d5e/c101 0 2024-03-24T00:49:07.955 INFO:tasks.workunit.client.0.smithi038.stdout:0/758: chown d0/d12/d15/d58/d64/da7/dbb/dd4/dea/l37 389 1 2024-03-24T00:49:07.955 INFO:tasks.workunit.client.1.smithi088.stdout:8/834: symlink d9/dc/d11/d24/d29/d36/d6a/d7b/d8f/l11c 0 2024-03-24T00:49:07.959 INFO:tasks.workunit.client.0.smithi038.stdout:0/759: rename d0/d1b/c30 to d0/d12/d42/d83/dfe/c102 0 2024-03-24T00:49:07.959 INFO:tasks.workunit.client.0.smithi038.stdout:0/760: write d0/d12/f68 [4441979,126307] 0 2024-03-24T00:49:07.959 INFO:tasks.workunit.client.0.smithi038.stdout:0/761: readlink d0/d12/d42/d9d/l66 0 2024-03-24T00:49:07.959 INFO:tasks.workunit.client.0.smithi038.stdout:0/762: chown d0/d12/d42/d9d/d28/cec 0 1 2024-03-24T00:49:07.959 INFO:tasks.workunit.client.0.smithi038.stdout:0/763: chown d0/f7 6 1 2024-03-24T00:49:07.961 INFO:tasks.workunit.client.1.smithi088.stdout:4/852: dwrite d5/d7/f66 [0,4194304] 0 2024-03-24T00:49:07.961 INFO:tasks.workunit.client.1.smithi088.stdout:8/835: rename d9/dc/d11/d24/d43/d4a/dee/cf2 to d9/c11d 0 2024-03-24T00:49:07.964 INFO:tasks.workunit.client.1.smithi088.stdout:2/806: dwrite db/dd/d96/d99/db2/d88/f97 [0,4194304] 0 2024-03-24T00:49:07.975 INFO:tasks.workunit.client.0.smithi038.stdout:7/757: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/fec [0,4194304] 0 2024-03-24T00:49:07.975 INFO:tasks.workunit.client.0.smithi038.stdout:7/758: chown d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/cdd 435041 1 2024-03-24T00:49:07.975 INFO:tasks.workunit.client.0.smithi038.stdout:0/764: stat d0/d12/d42/d83/dfe/c102 0 2024-03-24T00:49:07.983 INFO:tasks.workunit.client.1.smithi088.stdout:8/836: creat d9/dc/d11/dd2/f11e x:0 0 0 2024-03-24T00:49:07.983 INFO:tasks.workunit.client.1.smithi088.stdout:2/807: symlink db/d21/d73/l103 0 2024-03-24T00:49:07.983 INFO:tasks.workunit.client.1.smithi088.stdout:4/853: rename d5/d7/d15/c41 to d5/d7/d15/c12d 0 2024-03-24T00:49:07.983 INFO:tasks.workunit.client.1.smithi088.stdout:4/854: chown d5/d16/d94/da6/ffa 1 1 2024-03-24T00:49:07.984 INFO:tasks.workunit.client.0.smithi038.stdout:0/765: mknod d0/c103 0 2024-03-24T00:49:07.984 INFO:tasks.workunit.client.0.smithi038.stdout:0/766: chown d0/d12/d42/d9d/c1f 490305 1 2024-03-24T00:49:07.984 INFO:tasks.workunit.client.1.smithi088.stdout:0/867: dwrite d9/dc/de/f5c [4194304,4194304] 0 2024-03-24T00:49:07.984 INFO:tasks.workunit.client.1.smithi088.stdout:7/785: dwrite d3/d32/d34/d9c/d4b/d74/f82 [0,4194304] 0 2024-03-24T00:49:07.986 INFO:tasks.workunit.client.0.smithi038.stdout:5/782: sync 2024-03-24T00:49:07.986 INFO:tasks.workunit.client.0.smithi038.stdout:5/783: stat d6/c36 0 2024-03-24T00:49:07.988 INFO:tasks.workunit.client.0.smithi038.stdout:0/767: symlink d0/d12/d92/l104 0 2024-03-24T00:49:07.995 INFO:tasks.workunit.client.1.smithi088.stdout:1/854: dwrite d6/f2a [0,4194304] 0 2024-03-24T00:49:07.995 INFO:tasks.workunit.client.1.smithi088.stdout:1/855: fsync d6/d27/d67/d9b/db9/ff5 0 2024-03-24T00:49:07.995 INFO:tasks.workunit.client.1.smithi088.stdout:1/856: truncate d6/d6e/dfc/d108/d5b/d5d/d61/d7d/dc8/fd1 4889162 0 2024-03-24T00:49:07.995 INFO:tasks.workunit.client.1.smithi088.stdout:1/857: dread - d6/d27/d67/d88/fd2 zero size 2024-03-24T00:49:07.995 INFO:tasks.workunit.client.1.smithi088.stdout:2/808: dread db/dc/f93 [0,4194304] 0 2024-03-24T00:49:07.995 INFO:tasks.workunit.client.1.smithi088.stdout:2/809: chown db/d21/d35/lf9 1359 1 2024-03-24T00:49:07.996 INFO:tasks.workunit.client.1.smithi088.stdout:2/810: dread - db/dd/d96/d99/db2/ff0 zero size 2024-03-24T00:49:07.998 INFO:tasks.workunit.client.0.smithi038.stdout:7/759: dread d2/d9/da/d23/d2b/d5b/d72/d92/d33/f31 [0,4194304] 0 2024-03-24T00:49:07.999 INFO:tasks.workunit.client.0.smithi038.stdout:5/784: mknod d6/d19/da1/cfd 0 2024-03-24T00:49:08.000 INFO:tasks.workunit.client.0.smithi038.stdout:8/836: sync 2024-03-24T00:49:08.004 INFO:tasks.workunit.client.1.smithi088.stdout:5/903: write d1/d10/db6/d10b/d2f/d57/fa5 [560018,48892] 0 2024-03-24T00:49:08.009 INFO:tasks.workunit.client.1.smithi088.stdout:4/855: dread d5/fb5 [0,4194304] 0 2024-03-24T00:49:08.009 INFO:tasks.workunit.client.1.smithi088.stdout:4/856: chown d5/d13/d18/d6e/d7d/fbb 8178 1 2024-03-24T00:49:08.009 INFO:tasks.workunit.client.1.smithi088.stdout:4/857: write d5/d13/d18/d6e/d7d/f7e [2847360,92782] 0 2024-03-24T00:49:08.009 INFO:tasks.workunit.client.1.smithi088.stdout:4/858: truncate d5/d16/d94/da6/fc7 719545 0 2024-03-24T00:49:08.009 INFO:tasks.workunit.client.1.smithi088.stdout:4/859: chown d5/d13/d18 24740535 1 2024-03-24T00:49:08.011 INFO:tasks.workunit.client.0.smithi038.stdout:5/785: link d6/d16/d14/f8e d6/d19/da1/ffe 0 2024-03-24T00:49:08.012 INFO:tasks.workunit.client.0.smithi038.stdout:6/880: sync 2024-03-24T00:49:08.012 INFO:tasks.workunit.client.0.smithi038.stdout:9/904: sync 2024-03-24T00:49:08.012 INFO:tasks.workunit.client.0.smithi038.stdout:4/754: sync 2024-03-24T00:49:08.013 INFO:tasks.workunit.client.1.smithi088.stdout:3/814: dread d3/de/d6f/f7a [0,4194304] 0 2024-03-24T00:49:08.013 INFO:tasks.workunit.client.1.smithi088.stdout:3/815: readlink d3/d32/l51 0 2024-03-24T00:49:08.015 INFO:tasks.workunit.client.1.smithi088.stdout:0/868: dread d9/f55 [0,4194304] 0 2024-03-24T00:49:08.017 INFO:tasks.workunit.client.1.smithi088.stdout:5/904: mknod d1/d10/db6/d10b/d2f/d57/d9e/dba/df1/c132 0 2024-03-24T00:49:08.018 INFO:tasks.workunit.client.1.smithi088.stdout:4/860: creat d5/d8b/f12e x:0 0 0 2024-03-24T00:49:08.019 INFO:tasks.workunit.client.1.smithi088.stdout:4/861: stat d5/d13/d18/d6e 0 2024-03-24T00:49:08.023 INFO:tasks.workunit.client.0.smithi038.stdout:3/797: sync 2024-03-24T00:49:08.026 INFO:tasks.workunit.client.0.smithi038.stdout:0/768: getdents d0/d12/d42/d5e 0 2024-03-24T00:49:08.026 INFO:tasks.workunit.client.0.smithi038.stdout:0/769: chown d0/l9 349648985 1 2024-03-24T00:49:08.028 INFO:tasks.workunit.client.1.smithi088.stdout:3/816: link d3/d1a/d1e/lb9 d3/d1a/d1e/d2f/d45/db5/l104 0 2024-03-24T00:49:08.028 INFO:tasks.workunit.client.1.smithi088.stdout:4/862: truncate d5/d7/d72/f71 1449632 0 2024-03-24T00:49:08.028 INFO:tasks.workunit.client.1.smithi088.stdout:8/837: dread d9/dc/d11/d24/d29/f2c [0,4194304] 0 2024-03-24T00:49:08.029 INFO:tasks.workunit.client.1.smithi088.stdout:7/786: dread d3/d32/d34/d9c/d4b/d74/f95 [0,4194304] 0 2024-03-24T00:49:08.036 INFO:tasks.workunit.client.1.smithi088.stdout:3/817: dread d3/d1a/d1e/d2f/fcd [0,4194304] 0 2024-03-24T00:49:08.036 INFO:tasks.workunit.client.1.smithi088.stdout:1/858: dread d6/d6e/dfc/d108/d58/f81 [0,4194304] 0 2024-03-24T00:49:08.036 INFO:tasks.workunit.client.1.smithi088.stdout:9/852: dwrite d1/ff [0,4194304] 0 2024-03-24T00:49:08.038 INFO:tasks.workunit.client.1.smithi088.stdout:0/869: write d9/dc/de/f5c [4199153,102204] 0 2024-03-24T00:49:08.038 INFO:tasks.workunit.client.1.smithi088.stdout:4/863: rename d5/d7/d15/d60/lee to d5/d13/d63/d78/d12b/l12f 0 2024-03-24T00:49:08.038 INFO:tasks.workunit.client.1.smithi088.stdout:8/838: rmdir d9/dc/dce 39 2024-03-24T00:49:08.039 INFO:tasks.workunit.client.0.smithi038.stdout:6/881: link d1/da/d1b/d2f/l32 d1/da/d1b/d1d/d3c/l122 0 2024-03-24T00:49:08.039 INFO:tasks.workunit.client.0.smithi038.stdout:9/905: creat d12/d1a/d5f/df9/f124 x:0 0 0 2024-03-24T00:49:08.039 INFO:tasks.workunit.client.0.smithi038.stdout:4/755: rmdir da/d27/d89 0 2024-03-24T00:49:08.039 INFO:tasks.workunit.client.1.smithi088.stdout:6/914: dwrite db/d1d/f62 [0,4194304] 0 2024-03-24T00:49:08.042 INFO:tasks.workunit.client.1.smithi088.stdout:7/787: symlink d3/d31/d56/lf8 0 2024-03-24T00:49:08.053 INFO:tasks.workunit.client.0.smithi038.stdout:0/770: mkdir d0/d12/d42/d44/d105 0 2024-03-24T00:49:08.053 INFO:tasks.workunit.client.0.smithi038.stdout:0/771: chown d0/d12/d42/d44/f93 1915 1 2024-03-24T00:49:08.053 INFO:tasks.workunit.client.0.smithi038.stdout:0/772: chown d0/d12/d15/d58/d64/ffa 405 1 2024-03-24T00:49:08.055 INFO:tasks.workunit.client.0.smithi038.stdout:3/798: write d3/d30/d70/f7b [1412752,72555] 0 2024-03-24T00:49:08.055 INFO:tasks.workunit.client.0.smithi038.stdout:3/799: chown d3/d9/dc/da6/db5/dad/fbd 7 1 2024-03-24T00:49:08.055 INFO:tasks.workunit.client.0.smithi038.stdout:3/800: fsync d3/d9/dc/da6/db5/d19/dd2/df0/ff2 0 2024-03-24T00:49:08.055 INFO:tasks.workunit.client.0.smithi038.stdout:3/801: readlink d3/d30/d5d/d6a/laf 0 2024-03-24T00:49:08.057 INFO:tasks.workunit.client.0.smithi038.stdout:6/882: link d1/da/d61/fd1 d1/da/d61/d69/f123 0 2024-03-24T00:49:08.064 INFO:tasks.workunit.client.0.smithi038.stdout:9/906: dread d12/d28/d9c/ddd/d111/d26/f86 [0,4194304] 0 2024-03-24T00:49:08.065 INFO:tasks.workunit.client.0.smithi038.stdout:4/756: write da/d12/d37/dd/d15/f17 [2992480,126152] 0 2024-03-24T00:49:08.065 INFO:tasks.workunit.client.0.smithi038.stdout:4/757: fsync da/d12/d37/dd/f34 0 2024-03-24T00:49:08.065 INFO:tasks.workunit.client.0.smithi038.stdout:4/758: readlink da/d12/d37/dd/d15/d6d/l7f 0 2024-03-24T00:49:08.065 INFO:tasks.workunit.client.1.smithi088.stdout:4/864: rename d5/d13/d63/d78/df1/l100 to d5/dac/l130 0 2024-03-24T00:49:08.066 INFO:tasks.workunit.client.0.smithi038.stdout:3/802: rename d3/d9/dc/c60 to d3/d9/dc/df/c111 0 2024-03-24T00:49:08.068 INFO:tasks.workunit.client.0.smithi038.stdout:6/883: unlink d1/da/d73/c77 0 2024-03-24T00:49:08.069 INFO:tasks.workunit.client.0.smithi038.stdout:4/759: mkdir da/d12/d24/d68/d6c/d7c/d105 0 2024-03-24T00:49:08.070 INFO:tasks.workunit.client.1.smithi088.stdout:5/905: dread d1/d10/db6/d10b/d2f/d57/f5f [0,4194304] 0 2024-03-24T00:49:08.075 INFO:tasks.workunit.client.0.smithi038.stdout:6/884: creat d1/da/d1b/dbb/f124 x:0 0 0 2024-03-24T00:49:08.076 INFO:tasks.workunit.client.1.smithi088.stdout:6/915: creat db/d8e/d10d/f123 x:0 0 0 2024-03-24T00:49:08.076 INFO:tasks.workunit.client.1.smithi088.stdout:6/916: readlink db/dd/d61/d6d/d74/l81 0 2024-03-24T00:49:08.076 INFO:tasks.workunit.client.1.smithi088.stdout:6/917: readlink db/dd/la0 0 2024-03-24T00:49:08.076 INFO:tasks.workunit.client.1.smithi088.stdout:6/918: creat db/dc5/dc8/d4b/f124 x:0 0 0 2024-03-24T00:49:08.076 INFO:tasks.workunit.client.1.smithi088.stdout:6/919: stat db/dc5 0 2024-03-24T00:49:08.080 INFO:tasks.workunit.client.1.smithi088.stdout:7/788: rmdir d3/d32/d34/d9c 39 2024-03-24T00:49:08.083 INFO:tasks.workunit.client.0.smithi038.stdout:4/760: dread da/d12/d37/dd/d15/d4c/f53 [0,4194304] 0 2024-03-24T00:49:08.084 INFO:tasks.workunit.client.0.smithi038.stdout:0/773: dread d0/d12/d15/d1a/f98 [0,4194304] 0 2024-03-24T00:49:08.084 INFO:tasks.workunit.client.0.smithi038.stdout:0/774: chown d0/d12/d15/d58/d64/da7/dbb/dd4/dea/f6f 1 1 2024-03-24T00:49:08.085 INFO:tasks.workunit.client.0.smithi038.stdout:9/907: rename d12/d41/d57/l64 to d12/d28/d9c/ddd/df5/l125 0 2024-03-24T00:49:08.085 INFO:tasks.workunit.client.0.smithi038.stdout:9/908: fsync d12/d28/d9c/ddd/d111/d26/f5a 0 2024-03-24T00:49:08.086 INFO:tasks.workunit.client.1.smithi088.stdout:8/839: write d9/dc/d11/d24/d29/d39/fc6 [5872435,83923] 0 2024-03-24T00:49:08.097 INFO:tasks.workunit.client.1.smithi088.stdout:9/853: rmdir d1/d10/d9b/dab/db8/da7 39 2024-03-24T00:49:08.098 INFO:tasks.workunit.client.0.smithi038.stdout:3/803: write d3/d9/dc/da6/db5/d19/d82/d89/da8/f100 [5304692,123229] 0 2024-03-24T00:49:08.099 INFO:tasks.workunit.client.0.smithi038.stdout:7/760: dwrite d2/d9/da/d23/d2b/d5b/f110 [0,4194304] 0 2024-03-24T00:49:08.100 INFO:tasks.workunit.client.0.smithi038.stdout:6/885: rename d1/da/d1b/d1d/l119 to d1/da/d61/da2/l125 0 2024-03-24T00:49:08.104 INFO:tasks.workunit.client.1.smithi088.stdout:8/840: write d9/dc/d11/d24/d43/f50 [2059542,74262] 0 2024-03-24T00:49:08.104 INFO:tasks.workunit.client.1.smithi088.stdout:3/818: dwrite d3/de/d6f/d85/f15 [0,4194304] 0 2024-03-24T00:49:08.107 INFO:tasks.workunit.client.0.smithi038.stdout:8/837: dwrite d0/d1d/d27/d4d/daf/fb5 [0,4194304] 0 2024-03-24T00:49:08.112 INFO:tasks.workunit.client.1.smithi088.stdout:1/859: rename d6/d6e/dfc/d108/d5b/d5d/d61/d7d/da5 to d6/d6e/dfc/d108/d118 0 2024-03-24T00:49:08.114 INFO:tasks.workunit.client.0.smithi038.stdout:3/804: truncate d3/d9/dc/f81 370240 0 2024-03-24T00:49:08.121 INFO:tasks.workunit.client.0.smithi038.stdout:8/838: dread d0/d15/f98 [0,4194304] 0 2024-03-24T00:49:08.122 INFO:tasks.workunit.client.0.smithi038.stdout:7/761: dread - d2/d2c/fd7 zero size 2024-03-24T00:49:08.124 INFO:tasks.workunit.client.0.smithi038.stdout:4/761: rename da/l35 to da/d27/l106 0 2024-03-24T00:49:08.124 INFO:tasks.workunit.client.0.smithi038.stdout:4/762: dread - da/d12/d37/d83/dac/fad zero size 2024-03-24T00:49:08.125 INFO:tasks.workunit.client.0.smithi038.stdout:4/763: dread - da/d12/d24/d68/d6c/fe9 zero size 2024-03-24T00:49:08.125 INFO:tasks.workunit.client.0.smithi038.stdout:4/764: write da/d12/d37/dd/f95 [859050,38566] 0 2024-03-24T00:49:08.125 INFO:tasks.workunit.client.0.smithi038.stdout:4/765: readlink da/ldb 0 2024-03-24T00:49:08.125 INFO:tasks.workunit.client.0.smithi038.stdout:4/766: chown da/d12/d24/d68/d6c/f79 25722086 1 2024-03-24T00:49:08.125 INFO:tasks.workunit.client.0.smithi038.stdout:6/886: mkdir d1/da/d1b/d2f/dc0/d126 0 2024-03-24T00:49:08.125 INFO:tasks.workunit.client.0.smithi038.stdout:6/887: chown d1/da/fb 4239707 1 2024-03-24T00:49:08.129 INFO:tasks.workunit.client.0.smithi038.stdout:5/786: dwrite d6/d19/d20/d42/f72 [0,4194304] 0 2024-03-24T00:49:08.130 INFO:tasks.workunit.client.1.smithi088.stdout:5/906: unlink d1/d19/c9c 0 2024-03-24T00:49:08.132 INFO:tasks.workunit.client.0.smithi038.stdout:9/909: mknod d12/d28/d6c/dbc/dd7/dda/c126 0 2024-03-24T00:49:08.134 INFO:tasks.workunit.client.0.smithi038.stdout:3/805: creat d3/d9/dc/da6/db5/d19/dd2/f112 x:0 0 0 2024-03-24T00:49:08.134 INFO:tasks.workunit.client.0.smithi038.stdout:3/806: read d3/d9/dc/df/f3a [5221751,120418] 0 2024-03-24T00:49:08.134 INFO:tasks.workunit.client.0.smithi038.stdout:3/807: fsync d3/d9/dc/da6/db5/d19/dd2/df0/ff2 0 2024-03-24T00:49:08.134 INFO:tasks.workunit.client.1.smithi088.stdout:6/920: dwrite db/dd/d61/d6d/d74/f8b [0,4194304] 0 2024-03-24T00:49:08.135 INFO:tasks.workunit.client.0.smithi038.stdout:8/839: symlink d0/d45/d37/l119 0 2024-03-24T00:49:08.139 INFO:tasks.workunit.client.0.smithi038.stdout:7/762: creat d2/d9/d16/dfc/f119 x:0 0 0 2024-03-24T00:49:08.139 INFO:tasks.workunit.client.0.smithi038.stdout:7/763: readlink d2/d9/da/d23/d2b/d5b/d72/d92/l44 0 2024-03-24T00:49:08.139 INFO:tasks.workunit.client.0.smithi038.stdout:7/764: chown d2/d9/da/d23/d2b/d5b/d72/d92/d33 2 1 2024-03-24T00:49:08.139 INFO:tasks.workunit.client.0.smithi038.stdout:7/765: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fc8 0 2024-03-24T00:49:08.140 INFO:tasks.workunit.client.1.smithi088.stdout:9/854: mknod d1/d10/d77/dc0/c114 0 2024-03-24T00:49:08.143 INFO:tasks.workunit.client.0.smithi038.stdout:0/775: rename d0/d12/d92/cce to d0/d12/d9a/c106 0 2024-03-24T00:49:08.143 INFO:tasks.workunit.client.0.smithi038.stdout:0/776: creat d0/d12/d42/d83/f107 x:0 0 0 2024-03-24T00:49:08.143 INFO:tasks.workunit.client.1.smithi088.stdout:7/789: dwrite d3/d32/d34/d9c/d4b/d74/d79/d7f/fdd [0,4194304] 0 2024-03-24T00:49:08.146 INFO:tasks.workunit.client.0.smithi038.stdout:4/767: link da/d12/ff7 da/d12/d24/d5f/d8a/db6/dd1/f107 0 2024-03-24T00:49:08.146 INFO:tasks.workunit.client.1.smithi088.stdout:0/870: write d9/dc/de/d7e/dce/fe1 [1010858,13256] 0 2024-03-24T00:49:08.154 INFO:tasks.workunit.client.1.smithi088.stdout:1/860: symlink d6/d6e/dfc/d108/d5b/d5d/d61/l119 0 2024-03-24T00:49:08.154 INFO:tasks.workunit.client.0.smithi038.stdout:6/888: mkdir d1/da/d1b/d25/d49/d4e/d127 0 2024-03-24T00:49:08.154 INFO:tasks.workunit.client.0.smithi038.stdout:6/889: dread - d1/da/d1b/d25/d49/fcc zero size 2024-03-24T00:49:08.154 INFO:tasks.workunit.client.0.smithi038.stdout:6/890: chown d1/da/lbf 345 1 2024-03-24T00:49:08.159 INFO:tasks.workunit.client.1.smithi088.stdout:4/865: rename d5/d7/d15/d74/d114/l124 to d5/d13/d123/l131 0 2024-03-24T00:49:08.159 INFO:tasks.workunit.client.1.smithi088.stdout:4/866: fdatasync d5/d7/d72/f73 0 2024-03-24T00:49:08.166 INFO:tasks.workunit.client.1.smithi088.stdout:9/855: dread d1/d10/d29/d6d/f57 [0,4194304] 0 2024-03-24T00:49:08.166 INFO:tasks.workunit.client.1.smithi088.stdout:6/921: mknod db/dd/d61/c125 0 2024-03-24T00:49:08.169 INFO:tasks.workunit.client.0.smithi038.stdout:5/787: creat d6/fff x:0 0 0 2024-03-24T00:49:08.169 INFO:tasks.workunit.client.0.smithi038.stdout:5/788: fsync d6/d23/f3b 0 2024-03-24T00:49:08.173 INFO:tasks.workunit.client.0.smithi038.stdout:8/840: link d0/d1d/d27/d4d/fb2 d0/f11a 0 2024-03-24T00:49:08.176 INFO:tasks.workunit.client.0.smithi038.stdout:9/910: rename d12/d28/d9c/ddd/d111/d26/f4a to d12/d28/d9c/ddd/df5/f127 0 2024-03-24T00:49:08.176 INFO:tasks.workunit.client.0.smithi038.stdout:9/911: chown d12/d28/f36 1175 1 2024-03-24T00:49:08.177 INFO:tasks.workunit.client.0.smithi038.stdout:6/891: dread d1/f6b [0,4194304] 0 2024-03-24T00:49:08.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:07 smithi088 ceph-mon[36610]: pgmap v40: 65 pgs: 65 active+clean; 3.1 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 147 MiB/s rd, 165 MiB/s wr, 259 op/s 2024-03-24T00:49:08.181 INFO:tasks.workunit.client.1.smithi088.stdout:8/841: dwrite d9/dc/d11/d24/d29/f51 [0,4194304] 0 2024-03-24T00:49:08.181 INFO:tasks.workunit.client.1.smithi088.stdout:5/907: dwrite d1/d10/db6/d10b/d2f/d57/d9e/dd0/f11e [0,4194304] 0 2024-03-24T00:49:08.181 INFO:tasks.workunit.client.1.smithi088.stdout:8/842: write d9/dc/d11/d24/d29/da9/fab [597849,121772] 0 2024-03-24T00:49:08.184 INFO:tasks.workunit.client.0.smithi038.stdout:5/789: link d6/d1e/d97/fbe d6/d19/d20/d64/d6b/d77/f100 0 2024-03-24T00:49:08.184 INFO:tasks.workunit.client.0.smithi038.stdout:5/790: chown d6/d19/d20/d42/d43/d68/dc8/df5/cfa 0 1 2024-03-24T00:49:08.185 INFO:tasks.workunit.client.1.smithi088.stdout:3/819: rmdir d3/d6/d19/d2a/d5f 39 2024-03-24T00:49:08.190 INFO:tasks.workunit.client.1.smithi088.stdout:1/861: mknod d6/d23/dcb/c11a 0 2024-03-24T00:49:08.194 INFO:tasks.workunit.client.0.smithi038.stdout:9/912: symlink d12/d28/d71/l128 0 2024-03-24T00:49:08.204 INFO:tasks.workunit.client.1.smithi088.stdout:4/867: creat d5/d13/d18/d24/db3/f132 x:0 0 0 2024-03-24T00:49:08.213 INFO:tasks.workunit.client.0.smithi038.stdout:3/808: rename d3/d9/dc/da6/db5/d19/f6c to d3/d2a/f113 0 2024-03-24T00:49:08.217 INFO:tasks.workunit.client.1.smithi088.stdout:6/922: symlink db/dd/d61/d6d/d74/d88/da1/d106/l126 0 2024-03-24T00:49:08.219 INFO:tasks.workunit.client.1.smithi088.stdout:6/923: write db/d1d/d50/fef [22309,90404] 0 2024-03-24T00:49:08.221 INFO:tasks.workunit.client.0.smithi038.stdout:5/791: creat d6/d16/d67/df6/f101 x:0 0 0 2024-03-24T00:49:08.223 INFO:tasks.workunit.client.1.smithi088.stdout:6/924: write db/d3d/f122 [1980647,73628] 0 2024-03-24T00:49:08.227 INFO:tasks.workunit.client.0.smithi038.stdout:6/892: dread d1/d37/d84/f108 [0,4194304] 0 2024-03-24T00:49:08.227 INFO:tasks.workunit.client.0.smithi038.stdout:6/893: readlink d1/da/d61/d69/lf1 0 2024-03-24T00:49:08.231 INFO:tasks.workunit.client.1.smithi088.stdout:5/908: truncate d1/d10/db6/d10b/d2e/d9f/ddf/f80 39449 0 2024-03-24T00:49:08.234 INFO:tasks.workunit.client.0.smithi038.stdout:7/766: rename d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/laa to d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/l11a 0 2024-03-24T00:49:08.236 INFO:tasks.workunit.client.1.smithi088.stdout:8/843: link d9/dc/d11/d24/d43/lf1 d9/dc/d11/d24/l11f 0 2024-03-24T00:49:08.236 INFO:tasks.workunit.client.1.smithi088.stdout:8/844: write d9/dc/d11/d24/d29/d36/dc1/dc4/ffe [617230,69232] 0 2024-03-24T00:49:08.238 INFO:tasks.workunit.client.0.smithi038.stdout:6/894: rename d1/da/d61/d69/c109 to d1/da/d61/d69/c128 0 2024-03-24T00:49:08.242 INFO:tasks.workunit.client.0.smithi038.stdout:0/777: rename d0/d12/d42/d9d/d28/f9b to d0/d12/d15/d58/d64/f108 0 2024-03-24T00:49:08.246 INFO:tasks.workunit.client.0.smithi038.stdout:6/895: mkdir d1/da/d61/d129 0 2024-03-24T00:49:08.247 INFO:tasks.workunit.client.0.smithi038.stdout:6/896: chown d1/da/d1b/d25/f2a 7 1 2024-03-24T00:49:08.255 INFO:tasks.workunit.client.1.smithi088.stdout:9/856: dwrite d1/d15/d21/fc8 [0,4194304] 0 2024-03-24T00:49:08.255 INFO:tasks.workunit.client.1.smithi088.stdout:9/857: readlink d1/d15/d80/da5/le9 0 2024-03-24T00:49:08.256 INFO:tasks.workunit.client.0.smithi038.stdout:5/792: dread d6/d19/d20/d42/d43/d4c/f62 [0,4194304] 0 2024-03-24T00:49:08.256 INFO:tasks.workunit.client.0.smithi038.stdout:5/793: chown d6/d16/d21/lf2 1443100632 1 2024-03-24T00:49:08.257 INFO:tasks.workunit.client.1.smithi088.stdout:2/811: sync 2024-03-24T00:49:08.257 INFO:tasks.workunit.client.0.smithi038.stdout:9/913: write d12/d28/f70 [195737,29292] 0 2024-03-24T00:49:08.257 INFO:tasks.workunit.client.0.smithi038.stdout:4/768: rename da/d12/d24/d68/d6c/d7c/d105 to da/d12/d24/d5f/d108 0 2024-03-24T00:49:08.262 INFO:tasks.workunit.client.0.smithi038.stdout:6/897: creat d1/da/d1b/d1d/d3c/f12a x:0 0 0 2024-03-24T00:49:08.264 INFO:tasks.workunit.client.1.smithi088.stdout:0/871: dwrite d9/d26/d8f/da1/d62/f6a [4194304,4194304] 0 2024-03-24T00:49:08.266 INFO:tasks.workunit.client.0.smithi038.stdout:2/742: sync 2024-03-24T00:49:08.269 INFO:tasks.workunit.client.0.smithi038.stdout:1/867: sync 2024-03-24T00:49:08.270 INFO:tasks.workunit.client.1.smithi088.stdout:3/820: dwrite d3/de/d6f/d85/d31/f94 [0,4194304] 0 2024-03-24T00:49:08.274 INFO:tasks.workunit.client.0.smithi038.stdout:7/767: write d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fc7 [3566539,78387] 0 2024-03-24T00:49:08.274 INFO:tasks.workunit.client.0.smithi038.stdout:7/768: chown d2/d9/da/dc/d8a/d105 519410 1 2024-03-24T00:49:08.275 INFO:tasks.workunit.client.0.smithi038.stdout:3/809: rename d3/d30/d5d/f95 to d3/d30/d70/d90/d99/f114 0 2024-03-24T00:49:08.275 INFO:tasks.workunit.client.0.smithi038.stdout:3/810: creat d3/d30/d70/d90/d99/f115 x:0 0 0 2024-03-24T00:49:08.275 INFO:tasks.workunit.client.0.smithi038.stdout:3/811: dread - d3/d9/dc/da6/db5/d19/dd2/df0/ff2 zero size 2024-03-24T00:49:08.275 INFO:tasks.workunit.client.0.smithi038.stdout:3/812: fsync d3/d9/d77/f7f 0 2024-03-24T00:49:08.280 INFO:tasks.workunit.client.0.smithi038.stdout:4/769: dread da/d12/d24/d68/d6c/f85 [0,4194304] 0 2024-03-24T00:49:08.280 INFO:tasks.workunit.client.0.smithi038.stdout:4/770: readlink da/d12/d37/dd/d15/d4c/d70/lb8 0 2024-03-24T00:49:08.280 INFO:tasks.workunit.client.0.smithi038.stdout:4/771: write da/d12/d37/dd/f95 [995126,55160] 0 2024-03-24T00:49:08.281 INFO:tasks.workunit.client.1.smithi088.stdout:5/909: creat d1/d10/d4f/f133 x:0 0 0 2024-03-24T00:49:08.281 INFO:tasks.workunit.client.1.smithi088.stdout:8/845: rmdir d9/dc/d11/d24/d29/d36/d6a 39 2024-03-24T00:49:08.281 INFO:tasks.workunit.client.0.smithi038.stdout:8/841: dwrite d0/d1d/d27/f8e [0,4194304] 0 2024-03-24T00:49:08.282 INFO:tasks.workunit.client.0.smithi038.stdout:8/842: write d0/d1d/f22 [1308576,114689] 0 2024-03-24T00:49:08.282 INFO:tasks.workunit.client.0.smithi038.stdout:8/843: fsync d0/d15/fcf 0 2024-03-24T00:49:08.282 INFO:tasks.workunit.client.0.smithi038.stdout:8/844: chown d0/d1d/d27/l3f 6627387 1 2024-03-24T00:49:08.282 INFO:tasks.workunit.client.1.smithi088.stdout:1/862: mkdir d6/d6e/dfc/d108/d5b/d11b 0 2024-03-24T00:49:08.282 INFO:tasks.workunit.client.0.smithi038.stdout:8/845: fsync d0/d15/fbd 0 2024-03-24T00:49:08.283 INFO:tasks.workunit.client.0.smithi038.stdout:8/846: read - d0/d45/d37/d5f/db7/dcc/fff zero size 2024-03-24T00:49:08.284 INFO:tasks.workunit.client.0.smithi038.stdout:1/868: stat da/dd/d4a/d76/l115 0 2024-03-24T00:49:08.284 INFO:tasks.workunit.client.0.smithi038.stdout:1/869: chown da/dd/d2c/d41 31070248 1 2024-03-24T00:49:08.284 INFO:tasks.workunit.client.0.smithi038.stdout:7/769: getdents d2/d9/da/d23/d2b/d5b/d72/d92/d98 0 2024-03-24T00:49:08.284 INFO:tasks.workunit.client.1.smithi088.stdout:9/858: creat d1/d10/d11/d23/f115 x:0 0 0 2024-03-24T00:49:08.284 INFO:tasks.workunit.client.1.smithi088.stdout:9/859: dread - d1/d22/d27/d39/d98/fae zero size 2024-03-24T00:49:08.285 INFO:tasks.workunit.client.0.smithi038.stdout:0/778: rename d0/d12/d15/l49 to d0/d12/d42/d5e/db4/l109 0 2024-03-24T00:49:08.289 INFO:tasks.workunit.client.1.smithi088.stdout:0/872: read d9/d26/d8f/dc7/fed [943124,130929] 0 2024-03-24T00:49:08.289 INFO:tasks.workunit.client.1.smithi088.stdout:0/873: fsync d9/f1c 0 2024-03-24T00:49:08.289 INFO:tasks.workunit.client.1.smithi088.stdout:0/874: chown d9/d15/le9 1 1 2024-03-24T00:49:08.291 INFO:tasks.workunit.client.1.smithi088.stdout:0/875: write d9/d26/d8f/da1/fc2 [266609,48445] 0 2024-03-24T00:49:08.291 INFO:tasks.workunit.client.1.smithi088.stdout:0/876: readlink d9/dc/de/d19/l42 0 2024-03-24T00:49:08.300 INFO:tasks.workunit.client.1.smithi088.stdout:6/925: dwrite db/d3d/ff1 [0,4194304] 0 2024-03-24T00:49:08.301 INFO:tasks.workunit.client.1.smithi088.stdout:7/790: write d3/d32/d34/d9c/d4b/f5e [3692203,110979] 0 2024-03-24T00:49:08.302 INFO:tasks.workunit.client.1.smithi088.stdout:7/791: fdatasync d3/d31/f83 0 2024-03-24T00:49:08.302 INFO:tasks.workunit.client.0.smithi038.stdout:5/794: dread d6/d1e/d97/fc3 [0,4194304] 0 2024-03-24T00:49:08.305 INFO:tasks.workunit.client.0.smithi038.stdout:2/743: dread d4/d18/d38/f5a [0,4194304] 0 2024-03-24T00:49:08.305 INFO:tasks.workunit.client.0.smithi038.stdout:2/744: chown d4/d18/lfa 1 1 2024-03-24T00:49:08.306 INFO:tasks.workunit.client.0.smithi038.stdout:2/745: truncate d4/d18/d26/d37/ff7 410704 0 2024-03-24T00:49:08.313 INFO:tasks.workunit.client.0.smithi038.stdout:4/772: rmdir da/d12/d24/d5f 39 2024-03-24T00:49:08.313 INFO:tasks.workunit.client.0.smithi038.stdout:4/773: stat da/d12/f4e 0 2024-03-24T00:49:08.313 INFO:tasks.workunit.client.0.smithi038.stdout:4/774: write da/d38/fa3 [601037,17289] 0 2024-03-24T00:49:08.314 INFO:tasks.workunit.client.0.smithi038.stdout:8/847: dread d0/d15/f54 [0,4194304] 0 2024-03-24T00:49:08.314 INFO:tasks.workunit.client.0.smithi038.stdout:9/914: rmdir d12/d41/d57 39 2024-03-24T00:49:08.314 INFO:tasks.workunit.client.0.smithi038.stdout:9/915: write d12/d41/d57/dd3/dfe/f102 [688745,14181] 0 2024-03-24T00:49:08.314 INFO:tasks.workunit.client.0.smithi038.stdout:9/916: write d12/d28/d9c/ddd/d111/f77 [768196,105630] 0 2024-03-24T00:49:08.315 INFO:tasks.workunit.client.1.smithi088.stdout:8/846: mkdir d9/dc/d11/d24/d29/d36/dc1/dc4/d120 0 2024-03-24T00:49:08.319 INFO:tasks.workunit.client.1.smithi088.stdout:5/910: dread d1/d19/d48/fae [0,4194304] 0 2024-03-24T00:49:08.322 INFO:tasks.workunit.client.0.smithi038.stdout:1/870: mknod da/d3c/c11e 0 2024-03-24T00:49:08.325 INFO:tasks.workunit.client.1.smithi088.stdout:1/863: creat d6/d6e/dfc/d108/d5b/f11c x:0 0 0 2024-03-24T00:49:08.325 INFO:tasks.workunit.client.1.smithi088.stdout:1/864: chown d6/f17 495740 1 2024-03-24T00:49:08.340 INFO:tasks.workunit.client.1.smithi088.stdout:9/860: creat d1/d10/d9b/dab/d7a/f116 x:0 0 0 2024-03-24T00:49:08.342 INFO:tasks.workunit.client.0.smithi038.stdout:5/795: creat d6/d19/d20/d64/d6b/d77/f102 x:0 0 0 2024-03-24T00:49:08.345 INFO:tasks.workunit.client.1.smithi088.stdout:2/812: getdents db/dd 0 2024-03-24T00:49:08.347 INFO:tasks.workunit.client.0.smithi038.stdout:2/746: symlink d4/d27/d54/d53/l100 0 2024-03-24T00:49:08.347 INFO:tasks.workunit.client.0.smithi038.stdout:2/747: dread - d4/d18/fcf zero size 2024-03-24T00:49:08.358 INFO:tasks.workunit.client.0.smithi038.stdout:4/775: dread da/d12/d24/d5f/d9a/f92 [0,4194304] 0 2024-03-24T00:49:08.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:07 smithi038 ceph-mon[29271]: pgmap v40: 65 pgs: 65 active+clean; 3.1 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 147 MiB/s rd, 165 MiB/s wr, 259 op/s 2024-03-24T00:49:08.363 INFO:tasks.workunit.client.1.smithi088.stdout:3/821: dwrite d3/d6/d19/f52 [0,4194304] 0 2024-03-24T00:49:08.365 INFO:tasks.workunit.client.0.smithi038.stdout:6/898: dwrite d1/d37/fa5 [0,4194304] 0 2024-03-24T00:49:08.369 INFO:tasks.workunit.client.0.smithi038.stdout:0/779: dwrite d0/d1b/d2c/fac [0,4194304] 0 2024-03-24T00:49:08.372 INFO:tasks.workunit.client.0.smithi038.stdout:3/813: rename d3/d2a/l1b to d3/d9/dc/d51/l116 0 2024-03-24T00:49:08.372 INFO:tasks.workunit.client.0.smithi038.stdout:3/814: creat d3/d9/dc/da6/db5/d19/d82/f117 x:0 0 0 2024-03-24T00:49:08.373 INFO:tasks.workunit.client.1.smithi088.stdout:6/926: getdents db/d1d 0 2024-03-24T00:49:08.374 INFO:tasks.workunit.client.0.smithi038.stdout:8/848: dwrite d0/f32 [0,4194304] 0 2024-03-24T00:49:08.374 INFO:tasks.workunit.client.0.smithi038.stdout:8/849: fsync d0/d15/f98 0 2024-03-24T00:49:08.376 INFO:tasks.workunit.client.0.smithi038.stdout:2/748: truncate d4/d18/d38/f5a 763733 0 2024-03-24T00:49:08.376 INFO:tasks.workunit.client.0.smithi038.stdout:2/749: fdatasync d4/da3/fb7 0 2024-03-24T00:49:08.376 INFO:tasks.workunit.client.0.smithi038.stdout:2/750: creat d4/d18/d38/d63/f101 x:0 0 0 2024-03-24T00:49:08.376 INFO:tasks.workunit.client.0.smithi038.stdout:2/751: chown d4/d27/d2f/d30/f8d 1656587 1 2024-03-24T00:49:08.378 INFO:tasks.workunit.client.0.smithi038.stdout:4/776: creat da/d12/d24/d5f/d8a/db6/f109 x:0 0 0 2024-03-24T00:49:08.380 INFO:tasks.workunit.client.1.smithi088.stdout:8/847: creat d9/dc/d11/d24/d43/d4a/dee/f121 x:0 0 0 2024-03-24T00:49:08.380 INFO:tasks.workunit.client.1.smithi088.stdout:8/848: chown d9/dc/dce/ddc/l114 158 1 2024-03-24T00:49:08.380 INFO:tasks.workunit.client.1.smithi088.stdout:8/849: write d9/dc/dce/ddc/dde/f11a [751713,56207] 0 2024-03-24T00:49:08.382 INFO:tasks.workunit.client.1.smithi088.stdout:5/911: mknod d1/dd/d6f/da1/c134 0 2024-03-24T00:49:08.382 INFO:tasks.workunit.client.1.smithi088.stdout:5/912: dread - d1/d10/db6/d10b/d2f/d57/d9e/f11d zero size 2024-03-24T00:49:08.383 INFO:tasks.workunit.client.0.smithi038.stdout:7/770: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fd5 [0,4194304] 0 2024-03-24T00:49:08.383 INFO:tasks.workunit.client.0.smithi038.stdout:7/771: truncate d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f97 787259 0 2024-03-24T00:49:08.383 INFO:tasks.workunit.client.0.smithi038.stdout:7/772: dread - d2/d9/da/d23/d2b/d5b/d72/d92/d5d/dcd/f104 zero size 2024-03-24T00:49:08.385 INFO:tasks.workunit.client.1.smithi088.stdout:1/865: link d6/d6e/dfc/d108/d3f/f41 d6/d6e/f11d 0 2024-03-24T00:49:08.390 INFO:tasks.workunit.client.1.smithi088.stdout:0/877: dwrite d9/d26/d8f/da1/d62/d9b/db0/fb2 [0,4194304] 0 2024-03-24T00:49:08.398 INFO:tasks.workunit.client.1.smithi088.stdout:9/861: unlink d1/d10/d29/fa6 0 2024-03-24T00:49:08.398 INFO:tasks.workunit.client.1.smithi088.stdout:9/862: truncate d1/d22/d27/d39/ffd 547506 0 2024-03-24T00:49:08.398 INFO:tasks.workunit.client.1.smithi088.stdout:2/813: creat db/dd/d96/d99/db2/d88/f104 x:0 0 0 2024-03-24T00:49:08.399 INFO:tasks.workunit.client.0.smithi038.stdout:1/871: rmdir da/d3c/d86/def 39 2024-03-24T00:49:08.399 INFO:tasks.workunit.client.0.smithi038.stdout:1/872: fsync da/ddc/f10f 0 2024-03-24T00:49:08.401 INFO:tasks.workunit.client.1.smithi088.stdout:3/822: link d3/c98 d3/d1a/d1e/c105 0 2024-03-24T00:49:08.401 INFO:tasks.workunit.client.1.smithi088.stdout:3/823: fsync d3/f95 0 2024-03-24T00:49:08.402 INFO:tasks.workunit.client.1.smithi088.stdout:3/824: creat d3/d6/d58/da4/f106 x:0 0 0 2024-03-24T00:49:08.405 INFO:tasks.workunit.client.1.smithi088.stdout:7/792: symlink d3/lf9 0 2024-03-24T00:49:08.410 INFO:tasks.workunit.client.0.smithi038.stdout:3/815: write d3/d9/dc/da6/db5/d19/d82/d89/fa3 [334098,78910] 0 2024-03-24T00:49:08.410 INFO:tasks.workunit.client.0.smithi038.stdout:3/816: truncate d3/d9/dc/da6/db5/d19/d82/ffb 1517132 0 2024-03-24T00:49:08.410 INFO:tasks.workunit.client.0.smithi038.stdout:3/817: truncate d3/dc3/fcb 793715 0 2024-03-24T00:49:08.412 INFO:tasks.workunit.client.0.smithi038.stdout:0/780: mknod d0/daf/c10a 0 2024-03-24T00:49:08.412 INFO:tasks.workunit.client.0.smithi038.stdout:0/781: truncate d0/d12/d15/d58/d64/f108 96896 0 2024-03-24T00:49:08.420 INFO:tasks.workunit.client.1.smithi088.stdout:1/866: rmdir d6/d6e/dfc/d108/d5b/d5d/d61 39 2024-03-24T00:49:08.424 INFO:tasks.workunit.client.0.smithi038.stdout:8/850: creat d0/d1d/d27/d6b/da4/f11b x:0 0 0 2024-03-24T00:49:08.425 INFO:tasks.workunit.client.0.smithi038.stdout:8/851: fdatasync d0/d15/fcf 0 2024-03-24T00:49:08.425 INFO:tasks.workunit.client.0.smithi038.stdout:8/852: write d0/d45/d37/d5f/db7/dcc/fef [5070246,63693] 0 2024-03-24T00:49:08.427 INFO:tasks.workunit.client.0.smithi038.stdout:5/796: dwrite d6/d19/d20/fcf [0,4194304] 0 2024-03-24T00:49:08.427 INFO:tasks.workunit.client.0.smithi038.stdout:5/797: fdatasync d6/f26 0 2024-03-24T00:49:08.428 INFO:tasks.workunit.client.0.smithi038.stdout:5/798: readlink d6/d16/d21/d2d/l4a 0 2024-03-24T00:49:08.433 INFO:tasks.workunit.client.1.smithi088.stdout:0/878: write d9/dc/de/d19/d44/d46/d65/fb5 [442984,21877] 0 2024-03-24T00:49:08.433 INFO:tasks.workunit.client.1.smithi088.stdout:0/879: write d9/dc/de0/fe4 [3504832,109139] 0 2024-03-24T00:49:08.439 INFO:tasks.workunit.client.0.smithi038.stdout:4/777: mkdir da/d12/d37/dd/d15/d4c/d70/d10a 0 2024-03-24T00:49:08.445 INFO:tasks.workunit.client.1.smithi088.stdout:2/814: unlink db/dc/f93 0 2024-03-24T00:49:08.447 INFO:tasks.workunit.client.0.smithi038.stdout:7/773: symlink d2/d9/da/dc/d8a/d105/l11b 0 2024-03-24T00:49:08.447 INFO:tasks.workunit.client.0.smithi038.stdout:7/774: chown d2/d9/f54 1053 1 2024-03-24T00:49:08.448 INFO:tasks.workunit.client.0.smithi038.stdout:6/899: rmdir d1/da/d1b/d25/d49/d4e/ddf 39 2024-03-24T00:49:08.450 INFO:tasks.workunit.client.1.smithi088.stdout:8/850: dwrite d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/f9a [0,4194304] 0 2024-03-24T00:49:08.451 INFO:tasks.workunit.client.1.smithi088.stdout:8/851: getdents d9/dc/d11/d24/d29/d39/d84/deb 0 2024-03-24T00:49:08.451 INFO:tasks.workunit.client.1.smithi088.stdout:3/825: mkdir d3/de/d6f/d85/d107 0 2024-03-24T00:49:08.451 INFO:tasks.workunit.client.1.smithi088.stdout:3/826: truncate d3/d6/dca/dae/d4b/fbf 4986831 0 2024-03-24T00:49:08.451 INFO:tasks.workunit.client.1.smithi088.stdout:3/827: write d3/d6/d19/d2a/f53 [878679,47969] 0 2024-03-24T00:49:08.452 INFO:tasks.workunit.client.0.smithi038.stdout:1/873: symlink da/dd/d7a/d82/d98/d14/dbb/l11f 0 2024-03-24T00:49:08.452 INFO:tasks.workunit.client.0.smithi038.stdout:1/874: chown da/dd/d7a/d82/d98/f15 54245 1 2024-03-24T00:49:08.452 INFO:tasks.workunit.client.0.smithi038.stdout:5/799: read d6/d19/d20/d42/d43/d50/f81 [765352,35663] 0 2024-03-24T00:49:08.452 INFO:tasks.workunit.client.0.smithi038.stdout:5/800: readlink d6/d19/d20/d42/d43/d68/d6e/ld8 0 2024-03-24T00:49:08.453 INFO:tasks.workunit.client.1.smithi088.stdout:6/927: dwrite db/d1d/d23/d5c/f67 [0,4194304] 0 2024-03-24T00:49:08.453 INFO:tasks.workunit.client.1.smithi088.stdout:6/928: dread - db/d1d/d23/faa zero size 2024-03-24T00:49:08.453 INFO:tasks.workunit.client.1.smithi088.stdout:7/793: mknod d3/d32/d34/d9c/d4b/ddc/cfa 0 2024-03-24T00:49:08.453 INFO:tasks.workunit.client.1.smithi088.stdout:7/794: read - d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/fb6 zero size 2024-03-24T00:49:08.455 INFO:tasks.workunit.client.1.smithi088.stdout:9/863: dwrite d1/ff [0,4194304] 0 2024-03-24T00:49:08.457 INFO:tasks.workunit.client.0.smithi038.stdout:3/818: dwrite d3/d30/f33 [0,4194304] 0 2024-03-24T00:49:08.457 INFO:tasks.workunit.client.0.smithi038.stdout:3/819: chown d3/d30/f33 14292 1 2024-03-24T00:49:08.458 INFO:tasks.workunit.client.1.smithi088.stdout:1/867: creat d6/d27/d103/f11e x:0 0 0 2024-03-24T00:49:08.458 INFO:tasks.workunit.client.1.smithi088.stdout:1/868: write d6/fab [111810,61151] 0 2024-03-24T00:49:08.460 INFO:tasks.workunit.client.0.smithi038.stdout:0/782: rmdir d0/d1b/df1 39 2024-03-24T00:49:08.468 INFO:tasks.workunit.client.0.smithi038.stdout:2/752: mkdir d4/d102 0 2024-03-24T00:49:08.468 INFO:tasks.workunit.client.0.smithi038.stdout:2/753: fdatasync d4/d18/d38/d63/f7e 0 2024-03-24T00:49:08.472 INFO:tasks.workunit.client.0.smithi038.stdout:5/801: write d6/d19/d20/d42/f72 [69143,4021] 0 2024-03-24T00:49:08.473 INFO:tasks.workunit.client.1.smithi088.stdout:0/880: truncate d9/f56 7596995 0 2024-03-24T00:49:08.473 INFO:tasks.workunit.client.0.smithi038.stdout:3/820: write d3/f49 [2297176,41803] 0 2024-03-24T00:49:08.475 INFO:tasks.workunit.client.0.smithi038.stdout:4/778: mkdir da/dbd/d10b 0 2024-03-24T00:49:08.475 INFO:tasks.workunit.client.0.smithi038.stdout:4/779: dread - da/d12/d24/d68/d6c/fae zero size 2024-03-24T00:49:08.477 INFO:tasks.workunit.client.1.smithi088.stdout:2/815: mkdir db/dd/d96/d105 0 2024-03-24T00:49:08.478 INFO:tasks.workunit.client.0.smithi038.stdout:7/775: mknod d2/d9/da/d23/def/df0/c11c 0 2024-03-24T00:49:08.483 INFO:tasks.workunit.client.0.smithi038.stdout:6/900: write d1/d37/fa5 [2440990,70979] 0 2024-03-24T00:49:08.492 INFO:tasks.workunit.client.1.smithi088.stdout:8/852: rename d9/f37 to d9/dc/d11/d24/f122 0 2024-03-24T00:49:08.493 INFO:tasks.workunit.client.1.smithi088.stdout:7/795: truncate d3/f2b 432387 0 2024-03-24T00:49:08.493 INFO:tasks.workunit.client.1.smithi088.stdout:4/868: sync 2024-03-24T00:49:08.494 INFO:tasks.workunit.client.1.smithi088.stdout:4/869: fdatasync d5/d7/f44 0 2024-03-24T00:49:08.494 INFO:tasks.workunit.client.1.smithi088.stdout:9/864: creat d1/d15/f117 x:0 0 0 2024-03-24T00:49:08.497 INFO:tasks.workunit.client.1.smithi088.stdout:1/869: mknod d6/d23/daf/c11f 0 2024-03-24T00:49:08.504 INFO:tasks.workunit.client.0.smithi038.stdout:2/754: mkdir d4/d27/ddb/d7c/d103 0 2024-03-24T00:49:08.509 INFO:tasks.workunit.client.0.smithi038.stdout:4/780: truncate da/d12/d37/dd/d15/d4c/d70/f9b 2174012 0 2024-03-24T00:49:08.510 INFO:tasks.workunit.client.1.smithi088.stdout:0/881: mkdir d9/dc/de/d19/d84/dd6/d116 0 2024-03-24T00:49:08.510 INFO:tasks.workunit.client.1.smithi088.stdout:0/882: stat d9/dc/de/d19/d8d/d97 0 2024-03-24T00:49:08.512 INFO:tasks.workunit.client.1.smithi088.stdout:3/828: rename d3/d6/fee to d3/d6/dca/dae/d4b/f108 0 2024-03-24T00:49:08.512 INFO:tasks.workunit.client.1.smithi088.stdout:3/829: chown d3/de/d6f/dcc/ff0 120467 1 2024-03-24T00:49:08.513 INFO:tasks.workunit.client.0.smithi038.stdout:6/901: symlink d1/da/d1b/d1d/l12b 0 2024-03-24T00:49:08.517 INFO:tasks.workunit.client.0.smithi038.stdout:3/821: rmdir d3/d9/dc/da6/db5/d19/da4 39 2024-03-24T00:49:08.517 INFO:tasks.workunit.client.0.smithi038.stdout:3/822: fsync d3/d30/d5d/fca 0 2024-03-24T00:49:08.519 INFO:tasks.workunit.client.1.smithi088.stdout:8/853: mkdir d9/dc/d11/d24/d29/d36/dc1/dc4/d120/d123 0 2024-03-24T00:49:08.520 INFO:tasks.workunit.client.1.smithi088.stdout:7/796: creat d3/d32/d34/d9c/d4b/d74/d79/d7f/ffb x:0 0 0 2024-03-24T00:49:08.526 INFO:tasks.workunit.client.0.smithi038.stdout:4/781: link da/d12/d37/d83/ca8 da/d12/d37/dd/d15/d4c/c10c 0 2024-03-24T00:49:08.531 INFO:tasks.workunit.client.1.smithi088.stdout:4/870: mkdir d5/d7/d92/d121/d133 0 2024-03-24T00:49:08.531 INFO:tasks.workunit.client.1.smithi088.stdout:4/871: fdatasync d5/d13/d63/d78/fd6 0 2024-03-24T00:49:08.534 INFO:tasks.workunit.client.0.smithi038.stdout:0/783: write d0/d12/d15/d58/d64/f7d [136256,10409] 0 2024-03-24T00:49:08.534 INFO:tasks.workunit.client.0.smithi038.stdout:0/784: chown d0/d12/fba 16262 1 2024-03-24T00:49:08.534 INFO:tasks.workunit.client.0.smithi038.stdout:5/802: dwrite d6/d19/d20/d42/d43/d4c/f9d [0,4194304] 0 2024-03-24T00:49:08.534 INFO:tasks.workunit.client.0.smithi038.stdout:5/803: readlink d6/d19/d20/d42/d43/d68/d6e/lec 0 2024-03-24T00:49:08.534 INFO:tasks.workunit.client.0.smithi038.stdout:5/804: stat d6/d16/d21/d2d/lf8 0 2024-03-24T00:49:08.535 INFO:tasks.workunit.client.0.smithi038.stdout:1/875: dwrite da/dd/d7a/d82/d98/d34/fdf [0,4194304] 0 2024-03-24T00:49:08.535 INFO:tasks.workunit.client.1.smithi088.stdout:7/797: dread d3/f24 [0,4194304] 0 2024-03-24T00:49:08.536 INFO:tasks.workunit.client.0.smithi038.stdout:6/902: symlink d1/da/d1b/d25/d49/d4e/ddf/l12c 0 2024-03-24T00:49:08.537 INFO:tasks.workunit.client.1.smithi088.stdout:9/865: mkdir d1/db5/d118 0 2024-03-24T00:49:08.541 INFO:tasks.workunit.client.0.smithi038.stdout:3/823: link d3/d9/dc/da6/db5/d19/d82/fd3 d3/d30/d70/d90/d99/f118 0 2024-03-24T00:49:08.541 INFO:tasks.workunit.client.0.smithi038.stdout:3/824: read - d3/d5a/fc5 zero size 2024-03-24T00:49:08.542 INFO:tasks.workunit.client.0.smithi038.stdout:3/825: creat d3/d9/dc/da6/db5/d19/d82/f119 x:0 0 0 2024-03-24T00:49:08.542 INFO:tasks.workunit.client.0.smithi038.stdout:3/826: creat d3/d9/dc/da6/db5/dad/f11a x:0 0 0 2024-03-24T00:49:08.543 INFO:tasks.workunit.client.1.smithi088.stdout:0/883: creat d9/dc/de/d19/d84/dd6/f117 x:0 0 0 2024-03-24T00:49:08.544 INFO:tasks.workunit.client.1.smithi088.stdout:7/798: dread d3/d32/d76/f87 [0,4194304] 0 2024-03-24T00:49:08.544 INFO:tasks.workunit.client.1.smithi088.stdout:7/799: fsync d3/d32/d34/d9c/d4b/d74/d79/fb0 0 2024-03-24T00:49:08.545 INFO:tasks.workunit.client.0.smithi038.stdout:7/776: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fd5 [0,4194304] 0 2024-03-24T00:49:08.545 INFO:tasks.workunit.client.0.smithi038.stdout:4/782: rename da/dbd/d10b to da/d12/d29/d10d 0 2024-03-24T00:49:08.548 INFO:tasks.workunit.client.1.smithi088.stdout:6/929: rename db/dd/d61/c69 to db/d1d/d23/c127 0 2024-03-24T00:49:08.551 INFO:tasks.workunit.client.1.smithi088.stdout:1/870: dread d6/d27/f8e [0,4194304] 0 2024-03-24T00:49:08.551 INFO:tasks.workunit.client.1.smithi088.stdout:2/816: dwrite db/dd/d96/d4e/d8f/d9c/db0/ffc [0,4194304] 0 2024-03-24T00:49:08.556 INFO:tasks.workunit.client.0.smithi038.stdout:0/785: truncate d0/d12/d42/f61 8038982 0 2024-03-24T00:49:08.556 INFO:tasks.workunit.client.1.smithi088.stdout:8/854: mknod d9/dc/d11/d24/d119/c124 0 2024-03-24T00:49:08.556 INFO:tasks.workunit.client.1.smithi088.stdout:8/855: stat d9/dc/d11/d24/d29/d39/d84/lbb 0 2024-03-24T00:49:08.558 INFO:tasks.workunit.client.1.smithi088.stdout:0/884: link d9/dc/de/cc3 d9/d26/d8f/da1/d62/d9b/db0/c118 0 2024-03-24T00:49:08.569 INFO:tasks.workunit.client.0.smithi038.stdout:5/805: mknod d6/d19/d20/d42/d43/d50/c103 0 2024-03-24T00:49:08.570 INFO:tasks.workunit.client.0.smithi038.stdout:5/806: chown d6/d1e/d97/db0/cc5 16118 1 2024-03-24T00:49:08.571 INFO:tasks.workunit.client.0.smithi038.stdout:1/876: mknod da/d3c/d86/d111/c120 0 2024-03-24T00:49:08.573 INFO:tasks.workunit.client.1.smithi088.stdout:2/817: symlink db/dd/d96/d4e/d8f/d9c/db0/l106 0 2024-03-24T00:49:08.573 INFO:tasks.workunit.client.1.smithi088.stdout:2/818: stat db/dd/d96/cb8 0 2024-03-24T00:49:08.573 INFO:tasks.workunit.client.1.smithi088.stdout:3/830: dwrite d3/d6/dca/dae/f50 [0,4194304] 0 2024-03-24T00:49:08.574 INFO:tasks.workunit.client.0.smithi038.stdout:6/903: creat d1/d37/d68/f12d x:0 0 0 2024-03-24T00:49:08.574 INFO:tasks.workunit.client.1.smithi088.stdout:6/930: dread db/dd/d61/fc4 [0,4194304] 0 2024-03-24T00:49:08.576 INFO:tasks.workunit.client.1.smithi088.stdout:5/913: sync 2024-03-24T00:49:08.581 INFO:tasks.workunit.client.0.smithi038.stdout:4/783: dread da/d12/d37/dd/d15/d6d/f88 [0,4194304] 0 2024-03-24T00:49:08.581 INFO:tasks.workunit.client.0.smithi038.stdout:4/784: fdatasync da/d12/d29/d5b/d73/db4/fc3 0 2024-03-24T00:49:08.581 INFO:tasks.workunit.client.1.smithi088.stdout:8/856: creat d9/dc/dce/ddc/f125 x:0 0 0 2024-03-24T00:49:08.582 INFO:tasks.workunit.client.1.smithi088.stdout:8/857: dread - d9/dc/d11/d24/d29/da9/f110 zero size 2024-03-24T00:49:08.586 INFO:tasks.workunit.client.0.smithi038.stdout:2/755: dwrite d4/d27/ddb/f61 [0,4194304] 0 2024-03-24T00:49:08.586 INFO:tasks.workunit.client.0.smithi038.stdout:1/877: read da/dd/fb1 [55722,80249] 0 2024-03-24T00:49:08.586 INFO:tasks.workunit.client.0.smithi038.stdout:2/756: fdatasync d4/d18/d38/d63/f92 0 2024-03-24T00:49:08.586 INFO:tasks.workunit.client.0.smithi038.stdout:2/757: write d4/d18/d26/d37/d69/d83/fb0 [1418419,60162] 0 2024-03-24T00:49:08.589 INFO:tasks.workunit.client.0.smithi038.stdout:3/827: stat d3/d9/dc/d51/f75 0 2024-03-24T00:49:08.589 INFO:tasks.workunit.client.0.smithi038.stdout:3/828: chown d3/d9/dc/df/f3a 11842271 1 2024-03-24T00:49:08.595 INFO:tasks.workunit.client.1.smithi088.stdout:0/885: creat d9/f119 x:0 0 0 2024-03-24T00:49:08.595 INFO:tasks.workunit.client.1.smithi088.stdout:3/831: mknod d3/d67/dc3/dd7/c109 0 2024-03-24T00:49:08.595 INFO:tasks.workunit.client.1.smithi088.stdout:5/914: mkdir d1/dd/d6f/da1/d135 0 2024-03-24T00:49:08.600 INFO:tasks.workunit.client.0.smithi038.stdout:7/777: creat d2/d9/da/dc/f11d x:0 0 0 2024-03-24T00:49:08.601 INFO:tasks.workunit.client.1.smithi088.stdout:8/858: rmdir d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d 39 2024-03-24T00:49:08.604 INFO:tasks.workunit.client.1.smithi088.stdout:0/886: link d9/dc/de/d19/d84/ceb d9/dc/de/d19/d44/d46/d59/c11a 0 2024-03-24T00:49:08.611 INFO:tasks.workunit.client.1.smithi088.stdout:3/832: mkdir d3/d6/d19/d2a/dc2/dd5/d10a 0 2024-03-24T00:49:08.611 INFO:tasks.workunit.client.1.smithi088.stdout:2/819: dread db/dd/d96/d99/db2/d88/f97 [0,4194304] 0 2024-03-24T00:49:08.616 INFO:tasks.workunit.client.0.smithi038.stdout:4/785: creat da/d12/d24/d5f/d8a/db6/dd1/f10e x:0 0 0 2024-03-24T00:49:08.619 INFO:tasks.workunit.client.1.smithi088.stdout:2/820: unlink f1 0 2024-03-24T00:49:08.622 INFO:tasks.workunit.client.1.smithi088.stdout:0/887: dread d9/fd0 [0,4194304] 0 2024-03-24T00:49:08.623 INFO:tasks.workunit.client.1.smithi088.stdout:0/888: readlink d9/dc/de/d7e/lbd 0 2024-03-24T00:49:08.623 INFO:tasks.workunit.client.1.smithi088.stdout:0/889: truncate d9/dc/de/fcf 721880 0 2024-03-24T00:49:08.625 INFO:tasks.workunit.client.1.smithi088.stdout:2/821: mknod db/dd/d96/d4e/d8f/d9c/c107 0 2024-03-24T00:49:08.625 INFO:tasks.workunit.client.1.smithi088.stdout:2/822: dread - db/d21/d35/dcb/fce zero size 2024-03-24T00:49:08.635 INFO:tasks.workunit.client.1.smithi088.stdout:2/823: dread db/d21/f2e [0,4194304] 0 2024-03-24T00:49:08.636 INFO:tasks.workunit.client.1.smithi088.stdout:0/890: truncate d9/d26/d8f/da1/fc2 835624 0 2024-03-24T00:49:08.640 INFO:tasks.workunit.client.0.smithi038.stdout:3/829: creat d3/d9/dc/da6/db5/d19/d82/d89/f11b x:0 0 0 2024-03-24T00:49:08.640 INFO:tasks.workunit.client.0.smithi038.stdout:4/786: mknod da/c10f 0 2024-03-24T00:49:08.646 INFO:tasks.workunit.client.1.smithi088.stdout:5/915: dread d1/d19/fc0 [0,4194304] 0 2024-03-24T00:49:08.646 INFO:tasks.workunit.client.1.smithi088.stdout:5/916: chown d1/dd/l2a 5896451 1 2024-03-24T00:49:08.647 INFO:tasks.workunit.client.0.smithi038.stdout:3/830: mkdir d3/d9/dc/da6/db5/d19/d82/d89/da8/d11c 0 2024-03-24T00:49:08.658 INFO:tasks.workunit.client.1.smithi088.stdout:0/891: rename d9/d26/d8f/da1/d62/d9b/lc0 to d9/d26/d8f/da1/d62/d9b/l11b 0 2024-03-24T00:49:08.659 INFO:tasks.workunit.client.1.smithi088.stdout:5/917: symlink d1/d19/d48/dc2/dcc/l136 0 2024-03-24T00:49:08.659 INFO:tasks.workunit.client.1.smithi088.stdout:1/871: dwrite d6/d6e/dfc/d108/d5b/fc7 [0,4194304] 0 2024-03-24T00:49:08.663 INFO:tasks.workunit.client.0.smithi038.stdout:3/831: symlink d3/d30/d5d/d7c/db0/l11d 0 2024-03-24T00:49:08.663 INFO:tasks.workunit.client.0.smithi038.stdout:3/832: fsync d3/d9/d77/fc9 0 2024-03-24T00:49:08.664 INFO:tasks.workunit.client.1.smithi088.stdout:2/824: dread db/d21/d35/f5a [0,4194304] 0 2024-03-24T00:49:08.664 INFO:tasks.workunit.client.1.smithi088.stdout:2/825: truncate db/d21/fb6 269061 0 2024-03-24T00:49:08.668 INFO:tasks.workunit.client.1.smithi088.stdout:5/918: rename d1/dd/d6f/da1/d3f/d88/d8b/d94 to d1/dd/d6f/da1/d135/d137 0 2024-03-24T00:49:08.668 INFO:tasks.workunit.client.1.smithi088.stdout:5/919: truncate d1/d10/db6/d10b/d2f/d57/d78/ffe 114764 0 2024-03-24T00:49:08.669 INFO:tasks.workunit.client.1.smithi088.stdout:9/866: dwrite d1/d15/d80/da5/fb2 [0,4194304] 0 2024-03-24T00:49:08.669 INFO:tasks.workunit.client.1.smithi088.stdout:9/867: chown d1/d10/d29/def/lfa 3244 1 2024-03-24T00:49:08.669 INFO:tasks.workunit.client.1.smithi088.stdout:9/868: fsync d1/f4 0 2024-03-24T00:49:08.669 INFO:tasks.workunit.client.1.smithi088.stdout:9/869: chown d1/d10/d9b/c110 38059 1 2024-03-24T00:49:08.670 INFO:tasks.workunit.client.1.smithi088.stdout:9/870: chown d1/d15/f117 0 1 2024-03-24T00:49:08.673 INFO:tasks.workunit.client.0.smithi038.stdout:1/878: dwrite da/dd/d22/d94/fa0 [0,4194304] 0 2024-03-24T00:49:08.673 INFO:tasks.workunit.client.0.smithi038.stdout:1/879: write da/dd/d7a/d82/d98/d14/f2e [356660,29802] 0 2024-03-24T00:49:08.685 INFO:tasks.workunit.client.1.smithi088.stdout:5/920: dread d1/d10/db6/d10b/d2f/f4a [0,4194304] 0 2024-03-24T00:49:08.687 INFO:tasks.workunit.client.0.smithi038.stdout:1/880: symlink da/d3c/d114/l121 0 2024-03-24T00:49:08.693 INFO:tasks.workunit.client.0.smithi038.stdout:6/904: dwrite d1/da/d1b/d25/f44 [4194304,4194304] 0 2024-03-24T00:49:08.696 INFO:tasks.workunit.client.1.smithi088.stdout:5/921: symlink d1/d10/db6/d10b/d2f/d57/d9e/dba/l138 0 2024-03-24T00:49:08.700 INFO:tasks.workunit.client.0.smithi038.stdout:2/758: dwrite d4/d18/d26/d37/fd8 [4194304,4194304] 0 2024-03-24T00:49:08.701 INFO:tasks.workunit.client.0.smithi038.stdout:7/778: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d98/f9f [0,4194304] 0 2024-03-24T00:49:08.704 INFO:tasks.workunit.client.1.smithi088.stdout:2/826: dread db/f89 [0,4194304] 0 2024-03-24T00:49:08.704 INFO:tasks.workunit.client.1.smithi088.stdout:6/931: dwrite db/d1d/f114 [0,4194304] 0 2024-03-24T00:49:08.704 INFO:tasks.workunit.client.1.smithi088.stdout:6/932: fdatasync db/d1d/d50/d90/d55/f10f 0 2024-03-24T00:49:08.704 INFO:tasks.workunit.client.1.smithi088.stdout:6/933: write db/dd/d61/d6d/d74/d88/da1/f119 [478669,20086] 0 2024-03-24T00:49:08.708 INFO:tasks.workunit.client.0.smithi038.stdout:5/807: dwrite d6/d19/d20/d42/d43/d50/fdd [0,4194304] 0 2024-03-24T00:49:08.708 INFO:tasks.workunit.client.0.smithi038.stdout:9/917: sync 2024-03-24T00:49:08.713 INFO:tasks.workunit.client.0.smithi038.stdout:0/786: dwrite d0/d14/de4/feb [0,4194304] 0 2024-03-24T00:49:08.714 INFO:tasks.workunit.client.0.smithi038.stdout:8/853: sync 2024-03-24T00:49:08.718 INFO:tasks.workunit.client.1.smithi088.stdout:3/833: dwrite d3/de/d6f/d85/f15 [4194304,4194304] 0 2024-03-24T00:49:08.718 INFO:tasks.workunit.client.1.smithi088.stdout:3/834: readlink d3/d6/dca/dae/d4b/daf/lf8 0 2024-03-24T00:49:08.718 INFO:tasks.workunit.client.1.smithi088.stdout:3/835: creat d3/d67/dc3/dd7/f10b x:0 0 0 2024-03-24T00:49:08.718 INFO:tasks.workunit.client.1.smithi088.stdout:5/922: mknod d1/d10/db6/d10b/d2f/d57/d9e/dba/c139 0 2024-03-24T00:49:08.719 INFO:tasks.workunit.client.1.smithi088.stdout:5/923: stat d1/d10/d81/ld5 0 2024-03-24T00:49:08.719 INFO:tasks.workunit.client.1.smithi088.stdout:9/871: write d1/d10/d11/f5f [3364459,91918] 0 2024-03-24T00:49:08.719 INFO:tasks.workunit.client.1.smithi088.stdout:8/859: dwrite d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/f91 [0,4194304] 0 2024-03-24T00:49:08.719 INFO:tasks.workunit.client.0.smithi038.stdout:2/759: mkdir d4/d27/d54/dab/d104 0 2024-03-24T00:49:08.719 INFO:tasks.workunit.client.0.smithi038.stdout:6/905: link d1/da/l29 d1/da/d1b/dfd/l12e 0 2024-03-24T00:49:08.739 INFO:tasks.workunit.client.0.smithi038.stdout:2/760: chown d4/f2b 28 1 2024-03-24T00:49:08.745 INFO:tasks.workunit.client.1.smithi088.stdout:0/892: dwrite d9/dc/de/d19/f93 [0,4194304] 0 2024-03-24T00:49:08.746 INFO:tasks.workunit.client.0.smithi038.stdout:0/787: dread d0/d12/f89 [0,4194304] 0 2024-03-24T00:49:08.746 INFO:tasks.workunit.client.0.smithi038.stdout:0/788: fsync d0/d12/d15/d58/d64/da7/dbb/dd4/dea/fb3 0 2024-03-24T00:49:08.746 INFO:tasks.workunit.client.0.smithi038.stdout:6/906: write d1/da/d1b/d1d/d3c/fb4 [451067,109099] 0 2024-03-24T00:49:08.746 INFO:tasks.workunit.client.0.smithi038.stdout:4/787: dwrite da/d12/d24/d5f/f97 [0,4194304] 0 2024-03-24T00:49:08.746 INFO:tasks.workunit.client.0.smithi038.stdout:4/788: readlink da/d12/laf 0 2024-03-24T00:49:08.746 INFO:tasks.workunit.client.0.smithi038.stdout:7/779: write d2/d9/da/d23/d2b/d5b/d72/d92/d5d/fda [4106721,18600] 0 2024-03-24T00:49:08.747 INFO:tasks.workunit.client.0.smithi038.stdout:7/780: readlink d2/d9/l5c 0 2024-03-24T00:49:08.750 INFO:tasks.workunit.client.0.smithi038.stdout:6/907: mkdir d1/da/d1b/dbb/d12f 0 2024-03-24T00:49:08.750 INFO:tasks.workunit.client.0.smithi038.stdout:6/908: read - d1/d37/d84/f10b zero size 2024-03-24T00:49:08.750 INFO:tasks.workunit.client.0.smithi038.stdout:6/909: chown d1/c23 37 1 2024-03-24T00:49:08.750 INFO:tasks.workunit.client.0.smithi038.stdout:6/910: chown d1/da/d1b/dbb/f124 12888271 1 2024-03-24T00:49:08.754 INFO:tasks.workunit.client.1.smithi088.stdout:8/860: creat d9/dc/d11/f126 x:0 0 0 2024-03-24T00:49:08.754 INFO:tasks.workunit.client.1.smithi088.stdout:8/861: fdatasync d9/dc/d11/d24/d29/da9/f10d 0 2024-03-24T00:49:08.756 INFO:tasks.workunit.client.0.smithi038.stdout:7/781: mkdir d2/d9/da/d9b/d11e 0 2024-03-24T00:49:08.757 INFO:tasks.workunit.client.1.smithi088.stdout:9/872: rmdir d1/d15/d21/df3/d10a 0 2024-03-24T00:49:08.757 INFO:tasks.workunit.client.1.smithi088.stdout:5/924: dread d1/d10/ff3 [0,4194304] 0 2024-03-24T00:49:08.757 INFO:tasks.workunit.client.1.smithi088.stdout:5/925: dread - d1/d43/d4e/d5b/d66/fee zero size 2024-03-24T00:49:08.759 INFO:tasks.workunit.client.0.smithi038.stdout:7/782: write d2/d2c/fd7 [875693,122760] 0 2024-03-24T00:49:08.766 INFO:tasks.workunit.client.0.smithi038.stdout:6/911: unlink d1/da/d61/da2/faf 0 2024-03-24T00:49:08.767 INFO:tasks.workunit.client.0.smithi038.stdout:6/912: chown d1/da/d1b/d25/d49/d4e/c5d 455 1 2024-03-24T00:49:08.767 INFO:tasks.workunit.client.0.smithi038.stdout:5/808: dread d6/d19/d20/d42/d43/d4c/f62 [0,4194304] 0 2024-03-24T00:49:08.767 INFO:tasks.workunit.client.0.smithi038.stdout:5/809: creat d6/d19/d20/f104 x:0 0 0 2024-03-24T00:49:08.767 INFO:tasks.workunit.client.1.smithi088.stdout:0/893: dread d9/fd0 [0,4194304] 0 2024-03-24T00:49:08.768 INFO:tasks.workunit.client.1.smithi088.stdout:0/894: chown d9/dc/de/d19/d44/d46/d65/d2a/fcc 56 1 2024-03-24T00:49:08.768 INFO:tasks.workunit.client.1.smithi088.stdout:8/862: mknod d9/dc/d11/d24/d29/d5b/d8b/dd3/c127 0 2024-03-24T00:49:08.770 INFO:tasks.workunit.client.0.smithi038.stdout:6/913: mknod d1/d37/d84/df5/c130 0 2024-03-24T00:49:08.770 INFO:tasks.workunit.client.0.smithi038.stdout:6/914: fdatasync d1/da/d61/d69/fc2 0 2024-03-24T00:49:08.770 INFO:tasks.workunit.client.0.smithi038.stdout:6/915: chown d1/da/d1b/d1d/c4a 596 1 2024-03-24T00:49:08.771 INFO:tasks.workunit.client.0.smithi038.stdout:3/833: dwrite d3/d30/fac [0,4194304] 0 2024-03-24T00:49:08.771 INFO:tasks.workunit.client.1.smithi088.stdout:5/926: creat d1/dd/d6f/da1/d135/d137/f13a x:0 0 0 2024-03-24T00:49:08.771 INFO:tasks.workunit.client.1.smithi088.stdout:5/927: creat d1/dd/f13b x:0 0 0 2024-03-24T00:49:08.772 INFO:tasks.workunit.client.0.smithi038.stdout:3/834: write d3/d9/d77/f7f [196446,96106] 0 2024-03-24T00:49:08.772 INFO:tasks.workunit.client.0.smithi038.stdout:3/835: chown d3/d9/d77/dd4/ce6 138 1 2024-03-24T00:49:08.775 INFO:tasks.workunit.client.1.smithi088.stdout:0/895: getdents d9/d26/d8f 0 2024-03-24T00:49:08.775 INFO:tasks.workunit.client.1.smithi088.stdout:2/827: write db/d21/d35/f79 [3950986,28579] 0 2024-03-24T00:49:08.776 INFO:tasks.workunit.client.1.smithi088.stdout:2/828: write db/dd/d96/d4e/d8f/fa7 [21031,62447] 0 2024-03-24T00:49:08.777 INFO:tasks.workunit.client.0.smithi038.stdout:5/810: truncate d6/d19/d20/d33/d6f/f7a 3569073 0 2024-03-24T00:49:08.781 INFO:tasks.workunit.client.1.smithi088.stdout:8/863: link d9/dc/d11/d24/d43/dd8/ff4 d9/dc/d11/d24/d29/da9/d9c/f128 0 2024-03-24T00:49:08.781 INFO:tasks.workunit.client.1.smithi088.stdout:8/864: write d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/f60 [4979278,3940] 0 2024-03-24T00:49:08.781 INFO:tasks.workunit.client.1.smithi088.stdout:8/865: truncate d9/dc/d11/d24/d29/d36/dc1/dc4/ffe 1454380 0 2024-03-24T00:49:08.783 INFO:tasks.workunit.client.0.smithi038.stdout:2/761: dread d4/f13 [0,4194304] 0 2024-03-24T00:49:08.784 INFO:tasks.workunit.client.1.smithi088.stdout:9/873: dread d1/d22/d27/d39/d98/da3/f9f [0,4194304] 0 2024-03-24T00:49:08.788 INFO:tasks.workunit.client.1.smithi088.stdout:6/934: dwrite db/dd/d61/d6d/d74/d88/fda [0,4194304] 0 2024-03-24T00:49:08.789 INFO:tasks.workunit.client.0.smithi038.stdout:1/881: dwrite da/f21 [4194304,4194304] 0 2024-03-24T00:49:08.790 INFO:tasks.workunit.client.1.smithi088.stdout:1/872: dwrite d6/d6e/dfc/d108/d5b/f105 [4194304,4194304] 0 2024-03-24T00:49:08.790 INFO:tasks.workunit.client.1.smithi088.stdout:1/873: stat d6/d6e/dfc/d108/d5b/d5e 0 2024-03-24T00:49:08.797 INFO:tasks.workunit.client.1.smithi088.stdout:5/928: rename d1/dd/d6f/da1/d3f/d88/d8b/fd8 to d1/d10/db6/d10b/d2f/d57/d9e/db1/ded/f13c 0 2024-03-24T00:49:08.799 INFO:tasks.workunit.client.0.smithi038.stdout:7/783: dread d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fd5 [0,4194304] 0 2024-03-24T00:49:08.800 INFO:tasks.workunit.client.1.smithi088.stdout:7/800: sync 2024-03-24T00:49:08.800 INFO:tasks.workunit.client.1.smithi088.stdout:4/872: sync 2024-03-24T00:49:08.800 INFO:tasks.workunit.client.1.smithi088.stdout:4/873: creat d5/d7/de4/d107/f134 x:0 0 0 2024-03-24T00:49:08.803 INFO:tasks.workunit.client.0.smithi038.stdout:3/836: mknod d3/d30/d5d/deb/c11e 0 2024-03-24T00:49:08.803 INFO:tasks.workunit.client.0.smithi038.stdout:3/837: rename d3/d30 to d3/d30/d5d/deb/d11f 22 2024-03-24T00:49:08.803 INFO:tasks.workunit.client.0.smithi038.stdout:3/838: chown d3/dc3/fcb 12785993 1 2024-03-24T00:49:08.804 INFO:tasks.workunit.client.0.smithi038.stdout:0/789: dread d0/d12/fba [0,4194304] 0 2024-03-24T00:49:08.804 INFO:tasks.workunit.client.0.smithi038.stdout:0/790: creat d0/d12/d15/d58/d64/da7/dbb/dcd/f10b x:0 0 0 2024-03-24T00:49:08.804 INFO:tasks.workunit.client.0.smithi038.stdout:0/791: write d0/d12/d9a/f9c [1167202,37595] 0 2024-03-24T00:49:08.804 INFO:tasks.workunit.client.0.smithi038.stdout:0/792: write d0/d12/d15/d1a/fcc [935060,126658] 0 2024-03-24T00:49:08.807 INFO:tasks.workunit.client.1.smithi088.stdout:9/874: read d1/d22/d72/f79 [1621458,116179] 0 2024-03-24T00:49:08.807 INFO:tasks.workunit.client.1.smithi088.stdout:3/836: dwrite d3/d6/d27/d3c/f42 [0,4194304] 0 2024-03-24T00:49:08.807 INFO:tasks.workunit.client.1.smithi088.stdout:3/837: chown d3/d32 3475 1 2024-03-24T00:49:08.807 INFO:tasks.workunit.client.1.smithi088.stdout:3/838: chown d3/d6/d27/d3c/d80/l86 971425 1 2024-03-24T00:49:08.808 INFO:tasks.workunit.client.0.smithi038.stdout:9/918: dwrite d12/d41/f6a [4194304,4194304] 0 2024-03-24T00:49:08.808 INFO:tasks.workunit.client.0.smithi038.stdout:9/919: write d12/d1a/d5f/df9/f117 [3286985,108193] 0 2024-03-24T00:49:08.808 INFO:tasks.workunit.client.0.smithi038.stdout:8/854: dread d0/d45/d37/d5f/d68/f91 [0,4194304] 0 2024-03-24T00:49:08.808 INFO:tasks.workunit.client.1.smithi088.stdout:0/896: link d9/dc/de/d19/d44/d46/d65/c90 d9/dc/de/d19/dd2/d103/c11c 0 2024-03-24T00:49:08.825 INFO:tasks.workunit.client.0.smithi038.stdout:5/811: creat d6/d19/d20/d42/d43/d50/d92/f105 x:0 0 0 2024-03-24T00:49:08.826 INFO:tasks.workunit.client.0.smithi038.stdout:1/882: creat da/dd/d7a/d82/ddb/f122 x:0 0 0 2024-03-24T00:49:08.834 INFO:tasks.workunit.client.0.smithi038.stdout:7/784: dread d2/f63 [0,4194304] 0 2024-03-24T00:49:08.834 INFO:tasks.workunit.client.0.smithi038.stdout:7/785: fdatasync d2/d9/da/dc/d8a/db8/fce 0 2024-03-24T00:49:08.837 INFO:tasks.workunit.client.0.smithi038.stdout:0/793: mknod d0/c10c 0 2024-03-24T00:49:08.849 INFO:tasks.workunit.client.1.smithi088.stdout:8/866: symlink d9/dc/l129 0 2024-03-24T00:49:08.849 INFO:tasks.workunit.client.1.smithi088.stdout:8/867: fdatasync d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/ff9 0 2024-03-24T00:49:08.849 INFO:tasks.workunit.client.1.smithi088.stdout:8/868: dread - d9/dc/d11/d24/d29/da9/f115 zero size 2024-03-24T00:49:08.849 INFO:tasks.workunit.client.1.smithi088.stdout:8/869: dread - d9/dc/d11/d24/d29/d39/d84/f11b zero size 2024-03-24T00:49:08.850 INFO:tasks.workunit.client.1.smithi088.stdout:5/929: dwrite d1/d19/d48/dc2/fc9 [8388608,4194304] 0 2024-03-24T00:49:08.852 INFO:tasks.workunit.client.1.smithi088.stdout:6/935: dwrite db/dd/d61/d6d/fe1 [0,4194304] 0 2024-03-24T00:49:08.853 INFO:tasks.workunit.client.0.smithi038.stdout:4/789: dwrite da/d12/d37/dd/d15/d6d/d8f/f102 [0,4194304] 0 2024-03-24T00:49:08.853 INFO:tasks.workunit.client.0.smithi038.stdout:4/790: write da/d12/d24/f26 [2026271,67267] 0 2024-03-24T00:49:08.855 INFO:tasks.workunit.client.0.smithi038.stdout:9/920: mkdir d12/d28/d9c/ddd/d129 0 2024-03-24T00:49:08.856 INFO:tasks.workunit.client.0.smithi038.stdout:9/921: chown d12/d41/lbb 35951969 1 2024-03-24T00:49:08.858 INFO:tasks.workunit.client.0.smithi038.stdout:5/812: symlink d6/d19/d20/d64/d6b/d77/l106 0 2024-03-24T00:49:08.859 INFO:tasks.workunit.client.0.smithi038.stdout:5/813: fdatasync d6/d19/d20/d42/d43/d4c/f74 0 2024-03-24T00:49:08.863 INFO:tasks.workunit.client.1.smithi088.stdout:8/870: dread d9/dc/d11/d24/d29/d39/d84/faa [0,4194304] 0 2024-03-24T00:49:08.866 INFO:tasks.workunit.client.0.smithi038.stdout:8/855: dread d0/d45/d37/d5f/db7/fb4 [0,4194304] 0 2024-03-24T00:49:08.866 INFO:tasks.workunit.client.0.smithi038.stdout:8/856: write d0/d1d/d20/d79/f104 [440688,95432] 0 2024-03-24T00:49:08.871 INFO:tasks.workunit.client.1.smithi088.stdout:9/875: dwrite d1/fcc [0,4194304] 0 2024-03-24T00:49:08.872 INFO:tasks.workunit.client.0.smithi038.stdout:3/839: dwrite d3/d9/dc/df/f3a [0,4194304] 0 2024-03-24T00:49:08.872 INFO:tasks.workunit.client.0.smithi038.stdout:3/840: chown d3/d9/d77 17513684 1 2024-03-24T00:49:08.872 INFO:tasks.workunit.client.0.smithi038.stdout:3/841: chown d3/d9/dc/da6/fc7 469656313 1 2024-03-24T00:49:08.872 INFO:tasks.workunit.client.0.smithi038.stdout:3/842: write d3/d30/f7e [603135,31975] 0 2024-03-24T00:49:08.873 INFO:tasks.workunit.client.0.smithi038.stdout:3/843: dread - d3/d9/dc/da6/dfc/f109 zero size 2024-03-24T00:49:08.874 INFO:tasks.workunit.client.0.smithi038.stdout:1/883: mknod da/dd/d7a/d82/d98/d14/c123 0 2024-03-24T00:49:08.878 INFO:tasks.workunit.client.0.smithi038.stdout:2/762: dwrite d4/d27/d54/d53/f6b [0,4194304] 0 2024-03-24T00:49:08.881 INFO:tasks.workunit.client.0.smithi038.stdout:5/814: dread d6/f9 [0,4194304] 0 2024-03-24T00:49:08.881 INFO:tasks.workunit.client.0.smithi038.stdout:5/815: stat d6/d19/d20/d42/d43/d68/dc8/fd4 0 2024-03-24T00:49:08.883 INFO:tasks.workunit.client.1.smithi088.stdout:9/876: dread d1/d10/d77/fb4 [0,4194304] 0 2024-03-24T00:49:08.886 INFO:tasks.workunit.client.1.smithi088.stdout:1/874: rename d6/f6d to d6/d6e/dfc/d108/d3f/f120 0 2024-03-24T00:49:08.886 INFO:tasks.workunit.client.1.smithi088.stdout:1/875: chown d6/l62 1040059349 1 2024-03-24T00:49:08.890 INFO:tasks.workunit.client.1.smithi088.stdout:7/801: symlink d3/d31/d56/de4/lfc 0 2024-03-24T00:49:08.898 INFO:tasks.workunit.client.0.smithi038.stdout:7/786: creat d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/f11f x:0 0 0 2024-03-24T00:49:08.908 INFO:tasks.workunit.client.0.smithi038.stdout:0/794: dwrite d0/d14/fb5 [0,4194304] 0 2024-03-24T00:49:08.911 INFO:tasks.workunit.client.1.smithi088.stdout:1/876: dread d6/d6e/dfc/d108/d5b/d5d/d61/d7d/dc8/fd1 [0,4194304] 0 2024-03-24T00:49:08.913 INFO:tasks.workunit.client.1.smithi088.stdout:4/874: creat d5/d13/d63/df4/f135 x:0 0 0 2024-03-24T00:49:08.914 INFO:tasks.workunit.client.1.smithi088.stdout:4/875: fdatasync d5/f46 0 2024-03-24T00:49:08.914 INFO:tasks.workunit.client.1.smithi088.stdout:4/876: creat d5/d8b/f136 x:0 0 0 2024-03-24T00:49:08.918 INFO:tasks.workunit.client.1.smithi088.stdout:5/930: dwrite d1/d43/d4e/fc7 [0,4194304] 0 2024-03-24T00:49:08.926 INFO:tasks.workunit.client.1.smithi088.stdout:3/839: rmdir d3/de/d6f/d85/d107 0 2024-03-24T00:49:08.927 INFO:tasks.workunit.client.1.smithi088.stdout:3/840: write d3/d6/d19/d2a/dc2/dd5/dd6/ff2 [562203,108691] 0 2024-03-24T00:49:08.927 INFO:tasks.workunit.client.1.smithi088.stdout:3/841: write d3/de/d6f/d85/d49/ffe [31569,110118] 0 2024-03-24T00:49:08.927 INFO:tasks.workunit.client.0.smithi038.stdout:8/857: dwrite d0/d45/d66/f97 [0,4194304] 0 2024-03-24T00:49:08.927 INFO:tasks.workunit.client.1.smithi088.stdout:2/829: write db/f18 [3526381,6903] 0 2024-03-24T00:49:08.929 INFO:tasks.workunit.client.1.smithi088.stdout:0/897: symlink d9/dc/de/d7e/l11d 0 2024-03-24T00:49:08.934 INFO:tasks.workunit.client.0.smithi038.stdout:4/791: unlink da/d27/fb5 0 2024-03-24T00:49:08.941 INFO:tasks.workunit.client.0.smithi038.stdout:9/922: creat d12/d28/d9c/ddd/d129/f12a x:0 0 0 2024-03-24T00:49:08.941 INFO:tasks.workunit.client.0.smithi038.stdout:9/923: chown d12/d28/d9c/ddd/d111 7955893 1 2024-03-24T00:49:08.946 INFO:tasks.workunit.client.0.smithi038.stdout:8/858: dread d0/d1d/d20/fdd [0,4194304] 0 2024-03-24T00:49:08.946 INFO:tasks.workunit.client.0.smithi038.stdout:1/884: creat da/d3c/d86/dd7/f124 x:0 0 0 2024-03-24T00:49:08.946 INFO:tasks.workunit.client.0.smithi038.stdout:1/885: truncate da/dd/d4a/fd3 498456 0 2024-03-24T00:49:08.947 INFO:tasks.workunit.client.1.smithi088.stdout:6/936: mknod db/dd/d61/d6d/d74/df0/dad/c128 0 2024-03-24T00:49:08.947 INFO:tasks.workunit.client.1.smithi088.stdout:6/937: dread - db/dd/d61/d6d/d74/df0/fcf zero size 2024-03-24T00:49:08.949 INFO:tasks.workunit.client.0.smithi038.stdout:8/859: write d0/d45/d37/d5f/db7/f9a [490797,26194] 0 2024-03-24T00:49:08.951 INFO:tasks.workunit.client.1.smithi088.stdout:8/871: getdents d9/d6b/df5 0 2024-03-24T00:49:08.962 INFO:tasks.workunit.client.0.smithi038.stdout:5/816: rename d6/d19/da1/fac to d6/d19/d20/d33/f107 0 2024-03-24T00:49:08.962 INFO:tasks.workunit.client.0.smithi038.stdout:5/817: chown d6/d19/da1 446 1 2024-03-24T00:49:08.962 INFO:tasks.workunit.client.0.smithi038.stdout:5/818: read d6/d23/f3b [55784,57293] 0 2024-03-24T00:49:08.965 INFO:tasks.workunit.client.1.smithi088.stdout:9/877: mkdir d1/d15/d21/d119 0 2024-03-24T00:49:08.968 INFO:tasks.workunit.client.1.smithi088.stdout:7/802: rmdir d3/dbf 39 2024-03-24T00:49:08.971 INFO:tasks.workunit.client.0.smithi038.stdout:0/795: unlink d0/d12/c57 0 2024-03-24T00:49:08.971 INFO:tasks.workunit.client.0.smithi038.stdout:0/796: truncate d0/d12/d42/fdd 320431 0 2024-03-24T00:49:08.973 INFO:tasks.workunit.client.1.smithi088.stdout:1/877: rename d6/dc2/l100 to d6/d6e/dfc/d108/l121 0 2024-03-24T00:49:08.974 INFO:tasks.workunit.client.1.smithi088.stdout:4/877: dwrite d5/d17/f1d [0,4194304] 0 2024-03-24T00:49:08.974 INFO:tasks.workunit.client.1.smithi088.stdout:4/878: dread - d5/d16/d37/de7/f109 zero size 2024-03-24T00:49:08.977 INFO:tasks.workunit.client.1.smithi088.stdout:5/931: rmdir d1/d19/d48/dc2 39 2024-03-24T00:49:08.978 INFO:tasks.workunit.client.1.smithi088.stdout:5/932: creat d1/dd/d6f/da1/d135/d137/f13d x:0 0 0 2024-03-24T00:49:08.982 INFO:tasks.workunit.client.1.smithi088.stdout:3/842: truncate d3/d6/dca/f84 2938309 0 2024-03-24T00:49:08.985 INFO:tasks.workunit.client.1.smithi088.stdout:0/898: unlink d9/d26/d8f/da1/d62/d9b/fbb 0 2024-03-24T00:49:08.985 INFO:tasks.workunit.client.1.smithi088.stdout:0/899: chown d9/dc/de/d19/cf4 20281 1 2024-03-24T00:49:08.985 INFO:tasks.workunit.client.1.smithi088.stdout:6/938: truncate db/d1d/d23/d5c/f78 3316866 0 2024-03-24T00:49:08.985 INFO:tasks.workunit.client.1.smithi088.stdout:0/900: truncate d9/dc/de/d7e/d91/fc1 674585 0 2024-03-24T00:49:08.985 INFO:tasks.workunit.client.1.smithi088.stdout:0/901: chown d9/f10d 92370262 1 2024-03-24T00:49:08.986 INFO:tasks.workunit.client.1.smithi088.stdout:8/872: mkdir d9/dc/d11/d24/d29/d5b/d12a 0 2024-03-24T00:49:08.986 INFO:tasks.workunit.client.1.smithi088.stdout:8/873: chown d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/ff9 228412753 1 2024-03-24T00:49:08.986 INFO:tasks.workunit.client.1.smithi088.stdout:8/874: readlink d9/dc/l107 0 2024-03-24T00:49:08.986 INFO:tasks.workunit.client.1.smithi088.stdout:8/875: chown d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/c64 1188976 1 2024-03-24T00:49:08.986 INFO:tasks.workunit.client.1.smithi088.stdout:8/876: write d9/dc/d11/d24/d29/d39/fc6 [6790750,69621] 0 2024-03-24T00:49:08.986 INFO:tasks.workunit.client.1.smithi088.stdout:8/877: dread - d9/dc/dce/ddc/f125 zero size 2024-03-24T00:49:08.986 INFO:tasks.workunit.client.1.smithi088.stdout:8/878: chown d9/dc/dce/ddc 3 1 2024-03-24T00:49:08.990 INFO:tasks.workunit.client.1.smithi088.stdout:1/878: rename d6/d23/daf/c97 to d6/d6e/dfc/d108/d118/c122 0 2024-03-24T00:49:08.990 INFO:tasks.workunit.client.1.smithi088.stdout:7/803: creat d3/d32/d34/d9c/ffd x:0 0 0 2024-03-24T00:49:08.990 INFO:tasks.workunit.client.1.smithi088.stdout:1/879: fdatasync d6/d6e/dfc/d108/d3f/f57 0 2024-03-24T00:49:08.995 INFO:tasks.workunit.client.1.smithi088.stdout:0/902: creat d9/d26/d8f/da1/d62/dc4/f11e x:0 0 0 2024-03-24T00:49:08.995 INFO:tasks.workunit.client.1.smithi088.stdout:0/903: fsync d9/d26/d8f/da1/d63/fd8 0 2024-03-24T00:49:08.997 INFO:tasks.workunit.client.0.smithi038.stdout:8/860: mkdir d0/d45/d37/d5f/db7/d62/dfa/dc8/d11c 0 2024-03-24T00:49:08.998 INFO:tasks.workunit.client.0.smithi038.stdout:8/861: truncate d0/d10/f92 5166134 0 2024-03-24T00:49:08.998 INFO:tasks.workunit.client.0.smithi038.stdout:3/844: dwrite d3/d9/dc/da6/db5/d19/d82/d89/fa2 [0,4194304] 0 2024-03-24T00:49:09.000 INFO:tasks.workunit.client.1.smithi088.stdout:6/939: dread db/d3d/dcb/f116 [0,4194304] 0 2024-03-24T00:49:09.003 INFO:tasks.workunit.client.0.smithi038.stdout:5/819: mknod d6/d19/d20/d64/d6b/c108 0 2024-03-24T00:49:09.004 INFO:tasks.workunit.client.1.smithi088.stdout:8/879: creat d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/f12b x:0 0 0 2024-03-24T00:49:09.004 INFO:tasks.workunit.client.1.smithi088.stdout:7/804: mknod d3/d32/d34/d9c/d4b/d73/dab/dc3/cfe 0 2024-03-24T00:49:09.004 INFO:tasks.workunit.client.1.smithi088.stdout:7/805: fdatasync d3/d32/d34/d9c/ffd 0 2024-03-24T00:49:09.004 INFO:tasks.workunit.client.1.smithi088.stdout:7/806: readlink d3/d32/l53 0 2024-03-24T00:49:09.004 INFO:tasks.workunit.client.1.smithi088.stdout:0/904: link d9/d15/l23 d9/d26/d8f/l11f 0 2024-03-24T00:49:09.004 INFO:tasks.workunit.client.1.smithi088.stdout:0/905: fdatasync d9/d26/d8f/dc7/fed 0 2024-03-24T00:49:09.004 INFO:tasks.workunit.client.1.smithi088.stdout:0/906: truncate d9/dc/de0/fec 691353 0 2024-03-24T00:49:09.006 INFO:tasks.workunit.client.0.smithi038.stdout:7/787: rename l1 to d2/d9/da/d23/d2b/d5b/d72/d92/d33/l120 0 2024-03-24T00:49:09.006 INFO:tasks.workunit.client.0.smithi038.stdout:7/788: dread - d2/d9/da/d23/d2b/d5b/d72/fbf zero size 2024-03-24T00:49:09.006 INFO:tasks.workunit.client.1.smithi088.stdout:6/940: mknod db/dd/d61/d6d/d74/d88/c129 0 2024-03-24T00:49:09.006 INFO:tasks.workunit.client.1.smithi088.stdout:6/941: fsync db/dd/d61/d6d/d74/f8b 0 2024-03-24T00:49:09.008 INFO:tasks.workunit.client.0.smithi038.stdout:0/797: mkdir d0/d1b/df1/d10d 0 2024-03-24T00:49:09.008 INFO:tasks.workunit.client.0.smithi038.stdout:0/798: truncate d0/d1b/f87 1522745 0 2024-03-24T00:49:09.008 INFO:tasks.workunit.client.0.smithi038.stdout:0/799: fsync d0/d1b/d2c/f3b 0 2024-03-24T00:49:09.011 INFO:tasks.workunit.client.0.smithi038.stdout:9/924: symlink d12/d28/l12b 0 2024-03-24T00:49:09.014 INFO:tasks.workunit.client.1.smithi088.stdout:0/907: creat d9/d26/d8f/da1/d62/d9b/db0/f120 x:0 0 0 2024-03-24T00:49:09.019 INFO:tasks.workunit.client.1.smithi088.stdout:6/942: rmdir db/dc5/dc8/d11d 0 2024-03-24T00:49:09.022 INFO:tasks.workunit.client.1.smithi088.stdout:0/908: creat d9/dc/de/d19/d44/d46/d59/f121 x:0 0 0 2024-03-24T00:49:09.022 INFO:tasks.workunit.client.1.smithi088.stdout:0/909: dread - d9/d26/d108/f10c zero size 2024-03-24T00:49:09.023 INFO:tasks.workunit.client.0.smithi038.stdout:3/845: write d3/f25 [1272175,121111] 0 2024-03-24T00:49:09.023 INFO:tasks.workunit.client.0.smithi038.stdout:3/846: chown d3/d9/dc/da6/db5/lfe 993772311 1 2024-03-24T00:49:09.023 INFO:tasks.workunit.client.0.smithi038.stdout:3/847: creat d3/d9/dc/da6/db5/d19/d82/d89/da8/f120 x:0 0 0 2024-03-24T00:49:09.023 INFO:tasks.workunit.client.0.smithi038.stdout:3/848: fsync d3/d30/d5d/fde 0 2024-03-24T00:49:09.024 INFO:tasks.workunit.client.1.smithi088.stdout:6/943: stat db/d1d/d50/d90/d56/l79 0 2024-03-24T00:49:09.024 INFO:tasks.workunit.client.1.smithi088.stdout:6/944: chown db/d1d/ld4 3285 1 2024-03-24T00:49:09.025 INFO:tasks.workunit.client.0.smithi038.stdout:5/820: mkdir d6/d1e/d109 0 2024-03-24T00:49:09.026 INFO:tasks.workunit.client.1.smithi088.stdout:0/910: mknod d9/dc/de/d7e/de6/c122 0 2024-03-24T00:49:09.026 INFO:tasks.workunit.client.0.smithi038.stdout:7/789: symlink d2/d9/da/dc/d8a/l121 0 2024-03-24T00:49:09.029 INFO:tasks.workunit.client.1.smithi088.stdout:5/933: dread d1/d10/db6/d10b/d2f/d57/d9e/fab [8388608,4194304] 0 2024-03-24T00:49:09.029 INFO:tasks.workunit.client.1.smithi088.stdout:5/934: chown d1/d19/d48/dc2/dcc/l136 480857694 1 2024-03-24T00:49:09.029 INFO:tasks.workunit.client.1.smithi088.stdout:5/935: dread - d1/d43/d4e/f122 zero size 2024-03-24T00:49:09.029 INFO:tasks.workunit.client.1.smithi088.stdout:1/880: dread d6/d6e/dfc/d108/d3f/f120 [0,4194304] 0 2024-03-24T00:49:09.030 INFO:tasks.workunit.client.0.smithi038.stdout:8/862: rename d0/d45/d37/d5f/db7/d62/dfa/cd6 to d0/d45/d37/d5f/d68/c11d 0 2024-03-24T00:49:09.030 INFO:tasks.workunit.client.0.smithi038.stdout:8/863: readlink d0/d45/d37/d5f/db7/d62/l7f 0 2024-03-24T00:49:09.031 INFO:tasks.workunit.client.0.smithi038.stdout:0/800: creat d0/d12/d42/d83/dfe/f10e x:0 0 0 2024-03-24T00:49:09.031 INFO:tasks.workunit.client.0.smithi038.stdout:0/801: write d0/d12/d42/d83/dfe/f10e [373177,66905] 0 2024-03-24T00:49:09.031 INFO:tasks.workunit.client.1.smithi088.stdout:0/911: truncate d9/f85 1257385 0 2024-03-24T00:49:09.034 INFO:tasks.workunit.client.1.smithi088.stdout:3/843: write d3/d6/dca/fbd [2025642,119889] 0 2024-03-24T00:49:09.034 INFO:tasks.workunit.client.1.smithi088.stdout:2/830: dwrite db/dd/d96/d99/db2/d88/fd6 [0,4194304] 0 2024-03-24T00:49:09.037 INFO:tasks.workunit.client.0.smithi038.stdout:4/792: dwrite da/d12/d24/d5f/d8a/fc4 [0,4194304] 0 2024-03-24T00:49:09.039 INFO:tasks.workunit.client.0.smithi038.stdout:1/886: dwrite f5 [0,4194304] 0 2024-03-24T00:49:09.040 INFO:tasks.workunit.client.0.smithi038.stdout:3/849: unlink d3/d9/dc/df/f3a 0 2024-03-24T00:49:09.041 INFO:tasks.workunit.client.1.smithi088.stdout:4/879: dwrite d5/d7/d72/f104 [0,4194304] 0 2024-03-24T00:49:09.042 INFO:tasks.workunit.client.1.smithi088.stdout:4/880: chown d5/d13/d18/f79 25647 1 2024-03-24T00:49:09.042 INFO:tasks.workunit.client.1.smithi088.stdout:4/881: chown d5/d8b/lf9 5934 1 2024-03-24T00:49:09.042 INFO:tasks.workunit.client.0.smithi038.stdout:3/850: chown d3/d2a/cd8 4089 1 2024-03-24T00:49:09.042 INFO:tasks.workunit.client.0.smithi038.stdout:5/821: mknod d6/d19/d20/d33/c10a 0 2024-03-24T00:49:09.042 INFO:tasks.workunit.client.0.smithi038.stdout:0/802: dread d0/d12/d42/d5e/ff0 [0,4194304] 0 2024-03-24T00:49:09.042 INFO:tasks.workunit.client.0.smithi038.stdout:0/803: fdatasync d0/d12/d42/d44/f93 0 2024-03-24T00:49:09.042 INFO:tasks.workunit.client.0.smithi038.stdout:0/804: chown d0/l10 22854 1 2024-03-24T00:49:09.043 INFO:tasks.workunit.client.0.smithi038.stdout:0/805: chown d0/d12/d42/c8e 16 1 2024-03-24T00:49:09.043 INFO:tasks.workunit.client.0.smithi038.stdout:7/790: creat d2/d9/da/d23/f122 x:0 0 0 2024-03-24T00:49:09.046 INFO:tasks.workunit.client.1.smithi088.stdout:8/880: dwrite d9/dc/d11/d24/d29/da9/de7/ff6 [0,4194304] 0 2024-03-24T00:49:09.054 INFO:tasks.workunit.client.1.smithi088.stdout:1/881: dread d6/d6e/dfc/d108/d58/f32 [0,4194304] 0 2024-03-24T00:49:09.055 INFO:tasks.workunit.client.0.smithi038.stdout:9/925: rename d12/d28/f36 to d12/d28/d6c/f12c 0 2024-03-24T00:49:09.056 INFO:tasks.workunit.client.0.smithi038.stdout:9/926: creat d12/d81/d84/dec/dfc/f12d x:0 0 0 2024-03-24T00:49:09.062 INFO:tasks.workunit.client.1.smithi088.stdout:0/912: dread d9/ffe [0,4194304] 0 2024-03-24T00:49:09.066 INFO:tasks.workunit.client.0.smithi038.stdout:8/864: link d0/d45/db8/lc5 d0/d1d/d20/d79/d102/l11e 0 2024-03-24T00:49:09.068 INFO:tasks.workunit.client.0.smithi038.stdout:4/793: read da/d12/d24/d5f/d8a/fc4 [2843522,90986] 0 2024-03-24T00:49:09.073 INFO:tasks.workunit.client.0.smithi038.stdout:0/806: dread d0/f78 [0,4194304] 0 2024-03-24T00:49:09.074 INFO:tasks.workunit.client.0.smithi038.stdout:7/791: dread d2/d9/da/dc/d4a/f6d [0,4194304] 0 2024-03-24T00:49:09.074 INFO:tasks.workunit.client.1.smithi088.stdout:3/844: dread d3/de/d6f/f75 [0,4194304] 0 2024-03-24T00:49:09.074 INFO:tasks.workunit.client.1.smithi088.stdout:3/845: fdatasync d3/de/d6f/d85/d90/ff7 0 2024-03-24T00:49:09.076 INFO:tasks.workunit.client.1.smithi088.stdout:6/945: dwrite db/dc5/dc8/fd1 [0,4194304] 0 2024-03-24T00:49:09.076 INFO:tasks.workunit.client.1.smithi088.stdout:6/946: dread - db/d1d/d50/d90/ffa zero size 2024-03-24T00:49:09.076 INFO:tasks.workunit.client.1.smithi088.stdout:6/947: chown db/d3d/ff1 6116451 1 2024-03-24T00:49:09.083 INFO:tasks.workunit.client.1.smithi088.stdout:2/831: mknod db/d21/d73/c108 0 2024-03-24T00:49:09.086 INFO:tasks.workunit.client.0.smithi038.stdout:1/887: symlink da/d3c/d114/l125 0 2024-03-24T00:49:09.086 INFO:tasks.workunit.client.0.smithi038.stdout:1/888: stat da/dd/d7a/d82/d98/d34 0 2024-03-24T00:49:09.090 INFO:tasks.workunit.client.1.smithi088.stdout:8/881: dwrite d9/dc/d11/d24/d43/f50 [0,4194304] 0 2024-03-24T00:49:09.092 INFO:tasks.workunit.client.1.smithi088.stdout:5/936: rename d1/d10/d81 to d1/dd/d6f/da1/d135/d137/d13e 0 2024-03-24T00:49:09.093 INFO:tasks.workunit.client.1.smithi088.stdout:5/937: chown d1/d10/db6/d10b/d2e/d9f 63077603 1 2024-03-24T00:49:09.108 INFO:tasks.workunit.client.1.smithi088.stdout:4/882: mkdir d5/d7/dd/d88/d137 0 2024-03-24T00:49:09.108 INFO:tasks.workunit.client.1.smithi088.stdout:4/883: read d5/d7/de4/f127 [4292366,43143] 0 2024-03-24T00:49:09.109 INFO:tasks.workunit.client.1.smithi088.stdout:1/882: dwrite d6/d27/d67/d88/f92 [4194304,4194304] 0 2024-03-24T00:49:09.109 INFO:tasks.workunit.client.0.smithi038.stdout:3/851: dwrite d3/d9/dc/da6/db5/d19/d82/fd3 [0,4194304] 0 2024-03-24T00:49:09.116 INFO:tasks.workunit.client.0.smithi038.stdout:0/807: dwrite d0/d14/de4/feb [0,4194304] 0 2024-03-24T00:49:09.116 INFO:tasks.workunit.client.0.smithi038.stdout:0/808: creat d0/d12/d42/d83/dfe/f10f x:0 0 0 2024-03-24T00:49:09.117 INFO:tasks.workunit.client.0.smithi038.stdout:5/822: truncate d6/d19/d20/fcf 4059785 0 2024-03-24T00:49:09.132 INFO:tasks.workunit.client.0.smithi038.stdout:8/865: getdents d0/d45/d37/d5f/db7 0 2024-03-24T00:49:09.142 INFO:tasks.workunit.client.0.smithi038.stdout:4/794: unlink da/d12/d37/dd/d15/f77 0 2024-03-24T00:49:09.144 INFO:tasks.workunit.client.1.smithi088.stdout:0/913: symlink d9/d26/d8f/da1/l123 0 2024-03-24T00:49:09.145 INFO:tasks.workunit.client.0.smithi038.stdout:7/792: creat d2/d9/da/d23/def/f123 x:0 0 0 2024-03-24T00:49:09.156 INFO:tasks.workunit.client.1.smithi088.stdout:2/832: stat db/dd/d96/l98 0 2024-03-24T00:49:09.156 INFO:tasks.workunit.client.1.smithi088.stdout:2/833: creat db/f109 x:0 0 0 2024-03-24T00:49:09.160 INFO:tasks.workunit.client.0.smithi038.stdout:3/852: creat d3/d9/f121 x:0 0 0 2024-03-24T00:49:09.161 INFO:tasks.workunit.client.0.smithi038.stdout:3/853: write d3/d30/d5d/d7c/db0/fe1 [824711,102238] 0 2024-03-24T00:49:09.161 INFO:tasks.workunit.client.1.smithi088.stdout:8/882: symlink d9/dc/d11/d24/d29/d5b/db9/l12c 0 2024-03-24T00:49:09.161 INFO:tasks.workunit.client.1.smithi088.stdout:8/883: chown d9/dc/d11/d24/d29/d36/f97 2 1 2024-03-24T00:49:09.165 INFO:tasks.workunit.client.1.smithi088.stdout:6/948: rename db/c29 to db/dd/d87/ddf/c12a 0 2024-03-24T00:49:09.165 INFO:tasks.workunit.client.1.smithi088.stdout:6/949: readlink db/d1d/l22 0 2024-03-24T00:49:09.168 INFO:tasks.workunit.client.0.smithi038.stdout:5/823: getdents d6/d19/d20/d42/d43/d68 0 2024-03-24T00:49:09.171 INFO:tasks.workunit.client.0.smithi038.stdout:8/866: mkdir d0/d1d/d27/d6b/dfc/d100/d11f 0 2024-03-24T00:49:09.172 INFO:tasks.workunit.client.0.smithi038.stdout:6/916: sync 2024-03-24T00:49:09.173 INFO:tasks.workunit.client.0.smithi038.stdout:1/889: dwrite da/dd/d2c/d41/d60/feb [0,4194304] 0 2024-03-24T00:49:09.173 INFO:tasks.workunit.client.0.smithi038.stdout:1/890: write da/dd/d2c/d5a/f7b [3693232,68077] 0 2024-03-24T00:49:09.174 INFO:tasks.workunit.client.0.smithi038.stdout:2/763: sync 2024-03-24T00:49:09.179 INFO:tasks.workunit.client.0.smithi038.stdout:4/795: mknod da/d12/d24/d68/d6c/d7c/c110 0 2024-03-24T00:49:09.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:08 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:49:09.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:08 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:09.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:08 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:09.184 INFO:tasks.workunit.client.1.smithi088.stdout:0/914: unlink d9/dc/de/l2d 0 2024-03-24T00:49:09.184 INFO:tasks.workunit.client.0.smithi038.stdout:7/793: symlink d2/d9/da/dc/d8a/d105/l124 0 2024-03-24T00:49:09.188 INFO:tasks.workunit.client.1.smithi088.stdout:3/846: symlink d3/d6/dca/l10c 0 2024-03-24T00:49:09.197 INFO:tasks.workunit.client.0.smithi038.stdout:8/867: dread d0/d45/d66/f40 [0,4194304] 0 2024-03-24T00:49:09.197 INFO:tasks.workunit.client.0.smithi038.stdout:3/854: rename d3/d9/dc/c13 to d3/d2a/d107/c122 0 2024-03-24T00:49:09.200 INFO:tasks.workunit.client.1.smithi088.stdout:2/834: mknod db/dd/d96/d99/dc3/c10a 0 2024-03-24T00:49:09.210 INFO:tasks.workunit.client.0.smithi038.stdout:5/824: dread d6/d19/da1/ffe [0,4194304] 0 2024-03-24T00:49:09.211 INFO:tasks.workunit.client.0.smithi038.stdout:5/825: chown d6/d16/d21/d2d/la2 85 1 2024-03-24T00:49:09.219 INFO:tasks.workunit.client.1.smithi088.stdout:6/950: dwrite db/dd/d61/d6d/d74/d88/f9b [0,4194304] 0 2024-03-24T00:49:09.221 INFO:tasks.workunit.client.0.smithi038.stdout:9/927: rmdir d12/d81/d122 0 2024-03-24T00:49:09.222 INFO:tasks.workunit.client.0.smithi038.stdout:9/928: readlink d12/d28/d9c/ddd/d111/d26/l2c 0 2024-03-24T00:49:09.222 INFO:tasks.workunit.client.0.smithi038.stdout:9/929: write d12/d81/fd8 [4646057,17285] 0 2024-03-24T00:49:09.222 INFO:tasks.workunit.client.0.smithi038.stdout:0/809: rmdir d0/d12/d42/d83/dfe 39 2024-03-24T00:49:09.222 INFO:tasks.workunit.client.1.smithi088.stdout:5/938: rename d1/d10/db6/d10b/d2f/d57/d9e/dd0/f10a to d1/d10/db6/f13f 0 2024-03-24T00:49:09.223 INFO:tasks.workunit.client.0.smithi038.stdout:6/917: symlink d1/da/d1b/dbb/l131 0 2024-03-24T00:49:09.223 INFO:tasks.workunit.client.0.smithi038.stdout:6/918: chown d1/da/d73/fab 3011472 1 2024-03-24T00:49:09.226 INFO:tasks.workunit.client.0.smithi038.stdout:2/764: unlink d4/d55/lc2 0 2024-03-24T00:49:09.228 INFO:tasks.workunit.client.1.smithi088.stdout:9/878: sync 2024-03-24T00:49:09.228 INFO:tasks.workunit.client.1.smithi088.stdout:4/884: rmdir d5/d7/dd 39 2024-03-24T00:49:09.228 INFO:tasks.workunit.client.1.smithi088.stdout:7/807: sync 2024-03-24T00:49:09.228 INFO:tasks.workunit.client.1.smithi088.stdout:4/885: truncate d5/d16/d1a/d22/f68 829559 0 2024-03-24T00:49:09.229 INFO:tasks.workunit.client.1.smithi088.stdout:7/808: write d3/d32/d34/d9c/d4b/d74/d79/f97 [1527415,89043] 0 2024-03-24T00:49:09.229 INFO:tasks.workunit.client.1.smithi088.stdout:7/809: stat d3/d32/d34/d9c/d4b/d74/f95 0 2024-03-24T00:49:09.232 INFO:tasks.workunit.client.0.smithi038.stdout:8/868: stat d0/d45/d37/d5f/db7/d62/dfa/cf9 0 2024-03-24T00:49:09.232 INFO:tasks.workunit.client.1.smithi088.stdout:1/883: write d6/d27/f1d [488348,46990] 0 2024-03-24T00:49:09.237 INFO:tasks.workunit.client.1.smithi088.stdout:3/847: creat d3/de/d6f/d85/d90/f10d x:0 0 0 2024-03-24T00:49:09.238 INFO:tasks.workunit.client.0.smithi038.stdout:5/826: mknod d6/d19/d20/d42/d43/d68/c10b 0 2024-03-24T00:49:09.238 INFO:tasks.workunit.client.0.smithi038.stdout:5/827: readlink d6/d1e/l6c 0 2024-03-24T00:49:09.238 INFO:tasks.workunit.client.0.smithi038.stdout:5/828: fsync d6/d19/d20/d33/d6f/fe4 0 2024-03-24T00:49:09.239 INFO:tasks.workunit.client.1.smithi088.stdout:2/835: stat db/dd/d96/d99/feb 0 2024-03-24T00:49:09.240 INFO:tasks.workunit.client.0.smithi038.stdout:6/919: rename d1/da/d61/d129 to d1/da/d1b/dfd/d132 0 2024-03-24T00:49:09.242 INFO:tasks.workunit.client.0.smithi038.stdout:1/891: dwrite da/dd/d2c/d40/de9/f113 [0,4194304] 0 2024-03-24T00:49:09.242 INFO:tasks.workunit.client.0.smithi038.stdout:1/892: fsync da/dd/d7a/d82/d98/f15 0 2024-03-24T00:49:09.243 INFO:tasks.workunit.client.0.smithi038.stdout:2/765: truncate d4/d27/d54/d53/de2/f9a 4186486 0 2024-03-24T00:49:09.247 INFO:tasks.workunit.client.0.smithi038.stdout:3/855: symlink d3/d9/dc/d51/l123 0 2024-03-24T00:49:09.254 INFO:tasks.workunit.client.1.smithi088.stdout:5/939: mknod d1/dd/d6f/da1/d135/d137/d13e/c140 0 2024-03-24T00:49:09.254 INFO:tasks.workunit.client.1.smithi088.stdout:5/940: readlink d1/d10/db6/d10b/d2e/l74 0 2024-03-24T00:49:09.254 INFO:tasks.workunit.client.0.smithi038.stdout:8/869: creat d0/d1d/d27/d4d/daf/dea/f120 x:0 0 0 2024-03-24T00:49:09.255 INFO:tasks.workunit.client.0.smithi038.stdout:8/870: creat d0/d45/d66/d6d/f121 x:0 0 0 2024-03-24T00:49:09.255 INFO:tasks.workunit.client.0.smithi038.stdout:8/871: fsync d0/d1d/d27/f8e 0 2024-03-24T00:49:09.256 INFO:tasks.workunit.client.0.smithi038.stdout:5/829: mkdir d6/d19/d20/d42/d43/d10c 0 2024-03-24T00:49:09.262 INFO:tasks.workunit.client.0.smithi038.stdout:4/796: dwrite da/d12/d24/d5f/d8a/db6/f109 [0,4194304] 0 2024-03-24T00:49:09.263 INFO:tasks.workunit.client.1.smithi088.stdout:9/879: mknod d1/d10/d77/d8a/de0/c11a 0 2024-03-24T00:49:09.263 INFO:tasks.workunit.client.1.smithi088.stdout:9/880: write d1/d10/d11/f17 [8749993,18572] 0 2024-03-24T00:49:09.263 INFO:tasks.workunit.client.1.smithi088.stdout:4/886: getdents d5/d16 0 2024-03-24T00:49:09.265 INFO:tasks.workunit.client.0.smithi038.stdout:6/920: rename d1/da/d1b/d25/d49/d4e/l107 to d1/da/d1b/d25/d49/d4e/ddf/l133 0 2024-03-24T00:49:09.266 INFO:tasks.workunit.client.0.smithi038.stdout:6/921: creat d1/da/d61/d79/f134 x:0 0 0 2024-03-24T00:49:09.268 INFO:tasks.workunit.client.1.smithi088.stdout:7/810: mknod d3/d32/d34/d9c/d4b/d73/dab/dd8/cff 0 2024-03-24T00:49:09.272 INFO:tasks.workunit.client.1.smithi088.stdout:1/884: mknod d6/d6e/dfc/d108/d118/c123 0 2024-03-24T00:49:09.278 INFO:tasks.workunit.client.0.smithi038.stdout:1/893: symlink da/dd/d2c/d40/de9/d104/l126 0 2024-03-24T00:49:09.278 INFO:tasks.workunit.client.1.smithi088.stdout:0/915: rmdir d9/dc/de/d7e/dce 39 2024-03-24T00:49:09.284 INFO:tasks.workunit.client.1.smithi088.stdout:3/848: symlink d3/d6/d19/ddb/l10e 0 2024-03-24T00:49:09.284 INFO:tasks.workunit.client.1.smithi088.stdout:5/941: stat d1/d10/db6/d10b/d2f/d57/d9e/db1/cc4 0 2024-03-24T00:49:09.284 INFO:tasks.workunit.client.1.smithi088.stdout:2/836: write db/d21/d35/d49/fd8 [1675203,100000] 0 2024-03-24T00:49:09.284 INFO:tasks.workunit.client.1.smithi088.stdout:5/942: write d1/dd/d6f/da1/d3f/d106/f10f [993749,78326] 0 2024-03-24T00:49:09.285 INFO:tasks.workunit.client.1.smithi088.stdout:2/837: creat db/dd/d96/d99/db2/f10b x:0 0 0 2024-03-24T00:49:09.285 INFO:tasks.workunit.client.1.smithi088.stdout:2/838: chown db/dd/d96/d4e/d8f/d9c/db0/fd9 59193291 1 2024-03-24T00:49:09.285 INFO:tasks.workunit.client.0.smithi038.stdout:8/872: creat d0/d1d/d27/d6b/dfc/d100/f122 x:0 0 0 2024-03-24T00:49:09.288 INFO:tasks.workunit.client.1.smithi088.stdout:3/849: write d3/d6/d27/d3c/fa8 [100960,2572] 0 2024-03-24T00:49:09.293 INFO:tasks.workunit.client.1.smithi088.stdout:9/881: mkdir d1/db5/d118/d11b 0 2024-03-24T00:49:09.293 INFO:tasks.workunit.client.0.smithi038.stdout:5/830: write d6/d19/d20/d42/fd5 [1240715,31784] 0 2024-03-24T00:49:09.293 INFO:tasks.workunit.client.0.smithi038.stdout:5/831: readlink d6/d19/l2f 0 2024-03-24T00:49:09.296 INFO:tasks.workunit.client.0.smithi038.stdout:3/856: dread d3/d9/dc/da6/db5/d19/da4/ff5 [0,4194304] 0 2024-03-24T00:49:09.296 INFO:tasks.workunit.client.0.smithi038.stdout:3/857: fsync d3/d9/dc/df/fb8 0 2024-03-24T00:49:09.296 INFO:tasks.workunit.client.0.smithi038.stdout:3/858: write d3/d9/f96 [326847,58958] 0 2024-03-24T00:49:09.296 INFO:tasks.workunit.client.0.smithi038.stdout:3/859: readlink d3/d9/dc/da6/db5/lfe 0 2024-03-24T00:49:09.299 INFO:tasks.workunit.client.1.smithi088.stdout:7/811: rename d3/d32/d34/d9c/d4b/d73/dab/dd8/cff to d3/d32/d34/d9c/d4b/d74/d79/c100 0 2024-03-24T00:49:09.325 INFO:tasks.workunit.client.1.smithi088.stdout:4/887: dwrite d5/d13/d18/d24/db3/f132 [0,4194304] 0 2024-03-24T00:49:09.328 INFO:tasks.workunit.client.1.smithi088.stdout:2/839: creat db/dd/d96/d99/f10c x:0 0 0 2024-03-24T00:49:09.330 INFO:tasks.workunit.client.1.smithi088.stdout:3/850: write d3/de/d6f/d85/d49/f54 [9097598,99671] 0 2024-03-24T00:49:09.331 INFO:tasks.workunit.client.1.smithi088.stdout:3/851: readlink d3/d1a/d1e/d2f/d45/db5/l104 0 2024-03-24T00:49:09.331 INFO:tasks.workunit.client.1.smithi088.stdout:9/882: unlink d1/d15/d84/fdf 0 2024-03-24T00:49:09.336 INFO:tasks.workunit.client.0.smithi038.stdout:7/794: dwrite d2/d2c/d90/f115 [0,4194304] 0 2024-03-24T00:49:09.338 INFO:tasks.workunit.client.1.smithi088.stdout:5/943: dwrite d1/d10/db6/d10b/d2f/d57/f71 [0,4194304] 0 2024-03-24T00:49:09.338 INFO:tasks.workunit.client.1.smithi088.stdout:2/840: creat db/d21/d35/f10d x:0 0 0 2024-03-24T00:49:09.343 INFO:tasks.workunit.client.1.smithi088.stdout:3/852: unlink d3/d6/f3b 0 2024-03-24T00:49:09.349 INFO:tasks.workunit.client.1.smithi088.stdout:9/883: mknod d1/d10/d77/d8a/de0/c11c 0 2024-03-24T00:49:09.349 INFO:tasks.workunit.client.1.smithi088.stdout:9/884: write d1/d10/d29/f10c [59935,16620] 0 2024-03-24T00:49:09.356 INFO:tasks.workunit.client.0.smithi038.stdout:7/795: getdents d2/d9/da/d23/d2b/d5b/d72/d92/d5d 0 2024-03-24T00:49:09.357 INFO:tasks.workunit.client.0.smithi038.stdout:7/796: creat d2/d9/da/dc/f125 x:0 0 0 2024-03-24T00:49:09.357 INFO:tasks.workunit.client.0.smithi038.stdout:7/797: rename d2/d9/da/d23/d2b/d5b/d72/d92 to d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/d10a/d126 22 2024-03-24T00:49:09.358 INFO:tasks.workunit.client.1.smithi088.stdout:0/916: dwrite d9/dc/de/d19/d44/f6e [0,4194304] 0 2024-03-24T00:49:09.358 INFO:tasks.workunit.client.1.smithi088.stdout:0/917: chown d9/dc/de/d7e/de6/df5/d115/lf8 92561 1 2024-03-24T00:49:09.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:08 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:49:09.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:08 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:09.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:08 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:09.365 INFO:tasks.workunit.client.1.smithi088.stdout:5/944: truncate d1/d19/d48/dc2/dcc/fff 258633 0 2024-03-24T00:49:09.366 INFO:tasks.workunit.client.1.smithi088.stdout:3/853: fdatasync d3/de/d6f/d85/d49/f54 0 2024-03-24T00:49:09.367 INFO:tasks.workunit.client.1.smithi088.stdout:2/841: creat db/dd/dc9/d100/f10e x:0 0 0 2024-03-24T00:49:09.367 INFO:tasks.workunit.client.1.smithi088.stdout:2/842: chown db/dc/l23 30 1 2024-03-24T00:49:09.376 INFO:tasks.workunit.client.0.smithi038.stdout:4/797: dread da/d12/d37/dd/d15/f64 [0,4194304] 0 2024-03-24T00:49:09.376 INFO:tasks.workunit.client.0.smithi038.stdout:0/810: dwrite d0/d12/d42/d9d/f9e [0,4194304] 0 2024-03-24T00:49:09.376 INFO:tasks.workunit.client.0.smithi038.stdout:4/798: read da/d12/f3f [2501688,55572] 0 2024-03-24T00:49:09.376 INFO:tasks.workunit.client.0.smithi038.stdout:7/798: unlink d2/d2c/c86 0 2024-03-24T00:49:09.378 INFO:tasks.workunit.client.1.smithi088.stdout:2/843: dread db/dd/d96/d4e/d8f/fa7 [0,4194304] 0 2024-03-24T00:49:09.382 INFO:tasks.workunit.client.0.smithi038.stdout:2/766: dwrite d4/d18/fe1 [0,4194304] 0 2024-03-24T00:49:09.387 INFO:tasks.workunit.client.0.smithi038.stdout:8/873: dwrite d0/d15/d2c/f88 [0,4194304] 0 2024-03-24T00:49:09.392 INFO:tasks.workunit.client.1.smithi088.stdout:0/918: rename d9/dc/de/d19/d44/d46/d65/c22 to d9/dc/de/d19/d44/d46/d65/d6f/c124 0 2024-03-24T00:49:09.392 INFO:tasks.workunit.client.1.smithi088.stdout:0/919: write d9/dc/de/d7e/d91/f9f [3920973,5736] 0 2024-03-24T00:49:09.396 INFO:tasks.workunit.client.0.smithi038.stdout:0/811: mknod d0/d12/d92/c110 0 2024-03-24T00:49:09.398 INFO:tasks.workunit.client.0.smithi038.stdout:7/799: dread d2/d9/d16/fc6 [0,4194304] 0 2024-03-24T00:49:09.398 INFO:tasks.workunit.client.0.smithi038.stdout:7/800: truncate d2/d9/da/dc/da8/dc5/fd4 415707 0 2024-03-24T00:49:09.400 INFO:tasks.workunit.client.0.smithi038.stdout:4/799: rename da/d12/d29/l67 to da/d12/d29/d5b/d73/db4/dd8/l111 0 2024-03-24T00:49:09.400 INFO:tasks.workunit.client.0.smithi038.stdout:4/800: dread - da/d12/d24/d5f/df8/ff9 zero size 2024-03-24T00:49:09.405 INFO:tasks.workunit.client.0.smithi038.stdout:3/860: dread d3/d9/dc/d51/f7d [0,4194304] 0 2024-03-24T00:49:09.406 INFO:tasks.workunit.client.1.smithi088.stdout:5/945: mknod d1/dd/d6f/da1/d8e/de0/c141 0 2024-03-24T00:49:09.406 INFO:tasks.workunit.client.1.smithi088.stdout:5/946: creat d1/d10/db6/d10b/d2e/d9f/ddf/f142 x:0 0 0 2024-03-24T00:49:09.410 INFO:tasks.workunit.client.1.smithi088.stdout:2/844: getdents db/dd/d96/d4e/d8f/dad 0 2024-03-24T00:49:09.413 INFO:tasks.workunit.client.0.smithi038.stdout:2/767: creat d4/d18/d26/d37/de0/f105 x:0 0 0 2024-03-24T00:49:09.414 INFO:tasks.workunit.client.0.smithi038.stdout:0/812: symlink d0/d12/d42/d83/dfe/l111 0 2024-03-24T00:49:09.414 INFO:tasks.workunit.client.1.smithi088.stdout:8/884: sync 2024-03-24T00:49:09.414 INFO:tasks.workunit.client.1.smithi088.stdout:9/885: getdents d1/d15 0 2024-03-24T00:49:09.415 INFO:tasks.workunit.client.1.smithi088.stdout:0/920: symlink d9/d26/l125 0 2024-03-24T00:49:09.418 INFO:tasks.workunit.client.0.smithi038.stdout:1/894: dwrite da/dd/d2c/d41/d60/feb [0,4194304] 0 2024-03-24T00:49:09.418 INFO:tasks.workunit.client.0.smithi038.stdout:4/801: fsync da/d38/f63 0 2024-03-24T00:49:09.418 INFO:tasks.workunit.client.0.smithi038.stdout:4/802: fsync da/d12/d29/d5b/d73/db4/ff3 0 2024-03-24T00:49:09.420 INFO:tasks.workunit.client.0.smithi038.stdout:3/861: truncate d3/d9/dc/df/f12 3661548 0 2024-03-24T00:49:09.420 INFO:tasks.workunit.client.0.smithi038.stdout:3/862: dread - d3/d9/dc/da6/db5/d19/d82/d89/da8/f120 zero size 2024-03-24T00:49:09.422 INFO:tasks.workunit.client.1.smithi088.stdout:4/888: dwrite d5/d13/d18/f19 [0,4194304] 0 2024-03-24T00:49:09.422 INFO:tasks.workunit.client.1.smithi088.stdout:4/889: write d5/d17/f11d [592593,53284] 0 2024-03-24T00:49:09.422 INFO:tasks.workunit.client.1.smithi088.stdout:3/854: dread d3/de/d6f/d85/f15 [0,4194304] 0 2024-03-24T00:49:09.423 INFO:tasks.workunit.client.1.smithi088.stdout:5/947: mknod d1/d19/d48/dc2/dcc/c143 0 2024-03-24T00:49:09.425 INFO:tasks.workunit.client.0.smithi038.stdout:0/813: mknod d0/d12/d15/c112 0 2024-03-24T00:49:09.425 INFO:tasks.workunit.client.0.smithi038.stdout:0/814: read - d0/d12/d15/d58/fe7 zero size 2024-03-24T00:49:09.426 INFO:tasks.workunit.client.0.smithi038.stdout:0/815: creat d0/d12/d9a/f113 x:0 0 0 2024-03-24T00:49:09.428 INFO:tasks.workunit.client.1.smithi088.stdout:8/885: creat d9/dc/d11/d24/d29/d5b/d8b/f12d x:0 0 0 2024-03-24T00:49:09.429 INFO:tasks.workunit.client.1.smithi088.stdout:0/921: creat d9/dc/de0/f126 x:0 0 0 2024-03-24T00:49:09.435 INFO:tasks.workunit.client.1.smithi088.stdout:4/890: write d5/d13/d18/f79 [503733,77604] 0 2024-03-24T00:49:09.435 INFO:tasks.workunit.client.0.smithi038.stdout:2/768: creat d4/d27/f106 x:0 0 0 2024-03-24T00:49:09.435 INFO:tasks.workunit.client.0.smithi038.stdout:2/769: fdatasync d4/d27/d54/d53/f7f 0 2024-03-24T00:49:09.441 INFO:tasks.workunit.client.1.smithi088.stdout:3/855: symlink d3/d1a/d1e/d2f/d45/db5/l10f 0 2024-03-24T00:49:09.448 INFO:tasks.workunit.client.0.smithi038.stdout:0/816: dread d0/d12/f3c [0,4194304] 0 2024-03-24T00:49:09.449 INFO:tasks.workunit.client.0.smithi038.stdout:0/817: stat d0/d12/d15/d58/lf8 0 2024-03-24T00:49:09.454 INFO:tasks.workunit.client.1.smithi088.stdout:6/951: sync 2024-03-24T00:49:09.454 INFO:tasks.workunit.client.1.smithi088.stdout:6/952: write db/dd/d87/ddf/f101 [557595,89228] 0 2024-03-24T00:49:09.454 INFO:tasks.workunit.client.1.smithi088.stdout:6/953: chown db/d1d/d50/fd0 951 1 2024-03-24T00:49:09.454 INFO:tasks.workunit.client.1.smithi088.stdout:7/812: sync 2024-03-24T00:49:09.454 INFO:tasks.workunit.client.1.smithi088.stdout:7/813: chown d3/f2d 41770007 1 2024-03-24T00:49:09.455 INFO:tasks.workunit.client.1.smithi088.stdout:5/948: dread d1/d19/f2c [0,4194304] 0 2024-03-24T00:49:09.455 INFO:tasks.workunit.client.1.smithi088.stdout:6/954: getdents db/d1d/d23/d5c/df2 0 2024-03-24T00:49:09.455 INFO:tasks.workunit.client.1.smithi088.stdout:6/955: stat db/dd/d61/d6d/d74/d88/da1/d106/d121 0 2024-03-24T00:49:09.458 INFO:tasks.workunit.client.1.smithi088.stdout:5/949: getdents d1/dd/d6f/da1/d8e 0 2024-03-24T00:49:09.458 INFO:tasks.workunit.client.1.smithi088.stdout:5/950: chown d1/d10/db6/d10b 0 1 2024-03-24T00:49:09.459 INFO:tasks.workunit.client.1.smithi088.stdout:7/814: rmdir d3/d32/d34/d9c/d4b/da7/dc5 39 2024-03-24T00:49:09.460 INFO:tasks.workunit.client.0.smithi038.stdout:4/803: dread da/d12/f87 [0,4194304] 0 2024-03-24T00:49:09.460 INFO:tasks.workunit.client.1.smithi088.stdout:0/922: dread d9/dc/de/d19/f93 [0,4194304] 0 2024-03-24T00:49:09.461 INFO:tasks.workunit.client.0.smithi038.stdout:3/863: dread d3/d30/d70/f7b [0,4194304] 0 2024-03-24T00:49:09.461 INFO:tasks.workunit.client.0.smithi038.stdout:3/864: dread - d3/d30/d5d/deb/f108 zero size 2024-03-24T00:49:09.463 INFO:tasks.workunit.client.1.smithi088.stdout:5/951: mknod d1/dd/d111/c144 0 2024-03-24T00:49:09.463 INFO:tasks.workunit.client.1.smithi088.stdout:5/952: write d1/dd/d6f/da1/d3f/f11c [1543581,50257] 0 2024-03-24T00:49:09.465 INFO:tasks.workunit.client.1.smithi088.stdout:1/885: sync 2024-03-24T00:49:09.466 INFO:tasks.workunit.client.0.smithi038.stdout:4/804: mknod da/d12/d29/d5b/d73/db4/de3/c112 0 2024-03-24T00:49:09.466 INFO:tasks.workunit.client.0.smithi038.stdout:3/865: creat d3/d30/d5d/d6a/f124 x:0 0 0 2024-03-24T00:49:09.466 INFO:tasks.workunit.client.0.smithi038.stdout:3/866: chown d3 4513268 1 2024-03-24T00:49:09.466 INFO:tasks.workunit.client.0.smithi038.stdout:3/867: stat d3/d9/dc/da6/db5/l8e 0 2024-03-24T00:49:09.466 INFO:tasks.workunit.client.0.smithi038.stdout:3/868: stat d3/d2a/cd8 0 2024-03-24T00:49:09.468 INFO:tasks.workunit.client.0.smithi038.stdout:3/869: link d3/d30/f7a d3/f125 0 2024-03-24T00:49:09.469 INFO:tasks.workunit.client.1.smithi088.stdout:0/923: dread d9/d26/d8f/fad [0,4194304] 0 2024-03-24T00:49:09.473 INFO:tasks.workunit.client.1.smithi088.stdout:4/891: dread d5/d13/f86 [0,4194304] 0 2024-03-24T00:49:09.473 INFO:tasks.workunit.client.1.smithi088.stdout:0/924: rename d9/cb to d9/d26/d8f/dc7/c127 0 2024-03-24T00:49:09.474 INFO:tasks.workunit.client.1.smithi088.stdout:4/892: write d5/d13/d18/d24/d3d/fb7 [690336,87897] 0 2024-03-24T00:49:09.475 INFO:tasks.workunit.client.1.smithi088.stdout:0/925: getdents d9/dc/de/d19 0 2024-03-24T00:49:09.476 INFO:tasks.workunit.client.1.smithi088.stdout:0/926: dread - d9/f119 zero size 2024-03-24T00:49:09.476 INFO:tasks.workunit.client.1.smithi088.stdout:0/927: chown d9/dc/de/d19/d44/d46/f57 2048 1 2024-03-24T00:49:09.476 INFO:tasks.workunit.client.1.smithi088.stdout:0/928: readlink d9/dc/de/d7e/lbd 0 2024-03-24T00:49:09.479 INFO:tasks.workunit.client.1.smithi088.stdout:3/856: read d3/d6/d27/de9/fd2 [3272393,74718] 0 2024-03-24T00:49:09.479 INFO:tasks.workunit.client.1.smithi088.stdout:3/857: readlink d3/d6/dca/dae/d4b/l6a 0 2024-03-24T00:49:09.481 INFO:tasks.workunit.client.0.smithi038.stdout:0/818: dread d0/d1b/d2c/fac [0,4194304] 0 2024-03-24T00:49:09.481 INFO:tasks.workunit.client.0.smithi038.stdout:8/874: dwrite d0/d45/f89 [0,4194304] 0 2024-03-24T00:49:09.483 INFO:tasks.workunit.client.1.smithi088.stdout:4/893: rename d5/d16/d94/da6/ffa to d5/d13/d18/d24/db3/f138 0 2024-03-24T00:49:09.483 INFO:tasks.workunit.client.1.smithi088.stdout:0/929: getdents d9/dc/de/d7e/de6/df5 0 2024-03-24T00:49:09.483 INFO:tasks.workunit.client.1.smithi088.stdout:0/930: readlink d9/d15/l77 0 2024-03-24T00:49:09.485 INFO:tasks.workunit.client.0.smithi038.stdout:7/801: dwrite d2/d9/da/dc/d93/ff1 [0,4194304] 0 2024-03-24T00:49:09.488 INFO:tasks.workunit.client.0.smithi038.stdout:8/875: truncate d0/d1d/d27/d4d/f75 102234 0 2024-03-24T00:49:09.488 INFO:tasks.workunit.client.0.smithi038.stdout:8/876: dread - d0/d1d/d27/d4d/fd7 zero size 2024-03-24T00:49:09.488 INFO:tasks.workunit.client.0.smithi038.stdout:8/877: chown d0/d45/d37/d5f/db7/d62/dfa/dc8/le9 2259688 1 2024-03-24T00:49:09.490 INFO:tasks.workunit.client.1.smithi088.stdout:6/956: dread db/dd/d87/ddf/f101 [0,4194304] 0 2024-03-24T00:49:09.490 INFO:tasks.workunit.client.1.smithi088.stdout:6/957: dread - db/d8e/d10d/f123 zero size 2024-03-24T00:49:09.491 INFO:tasks.workunit.client.1.smithi088.stdout:6/958: truncate db/dd/d61/d6d/f91 4899326 0 2024-03-24T00:49:09.491 INFO:tasks.workunit.client.1.smithi088.stdout:6/959: dread - db/d3d/d83/deb/ff6 zero size 2024-03-24T00:49:09.491 INFO:tasks.workunit.client.1.smithi088.stdout:6/960: fsync db/dc5/dc8/d4b/fd5 0 2024-03-24T00:49:09.491 INFO:tasks.workunit.client.0.smithi038.stdout:8/878: rmdir d0/d1d/d27/d4d/d81/d109 0 2024-03-24T00:49:09.492 INFO:tasks.workunit.client.1.smithi088.stdout:5/953: write d1/d19/f45 [1732867,114279] 0 2024-03-24T00:49:09.492 INFO:tasks.workunit.client.1.smithi088.stdout:5/954: stat d1/d19/d48/dc2/dcc/ce2 0 2024-03-24T00:49:09.492 INFO:tasks.workunit.client.1.smithi088.stdout:5/955: write d1/d19/fc0 [2482539,106040] 0 2024-03-24T00:49:09.492 INFO:tasks.workunit.client.1.smithi088.stdout:5/956: readlink d1/d10/db6/d10b/la8 0 2024-03-24T00:49:09.492 INFO:tasks.workunit.client.1.smithi088.stdout:5/957: truncate d1/dd/d6f/da1/ff4 586991 0 2024-03-24T00:49:09.493 INFO:tasks.workunit.client.1.smithi088.stdout:8/886: write d9/dc/f16 [46969,73878] 0 2024-03-24T00:49:09.493 INFO:tasks.workunit.client.1.smithi088.stdout:8/887: chown d9/dc/d11/d24/d29/f51 2430007 1 2024-03-24T00:49:09.493 INFO:tasks.workunit.client.1.smithi088.stdout:2/845: dwrite db/dd/fcf [0,4194304] 0 2024-03-24T00:49:09.494 INFO:tasks.workunit.client.0.smithi038.stdout:8/879: link d0/d45/d37/d5f/db7/dcc/fef d0/d45/d37/d5f/db7/d62/f123 0 2024-03-24T00:49:09.497 INFO:tasks.workunit.client.0.smithi038.stdout:8/880: rename d0/d1d/d27/l3f to d0/d45/d37/d5f/db7/dcc/l124 0 2024-03-24T00:49:09.497 INFO:tasks.workunit.client.1.smithi088.stdout:6/961: rename db/dd/d61/d6d/d74/df0/fcf to db/dc5/f12b 0 2024-03-24T00:49:09.501 INFO:tasks.workunit.client.1.smithi088.stdout:5/958: rename d1/d10/d4f/d10c to d1/dd/d6f/da1/d8e/d145 0 2024-03-24T00:49:09.504 INFO:tasks.workunit.client.1.smithi088.stdout:3/858: write d3/de/d6f/d85/d31/f94 [3259078,58751] 0 2024-03-24T00:49:09.508 INFO:tasks.workunit.client.0.smithi038.stdout:0/819: write d0/d14/d84/fd2 [1440914,76869] 0 2024-03-24T00:49:09.508 INFO:tasks.workunit.client.0.smithi038.stdout:0/820: dread - d0/d12/d42/d83/fe3 zero size 2024-03-24T00:49:09.510 INFO:tasks.workunit.client.0.smithi038.stdout:0/821: rename d0/d12/d15/d58/d64/da7/dbb/dd4/dea/c2e to d0/d14/c114 0 2024-03-24T00:49:09.510 INFO:tasks.workunit.client.0.smithi038.stdout:0/822: fsync d0/d12/d42/d83/fb1 0 2024-03-24T00:49:09.511 INFO:tasks.workunit.client.1.smithi088.stdout:4/894: write d5/d13/d18/d24/d3d/f55 [1831881,27264] 0 2024-03-24T00:49:09.515 INFO:tasks.workunit.client.1.smithi088.stdout:3/859: creat d3/d6/dca/dae/f110 x:0 0 0 2024-03-24T00:49:09.515 INFO:tasks.workunit.client.1.smithi088.stdout:4/895: creat d5/d13/d63/f139 x:0 0 0 2024-03-24T00:49:09.516 INFO:tasks.workunit.client.0.smithi038.stdout:8/881: write d0/d45/d37/f48 [392298,66188] 0 2024-03-24T00:49:09.517 INFO:tasks.workunit.client.1.smithi088.stdout:4/896: rename d5/d13/d63/fc8 to d5/d16/d37/f13a 0 2024-03-24T00:49:09.517 INFO:tasks.workunit.client.1.smithi088.stdout:4/897: fsync d5/d7/d15/d74/d114/f129 0 2024-03-24T00:49:09.517 INFO:tasks.workunit.client.1.smithi088.stdout:4/898: truncate d5/d13/d18/f79 1859732 0 2024-03-24T00:49:09.518 INFO:tasks.workunit.client.1.smithi088.stdout:4/899: truncate d5/d7/de4/d107/f134 623570 0 2024-03-24T00:49:09.518 INFO:tasks.workunit.client.1.smithi088.stdout:4/900: rmdir d5/d7/d15/d74 39 2024-03-24T00:49:09.518 INFO:tasks.workunit.client.1.smithi088.stdout:4/901: dread - d5/d16/d37/de7/f109 zero size 2024-03-24T00:49:09.531 INFO:tasks.workunit.client.0.smithi038.stdout:0/823: write d0/d1b/d2c/f3b [4019200,58919] 0 2024-03-24T00:49:09.534 INFO:tasks.workunit.client.0.smithi038.stdout:0/824: mkdir d0/d1b/df1/d115 0 2024-03-24T00:49:09.540 INFO:tasks.workunit.client.0.smithi038.stdout:8/882: read d0/d45/d37/d5f/d68/f85 [4085226,5375] 0 2024-03-24T00:49:09.540 INFO:tasks.workunit.client.0.smithi038.stdout:3/870: dwrite d3/d9/dc/da6/fab [0,4194304] 0 2024-03-24T00:49:09.540 INFO:tasks.workunit.client.0.smithi038.stdout:8/883: rmdir d0/d1d 39 2024-03-24T00:49:09.560 INFO:tasks.workunit.client.0.smithi038.stdout:0/825: dread d0/d12/d92/fd6 [0,4194304] 0 2024-03-24T00:49:09.562 INFO:tasks.workunit.client.0.smithi038.stdout:0/826: getdents d0/d12/d15/d58/d64/da7/dbb/dd4/dea 0 2024-03-24T00:49:09.563 INFO:tasks.workunit.client.0.smithi038.stdout:0/827: creat d0/d14/d88/db8/f116 x:0 0 0 2024-03-24T00:49:09.563 INFO:tasks.workunit.client.0.smithi038.stdout:0/828: fdatasync d0/d12/d15/d58/d64/da7/dbb/dcd/f10b 0 2024-03-24T00:49:09.564 INFO:tasks.workunit.client.1.smithi088.stdout:7/815: dread d3/d31/fcc [0,4194304] 0 2024-03-24T00:49:09.564 INFO:tasks.workunit.client.1.smithi088.stdout:7/816: stat d3/f51 0 2024-03-24T00:49:09.564 INFO:tasks.workunit.client.1.smithi088.stdout:7/817: write d3/d32/d34/d9c/d4b/f5e [1348729,79401] 0 2024-03-24T00:49:09.564 INFO:tasks.workunit.client.1.smithi088.stdout:7/818: fsync d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/fc4 0 2024-03-24T00:49:09.565 INFO:tasks.workunit.client.0.smithi038.stdout:8/884: dread d0/d15/f94 [0,4194304] 0 2024-03-24T00:49:09.566 INFO:tasks.workunit.client.1.smithi088.stdout:7/819: mkdir d3/d32/d34/d9c/d4b/da7/dc5/d101 0 2024-03-24T00:49:09.566 INFO:tasks.workunit.client.1.smithi088.stdout:7/820: write d3/d32/d76/d8f/f90 [784098,53780] 0 2024-03-24T00:49:09.570 INFO:tasks.workunit.client.0.smithi038.stdout:8/885: rmdir d0/d1d/d27/d6b 39 2024-03-24T00:49:09.574 INFO:tasks.workunit.client.1.smithi088.stdout:1/886: write d6/d6e/dfc/d108/d3f/f4b [3251636,18873] 0 2024-03-24T00:49:09.574 INFO:tasks.workunit.client.1.smithi088.stdout:7/821: mkdir d3/d32/d34/d9c/d4b/d73/d102 0 2024-03-24T00:49:09.575 INFO:tasks.workunit.client.1.smithi088.stdout:7/822: fdatasync d3/d32/d34/d9c/d4b/d73/dab/fec 0 2024-03-24T00:49:09.575 INFO:tasks.workunit.client.1.smithi088.stdout:7/823: dread - d3/d32/d34/d9c/d4b/d73/dab/fb3 zero size 2024-03-24T00:49:09.576 INFO:tasks.workunit.client.1.smithi088.stdout:2/846: dwrite db/dd/d96/d99/db2/f102 [0,4194304] 0 2024-03-24T00:49:09.577 INFO:tasks.workunit.client.0.smithi038.stdout:8/886: unlink d0/d45/ce2 0 2024-03-24T00:49:09.580 INFO:tasks.workunit.client.1.smithi088.stdout:6/962: dwrite db/d1d/d50/f8d [0,4194304] 0 2024-03-24T00:49:09.580 INFO:tasks.workunit.client.1.smithi088.stdout:1/887: rename d6/d23/lbf to d6/d27/d67/d9b/l124 0 2024-03-24T00:49:09.580 INFO:tasks.workunit.client.1.smithi088.stdout:1/888: truncate d6/d6e/dfc/d108/d5b/d5d/d61/d7d/fde 1292641 0 2024-03-24T00:49:09.582 INFO:tasks.workunit.client.0.smithi038.stdout:7/802: dread d2/f1f [0,4194304] 0 2024-03-24T00:49:09.583 INFO:tasks.workunit.client.1.smithi088.stdout:7/824: mkdir d3/d32/d76/df3/d103 0 2024-03-24T00:49:09.587 INFO:tasks.workunit.client.0.smithi038.stdout:7/803: rename d2/d9/da/c15 to d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/d10b/c127 0 2024-03-24T00:49:09.588 INFO:tasks.workunit.client.0.smithi038.stdout:3/871: dread d3/d9/dc/d51/f7d [0,4194304] 0 2024-03-24T00:49:09.591 INFO:tasks.workunit.client.1.smithi088.stdout:4/902: dwrite d5/d16/fb1 [0,4194304] 0 2024-03-24T00:49:09.591 INFO:tasks.workunit.client.0.smithi038.stdout:6/922: sync 2024-03-24T00:49:09.592 INFO:tasks.workunit.client.0.smithi038.stdout:5/832: sync 2024-03-24T00:49:09.592 INFO:tasks.workunit.client.0.smithi038.stdout:7/804: creat d2/d9/d16/f128 x:0 0 0 2024-03-24T00:49:09.593 INFO:tasks.workunit.client.0.smithi038.stdout:3/872: mknod d3/d9/def/c126 0 2024-03-24T00:49:09.595 INFO:tasks.workunit.client.1.smithi088.stdout:1/889: symlink d6/l125 0 2024-03-24T00:49:09.595 INFO:tasks.workunit.client.1.smithi088.stdout:1/890: dread - d6/d27/d103/f11e zero size 2024-03-24T00:49:09.597 INFO:tasks.workunit.client.0.smithi038.stdout:6/923: symlink d1/da/d1b/l135 0 2024-03-24T00:49:09.597 INFO:tasks.workunit.client.0.smithi038.stdout:6/924: fsync d1/f5 0 2024-03-24T00:49:09.600 INFO:tasks.workunit.client.0.smithi038.stdout:9/930: sync 2024-03-24T00:49:09.600 INFO:tasks.workunit.client.1.smithi088.stdout:4/903: mknod d5/d13/c13b 0 2024-03-24T00:49:09.601 INFO:tasks.workunit.client.1.smithi088.stdout:9/886: dwrite d1/d22/d27/fbf [4194304,4194304] 0 2024-03-24T00:49:09.601 INFO:tasks.workunit.client.1.smithi088.stdout:9/887: creat d1/d10/d29/def/f11d x:0 0 0 2024-03-24T00:49:09.601 INFO:tasks.workunit.client.0.smithi038.stdout:5/833: creat d6/d1e/d97/db0/f10d x:0 0 0 2024-03-24T00:49:09.603 INFO:tasks.workunit.client.1.smithi088.stdout:2/847: dread db/f87 [0,4194304] 0 2024-03-24T00:49:09.610 INFO:tasks.workunit.client.0.smithi038.stdout:7/805: mknod d2/d2c/d9a/d109/c129 0 2024-03-24T00:49:09.610 INFO:tasks.workunit.client.0.smithi038.stdout:0/829: write d0/d12/d42/d9d/d28/f48 [604408,114316] 0 2024-03-24T00:49:09.612 INFO:tasks.workunit.client.0.smithi038.stdout:3/873: symlink d3/d9/dc/da6/db5/d19/d82/d89/da8/d105/l127 0 2024-03-24T00:49:09.614 INFO:tasks.workunit.client.0.smithi038.stdout:5/834: link d6/d16/d21/d2d/da6/lb3 d6/d19/d20/d70/da3/l10e 0 2024-03-24T00:49:09.617 INFO:tasks.workunit.client.1.smithi088.stdout:4/904: write d5/d16/d1a/d22/dc6/fdc [278232,104591] 0 2024-03-24T00:49:09.617 INFO:tasks.workunit.client.0.smithi038.stdout:6/925: write d1/da/d1b/d25/d49/d4e/ddf/fe9 [3070740,64608] 0 2024-03-24T00:49:09.618 INFO:tasks.workunit.client.0.smithi038.stdout:7/806: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d75/d12a 0 2024-03-24T00:49:09.619 INFO:tasks.workunit.client.0.smithi038.stdout:0/830: mknod d0/d1b/c117 0 2024-03-24T00:49:09.619 INFO:tasks.workunit.client.0.smithi038.stdout:0/831: fdatasync d0/d12/d15/d58/d64/da7/dbb/dd4/dea/fb3 0 2024-03-24T00:49:09.620 INFO:tasks.workunit.client.1.smithi088.stdout:5/959: dwrite d1/dd/f112 [0,4194304] 0 2024-03-24T00:49:09.620 INFO:tasks.workunit.client.1.smithi088.stdout:3/860: dwrite d3/de/d6f/f75 [0,4194304] 0 2024-03-24T00:49:09.620 INFO:tasks.workunit.client.1.smithi088.stdout:3/861: dread - d3/d1a/d1e/fd4 zero size 2024-03-24T00:49:09.620 INFO:tasks.workunit.client.1.smithi088.stdout:5/960: creat d1/d10/db6/d10b/d2f/d57/d9e/f146 x:0 0 0 2024-03-24T00:49:09.620 INFO:tasks.workunit.client.1.smithi088.stdout:5/961: write d1/d10/db6/d10b/d2f/d93/fa9 [791357,74977] 0 2024-03-24T00:49:09.620 INFO:tasks.workunit.client.1.smithi088.stdout:6/963: dwrite db/d1d/d23/faa [0,4194304] 0 2024-03-24T00:49:09.620 INFO:tasks.workunit.client.1.smithi088.stdout:6/964: chown db/f47 2898 1 2024-03-24T00:49:09.626 INFO:tasks.workunit.client.0.smithi038.stdout:3/874: mknod d3/d9/dc/da6/db5/d19/d82/d89/da8/d11c/c128 0 2024-03-24T00:49:09.626 INFO:tasks.workunit.client.0.smithi038.stdout:8/887: dread d0/d1d/d20/fdd [0,4194304] 0 2024-03-24T00:49:09.626 INFO:tasks.workunit.client.0.smithi038.stdout:3/875: readlink d3/d2a/la7 0 2024-03-24T00:49:09.629 INFO:tasks.workunit.client.1.smithi088.stdout:9/888: dread d1/d10/d29/db0/fc9 [0,4194304] 0 2024-03-24T00:49:09.629 INFO:tasks.workunit.client.1.smithi088.stdout:9/889: dread - d1/d10/d29/db0/f109 zero size 2024-03-24T00:49:09.629 INFO:tasks.workunit.client.1.smithi088.stdout:9/890: truncate d1/d10/d9b/dab/d7a/f116 679356 0 2024-03-24T00:49:09.631 INFO:tasks.workunit.client.1.smithi088.stdout:7/825: dwrite d3/d32/d34/d9c/fae [0,4194304] 0 2024-03-24T00:49:09.631 INFO:tasks.workunit.client.1.smithi088.stdout:7/826: write d3/d32/d34/f3b [2107833,94040] 0 2024-03-24T00:49:09.636 INFO:tasks.workunit.client.0.smithi038.stdout:5/835: rename d6/d19/d20/d42/d43/d50 to d6/d16/d10f 0 2024-03-24T00:49:09.637 INFO:tasks.workunit.client.1.smithi088.stdout:1/891: rename d6/d23/dcb to d6/d6e/dfc/d126 0 2024-03-24T00:49:09.639 INFO:tasks.workunit.client.1.smithi088.stdout:4/905: unlink d5/d13/d18/d24/f90 0 2024-03-24T00:49:09.651 INFO:tasks.workunit.client.1.smithi088.stdout:5/962: creat d1/d19/f147 x:0 0 0 2024-03-24T00:49:09.652 INFO:tasks.workunit.client.1.smithi088.stdout:5/963: write d1/d19/fa6 [4882060,13642] 0 2024-03-24T00:49:09.655 INFO:tasks.workunit.client.1.smithi088.stdout:6/965: symlink db/dc5/dc8/d43/daf/l12c 0 2024-03-24T00:49:09.655 INFO:tasks.workunit.client.1.smithi088.stdout:9/891: symlink d1/db5/l11e 0 2024-03-24T00:49:09.658 INFO:tasks.workunit.client.1.smithi088.stdout:7/827: creat d3/d32/d76/f104 x:0 0 0 2024-03-24T00:49:09.672 INFO:tasks.workunit.client.1.smithi088.stdout:3/862: rename d3/de/d6f/d85/d31/f72 to d3/d1a/d1e/d2f/f111 0 2024-03-24T00:49:09.672 INFO:tasks.workunit.client.1.smithi088.stdout:1/892: mknod d6/d27/c127 0 2024-03-24T00:49:09.672 INFO:tasks.workunit.client.1.smithi088.stdout:4/906: symlink d5/d16/d37/l13c 0 2024-03-24T00:49:09.672 INFO:tasks.workunit.client.1.smithi088.stdout:5/964: creat d1/d43/d4e/d5b/f148 x:0 0 0 2024-03-24T00:49:09.672 INFO:tasks.workunit.client.1.smithi088.stdout:5/965: creat d1/d10/d79/dbc/f149 x:0 0 0 2024-03-24T00:49:09.673 INFO:tasks.workunit.client.0.smithi038.stdout:5/836: dread d6/d1e/d97/fbe [0,4194304] 0 2024-03-24T00:49:09.674 INFO:tasks.workunit.client.1.smithi088.stdout:9/892: creat d1/d15/d80/f11f x:0 0 0 2024-03-24T00:49:09.674 INFO:tasks.workunit.client.1.smithi088.stdout:9/893: dread - d1/d10/d29/def/f11d zero size 2024-03-24T00:49:09.674 INFO:tasks.workunit.client.1.smithi088.stdout:9/894: stat d1/d10/d9b 0 2024-03-24T00:49:09.674 INFO:tasks.workunit.client.1.smithi088.stdout:9/895: chown d1/cd 150 1 2024-03-24T00:49:09.675 INFO:tasks.workunit.client.0.smithi038.stdout:6/926: dread d1/da/d61/d69/fdb [0,4194304] 0 2024-03-24T00:49:09.677 INFO:tasks.workunit.client.0.smithi038.stdout:3/876: write d3/f5 [225502,96099] 0 2024-03-24T00:49:09.679 INFO:tasks.workunit.client.1.smithi088.stdout:7/828: unlink c2 0 2024-03-24T00:49:09.680 INFO:tasks.workunit.client.1.smithi088.stdout:1/893: symlink d6/d27/d103/l128 0 2024-03-24T00:49:09.680 INFO:tasks.workunit.client.1.smithi088.stdout:1/894: readlink d6/lb6 0 2024-03-24T00:49:09.682 INFO:tasks.workunit.client.0.smithi038.stdout:7/807: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d33/f31 [0,4194304] 0 2024-03-24T00:49:09.682 INFO:tasks.workunit.client.0.smithi038.stdout:5/837: rename d6/d19/d20/d42/d43/d68/cc4 to d6/d16/d14/c110 0 2024-03-24T00:49:09.684 INFO:tasks.workunit.client.1.smithi088.stdout:5/966: mkdir d1/d19/d48/d84/d131/d14a 0 2024-03-24T00:49:09.684 INFO:tasks.workunit.client.1.smithi088.stdout:5/967: chown d1/dd/d6f/da1/d3f/d88/d8f/d69/f82 30 1 2024-03-24T00:49:09.686 INFO:tasks.workunit.client.0.smithi038.stdout:7/808: symlink d2/d9/da/dc/da8/dc5/l12b 0 2024-03-24T00:49:09.689 INFO:tasks.workunit.client.1.smithi088.stdout:3/863: dread d3/d1a/fe4 [0,4194304] 0 2024-03-24T00:49:09.691 INFO:tasks.workunit.client.1.smithi088.stdout:0/931: sync 2024-03-24T00:49:09.692 INFO:tasks.workunit.client.1.smithi088.stdout:8/888: sync 2024-03-24T00:49:09.692 INFO:tasks.workunit.client.1.smithi088.stdout:0/932: chown d9/dc/de/d19/d8d/l112 36361 1 2024-03-24T00:49:09.692 INFO:tasks.workunit.client.1.smithi088.stdout:0/933: fsync d9/d26/d8f/da1/d62/d9b/db0/db4/fd9 0 2024-03-24T00:49:09.695 INFO:tasks.workunit.client.0.smithi038.stdout:3/877: rename d3/d30/d5d/d7c/le5 to d3/d9/dc/da6/db5/d19/da4/l129 0 2024-03-24T00:49:09.696 INFO:tasks.workunit.client.0.smithi038.stdout:3/878: chown d3/d9/dc/da6/db5/l8e 19262 1 2024-03-24T00:49:09.696 INFO:tasks.workunit.client.1.smithi088.stdout:7/829: dread d3/d32/d34/d9c/d4b/f4c [0,4194304] 0 2024-03-24T00:49:09.696 INFO:tasks.workunit.client.1.smithi088.stdout:7/830: stat d3/d32/d34/d9c/d4b/d73/dab/dc3/feb 0 2024-03-24T00:49:09.696 INFO:tasks.workunit.client.1.smithi088.stdout:7/831: read d3/f27 [738488,14423] 0 2024-03-24T00:49:09.698 INFO:tasks.workunit.client.1.smithi088.stdout:3/864: link d3/de/d6f/d85/d49/d70/lab d3/d6/d19/d2a/dc2/dd5/l112 0 2024-03-24T00:49:09.699 INFO:tasks.workunit.client.1.smithi088.stdout:4/907: dread d5/d7/f66 [0,4194304] 0 2024-03-24T00:49:09.706 INFO:tasks.workunit.client.0.smithi038.stdout:3/879: unlink d3/d30/f33 0 2024-03-24T00:49:09.706 INFO:tasks.workunit.client.0.smithi038.stdout:3/880: fdatasync d3/d9/dc/da6/db5/f54 0 2024-03-24T00:49:09.706 INFO:tasks.workunit.client.0.smithi038.stdout:3/881: chown d3/d9/dc/d51/f75 87963382 1 2024-03-24T00:49:09.707 INFO:tasks.workunit.client.1.smithi088.stdout:0/934: creat d9/dc/de/d19/d44/d46/d65/f128 x:0 0 0 2024-03-24T00:49:09.708 INFO:tasks.workunit.client.1.smithi088.stdout:0/935: fsync d9/d26/d8f/da1/d62/d9b/db0/f120 0 2024-03-24T00:49:09.708 INFO:tasks.workunit.client.1.smithi088.stdout:8/889: dread d9/dc/d11/d80/db0/f90 [0,4194304] 0 2024-03-24T00:49:09.711 INFO:tasks.workunit.client.0.smithi038.stdout:3/882: unlink d3/d30/d70/d90/lc6 0 2024-03-24T00:49:09.714 INFO:tasks.workunit.client.1.smithi088.stdout:8/890: mkdir d9/dc/d11/d24/d29/d5b/d8b/d101/d12e 0 2024-03-24T00:49:09.714 INFO:tasks.workunit.client.1.smithi088.stdout:8/891: write d9/dc/d11/d24/d29/d36/dc1/dc4/fe2 [551479,30418] 0 2024-03-24T00:49:09.714 INFO:tasks.workunit.client.1.smithi088.stdout:8/892: stat d9/dc/d11/d24/d29/da9 0 2024-03-24T00:49:09.714 INFO:tasks.workunit.client.1.smithi088.stdout:8/893: write d9/dc/d11/d24/d29/d5b/db9/da5/fe0 [256738,58728] 0 2024-03-24T00:49:09.720 INFO:tasks.workunit.client.1.smithi088.stdout:8/894: mkdir d9/dc/d11/d24/d29/d5b/d8b/d12f 0 2024-03-24T00:49:09.721 INFO:tasks.workunit.client.1.smithi088.stdout:8/895: write d9/dc/d11/d24/fff [1149529,106963] 0 2024-03-24T00:49:09.721 INFO:tasks.workunit.client.1.smithi088.stdout:8/896: fsync d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/f12b 0 2024-03-24T00:49:09.724 INFO:tasks.workunit.client.1.smithi088.stdout:9/896: write d1/d22/d72/faf [801151,63129] 0 2024-03-24T00:49:09.730 INFO:tasks.workunit.client.1.smithi088.stdout:9/897: link d1/d10/d2c/cc6 d1/d22/d27/d39/d98/de6/c120 0 2024-03-24T00:49:09.741 INFO:tasks.workunit.client.1.smithi088.stdout:5/968: write d1/d19/f6c [888060,90941] 0 2024-03-24T00:49:09.741 INFO:tasks.workunit.client.1.smithi088.stdout:5/969: chown d1/d10/db6/d10b/d2f/d57/d9e/fb5 16 1 2024-03-24T00:49:09.742 INFO:tasks.workunit.client.1.smithi088.stdout:5/970: fdatasync d1/d10/db6/d10b/d2f/d57/d9e/daa/f12c 0 2024-03-24T00:49:09.744 INFO:tasks.workunit.client.1.smithi088.stdout:5/971: creat d1/d19/d48/d84/f14b x:0 0 0 2024-03-24T00:49:09.751 INFO:tasks.workunit.client.1.smithi088.stdout:5/972: creat d1/d10/f14c x:0 0 0 2024-03-24T00:49:09.751 INFO:tasks.workunit.client.1.smithi088.stdout:5/973: creat d1/d10/db6/d10b/d2e/d9f/ddf/f14d x:0 0 0 2024-03-24T00:49:09.754 INFO:tasks.workunit.client.1.smithi088.stdout:9/898: write d1/d10/d11/f17 [4909428,38586] 0 2024-03-24T00:49:09.757 INFO:tasks.workunit.client.1.smithi088.stdout:5/974: unlink d1/d10/db6/d10b/d2e/d9f/ddf/f142 0 2024-03-24T00:49:09.757 INFO:tasks.workunit.client.1.smithi088.stdout:5/975: readlink d1/dd/d6f/da1/l34 0 2024-03-24T00:49:09.765 INFO:tasks.workunit.client.1.smithi088.stdout:2/848: dwrite db/f3c [0,4194304] 0 2024-03-24T00:49:09.781 INFO:tasks.workunit.client.0.smithi038.stdout:3/883: write d3/d9/f2b [1873982,4106] 0 2024-03-24T00:49:09.781 INFO:tasks.workunit.client.0.smithi038.stdout:3/884: truncate d3/d9/dc/da6/db5/d19/dd2/f112 518401 0 2024-03-24T00:49:09.781 INFO:tasks.workunit.client.0.smithi038.stdout:3/885: read d3/d9/dc/da6/db5/d19/d82/d89/fa3 [5227,90229] 0 2024-03-24T00:49:09.781 INFO:tasks.workunit.client.1.smithi088.stdout:6/966: dwrite db/d3d/ff1 [4194304,4194304] 0 2024-03-24T00:49:09.783 INFO:tasks.workunit.client.1.smithi088.stdout:6/967: symlink db/d8e/l12d 0 2024-03-24T00:49:09.785 INFO:tasks.workunit.client.0.smithi038.stdout:3/886: symlink d3/d30/d5d/d7c/l12a 0 2024-03-24T00:49:09.787 INFO:tasks.workunit.client.0.smithi038.stdout:6/927: dwrite d1/da/d1b/d25/d49/d4e/d51/fb2 [0,4194304] 0 2024-03-24T00:49:09.787 INFO:tasks.workunit.client.0.smithi038.stdout:6/928: fdatasync d1/d37/d84/ffa 0 2024-03-24T00:49:09.788 INFO:tasks.workunit.client.0.smithi038.stdout:3/887: truncate d3/f25 3043682 0 2024-03-24T00:49:09.788 INFO:tasks.workunit.client.0.smithi038.stdout:3/888: creat d3/d9/f12b x:0 0 0 2024-03-24T00:49:09.788 INFO:tasks.workunit.client.0.smithi038.stdout:3/889: readlink d3/d9/dc/da6/l9f 0 2024-03-24T00:49:09.796 INFO:tasks.workunit.client.1.smithi088.stdout:2/849: dread db/d21/d35/f60 [0,4194304] 0 2024-03-24T00:49:09.796 INFO:tasks.workunit.client.1.smithi088.stdout:2/850: chown db/f4f 3 1 2024-03-24T00:49:09.796 INFO:tasks.workunit.client.1.smithi088.stdout:2/851: chown db/dd/dc9 1 1 2024-03-24T00:49:09.796 INFO:tasks.workunit.client.1.smithi088.stdout:2/852: chown db/dd/d96/d99/db2/fb4 50 1 2024-03-24T00:49:09.796 INFO:tasks.workunit.client.1.smithi088.stdout:2/853: write db/dd/dc9/d100/f10e [387901,92610] 0 2024-03-24T00:49:09.796 INFO:tasks.workunit.client.1.smithi088.stdout:6/968: dread db/d1d/d23/fa5 [0,4194304] 0 2024-03-24T00:49:09.797 INFO:tasks.workunit.client.1.smithi088.stdout:2/854: creat db/d21/d35/f10f x:0 0 0 2024-03-24T00:49:09.797 INFO:tasks.workunit.client.0.smithi038.stdout:3/890: dread d3/d30/d5d/fde [0,4194304] 0 2024-03-24T00:49:09.798 INFO:tasks.workunit.client.1.smithi088.stdout:6/969: symlink db/dd/d61/d6d/d74/df0/l12e 0 2024-03-24T00:49:09.798 INFO:tasks.workunit.client.1.smithi088.stdout:6/970: chown db/dc5/dc8/d43/fa2 34 1 2024-03-24T00:49:09.798 INFO:tasks.workunit.client.1.smithi088.stdout:6/971: fsync db/dd/d87/ddf/f111 0 2024-03-24T00:49:09.799 INFO:tasks.workunit.client.0.smithi038.stdout:3/891: creat d3/d9/dc/da6/db5/d19/dd2/df0/f12c x:0 0 0 2024-03-24T00:49:09.799 INFO:tasks.workunit.client.1.smithi088.stdout:2/855: rmdir db/dd/d96/d4e/d8f 39 2024-03-24T00:49:09.799 INFO:tasks.workunit.client.1.smithi088.stdout:2/856: readlink db/dd/d96/d99/db2/la6 0 2024-03-24T00:49:09.801 INFO:tasks.workunit.client.1.smithi088.stdout:7/832: dwrite d3/d32/d76/df3/ff6 [0,4194304] 0 2024-03-24T00:49:09.803 INFO:tasks.workunit.client.1.smithi088.stdout:6/972: unlink db/dd/lb8 0 2024-03-24T00:49:09.803 INFO:tasks.workunit.client.1.smithi088.stdout:6/973: fdatasync db/dd/d61/d6d/f91 0 2024-03-24T00:49:09.804 INFO:tasks.workunit.client.1.smithi088.stdout:0/936: dwrite d9/d26/d8f/da1/fc2 [0,4194304] 0 2024-03-24T00:49:09.804 INFO:tasks.workunit.client.0.smithi038.stdout:3/892: creat d3/dc3/f12d x:0 0 0 2024-03-24T00:49:09.805 INFO:tasks.workunit.client.1.smithi088.stdout:2/857: symlink db/dd/de7/l110 0 2024-03-24T00:49:09.805 INFO:tasks.workunit.client.1.smithi088.stdout:2/858: chown db/d21/l5c 37 1 2024-03-24T00:49:09.805 INFO:tasks.workunit.client.1.smithi088.stdout:2/859: read db/d21/d73/f83 [4172816,130208] 0 2024-03-24T00:49:09.806 INFO:tasks.workunit.client.1.smithi088.stdout:1/895: dwrite d6/d6e/dfc/d108/f39 [0,4194304] 0 2024-03-24T00:49:09.806 INFO:tasks.workunit.client.0.smithi038.stdout:3/893: creat d3/d30/d5d/d7c/f12e x:0 0 0 2024-03-24T00:49:09.808 INFO:tasks.workunit.client.1.smithi088.stdout:7/833: creat d3/d32/d76/df3/d103/f105 x:0 0 0 2024-03-24T00:49:09.811 INFO:tasks.workunit.client.1.smithi088.stdout:6/974: symlink db/dc5/dc8/d43/l12f 0 2024-03-24T00:49:09.812 INFO:tasks.workunit.client.1.smithi088.stdout:5/976: dwrite d1/d19/d48/dc2/fc9 [8388608,4194304] 0 2024-03-24T00:49:09.812 INFO:tasks.workunit.client.0.smithi038.stdout:6/929: dread d1/da/d1b/d2f/dc0/f104 [0,4194304] 0 2024-03-24T00:49:09.814 INFO:tasks.workunit.client.1.smithi088.stdout:4/908: dwrite d5/fa0 [0,4194304] 0 2024-03-24T00:49:09.814 INFO:tasks.workunit.client.1.smithi088.stdout:3/865: dwrite d3/d1a/d1e/f9b [0,4194304] 0 2024-03-24T00:49:09.814 INFO:tasks.workunit.client.0.smithi038.stdout:5/838: dwrite d6/d19/d20/d42/d43/d68/f8b [4194304,4194304] 0 2024-03-24T00:49:09.815 INFO:tasks.workunit.client.1.smithi088.stdout:0/937: mknod d9/dc/de/d19/d84/c129 0 2024-03-24T00:49:09.815 INFO:tasks.workunit.client.1.smithi088.stdout:0/938: stat d9/dc/de/d19/dd2 0 2024-03-24T00:49:09.815 INFO:tasks.workunit.client.1.smithi088.stdout:9/899: fsync d1/d10/d11/f17 0 2024-03-24T00:49:09.817 INFO:tasks.workunit.client.0.smithi038.stdout:7/809: dwrite d2/d9/da/d9b/ff9 [0,4194304] 0 2024-03-24T00:49:09.817 INFO:tasks.workunit.client.0.smithi038.stdout:7/810: fsync d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fca 0 2024-03-24T00:49:09.817 INFO:tasks.workunit.client.0.smithi038.stdout:7/811: creat d2/d9/d16/dfc/f12c x:0 0 0 2024-03-24T00:49:09.817 INFO:tasks.workunit.client.0.smithi038.stdout:6/930: unlink d1/da/de6/f118 0 2024-03-24T00:49:09.817 INFO:tasks.workunit.client.0.smithi038.stdout:7/812: readlink d2/d9/da/dc/l102 0 2024-03-24T00:49:09.817 INFO:tasks.workunit.client.0.smithi038.stdout:7/813: truncate d2/d9/da/d23/d2b/d5b/d72/d92/d75/ffd 470418 0 2024-03-24T00:49:09.818 INFO:tasks.workunit.client.1.smithi088.stdout:8/897: dwrite d9/dc/d11/fd6 [0,4194304] 0 2024-03-24T00:49:09.818 INFO:tasks.workunit.client.1.smithi088.stdout:8/898: dread - d9/dc/d11/dd2/f11e zero size 2024-03-24T00:49:09.819 INFO:tasks.workunit.client.1.smithi088.stdout:8/899: read d9/dc/d11/d24/d29/d5b/db9/fc5 [283865,19828] 0 2024-03-24T00:49:09.819 INFO:tasks.workunit.client.1.smithi088.stdout:8/900: write d9/dc/d11/d80/db0/fef [1372818,56968] 0 2024-03-24T00:49:09.819 INFO:tasks.workunit.client.1.smithi088.stdout:5/977: mknod d1/dd/d6f/da1/d8e/de0/c14e 0 2024-03-24T00:49:09.820 INFO:tasks.workunit.client.1.smithi088.stdout:6/975: truncate db/f13 6547358 0 2024-03-24T00:49:09.822 INFO:tasks.workunit.client.1.smithi088.stdout:1/896: dread d6/d27/d67/d88/f92 [4194304,4194304] 0 2024-03-24T00:49:09.830 INFO:tasks.workunit.client.0.smithi038.stdout:3/894: dread d3/d9/dc/da6/db5/dad/fbd [0,4194304] 0 2024-03-24T00:49:09.832 INFO:tasks.workunit.client.1.smithi088.stdout:4/909: truncate d5/d13/d18/f1b 2235430 0 2024-03-24T00:49:09.833 INFO:tasks.workunit.client.1.smithi088.stdout:7/834: write d3/d32/d34/d9c/d4b/d74/d79/fb0 [1332806,51880] 0 2024-03-24T00:49:09.834 INFO:tasks.workunit.client.1.smithi088.stdout:3/866: link d3/d1a/d1e/d2f/d45/db5/l10f d3/d1a/d1e/d2f/l113 0 2024-03-24T00:49:09.834 INFO:tasks.workunit.client.1.smithi088.stdout:3/867: readlink d3/d6/dca/dae/d4b/daf/lf8 0 2024-03-24T00:49:09.838 INFO:tasks.workunit.client.1.smithi088.stdout:2/860: dread db/d21/d35/f95 [0,4194304] 0 2024-03-24T00:49:09.843 INFO:tasks.workunit.client.0.smithi038.stdout:6/931: symlink d1/d37/d84/df5/d113/l136 0 2024-03-24T00:49:09.844 INFO:tasks.workunit.client.1.smithi088.stdout:9/900: dread d1/d22/d27/d39/d98/da3/f9f [0,4194304] 0 2024-03-24T00:49:09.849 INFO:tasks.workunit.client.0.smithi038.stdout:7/814: getdents d2/d9/da/d23/d2b/d5b/d72/d92/d33 0 2024-03-24T00:49:09.852 INFO:tasks.workunit.client.1.smithi088.stdout:5/978: symlink d1/dd/d6f/da1/d3f/d88/l14f 0 2024-03-24T00:49:09.852 INFO:tasks.workunit.client.1.smithi088.stdout:5/979: fsync d1/dd/d6f/da1/d135/d137/f13a 0 2024-03-24T00:49:09.855 INFO:tasks.workunit.client.1.smithi088.stdout:7/835: write d3/d32/d34/d9c/d4b/f99 [1290541,50166] 0 2024-03-24T00:49:09.855 INFO:tasks.workunit.client.1.smithi088.stdout:8/901: dread d9/dc/d11/d24/d29/d36/d6a/d7b/fa0 [0,4194304] 0 2024-03-24T00:49:09.855 INFO:tasks.workunit.client.1.smithi088.stdout:8/902: write d9/dc/d11/d24/f5e [3653928,5627] 0 2024-03-24T00:49:09.858 INFO:tasks.workunit.client.1.smithi088.stdout:1/897: unlink d6/d6e/dfc/d108/d5b/d5d/d61/d7d/le4 0 2024-03-24T00:49:09.858 INFO:tasks.workunit.client.1.smithi088.stdout:1/898: creat d6/d6e/dfc/f129 x:0 0 0 2024-03-24T00:49:09.860 INFO:tasks.workunit.client.0.smithi038.stdout:3/895: creat d3/d2a/d107/f12f x:0 0 0 2024-03-24T00:49:09.862 INFO:tasks.workunit.client.0.smithi038.stdout:6/932: mknod d1/d37/d84/df5/c137 0 2024-03-24T00:49:09.864 INFO:tasks.workunit.client.0.smithi038.stdout:7/815: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/d10b/c12d 0 2024-03-24T00:49:09.869 INFO:tasks.workunit.client.1.smithi088.stdout:4/910: mknod d5/d7/dd/d88/c13d 0 2024-03-24T00:49:09.869 INFO:tasks.workunit.client.1.smithi088.stdout:4/911: read d5/d13/d63/d78/fd1 [76739,53378] 0 2024-03-24T00:49:09.870 INFO:tasks.workunit.client.0.smithi038.stdout:3/896: link d3/d9/dc/da6/db5/d19/d82/d89/fa2 d3/d9/dc/f130 0 2024-03-24T00:49:09.877 INFO:tasks.workunit.client.0.smithi038.stdout:7/816: creat d2/d9/da/d23/d2b/d5b/d72/d92/d98/f12e x:0 0 0 2024-03-24T00:49:09.880 INFO:tasks.workunit.client.0.smithi038.stdout:3/897: creat d3/d9/dc/da6/db5/d19/da4/f131 x:0 0 0 2024-03-24T00:49:09.882 INFO:tasks.workunit.client.1.smithi088.stdout:0/939: getdents d9/d26/d8f/da1/d62 0 2024-03-24T00:49:09.882 INFO:tasks.workunit.client.1.smithi088.stdout:0/940: dread - d9/f69 zero size 2024-03-24T00:49:09.882 INFO:tasks.workunit.client.1.smithi088.stdout:0/941: chown d9/dc/de/d19/dd2/d103 3 1 2024-03-24T00:49:09.883 INFO:tasks.workunit.client.1.smithi088.stdout:3/868: dwrite d3/d6/d27/d3c/f42 [0,4194304] 0 2024-03-24T00:49:09.884 INFO:tasks.workunit.client.0.smithi038.stdout:7/817: mkdir d2/d2c/d9a/deb/d12f 0 2024-03-24T00:49:09.885 INFO:tasks.workunit.client.0.smithi038.stdout:6/933: getdents d1/d37/d84/df5/d113 0 2024-03-24T00:49:09.886 INFO:tasks.workunit.client.0.smithi038.stdout:3/898: mknod d3/d9/dc/da6/db5/d19/d82/d89/da8/c132 0 2024-03-24T00:49:09.886 INFO:tasks.workunit.client.0.smithi038.stdout:3/899: readlink d3/d9/def/lf3 0 2024-03-24T00:49:09.891 INFO:tasks.workunit.client.1.smithi088.stdout:5/980: dwrite d1/d10/db6/d10b/d2f/d57/d9e/dd0/f120 [0,4194304] 0 2024-03-24T00:49:09.891 INFO:tasks.workunit.client.0.smithi038.stdout:6/934: mkdir d1/da/d61/d79/d138 0 2024-03-24T00:49:09.892 INFO:tasks.workunit.client.0.smithi038.stdout:6/935: write d1/f1c [4108327,82875] 0 2024-03-24T00:49:09.892 INFO:tasks.workunit.client.0.smithi038.stdout:6/936: write d1/da/d1b/dbb/f124 [363897,861] 0 2024-03-24T00:49:09.894 INFO:tasks.workunit.client.1.smithi088.stdout:9/901: getdents d1/db5 0 2024-03-24T00:49:09.894 INFO:tasks.workunit.client.1.smithi088.stdout:9/902: fsync d1/d10/d11/d23/f70 0 2024-03-24T00:49:09.895 INFO:tasks.workunit.client.0.smithi038.stdout:7/818: rmdir d2/d9/d16 39 2024-03-24T00:49:09.897 INFO:tasks.workunit.client.0.smithi038.stdout:3/900: getdents d3/dc3 0 2024-03-24T00:49:09.898 INFO:tasks.workunit.client.0.smithi038.stdout:7/819: rmdir d2 39 2024-03-24T00:49:09.902 INFO:tasks.workunit.client.0.smithi038.stdout:7/820: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d130 0 2024-03-24T00:49:09.904 INFO:tasks.workunit.client.0.smithi038.stdout:3/901: getdents d3/d30 0 2024-03-24T00:49:09.907 INFO:tasks.workunit.client.1.smithi088.stdout:7/836: mkdir d3/d32/d34/d9c/d4b/d73/dab/d106 0 2024-03-24T00:49:09.910 INFO:tasks.workunit.client.0.smithi038.stdout:7/821: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/d10b/d131 0 2024-03-24T00:49:09.910 INFO:tasks.workunit.client.0.smithi038.stdout:7/822: dread - d2/d9/da/dc/d8a/ff8 zero size 2024-03-24T00:49:09.911 INFO:tasks.workunit.client.0.smithi038.stdout:3/902: symlink d3/d9/dc/da6/dfc/l133 0 2024-03-24T00:49:09.912 INFO:tasks.workunit.client.0.smithi038.stdout:3/903: creat d3/d30/d70/d90/d99/f134 x:0 0 0 2024-03-24T00:49:09.914 INFO:tasks.workunit.client.1.smithi088.stdout:8/903: mknod d9/dc/d11/d24/d29/d5b/db9/c130 0 2024-03-24T00:49:09.919 INFO:tasks.workunit.client.0.smithi038.stdout:1/895: sync 2024-03-24T00:49:09.919 INFO:tasks.workunit.client.0.smithi038.stdout:4/805: sync 2024-03-24T00:49:09.919 INFO:tasks.workunit.client.0.smithi038.stdout:1/896: chown da/dd/d2c/d5a/d74/c90 125568 1 2024-03-24T00:49:09.919 INFO:tasks.workunit.client.0.smithi038.stdout:1/897: write da/dd/d2c/d41/d60/dae/fc3 [679635,63180] 0 2024-03-24T00:49:09.919 INFO:tasks.workunit.client.0.smithi038.stdout:2/770: sync 2024-03-24T00:49:09.919 INFO:tasks.workunit.client.0.smithi038.stdout:9/931: sync 2024-03-24T00:49:09.920 INFO:tasks.workunit.client.0.smithi038.stdout:0/832: sync 2024-03-24T00:49:09.921 INFO:tasks.workunit.client.0.smithi038.stdout:8/888: sync 2024-03-24T00:49:09.927 INFO:tasks.workunit.client.0.smithi038.stdout:7/823: write d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/f70 [4094283,12900] 0 2024-03-24T00:49:09.927 INFO:tasks.workunit.client.0.smithi038.stdout:7/824: truncate d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/f101 759845 0 2024-03-24T00:49:09.927 INFO:tasks.workunit.client.0.smithi038.stdout:7/825: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/f11f 0 2024-03-24T00:49:09.930 INFO:tasks.workunit.client.1.smithi088.stdout:3/869: dwrite d3/de/d6f/d85/d49/d70/d96/fe3 [0,4194304] 0 2024-03-24T00:49:09.930 INFO:tasks.workunit.client.1.smithi088.stdout:3/870: dread - d3/d6/d58/da4/f106 zero size 2024-03-24T00:49:09.930 INFO:tasks.workunit.client.0.smithi038.stdout:4/806: rmdir da/d12/d24 39 2024-03-24T00:49:09.931 INFO:tasks.workunit.client.1.smithi088.stdout:1/899: truncate d6/d6e/dfc/d108/d5b/d5d/fa9 3194 0 2024-03-24T00:49:09.932 INFO:tasks.workunit.client.0.smithi038.stdout:3/904: write d3/d9/dc/da6/db5/f53 [461891,62509] 0 2024-03-24T00:49:09.935 INFO:tasks.workunit.client.0.smithi038.stdout:1/898: rename da/c32 to da/dd/d7a/d82/ddb/d11a/c127 0 2024-03-24T00:49:09.940 INFO:tasks.workunit.client.1.smithi088.stdout:4/912: creat d5/d7/d92/d121/d133/f13e x:0 0 0 2024-03-24T00:49:09.942 INFO:tasks.workunit.client.0.smithi038.stdout:9/932: creat d12/d41/f12e x:0 0 0 2024-03-24T00:49:09.946 INFO:tasks.workunit.client.0.smithi038.stdout:2/771: creat d4/d27/d2f/d30/df8/f107 x:0 0 0 2024-03-24T00:49:09.948 INFO:tasks.workunit.client.1.smithi088.stdout:0/942: creat d9/dc/de/d19/d84/dd6/d116/f12a x:0 0 0 2024-03-24T00:49:09.950 INFO:tasks.workunit.client.0.smithi038.stdout:7/826: mkdir d2/d2c/d9a/dcb/d132 0 2024-03-24T00:49:09.951 INFO:tasks.workunit.client.1.smithi088.stdout:2/861: mknod db/dd/d96/d4e/d8f/c111 0 2024-03-24T00:49:09.953 INFO:tasks.workunit.client.0.smithi038.stdout:4/807: symlink da/d12/d24/d68/d6c/dce/l113 0 2024-03-24T00:49:09.953 INFO:tasks.workunit.client.0.smithi038.stdout:4/808: fdatasync da/fa7 0 2024-03-24T00:49:09.953 INFO:tasks.workunit.client.0.smithi038.stdout:4/809: write da/d12/d24/d5f/fd4 [1885035,84448] 0 2024-03-24T00:49:09.953 INFO:tasks.workunit.client.0.smithi038.stdout:4/810: dread - da/d12/d29/d5b/d73/db4/ff3 zero size 2024-03-24T00:49:09.955 INFO:tasks.workunit.client.0.smithi038.stdout:3/905: mkdir d3/dc3/d102/d135 0 2024-03-24T00:49:09.957 INFO:tasks.workunit.client.1.smithi088.stdout:5/981: rename d1/d19/d1e/l28 to d1/dd/d6f/da1/d3f/d88/d8f/d69/d105/l150 0 2024-03-24T00:49:09.959 INFO:tasks.workunit.client.0.smithi038.stdout:1/899: link da/f35 da/dd/d7a/d82/d98/f128 0 2024-03-24T00:49:09.961 INFO:tasks.workunit.client.1.smithi088.stdout:9/903: mkdir d1/d15/d121 0 2024-03-24T00:49:09.964 INFO:tasks.workunit.client.1.smithi088.stdout:8/904: mkdir d9/dc/d11/d24/d131 0 2024-03-24T00:49:09.964 INFO:tasks.workunit.client.0.smithi038.stdout:6/937: dwrite d1/da/d73/fab [0,4194304] 0 2024-03-24T00:49:09.965 INFO:tasks.workunit.client.0.smithi038.stdout:6/938: chown d1/d37/d68/c7e 2440521 1 2024-03-24T00:49:09.965 INFO:tasks.workunit.client.0.smithi038.stdout:2/772: truncate d4/d27/ddb/d7c/fd9 3001698 0 2024-03-24T00:49:09.969 INFO:tasks.workunit.client.1.smithi088.stdout:7/837: dwrite d3/d32/d34/d9c/d4b/d74/f95 [0,4194304] 0 2024-03-24T00:49:09.974 INFO:tasks.workunit.client.0.smithi038.stdout:7/827: mkdir d2/d2c/d9a/deb/d12f/d133 0 2024-03-24T00:49:09.979 INFO:tasks.workunit.client.0.smithi038.stdout:4/811: creat da/d38/f114 x:0 0 0 2024-03-24T00:49:09.979 INFO:tasks.workunit.client.0.smithi038.stdout:4/812: creat da/d12/d24/d5f/d8a/db6/f115 x:0 0 0 2024-03-24T00:49:09.981 INFO:tasks.workunit.client.0.smithi038.stdout:3/906: truncate d3/d9/dc/da6/db5/fcf 2109824 0 2024-03-24T00:49:09.990 INFO:tasks.workunit.client.0.smithi038.stdout:1/900: rename da/dd/d7a/d82/lc1 to da/dd/l129 0 2024-03-24T00:49:09.994 INFO:tasks.workunit.client.1.smithi088.stdout:3/871: link d3/d6/d19/d2a/l23 d3/d6/d19/d2a/dc2/dd5/d10a/l114 0 2024-03-24T00:49:09.995 INFO:tasks.workunit.client.0.smithi038.stdout:0/833: dwrite d0/d12/d15/f22 [0,4194304] 0 2024-03-24T00:49:09.995 INFO:tasks.workunit.client.0.smithi038.stdout:8/889: dwrite d0/d1d/d27/d4d/daf/f110 [0,4194304] 0 2024-03-24T00:49:09.997 INFO:tasks.workunit.client.0.smithi038.stdout:9/933: getdents d12/d28/d9c/ddd/d111 0 2024-03-24T00:49:09.998 INFO:tasks.workunit.client.1.smithi088.stdout:1/900: mkdir d6/d6e/dfc/d108/d5b/d5d/d61/d7d/dc8/de9/d12a 0 2024-03-24T00:49:10.000 INFO:tasks.workunit.client.1.smithi088.stdout:4/913: mknod d5/d7/dd/d10b/c13f 0 2024-03-24T00:49:10.001 INFO:tasks.workunit.client.0.smithi038.stdout:2/773: mknod d4/d27/ddb/d7c/c108 0 2024-03-24T00:49:10.002 INFO:tasks.workunit.client.1.smithi088.stdout:2/862: chown db/d21/d35/f3d 992 1 2024-03-24T00:49:10.003 INFO:tasks.workunit.client.0.smithi038.stdout:9/934: dread d12/f35 [0,4194304] 0 2024-03-24T00:49:10.003 INFO:tasks.workunit.client.0.smithi038.stdout:9/935: fdatasync d12/d41/dc0/f100 0 2024-03-24T00:49:10.007 INFO:tasks.workunit.client.0.smithi038.stdout:7/828: creat d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/f134 x:0 0 0 2024-03-24T00:49:10.007 INFO:tasks.workunit.client.0.smithi038.stdout:7/829: chown d2/d9/da/d23/d2b/d5b/d72/d92/d33/f31 6797954 1 2024-03-24T00:49:10.008 INFO:tasks.workunit.client.1.smithi088.stdout:3/872: write d3/d6/d19/f52 [428227,34836] 0 2024-03-24T00:49:10.008 INFO:tasks.workunit.client.1.smithi088.stdout:3/873: chown d3 23 1 2024-03-24T00:49:10.014 INFO:tasks.workunit.client.1.smithi088.stdout:5/982: getdents d1/d10/d79 0 2024-03-24T00:49:10.016 INFO:tasks.workunit.client.1.smithi088.stdout:6/976: sync 2024-03-24T00:49:10.017 INFO:tasks.workunit.client.0.smithi038.stdout:4/813: symlink da/d12/d24/d68/d6c/da6/de7/l116 0 2024-03-24T00:49:10.020 INFO:tasks.workunit.client.1.smithi088.stdout:0/943: rename d9/dc/de/c47 to d9/dc/de0/c12b 0 2024-03-24T00:49:10.020 INFO:tasks.workunit.client.1.smithi088.stdout:0/944: stat d9/dc/de/d19/d8d/ffb 0 2024-03-24T00:49:10.021 INFO:tasks.workunit.client.1.smithi088.stdout:0/945: creat d9/dc/de/d19/d84/dd6/f12c x:0 0 0 2024-03-24T00:49:10.021 INFO:tasks.workunit.client.1.smithi088.stdout:0/946: fdatasync d9/f69 0 2024-03-24T00:49:10.023 INFO:tasks.workunit.client.0.smithi038.stdout:3/907: mknod d3/d30/d5d/d6a/c136 0 2024-03-24T00:49:10.023 INFO:tasks.workunit.client.0.smithi038.stdout:3/908: truncate d3/d30/d70/f8a 331096 0 2024-03-24T00:49:10.024 INFO:tasks.workunit.client.0.smithi038.stdout:1/901: mknod da/dd/d4a/d76/c12a 0 2024-03-24T00:49:10.024 INFO:tasks.workunit.client.1.smithi088.stdout:8/905: rmdir d9/dc/d11/d24/d29 39 2024-03-24T00:49:10.025 INFO:tasks.workunit.client.0.smithi038.stdout:1/902: fdatasync da/dd/d7a/d82/d98/d14/f47 0 2024-03-24T00:49:10.026 INFO:tasks.workunit.client.1.smithi088.stdout:7/838: dwrite d3/fbb [4194304,4194304] 0 2024-03-24T00:49:10.026 INFO:tasks.workunit.client.1.smithi088.stdout:7/839: chown d3/d32/d34/d9c/d4b/d73/dab/dc3/cfe 365 1 2024-03-24T00:49:10.027 INFO:tasks.workunit.client.0.smithi038.stdout:6/939: rename d1/d37/f11e to d1/da/d1b/d25/d49/d4e/d127/f139 0 2024-03-24T00:49:10.028 INFO:tasks.workunit.client.0.smithi038.stdout:0/834: mkdir d0/d12/d42/d9d/d28/d118 0 2024-03-24T00:49:10.033 INFO:tasks.workunit.client.1.smithi088.stdout:4/914: symlink d5/d13/l140 0 2024-03-24T00:49:10.033 INFO:tasks.workunit.client.1.smithi088.stdout:2/863: mkdir db/dd/d96/d99/db2/d88/d112 0 2024-03-24T00:49:10.037 INFO:tasks.workunit.client.1.smithi088.stdout:3/874: symlink d3/d6/d19/d2a/dc2/l115 0 2024-03-24T00:49:10.039 INFO:tasks.workunit.client.1.smithi088.stdout:5/983: rmdir d1/d19/d48/dc2/dcc 39 2024-03-24T00:49:10.041 INFO:tasks.workunit.client.0.smithi038.stdout:8/890: symlink d0/d45/d37/d5f/db7/dcc/l125 0 2024-03-24T00:49:10.043 INFO:tasks.workunit.client.1.smithi088.stdout:6/977: mknod db/d3d/dcb/d112/c130 0 2024-03-24T00:49:10.043 INFO:tasks.workunit.client.1.smithi088.stdout:6/978: chown db/d1d/d50/d90/d55/c7c 5338821 1 2024-03-24T00:49:10.043 INFO:tasks.workunit.client.1.smithi088.stdout:6/979: fsync db/d1d/d50/d90/d55/d99/f10a 0 2024-03-24T00:49:10.043 INFO:tasks.workunit.client.1.smithi088.stdout:6/980: write db/f105 [312853,46547] 0 2024-03-24T00:49:10.053 INFO:tasks.workunit.client.0.smithi038.stdout:0/835: dread d0/d12/d15/d1a/f98 [0,4194304] 0 2024-03-24T00:49:10.053 INFO:tasks.workunit.client.0.smithi038.stdout:0/836: fdatasync d0/d14/d84/ffb 0 2024-03-24T00:49:10.056 INFO:tasks.workunit.client.1.smithi088.stdout:1/901: rename d6/fab to d6/d6e/dfc/d108/d5b/d5d/d61/d7d/dc8/de9/f12b 0 2024-03-24T00:49:10.056 INFO:tasks.workunit.client.1.smithi088.stdout:1/902: dread - d6/d27/d67/d88/fd2 zero size 2024-03-24T00:49:10.056 INFO:tasks.workunit.client.1.smithi088.stdout:1/903: truncate d6/d27/d67/d9b/db9/ff5 925505 0 2024-03-24T00:49:10.057 INFO:tasks.workunit.client.1.smithi088.stdout:0/947: mkdir d9/d26/d108/d12d 0 2024-03-24T00:49:10.057 INFO:tasks.workunit.client.1.smithi088.stdout:9/904: getdents d1 0 2024-03-24T00:49:10.060 INFO:tasks.workunit.client.0.smithi038.stdout:2/774: mknod d4/d27/d54/d53/c109 0 2024-03-24T00:49:10.060 INFO:tasks.workunit.client.0.smithi038.stdout:7/830: mkdir d2/d9/da/d9b/d135 0 2024-03-24T00:49:10.060 INFO:tasks.workunit.client.0.smithi038.stdout:7/831: fdatasync d2/d9/da/dc/f125 0 2024-03-24T00:49:10.060 INFO:tasks.workunit.client.0.smithi038.stdout:7/832: write d2/fdf [489843,54657] 0 2024-03-24T00:49:10.060 INFO:tasks.workunit.client.0.smithi038.stdout:7/833: dread - d2/d9/da/dc/d8a/db8/fe4 zero size 2024-03-24T00:49:10.060 INFO:tasks.workunit.client.0.smithi038.stdout:7/834: chown d2/d2c/d9a/fd0 154904222 1 2024-03-24T00:49:10.060 INFO:tasks.workunit.client.0.smithi038.stdout:7/835: dread - d2/d9/da/d23/d2b/d5b/d72/fbf zero size 2024-03-24T00:49:10.065 INFO:tasks.workunit.client.1.smithi088.stdout:6/981: unlink db/d1d/d50/d90/d56/lec 0 2024-03-24T00:49:10.070 INFO:tasks.workunit.client.0.smithi038.stdout:4/814: truncate da/d12/d37/dd/d15/f17 4465730 0 2024-03-24T00:49:10.070 INFO:tasks.workunit.client.1.smithi088.stdout:7/840: rename d3/d32/d34/d9c/d4b/d73/dab/fec to d3/d32/d34/d9c/d4b/da7/dc5/f107 0 2024-03-24T00:49:10.074 INFO:tasks.workunit.client.1.smithi088.stdout:1/904: mknod d6/d6e/dfc/d108/d5b/d11b/c12c 0 2024-03-24T00:49:10.077 INFO:tasks.workunit.client.0.smithi038.stdout:6/940: creat d1/da/d61/f13a x:0 0 0 2024-03-24T00:49:10.081 INFO:tasks.workunit.client.1.smithi088.stdout:0/948: symlink d9/dc/de/d19/d44/d46/d59/l12e 0 2024-03-24T00:49:10.091 INFO:tasks.workunit.client.1.smithi088.stdout:9/905: mkdir d1/d10/d2c/d122 0 2024-03-24T00:49:10.091 INFO:tasks.workunit.client.1.smithi088.stdout:9/906: truncate d1/d10/d29/f10c 664034 0 2024-03-24T00:49:10.092 INFO:tasks.workunit.client.0.smithi038.stdout:7/836: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/d136 0 2024-03-24T00:49:10.095 INFO:tasks.workunit.client.0.smithi038.stdout:7/837: write d2/d9/da/dc/d8a/db8/fce [115494,83427] 0 2024-03-24T00:49:10.095 INFO:tasks.workunit.client.0.smithi038.stdout:7/838: readlink d2/d9/da/d23/d2b/lde 0 2024-03-24T00:49:10.095 INFO:tasks.workunit.client.0.smithi038.stdout:7/839: readlink d2/d2c/l49 0 2024-03-24T00:49:10.095 INFO:tasks.workunit.client.0.smithi038.stdout:7/840: dread - d2/d9/da/d23/f5a zero size 2024-03-24T00:49:10.095 INFO:tasks.workunit.client.0.smithi038.stdout:7/841: chown d2/d9/da/d23/d2b/d5b/d72/d92/c6a 30420370 1 2024-03-24T00:49:10.097 INFO:tasks.workunit.client.0.smithi038.stdout:3/909: rmdir d3/d30/d5d/d7c/db0 39 2024-03-24T00:49:10.098 INFO:tasks.workunit.client.1.smithi088.stdout:4/915: mkdir d5/d7/d72/d141 0 2024-03-24T00:49:10.098 INFO:tasks.workunit.client.1.smithi088.stdout:4/916: fdatasync d5/d7/f112 0 2024-03-24T00:49:10.098 INFO:tasks.workunit.client.1.smithi088.stdout:4/917: creat d5/d13/d63/f142 x:0 0 0 2024-03-24T00:49:10.099 INFO:tasks.workunit.client.1.smithi088.stdout:3/875: link d3/d6/d19/d2a/dc2/dd5/l112 d3/l116 0 2024-03-24T00:49:10.106 INFO:tasks.workunit.client.1.smithi088.stdout:5/984: rename d1/d43/d4e/df8/c125 to d1/dd/d6f/da1/d3f/d88/c151 0 2024-03-24T00:49:10.107 INFO:tasks.workunit.client.0.smithi038.stdout:6/941: mknod d1/da/d61/da2/c13b 0 2024-03-24T00:49:10.107 INFO:tasks.workunit.client.0.smithi038.stdout:5/839: sync 2024-03-24T00:49:10.107 INFO:tasks.workunit.client.1.smithi088.stdout:7/841: mknod d3/d32/d34/d9c/d4b/d73/c108 0 2024-03-24T00:49:10.108 INFO:tasks.workunit.client.1.smithi088.stdout:7/842: truncate d3/d32/d34/d9c/d4b/d74/d79/fa1 558218 0 2024-03-24T00:49:10.108 INFO:tasks.workunit.client.0.smithi038.stdout:3/910: creat d3/d30/d5d/f137 x:0 0 0 2024-03-24T00:49:10.108 INFO:tasks.workunit.client.0.smithi038.stdout:3/911: dread - d3/d30/d70/d90/d99/f110 zero size 2024-03-24T00:49:10.109 INFO:tasks.workunit.client.0.smithi038.stdout:3/912: creat d3/d9/dc/da6/db5/d19/dd2/f138 x:0 0 0 2024-03-24T00:49:10.111 INFO:tasks.workunit.client.1.smithi088.stdout:8/906: dwrite d9/dc/d11/d24/d29/d5b/fad [0,4194304] 0 2024-03-24T00:49:10.117 INFO:tasks.workunit.client.1.smithi088.stdout:1/905: symlink d6/d27/db0/l12d 0 2024-03-24T00:49:10.117 INFO:tasks.workunit.client.1.smithi088.stdout:3/876: mknod d3/d1a/d1e/d2f/d45/db5/c117 0 2024-03-24T00:49:10.117 INFO:tasks.workunit.client.1.smithi088.stdout:0/949: truncate d9/dc/de/d19/d44/d46/d65/f58 483405 0 2024-03-24T00:49:10.117 INFO:tasks.workunit.client.1.smithi088.stdout:0/950: fsync d9/dc/de/d7e/de6/df5/d115/f5f 0 2024-03-24T00:49:10.117 INFO:tasks.workunit.client.0.smithi038.stdout:1/903: dwrite da/d3c/f81 [0,4194304] 0 2024-03-24T00:49:10.118 INFO:tasks.workunit.client.1.smithi088.stdout:5/985: mknod d1/d10/db6/d10b/d2f/c152 0 2024-03-24T00:49:10.118 INFO:tasks.workunit.client.1.smithi088.stdout:4/918: rename d5/d16/d1a/d22/dc6/fce to d5/d8b/f143 0 2024-03-24T00:49:10.118 INFO:tasks.workunit.client.1.smithi088.stdout:4/919: getdents d5/d16/d37/dff 0 2024-03-24T00:49:10.120 INFO:tasks.workunit.client.0.smithi038.stdout:5/840: truncate d6/d19/da1/ffe 2990669 0 2024-03-24T00:49:10.122 INFO:tasks.workunit.client.0.smithi038.stdout:9/936: dwrite d12/d28/d9c/ddd/d111/d26/f93 [0,4194304] 0 2024-03-24T00:49:10.124 INFO:tasks.workunit.client.0.smithi038.stdout:9/937: write d12/f1e [495927,101993] 0 2024-03-24T00:49:10.128 INFO:tasks.workunit.client.1.smithi088.stdout:9/907: rmdir d1/d10/d29 39 2024-03-24T00:49:10.128 INFO:tasks.workunit.client.1.smithi088.stdout:9/908: readlink d1/lfb 0 2024-03-24T00:49:10.128 INFO:tasks.workunit.client.1.smithi088.stdout:9/909: chown d1/d10/d11 190 1 2024-03-24T00:49:10.130 INFO:tasks.workunit.client.1.smithi088.stdout:3/877: creat d3/d1a/d1e/d2f/d45/f118 x:0 0 0 2024-03-24T00:49:10.130 INFO:tasks.workunit.client.1.smithi088.stdout:4/920: rmdir d5/d13/d63/d78/d12b 39 2024-03-24T00:49:10.133 INFO:tasks.workunit.client.1.smithi088.stdout:2/864: dwrite db/dd/d96/d99/fca [0,4194304] 0 2024-03-24T00:49:10.133 INFO:tasks.workunit.client.1.smithi088.stdout:9/910: creat d1/d22/d27/d39/d98/dda/f123 x:0 0 0 2024-03-24T00:49:10.133 INFO:tasks.workunit.client.1.smithi088.stdout:9/911: dread - d1/d10/d11/d23/ff6 zero size 2024-03-24T00:49:10.138 INFO:tasks.workunit.client.1.smithi088.stdout:3/878: rename d3/de/d6f/d85/d49/f54 to d3/d6/d27/f119 0 2024-03-24T00:49:10.140 INFO:tasks.workunit.client.0.smithi038.stdout:1/904: creat da/f12b x:0 0 0 2024-03-24T00:49:10.140 INFO:tasks.workunit.client.0.smithi038.stdout:5/841: mkdir d6/d19/d20/d42/d43/d4c/d111 0 2024-03-24T00:49:10.140 INFO:tasks.workunit.client.0.smithi038.stdout:1/905: readlink da/d3c/d86/lde 0 2024-03-24T00:49:10.140 INFO:tasks.workunit.client.0.smithi038.stdout:1/906: chown da/dd/d4a 1 1 2024-03-24T00:49:10.140 INFO:tasks.workunit.client.0.smithi038.stdout:1/907: chown da/dd/d7a/d82/d98/d34/c79 10562 1 2024-03-24T00:49:10.140 INFO:tasks.workunit.client.0.smithi038.stdout:1/908: stat da/d3c/c80 0 2024-03-24T00:49:10.140 INFO:tasks.workunit.client.0.smithi038.stdout:1/909: write da/dd/d7a/d82/d98/d34/f39 [372481,6504] 0 2024-03-24T00:49:10.141 INFO:tasks.workunit.client.1.smithi088.stdout:6/982: dwrite db/dc5/dc8/d43/fa2 [0,4194304] 0 2024-03-24T00:49:10.141 INFO:tasks.workunit.client.1.smithi088.stdout:9/912: rmdir d1/d10 39 2024-03-24T00:49:10.142 INFO:tasks.workunit.client.1.smithi088.stdout:9/913: write d1/d15/d21/f66 [1294300,87971] 0 2024-03-24T00:49:10.142 INFO:tasks.workunit.client.0.smithi038.stdout:9/938: rmdir d12/d41/d57/dd3 39 2024-03-24T00:49:10.143 INFO:tasks.workunit.client.1.smithi088.stdout:3/879: mkdir d3/de/d6f/d85/d90/d11a 0 2024-03-24T00:49:10.143 INFO:tasks.workunit.client.1.smithi088.stdout:3/880: truncate d3/d6/d27/d3c/f39 957235 0 2024-03-24T00:49:10.144 INFO:tasks.workunit.client.1.smithi088.stdout:8/907: dread d9/dc/d11/fd6 [0,4194304] 0 2024-03-24T00:49:10.150 INFO:tasks.workunit.client.1.smithi088.stdout:0/951: write d9/dc/de/d19/d44/f7d [1918216,82101] 0 2024-03-24T00:49:10.150 INFO:tasks.workunit.client.1.smithi088.stdout:0/952: readlink d9/d26/d8f/da1/d62/d9b/l11b 0 2024-03-24T00:49:10.154 INFO:tasks.workunit.client.1.smithi088.stdout:5/986: read d1/d19/d48/fae [3149014,82952] 0 2024-03-24T00:49:10.155 INFO:tasks.workunit.client.0.smithi038.stdout:4/815: dwrite da/f3b [4194304,4194304] 0 2024-03-24T00:49:10.167 INFO:tasks.workunit.client.0.smithi038.stdout:8/891: dwrite d0/d45/d37/d5f/db7/dcc/fef [0,4194304] 0 2024-03-24T00:49:10.169 INFO:tasks.workunit.client.1.smithi088.stdout:0/953: write d9/d26/d8f/da1/d62/f6a [4420942,34996] 0 2024-03-24T00:49:10.170 INFO:tasks.workunit.client.1.smithi088.stdout:8/908: write d9/dc/d11/d24/d43/f50 [125741,10339] 0 2024-03-24T00:49:10.170 INFO:tasks.workunit.client.1.smithi088.stdout:8/909: chown d9/d6b/df5 6222192 1 2024-03-24T00:49:10.170 INFO:tasks.workunit.client.1.smithi088.stdout:8/910: creat d9/dc/d11/dd2/f132 x:0 0 0 2024-03-24T00:49:10.170 INFO:tasks.workunit.client.1.smithi088.stdout:7/843: dwrite d3/d32/d34/d9c/d4b/d73/dab/fb3 [0,4194304] 0 2024-03-24T00:49:10.172 INFO:tasks.workunit.client.0.smithi038.stdout:2/775: write d4/d27/d54/d53/fae [127345,111566] 0 2024-03-24T00:49:10.172 INFO:tasks.workunit.client.0.smithi038.stdout:2/776: chown d4/d18/da1/cbb 324 1 2024-03-24T00:49:10.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:09 smithi088 ceph-mon[36610]: pgmap v41: 65 pgs: 65 active+clean; 3.1 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 147 MiB/s rd, 165 MiB/s wr, 259 op/s 2024-03-24T00:49:10.184 INFO:tasks.workunit.client.0.smithi038.stdout:5/842: symlink d6/d16/d21/l112 0 2024-03-24T00:49:10.185 INFO:tasks.workunit.client.0.smithi038.stdout:3/913: dread d3/d9/f3c [0,4194304] 0 2024-03-24T00:49:10.185 INFO:tasks.workunit.client.0.smithi038.stdout:3/914: write d3/d9/f3c [3343964,44330] 0 2024-03-24T00:49:10.186 INFO:tasks.workunit.client.0.smithi038.stdout:3/915: dread - d3/d30/d70/d90/d99/f134 zero size 2024-03-24T00:49:10.186 INFO:tasks.workunit.client.0.smithi038.stdout:3/916: dread - d3/d30/d5d/d6a/f124 zero size 2024-03-24T00:49:10.186 INFO:tasks.workunit.client.0.smithi038.stdout:3/917: creat d3/d30/d5d/d7c/f139 x:0 0 0 2024-03-24T00:49:10.193 INFO:tasks.workunit.client.0.smithi038.stdout:7/842: dwrite d2/f1f [4194304,4194304] 0 2024-03-24T00:49:10.199 INFO:tasks.workunit.client.0.smithi038.stdout:9/939: creat d12/d81/d84/d88/f12f x:0 0 0 2024-03-24T00:49:10.199 INFO:tasks.workunit.client.0.smithi038.stdout:9/940: write d12/d41/f12e [665295,117032] 0 2024-03-24T00:49:10.199 INFO:tasks.workunit.client.1.smithi088.stdout:9/914: chown d1/d10/d77/d8a/f99 433638848 1 2024-03-24T00:49:10.199 INFO:tasks.workunit.client.1.smithi088.stdout:9/915: chown d1/d10/d29/def/lfa 1081 1 2024-03-24T00:49:10.200 INFO:tasks.workunit.client.1.smithi088.stdout:4/921: rename d5/d13/d63/d78/fd1 to d5/d7/dd/f144 0 2024-03-24T00:49:10.202 INFO:tasks.workunit.client.1.smithi088.stdout:3/881: dwrite d3/d6/d19/d2a/dc2/dd5/dd6/ff2 [0,4194304] 0 2024-03-24T00:49:10.202 INFO:tasks.workunit.client.1.smithi088.stdout:3/882: chown d3/d6/d27/l64 89356 1 2024-03-24T00:49:10.202 INFO:tasks.workunit.client.1.smithi088.stdout:3/883: stat d3/d67/dc3/dd7 0 2024-03-24T00:49:10.208 INFO:tasks.workunit.client.0.smithi038.stdout:4/816: mkdir da/d12/d24/d5f/d8a/d117 0 2024-03-24T00:49:10.208 INFO:tasks.workunit.client.0.smithi038.stdout:7/843: write d2/d2c/f80 [917923,80420] 0 2024-03-24T00:49:10.210 INFO:tasks.workunit.client.0.smithi038.stdout:6/942: dwrite d1/da/d1b/fb8 [0,4194304] 0 2024-03-24T00:49:10.211 INFO:tasks.workunit.client.0.smithi038.stdout:2/777: mkdir d4/d27/ddb/d10a 0 2024-03-24T00:49:10.213 INFO:tasks.workunit.client.1.smithi088.stdout:9/916: dread d1/fcc [0,4194304] 0 2024-03-24T00:49:10.213 INFO:tasks.workunit.client.1.smithi088.stdout:9/917: chown d1/d22/d27/c31 139609 1 2024-03-24T00:49:10.215 INFO:tasks.workunit.client.0.smithi038.stdout:5/843: creat d6/d19/d20/d42/d43/d68/dc8/de7/f113 x:0 0 0 2024-03-24T00:49:10.218 INFO:tasks.workunit.client.1.smithi088.stdout:0/954: mknod d9/dc/de/d19/d44/c12f 0 2024-03-24T00:49:10.218 INFO:tasks.workunit.client.1.smithi088.stdout:0/955: truncate d9/dc/de/d19/d44/d46/d59/f121 893458 0 2024-03-24T00:49:10.233 INFO:tasks.workunit.client.0.smithi038.stdout:3/918: mkdir d3/d30/d13a 0 2024-03-24T00:49:10.242 INFO:tasks.workunit.client.0.smithi038.stdout:1/910: write da/dd/d22/dab/fbc [1179861,15482] 0 2024-03-24T00:49:10.264 INFO:tasks.workunit.client.0.smithi038.stdout:7/844: link d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/cff d2/d9/da/d23/d2b/d5b/d72/d92/d5d/c137 0 2024-03-24T00:49:10.271 INFO:tasks.workunit.client.1.smithi088.stdout:3/884: dwrite d3/d6/dca/dae/d4b/daf/fb2 [0,4194304] 0 2024-03-24T00:49:10.274 INFO:tasks.workunit.client.0.smithi038.stdout:4/817: dwrite da/d12/d37/dd/d15/d4c/d70/fa5 [4194304,4194304] 0 2024-03-24T00:49:10.274 INFO:tasks.workunit.client.0.smithi038.stdout:6/943: dread d1/d37/f90 [0,4194304] 0 2024-03-24T00:49:10.274 INFO:tasks.workunit.client.0.smithi038.stdout:6/944: readlink d1/d37/d84/df5/d113/l136 0 2024-03-24T00:49:10.281 INFO:tasks.workunit.client.1.smithi088.stdout:6/983: rmdir db/d8e/d10d 39 2024-03-24T00:49:10.281 INFO:tasks.workunit.client.1.smithi088.stdout:6/984: chown db/dc5/dc8/d43/daf/l12c 37 1 2024-03-24T00:49:10.287 INFO:tasks.workunit.client.1.smithi088.stdout:2/865: rename db/dc/ca4 to db/dd4/c113 0 2024-03-24T00:49:10.287 INFO:tasks.workunit.client.1.smithi088.stdout:2/866: creat db/dd/d96/d4e/d8f/d9c/db0/f114 x:0 0 0 2024-03-24T00:49:10.289 INFO:tasks.workunit.client.1.smithi088.stdout:4/922: mknod d5/d16/d94/da6/c145 0 2024-03-24T00:49:10.289 INFO:tasks.workunit.client.0.smithi038.stdout:2/778: symlink d4/d27/ddb/d7c/d103/l10b 0 2024-03-24T00:49:10.294 INFO:tasks.workunit.client.0.smithi038.stdout:5/844: rename d6/d16/d67/f8f to d6/d16/d67/df6/f114 0 2024-03-24T00:49:10.296 INFO:tasks.workunit.client.0.smithi038.stdout:7/845: symlink d2/d9/da/d23/d2b/d5b/d72/d92/l138 0 2024-03-24T00:49:10.299 INFO:tasks.workunit.client.1.smithi088.stdout:5/987: getdents d1/d10/db6/d10b/d2f/d57/d9e/daa 0 2024-03-24T00:49:10.300 INFO:tasks.workunit.client.1.smithi088.stdout:9/918: mkdir d1/d15/d21/d9c/d124 0 2024-03-24T00:49:10.300 INFO:tasks.workunit.client.1.smithi088.stdout:9/919: write d1/d10/d11/fcb [770432,39643] 0 2024-03-24T00:49:10.306 INFO:tasks.workunit.client.0.smithi038.stdout:8/892: dwrite d0/d1d/d27/d6b/fc9 [0,4194304] 0 2024-03-24T00:49:10.307 INFO:tasks.workunit.client.0.smithi038.stdout:8/893: creat d0/d45/d37/d5f/db7/d62/dfa/dc8/f126 x:0 0 0 2024-03-24T00:49:10.320 INFO:tasks.workunit.client.0.smithi038.stdout:2/779: creat d4/d27/d2f/dc5/f10c x:0 0 0 2024-03-24T00:49:10.320 INFO:tasks.workunit.client.0.smithi038.stdout:2/780: creat d4/d18/d26/d37/de0/f10d x:0 0 0 2024-03-24T00:49:10.322 INFO:tasks.workunit.client.0.smithi038.stdout:1/911: rename da/dd/d2c/d40/lb7 to da/dd/d7a/d82/d98/d14/dc0/l12c 0 2024-03-24T00:49:10.323 INFO:tasks.workunit.client.0.smithi038.stdout:1/912: write da/dd/d2c/d41/d60/fc5 [201887,110453] 0 2024-03-24T00:49:10.324 INFO:tasks.workunit.client.0.smithi038.stdout:5/845: rmdir d6/d16/d67/ddc 0 2024-03-24T00:49:10.327 INFO:tasks.workunit.client.1.smithi088.stdout:7/844: rename d3/d31/d56/f7b to d3/d32/d34/d9c/d4b/d73/dab/dd8/f109 0 2024-03-24T00:49:10.329 INFO:tasks.workunit.client.1.smithi088.stdout:2/867: symlink db/d21/d35/d49/l115 0 2024-03-24T00:49:10.329 INFO:tasks.workunit.client.1.smithi088.stdout:2/868: creat db/dd/d96/d99/db2/f116 x:0 0 0 2024-03-24T00:49:10.333 INFO:tasks.workunit.client.1.smithi088.stdout:4/923: symlink d5/d16/d1a/d22/dc6/l146 0 2024-03-24T00:49:10.333 INFO:tasks.workunit.client.1.smithi088.stdout:4/924: chown d5/d16/d1a/d22/f68 363243 1 2024-03-24T00:49:10.333 INFO:tasks.workunit.client.1.smithi088.stdout:5/988: creat d1/d10/db6/d10b/d2f/d8a/f153 x:0 0 0 2024-03-24T00:49:10.334 INFO:tasks.workunit.client.0.smithi038.stdout:8/894: dread d0/d45/d37/f61 [4194304,4194304] 0 2024-03-24T00:49:10.336 INFO:tasks.workunit.client.0.smithi038.stdout:2/781: unlink d4/d18/fea 0 2024-03-24T00:49:10.339 INFO:tasks.workunit.client.1.smithi088.stdout:9/920: creat d1/d10/d77/d8a/f125 x:0 0 0 2024-03-24T00:49:10.341 INFO:tasks.workunit.client.0.smithi038.stdout:9/941: dwrite d12/d28/d9c/ddd/d111/d26/f5a [0,4194304] 0 2024-03-24T00:49:10.341 INFO:tasks.workunit.client.1.smithi088.stdout:3/885: dwrite d3/d6/dca/f84 [0,4194304] 0 2024-03-24T00:49:10.343 INFO:tasks.workunit.client.1.smithi088.stdout:0/956: getdents d9/dc/de/d19/d44 0 2024-03-24T00:49:10.343 INFO:tasks.workunit.client.0.smithi038.stdout:1/913: symlink da/dd/d2c/d5a/d101/de8/d108/dfb/l12d 0 2024-03-24T00:49:10.345 INFO:tasks.workunit.client.0.smithi038.stdout:6/945: dwrite d1/da/f1a [0,4194304] 0 2024-03-24T00:49:10.345 INFO:tasks.workunit.client.0.smithi038.stdout:5/846: link d6/d16/d21/d2d/c65 d6/d19/d20/d42/c115 0 2024-03-24T00:49:10.345 INFO:tasks.workunit.client.0.smithi038.stdout:5/847: fsync d6/d16/d14/ffb 0 2024-03-24T00:49:10.345 INFO:tasks.workunit.client.0.smithi038.stdout:5/848: chown d6/d16/d21/d2d/da6/ce5 0 1 2024-03-24T00:49:10.347 INFO:tasks.workunit.client.0.smithi038.stdout:7/846: getdents d2/d9/da/dc/da8/dc5 0 2024-03-24T00:49:10.349 INFO:tasks.workunit.client.1.smithi088.stdout:1/906: sync 2024-03-24T00:49:10.349 INFO:tasks.workunit.client.1.smithi088.stdout:7/845: rmdir d3/d32/d34/d9c/d4b/d73/dab/dd8 39 2024-03-24T00:49:10.349 INFO:tasks.workunit.client.1.smithi088.stdout:7/846: chown d3/d32/d34/d9c/d4b/da7/dc5/lf7 1 1 2024-03-24T00:49:10.359 INFO:tasks.workunit.client.1.smithi088.stdout:8/911: dwrite d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/fd4 [0,4194304] 0 2024-03-24T00:49:10.359 INFO:tasks.workunit.client.1.smithi088.stdout:8/912: read d9/dc/d11/d24/d29/d5b/db9/da5/fb7 [718719,3533] 0 2024-03-24T00:49:10.359 INFO:tasks.workunit.client.1.smithi088.stdout:8/913: write d9/dc/d11/d24/d29/d39/d84/f6e [3582495,35105] 0 2024-03-24T00:49:10.359 INFO:tasks.workunit.client.1.smithi088.stdout:5/989: creat d1/dd/d111/f154 x:0 0 0 2024-03-24T00:49:10.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:09 smithi038 ceph-mon[29271]: pgmap v41: 65 pgs: 65 active+clean; 3.1 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 147 MiB/s rd, 165 MiB/s wr, 259 op/s 2024-03-24T00:49:10.369 INFO:tasks.workunit.client.0.smithi038.stdout:8/895: mkdir d0/d45/d66/d107/d127 0 2024-03-24T00:49:10.369 INFO:tasks.workunit.client.0.smithi038.stdout:8/896: creat d0/d1d/d27/d4d/f128 x:0 0 0 2024-03-24T00:49:10.369 INFO:tasks.workunit.client.0.smithi038.stdout:2/782: symlink d4/d27/d54/d53/l10e 0 2024-03-24T00:49:10.371 INFO:tasks.workunit.client.1.smithi088.stdout:9/921: rename d1/d10/d29/d6d/f57 to d1/d10/d11/f126 0 2024-03-24T00:49:10.371 INFO:tasks.workunit.client.0.smithi038.stdout:1/914: mkdir da/dd/d2c/d5a/d12e 0 2024-03-24T00:49:10.372 INFO:tasks.workunit.client.0.smithi038.stdout:1/915: fsync da/f7e 0 2024-03-24T00:49:10.372 INFO:tasks.workunit.client.0.smithi038.stdout:1/916: write da/dd/d2c/d40/f5d [497530,97826] 0 2024-03-24T00:49:10.373 INFO:tasks.workunit.client.0.smithi038.stdout:4/818: dwrite da/d38/f76 [0,4194304] 0 2024-03-24T00:49:10.373 INFO:tasks.workunit.client.0.smithi038.stdout:6/946: mkdir d1/d37/d84/df5/d113/d13c 0 2024-03-24T00:49:10.373 INFO:tasks.workunit.client.0.smithi038.stdout:4/819: write da/d12/d24/d5f/d8a/da4/faa [961477,13256] 0 2024-03-24T00:49:10.376 INFO:tasks.workunit.client.0.smithi038.stdout:7/847: creat d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/f139 x:0 0 0 2024-03-24T00:49:10.380 INFO:tasks.workunit.client.1.smithi088.stdout:8/914: truncate d9/dc/d11/d24/d29/d5b/f53 572451 0 2024-03-24T00:49:10.380 INFO:tasks.workunit.client.1.smithi088.stdout:7/847: dread d3/d31/f3c [0,4194304] 0 2024-03-24T00:49:10.380 INFO:tasks.workunit.client.1.smithi088.stdout:5/990: mknod d1/d10/db6/d10b/d2f/d8a/deb/c155 0 2024-03-24T00:49:10.381 INFO:tasks.workunit.client.1.smithi088.stdout:5/991: chown d1/dd/d6f/da1/d135/d137/l12a 2 1 2024-03-24T00:49:10.381 INFO:tasks.workunit.client.1.smithi088.stdout:0/957: dread d9/d26/d8f/da1/fc2 [0,4194304] 0 2024-03-24T00:49:10.382 INFO:tasks.workunit.client.1.smithi088.stdout:9/922: truncate d1/d15/d21/f66 544376 0 2024-03-24T00:49:10.382 INFO:tasks.workunit.client.0.smithi038.stdout:1/917: mkdir da/dd/d2c/d41/d12f 0 2024-03-24T00:49:10.384 INFO:tasks.workunit.client.1.smithi088.stdout:2/869: dwrite db/f3c [0,4194304] 0 2024-03-24T00:49:10.384 INFO:tasks.workunit.client.1.smithi088.stdout:2/870: creat db/dd/d96/d99/db2/d88/f117 x:0 0 0 2024-03-24T00:49:10.384 INFO:tasks.workunit.client.1.smithi088.stdout:2/871: chown db/d21/d73/c108 660 1 2024-03-24T00:49:10.388 INFO:tasks.workunit.client.0.smithi038.stdout:6/947: unlink d1/da/d1b/d25/d49/l65 0 2024-03-24T00:49:10.390 INFO:tasks.workunit.client.1.smithi088.stdout:4/925: dwrite d5/d13/d18/d24/d3d/f42 [4194304,4194304] 0 2024-03-24T00:49:10.397 INFO:tasks.workunit.client.0.smithi038.stdout:7/848: dread d2/d9/da/d23/d2b/d5b/d72/d92/d98/fad [0,4194304] 0 2024-03-24T00:49:10.402 INFO:tasks.workunit.client.1.smithi088.stdout:7/848: stat d3/d32/d34/d9c/d4b/d73/dab/lb1 0 2024-03-24T00:49:10.402 INFO:tasks.workunit.client.1.smithi088.stdout:7/849: chown d3/d32/d34/d9c/d4b/d73/la4 66773682 1 2024-03-24T00:49:10.402 INFO:tasks.workunit.client.1.smithi088.stdout:5/992: creat d1/dd/d6f/f156 x:0 0 0 2024-03-24T00:49:10.402 INFO:tasks.workunit.client.1.smithi088.stdout:9/923: rename d1/d10/l46 to d1/d10/d77/d8a/de0/l127 0 2024-03-24T00:49:10.402 INFO:tasks.workunit.client.1.smithi088.stdout:5/993: chown d1/dd/d6f/da1/d8e/d145 1770622 1 2024-03-24T00:49:10.402 INFO:tasks.workunit.client.1.smithi088.stdout:3/886: dread d3/d6/dca/dae/d4b/fbf [0,4194304] 0 2024-03-24T00:49:10.403 INFO:tasks.workunit.client.0.smithi038.stdout:1/918: mkdir da/dd/d7a/d130 0 2024-03-24T00:49:10.403 INFO:tasks.workunit.client.0.smithi038.stdout:6/948: mkdir d1/d37/d84/df5/d113/d13c/d13d 0 2024-03-24T00:49:10.403 INFO:tasks.workunit.client.0.smithi038.stdout:6/949: dread - d1/da/d1b/d25/d49/fda zero size 2024-03-24T00:49:10.403 INFO:tasks.workunit.client.0.smithi038.stdout:6/950: write d1/da/d61/d79/fa3 [178751,74384] 0 2024-03-24T00:49:10.409 INFO:tasks.workunit.client.0.smithi038.stdout:1/919: link da/dd/d22/dab/fbc da/dd/d2c/d5a/d101/dda/de6/df8/f131 0 2024-03-24T00:49:10.409 INFO:tasks.workunit.client.0.smithi038.stdout:1/920: write da/dd/d4a/d76/fad [2507048,44709] 0 2024-03-24T00:49:10.409 INFO:tasks.workunit.client.0.smithi038.stdout:1/921: fdatasync da/dd/d2c/d41/d60/dae/fc3 0 2024-03-24T00:49:10.409 INFO:tasks.workunit.client.0.smithi038.stdout:1/922: fsync da/f58 0 2024-03-24T00:49:10.411 INFO:tasks.workunit.client.1.smithi088.stdout:7/850: rename d3/d32/f5a to d3/d32/d76/df3/d103/f10a 0 2024-03-24T00:49:10.411 INFO:tasks.workunit.client.1.smithi088.stdout:7/851: chown d3/d32/d34/d9c/d4b/d74/f82 14089032 1 2024-03-24T00:49:10.414 INFO:tasks.workunit.client.0.smithi038.stdout:6/951: mkdir d1/d37/d84/d13e 0 2024-03-24T00:49:10.414 INFO:tasks.workunit.client.0.smithi038.stdout:6/952: fdatasync d1/d37/ff4 0 2024-03-24T00:49:10.414 INFO:tasks.workunit.client.0.smithi038.stdout:6/953: readlink d1/da/d73/lff 0 2024-03-24T00:49:10.417 INFO:tasks.workunit.client.0.smithi038.stdout:8/897: dread d0/d10/f92 [0,4194304] 0 2024-03-24T00:49:10.419 INFO:tasks.workunit.client.0.smithi038.stdout:9/942: dwrite d12/d81/d84/d88/fa9 [0,4194304] 0 2024-03-24T00:49:10.420 INFO:tasks.workunit.client.1.smithi088.stdout:0/958: read d9/dc/de/d19/d44/d46/d65/d6f/f9a [470122,112495] 0 2024-03-24T00:49:10.420 INFO:tasks.workunit.client.1.smithi088.stdout:0/959: dread - d9/dc/de/d19/f106 zero size 2024-03-24T00:49:10.424 INFO:tasks.workunit.client.0.smithi038.stdout:7/849: read d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/fe2 [3439780,84584] 0 2024-03-24T00:49:10.424 INFO:tasks.workunit.client.1.smithi088.stdout:9/924: creat d1/d22/d27/f128 x:0 0 0 2024-03-24T00:49:10.426 INFO:tasks.workunit.client.0.smithi038.stdout:6/954: rename d1/da/d1b/d2f/l32 to d1/d37/d68/l13f 0 2024-03-24T00:49:10.426 INFO:tasks.workunit.client.0.smithi038.stdout:6/955: fdatasync d1/d37/d84/ffa 0 2024-03-24T00:49:10.426 INFO:tasks.workunit.client.0.smithi038.stdout:6/956: chown d1/da/d73/fab 2007 1 2024-03-24T00:49:10.427 INFO:tasks.workunit.client.1.smithi088.stdout:3/887: getdents d3/d1a/d1e/d2f/d45/db5/d6e 0 2024-03-24T00:49:10.427 INFO:tasks.workunit.client.1.smithi088.stdout:3/888: truncate d3/d6/d58/da4/f106 606581 0 2024-03-24T00:49:10.428 INFO:tasks.workunit.client.1.smithi088.stdout:3/889: chown d3/de/ce8 17255 1 2024-03-24T00:49:10.428 INFO:tasks.workunit.client.1.smithi088.stdout:3/890: dread - d3/d6/d58/da4/ff9 zero size 2024-03-24T00:49:10.428 INFO:tasks.workunit.client.1.smithi088.stdout:3/891: stat d3/d6/d19/f77 0 2024-03-24T00:49:10.428 INFO:tasks.workunit.client.1.smithi088.stdout:3/892: chown d3/d6/l1b 435 1 2024-03-24T00:49:10.429 INFO:tasks.workunit.client.0.smithi038.stdout:8/898: getdents d0/d1d/d20/d79/d102 0 2024-03-24T00:49:10.429 INFO:tasks.workunit.client.0.smithi038.stdout:8/899: dread - d0/d10/fd3 zero size 2024-03-24T00:49:10.436 INFO:tasks.workunit.client.1.smithi088.stdout:0/960: creat d9/dc/de/d19/d8d/f130 x:0 0 0 2024-03-24T00:49:10.438 INFO:tasks.workunit.client.0.smithi038.stdout:7/850: symlink d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/l13a 0 2024-03-24T00:49:10.440 INFO:tasks.workunit.client.1.smithi088.stdout:3/893: mkdir d3/d6/d19/d2a/dc2/d11b 0 2024-03-24T00:49:10.445 INFO:tasks.workunit.client.1.smithi088.stdout:7/852: dread d3/f2d [0,4194304] 0 2024-03-24T00:49:10.449 INFO:tasks.workunit.client.0.smithi038.stdout:6/957: symlink d1/da/d1b/d25/d49/d72/l140 0 2024-03-24T00:49:10.449 INFO:tasks.workunit.client.0.smithi038.stdout:1/923: dread da/d3c/f81 [0,4194304] 0 2024-03-24T00:49:10.450 INFO:tasks.workunit.client.1.smithi088.stdout:1/907: dwrite d6/d27/d67/f6b [0,4194304] 0 2024-03-24T00:49:10.450 INFO:tasks.workunit.client.1.smithi088.stdout:0/961: unlink d9/d26/f5a 0 2024-03-24T00:49:10.450 INFO:tasks.workunit.client.0.smithi038.stdout:9/943: symlink d12/d55/d56/l130 0 2024-03-24T00:49:10.451 INFO:tasks.workunit.client.1.smithi088.stdout:9/925: dread d1/d10/f1e [0,4194304] 0 2024-03-24T00:49:10.452 INFO:tasks.workunit.client.0.smithi038.stdout:5/849: dwrite d6/d16/d21/d2d/fd2 [0,4194304] 0 2024-03-24T00:49:10.457 INFO:tasks.workunit.client.0.smithi038.stdout:2/783: dwrite d4/d18/d26/f62 [0,4194304] 0 2024-03-24T00:49:10.458 INFO:tasks.workunit.client.1.smithi088.stdout:3/894: rename d3/de/ce8 to d3/d6/d19/d2a/dc2/d11b/c11c 0 2024-03-24T00:49:10.459 INFO:tasks.workunit.client.1.smithi088.stdout:3/895: read d3/de/d6f/d85/f9a [51075,7187] 0 2024-03-24T00:49:10.459 INFO:tasks.workunit.client.1.smithi088.stdout:3/896: write d3/fb3 [927028,2847] 0 2024-03-24T00:49:10.459 INFO:tasks.workunit.client.1.smithi088.stdout:3/897: read d3/d6/d58/da4/f106 [369057,127232] 0 2024-03-24T00:49:10.462 INFO:tasks.workunit.client.0.smithi038.stdout:9/944: rmdir d12/d1a 39 2024-03-24T00:49:10.467 INFO:tasks.workunit.client.0.smithi038.stdout:5/850: getdents d6/d16/d10f/d92 0 2024-03-24T00:49:10.467 INFO:tasks.workunit.client.0.smithi038.stdout:7/851: dread d2/d9/da/f36 [0,4194304] 0 2024-03-24T00:49:10.468 INFO:tasks.workunit.client.0.smithi038.stdout:5/851: creat d6/d19/d20/d64/d6b/d77/f116 x:0 0 0 2024-03-24T00:49:10.471 INFO:tasks.workunit.client.1.smithi088.stdout:8/915: dwrite d9/dc/d11/d24/d29/d36/d6a/d7b/d8f/dbc/fcf [0,4194304] 0 2024-03-24T00:49:10.471 INFO:tasks.workunit.client.1.smithi088.stdout:8/916: chown d9/dc/d11/d24/d29/d36/d6a/l7f 50308 1 2024-03-24T00:49:10.471 INFO:tasks.workunit.client.1.smithi088.stdout:8/917: chown d9/dc/d11/d24/d29/d36/dc1/dc4/dd7/d54/d5d/db3/lbf 0 1 2024-03-24T00:49:10.472 INFO:tasks.workunit.client.1.smithi088.stdout:0/962: creat d9/d26/d8f/f131 x:0 0 0 2024-03-24T00:49:10.474 INFO:tasks.workunit.client.1.smithi088.stdout:5/994: dwrite d1/d10/db6/d10b/d2f/d57/d9e/daa/f12c [4194304,4194304] 0 2024-03-24T00:49:10.474 INFO:tasks.workunit.client.1.smithi088.stdout:5/995: fsync d1/d43/d4e/d5b/fcb 0 2024-03-24T00:49:10.477 INFO:tasks.workunit.client.0.smithi038.stdout:4/820: dwrite da/d38/fa3 [0,4194304] 0 2024-03-24T00:49:10.480 INFO:tasks.workunit.client.0.smithi038.stdout:2/784: symlink d4/d18/d26/l10f 0 2024-03-24T00:49:10.480 INFO:tasks.workunit.client.0.smithi038.stdout:2/785: fsync d4/d27/d2f/f3c 0 2024-03-24T00:49:10.480 INFO:tasks.workunit.client.0.smithi038.stdout:2/786: chown d4/f1b 39 1 2024-03-24T00:49:10.481 INFO:tasks.workunit.client.1.smithi088.stdout:7/853: getdents d3/d32/d34/d9c/d4b/d74/d79/d7f 0 2024-03-24T00:49:10.490 INFO:tasks.workunit.client.0.smithi038.stdout:1/924: dread da/dd/d2c/d40/de9/f113 [0,4194304] 0 2024-03-24T00:49:10.490 INFO:tasks.workunit.client.0.smithi038.stdout:1/925: dread - da/dd/d7a/d82/d98/d14/dc0/f119 zero size 2024-03-24T00:49:10.491 INFO:tasks.workunit.client.0.smithi038.stdout:1/926: write da/dd/d2c/d41/d60/dae/fc3 [1311722,104091] 0 2024-03-24T00:49:10.491 INFO:tasks.workunit.client.0.smithi038.stdout:1/927: creat da/dd/d4a/d8b/f132 x:0 0 0 2024-03-24T00:49:10.491 INFO:tasks.workunit.client.0.smithi038.stdout:9/945: dread d12/d28/f4c [0,4194304] 0 2024-03-24T00:49:10.494 INFO:tasks.workunit.client.1.smithi088.stdout:0/963: dread d9/dc/de/d19/f7f [0,4194304] 0 2024-03-24T00:49:10.494 INFO:tasks.workunit.client.1.smithi088.stdout:8/918: creat d9/dc/d11/d24/d29/d36/dc1/f133 x:0 0 0 2024-03-24T00:49:10.502 INFO:tasks.workunit.client.1.smithi088.stdout:5/996: truncate d1/dd/d6f/da1/d3f/f11c 1271032 0 2024-03-24T00:49:10.502 INFO:tasks.workunit.client.0.smithi038.stdout:2/787: mkdir d4/d27/ddb/de8/d110 0 2024-03-24T00:49:10.502 INFO:tasks.workunit.client.0.smithi038.stdout:2/788: fsync d4/d27/ddb/f78 0 2024-03-24T00:49:10.504 INFO:tasks.workunit.client.1.smithi088.stdout:9/926: link d1/d10/d9b/dab/db8/f7b d1/d10/d77/d8a/f129 0 2024-03-24T00:49:10.508 INFO:tasks.workunit.client.0.smithi038.stdout:1/928: mknod da/dd/d2c/d5a/d101/de8/d108/dc4/c133 0 2024-03-24T00:49:10.508 INFO:tasks.workunit.client.1.smithi088.stdout:6/985: sync 2024-03-24T00:49:10.509 INFO:tasks.workunit.client.1.smithi088.stdout:1/908: rename d6/d27/d67/d9b/l9d to d6/d6e/dfc/d108/d5b/l12e 0 2024-03-24T00:49:10.509 INFO:tasks.workunit.client.0.smithi038.stdout:9/946: rename d12/d28/d9c/ddd/d111/d26/l27 to d12/d81/d84/dec/l131 0 2024-03-24T00:49:10.514 INFO:tasks.workunit.client.1.smithi088.stdout:7/854: unlink d3/f2b 0 2024-03-24T00:49:10.516 INFO:tasks.workunit.client.1.smithi088.stdout:0/964: link d9/dc/de/d19/d44/d46/d65/f58 d9/dc/de/d7e/d91/f132 0 2024-03-24T00:49:10.517 INFO:tasks.workunit.client.1.smithi088.stdout:8/919: getdents d9/dc/d11/d24/d29/d5b/d8b/dd3 0 2024-03-24T00:49:10.520 INFO:tasks.workunit.client.1.smithi088.stdout:5/997: creat d1/dd/d6f/da1/d8e/de0/f157 x:0 0 0 2024-03-24T00:49:10.520 INFO:tasks.workunit.client.1.smithi088.stdout:5/998: readlink d1/d10/d79/dbc/l10e 0 2024-03-24T00:49:10.522 INFO:tasks.workunit.client.1.smithi088.stdout:6/986: mkdir db/dc5/dc8/d43/d5b/d131 0 2024-03-24T00:49:10.525 INFO:tasks.workunit.client.1.smithi088.stdout:7/855: creat d3/d32/f10b x:0 0 0 2024-03-24T00:49:10.526 INFO:tasks.workunit.client.1.smithi088.stdout:0/965: mkdir d9/d26/d8f/dc7/ddd/d133 0 2024-03-24T00:49:10.527 INFO:tasks.workunit.client.1.smithi088.stdout:8/920: rmdir d9 39 2024-03-24T00:49:10.527 INFO:tasks.workunit.client.1.smithi088.stdout:8/921: dread - d9/dc/d11/d24/fca zero size 2024-03-24T00:49:10.529 INFO:tasks.workunit.client.1.smithi088.stdout:5/999: rmdir d1/dd/d6f/da1/d3f/d106/d115 39 2024-03-24T00:49:10.531 INFO:tasks.workunit.client.0.smithi038.stdout:6/958: write d1/da/d1b/d2f/dc0/f104 [2998625,106618] 0 2024-03-24T00:49:10.532 INFO:tasks.workunit.client.0.smithi038.stdout:8/900: dwrite d0/d45/d37/d5f/db7/f4f [4194304,4194304] 0 2024-03-24T00:49:10.532 INFO:tasks.workunit.client.1.smithi088.stdout:6/987: mkdir db/dd/d61/d6d/d74/d88/d132 0 2024-03-24T00:49:10.533 INFO:tasks.workunit.client.1.smithi088.stdout:6/988: readlink db/d3d/l11b 0 2024-03-24T00:49:10.534 INFO:tasks.workunit.client.0.smithi038.stdout:6/959: truncate d1/da/d1b/fb3 919731 0 2024-03-24T00:49:10.534 INFO:tasks.workunit.client.0.smithi038.stdout:6/960: readlink d1/da/d1b/dfd/l12e 0 2024-03-24T00:49:10.534 INFO:tasks.workunit.client.0.smithi038.stdout:9/947: dread d12/d55/fb5 [4194304,4194304] 0 2024-03-24T00:49:10.534 INFO:tasks.workunit.client.0.smithi038.stdout:2/789: dread d4/f1b [4194304,4194304] 0 2024-03-24T00:49:10.535 INFO:tasks.workunit.client.0.smithi038.stdout:9/948: write d12/d81/f105 [1032035,30522] 0 2024-03-24T00:49:10.535 INFO:tasks.workunit.client.0.smithi038.stdout:9/949: readlink d12/l19 0 2024-03-24T00:49:10.535 INFO:tasks.workunit.client.0.smithi038.stdout:9/950: write d12/d28/da8/fae [620834,101982] 0 2024-03-24T00:49:10.535 INFO:tasks.workunit.client.1.smithi088.stdout:7/856: creat d3/d32/d76/df3/f10c x:0 0 0 2024-03-24T00:49:10.535 INFO:tasks.workunit.client.0.smithi038.stdout:9/951: write d12/d81/d84/dec/ff6 [1932884,28715] 0 2024-03-24T00:49:10.538 INFO:tasks.workunit.client.1.smithi088.stdout:9/927: write d1/d22/f8c [1905024,107231] 0 2024-03-24T00:49:10.541 INFO:tasks.workunit.client.1.smithi088.stdout:0/966: rename d9/d26/d8f/da1/d62/f6a to d9/dc/de/d19/dd2/d103/f134 0 2024-03-24T00:49:10.541 INFO:tasks.workunit.client.1.smithi088.stdout:0/967: readlink d9/d15/l23 0 2024-03-24T00:49:10.541 INFO:tasks.workunit.client.1.smithi088.stdout:6/989: mkdir db/d3d/dcb/d112/d133 0 2024-03-24T00:49:10.542 INFO:tasks.workunit.client.1.smithi088.stdout:7/857: symlink d3/d32/d34/d9c/d4b/da7/dc5/d101/l10d 0 2024-03-24T00:49:10.542 INFO:tasks.workunit.client.1.smithi088.stdout:6/990: write db/dc5/dc8/d4b/f6e [4856792,129235] 0 2024-03-24T00:49:10.544 INFO:tasks.workunit.client.1.smithi088.stdout:3/898: dwrite d3/de/d6f/d85/d49/f99 [0,4194304] 0 2024-03-24T00:49:10.546 INFO:tasks.workunit.client.0.smithi038.stdout:2/790: truncate d4/d18/f22 2106611 0 2024-03-24T00:49:10.546 INFO:tasks.workunit.client.0.smithi038.stdout:2/791: fsync d4/d18/d26/f8c 0 2024-03-24T00:49:10.546 INFO:tasks.workunit.client.0.smithi038.stdout:8/901: getdents d0/d45/d37/d5f 0 2024-03-24T00:49:10.546 INFO:tasks.workunit.client.0.smithi038.stdout:8/902: chown d0/d1d/d20/cc3 1 1 2024-03-24T00:49:10.546 INFO:tasks.workunit.client.0.smithi038.stdout:8/903: creat d0/d45/d37/d5f/db7/dcc/f129 x:0 0 0 2024-03-24T00:49:10.547 INFO:tasks.workunit.client.0.smithi038.stdout:9/952: mknod d12/d28/d9c/ddd/d111/d8f/d10c/c132 0 2024-03-24T00:49:10.555 INFO:tasks.workunit.client.1.smithi088.stdout:7/858: getdents d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/dd6 0 2024-03-24T00:49:10.555 INFO:tasks.workunit.client.1.smithi088.stdout:7/859: fsync d3/d31/d56/fe7 0 2024-03-24T00:49:10.555 INFO:tasks.workunit.client.0.smithi038.stdout:4/821: dwrite da/d12/d37/fe6 [0,4194304] 0 2024-03-24T00:49:10.557 INFO:tasks.workunit.client.0.smithi038.stdout:8/904: mknod d0/d1d/d27/d6b/dfc/c12a 0 2024-03-24T00:49:10.557 INFO:tasks.workunit.client.0.smithi038.stdout:8/905: chown d0/d1d/d27/d4d 14508176 1 2024-03-24T00:49:10.562 INFO:tasks.workunit.client.0.smithi038.stdout:9/953: mkdir d12/d41/d57/d133 0 2024-03-24T00:49:10.562 INFO:tasks.workunit.client.0.smithi038.stdout:9/954: dread - d12/d1a/d5f/f10f zero size 2024-03-24T00:49:10.562 INFO:tasks.workunit.client.0.smithi038.stdout:9/955: truncate d12/d28/d9c/ddd/d111/f3f 6003267 0 2024-03-24T00:49:10.572 INFO:tasks.workunit.client.0.smithi038.stdout:2/792: dread d4/d55/fb2 [0,4194304] 0 2024-03-24T00:49:10.572 INFO:tasks.workunit.client.0.smithi038.stdout:2/793: chown d4/d55 40787509 1 2024-03-24T00:49:10.573 INFO:tasks.workunit.client.0.smithi038.stdout:4/822: dread da/d12/d24/d68/d6c/f85 [0,4194304] 0 2024-03-24T00:49:10.574 INFO:tasks.workunit.client.1.smithi088.stdout:3/899: symlink d3/de/d6f/d85/d49/l11d 0 2024-03-24T00:49:10.574 INFO:tasks.workunit.client.1.smithi088.stdout:3/900: chown d3/d6/d27/d3c/lc0 7485619 1 2024-03-24T00:49:10.574 INFO:tasks.workunit.client.1.smithi088.stdout:3/901: creat d3/d32/f11e x:0 0 0 2024-03-24T00:49:10.574 INFO:tasks.workunit.client.1.smithi088.stdout:3/902: truncate d3/f95 4988545 0 2024-03-24T00:49:10.575 INFO:tasks.workunit.client.1.smithi088.stdout:3/903: chown d3/d1a/d1e/d2f/c3d 76298899 1 2024-03-24T00:49:10.575 INFO:tasks.workunit.client.1.smithi088.stdout:3/904: stat d3/d1a/d1e/d2f/d45/f5d 0 2024-03-24T00:49:10.575 INFO:tasks.workunit.client.0.smithi038.stdout:8/906: symlink d0/d45/d66/d6d/d106/l12b 0 2024-03-24T00:49:10.575 INFO:tasks.workunit.client.0.smithi038.stdout:8/907: chown d0/d45/d66 918 1 2024-03-24T00:49:10.575 INFO:tasks.workunit.client.0.smithi038.stdout:8/908: creat d0/d45/d37/d5f/db7/d62/dfa/dc8/f12c x:0 0 0 2024-03-24T00:49:10.578 INFO:tasks.workunit.client.0.smithi038.stdout:9/956: mknod d12/d28/d71/c134 0 2024-03-24T00:49:10.578 INFO:tasks.workunit.client.0.smithi038.stdout:9/957: stat d12/d28/d9c/ddd/d111 0 2024-03-24T00:49:10.580 INFO:tasks.workunit.client.0.smithi038.stdout:2/794: mkdir d4/d27/d54/d67/d111 0 2024-03-24T00:49:10.582 INFO:tasks.workunit.client.1.smithi088.stdout:8/922: rename d9/dc/d11/d24/d29/d36/dc1/dc4/dd7 to d9/dc/d11/d24/d43/d4a/d118/d134 0 2024-03-24T00:49:10.583 INFO:tasks.workunit.client.0.smithi038.stdout:4/823: getdents da/d12/d29 0 2024-03-24T00:49:10.583 INFO:tasks.workunit.client.0.smithi038.stdout:4/824: stat da/d12/d24/d5f/fd4 0 2024-03-24T00:49:10.586 INFO:tasks.workunit.client.1.smithi088.stdout:7/860: symlink d3/d32/d34/d9c/d4b/da7/dc5/d101/l10e 0 2024-03-24T00:49:10.586 INFO:tasks.workunit.client.1.smithi088.stdout:7/861: fdatasync d3/d31/d56/fe7 0 2024-03-24T00:49:10.588 INFO:tasks.workunit.client.0.smithi038.stdout:9/958: symlink d12/d28/d9c/ddd/l135 0 2024-03-24T00:49:10.589 INFO:tasks.workunit.client.1.smithi088.stdout:3/905: symlink d3/d1a/l11f 0 2024-03-24T00:49:10.590 INFO:tasks.workunit.client.0.smithi038.stdout:6/961: dwrite d1/da/d61/f106 [0,4194304] 0 2024-03-24T00:49:10.593 INFO:tasks.workunit.client.1.smithi088.stdout:0/968: rename d9/dc/de/d7e/de6/df5/d115/lf9 to d9/dc/de/d7e/l135 0 2024-03-24T00:49:10.593 INFO:tasks.workunit.client.1.smithi088.stdout:0/969: dread - d9/d26/d8f/da1/d62/d9b/db0/db4/fd9 zero size 2024-03-24T00:49:10.594 INFO:tasks.workunit.client.1.smithi088.stdout:9/928: dwrite d1/d10/f34 [0,4194304] 0 2024-03-24T00:49:10.595 INFO:tasks.workunit.client.0.smithi038.stdout:9/959: fdatasync d12/d81/d84/dec/ff6 0 2024-03-24T00:49:10.595 INFO:tasks.workunit.client.0.smithi038.stdout:2/795: creat d4/d27/ddb/f112 x:0 0 0 2024-03-24T00:49:10.596 INFO:tasks.workunit.client.0.smithi038.stdout:2/796: readlink d4/d27/d54/d67/lbd 0 2024-03-24T00:49:10.596 INFO:tasks.workunit.client.0.smithi038.stdout:1/929: dwrite da/f12b [0,4194304] 0 2024-03-24T00:49:10.596 INFO:tasks.workunit.client.0.smithi038.stdout:1/930: chown da/dd/d2c/d41/d60/ce7 2210340 1 2024-03-24T00:49:10.597 INFO:tasks.workunit.client.1.smithi088.stdout:7/862: symlink d3/d32/d34/d9c/d4b/d73/dab/dc3/l10f 0 2024-03-24T00:49:10.597 INFO:tasks.workunit.client.1.smithi088.stdout:7/863: stat d3/d32/d34/d9c/d4b/d74/d79/dea 0 2024-03-24T00:49:10.604 INFO:tasks.workunit.client.1.smithi088.stdout:9/929: dread d1/d10/d9b/dab/d7a/fd9 [0,4194304] 0 2024-03-24T00:49:10.605 INFO:tasks.workunit.client.0.smithi038.stdout:4/825: write da/d38/fa3 [3291877,75300] 0 2024-03-24T00:49:10.614 INFO:tasks.workunit.client.1.smithi088.stdout:9/930: creat d1/d15/d121/f12a x:0 0 0 2024-03-24T00:49:10.614 INFO:tasks.workunit.client.1.smithi088.stdout:9/931: write d1/d22/f3f [954617,15946] 0 2024-03-24T00:49:10.614 INFO:tasks.workunit.client.1.smithi088.stdout:7/864: dread d3/d32/d34/d9c/d4b/d74/d79/d7f/fdd [0,4194304] 0 2024-03-24T00:49:10.617 INFO:tasks.workunit.client.0.smithi038.stdout:3/919: sync 2024-03-24T00:49:10.617 INFO:tasks.workunit.client.0.smithi038.stdout:0/837: sync 2024-03-24T00:49:10.617 INFO:tasks.workunit.client.0.smithi038.stdout:1/931: unlink da/f72 0 2024-03-24T00:49:10.617 INFO:tasks.workunit.client.0.smithi038.stdout:1/932: chown da/dd/d2c/d40 0 1 2024-03-24T00:49:10.617 INFO:tasks.workunit.client.0.smithi038.stdout:1/933: readlink da/l4b 0 2024-03-24T00:49:10.617 INFO:tasks.workunit.client.0.smithi038.stdout:1/934: readlink da/dd/d4a/l112 0 2024-03-24T00:49:10.620 INFO:tasks.workunit.client.0.smithi038.stdout:3/920: mknod d3/d9/dc/da6/db5/d19/d82/d89/da8/d105/c13b 0 2024-03-24T00:49:10.622 INFO:tasks.workunit.client.0.smithi038.stdout:0/838: mkdir d0/d14/d84/d119 0 2024-03-24T00:49:10.628 INFO:tasks.workunit.client.0.smithi038.stdout:1/935: symlink da/dd/d7a/d82/l134 0 2024-03-24T00:49:10.628 INFO:tasks.workunit.client.0.smithi038.stdout:3/921: symlink d3/d9/dc/df/l13c 0 2024-03-24T00:49:10.628 INFO:tasks.workunit.client.0.smithi038.stdout:3/922: truncate d3/d9/dc/da6/db5/d19/dd2/f112 163484 0 2024-03-24T00:49:10.631 INFO:tasks.workunit.client.0.smithi038.stdout:3/923: creat d3/d9/dc/da6/db5/d19/f13d x:0 0 0 2024-03-24T00:49:10.631 INFO:tasks.workunit.client.1.smithi088.stdout:3/906: dread d3/d6/d27/f119 [4194304,4194304] 0 2024-03-24T00:49:10.631 INFO:tasks.workunit.client.1.smithi088.stdout:3/907: write d3/d6/d19/d2a/d5f/f65 [4064442,53263] 0 2024-03-24T00:49:10.632 INFO:tasks.workunit.client.1.smithi088.stdout:3/908: creat d3/d1a/d1e/d2f/d45/f120 x:0 0 0 2024-03-24T00:49:10.632 INFO:tasks.workunit.client.0.smithi038.stdout:3/924: creat d3/d9/def/f13e x:0 0 0 2024-03-24T00:49:10.632 INFO:tasks.workunit.client.0.smithi038.stdout:3/925: fdatasync d3/d9/dc/da6/db5/d19/f42 0 2024-03-24T00:49:10.632 INFO:tasks.workunit.client.1.smithi088.stdout:8/923: dwrite d9/dc/d11/d24/d29/d36/d6a/d7b/d8f/ff0 [0,4194304] 0 2024-03-24T00:49:10.649 INFO:tasks.workunit.client.1.smithi088.stdout:4/926: sync 2024-03-24T00:49:10.649 INFO:tasks.workunit.client.1.smithi088.stdout:2/872: sync 2024-03-24T00:49:10.649 INFO:tasks.workunit.client.1.smithi088.stdout:2/873: chown db/f18 4805653 1 2024-03-24T00:49:10.649 INFO:tasks.workunit.client.1.smithi088.stdout:3/909: symlink d3/d1a/d1e/d2f/d45/db5/d6e/l121 0 2024-03-24T00:49:10.649 INFO:tasks.workunit.client.1.smithi088.stdout:3/910: chown d3/de/cb0 0 1 2024-03-24T00:49:10.649 INFO:tasks.workunit.client.1.smithi088.stdout:8/924: symlink d9/dc/d11/d24/d29/d39/d84/l135 0 2024-03-24T00:49:10.654 INFO:tasks.workunit.client.1.smithi088.stdout:4/927: truncate d5/fa0 2430737 0 2024-03-24T00:49:10.661 INFO:tasks.workunit.client.1.smithi088.stdout:4/928: truncate d5/d13/d18/d24/f53 2982056 0 2024-03-24T00:49:10.663 INFO:tasks.workunit.client.0.smithi038.stdout:4/826: dread da/d12/d37/dd/d15/d4c/d70/f9b [0,4194304] 0 2024-03-24T00:49:10.663 INFO:tasks.workunit.client.0.smithi038.stdout:4/827: truncate da/d12/d24/d68/d6c/fe9 433576 0 2024-03-24T00:49:10.663 INFO:tasks.workunit.client.0.smithi038.stdout:4/828: chown da/d12/d24/d68/d6c/dce/l113 521959177 1 2024-03-24T00:49:10.664 INFO:tasks.workunit.client.1.smithi088.stdout:1/909: sync 2024-03-24T00:49:10.664 INFO:tasks.workunit.client.1.smithi088.stdout:3/911: rename d3/d1a/d1e/d2f/d45/db5/d6e to d3/d6/d58/da4/d122 0 2024-03-24T00:49:10.664 INFO:tasks.workunit.client.1.smithi088.stdout:1/910: chown d6/d6e/dfc/d108/d5b/d5d/d61/d7d/f10b 29837 1 2024-03-24T00:49:10.665 INFO:tasks.workunit.client.1.smithi088.stdout:0/970: dwrite d9/dc/de/d19/d8d/ffb [0,4194304] 0 2024-03-24T00:49:10.666 INFO:tasks.workunit.client.1.smithi088.stdout:3/912: rmdir d3/de/d6f 39 2024-03-24T00:49:10.666 INFO:tasks.workunit.client.1.smithi088.stdout:3/913: fdatasync d3/de/d6f/d85/d90/ff7 0 2024-03-24T00:49:10.668 INFO:tasks.workunit.client.0.smithi038.stdout:6/962: dwrite d1/da/d1b/d25/d49/d72/f76 [0,4194304] 0 2024-03-24T00:49:10.668 INFO:tasks.workunit.client.0.smithi038.stdout:6/963: write d1/da/d1b/d2f/dc0/f104 [2661642,87009] 0 2024-03-24T00:49:10.669 INFO:tasks.workunit.client.0.smithi038.stdout:6/964: creat d1/d37/d84/f141 x:0 0 0 2024-03-24T00:49:10.669 INFO:tasks.workunit.client.1.smithi088.stdout:8/925: dread d9/dc/d11/d24/d43/d4a/d118/d134/d54/d5d/db3/fb2 [0,4194304] 0 2024-03-24T00:49:10.669 INFO:tasks.workunit.client.1.smithi088.stdout:2/874: write db/dd/d96/d99/db2/d88/fd6 [269023,60232] 0 2024-03-24T00:49:10.669 INFO:tasks.workunit.client.1.smithi088.stdout:2/875: fsync db/dd/d96/d99/f10c 0 2024-03-24T00:49:10.669 INFO:tasks.workunit.client.1.smithi088.stdout:1/911: write d6/d27/f1d [408121,52471] 0 2024-03-24T00:49:10.670 INFO:tasks.workunit.client.1.smithi088.stdout:1/912: dread - d6/d23/daf/f9f zero size 2024-03-24T00:49:10.674 INFO:tasks.workunit.client.1.smithi088.stdout:0/971: mknod d9/dc/de/c136 0 2024-03-24T00:49:10.675 INFO:tasks.workunit.client.1.smithi088.stdout:8/926: mkdir d9/dc/d11/d24/d29/d5b/db9/da5/d136 0 2024-03-24T00:49:10.675 INFO:tasks.workunit.client.1.smithi088.stdout:8/927: dread - d9/dc/d11/d24/d29/d36/dc1/f133 zero size 2024-03-24T00:49:10.675 INFO:tasks.workunit.client.1.smithi088.stdout:8/928: write d9/dc/d11/d24/d29/da9/fab [1247681,52588] 0 2024-03-24T00:49:10.676 INFO:tasks.workunit.client.1.smithi088.stdout:2/876: mkdir db/dd/d118 0 2024-03-24T00:49:10.677 INFO:tasks.workunit.client.1.smithi088.stdout:1/913: symlink d6/d27/l12f 0 2024-03-24T00:49:10.677 INFO:tasks.workunit.client.1.smithi088.stdout:1/914: chown d6/d23/c4e 10076231 1 2024-03-24T00:49:10.680 INFO:tasks.workunit.client.1.smithi088.stdout:8/929: symlink d9/dc/d11/d24/d29/d36/d6a/d7b/d8f/l137 0 2024-03-24T00:49:10.680 INFO:tasks.workunit.client.1.smithi088.stdout:1/915: mknod d6/c130 0 2024-03-24T00:49:10.680 INFO:tasks.workunit.client.1.smithi088.stdout:1/916: truncate d6/d6e/dfc/d108/d3f/d53/ff6 1453071 0 2024-03-24T00:49:10.682 INFO:tasks.workunit.client.1.smithi088.stdout:8/930: creat d9/dc/d11/d24/d29/d36/d6a/d7b/d8f/f138 x:0 0 0 2024-03-24T00:49:10.688 INFO:tasks.workunit.client.0.smithi038.stdout:3/926: dread d3/d9/dc/df/f31 [0,4194304] 0 2024-03-24T00:49:10.688 INFO:tasks.workunit.client.0.smithi038.stdout:3/927: creat d3/d30/d5d/f13f x:0 0 0 2024-03-24T00:49:10.689 INFO:tasks.workunit.client.0.smithi038.stdout:3/928: creat d3/d2a/d107/f140 x:0 0 0 2024-03-24T00:49:10.694 INFO:tasks.workunit.client.1.smithi088.stdout:1/917: truncate d6/f17 3036886 0 2024-03-24T00:49:10.695 INFO:tasks.workunit.client.1.smithi088.stdout:2/877: truncate db/d21/d35/f5a 3175353 0 2024-03-24T00:49:10.695 INFO:tasks.workunit.client.0.smithi038.stdout:3/929: mknod d3/d9/dc/da6/db5/d19/d82/d89/da8/c141 0 2024-03-24T00:49:10.695 INFO:tasks.workunit.client.0.smithi038.stdout:3/930: readlink d3/l36 0 2024-03-24T00:49:10.698 INFO:tasks.workunit.client.1.smithi088.stdout:1/918: link d6/d27/d67/d88/da4/ced d6/c131 0 2024-03-24T00:49:10.698 INFO:tasks.workunit.client.1.smithi088.stdout:2/878: creat db/dd/d96/f119 x:0 0 0 2024-03-24T00:49:10.702 INFO:tasks.workunit.client.1.smithi088.stdout:2/879: symlink db/dd/d96/d99/db2/ddc/l11a 0 2024-03-24T00:49:10.703 INFO:tasks.workunit.client.0.smithi038.stdout:6/965: write d1/d37/d84/df5/f121 [452642,26101] 0 2024-03-24T00:49:10.703 INFO:tasks.workunit.client.0.smithi038.stdout:6/966: readlink d1/da/d61/da2/lfb 0 2024-03-24T00:49:10.710 INFO:tasks.workunit.client.1.smithi088.stdout:2/880: truncate db/f38 1296111 0 2024-03-24T00:49:10.711 INFO:tasks.workunit.client.1.smithi088.stdout:3/914: dread f1 [0,4194304] 0 2024-03-24T00:49:10.711 INFO:tasks.workunit.client.1.smithi088.stdout:3/915: chown d3/d1a/d1e 5929 1 2024-03-24T00:49:10.712 INFO:tasks.workunit.client.0.smithi038.stdout:2/797: dwrite d4/d55/fb2 [0,4194304] 0 2024-03-24T00:49:10.718 INFO:tasks.workunit.client.0.smithi038.stdout:1/936: dwrite da/dd/d2c/d5a/f7b [0,4194304] 0 2024-03-24T00:49:10.719 INFO:tasks.workunit.client.0.smithi038.stdout:0/839: dwrite d0/d14/fcf [0,4194304] 0 2024-03-24T00:49:10.726 INFO:tasks.workunit.client.1.smithi088.stdout:1/919: dread d6/d23/f34 [0,4194304] 0 2024-03-24T00:49:10.727 INFO:tasks.workunit.client.1.smithi088.stdout:3/916: unlink d3/d67/dc3/dd7/c109 0 2024-03-24T00:49:10.728 INFO:tasks.workunit.client.1.smithi088.stdout:2/881: dread db/dd/d96/d4e/d8f/dad/fd7 [0,4194304] 0 2024-03-24T00:49:10.732 INFO:tasks.workunit.client.0.smithi038.stdout:2/798: rename d4/d27/d2f/fef to d4/d18/d5b/f113 0 2024-03-24T00:49:10.732 INFO:tasks.workunit.client.0.smithi038.stdout:1/937: getdents da/dd/d7a/d82 0 2024-03-24T00:49:10.736 INFO:tasks.workunit.client.1.smithi088.stdout:4/929: dwrite d5/d7/f44 [0,4194304] 0 2024-03-24T00:49:10.736 INFO:tasks.workunit.client.1.smithi088.stdout:4/930: fdatasync d5/f9 0 2024-03-24T00:49:10.736 INFO:tasks.workunit.client.1.smithi088.stdout:4/931: readlink d5/d16/d1a/d22/dc6/lef 0 2024-03-24T00:49:10.737 INFO:tasks.workunit.client.0.smithi038.stdout:3/931: read d3/d9/dc/d51/f7d [2506728,130260] 0 2024-03-24T00:49:10.737 INFO:tasks.workunit.client.0.smithi038.stdout:4/829: dwrite da/d12/d24/d5f/d8a/db6/f109 [0,4194304] 0 2024-03-24T00:49:10.738 INFO:tasks.workunit.client.0.smithi038.stdout:4/830: creat da/d12/d24/d68/d6c/d7c/f118 x:0 0 0 2024-03-24T00:49:10.738 INFO:tasks.workunit.client.1.smithi088.stdout:7/865: dwrite d3/d32/d34/d9c/fae [0,4194304] 0 2024-03-24T00:49:10.738 INFO:tasks.workunit.client.1.smithi088.stdout:7/866: creat d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/f110 x:0 0 0 2024-03-24T00:49:10.738 INFO:tasks.workunit.client.1.smithi088.stdout:7/867: stat d3/d32/d76/df3/d103 0 2024-03-24T00:49:10.738 INFO:tasks.workunit.client.1.smithi088.stdout:7/868: creat d3/d32/d76/f111 x:0 0 0 2024-03-24T00:49:10.738 INFO:tasks.workunit.client.1.smithi088.stdout:7/869: readlink d3/d32/d34/d9c/d4b/d73/dab/dd8/le3 0 2024-03-24T00:49:10.741 INFO:tasks.workunit.client.0.smithi038.stdout:9/960: dwrite d12/d41/d57/fa0 [0,4194304] 0 2024-03-24T00:49:10.742 INFO:tasks.workunit.client.1.smithi088.stdout:0/972: dwrite d9/d26/d8f/da1/d62/f10f [0,4194304] 0 2024-03-24T00:49:10.742 INFO:tasks.workunit.client.1.smithi088.stdout:0/973: write d9/dc/de/d19/d44/d46/d65/f35 [1311313,123395] 0 2024-03-24T00:49:10.743 INFO:tasks.workunit.client.0.smithi038.stdout:1/938: rename da/dd/d22/dab/f107 to da/dd/d2c/d5a/d101/dda/f135 0 2024-03-24T00:49:10.745 INFO:tasks.workunit.client.1.smithi088.stdout:3/917: mkdir d3/de/d6f/d85/d90/d123 0 2024-03-24T00:49:10.748 INFO:tasks.workunit.client.1.smithi088.stdout:2/882: creat db/dd/d96/d4e/d8f/d9c/f11b x:0 0 0 2024-03-24T00:49:10.749 INFO:tasks.workunit.client.1.smithi088.stdout:2/883: chown db/d21/d35/d49 3 1 2024-03-24T00:49:10.749 INFO:tasks.workunit.client.1.smithi088.stdout:2/884: write db/d21/d35/d49/f71 [2410795,73278] 0 2024-03-24T00:49:10.749 INFO:tasks.workunit.client.1.smithi088.stdout:2/885: creat db/dd/dc9/d100/f11c x:0 0 0 2024-03-24T00:49:10.759 INFO:tasks.workunit.client.0.smithi038.stdout:9/961: unlink d12/d41/d57/c5d 0 2024-03-24T00:49:10.759 INFO:tasks.workunit.client.0.smithi038.stdout:9/962: fdatasync d12/d28/d9c/ddd/d111/fce 0 2024-03-24T00:49:10.760 INFO:tasks.workunit.client.1.smithi088.stdout:7/870: dread d3/d32/d34/faa [0,4194304] 0 2024-03-24T00:49:10.765 INFO:tasks.workunit.client.1.smithi088.stdout:4/932: creat d5/d13/d18/d6e/d7d/f147 x:0 0 0 2024-03-24T00:49:10.766 INFO:tasks.workunit.client.0.smithi038.stdout:1/939: truncate da/dd/d7a/d82/fb4 319663 0 2024-03-24T00:49:10.771 INFO:tasks.workunit.client.1.smithi088.stdout:3/918: dread d3/d6/d27/d3c/f30 [0,4194304] 0 2024-03-24T00:49:10.773 INFO:tasks.workunit.client.0.smithi038.stdout:6/967: dwrite d1/da/d61/fd1 [0,4194304] 0 2024-03-24T00:49:10.777 INFO:tasks.workunit.client.0.smithi038.stdout:9/963: truncate d12/d41/f58 73653 0 2024-03-24T00:49:10.781 INFO:tasks.workunit.client.1.smithi088.stdout:1/920: write d6/d6e/dfc/d108/d5b/d5d/d61/d7d/fde [197218,97491] 0 2024-03-24T00:49:10.784 INFO:tasks.workunit.client.0.smithi038.stdout:6/968: rmdir d1/d37/d84/d13e 0 2024-03-24T00:49:10.785 INFO:tasks.workunit.client.0.smithi038.stdout:1/940: getdents da/dd/d22/dab 0 2024-03-24T00:49:10.789 INFO:tasks.workunit.client.1.smithi088.stdout:4/933: dread d5/d13/f12 [0,4194304] 0 2024-03-24T00:49:10.790 INFO:tasks.workunit.client.0.smithi038.stdout:1/941: write da/dd/f85 [654684,44584] 0 2024-03-24T00:49:10.793 INFO:tasks.workunit.client.1.smithi088.stdout:4/934: read d5/d7/de4/d107/f134 [540156,37724] 0 2024-03-24T00:49:10.798 INFO:tasks.workunit.client.1.smithi088.stdout:2/886: symlink db/dd/l11d 0 2024-03-24T00:49:10.799 INFO:tasks.workunit.client.0.smithi038.stdout:3/932: dwrite d3/d9/dc/da6/db5/f54 [0,4194304] 0 2024-03-24T00:49:10.799 INFO:tasks.workunit.client.0.smithi038.stdout:3/933: fsync d3/d9/dc/df/f1d 0 2024-03-24T00:49:10.799 INFO:tasks.workunit.client.1.smithi088.stdout:3/919: creat d3/d1a/d1e/f124 x:0 0 0 2024-03-24T00:49:10.801 INFO:tasks.workunit.client.1.smithi088.stdout:6/991: sync 2024-03-24T00:49:10.802 INFO:tasks.workunit.client.0.smithi038.stdout:9/964: creat d12/d41/d57/dd3/f136 x:0 0 0 2024-03-24T00:49:10.803 INFO:tasks.workunit.client.0.smithi038.stdout:6/969: symlink d1/d37/d84/df5/l142 0 2024-03-24T00:49:10.803 INFO:tasks.workunit.client.1.smithi088.stdout:1/921: symlink d6/d27/d67/d88/da4/l132 0 2024-03-24T00:49:10.805 INFO:tasks.workunit.client.0.smithi038.stdout:2/799: dwrite d4/d18/d26/d37/fd8 [4194304,4194304] 0 2024-03-24T00:49:10.805 INFO:tasks.workunit.client.0.smithi038.stdout:2/800: chown d4/d27/d2f/d30/f8d 4420 1 2024-03-24T00:49:10.805 INFO:tasks.workunit.client.0.smithi038.stdout:2/801: write d4/d27/d54/d67/f87 [4494047,68389] 0 2024-03-24T00:49:10.806 INFO:tasks.workunit.client.1.smithi088.stdout:4/935: rename d5/d7/d72/d106 to d5/dac/d148 0 2024-03-24T00:49:10.806 INFO:tasks.workunit.client.1.smithi088.stdout:3/920: creat d3/d6/d19/d2a/dc2/f125 x:0 0 0 2024-03-24T00:49:10.806 INFO:tasks.workunit.client.1.smithi088.stdout:3/921: truncate d3/d1a/d1e/d2f/d45/f118 65089 0 2024-03-24T00:49:10.806 INFO:tasks.workunit.client.1.smithi088.stdout:3/922: creat d3/d6/d58/f126 x:0 0 0 2024-03-24T00:49:10.809 INFO:tasks.workunit.client.1.smithi088.stdout:6/992: creat db/dd/d61/d6d/f134 x:0 0 0 2024-03-24T00:49:10.809 INFO:tasks.workunit.client.1.smithi088.stdout:6/993: chown db/dc5/dc8/d4b 26430863 1 2024-03-24T00:49:10.811 INFO:tasks.workunit.client.1.smithi088.stdout:1/922: mkdir d6/d27/db0/d133 0 2024-03-24T00:49:10.814 INFO:tasks.workunit.client.1.smithi088.stdout:4/936: mknod d5/d13/d18/d118/c149 0 2024-03-24T00:49:10.814 INFO:tasks.workunit.client.1.smithi088.stdout:9/932: sync 2024-03-24T00:49:10.818 INFO:tasks.workunit.client.0.smithi038.stdout:6/970: truncate d1/da/d1b/fa1 575298 0 2024-03-24T00:49:10.818 INFO:tasks.workunit.client.0.smithi038.stdout:6/971: write d1/d37/d84/f108 [5231187,57452] 0 2024-03-24T00:49:10.823 INFO:tasks.workunit.client.0.smithi038.stdout:1/942: dread da/dd/d4a/d76/fad [0,4194304] 0 2024-03-24T00:49:10.828 INFO:tasks.workunit.client.0.smithi038.stdout:9/965: read d12/d28/d71/d99/fc6 [142614,52620] 0 2024-03-24T00:49:10.833 INFO:tasks.workunit.client.1.smithi088.stdout:6/994: dread db/dc5/dc8/fd1 [0,4194304] 0 2024-03-24T00:49:10.834 INFO:tasks.workunit.client.1.smithi088.stdout:3/923: dread d3/d1a/d1e/f9b [0,4194304] 0 2024-03-24T00:49:10.834 INFO:tasks.workunit.client.1.smithi088.stdout:0/974: dwrite d9/dc/de0/fe4 [0,4194304] 0 2024-03-24T00:49:10.837 INFO:tasks.workunit.client.1.smithi088.stdout:0/975: creat d9/dc/de/d19/d44/de2/f137 x:0 0 0 2024-03-24T00:49:10.839 INFO:tasks.workunit.client.1.smithi088.stdout:0/976: symlink d9/d26/d8f/da1/l138 0 2024-03-24T00:49:10.840 INFO:tasks.workunit.client.1.smithi088.stdout:0/977: creat d9/dc/de/d19/d84/dd6/f139 x:0 0 0 2024-03-24T00:49:10.843 INFO:tasks.workunit.client.1.smithi088.stdout:0/978: mknod d9/d26/d8f/dc7/ddd/c13a 0 2024-03-24T00:49:10.843 INFO:tasks.workunit.client.1.smithi088.stdout:0/979: fdatasync d9/dc/de/d7e/f89 0 2024-03-24T00:49:10.843 INFO:tasks.workunit.client.1.smithi088.stdout:0/980: truncate d9/dc/de/d19/d44/f94 140 0 2024-03-24T00:49:10.844 INFO:tasks.workunit.client.1.smithi088.stdout:0/981: creat d9/d26/d8f/f13b x:0 0 0 2024-03-24T00:49:10.844 INFO:tasks.workunit.client.1.smithi088.stdout:0/982: write d9/dc/de/d19/dd2/d103/f134 [8590645,35073] 0 2024-03-24T00:49:10.844 INFO:tasks.workunit.client.1.smithi088.stdout:0/983: chown d9/fd0 0 1 2024-03-24T00:49:10.847 INFO:tasks.workunit.client.1.smithi088.stdout:0/984: link d9/dc/de/d19/d44/d46/cb8 d9/d26/d8f/da1/d62/d9b/db0/db4/c13c 0 2024-03-24T00:49:10.847 INFO:tasks.workunit.client.1.smithi088.stdout:0/985: creat d9/dc/de/d7e/d91/f13d x:0 0 0 2024-03-24T00:49:10.849 INFO:tasks.workunit.client.0.smithi038.stdout:0/840: dwrite d0/d12/f3c [0,4194304] 0 2024-03-24T00:49:10.849 INFO:tasks.workunit.client.0.smithi038.stdout:0/841: creat d0/d14/de4/f11a x:0 0 0 2024-03-24T00:49:10.849 INFO:tasks.workunit.client.0.smithi038.stdout:0/842: creat d0/d12/d15/d58/d64/da7/dbb/dcd/f11b x:0 0 0 2024-03-24T00:49:10.849 INFO:tasks.workunit.client.0.smithi038.stdout:0/843: creat d0/d12/d15/d58/f11c x:0 0 0 2024-03-24T00:49:10.849 INFO:tasks.workunit.client.0.smithi038.stdout:0/844: dread - d0/d14/d88/db8/ff2 zero size 2024-03-24T00:49:10.852 INFO:tasks.workunit.client.0.smithi038.stdout:2/802: dread d4/f24 [0,4194304] 0 2024-03-24T00:49:10.852 INFO:tasks.workunit.client.0.smithi038.stdout:6/972: dread d1/da/d1b/d25/d49/d72/fb7 [0,4194304] 0 2024-03-24T00:49:10.867 INFO:tasks.workunit.client.0.smithi038.stdout:6/973: mkdir d1/da/d1b/dfd/d143 0 2024-03-24T00:49:10.867 INFO:tasks.workunit.client.1.smithi088.stdout:8/931: sync 2024-03-24T00:49:10.870 INFO:tasks.workunit.client.1.smithi088.stdout:2/887: dwrite db/f4f [0,4194304] 0 2024-03-24T00:49:10.871 INFO:tasks.workunit.client.0.smithi038.stdout:9/966: dread d12/d28/d71/fbe [0,4194304] 0 2024-03-24T00:49:10.872 INFO:tasks.workunit.client.0.smithi038.stdout:9/967: chown d12/d28/d9c/ddd/ffa 12 1 2024-03-24T00:49:10.872 INFO:tasks.workunit.client.0.smithi038.stdout:9/968: write d12/d28/f82 [471086,96605] 0 2024-03-24T00:49:10.874 INFO:tasks.workunit.client.0.smithi038.stdout:9/969: rmdir d12/d1a/d80 39 2024-03-24T00:49:10.875 INFO:tasks.workunit.client.1.smithi088.stdout:3/924: read d3/d6/d27/d3c/f30 [2806330,16084] 0 2024-03-24T00:49:10.876 INFO:tasks.workunit.client.0.smithi038.stdout:6/974: read d1/da/f89 [4346658,56825] 0 2024-03-24T00:49:10.878 INFO:tasks.workunit.client.0.smithi038.stdout:9/970: mkdir d12/d1a/d80/d137 0 2024-03-24T00:49:10.882 INFO:tasks.workunit.client.1.smithi088.stdout:3/925: rename d3/de/d6f/d85/d49/f8e to d3/de/d6f/d85/d49/d70/d96/f127 0 2024-03-24T00:49:10.885 INFO:tasks.workunit.client.0.smithi038.stdout:4/831: dwrite da/d12/d37/dd/f34 [0,4194304] 0 2024-03-24T00:49:10.886 INFO:tasks.workunit.client.0.smithi038.stdout:6/975: rename d1/da/d1b/f2b to d1/da/d1b/dbb/f144 0 2024-03-24T00:49:10.890 INFO:tasks.workunit.client.1.smithi088.stdout:8/932: dread d9/dc/d11/d24/d29/d36/f4d [0,4194304] 0 2024-03-24T00:49:10.891 INFO:tasks.workunit.client.1.smithi088.stdout:4/937: dwrite d5/d7/d72/f104 [0,4194304] 0 2024-03-24T00:49:10.892 INFO:tasks.workunit.client.0.smithi038.stdout:0/845: write d0/d12/f89 [899788,16133] 0 2024-03-24T00:49:10.892 INFO:tasks.workunit.client.0.smithi038.stdout:0/846: creat d0/d14/d84/f11d x:0 0 0 2024-03-24T00:49:10.895 INFO:tasks.workunit.client.1.smithi088.stdout:3/926: link d3/de/d6f/d85/d90/f10d d3/d6/d27/d3c/d80/f128 0 2024-03-24T00:49:10.895 INFO:tasks.workunit.client.0.smithi038.stdout:0/847: write d0/d12/d42/d9d/f90 [3525416,95076] 0 2024-03-24T00:49:10.896 INFO:tasks.workunit.client.1.smithi088.stdout:8/933: rmdir d9/dc/d11/d24 39 2024-03-24T00:49:10.902 INFO:tasks.workunit.client.1.smithi088.stdout:0/986: dwrite d9/fd0 [0,4194304] 0 2024-03-24T00:49:10.902 INFO:tasks.workunit.client.1.smithi088.stdout:0/987: write d9/dc/de/d19/d44/d46/d65/f68 [2430768,100720] 0 2024-03-24T00:49:10.902 INFO:tasks.workunit.client.1.smithi088.stdout:0/988: dread - d9/dc/f104 zero size 2024-03-24T00:49:10.907 INFO:tasks.workunit.client.1.smithi088.stdout:2/888: write db/f4f [2017500,122055] 0 2024-03-24T00:49:10.907 INFO:tasks.workunit.client.1.smithi088.stdout:9/933: dwrite d1/d22/f43 [0,4194304] 0 2024-03-24T00:49:10.907 INFO:tasks.workunit.client.1.smithi088.stdout:3/927: rename d3/d6/d19/d2a/dc2/dd5/d10a to d3/de/d6f/d85/d49/d70/d96/d129 0 2024-03-24T00:49:10.907 INFO:tasks.workunit.client.1.smithi088.stdout:3/928: chown d3/d6/d27/l5e 563024 1 2024-03-24T00:49:10.910 INFO:tasks.workunit.client.0.smithi038.stdout:0/848: write d0/d12/d15/d58/d64/da7/dbb/dd4/dea/f8d [1152959,57917] 0 2024-03-24T00:49:10.911 INFO:tasks.workunit.client.1.smithi088.stdout:0/989: mknod d9/dc/de0/c13e 0 2024-03-24T00:49:10.911 INFO:tasks.workunit.client.0.smithi038.stdout:0/849: getdents d0/d12/d42/d5e/db0 0 2024-03-24T00:49:10.911 INFO:tasks.workunit.client.0.smithi038.stdout:3/934: dwrite d3/f8c [0,4194304] 0 2024-03-24T00:49:10.919 INFO:tasks.workunit.client.0.smithi038.stdout:1/943: dwrite da/ff4 [0,4194304] 0 2024-03-24T00:49:10.932 INFO:tasks.workunit.client.0.smithi038.stdout:9/971: dread f4 [0,4194304] 0 2024-03-24T00:49:10.932 INFO:tasks.workunit.client.0.smithi038.stdout:9/972: write d12/d28/d9c/ddd/d111/d26/fb7 [4348969,417] 0 2024-03-24T00:49:10.933 INFO:tasks.workunit.client.0.smithi038.stdout:2/803: dwrite d4/d18/d38/d63/f101 [0,4194304] 0 2024-03-24T00:49:10.937 INFO:tasks.workunit.client.1.smithi088.stdout:4/938: write d5/d13/f96 [961492,33619] 0 2024-03-24T00:49:10.937 INFO:tasks.workunit.client.1.smithi088.stdout:4/939: fdatasync d5/d13/d63/d78/fd6 0 2024-03-24T00:49:10.940 INFO:tasks.workunit.client.0.smithi038.stdout:3/935: dread d3/f5 [0,4194304] 0 2024-03-24T00:49:10.940 INFO:tasks.workunit.client.0.smithi038.stdout:3/936: chown d3/d9/dc/da6/db5/d19/f42 7363772 1 2024-03-24T00:49:10.942 INFO:tasks.workunit.client.0.smithi038.stdout:1/944: write da/dd/d22/d94/fa0 [1189843,68561] 0 2024-03-24T00:49:10.949 INFO:tasks.workunit.client.1.smithi088.stdout:2/889: dwrite db/dd/d96/d4e/d8f/d9c/f11b [0,4194304] 0 2024-03-24T00:49:10.949 INFO:tasks.workunit.client.1.smithi088.stdout:1/923: sync 2024-03-24T00:49:10.949 INFO:tasks.workunit.client.1.smithi088.stdout:6/995: sync 2024-03-24T00:49:10.950 INFO:tasks.workunit.client.1.smithi088.stdout:7/871: sync 2024-03-24T00:49:10.950 INFO:tasks.workunit.client.1.smithi088.stdout:6/996: fsync db/dc5/dc8/d4b/f6e 0 2024-03-24T00:49:10.951 INFO:tasks.workunit.client.1.smithi088.stdout:9/934: creat d1/d15/d21/df3/f12b x:0 0 0 2024-03-24T00:49:10.951 INFO:tasks.workunit.client.1.smithi088.stdout:9/935: fsync d1/d10/d77/d8a/f71 0 2024-03-24T00:49:10.954 INFO:tasks.workunit.client.1.smithi088.stdout:8/934: rename d9/dc/d11/d24/d43/d4a/dee/f121 to d9/dc/d11/d24/d29/d36/f139 0 2024-03-24T00:49:10.957 INFO:tasks.workunit.client.1.smithi088.stdout:0/990: symlink d9/d26/d8f/dc7/ddd/d133/l13f 0 2024-03-24T00:49:10.957 INFO:tasks.workunit.client.1.smithi088.stdout:0/991: stat d9/d26/d8f/da1/d62/d9b/db0/db4/fd9 0 2024-03-24T00:49:10.958 INFO:tasks.workunit.client.1.smithi088.stdout:0/992: truncate d9/dc/de/d7e/d91/f13d 207094 0 2024-03-24T00:49:10.961 INFO:tasks.workunit.client.1.smithi088.stdout:4/940: symlink d5/d7/d92/d121/l14a 0 2024-03-24T00:49:10.965 INFO:tasks.workunit.client.1.smithi088.stdout:3/929: dwrite d3/de/d6f/dcc/fd3 [0,4194304] 0 2024-03-24T00:49:10.965 INFO:tasks.workunit.client.1.smithi088.stdout:3/930: write d3/d6/d58/fb6 [899064,36445] 0 2024-03-24T00:49:10.966 INFO:tasks.workunit.client.1.smithi088.stdout:3/931: chown d3/d1a/d1e/d2f/d45/f5d 51472 1 2024-03-24T00:49:10.966 INFO:tasks.workunit.client.1.smithi088.stdout:3/932: creat d3/d1a/d1e/f12a x:0 0 0 2024-03-24T00:49:10.982 INFO:tasks.workunit.client.1.smithi088.stdout:6/997: symlink db/dd/d61/d6d/d74/l135 0 2024-03-24T00:49:10.982 INFO:tasks.workunit.client.1.smithi088.stdout:8/935: truncate d9/dc/d11/d24/d29/d36/f97 1375077 0 2024-03-24T00:49:10.983 INFO:tasks.workunit.client.1.smithi088.stdout:6/998: write db/dd/d61/fdb [822239,20965] 0 2024-03-24T00:49:10.983 INFO:tasks.workunit.client.1.smithi088.stdout:1/924: write d6/f3e [253077,67068] 0 2024-03-24T00:49:10.983 INFO:tasks.workunit.client.1.smithi088.stdout:6/999: dread - db/dd/d61/d6d/f134 zero size 2024-03-24T00:49:10.984 INFO:tasks.workunit.client.0.smithi038.stdout:2/804: symlink d4/d55/l114 0 2024-03-24T00:49:10.985 INFO:tasks.workunit.client.1.smithi088.stdout:7/872: dread d3/d32/d34/d9c/d4b/f5e [0,4194304] 0 2024-03-24T00:49:10.988 INFO:tasks.workunit.client.0.smithi038.stdout:4/832: unlink da/d38/c84 0 2024-03-24T00:49:10.990 INFO:tasks.workunit.client.1.smithi088.stdout:9/936: dwrite d1/d10/d77/fb4 [0,4194304] 0 2024-03-24T00:49:10.993 INFO:tasks.workunit.client.1.smithi088.stdout:3/933: symlink d3/d6/dca/dae/l12b 0 2024-03-24T00:49:10.997 INFO:tasks.workunit.client.1.smithi088.stdout:2/890: mkdir db/d21/d35/d11e 0 2024-03-24T00:49:10.999 INFO:tasks.workunit.client.0.smithi038.stdout:1/945: symlink da/dd/d2c/d5a/l136 0 2024-03-24T00:49:11.000 INFO:tasks.workunit.client.0.smithi038.stdout:1/946: read da/dd/d4a/f6b [263779,7597] 0 2024-03-24T00:49:11.000 INFO:tasks.workunit.client.0.smithi038.stdout:9/973: getdents d12/d28/d9c/ddd/d129 0 2024-03-24T00:49:11.002 INFO:tasks.workunit.client.0.smithi038.stdout:4/833: creat da/d12/d24/d5f/d9a/f119 x:0 0 0 2024-03-24T00:49:11.002 INFO:tasks.workunit.client.1.smithi088.stdout:7/873: write d3/ff [7066835,66578] 0 2024-03-24T00:49:11.002 INFO:tasks.workunit.client.1.smithi088.stdout:7/874: chown d3/d32/d76/f104 3723256 1 2024-03-24T00:49:11.007 INFO:tasks.workunit.client.1.smithi088.stdout:8/936: mkdir d9/dc/d11/d24/d29/d5b/d6f/d13a 0 2024-03-24T00:49:11.007 INFO:tasks.workunit.client.0.smithi038.stdout:1/947: mknod da/dd/d22/c137 0 2024-03-24T00:49:11.009 INFO:tasks.workunit.client.0.smithi038.stdout:9/974: mkdir d12/d1a/d5f/df9/d138 0 2024-03-24T00:49:11.009 INFO:tasks.workunit.client.0.smithi038.stdout:9/975: fsync d12/d1a/f20 0 2024-03-24T00:49:11.010 INFO:tasks.workunit.client.0.smithi038.stdout:4/834: rename da/d12/d24/d5f/d8a/c9f to da/d12/d24/c11a 0 2024-03-24T00:49:11.010 INFO:tasks.workunit.client.1.smithi088.stdout:9/937: getdents d1/d22/d27/d39/d98/da3/dd8 0 2024-03-24T00:49:11.018 INFO:tasks.workunit.client.0.smithi038.stdout:1/948: mkdir da/dd/d7a/d82/d98/d9e/d138 0 2024-03-24T00:49:11.018 INFO:tasks.workunit.client.0.smithi038.stdout:1/949: chown da/d3c/l3b 92084 1 2024-03-24T00:49:11.019 INFO:tasks.workunit.client.1.smithi088.stdout:3/934: creat d3/de/d6f/d85/d31/f12c x:0 0 0 2024-03-24T00:49:11.019 INFO:tasks.workunit.client.1.smithi088.stdout:2/891: symlink db/dd/dc9/d100/l11f 0 2024-03-24T00:49:11.019 INFO:tasks.workunit.client.1.smithi088.stdout:8/937: creat d9/dc/d11/d24/d29/d39/d84/deb/f13b x:0 0 0 2024-03-24T00:49:11.019 INFO:tasks.workunit.client.1.smithi088.stdout:8/938: chown d9/dc/d11/d24/d29/l69 3040274 1 2024-03-24T00:49:11.019 INFO:tasks.workunit.client.1.smithi088.stdout:8/939: read - d9/dc/d11/d24/d29/d5b/db9/da5/fcb zero size 2024-03-24T00:49:11.019 INFO:tasks.workunit.client.1.smithi088.stdout:9/938: creat d1/d10/d29/f12c x:0 0 0 2024-03-24T00:49:11.021 INFO:tasks.workunit.client.1.smithi088.stdout:0/993: dwrite d9/d26/f3c [4194304,4194304] 0 2024-03-24T00:49:11.021 INFO:tasks.workunit.client.1.smithi088.stdout:0/994: fsync d9/dc/f49 0 2024-03-24T00:49:11.021 INFO:tasks.workunit.client.1.smithi088.stdout:0/995: readlink d9/d15/l23 0 2024-03-24T00:49:11.021 INFO:tasks.workunit.client.1.smithi088.stdout:0/996: fsync d9/dc/de/fcf 0 2024-03-24T00:49:11.021 INFO:tasks.workunit.client.1.smithi088.stdout:0/997: creat d9/dc/de/d19/d44/d46/d65/f140 x:0 0 0 2024-03-24T00:49:11.021 INFO:tasks.workunit.client.1.smithi088.stdout:0/998: readlink d9/dc/lef 0 2024-03-24T00:49:11.022 INFO:tasks.workunit.client.1.smithi088.stdout:0/999: creat d9/dc/de0/f141 x:0 0 0 2024-03-24T00:49:11.031 INFO:tasks.workunit.client.1.smithi088.stdout:8/940: truncate d9/dc/d11/d24/d29/d5b/fad 3590349 0 2024-03-24T00:49:11.031 INFO:tasks.workunit.client.1.smithi088.stdout:3/935: dread d3/d6/d27/de9/fa0 [0,4194304] 0 2024-03-24T00:49:11.037 INFO:tasks.workunit.client.1.smithi088.stdout:9/939: link d1/d15/c19 d1/d10/d9b/dab/d7a/c12d 0 2024-03-24T00:49:11.037 INFO:tasks.workunit.client.1.smithi088.stdout:2/892: symlink db/dd/d96/d4e/l120 0 2024-03-24T00:49:11.048 INFO:tasks.workunit.client.1.smithi088.stdout:4/941: dwrite d5/d13/d18/d24/fa1 [0,4194304] 0 2024-03-24T00:49:11.048 INFO:tasks.workunit.client.1.smithi088.stdout:3/936: mkdir d3/de/d6f/d85/d49/d70/d96/d12d 0 2024-03-24T00:49:11.050 INFO:tasks.workunit.client.1.smithi088.stdout:8/941: rmdir d9/dc/d11/d24/d29/d39 39 2024-03-24T00:49:11.050 INFO:tasks.workunit.client.1.smithi088.stdout:9/940: symlink d1/d10/d29/def/l12e 0 2024-03-24T00:49:11.052 INFO:tasks.workunit.client.1.smithi088.stdout:2/893: creat db/dd/f121 x:0 0 0 2024-03-24T00:49:11.054 INFO:tasks.workunit.client.0.smithi038.stdout:3/937: dwrite d3/d9/dc/df/fdd [0,4194304] 0 2024-03-24T00:49:11.057 INFO:tasks.workunit.client.0.smithi038.stdout:4/835: dread da/d12/d24/d5f/d8a/fe0 [0,4194304] 0 2024-03-24T00:49:11.057 INFO:tasks.workunit.client.0.smithi038.stdout:4/836: stat da/d12/d29/d5b/d73/lbc 0 2024-03-24T00:49:11.057 INFO:tasks.workunit.client.0.smithi038.stdout:4/837: chown da/d12/d37/f99 560766606 1 2024-03-24T00:49:11.058 INFO:tasks.workunit.client.1.smithi088.stdout:4/942: rename d5/d16/d1a/l126 to d5/d7/d15/d74/de2/l14b 0 2024-03-24T00:49:11.060 INFO:tasks.workunit.client.0.smithi038.stdout:1/950: write da/dd/d2c/d5a/d74/f75 [2158419,64771] 0 2024-03-24T00:49:11.060 INFO:tasks.workunit.client.1.smithi088.stdout:8/942: creat d9/dc/d11/d24/d29/d5b/db9/da5/f13c x:0 0 0 2024-03-24T00:49:11.062 INFO:tasks.workunit.client.1.smithi088.stdout:9/941: link d1/d22/d27/d39/f58 d1/d15/d121/f12f 0 2024-03-24T00:49:11.062 INFO:tasks.workunit.client.1.smithi088.stdout:9/942: readlink d1/d22/d27/d39/d98/da3/dd8/le3 0 2024-03-24T00:49:11.063 INFO:tasks.workunit.client.1.smithi088.stdout:1/925: dwrite d6/d6e/dfc/d108/d5b/d5d/d61/d7d/fde [0,4194304] 0 2024-03-24T00:49:11.064 INFO:tasks.workunit.client.0.smithi038.stdout:9/976: dwrite d12/d28/d9c/ddd/d111/f4e [0,4194304] 0 2024-03-24T00:49:11.064 INFO:tasks.workunit.client.0.smithi038.stdout:9/977: write d12/d28/d9c/ddd/d111/f4b [1165541,95534] 0 2024-03-24T00:49:11.064 INFO:tasks.workunit.client.1.smithi088.stdout:7/875: dwrite d3/d32/d34/d9c/d4b/d73/fe8 [0,4194304] 0 2024-03-24T00:49:11.065 INFO:tasks.workunit.client.0.smithi038.stdout:2/805: dwrite d4/d18/d26/d37/d69/d83/fd3 [0,4194304] 0 2024-03-24T00:49:11.065 INFO:tasks.workunit.client.0.smithi038.stdout:2/806: write d4/d18/d26/f6d [2123253,16525] 0 2024-03-24T00:49:11.065 INFO:tasks.workunit.client.1.smithi088.stdout:2/894: truncate db/dd/f1f 3344538 0 2024-03-24T00:49:11.067 INFO:tasks.workunit.client.0.smithi038.stdout:3/938: rmdir d3/d30/ded 0 2024-03-24T00:49:11.067 INFO:tasks.workunit.client.0.smithi038.stdout:3/939: dread - d3/d2a/f10c zero size 2024-03-24T00:49:11.067 INFO:tasks.workunit.client.0.smithi038.stdout:3/940: readlink d3/d30/l4a 0 2024-03-24T00:49:11.067 INFO:tasks.workunit.client.0.smithi038.stdout:3/941: dread - d3/d2a/f10c zero size 2024-03-24T00:49:11.070 INFO:tasks.workunit.client.0.smithi038.stdout:4/838: creat da/d12/d37/dd/d15/df1/f11b x:0 0 0 2024-03-24T00:49:11.072 INFO:tasks.workunit.client.1.smithi088.stdout:3/937: rename d3/d1a/d1e/f124 to d3/d6/d19/d2a/dc2/dd5/f12e 0 2024-03-24T00:49:11.072 INFO:tasks.workunit.client.1.smithi088.stdout:3/938: stat d3/d67/dc3 0 2024-03-24T00:49:11.074 INFO:tasks.workunit.client.0.smithi038.stdout:1/951: creat da/dd/d22/dab/f139 x:0 0 0 2024-03-24T00:49:11.076 INFO:tasks.workunit.client.1.smithi088.stdout:4/943: mkdir d5/d13/d18/d24/db3/d14c 0 2024-03-24T00:49:11.078 INFO:tasks.workunit.client.1.smithi088.stdout:8/943: creat d9/dc/d11/d24/d29/d39/f13d x:0 0 0 2024-03-24T00:49:11.078 INFO:tasks.workunit.client.1.smithi088.stdout:9/943: dread d1/d10/d29/db0/fc9 [0,4194304] 0 2024-03-24T00:49:11.086 INFO:tasks.workunit.client.0.smithi038.stdout:9/978: symlink d12/d28/d9c/ddd/d111/d26/l139 0 2024-03-24T00:49:11.086 INFO:tasks.workunit.client.0.smithi038.stdout:9/979: write d12/d28/d9c/ddd/d111/fe0 [979584,80733] 0 2024-03-24T00:49:11.086 INFO:tasks.workunit.client.0.smithi038.stdout:1/952: dread da/dd/d7a/d82/d98/d34/f39 [0,4194304] 0 2024-03-24T00:49:11.086 INFO:tasks.workunit.client.0.smithi038.stdout:1/953: dread - da/dd/d22/d94/fce zero size 2024-03-24T00:49:11.087 INFO:tasks.workunit.client.1.smithi088.stdout:1/926: link d6/d23/l49 d6/d6e/dfc/d108/d3f/l134 0 2024-03-24T00:49:11.088 INFO:tasks.workunit.client.1.smithi088.stdout:1/927: dread - d6/dc2/fc4 zero size 2024-03-24T00:49:11.088 INFO:tasks.workunit.client.1.smithi088.stdout:1/928: fsync f2 0 2024-03-24T00:49:11.088 INFO:tasks.workunit.client.1.smithi088.stdout:1/929: dread - d6/d6e/dfc/d108/d5b/d93/f106 zero size 2024-03-24T00:49:11.088 INFO:tasks.workunit.client.0.smithi038.stdout:4/839: write da/d12/d37/dd/f34 [3569963,108857] 0 2024-03-24T00:49:11.103 INFO:tasks.workunit.client.0.smithi038.stdout:9/980: mknod d12/d28/d9c/ddd/d111/d8f/d10c/c13a 0 2024-03-24T00:49:11.109 INFO:tasks.workunit.client.1.smithi088.stdout:4/944: mkdir d5/d16/d37/dff/d14d 0 2024-03-24T00:49:11.109 INFO:tasks.workunit.client.1.smithi088.stdout:4/945: chown d5/d7/dd/c84 18 1 2024-03-24T00:49:11.114 INFO:tasks.workunit.client.1.smithi088.stdout:4/946: write d5/d16/d94/da6/fc7 [174888,33719] 0 2024-03-24T00:49:11.114 INFO:tasks.workunit.client.1.smithi088.stdout:4/947: write d5/d16/d37/f48 [2442155,14684] 0 2024-03-24T00:49:11.120 INFO:tasks.workunit.client.0.smithi038.stdout:3/942: rmdir d3/d9/dc/df 39 2024-03-24T00:49:11.120 INFO:tasks.workunit.client.0.smithi038.stdout:3/943: stat d3/d30/d5d/deb/f108 0 2024-03-24T00:49:11.123 INFO:tasks.workunit.client.0.smithi038.stdout:9/981: rmdir d12/d41/d57/dd3/dfe 39 2024-03-24T00:49:11.123 INFO:tasks.workunit.client.0.smithi038.stdout:9/982: fdatasync d12/d28/d9c/ddd/d111/f4e 0 2024-03-24T00:49:11.125 INFO:tasks.workunit.client.0.smithi038.stdout:4/840: rename da/d12/d24/d5f/fda to da/d12/d37/dd/f11c 0 2024-03-24T00:49:11.127 INFO:tasks.workunit.client.1.smithi088.stdout:9/944: chown d1/d10/d77/d8a/c24 89738 1 2024-03-24T00:49:11.127 INFO:tasks.workunit.client.1.smithi088.stdout:8/944: dwrite d9/dc/d11/d24/d43/d4a/d118/d134/d54/f9a [0,4194304] 0 2024-03-24T00:49:11.127 INFO:tasks.workunit.client.1.smithi088.stdout:8/945: stat d9/dc/d11/dd2/f11e 0 2024-03-24T00:49:11.127 INFO:tasks.workunit.client.1.smithi088.stdout:3/939: dwrite d3/de/d6f/f75 [4194304,4194304] 0 2024-03-24T00:49:11.127 INFO:tasks.workunit.client.1.smithi088.stdout:1/930: dwrite d6/fb4 [0,4194304] 0 2024-03-24T00:49:11.128 INFO:tasks.workunit.client.1.smithi088.stdout:1/931: readlink d6/d23/l72 0 2024-03-24T00:49:11.128 INFO:tasks.workunit.client.1.smithi088.stdout:1/932: write d6/f10 [1779436,49259] 0 2024-03-24T00:49:11.128 INFO:tasks.workunit.client.0.smithi038.stdout:7/852: sync 2024-03-24T00:49:11.129 INFO:tasks.workunit.client.0.smithi038.stdout:9/983: mknod d12/d28/d9c/ddd/c13b 0 2024-03-24T00:49:11.132 INFO:tasks.workunit.client.1.smithi088.stdout:4/948: creat d5/d13/d63/d78/f14e x:0 0 0 2024-03-24T00:49:11.132 INFO:tasks.workunit.client.1.smithi088.stdout:4/949: read d5/d13/d18/d24/d3d/df0/ffc [936843,119027] 0 2024-03-24T00:49:11.132 INFO:tasks.workunit.client.1.smithi088.stdout:4/950: chown d5/d13/d63/d78/f11e 9 1 2024-03-24T00:49:11.134 INFO:tasks.workunit.client.0.smithi038.stdout:4/841: mknod da/d12/d37/d83/c11d 0 2024-03-24T00:49:11.136 INFO:tasks.workunit.client.1.smithi088.stdout:9/945: creat d1/d10/d29/db0/f130 x:0 0 0 2024-03-24T00:49:11.136 INFO:tasks.workunit.client.1.smithi088.stdout:9/946: read d1/f20 [386250,118194] 0 2024-03-24T00:49:11.136 INFO:tasks.workunit.client.1.smithi088.stdout:9/947: write d1/d10/d29/fe2 [1020971,11043] 0 2024-03-24T00:49:11.136 INFO:tasks.workunit.client.1.smithi088.stdout:9/948: truncate d1/d10/d9b/dab/db8/da7/f10d 739445 0 2024-03-24T00:49:11.136 INFO:tasks.workunit.client.1.smithi088.stdout:9/949: dread - d1/d22/d27/f83 zero size 2024-03-24T00:49:11.137 INFO:tasks.workunit.client.0.smithi038.stdout:3/944: getdents d3/d30/d5d/d6a 0 2024-03-24T00:49:11.142 INFO:tasks.workunit.client.0.smithi038.stdout:3/945: symlink d3/d9/dc/da6/db5/d19/d82/d89/l142 0 2024-03-24T00:49:11.144 INFO:tasks.workunit.client.1.smithi088.stdout:3/940: getdents d3/de/d6f/dcc 0 2024-03-24T00:49:11.146 INFO:tasks.workunit.client.1.smithi088.stdout:8/946: dread d9/dc/d11/d24/f2a [0,4194304] 0 2024-03-24T00:49:11.146 INFO:tasks.workunit.client.1.smithi088.stdout:9/950: creat d1/db5/d118/f131 x:0 0 0 2024-03-24T00:49:11.146 INFO:tasks.workunit.client.1.smithi088.stdout:9/951: dread - d1/d22/d27/f128 zero size 2024-03-24T00:49:11.148 INFO:tasks.workunit.client.0.smithi038.stdout:3/946: symlink d3/d30/d70/d90/d99/l143 0 2024-03-24T00:49:11.149 INFO:tasks.workunit.client.1.smithi088.stdout:1/933: dread d6/d27/f8e [0,4194304] 0 2024-03-24T00:49:11.155 INFO:tasks.workunit.client.1.smithi088.stdout:8/947: rename d9/dc/d11/d24/d43/d4a/c5f to d9/dc/d11/d24/d43/c13e 0 2024-03-24T00:49:11.156 INFO:tasks.workunit.client.1.smithi088.stdout:4/951: dread d5/d7/dd/d10b/f11a [0,4194304] 0 2024-03-24T00:49:11.159 INFO:tasks.workunit.client.1.smithi088.stdout:4/952: creat d5/d13/d18/d118/f14f x:0 0 0 2024-03-24T00:49:11.159 INFO:tasks.workunit.client.1.smithi088.stdout:4/953: write d5/d16/d94/fd4 [1369805,14827] 0 2024-03-24T00:49:11.162 INFO:tasks.workunit.client.0.smithi038.stdout:4/842: dread da/d12/d37/dd/d15/fb3 [0,4194304] 0 2024-03-24T00:49:11.164 INFO:tasks.workunit.client.1.smithi088.stdout:8/948: rmdir d9/dc/d11/d24/d29/d39 39 2024-03-24T00:49:11.166 INFO:tasks.workunit.client.0.smithi038.stdout:3/947: write d3/d9/dc/df/f1d [1219259,59457] 0 2024-03-24T00:49:11.166 INFO:tasks.workunit.client.0.smithi038.stdout:3/948: chown d3/f29 8006 1 2024-03-24T00:49:11.166 INFO:tasks.workunit.client.0.smithi038.stdout:1/954: dwrite da/f35 [0,4194304] 0 2024-03-24T00:49:11.168 INFO:tasks.workunit.client.1.smithi088.stdout:3/941: dread d3/d6/dca/dae/f50 [0,4194304] 0 2024-03-24T00:49:11.169 INFO:tasks.workunit.client.1.smithi088.stdout:8/949: creat d9/dc/d11/d24/d29/d36/d6a/f13f x:0 0 0 2024-03-24T00:49:11.169 INFO:tasks.workunit.client.1.smithi088.stdout:8/950: getdents d9/dc/d11/d24/d43/d4a/d118/d134/d54/d5d 0 2024-03-24T00:49:11.172 INFO:tasks.workunit.client.0.smithi038.stdout:3/949: rename d3/d30/d70/d90/ce4 to d3/d2a/c144 0 2024-03-24T00:49:11.177 INFO:tasks.workunit.client.1.smithi088.stdout:3/942: rename d3/d1a/d1e/d2f/d45/f120 to d3/de/d6f/dcc/f12f 0 2024-03-24T00:49:11.181 INFO:tasks.workunit.client.0.smithi038.stdout:4/843: symlink da/d38/l11e 0 2024-03-24T00:49:11.181 INFO:tasks.workunit.client.0.smithi038.stdout:4/844: stat da/d12/d29/l47 0 2024-03-24T00:49:11.182 INFO:tasks.workunit.client.0.smithi038.stdout:3/950: creat d3/d9/dc/da6/f145 x:0 0 0 2024-03-24T00:49:11.184 INFO:tasks.workunit.client.1.smithi088.stdout:8/951: rename d9/dc/d11/d24/d29/d73 to d9/dc/d11/d24/d43/d4a/d140 0 2024-03-24T00:49:11.188 INFO:tasks.workunit.client.0.smithi038.stdout:1/955: rename da/dd/d7a/d82/d98/d14/c123 to da/dd/d4a/c13a 0 2024-03-24T00:49:11.188 INFO:tasks.workunit.client.0.smithi038.stdout:1/956: readlink da/d3c/d86/lde 0 2024-03-24T00:49:11.193 INFO:tasks.workunit.client.0.smithi038.stdout:3/951: mknod d3/d9/dc/da6/db5/d19/d82/d89/da8/c146 0 2024-03-24T00:49:11.194 INFO:tasks.workunit.client.0.smithi038.stdout:1/957: creat da/dd/d7a/d82/d98/d14/f13b x:0 0 0 2024-03-24T00:49:11.198 INFO:tasks.workunit.client.1.smithi088.stdout:9/952: rmdir d1/d10/d29/db0 39 2024-03-24T00:49:11.203 INFO:tasks.workunit.client.0.smithi038.stdout:4/845: rename da/d12/laf to da/d12/d24/d5f/d8a/d117/l11f 0 2024-03-24T00:49:11.208 INFO:tasks.workunit.client.1.smithi088.stdout:9/953: mknod d1/d10/d9b/dab/c132 0 2024-03-24T00:49:11.211 INFO:tasks.workunit.client.0.smithi038.stdout:1/958: truncate da/f56 730141 0 2024-03-24T00:49:11.211 INFO:tasks.workunit.client.0.smithi038.stdout:1/959: chown da/dd/d2c/d5a/l9f 61702 1 2024-03-24T00:49:11.211 INFO:tasks.workunit.client.0.smithi038.stdout:1/960: fdatasync da/dd/d22/dab/ff5 0 2024-03-24T00:49:11.212 INFO:tasks.workunit.client.0.smithi038.stdout:1/961: chown da/dd/d2c/d41/d60/fc5 1129 1 2024-03-24T00:49:11.212 INFO:tasks.workunit.client.0.smithi038.stdout:1/962: truncate da/dd/d4a/f6b 939312 0 2024-03-24T00:49:11.212 INFO:tasks.workunit.client.0.smithi038.stdout:1/963: dread - da/d3c/d86/fb3 zero size 2024-03-24T00:49:11.214 INFO:tasks.workunit.client.1.smithi088.stdout:1/934: dwrite d6/d6e/dfc/d108/d3f/fc9 [0,4194304] 0 2024-03-24T00:49:11.217 INFO:tasks.workunit.client.1.smithi088.stdout:9/954: mknod d1/d10/d2c/d122/c133 0 2024-03-24T00:49:11.221 INFO:tasks.workunit.client.0.smithi038.stdout:4/846: creat da/d12/d29/d5b/d73/db4/f120 x:0 0 0 2024-03-24T00:49:11.227 INFO:tasks.workunit.client.0.smithi038.stdout:3/952: write d3/d9/dc/da6/fab [2702,67717] 0 2024-03-24T00:49:11.227 INFO:tasks.workunit.client.0.smithi038.stdout:3/953: dread - d3/d9/dc/da6/db5/ff7 zero size 2024-03-24T00:49:11.229 INFO:tasks.workunit.client.0.smithi038.stdout:1/964: rmdir da/dd/d2c/d40/de9/d117 0 2024-03-24T00:49:11.229 INFO:tasks.workunit.client.1.smithi088.stdout:4/954: dwrite d5/d7/d15/d74/d114/f129 [0,4194304] 0 2024-03-24T00:49:11.230 INFO:tasks.workunit.client.0.smithi038.stdout:9/984: dwrite d12/d81/fa5 [0,4194304] 0 2024-03-24T00:49:11.230 INFO:tasks.workunit.client.0.smithi038.stdout:4/847: mkdir da/d12/d37/dd/d15/d6d/d8f/df0/d121 0 2024-03-24T00:49:11.232 INFO:tasks.workunit.client.0.smithi038.stdout:3/954: mknod d3/d5a/c147 0 2024-03-24T00:49:11.232 INFO:tasks.workunit.client.0.smithi038.stdout:3/955: dread - d3/d9/dc/da6/db5/d19/d82/d89/da8/fb2 zero size 2024-03-24T00:49:11.233 INFO:tasks.workunit.client.0.smithi038.stdout:1/965: mknod da/dd/d2c/d5a/d101/de8/d108/dfb/c13c 0 2024-03-24T00:49:11.234 INFO:tasks.workunit.client.1.smithi088.stdout:3/943: dwrite d3/d6/d58/f126 [0,4194304] 0 2024-03-24T00:49:11.235 INFO:tasks.workunit.client.0.smithi038.stdout:9/985: mknod d12/d81/d84/c13c 0 2024-03-24T00:49:11.235 INFO:tasks.workunit.client.0.smithi038.stdout:9/986: write d12/d41/d57/dd3/dfe/f92 [1223201,63340] 0 2024-03-24T00:49:11.236 INFO:tasks.workunit.client.0.smithi038.stdout:4/848: stat da/d12/d29/d5b/d73/cfa 0 2024-03-24T00:49:11.237 INFO:tasks.workunit.client.0.smithi038.stdout:7/853: dwrite d2/d9/fa7 [0,4194304] 0 2024-03-24T00:49:11.237 INFO:tasks.workunit.client.1.smithi088.stdout:1/935: dread d6/d6e/dfc/d108/d5b/d5d/d61/d7d/fad [0,4194304] 0 2024-03-24T00:49:11.237 INFO:tasks.workunit.client.1.smithi088.stdout:1/936: write d6/d6e/dfc/d108/d5b/f11c [488162,6390] 0 2024-03-24T00:49:11.237 INFO:tasks.workunit.client.1.smithi088.stdout:9/955: unlink d1/d10/d9b/dab/db8/da7/cc1 0 2024-03-24T00:49:11.237 INFO:tasks.workunit.client.1.smithi088.stdout:8/952: dwrite d9/dc/d11/f104 [0,4194304] 0 2024-03-24T00:49:11.247 INFO:tasks.workunit.client.1.smithi088.stdout:4/955: rename d5/d17/f11d to d5/d16/d37/dff/f150 0 2024-03-24T00:49:11.250 INFO:tasks.workunit.client.0.smithi038.stdout:1/966: write da/dd/d7a/d82/d98/d34/fa5 [139250,120497] 0 2024-03-24T00:49:11.258 INFO:tasks.workunit.client.0.smithi038.stdout:9/987: rename d12/d41/c46 to d12/d28/d6c/dbc/dd7/dda/c13d 0 2024-03-24T00:49:11.266 INFO:tasks.workunit.client.0.smithi038.stdout:1/967: dread da/dd/d22/dab/fbc [0,4194304] 0 2024-03-24T00:49:11.266 INFO:tasks.workunit.client.0.smithi038.stdout:1/968: stat da/dd/d7a/d82/ddb/f10d 0 2024-03-24T00:49:11.267 INFO:tasks.workunit.client.1.smithi088.stdout:1/937: mkdir d6/d27/db0/de3/d135 0 2024-03-24T00:49:11.268 INFO:tasks.workunit.client.1.smithi088.stdout:9/956: mknod d1/d15/d80/da5/c134 0 2024-03-24T00:49:11.271 INFO:tasks.workunit.client.0.smithi038.stdout:4/849: rmdir da/d12/d37/dd/d15/df1 39 2024-03-24T00:49:11.273 INFO:tasks.workunit.client.1.smithi088.stdout:8/953: dwrite d9/dc/d11/d24/d29/d36/dc1/dc4/fe2 [0,4194304] 0 2024-03-24T00:49:11.273 INFO:tasks.workunit.client.1.smithi088.stdout:4/956: symlink d5/d7/dd/d88/l151 0 2024-03-24T00:49:11.276 INFO:tasks.workunit.client.0.smithi038.stdout:9/988: creat d12/d28/f13e x:0 0 0 2024-03-24T00:49:11.279 INFO:tasks.workunit.client.1.smithi088.stdout:3/944: getdents d3/d6/d58 0 2024-03-24T00:49:11.279 INFO:tasks.workunit.client.1.smithi088.stdout:3/945: write d3/de/d6f/dcc/ff0 [727942,22441] 0 2024-03-24T00:49:11.279 INFO:tasks.workunit.client.1.smithi088.stdout:3/946: fsync d3/de/d6f/d85/d31/f7e 0 2024-03-24T00:49:11.279 INFO:tasks.workunit.client.1.smithi088.stdout:3/947: chown d3/d1a/d1e/d2f/d45 4452607 1 2024-03-24T00:49:11.279 INFO:tasks.workunit.client.1.smithi088.stdout:3/948: write d3/d6/d27/d3c/f30 [3553712,95690] 0 2024-03-24T00:49:11.281 INFO:tasks.workunit.client.0.smithi038.stdout:1/969: creat da/d3c/d86/def/f13d x:0 0 0 2024-03-24T00:49:11.286 INFO:tasks.workunit.client.0.smithi038.stdout:4/850: link da/d12/d24/d68/f93 da/d12/d24/d68/f122 0 2024-03-24T00:49:11.286 INFO:tasks.workunit.client.0.smithi038.stdout:3/956: dwrite d3/d30/d70/d90/d99/f118 [0,4194304] 0 2024-03-24T00:49:11.287 INFO:tasks.workunit.client.1.smithi088.stdout:1/938: mkdir d6/d27/d136 0 2024-03-24T00:49:11.287 INFO:tasks.workunit.client.1.smithi088.stdout:1/939: write d6/f3e [1465895,43090] 0 2024-03-24T00:49:11.287 INFO:tasks.workunit.client.1.smithi088.stdout:8/954: creat d9/dc/d11/d24/d29/d36/d6a/d7b/f141 x:0 0 0 2024-03-24T00:49:11.288 INFO:tasks.workunit.client.1.smithi088.stdout:1/940: truncate d6/d6e/dfc/d108/d5b/d5d/d61/d7d/fd3 17595 0 2024-03-24T00:49:11.288 INFO:tasks.workunit.client.0.smithi038.stdout:9/989: truncate d12/d28/da8/fae 292054 0 2024-03-24T00:49:11.289 INFO:tasks.workunit.client.0.smithi038.stdout:1/970: rmdir da/dd/d7a/d82/d98/d14/dbb 39 2024-03-24T00:49:11.289 INFO:tasks.workunit.client.1.smithi088.stdout:7/876: sync 2024-03-24T00:49:11.289 INFO:tasks.workunit.client.1.smithi088.stdout:2/895: sync 2024-03-24T00:49:11.290 INFO:tasks.workunit.client.1.smithi088.stdout:8/955: rename d9/dc/d11/d24/fca to d9/dc/d11/d24/d29/da9/f142 0 2024-03-24T00:49:11.294 INFO:tasks.workunit.client.1.smithi088.stdout:1/941: mkdir d6/d6e/dfc/d108/d5b/d5d/d61/d7d/d137 0 2024-03-24T00:49:11.294 INFO:tasks.workunit.client.1.smithi088.stdout:1/942: chown d6/d6e/dfc/d108/d5b/d5e/ce8 16 1 2024-03-24T00:49:11.295 INFO:tasks.workunit.client.0.smithi038.stdout:9/990: rmdir d12/d55/d56 39 2024-03-24T00:49:11.295 INFO:tasks.workunit.client.0.smithi038.stdout:9/991: fdatasync d12/d41/d57/f109 0 2024-03-24T00:49:11.295 INFO:tasks.workunit.client.0.smithi038.stdout:9/992: chown d12/d55/d56/la4 0 1 2024-03-24T00:49:11.299 INFO:tasks.workunit.client.0.smithi038.stdout:1/971: mkdir da/dd/d7a/d82/d98/d14/d11d/d13e 0 2024-03-24T00:49:11.301 INFO:tasks.workunit.client.1.smithi088.stdout:3/949: dread d3/d6/d27/d3c/f59 [0,4194304] 0 2024-03-24T00:49:11.301 INFO:tasks.workunit.client.1.smithi088.stdout:2/896: rename db/dd/d118 to db/dd4/d122 0 2024-03-24T00:49:11.301 INFO:tasks.workunit.client.1.smithi088.stdout:3/950: creat d3/d6/d58/da4/f130 x:0 0 0 2024-03-24T00:49:11.302 INFO:tasks.workunit.client.1.smithi088.stdout:1/943: link d6/d6e/dfc/d108/d5b/d80/l10c d6/d27/d136/l138 0 2024-03-24T00:49:11.304 INFO:tasks.workunit.client.0.smithi038.stdout:9/993: mknod d12/d1a/d5f/c13f 0 2024-03-24T00:49:11.309 INFO:tasks.workunit.client.1.smithi088.stdout:2/897: mkdir db/dd/d96/d99/db2/d123 0 2024-03-24T00:49:11.310 INFO:tasks.workunit.client.0.smithi038.stdout:9/994: chown d12/d41/d57/l76 16 1 2024-03-24T00:49:11.315 INFO:tasks.workunit.client.1.smithi088.stdout:7/877: rename d3/d32/d34/d9c/d4b/d74/d79/fda to d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/dd6/f112 0 2024-03-24T00:49:11.315 INFO:tasks.workunit.client.1.smithi088.stdout:3/951: unlink d3/d1a/d1e/d2f/ce5 0 2024-03-24T00:49:11.317 INFO:tasks.workunit.client.1.smithi088.stdout:1/944: truncate d6/d6e/dfc/d108/fdd 159989 0 2024-03-24T00:49:11.317 INFO:tasks.workunit.client.1.smithi088.stdout:1/945: write d6/d23/f29 [5887,69318] 0 2024-03-24T00:49:11.317 INFO:tasks.workunit.client.1.smithi088.stdout:1/946: fdatasync d6/d6e/dfc/d108/d5b/d5d/d61/ff1 0 2024-03-24T00:49:11.318 INFO:tasks.workunit.client.0.smithi038.stdout:3/957: dread d3/d30/d70/d90/d99/f118 [0,4194304] 0 2024-03-24T00:49:11.318 INFO:tasks.workunit.client.0.smithi038.stdout:3/958: stat d3/d30/d5d/deb/c11e 0 2024-03-24T00:49:11.320 INFO:tasks.workunit.client.1.smithi088.stdout:7/878: mknod d3/d32/d34/d9c/d4b/d73/dab/dc3/c113 0 2024-03-24T00:49:11.321 INFO:tasks.workunit.client.1.smithi088.stdout:3/952: mkdir d3/de/d6f/d85/d49/d131 0 2024-03-24T00:49:11.329 INFO:tasks.workunit.client.1.smithi088.stdout:4/957: dwrite d5/d16/d1a/f50 [0,4194304] 0 2024-03-24T00:49:11.330 INFO:tasks.workunit.client.1.smithi088.stdout:1/947: creat d6/d6e/dfc/d108/d5b/f139 x:0 0 0 2024-03-24T00:49:11.331 INFO:tasks.workunit.client.1.smithi088.stdout:1/948: readlink d6/d6e/dfc/d108/d5b/d80/l10c 0 2024-03-24T00:49:11.332 INFO:tasks.workunit.client.1.smithi088.stdout:7/879: rmdir d3/d32/d34 39 2024-03-24T00:49:11.340 INFO:tasks.workunit.client.1.smithi088.stdout:2/898: write db/d21/d35/f7d [3998651,62194] 0 2024-03-24T00:49:11.344 INFO:tasks.workunit.client.0.smithi038.stdout:3/959: read d3/d9/dc/da6/db5/d19/d82/d89/da8/f100 [886674,22714] 0 2024-03-24T00:49:11.347 INFO:tasks.workunit.client.1.smithi088.stdout:9/957: dwrite d1/f106 [0,4194304] 0 2024-03-24T00:49:11.347 INFO:tasks.workunit.client.1.smithi088.stdout:9/958: readlink d1/d10/d9b/dab/db8/l4b 0 2024-03-24T00:49:11.349 INFO:tasks.workunit.client.1.smithi088.stdout:4/958: rename d5/d16/d1a/f3e to d5/d13/d18/d24/d3d/f152 0 2024-03-24T00:49:11.349 INFO:tasks.workunit.client.1.smithi088.stdout:4/959: readlink d5/d13/d63/d78/lc9 0 2024-03-24T00:49:11.349 INFO:tasks.workunit.client.1.smithi088.stdout:4/960: creat d5/d13/d18/d24/d3d/f153 x:0 0 0 2024-03-24T00:49:11.352 INFO:tasks.workunit.client.0.smithi038.stdout:4/851: dwrite da/d12/d37/dd/d15/d4c/f6a [4194304,4194304] 0 2024-03-24T00:49:11.352 INFO:tasks.workunit.client.1.smithi088.stdout:7/880: getdents d3/d32/d34 0 2024-03-24T00:49:11.352 INFO:tasks.workunit.client.1.smithi088.stdout:7/881: dread - d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/dd6/f112 zero size 2024-03-24T00:49:11.353 INFO:tasks.workunit.client.0.smithi038.stdout:5/852: sync 2024-03-24T00:49:11.354 INFO:tasks.workunit.client.0.smithi038.stdout:6/976: sync 2024-03-24T00:49:11.354 INFO:tasks.workunit.client.0.smithi038.stdout:8/909: sync 2024-03-24T00:49:11.356 INFO:tasks.workunit.client.1.smithi088.stdout:1/949: dread d6/f68 [0,4194304] 0 2024-03-24T00:49:11.356 INFO:tasks.workunit.client.1.smithi088.stdout:1/950: chown d6/d6e/dfc/d108/d5b/d5d/d61/d7d/dc8/de9/lfb 1 1 2024-03-24T00:49:11.357 INFO:tasks.workunit.client.0.smithi038.stdout:0/850: sync 2024-03-24T00:49:11.359 INFO:tasks.workunit.client.0.smithi038.stdout:0/851: write d0/d14/d88/db8/f116 [438531,27924] 0 2024-03-24T00:49:11.359 INFO:tasks.workunit.client.0.smithi038.stdout:2/807: sync 2024-03-24T00:49:11.361 INFO:tasks.workunit.client.1.smithi088.stdout:4/961: truncate d5/d13/d18/f19 1551534 0 2024-03-24T00:49:11.363 INFO:tasks.workunit.client.0.smithi038.stdout:2/808: creat d4/d18/d38/d63/f115 x:0 0 0 2024-03-24T00:49:11.363 INFO:tasks.workunit.client.0.smithi038.stdout:2/809: read - d4/d27/ddb/d9b/fb8 zero size 2024-03-24T00:49:11.363 INFO:tasks.workunit.client.0.smithi038.stdout:1/972: dread da/dd/d22/f55 [0,4194304] 0 2024-03-24T00:49:11.365 INFO:tasks.workunit.client.1.smithi088.stdout:7/882: link d3/d32/d34/d7c/ca9 d3/d32/d34/d9c/d4b/d73/dab/dc3/c114 0 2024-03-24T00:49:11.367 INFO:tasks.workunit.client.0.smithi038.stdout:3/960: write d3/d9/dc/df/f31 [1820115,117638] 0 2024-03-24T00:49:11.374 INFO:tasks.workunit.client.1.smithi088.stdout:4/962: dread d5/d17/f81 [0,4194304] 0 2024-03-24T00:49:11.397 INFO:tasks.workunit.client.0.smithi038.stdout:4/852: write da/d12/d37/dd/d15/d6d/f88 [2265828,115749] 0 2024-03-24T00:49:11.398 INFO:tasks.workunit.client.0.smithi038.stdout:4/853: chown da/d12/d24/d68/d6c/da6 672947230 1 2024-03-24T00:49:11.399 INFO:tasks.workunit.client.0.smithi038.stdout:9/995: dwrite d12/d1a/d5f/df9/f124 [0,4194304] 0 2024-03-24T00:49:11.404 INFO:tasks.workunit.client.1.smithi088.stdout:4/963: dread d5/d13/f12 [0,4194304] 0 2024-03-24T00:49:11.426 INFO:tasks.workunit.client.1.smithi088.stdout:2/899: symlink db/d21/d35/dcb/l124 0 2024-03-24T00:49:11.429 INFO:tasks.workunit.client.0.smithi038.stdout:5/853: dwrite d6/d1e/d97/db0/fe0 [0,4194304] 0 2024-03-24T00:49:11.429 INFO:tasks.workunit.client.0.smithi038.stdout:0/852: creat d0/d12/d42/d44/f11e x:0 0 0 2024-03-24T00:49:11.429 INFO:tasks.workunit.client.0.smithi038.stdout:0/853: truncate d0/d14/d88/db8/ff2 888552 0 2024-03-24T00:49:11.429 INFO:tasks.workunit.client.0.smithi038.stdout:0/854: chown d0/d12/d15/d58/lf8 56098 1 2024-03-24T00:49:11.431 INFO:tasks.workunit.client.1.smithi088.stdout:3/953: truncate d3/d6/d27/d3c/f30 6965308 0 2024-03-24T00:49:11.433 INFO:tasks.workunit.client.1.smithi088.stdout:7/883: creat d3/d32/d34/d9c/d4b/d74/d79/dea/f115 x:0 0 0 2024-03-24T00:49:11.434 INFO:tasks.workunit.client.0.smithi038.stdout:1/973: mkdir da/dd/d4a/d8b/d13f 0 2024-03-24T00:49:11.435 INFO:tasks.workunit.client.0.smithi038.stdout:8/910: getdents d0/d1d/d27/d4d/daf 0 2024-03-24T00:49:11.440 INFO:tasks.workunit.client.0.smithi038.stdout:2/810: dwrite d4/d27/f2a [0,4194304] 0 2024-03-24T00:49:11.444 INFO:tasks.workunit.client.1.smithi088.stdout:9/959: dwrite d1/d10/d9b/dab/db8/da7/fd6 [0,4194304] 0 2024-03-24T00:49:11.450 INFO:tasks.workunit.client.0.smithi038.stdout:9/996: dwrite d12/d28/d9c/ddd/d111/d26/fb7 [0,4194304] 0 2024-03-24T00:49:11.451 INFO:tasks.workunit.client.0.smithi038.stdout:9/997: write d12/d1a/d80/fb4 [70895,69163] 0 2024-03-24T00:49:11.458 INFO:tasks.workunit.client.1.smithi088.stdout:7/884: write d3/d31/f70 [3327629,21775] 0 2024-03-24T00:49:11.480 INFO:tasks.workunit.client.1.smithi088.stdout:4/964: dwrite d5/d7/de4/f127 [0,4194304] 0 2024-03-24T00:49:11.493 INFO:tasks.workunit.client.0.smithi038.stdout:4/854: symlink da/d12/d29/d5b/d73/db4/l123 0 2024-03-24T00:49:11.543 INFO:tasks.workunit.client.1.smithi088.stdout:2/900: mknod db/dd/dc9/c125 0 2024-03-24T00:49:11.543 INFO:tasks.workunit.client.1.smithi088.stdout:2/901: fsync db/d21/d35/f7d 0 2024-03-24T00:49:11.544 INFO:tasks.workunit.client.1.smithi088.stdout:2/902: read db/dd/d96/d99/feb [515547,65384] 0 2024-03-24T00:49:11.546 INFO:tasks.workunit.client.1.smithi088.stdout:3/954: mknod d3/de/d6f/c132 0 2024-03-24T00:49:11.550 INFO:tasks.workunit.client.1.smithi088.stdout:3/955: write d3/d6/d19/d2a/f53 [972855,108297] 0 2024-03-24T00:49:11.550 INFO:tasks.workunit.client.1.smithi088.stdout:9/960: rmdir d1/d10/d11 39 2024-03-24T00:49:11.559 INFO:tasks.workunit.client.0.smithi038.stdout:5/854: creat d6/d16/d67/f117 x:0 0 0 2024-03-24T00:49:11.559 INFO:tasks.workunit.client.0.smithi038.stdout:5/855: write d6/d19/d20/d42/d43/d68/d6e/f7f [3771094,63755] 0 2024-03-24T00:49:11.562 INFO:tasks.workunit.client.0.smithi038.stdout:0/855: mkdir d0/d12/d42/d5e/d11f 0 2024-03-24T00:49:11.562 INFO:tasks.workunit.client.0.smithi038.stdout:0/856: chown d0/d12/d92/l104 4812 1 2024-03-24T00:49:11.563 INFO:tasks.workunit.client.1.smithi088.stdout:4/965: mkdir d5/d7/d72/d141/d154 0 2024-03-24T00:49:11.563 INFO:tasks.workunit.client.1.smithi088.stdout:4/966: fdatasync d5/d13/d18/d24/db3/f132 0 2024-03-24T00:49:11.565 INFO:tasks.workunit.client.0.smithi038.stdout:1/974: creat da/dd/d7a/d82/d98/d34/dd9/f140 x:0 0 0 2024-03-24T00:49:11.568 INFO:tasks.workunit.client.0.smithi038.stdout:8/911: mknod d0/d45/d37/d5f/d68/c12d 0 2024-03-24T00:49:11.574 INFO:tasks.workunit.client.0.smithi038.stdout:3/961: symlink d3/d9/d77/dd4/l148 0 2024-03-24T00:49:11.574 INFO:tasks.workunit.client.0.smithi038.stdout:3/962: chown d3/d30/d5d/deb/lf9 16139838 1 2024-03-24T00:49:11.576 INFO:tasks.workunit.client.1.smithi088.stdout:4/967: creat d5/d13/d63/d78/df1/f155 x:0 0 0 2024-03-24T00:49:11.576 INFO:tasks.workunit.client.0.smithi038.stdout:5/856: read d6/d1e/f1f [3896299,72743] 0 2024-03-24T00:49:11.576 INFO:tasks.workunit.client.0.smithi038.stdout:5/857: creat d6/d19/da1/f118 x:0 0 0 2024-03-24T00:49:11.576 INFO:tasks.workunit.client.0.smithi038.stdout:5/858: write d6/d19/d20/d33/d6f/fe4 [461888,128065] 0 2024-03-24T00:49:11.576 INFO:tasks.workunit.client.0.smithi038.stdout:5/859: write d6/d19/d20/d42/d43/d84/fc6 [4075543,60000] 0 2024-03-24T00:49:11.576 INFO:tasks.workunit.client.0.smithi038.stdout:5/860: read d6/d19/d20/d42/d43/d84/fc6 [1716847,31332] 0 2024-03-24T00:49:11.579 INFO:tasks.workunit.client.0.smithi038.stdout:2/811: fdatasync d4/dc7/ff4 0 2024-03-24T00:49:11.582 INFO:tasks.workunit.client.1.smithi088.stdout:4/968: mkdir d5/d13/d156 0 2024-03-24T00:49:11.582 INFO:tasks.workunit.client.1.smithi088.stdout:4/969: fsync d5/d13/d63/f142 0 2024-03-24T00:49:11.582 INFO:tasks.workunit.client.1.smithi088.stdout:4/970: creat d5/d17/d91/f157 x:0 0 0 2024-03-24T00:49:11.582 INFO:tasks.workunit.client.1.smithi088.stdout:4/971: truncate d5/d17/f81 1375287 0 2024-03-24T00:49:11.583 INFO:tasks.workunit.client.0.smithi038.stdout:1/975: dread da/dd/d22/d94/fa0 [0,4194304] 0 2024-03-24T00:49:11.584 INFO:tasks.workunit.client.0.smithi038.stdout:1/976: write da/dd/d7a/d82/d98/d14/f13b [859338,100417] 0 2024-03-24T00:49:11.584 INFO:tasks.workunit.client.0.smithi038.stdout:1/977: stat da/dd/d7a/d82/d98/d14/f13b 0 2024-03-24T00:49:11.586 INFO:tasks.workunit.client.1.smithi088.stdout:4/972: creat d5/d7/d72/d141/f158 x:0 0 0 2024-03-24T00:49:11.589 INFO:tasks.workunit.client.1.smithi088.stdout:4/973: symlink d5/d7/dd/d88/l159 0 2024-03-24T00:49:11.591 INFO:tasks.workunit.client.0.smithi038.stdout:9/998: getdents d12/d28/d9c/ddd/d111/d8f 0 2024-03-24T00:49:11.592 INFO:tasks.workunit.client.1.smithi088.stdout:4/974: stat d5/d13/d18/d6e/d7d/l97 0 2024-03-24T00:49:11.593 INFO:tasks.workunit.client.1.smithi088.stdout:4/975: chown d5/d7/de4/d107 0 1 2024-03-24T00:49:11.594 INFO:tasks.workunit.client.0.smithi038.stdout:4/855: creat da/d12/d37/dd/d15/d6d/d8f/df0/f124 x:0 0 0 2024-03-24T00:49:11.595 INFO:tasks.workunit.client.1.smithi088.stdout:4/976: creat d5/d13/d123/f15a x:0 0 0 2024-03-24T00:49:11.597 INFO:tasks.workunit.client.1.smithi088.stdout:7/885: rename d3/l22 to d3/d32/d76/l116 0 2024-03-24T00:49:11.598 INFO:tasks.workunit.client.1.smithi088.stdout:4/977: write d5/f46 [109599,86228] 0 2024-03-24T00:49:11.605 INFO:tasks.workunit.client.1.smithi088.stdout:9/961: rename d1/d15/d21/d9c/cea to d1/d10/d29/d85/c135 0 2024-03-24T00:49:11.607 INFO:tasks.workunit.client.0.smithi038.stdout:3/963: mknod d3/d30/d5d/deb/c149 0 2024-03-24T00:49:11.607 INFO:tasks.workunit.client.0.smithi038.stdout:3/964: chown d3/d9/dc/da6/db5/d19/d82/d89/da8/c141 54 1 2024-03-24T00:49:11.607 INFO:tasks.workunit.client.0.smithi038.stdout:3/965: dread - d3/d9/dc/da6/db5/d19/dd2/df0/ff2 zero size 2024-03-24T00:49:11.607 INFO:tasks.workunit.client.0.smithi038.stdout:3/966: write d3/d30/d70/d90/d99/f134 [108354,82571] 0 2024-03-24T00:49:11.607 INFO:tasks.workunit.client.0.smithi038.stdout:3/967: write d3/d9/dc/da6/db5/d19/d82/fba [744621,82277] 0 2024-03-24T00:49:11.612 INFO:tasks.workunit.client.1.smithi088.stdout:9/962: mkdir d1/d10/d29/def/d136 0 2024-03-24T00:49:11.615 INFO:tasks.workunit.client.0.smithi038.stdout:5/861: rename d6/d16/d14/c31 to d6/d1e/d109/c119 0 2024-03-24T00:49:11.615 INFO:tasks.workunit.client.0.smithi038.stdout:5/862: stat d6/d1e/d97/fc3 0 2024-03-24T00:49:11.617 INFO:tasks.workunit.client.1.smithi088.stdout:4/978: dread d5/d13/d18/f9e [0,4194304] 0 2024-03-24T00:49:11.617 INFO:tasks.workunit.client.1.smithi088.stdout:4/979: write d5/fa0 [2768795,36257] 0 2024-03-24T00:49:11.620 INFO:tasks.workunit.client.0.smithi038.stdout:2/812: symlink d4/d27/d54/d67/d111/l116 0 2024-03-24T00:49:11.624 INFO:tasks.workunit.client.1.smithi088.stdout:3/956: dwrite d3/fb3 [0,4194304] 0 2024-03-24T00:49:11.625 INFO:tasks.workunit.client.1.smithi088.stdout:3/957: dread - d3/de/d6f/d85/fcb zero size 2024-03-24T00:49:11.629 INFO:tasks.workunit.client.0.smithi038.stdout:0/857: dwrite d0/d12/d15/d58/d64/da7/dbb/dd4/dea/fbe [0,4194304] 0 2024-03-24T00:49:11.632 INFO:tasks.workunit.client.0.smithi038.stdout:5/863: rmdir d6/d19/d20/d42/d43/d10c 0 2024-03-24T00:49:11.633 INFO:tasks.workunit.client.0.smithi038.stdout:2/813: mkdir d4/d18/d26/d117 0 2024-03-24T00:49:11.635 INFO:tasks.workunit.client.1.smithi088.stdout:3/958: mknod d3/d6/d27/de9/ddf/c133 0 2024-03-24T00:49:11.636 INFO:tasks.workunit.client.0.smithi038.stdout:1/978: truncate da/dd/d7a/d82/fb4 536908 0 2024-03-24T00:49:11.637 INFO:tasks.workunit.client.1.smithi088.stdout:4/980: dread d5/dac/fb8 [0,4194304] 0 2024-03-24T00:49:11.638 INFO:tasks.workunit.client.0.smithi038.stdout:4/856: mkdir da/d12/d37/dd/d125 0 2024-03-24T00:49:11.639 INFO:tasks.workunit.client.0.smithi038.stdout:3/968: dread d3/d30/d70/d90/d99/f118 [0,4194304] 0 2024-03-24T00:49:11.642 INFO:tasks.workunit.client.0.smithi038.stdout:8/912: creat d0/d45/d37/d5f/db7/d62/f12e x:0 0 0 2024-03-24T00:49:11.646 INFO:tasks.workunit.client.1.smithi088.stdout:4/981: mknod d5/d7/d72/d141/c15b 0 2024-03-24T00:49:11.646 INFO:tasks.workunit.client.1.smithi088.stdout:4/982: dread - d5/f110 zero size 2024-03-24T00:49:11.647 INFO:tasks.workunit.client.0.smithi038.stdout:2/814: rename d4/d27/ddb to d4/da3/d118 0 2024-03-24T00:49:11.647 INFO:tasks.workunit.client.0.smithi038.stdout:2/815: stat d4/d27/d54/dab 0 2024-03-24T00:49:11.649 INFO:tasks.workunit.client.0.smithi038.stdout:1/979: unlink da/dd/d7a/d82/d98/d14/cb9 0 2024-03-24T00:49:11.652 INFO:tasks.workunit.client.0.smithi038.stdout:8/913: symlink d0/d45/d37/d5f/db7/l12f 0 2024-03-24T00:49:11.652 INFO:tasks.workunit.client.0.smithi038.stdout:8/914: stat d0/d1d/d27/d4d/daf 0 2024-03-24T00:49:11.653 INFO:tasks.workunit.client.1.smithi088.stdout:3/959: mkdir d3/d6/d19/d2a/d134 0 2024-03-24T00:49:11.653 INFO:tasks.workunit.client.1.smithi088.stdout:3/960: fsync f1 0 2024-03-24T00:49:11.653 INFO:tasks.workunit.client.0.smithi038.stdout:2/816: symlink d4/d27/d54/d67/l119 0 2024-03-24T00:49:11.653 INFO:tasks.workunit.client.0.smithi038.stdout:2/817: fsync d4/f66 0 2024-03-24T00:49:11.654 INFO:tasks.workunit.client.0.smithi038.stdout:1/980: creat da/dd/d2c/d5a/d12e/f141 x:0 0 0 2024-03-24T00:49:11.655 INFO:tasks.workunit.client.0.smithi038.stdout:9/999: dwrite d12/d1a/d5f/df9/f117 [4194304,4194304] 0 2024-03-24T00:49:11.656 INFO:tasks.workunit.client.0.smithi038.stdout:8/915: truncate d0/d45/d37/d5f/db7/dcc/fef 1604923 0 2024-03-24T00:49:11.656 INFO:tasks.workunit.client.0.smithi038.stdout:8/916: creat d0/d45/d37/d5f/db7/d62/f130 x:0 0 0 2024-03-24T00:49:11.658 INFO:tasks.workunit.client.0.smithi038.stdout:1/981: rename da/dd/d7a/d82/d98/f9a to da/d3c/d86/d111/f142 0 2024-03-24T00:49:11.661 INFO:tasks.workunit.client.0.smithi038.stdout:8/917: link d0/d1d/d27/d4d/daf/fb5 d0/d15/d2c/f131 0 2024-03-24T00:49:11.663 INFO:tasks.workunit.client.0.smithi038.stdout:1/982: getdents da/dd/d22 0 2024-03-24T00:49:11.664 INFO:tasks.workunit.client.1.smithi088.stdout:9/963: dwrite d1/d10/d9b/dab/db8/da7/fd6 [4194304,4194304] 0 2024-03-24T00:49:11.664 INFO:tasks.workunit.client.1.smithi088.stdout:9/964: write d1/d10/d11/d23/f81 [2942245,52305] 0 2024-03-24T00:49:11.667 INFO:tasks.workunit.client.1.smithi088.stdout:9/965: mknod d1/d15/d80/da5/c137 0 2024-03-24T00:49:11.670 INFO:tasks.workunit.client.0.smithi038.stdout:2/818: rmdir d4/da3/d118/d7c 39 2024-03-24T00:49:11.670 INFO:tasks.workunit.client.0.smithi038.stdout:2/819: chown d4/d27/d54/fd7 19477 1 2024-03-24T00:49:11.670 INFO:tasks.workunit.client.0.smithi038.stdout:2/820: getdents d4/d18/d26/d117 0 2024-03-24T00:49:11.671 INFO:tasks.workunit.client.0.smithi038.stdout:2/821: mknod d4/d18/d26/d117/c11a 0 2024-03-24T00:49:11.675 INFO:tasks.workunit.client.0.smithi038.stdout:2/822: mknod d4/d18/d38/d63/c11b 0 2024-03-24T00:49:11.687 INFO:tasks.workunit.client.0.smithi038.stdout:8/918: dread d0/d45/d66/f40 [0,4194304] 0 2024-03-24T00:49:11.687 INFO:tasks.workunit.client.0.smithi038.stdout:8/919: read d0/d45/d37/f48 [1200646,32186] 0 2024-03-24T00:49:11.687 INFO:tasks.workunit.client.0.smithi038.stdout:8/920: dread - d0/d1d/d27/d6b/fbb zero size 2024-03-24T00:49:11.688 INFO:tasks.workunit.client.0.smithi038.stdout:8/921: mknod d0/d1d/d27/d6b/dfc/d9e/c132 0 2024-03-24T00:49:11.690 INFO:tasks.workunit.client.0.smithi038.stdout:8/922: link d0/d15/f54 d0/d1d/d27/d4d/daf/dea/f133 0 2024-03-24T00:49:11.692 INFO:tasks.workunit.client.0.smithi038.stdout:3/969: symlink d3/d9/dc/da6/db5/d19/d82/d89/l14a 0 2024-03-24T00:49:11.693 INFO:tasks.workunit.client.1.smithi088.stdout:8/956: sync 2024-03-24T00:49:11.694 INFO:tasks.workunit.client.0.smithi038.stdout:3/970: symlink d3/d9/def/l14b 0 2024-03-24T00:49:11.694 INFO:tasks.workunit.client.0.smithi038.stdout:3/971: stat d3/d30/d5d 0 2024-03-24T00:49:11.695 INFO:tasks.workunit.client.1.smithi088.stdout:4/983: dwrite d5/d7/d72/d141/f158 [0,4194304] 0 2024-03-24T00:49:11.698 INFO:tasks.workunit.client.1.smithi088.stdout:7/886: rename d3/d32/d34/fc2 to d3/d32/d34/d9c/d4b/d73/f117 0 2024-03-24T00:49:11.698 INFO:tasks.workunit.client.1.smithi088.stdout:7/887: fdatasync d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/dd6/f112 0 2024-03-24T00:49:11.698 INFO:tasks.workunit.client.1.smithi088.stdout:7/888: chown d3/d32/d34/d9c/d4b/d73/dab/fb3 1 1 2024-03-24T00:49:11.699 INFO:tasks.workunit.client.0.smithi038.stdout:5/864: dwrite d6/d16/d21/d2d/fd2 [4194304,4194304] 0 2024-03-24T00:49:11.699 INFO:tasks.workunit.client.0.smithi038.stdout:5/865: chown d6/d19/d20/d42/d43/d68/dc8 13108 1 2024-03-24T00:49:11.699 INFO:tasks.workunit.client.0.smithi038.stdout:5/866: write d6/d19/d20/d42/fd5 [19827,51723] 0 2024-03-24T00:49:11.700 INFO:tasks.workunit.client.1.smithi088.stdout:1/951: sync 2024-03-24T00:49:11.704 INFO:tasks.workunit.client.1.smithi088.stdout:7/889: rename d3/d32/d34/d9c/d4b/d73/c108 to d3/d31/d56/de4/c118 0 2024-03-24T00:49:11.704 INFO:tasks.workunit.client.1.smithi088.stdout:7/890: chown d3/d31/c67 444 1 2024-03-24T00:49:11.706 INFO:tasks.workunit.client.0.smithi038.stdout:5/867: mknod d6/d19/d20/d64/c11a 0 2024-03-24T00:49:11.708 INFO:tasks.workunit.client.0.smithi038.stdout:8/923: write d0/d15/d2c/f88 [1238832,3028] 0 2024-03-24T00:49:11.709 INFO:tasks.workunit.client.1.smithi088.stdout:7/891: getdents d3/d32/d34/d9c/d4b/d73/dab 0 2024-03-24T00:49:11.710 INFO:tasks.workunit.client.0.smithi038.stdout:5/868: mknod d6/d16/d10f/d92/c11b 0 2024-03-24T00:49:11.711 INFO:tasks.workunit.client.0.smithi038.stdout:8/924: symlink d0/d45/d66/d6d/l134 0 2024-03-24T00:49:11.712 INFO:tasks.workunit.client.1.smithi088.stdout:8/957: write d9/f2b [3116019,63808] 0 2024-03-24T00:49:11.715 INFO:tasks.workunit.client.0.smithi038.stdout:8/925: unlink d0/d1d/d20/l3d 0 2024-03-24T00:49:11.719 INFO:tasks.workunit.client.1.smithi088.stdout:8/958: mknod d9/dc/d11/d24/d29/d5b/db9/da5/d136/c143 0 2024-03-24T00:49:11.723 INFO:tasks.workunit.client.1.smithi088.stdout:1/952: dread d6/d27/d67/f6b [0,4194304] 0 2024-03-24T00:49:11.723 INFO:tasks.workunit.client.0.smithi038.stdout:0/858: creat d0/d12/d42/f120 x:0 0 0 2024-03-24T00:49:11.723 INFO:tasks.workunit.client.1.smithi088.stdout:3/961: rename d3/d1a/d1e/d2f/d45 to d3/de/d6f/d85/d49/d70/d96/d12d/d135 0 2024-03-24T00:49:11.724 INFO:tasks.workunit.client.1.smithi088.stdout:1/953: rmdir d6/d27/d67/d88/da4 39 2024-03-24T00:49:11.726 INFO:tasks.workunit.client.1.smithi088.stdout:3/962: mknod d3/d6/d19/d2a/dc2/d11b/c136 0 2024-03-24T00:49:11.727 INFO:tasks.workunit.client.1.smithi088.stdout:7/892: getdents d3/d32/d76/df3 0 2024-03-24T00:49:11.727 INFO:tasks.workunit.client.1.smithi088.stdout:7/893: write d3/d32/d34/d9c/d4b/d73/dab/dc3/feb [42017,58680] 0 2024-03-24T00:49:11.727 INFO:tasks.workunit.client.0.smithi038.stdout:0/859: getdents d0/d12/d42/d5e/db4 0 2024-03-24T00:49:11.728 INFO:tasks.workunit.client.1.smithi088.stdout:9/966: rename d1/d10/d29/def/l105 to d1/d22/d27/df0/l138 0 2024-03-24T00:49:11.730 INFO:tasks.workunit.client.1.smithi088.stdout:1/954: unlink d6/d23/daf/ce6 0 2024-03-24T00:49:11.730 INFO:tasks.workunit.client.1.smithi088.stdout:1/955: chown d6/d6e/dfc/d108/d5b/d5d/d61/fdb 35869 1 2024-03-24T00:49:11.730 INFO:tasks.workunit.client.1.smithi088.stdout:1/956: creat d6/d6e/dfc/f13a x:0 0 0 2024-03-24T00:49:11.731 INFO:tasks.workunit.client.0.smithi038.stdout:7/854: sync 2024-03-24T00:49:11.731 INFO:tasks.workunit.client.0.smithi038.stdout:6/977: sync 2024-03-24T00:49:11.732 INFO:tasks.workunit.client.0.smithi038.stdout:2/823: dwrite d4/d27/d2f/d30/f8d [0,4194304] 0 2024-03-24T00:49:11.732 INFO:tasks.workunit.client.0.smithi038.stdout:2/824: chown d4/d18/d5b/l74 81 1 2024-03-24T00:49:11.734 INFO:tasks.workunit.client.0.smithi038.stdout:0/860: rmdir d0/d14/d84 39 2024-03-24T00:49:11.734 INFO:tasks.workunit.client.0.smithi038.stdout:0/861: write d0/d12/d15/d58/d64/ffa [330650,1982] 0 2024-03-24T00:49:11.735 INFO:tasks.workunit.client.1.smithi088.stdout:7/894: unlink d3/d32/d76/ce1 0 2024-03-24T00:49:11.735 INFO:tasks.workunit.client.1.smithi088.stdout:3/963: creat d3/d6/dca/dae/d4b/f137 x:0 0 0 2024-03-24T00:49:11.736 INFO:tasks.workunit.client.0.smithi038.stdout:7/855: dread - d2/d2c/d99/fcf zero size 2024-03-24T00:49:11.736 INFO:tasks.workunit.client.0.smithi038.stdout:7/856: dread - d2/d9/da/d23/d2b/d5b/d72/fbf zero size 2024-03-24T00:49:11.739 INFO:tasks.workunit.client.0.smithi038.stdout:8/926: write d0/d45/d66/f97 [887043,79788] 0 2024-03-24T00:49:11.739 INFO:tasks.workunit.client.1.smithi088.stdout:3/964: creat d3/d6/d27/f138 x:0 0 0 2024-03-24T00:49:11.739 INFO:tasks.workunit.client.1.smithi088.stdout:3/965: stat d3/d6/dca/dae/f110 0 2024-03-24T00:49:11.742 INFO:tasks.workunit.client.1.smithi088.stdout:8/959: dread d9/dc/d11/d24/d43/d4a/d118/d134/d54/f9a [0,4194304] 0 2024-03-24T00:49:11.742 INFO:tasks.workunit.client.1.smithi088.stdout:8/960: fdatasync d9/dc/dce/ddc/fe8 0 2024-03-24T00:49:11.742 INFO:tasks.workunit.client.1.smithi088.stdout:4/984: dwrite d5/d7/dd/f144 [4194304,4194304] 0 2024-03-24T00:49:11.742 INFO:tasks.workunit.client.0.smithi038.stdout:2/825: creat d4/d18/f11c x:0 0 0 2024-03-24T00:49:11.743 INFO:tasks.workunit.client.0.smithi038.stdout:2/826: stat d4/d55 0 2024-03-24T00:49:11.744 INFO:tasks.workunit.client.1.smithi088.stdout:3/966: rmdir d3/de 39 2024-03-24T00:49:11.744 INFO:tasks.workunit.client.1.smithi088.stdout:3/967: truncate d3/de/d6f/d85/d49/f99 4288929 0 2024-03-24T00:49:11.744 INFO:tasks.workunit.client.1.smithi088.stdout:3/968: dread - d3/de/d6f/d85/d31/f12c zero size 2024-03-24T00:49:11.747 INFO:tasks.workunit.client.0.smithi038.stdout:2/827: mknod d4/d27/d2f/dc5/c11d 0 2024-03-24T00:49:11.749 INFO:tasks.workunit.client.1.smithi088.stdout:8/961: creat d9/dc/d11/d24/d29/da9/d9c/f144 x:0 0 0 2024-03-24T00:49:11.749 INFO:tasks.workunit.client.1.smithi088.stdout:8/962: chown d9/dc/d11/d24/d29/da9/f142 11344447 1 2024-03-24T00:49:11.749 INFO:tasks.workunit.client.1.smithi088.stdout:7/895: dread d3/d32/d34/d9c/d4b/d73/dab/dd8/fd7 [0,4194304] 0 2024-03-24T00:49:11.749 INFO:tasks.workunit.client.1.smithi088.stdout:7/896: write d3/d32/f10b [11395,13659] 0 2024-03-24T00:49:11.750 INFO:tasks.workunit.client.1.smithi088.stdout:2/903: unlink db/d21/d35/l5d 0 2024-03-24T00:49:11.752 INFO:tasks.workunit.client.0.smithi038.stdout:7/857: dread d2/d9/da/d23/d2b/d5b/d72/d92/d5d/fda [0,4194304] 0 2024-03-24T00:49:11.752 INFO:tasks.workunit.client.0.smithi038.stdout:7/858: readlink d2/d9/da/d23/lc0 0 2024-03-24T00:49:11.756 INFO:tasks.workunit.client.1.smithi088.stdout:9/967: dread d1/d15/f37 [0,4194304] 0 2024-03-24T00:49:11.756 INFO:tasks.workunit.client.1.smithi088.stdout:7/897: chown d3/l4f 83313567 1 2024-03-24T00:49:11.757 INFO:tasks.workunit.client.1.smithi088.stdout:2/904: creat db/d21/dea/f126 x:0 0 0 2024-03-24T00:49:11.765 INFO:tasks.workunit.client.1.smithi088.stdout:8/963: truncate f1 1760371 0 2024-03-24T00:49:11.775 INFO:tasks.workunit.client.0.smithi038.stdout:7/859: dread d2/d9/da/f1d [0,4194304] 0 2024-03-24T00:49:11.775 INFO:tasks.workunit.client.0.smithi038.stdout:7/860: stat d2/d9/da/dc/d4a/f6d 0 2024-03-24T00:49:11.777 INFO:tasks.workunit.client.1.smithi088.stdout:9/968: dread d1/d10/d77/fb4 [0,4194304] 0 2024-03-24T00:49:11.780 INFO:tasks.workunit.client.0.smithi038.stdout:6/978: rename d1/da/d1b/d25/d49/d72 to d1/d37/d68/d145 0 2024-03-24T00:49:11.785 INFO:tasks.workunit.client.1.smithi088.stdout:9/969: symlink d1/d15/l139 0 2024-03-24T00:49:11.786 INFO:tasks.workunit.client.1.smithi088.stdout:1/957: dwrite d6/d27/fb3 [4194304,4194304] 0 2024-03-24T00:49:11.786 INFO:tasks.workunit.client.1.smithi088.stdout:1/958: readlink d6/d6e/dfc/d108/d5b/d80/lff 0 2024-03-24T00:49:11.788 INFO:tasks.workunit.client.0.smithi038.stdout:0/862: rename d0/f5 to d0/d14/d84/f121 0 2024-03-24T00:49:11.793 INFO:tasks.workunit.client.1.smithi088.stdout:9/970: rename d1/d10/d29/def/l12e to d1/d22/d27/l13a 0 2024-03-24T00:49:11.795 INFO:tasks.workunit.client.0.smithi038.stdout:7/861: rename d2/d9/da/f41 to d2/d9/da/dc/f13b 0 2024-03-24T00:49:11.795 INFO:tasks.workunit.client.0.smithi038.stdout:7/862: readlink d2/d9/da/d23/d2b/d5b/d72/d92/l138 0 2024-03-24T00:49:11.797 INFO:tasks.workunit.client.0.smithi038.stdout:6/979: dread d1/d37/d68/d145/fb7 [0,4194304] 0 2024-03-24T00:49:11.798 INFO:tasks.workunit.client.1.smithi088.stdout:1/959: mkdir d6/d6e/dfc/d108/d5b/d5d/d61/d7d/d13b 0 2024-03-24T00:49:11.799 INFO:tasks.workunit.client.0.smithi038.stdout:8/927: dwrite d0/d45/fe5 [0,4194304] 0 2024-03-24T00:49:11.799 INFO:tasks.workunit.client.0.smithi038.stdout:8/928: readlink d0/d1d/d8c/le7 0 2024-03-24T00:49:11.805 INFO:tasks.workunit.client.0.smithi038.stdout:0/863: mknod d0/d12/d9a/c122 0 2024-03-24T00:49:11.806 INFO:tasks.workunit.client.1.smithi088.stdout:1/960: symlink d6/d27/d103/l13c 0 2024-03-24T00:49:11.806 INFO:tasks.workunit.client.1.smithi088.stdout:1/961: read - d6/d27/d67/d9b/ff2 zero size 2024-03-24T00:49:11.806 INFO:tasks.workunit.client.1.smithi088.stdout:1/962: write d6/f68 [3635840,38266] 0 2024-03-24T00:49:11.811 INFO:tasks.workunit.client.0.smithi038.stdout:6/980: creat d1/d37/d84/f146 x:0 0 0 2024-03-24T00:49:11.811 INFO:tasks.workunit.client.0.smithi038.stdout:6/981: read - d1/da/d1b/d25/d49/d4e/f110 zero size 2024-03-24T00:49:11.815 INFO:tasks.workunit.client.1.smithi088.stdout:9/971: write d1/d10/d11/f126 [3519947,112506] 0 2024-03-24T00:49:11.818 INFO:tasks.workunit.client.1.smithi088.stdout:1/963: rename d6/d6e/dfc/d108/d3f/l134 to d6/d6e/l13d 0 2024-03-24T00:49:11.818 INFO:tasks.workunit.client.0.smithi038.stdout:0/864: creat d0/d1b/df1/d10d/f123 x:0 0 0 2024-03-24T00:49:11.818 INFO:tasks.workunit.client.0.smithi038.stdout:0/865: chown d0/d12/d42/d9d/d28/cec 1314327 1 2024-03-24T00:49:11.821 INFO:tasks.workunit.client.0.smithi038.stdout:6/982: rename d1/d37/d84/c8a to d1/d37/d84/df5/d113/c147 0 2024-03-24T00:49:11.824 INFO:tasks.workunit.client.0.smithi038.stdout:8/929: dread d0/d15/d2c/f131 [0,4194304] 0 2024-03-24T00:49:11.827 INFO:tasks.workunit.client.0.smithi038.stdout:2/828: dwrite d4/d27/d54/fd7 [0,4194304] 0 2024-03-24T00:49:11.828 INFO:tasks.workunit.client.0.smithi038.stdout:2/829: write d4/d18/d26/d37/d69/d83/fd3 [4398776,74994] 0 2024-03-24T00:49:11.828 INFO:tasks.workunit.client.0.smithi038.stdout:2/830: fsync d4/d27/d8f/ffc 0 2024-03-24T00:49:11.829 INFO:tasks.workunit.client.0.smithi038.stdout:8/930: mkdir d0/d1d/d27/d4d/daf/d10c/d135 0 2024-03-24T00:49:11.829 INFO:tasks.workunit.client.0.smithi038.stdout:8/931: chown d0/d1d/d27/d6b/da4/ca7 2420 1 2024-03-24T00:49:11.834 INFO:tasks.workunit.client.1.smithi088.stdout:7/898: dwrite d3/d32/d76/df3/ff6 [0,4194304] 0 2024-03-24T00:49:11.835 INFO:tasks.workunit.client.0.smithi038.stdout:3/972: symlink d3/d9/dc/da6/db5/d19/d82/l14c 0 2024-03-24T00:49:11.835 INFO:tasks.workunit.client.0.smithi038.stdout:3/973: chown d3/d2a/f10c 62046 1 2024-03-24T00:49:11.838 INFO:tasks.workunit.client.0.smithi038.stdout:6/983: write d1/da/d1b/d2f/f7f [1869084,19560] 0 2024-03-24T00:49:11.839 INFO:tasks.workunit.client.0.smithi038.stdout:8/932: unlink d0/d45/d37/d5f/db7/d62/dfa/dc8/ldc 0 2024-03-24T00:49:11.840 INFO:tasks.workunit.client.0.smithi038.stdout:8/933: fsync d0/d15/f54 0 2024-03-24T00:49:11.844 INFO:tasks.workunit.client.0.smithi038.stdout:0/866: dread d0/d1b/d2c/fac [0,4194304] 0 2024-03-24T00:49:11.844 INFO:tasks.workunit.client.0.smithi038.stdout:0/867: creat d0/d14/d88/db8/f124 x:0 0 0 2024-03-24T00:49:11.844 INFO:tasks.workunit.client.0.smithi038.stdout:0/868: chown d0/d12/d15/d58/d64/da7/dbb/dcd/f10b 279827 1 2024-03-24T00:49:11.844 INFO:tasks.workunit.client.0.smithi038.stdout:0/869: fdatasync d0/d14/d88/db8/fed 0 2024-03-24T00:49:11.846 INFO:tasks.workunit.client.0.smithi038.stdout:6/984: symlink d1/da/d61/da2/l148 0 2024-03-24T00:49:11.847 INFO:tasks.workunit.client.1.smithi088.stdout:7/899: getdents d3/d32/d34/d9c/d4b/da7/dc5/d101 0 2024-03-24T00:49:11.847 INFO:tasks.workunit.client.0.smithi038.stdout:4/857: write da/d12/d37/dd/d15/d4c/d70/f9b [941477,115175] 0 2024-03-24T00:49:11.848 INFO:tasks.workunit.client.0.smithi038.stdout:4/858: dread - da/d12/d24/d5f/df8/ff9 zero size 2024-03-24T00:49:11.849 INFO:tasks.workunit.client.0.smithi038.stdout:2/831: dread d4/d27/d54/d53/de2/f9f [0,4194304] 0 2024-03-24T00:49:11.849 INFO:tasks.workunit.client.0.smithi038.stdout:2/832: fdatasync d4/d18/fe1 0 2024-03-24T00:49:11.855 INFO:tasks.workunit.client.0.smithi038.stdout:3/974: symlink d3/d9/dc/d51/l14d 0 2024-03-24T00:49:11.855 INFO:tasks.workunit.client.0.smithi038.stdout:3/975: chown d3/l4f 209 1 2024-03-24T00:49:11.855 INFO:tasks.workunit.client.0.smithi038.stdout:3/976: readlink d3/d9/dc/l106 0 2024-03-24T00:49:11.857 INFO:tasks.workunit.client.0.smithi038.stdout:0/870: rmdir d0/d12/d42/d83/dfe 39 2024-03-24T00:49:11.861 INFO:tasks.workunit.client.0.smithi038.stdout:4/859: symlink da/d12/d37/dd/d125/l126 0 2024-03-24T00:49:11.861 INFO:tasks.workunit.client.1.smithi088.stdout:9/972: dwrite d1/d10/d29/def/ff1 [0,4194304] 0 2024-03-24T00:49:11.862 INFO:tasks.workunit.client.1.smithi088.stdout:9/973: chown d1/d10/d77/d8a/c24 99701545 1 2024-03-24T00:49:11.862 INFO:tasks.workunit.client.0.smithi038.stdout:8/934: dread d0/d1d/d27/d4d/daf/f110 [0,4194304] 0 2024-03-24T00:49:11.863 INFO:tasks.workunit.client.0.smithi038.stdout:7/863: dwrite d2/d9/da/dc/d8a/fc9 [0,4194304] 0 2024-03-24T00:49:11.863 INFO:tasks.workunit.client.0.smithi038.stdout:7/864: readlink d2/d9/da/d23/d2b/d5b/d72/d92/d75/l8c 0 2024-03-24T00:49:11.863 INFO:tasks.workunit.client.0.smithi038.stdout:2/833: symlink d4/d18/d38/d63/l11e 0 2024-03-24T00:49:11.863 INFO:tasks.workunit.client.0.smithi038.stdout:7/865: write d2/d9/f54 [158020,128561] 0 2024-03-24T00:49:11.865 INFO:tasks.workunit.client.1.smithi088.stdout:7/900: link d3/d32/d34/d7c/ca9 d3/d32/d34/d9c/d4b/ddc/c119 0 2024-03-24T00:49:11.865 INFO:tasks.workunit.client.1.smithi088.stdout:7/901: fdatasync d3/f8c 0 2024-03-24T00:49:11.868 INFO:tasks.workunit.client.0.smithi038.stdout:3/977: creat d3/d30/d70/d90/f14e x:0 0 0 2024-03-24T00:49:11.871 INFO:tasks.workunit.client.0.smithi038.stdout:0/871: rename d0/d12/d42/fdd to d0/d12/d15/d58/d64/da7/dbb/dd4/dea/f125 0 2024-03-24T00:49:11.872 INFO:tasks.workunit.client.1.smithi088.stdout:7/902: symlink d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/l11a 0 2024-03-24T00:49:11.874 INFO:tasks.workunit.client.0.smithi038.stdout:4/860: symlink da/d12/d29/d5b/d73/db4/l127 0 2024-03-24T00:49:11.874 INFO:tasks.workunit.client.1.smithi088.stdout:9/974: read d1/d10/f3e [1992605,86223] 0 2024-03-24T00:49:11.876 INFO:tasks.workunit.client.1.smithi088.stdout:4/985: write d5/d13/d18/d24/f53 [3163556,45796] 0 2024-03-24T00:49:11.876 INFO:tasks.workunit.client.1.smithi088.stdout:4/986: write d5/d13/d63/d78/f11e [28157,130412] 0 2024-03-24T00:49:11.877 INFO:tasks.workunit.client.0.smithi038.stdout:8/935: read d0/d1d/d27/d4d/f7b [3715487,5383] 0 2024-03-24T00:49:11.877 INFO:tasks.workunit.client.0.smithi038.stdout:8/936: chown d0/d15/c51 3653273 1 2024-03-24T00:49:11.878 INFO:tasks.workunit.client.0.smithi038.stdout:2/834: fsync d4/f15 0 2024-03-24T00:49:11.879 INFO:tasks.workunit.client.1.smithi088.stdout:2/905: write db/d21/fb6 [1204551,59740] 0 2024-03-24T00:49:11.879 INFO:tasks.workunit.client.0.smithi038.stdout:7/866: rmdir d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab 39 2024-03-24T00:49:11.881 INFO:tasks.workunit.client.1.smithi088.stdout:9/975: creat d1/d10/d29/d6d/f13b x:0 0 0 2024-03-24T00:49:11.881 INFO:tasks.workunit.client.1.smithi088.stdout:9/976: write d1/d10/d9b/dab/db8/fce [975598,10502] 0 2024-03-24T00:49:11.883 INFO:tasks.workunit.client.0.smithi038.stdout:3/978: rename d3/d9/def/c101 to d3/dc3/d102/c14f 0 2024-03-24T00:49:11.885 INFO:tasks.workunit.client.0.smithi038.stdout:0/872: rmdir d0/d12/d42/d83/dfe 39 2024-03-24T00:49:11.889 INFO:tasks.workunit.client.0.smithi038.stdout:4/861: link da/d12/d24/d68/d6c/da6/de7/l116 da/d12/d37/dd/d15/d6d/d8f/l128 0 2024-03-24T00:49:11.890 INFO:tasks.workunit.client.0.smithi038.stdout:2/835: write d4/fdd [979927,101023] 0 2024-03-24T00:49:11.894 INFO:tasks.workunit.client.1.smithi088.stdout:9/977: rename d1/d10/cff to d1/d22/c13c 0 2024-03-24T00:49:11.895 INFO:tasks.workunit.client.0.smithi038.stdout:8/937: dread d0/d45/d66/da6/fae [0,4194304] 0 2024-03-24T00:49:11.895 INFO:tasks.workunit.client.0.smithi038.stdout:8/938: chown d0/d45/d66/d107 21 1 2024-03-24T00:49:11.895 INFO:tasks.workunit.client.0.smithi038.stdout:8/939: read - d0/d45/d37/d5f/db7/d62/dfa/dc8/f10f zero size 2024-03-24T00:49:11.896 INFO:tasks.workunit.client.0.smithi038.stdout:0/873: creat d0/d12/d42/d5e/db4/f126 x:0 0 0 2024-03-24T00:49:11.897 INFO:tasks.workunit.client.1.smithi088.stdout:9/978: mkdir d1/d10/d11/d13d 0 2024-03-24T00:49:11.899 INFO:tasks.workunit.client.0.smithi038.stdout:4/862: symlink da/d12/d29/d5b/d73/db4/l129 0 2024-03-24T00:49:11.901 INFO:tasks.workunit.client.1.smithi088.stdout:9/979: getdents d1/d10/d9b 0 2024-03-24T00:49:11.905 INFO:tasks.workunit.client.1.smithi088.stdout:9/980: unlink d1/f4 0 2024-03-24T00:49:11.917 INFO:tasks.workunit.client.0.smithi038.stdout:7/867: getdents d2/d9/da/d23/d2b/d5b/d72/d92/d5d/dcd 0 2024-03-24T00:49:11.918 INFO:tasks.workunit.client.0.smithi038.stdout:8/940: mkdir d0/d1d/d8c/d136 0 2024-03-24T00:49:11.919 INFO:tasks.workunit.client.0.smithi038.stdout:0/874: dread d0/d1b/d2c/f86 [0,4194304] 0 2024-03-24T00:49:11.920 INFO:tasks.workunit.client.0.smithi038.stdout:2/836: rename d4/d27/d2f/dc5 to d4/da3/d118/d11f 0 2024-03-24T00:49:11.924 INFO:tasks.workunit.client.0.smithi038.stdout:8/941: link d0/d1d/f22 d0/d1d/d20/d79/f137 0 2024-03-24T00:49:11.924 INFO:tasks.workunit.client.0.smithi038.stdout:4/863: dread da/d12/d37/dd/f2b [0,4194304] 0 2024-03-24T00:49:11.924 INFO:tasks.workunit.client.0.smithi038.stdout:4/864: write da/d38/f63 [559714,97438] 0 2024-03-24T00:49:11.925 INFO:tasks.workunit.client.0.smithi038.stdout:0/875: link d0/d12/d15/d58/d64/ffa d0/d12/d15/f127 0 2024-03-24T00:49:11.925 INFO:tasks.workunit.client.0.smithi038.stdout:0/876: creat d0/daf/f128 x:0 0 0 2024-03-24T00:49:11.929 INFO:tasks.workunit.client.0.smithi038.stdout:4/865: creat da/d12/d24/f12a x:0 0 0 2024-03-24T00:49:11.929 INFO:tasks.workunit.client.0.smithi038.stdout:8/942: truncate d0/d45/d66/d6d/fbc 1445841 0 2024-03-24T00:49:11.932 INFO:tasks.workunit.client.0.smithi038.stdout:4/866: rename da/d38/f63 to da/d12/d37/d83/dac/f12b 0 2024-03-24T00:49:11.933 INFO:tasks.workunit.client.1.smithi088.stdout:9/981: write d1/d15/fdd [2903569,126288] 0 2024-03-24T00:49:11.934 INFO:tasks.workunit.client.0.smithi038.stdout:8/943: symlink d0/d1d/d27/d4d/daf/l138 0 2024-03-24T00:49:11.934 INFO:tasks.workunit.client.1.smithi088.stdout:2/906: dwrite db/dd/d96/d4e/d8f/d9c/fee [0,4194304] 0 2024-03-24T00:49:11.938 INFO:tasks.workunit.client.1.smithi088.stdout:9/982: mknod d1/d10/d2c/c13e 0 2024-03-24T00:49:11.943 INFO:tasks.workunit.client.1.smithi088.stdout:4/987: dwrite d5/d13/d63/f139 [0,4194304] 0 2024-03-24T00:49:11.943 INFO:tasks.workunit.client.1.smithi088.stdout:9/983: creat d1/d10/d9b/dab/f13f x:0 0 0 2024-03-24T00:49:11.949 INFO:tasks.workunit.client.0.smithi038.stdout:7/868: write d2/d9/da/dc/f1c [610711,130958] 0 2024-03-24T00:49:11.953 INFO:tasks.workunit.client.0.smithi038.stdout:7/869: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d33/c13c 0 2024-03-24T00:49:11.954 INFO:tasks.workunit.client.0.smithi038.stdout:7/870: stat d2/d9/d16/dfc/f119 0 2024-03-24T00:49:11.958 INFO:tasks.workunit.client.0.smithi038.stdout:7/871: link d2/d2c/f32 d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f13d 0 2024-03-24T00:49:11.960 INFO:tasks.workunit.client.0.smithi038.stdout:0/877: dread d0/d12/d42/d9d/fc7 [0,4194304] 0 2024-03-24T00:49:11.960 INFO:tasks.workunit.client.0.smithi038.stdout:0/878: truncate d0/d12/d15/d58/d64/da7/dbb/dd4/dea/fb3 4388316 0 2024-03-24T00:49:11.968 INFO:tasks.workunit.client.1.smithi088.stdout:7/903: dwrite d3/d32/d34/d9c/d4b/d73/f117 [0,4194304] 0 2024-03-24T00:49:11.969 INFO:tasks.workunit.client.1.smithi088.stdout:4/988: rename d5/d7/dd/d88/d137 to d5/d7/d72/d15c 0 2024-03-24T00:49:11.969 INFO:tasks.workunit.client.0.smithi038.stdout:0/879: link d0/d14/d88/db8/fbf d0/f129 0 2024-03-24T00:49:11.969 INFO:tasks.workunit.client.0.smithi038.stdout:0/880: write d0/f7 [559705,42292] 0 2024-03-24T00:49:11.977 INFO:tasks.workunit.client.0.smithi038.stdout:4/867: write da/d12/d24/d5f/d8a/fe0 [3735739,42193] 0 2024-03-24T00:49:11.977 INFO:tasks.workunit.client.0.smithi038.stdout:4/868: dread - da/d12/d24/d68/d6c/ff4 zero size 2024-03-24T00:49:11.977 INFO:tasks.workunit.client.0.smithi038.stdout:4/869: stat da/d12/d37/dd/d15/d6d/lfb 0 2024-03-24T00:49:11.978 INFO:tasks.workunit.client.1.smithi088.stdout:7/904: symlink d3/d32/d34/l11b 0 2024-03-24T00:49:11.978 INFO:tasks.workunit.client.1.smithi088.stdout:7/905: write d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/dd6/f112 [928941,34384] 0 2024-03-24T00:49:11.978 INFO:tasks.workunit.client.1.smithi088.stdout:7/906: chown d3/d32/d34/fd2 7370 1 2024-03-24T00:49:11.979 INFO:tasks.workunit.client.0.smithi038.stdout:4/870: getdents da/d12/d37/dd/d15/df1 0 2024-03-24T00:49:11.982 INFO:tasks.workunit.client.1.smithi088.stdout:1/964: sync 2024-03-24T00:49:11.982 INFO:tasks.workunit.client.1.smithi088.stdout:8/964: sync 2024-03-24T00:49:11.982 INFO:tasks.workunit.client.1.smithi088.stdout:3/969: sync 2024-03-24T00:49:11.982 INFO:tasks.workunit.client.1.smithi088.stdout:8/965: truncate d9/dc/d11/d24/d43/dd8/ff4 724056 0 2024-03-24T00:49:11.982 INFO:tasks.workunit.client.1.smithi088.stdout:3/970: dread - d3/d6/d58/fe6 zero size 2024-03-24T00:49:11.982 INFO:tasks.workunit.client.1.smithi088.stdout:3/971: fdatasync d3/d6/d27/de9/fa2 0 2024-03-24T00:49:11.982 INFO:tasks.workunit.client.1.smithi088.stdout:3/972: truncate d3/d6/d27/d3c/d80/f128 1007731 0 2024-03-24T00:49:11.983 INFO:tasks.workunit.client.1.smithi088.stdout:7/907: unlink d3/d32/l55 0 2024-03-24T00:49:11.983 INFO:tasks.workunit.client.1.smithi088.stdout:1/965: mknod d6/d27/db0/d10e/c13e 0 2024-03-24T00:49:11.985 INFO:tasks.workunit.client.1.smithi088.stdout:8/966: creat d9/dc/d11/d24/d29/da9/de7/f145 x:0 0 0 2024-03-24T00:49:11.985 INFO:tasks.workunit.client.1.smithi088.stdout:8/967: truncate d9/dc/d11/d24/d29/d5b/db9/fc5 2394987 0 2024-03-24T00:49:11.985 INFO:tasks.workunit.client.1.smithi088.stdout:8/968: write d9/dc/d11/d24/d29/d39/d84/deb/f13b [458555,128139] 0 2024-03-24T00:49:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:7/872: dread d2/d2c/d90/f115 [0,4194304] 0 2024-03-24T00:49:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:2/837: dwrite d4/d18/d26/d37/ff7 [0,4194304] 0 2024-03-24T00:49:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:2/838: write d4/d18/d26/f62 [5045726,88929] 0 2024-03-24T00:49:11.988 INFO:tasks.workunit.client.1.smithi088.stdout:8/969: mkdir d9/d6b/df5/d146 0 2024-03-24T00:49:11.997 INFO:tasks.workunit.client.1.smithi088.stdout:3/973: getdents d3/de/d6f/d85/d49 0 2024-03-24T00:49:11.998 INFO:tasks.workunit.client.1.smithi088.stdout:3/974: chown d3/d6/d19/ddb/l10e 27426130 1 2024-03-24T00:49:12.002 INFO:tasks.workunit.client.0.smithi038.stdout:7/873: mkdir d2/d9/da/dc/d93/d13e 0 2024-03-24T00:49:12.007 INFO:tasks.workunit.client.0.smithi038.stdout:4/871: write da/dbd/fed [2365754,79853] 0 2024-03-24T00:49:12.008 INFO:tasks.workunit.client.1.smithi088.stdout:8/970: symlink d9/dc/d11/d24/d43/d4a/d118/d134/d54/d5d/l147 0 2024-03-24T00:49:12.008 INFO:tasks.workunit.client.1.smithi088.stdout:1/966: write d6/d6e/dfc/d108/d58/f81 [154305,71733] 0 2024-03-24T00:49:12.008 INFO:tasks.workunit.client.0.smithi038.stdout:8/944: dwrite d0/d1d/d27/d4d/daf/fed [0,4194304] 0 2024-03-24T00:49:12.015 INFO:tasks.workunit.client.1.smithi088.stdout:1/967: rename d6/d27/d103/f11e to d6/d6e/dfc/d108/d5b/d80/f13f 0 2024-03-24T00:49:12.015 INFO:tasks.workunit.client.1.smithi088.stdout:1/968: fdatasync d6/d6e/dfc/d108/d58/fbe 0 2024-03-24T00:49:12.015 INFO:tasks.workunit.client.1.smithi088.stdout:1/969: chown d6/d6e/dfc/d108/d5b/f75 3256354 1 2024-03-24T00:49:12.016 INFO:tasks.workunit.client.0.smithi038.stdout:7/874: mkdir d2/d9/da/dc/d93/d13f 0 2024-03-24T00:49:12.020 INFO:tasks.workunit.client.0.smithi038.stdout:4/872: unlink da/d12/d29/l51 0 2024-03-24T00:49:12.022 INFO:tasks.workunit.client.1.smithi088.stdout:1/970: symlink d6/d6e/dfc/d108/d5b/l140 0 2024-03-24T00:49:12.031 INFO:tasks.workunit.client.0.smithi038.stdout:7/875: creat d2/d2c/d9a/dcb/d132/f140 x:0 0 0 2024-03-24T00:49:12.031 INFO:tasks.workunit.client.0.smithi038.stdout:7/876: fdatasync d2/d9/f7c 0 2024-03-24T00:49:12.031 INFO:tasks.workunit.client.0.smithi038.stdout:7/877: chown d2/d9/da/d23/d2b/d5b/d72/d92/d98/db1/cee 2576092 1 2024-03-24T00:49:12.032 INFO:tasks.workunit.client.0.smithi038.stdout:8/945: unlink d0/d1d/ce3 0 2024-03-24T00:49:12.039 INFO:tasks.workunit.client.0.smithi038.stdout:4/873: getdents da/d12 0 2024-03-24T00:49:12.039 INFO:tasks.workunit.client.0.smithi038.stdout:7/878: stat d2/d9/da/d23/d2b/d5b/d72/d92/d5d/c137 0 2024-03-24T00:49:12.039 INFO:tasks.workunit.client.0.smithi038.stdout:7/879: chown d2/d9/da/d9b/d11e 479 1 2024-03-24T00:49:12.039 INFO:tasks.workunit.client.1.smithi088.stdout:4/989: dwrite d5/d16/d94/da6/fc7 [0,4194304] 0 2024-03-24T00:49:12.043 INFO:tasks.workunit.client.1.smithi088.stdout:2/907: dwrite db/d21/d35/f9e [0,4194304] 0 2024-03-24T00:49:12.044 INFO:tasks.workunit.client.0.smithi038.stdout:8/946: truncate d0/d1d/d20/f58 2684033 0 2024-03-24T00:49:12.045 INFO:tasks.workunit.client.0.smithi038.stdout:8/947: creat d0/d45/d37/d5f/db7/d62/f139 x:0 0 0 2024-03-24T00:49:12.047 INFO:tasks.workunit.client.1.smithi088.stdout:2/908: mknod db/dd4/c127 0 2024-03-24T00:49:12.048 INFO:tasks.workunit.client.1.smithi088.stdout:4/990: write d5/d7/dd/f65 [46895,26356] 0 2024-03-24T00:49:12.051 INFO:tasks.workunit.client.1.smithi088.stdout:2/909: rmdir db/dd/d96/d99/db2/d88/d8e 39 2024-03-24T00:49:12.051 INFO:tasks.workunit.client.1.smithi088.stdout:2/910: creat db/dd/de7/f128 x:0 0 0 2024-03-24T00:49:12.051 INFO:tasks.workunit.client.1.smithi088.stdout:2/911: chown db/dd/d96/d4e/f6e 196 1 2024-03-24T00:49:12.054 INFO:tasks.workunit.client.1.smithi088.stdout:4/991: rmdir d5/d13/d18/d6e/d11b 0 2024-03-24T00:49:12.056 INFO:tasks.workunit.client.1.smithi088.stdout:2/912: mkdir db/dc/d129 0 2024-03-24T00:49:12.062 INFO:tasks.workunit.client.1.smithi088.stdout:2/913: unlink db/fe0 0 2024-03-24T00:49:12.065 INFO:tasks.workunit.client.1.smithi088.stdout:3/975: dread d3/d6/d27/f8d [0,4194304] 0 2024-03-24T00:49:12.065 INFO:tasks.workunit.client.1.smithi088.stdout:2/914: truncate db/d21/d35/d49/fac 3185928 0 2024-03-24T00:49:12.072 INFO:tasks.workunit.client.1.smithi088.stdout:1/971: dwrite d6/d6e/dfc/d108/d5b/d80/fd4 [0,4194304] 0 2024-03-24T00:49:12.075 INFO:tasks.workunit.client.0.smithi038.stdout:8/948: dread d0/d1d/d20/d79/fcd [0,4194304] 0 2024-03-24T00:49:12.076 INFO:tasks.workunit.client.0.smithi038.stdout:8/949: write d0/d45/d66/f97 [873908,101946] 0 2024-03-24T00:49:12.076 INFO:tasks.workunit.client.1.smithi088.stdout:7/908: dwrite d3/d32/d76/df3/ff6 [0,4194304] 0 2024-03-24T00:49:12.079 INFO:tasks.workunit.client.1.smithi088.stdout:1/972: getdents d6/d6e/dfc/d108/d5b/d5d/d61/d7d 0 2024-03-24T00:49:12.079 INFO:tasks.workunit.client.1.smithi088.stdout:1/973: truncate d6/f3e 2189055 0 2024-03-24T00:49:12.080 INFO:tasks.workunit.client.0.smithi038.stdout:2/839: dwrite f2 [4194304,4194304] 0 2024-03-24T00:49:12.082 INFO:tasks.workunit.client.1.smithi088.stdout:8/971: dwrite d9/dc/d11/d24/d29/da9/fab [0,4194304] 0 2024-03-24T00:49:12.082 INFO:tasks.workunit.client.1.smithi088.stdout:9/984: dwrite d1/f5 [0,4194304] 0 2024-03-24T00:49:12.082 INFO:tasks.workunit.client.1.smithi088.stdout:7/909: mknod d3/d32/d76/df3/c11c 0 2024-03-24T00:49:12.083 INFO:tasks.workunit.client.1.smithi088.stdout:7/910: truncate d3/d32/d34/d9c/d4b/d73/dab/dc3/feb 419811 0 2024-03-24T00:49:12.083 INFO:tasks.workunit.client.1.smithi088.stdout:1/974: mknod d6/d6e/dfc/d126/c141 0 2024-03-24T00:49:12.083 INFO:tasks.workunit.client.1.smithi088.stdout:1/975: dread - d6/d27/d67/f10f zero size 2024-03-24T00:49:12.087 INFO:tasks.workunit.client.0.smithi038.stdout:2/840: symlink d4/d55/l120 0 2024-03-24T00:49:12.088 INFO:tasks.workunit.client.0.smithi038.stdout:0/881: dwrite d0/f2 [0,4194304] 0 2024-03-24T00:49:12.088 INFO:tasks.workunit.client.0.smithi038.stdout:0/882: truncate d0/d12/d15/d58/d64/ffa 985189 0 2024-03-24T00:49:12.088 INFO:tasks.workunit.client.1.smithi088.stdout:4/992: dwrite d5/d16/fb1 [4194304,4194304] 0 2024-03-24T00:49:12.092 INFO:tasks.workunit.client.0.smithi038.stdout:2/841: mkdir d4/d18/d26/d37/d69/da6/d121 0 2024-03-24T00:49:12.093 INFO:tasks.workunit.client.0.smithi038.stdout:2/842: creat d4/d18/d38/d63/f122 x:0 0 0 2024-03-24T00:49:12.095 INFO:tasks.workunit.client.0.smithi038.stdout:0/883: mknod d0/d1b/c12a 0 2024-03-24T00:49:12.099 INFO:tasks.workunit.client.1.smithi088.stdout:9/985: rename d1/d10/d11/fcb to d1/d10/d9b/dab/db8/da7/f140 0 2024-03-24T00:49:12.100 INFO:tasks.workunit.client.0.smithi038.stdout:0/884: write d0/d1b/d2c/f3b [1867093,72881] 0 2024-03-24T00:49:12.106 INFO:tasks.workunit.client.1.smithi088.stdout:7/911: creat d3/d32/d34/d9c/d4b/ddc/f11d x:0 0 0 2024-03-24T00:49:12.108 INFO:tasks.workunit.client.0.smithi038.stdout:2/843: mknod d4/da3/d118/d11f/c123 0 2024-03-24T00:49:12.109 INFO:tasks.workunit.client.0.smithi038.stdout:2/844: truncate d4/d18/d5b/fa8 738882 0 2024-03-24T00:49:12.112 INFO:tasks.workunit.client.1.smithi088.stdout:1/976: creat d6/d6e/dfc/d108/d5b/d5e/d90/f142 x:0 0 0 2024-03-24T00:49:12.113 INFO:tasks.workunit.client.1.smithi088.stdout:7/912: read d3/d32/d34/d9c/d4b/fdf [4737980,101726] 0 2024-03-24T00:49:12.115 INFO:tasks.workunit.client.1.smithi088.stdout:4/993: link d5/d13/d18/d24/f125 d5/d16/d37/de7/f15d 0 2024-03-24T00:49:12.115 INFO:tasks.workunit.client.0.smithi038.stdout:0/885: read d0/d1b/d2c/fac [139551,35490] 0 2024-03-24T00:49:12.115 INFO:tasks.workunit.client.0.smithi038.stdout:0/886: dread - d0/d12/d15/d58/fe7 zero size 2024-03-24T00:49:12.115 INFO:tasks.workunit.client.0.smithi038.stdout:0/887: write d0/d14/d88/db8/fbf [234708,120344] 0 2024-03-24T00:49:12.118 INFO:tasks.workunit.client.1.smithi088.stdout:7/913: mkdir d3/d32/d34/d9c/d4b/da7/dc5/d101/d11e 0 2024-03-24T00:49:12.118 INFO:tasks.workunit.client.1.smithi088.stdout:7/914: creat d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/f11f x:0 0 0 2024-03-24T00:49:12.122 INFO:tasks.workunit.client.1.smithi088.stdout:3/976: dwrite d3/d32/fc5 [0,4194304] 0 2024-03-24T00:49:12.122 INFO:tasks.workunit.client.1.smithi088.stdout:4/994: mknod d5/d13/d63/d78/df1/c15e 0 2024-03-24T00:49:12.128 INFO:tasks.workunit.client.1.smithi088.stdout:4/995: symlink d5/l15f 0 2024-03-24T00:49:12.135 INFO:tasks.workunit.client.1.smithi088.stdout:1/977: dread d6/d27/d67/f6b [0,4194304] 0 2024-03-24T00:49:12.138 INFO:tasks.workunit.client.1.smithi088.stdout:1/978: mknod d6/d27/db0/de3/c143 0 2024-03-24T00:49:12.140 INFO:tasks.workunit.client.1.smithi088.stdout:8/972: dwrite d9/dc/d11/d24/d29/da9/d9c/f144 [0,4194304] 0 2024-03-24T00:49:12.140 INFO:tasks.workunit.client.1.smithi088.stdout:8/973: dread - d9/dc/d11/d24/d43/d4a/d118/d134/d54/d5d/db3/fb4 zero size 2024-03-24T00:49:12.140 INFO:tasks.workunit.client.1.smithi088.stdout:8/974: write d9/dc/d11/d24/d29/d36/f139 [328034,106583] 0 2024-03-24T00:49:12.142 INFO:tasks.workunit.client.1.smithi088.stdout:7/915: dread d3/d32/d34/d9c/fae [0,4194304] 0 2024-03-24T00:49:12.143 INFO:tasks.workunit.client.1.smithi088.stdout:2/915: dwrite db/dd/d96/d4e/fcc [0,4194304] 0 2024-03-24T00:49:12.143 INFO:tasks.workunit.client.1.smithi088.stdout:2/916: write db/dd/de7/f128 [129634,39538] 0 2024-03-24T00:49:12.148 INFO:tasks.workunit.client.1.smithi088.stdout:8/975: mknod d9/d6b/df5/d146/c148 0 2024-03-24T00:49:12.148 INFO:tasks.workunit.client.1.smithi088.stdout:8/976: dread - d9/dc/d11/d24/d29/da9/f142 zero size 2024-03-24T00:49:12.154 INFO:tasks.workunit.client.1.smithi088.stdout:8/977: creat d9/dc/d11/d24/d29/d5b/db9/f149 x:0 0 0 2024-03-24T00:49:12.162 INFO:tasks.workunit.client.1.smithi088.stdout:3/977: read d3/de/d6f/d85/d90/ff7 [3983629,124052] 0 2024-03-24T00:49:12.163 INFO:tasks.workunit.client.0.smithi038.stdout:4/874: dwrite da/d12/d37/d83/dac/f12b [0,4194304] 0 2024-03-24T00:49:12.165 INFO:tasks.workunit.client.1.smithi088.stdout:3/978: symlink d3/de/d6f/d85/de0/d100/l139 0 2024-03-24T00:49:12.168 INFO:tasks.workunit.client.1.smithi088.stdout:3/979: chown d3/d1a/d1e/ce2 10052 1 2024-03-24T00:49:12.168 INFO:tasks.workunit.client.1.smithi088.stdout:3/980: truncate d3/de/f81 577820 0 2024-03-24T00:49:12.171 INFO:tasks.workunit.client.0.smithi038.stdout:4/875: stat da/d38/f3c 0 2024-03-24T00:49:12.171 INFO:tasks.workunit.client.0.smithi038.stdout:4/876: creat da/d12/d37/dd/d15/d6d/d8f/f12c x:0 0 0 2024-03-24T00:49:12.171 INFO:tasks.workunit.client.0.smithi038.stdout:4/877: readlink da/d12/d37/d83/lc6 0 2024-03-24T00:49:12.175 INFO:tasks.workunit.client.0.smithi038.stdout:4/878: symlink da/d27/l12d 0 2024-03-24T00:49:12.188 INFO:tasks.workunit.client.0.smithi038.stdout:8/950: dwrite d0/d15/d2c/f88 [0,4194304] 0 2024-03-24T00:49:12.189 INFO:tasks.workunit.client.0.smithi038.stdout:8/951: creat d0/d45/d37/f13a x:0 0 0 2024-03-24T00:49:12.193 INFO:tasks.workunit.client.1.smithi088.stdout:4/996: dwrite d5/d16/d37/f48 [4194304,4194304] 0 2024-03-24T00:49:12.193 INFO:tasks.workunit.client.1.smithi088.stdout:4/997: write d5/d7/d15/d74/fe9 [1279627,79744] 0 2024-03-24T00:49:12.194 INFO:tasks.workunit.client.1.smithi088.stdout:4/998: rename d5/d13/d18/d24/db3 to d5/d13/d18/d24/db3/d14c/d160 22 2024-03-24T00:49:12.196 INFO:tasks.workunit.client.1.smithi088.stdout:4/999: mkdir d5/d13/d63/d78/d161 0 2024-03-24T00:49:12.197 INFO:tasks.workunit.client.1.smithi088.stdout:9/986: dwrite d1/d10/d2c/f75 [0,4194304] 0 2024-03-24T00:49:12.205 INFO:tasks.workunit.client.0.smithi038.stdout:2/845: dwrite d4/d18/d38/f82 [0,4194304] 0 2024-03-24T00:49:12.205 INFO:tasks.workunit.client.1.smithi088.stdout:2/917: dwrite db/dd/d96/d99/db2/d88/f104 [0,4194304] 0 2024-03-24T00:49:12.205 INFO:tasks.workunit.client.1.smithi088.stdout:2/918: fdatasync db/dc/f2c 0 2024-03-24T00:49:12.209 INFO:tasks.workunit.client.1.smithi088.stdout:7/916: dwrite d3/d32/d34/d9c/d4b/d73/dab/dd8/f109 [0,4194304] 0 2024-03-24T00:49:12.210 INFO:tasks.workunit.client.1.smithi088.stdout:7/917: fdatasync d3/f21 0 2024-03-24T00:49:12.212 INFO:tasks.workunit.client.1.smithi088.stdout:7/918: rename d3/c25 to d3/d31/db5/c120 0 2024-03-24T00:49:12.215 INFO:tasks.workunit.client.1.smithi088.stdout:7/919: rename d3/d32/d34/d9c/d4b/d74/d79/f97 to d3/d31/db5/f121 0 2024-03-24T00:49:12.216 INFO:tasks.workunit.client.0.smithi038.stdout:2/846: dread d4/d76/fb3 [0,4194304] 0 2024-03-24T00:49:12.217 INFO:tasks.workunit.client.1.smithi088.stdout:7/920: unlink d3/d32/d34/d9c/d4b/l58 0 2024-03-24T00:49:12.220 INFO:tasks.workunit.client.1.smithi088.stdout:7/921: creat d3/d32/d34/d9c/d4b/da7/dc5/f122 x:0 0 0 2024-03-24T00:49:12.220 INFO:tasks.workunit.client.1.smithi088.stdout:7/922: chown d3/d32/d34/d7c/c86 2977 1 2024-03-24T00:49:12.235 INFO:tasks.workunit.client.1.smithi088.stdout:3/981: dwrite d3/f95 [4194304,4194304] 0 2024-03-24T00:49:12.237 INFO:tasks.workunit.client.1.smithi088.stdout:3/982: chown d3/d6/d27/d3c/d80/f128 31356063 1 2024-03-24T00:49:12.241 INFO:tasks.workunit.client.1.smithi088.stdout:8/978: dwrite d9/dc/d11/d24/d29/d5b/db9/da5/f13c [0,4194304] 0 2024-03-24T00:49:12.241 INFO:tasks.workunit.client.1.smithi088.stdout:3/983: rmdir d3/de/d6f/d85/d49/d70 39 2024-03-24T00:49:12.246 INFO:tasks.workunit.client.1.smithi088.stdout:3/984: mknod d3/d1a/d1e/d2f/c13a 0 2024-03-24T00:49:12.247 INFO:tasks.workunit.client.1.smithi088.stdout:8/979: rename d9/dc/d11/d24/f122 to d9/dc/d11/f14a 0 2024-03-24T00:49:12.248 INFO:tasks.workunit.client.1.smithi088.stdout:1/979: dwrite d6/d27/d67/d9b/ff2 [0,4194304] 0 2024-03-24T00:49:12.248 INFO:tasks.workunit.client.1.smithi088.stdout:1/980: stat d6/d27/f3a 0 2024-03-24T00:49:12.248 INFO:tasks.workunit.client.1.smithi088.stdout:1/981: fsync d6/d6e/dfc/d108/d5b/d80/ff9 0 2024-03-24T00:49:12.251 INFO:tasks.workunit.client.0.smithi038.stdout:8/952: dwrite d0/d45/d66/f97 [0,4194304] 0 2024-03-24T00:49:12.256 INFO:tasks.workunit.client.1.smithi088.stdout:2/919: dwrite db/dd/d96/d99/db2/ff0 [0,4194304] 0 2024-03-24T00:49:12.264 INFO:tasks.workunit.client.1.smithi088.stdout:9/987: dwrite d1/d15/d80/da5/fb2 [0,4194304] 0 2024-03-24T00:49:12.281 INFO:tasks.workunit.client.1.smithi088.stdout:3/985: dread d3/d6/d27/d3c/f30 [0,4194304] 0 2024-03-24T00:49:12.282 INFO:tasks.workunit.client.1.smithi088.stdout:3/986: read d3/de/f61 [1616628,93005] 0 2024-03-24T00:49:12.288 INFO:tasks.workunit.client.1.smithi088.stdout:7/923: dwrite d3/d32/d34/d7c/fc6 [0,4194304] 0 2024-03-24T00:49:12.290 INFO:tasks.workunit.client.1.smithi088.stdout:7/924: write d3/f24 [79342,5061] 0 2024-03-24T00:49:12.306 INFO:tasks.workunit.client.1.smithi088.stdout:8/980: symlink d9/dc/d11/l14b 0 2024-03-24T00:49:12.306 INFO:tasks.workunit.client.1.smithi088.stdout:8/981: readlink d9/dc/d11/d24/d43/d4a/l9e 0 2024-03-24T00:49:12.307 INFO:tasks.workunit.client.1.smithi088.stdout:8/982: fsync f2 0 2024-03-24T00:49:12.307 INFO:tasks.workunit.client.1.smithi088.stdout:7/925: dread d3/d31/f83 [0,4194304] 0 2024-03-24T00:49:12.310 INFO:tasks.workunit.client.1.smithi088.stdout:1/982: truncate d6/d27/fb3 4706293 0 2024-03-24T00:49:12.311 INFO:tasks.workunit.client.1.smithi088.stdout:1/983: fsync d6/d6e/dfc/d126/de7/fea 0 2024-03-24T00:49:12.323 INFO:tasks.workunit.client.1.smithi088.stdout:8/983: rename d9/dc/d11/d24/d29/d36/d6a/d7b/fa0 to d9/dc/d11/d24/d29/d5b/d8b/d101/d12e/f14c 0 2024-03-24T00:49:12.323 INFO:tasks.workunit.client.1.smithi088.stdout:8/984: readlink d9/dc/d11/d24/d43/lf1 0 2024-03-24T00:49:12.323 INFO:tasks.workunit.client.1.smithi088.stdout:8/985: chown d9/c68 761239220 1 2024-03-24T00:49:12.324 INFO:tasks.workunit.client.1.smithi088.stdout:7/926: unlink d3/d32/d34/la0 0 2024-03-24T00:49:12.326 INFO:tasks.workunit.client.1.smithi088.stdout:1/984: mkdir d6/d27/d136/d144 0 2024-03-24T00:49:12.327 INFO:tasks.workunit.client.1.smithi088.stdout:2/920: getdents db/dd/d96/d4e/d8f/d9c 0 2024-03-24T00:49:12.331 INFO:tasks.workunit.client.0.smithi038.stdout:8/953: write d0/d1d/d27/d4d/f75 [853736,5652] 0 2024-03-24T00:49:12.331 INFO:tasks.workunit.client.0.smithi038.stdout:8/954: dread - d0/d45/d66/da6/fe8 zero size 2024-03-24T00:49:12.331 INFO:tasks.workunit.client.0.smithi038.stdout:8/955: creat d0/d45/d37/d5f/db7/dcc/f13b x:0 0 0 2024-03-24T00:49:12.332 INFO:tasks.workunit.client.1.smithi088.stdout:8/986: mknod d9/dc/d11/d24/d29/d39/d84/deb/c14d 0 2024-03-24T00:49:12.336 INFO:tasks.workunit.client.1.smithi088.stdout:3/987: dwrite d3/de/d6f/d85/d90/ff7 [0,4194304] 0 2024-03-24T00:49:12.336 INFO:tasks.workunit.client.1.smithi088.stdout:1/985: rename d6/d6e/dfc/d126/c11a to d6/d6e/dfc/d108/d5b/d11b/c145 0 2024-03-24T00:49:12.346 INFO:tasks.workunit.client.1.smithi088.stdout:3/988: mkdir d3/de/d6f/d85/de0/d100/d13b 0 2024-03-24T00:49:12.346 INFO:tasks.workunit.client.1.smithi088.stdout:3/989: fdatasync d3/d67/dc3/dd7/ffc 0 2024-03-24T00:49:12.348 INFO:tasks.workunit.client.1.smithi088.stdout:1/986: unlink d6/d6e/dfc/d108/d5b/d80/ff9 0 2024-03-24T00:49:12.351 INFO:tasks.workunit.client.1.smithi088.stdout:1/987: link d6/f2a d6/d27/d103/f146 0 2024-03-24T00:49:12.351 INFO:tasks.workunit.client.1.smithi088.stdout:1/988: chown d6/d27/fb3 1 1 2024-03-24T00:49:12.353 INFO:tasks.workunit.client.1.smithi088.stdout:1/989: creat d6/f147 x:0 0 0 2024-03-24T00:49:12.353 INFO:tasks.workunit.client.1.smithi088.stdout:1/990: dread - d6/d6e/dfc/d108/d5b/d93/f106 zero size 2024-03-24T00:49:12.353 INFO:tasks.workunit.client.1.smithi088.stdout:1/991: write d6/d6e/dfc/d108/d5b/d5e/f64 [1116492,53102] 0 2024-03-24T00:49:12.354 INFO:tasks.workunit.client.0.smithi038.stdout:8/956: dread d0/d45/d37/f87 [0,4194304] 0 2024-03-24T00:49:12.355 INFO:tasks.workunit.client.1.smithi088.stdout:9/988: dwrite d1/d22/d27/d39/d98/dda/f123 [0,4194304] 0 2024-03-24T00:49:12.357 INFO:tasks.workunit.client.1.smithi088.stdout:1/992: creat d6/d6e/dfc/d108/d5b/d5d/d61/d7d/f148 x:0 0 0 2024-03-24T00:49:12.357 INFO:tasks.workunit.client.1.smithi088.stdout:8/987: write f1 [1239972,110709] 0 2024-03-24T00:49:12.360 INFO:tasks.workunit.client.1.smithi088.stdout:1/993: write d6/d6e/dfc/d108/d5b/f11c [47157,5544] 0 2024-03-24T00:49:12.360 INFO:tasks.workunit.client.1.smithi088.stdout:8/988: link d9/dc/d11/d24/l25 d9/dc/d11/d24/d131/l14e 0 2024-03-24T00:49:12.360 INFO:tasks.workunit.client.1.smithi088.stdout:8/989: chown d9/dc/d11/d24/d29/d5b/db9/da5/lea 2894222 1 2024-03-24T00:49:12.362 INFO:tasks.workunit.client.0.smithi038.stdout:8/957: write d0/d45/d37/d5f/db7/dcc/ff7 [430509,47897] 0 2024-03-24T00:49:12.362 INFO:tasks.workunit.client.0.smithi038.stdout:8/958: chown d0/d45/d66/c2a 5611601 1 2024-03-24T00:49:12.362 INFO:tasks.workunit.client.0.smithi038.stdout:8/959: readlink d0/d1d/d27/d4d/d81/ldb 0 2024-03-24T00:49:12.363 INFO:tasks.workunit.client.1.smithi088.stdout:8/990: rename d9/dc/d11/d24/d43/d4a/d140/faf to d9/dc/d11/d24/d29/d36/d6a/f14f 0 2024-03-24T00:49:12.363 INFO:tasks.workunit.client.1.smithi088.stdout:8/991: chown d9/dc/d11/d24/d29/d39/f93 1 1 2024-03-24T00:49:12.366 INFO:tasks.workunit.client.0.smithi038.stdout:8/960: mknod d0/d45/d37/d5f/db7/d62/df2/c13c 0 2024-03-24T00:49:12.366 INFO:tasks.workunit.client.0.smithi038.stdout:8/961: chown d0/d1d/f22 418993 1 2024-03-24T00:49:12.366 INFO:tasks.workunit.client.1.smithi088.stdout:8/992: truncate d9/dc/d11/d24/d29/f51 1940248 0 2024-03-24T00:49:12.368 INFO:tasks.workunit.client.1.smithi088.stdout:8/993: creat d9/d6b/f150 x:0 0 0 2024-03-24T00:49:12.377 INFO:tasks.workunit.client.1.smithi088.stdout:8/994: read d9/dc/dce/ddc/dde/f11a [142995,118506] 0 2024-03-24T00:49:12.379 INFO:tasks.workunit.client.1.smithi088.stdout:7/927: dwrite d3/d32/d34/d9c/d4b/f99 [0,4194304] 0 2024-03-24T00:49:12.379 INFO:tasks.workunit.client.1.smithi088.stdout:8/995: creat d9/dc/f151 x:0 0 0 2024-03-24T00:49:12.386 INFO:tasks.workunit.client.1.smithi088.stdout:2/921: dwrite db/dd/d96/d4e/fa0 [0,4194304] 0 2024-03-24T00:49:12.386 INFO:tasks.workunit.client.1.smithi088.stdout:7/928: getdents d3/d32/d34/d9c/d4b/d74 0 2024-03-24T00:49:12.389 INFO:tasks.workunit.client.1.smithi088.stdout:7/929: truncate d3/d32/d34/d7c/fc6 2284222 0 2024-03-24T00:49:12.394 INFO:tasks.workunit.client.0.smithi038.stdout:8/962: write d0/d15/f54 [1779091,18984] 0 2024-03-24T00:49:12.395 INFO:tasks.workunit.client.1.smithi088.stdout:8/996: rename d9/dc/d11/dd2 to d9/dc/d11/d24/d29/d5b/d152 0 2024-03-24T00:49:12.396 INFO:tasks.workunit.client.1.smithi088.stdout:8/997: creat d9/dc/d11/d80/db0/f153 x:0 0 0 2024-03-24T00:49:12.396 INFO:tasks.workunit.client.1.smithi088.stdout:3/990: dwrite d3/d1a/d1e/f9b [0,4194304] 0 2024-03-24T00:49:12.406 INFO:tasks.workunit.client.1.smithi088.stdout:2/922: write db/dd/d96/d99/db2/f102 [699949,52693] 0 2024-03-24T00:49:12.406 INFO:tasks.workunit.client.1.smithi088.stdout:2/923: chown db/dd/d96/d4e/d8f/d9c/db0/le2 7 1 2024-03-24T00:49:12.406 INFO:tasks.workunit.client.1.smithi088.stdout:2/924: truncate db/dd/d96/d99/db2/ff5 99635 0 2024-03-24T00:49:12.406 INFO:tasks.workunit.client.1.smithi088.stdout:2/925: truncate db/d21/d35/f10f 975306 0 2024-03-24T00:49:12.406 INFO:tasks.workunit.client.1.smithi088.stdout:2/926: fsync db/dd4/fec 0 2024-03-24T00:49:12.406 INFO:tasks.workunit.client.1.smithi088.stdout:2/927: chown db/dd/d96/d4e/d8f/dad 1540610 1 2024-03-24T00:49:12.406 INFO:tasks.workunit.client.1.smithi088.stdout:2/928: fdatasync db/f89 0 2024-03-24T00:49:12.407 INFO:tasks.workunit.client.1.smithi088.stdout:2/929: dread - db/d21/d73/fc5 zero size 2024-03-24T00:49:12.407 INFO:tasks.workunit.client.1.smithi088.stdout:2/930: write db/dd/f121 [649570,69667] 0 2024-03-24T00:49:12.409 INFO:tasks.workunit.client.1.smithi088.stdout:7/930: getdents d3/d32 0 2024-03-24T00:49:12.412 INFO:tasks.workunit.client.1.smithi088.stdout:2/931: symlink db/dc/d129/l12a 0 2024-03-24T00:49:12.413 INFO:tasks.workunit.client.1.smithi088.stdout:7/931: creat d3/d32/d34/d9c/d4b/da7/dc5/d101/d11e/f123 x:0 0 0 2024-03-24T00:49:12.414 INFO:tasks.workunit.client.1.smithi088.stdout:8/998: dread d9/dc/d11/f104 [0,4194304] 0 2024-03-24T00:49:12.415 INFO:tasks.workunit.client.1.smithi088.stdout:2/932: creat db/dd/dc9/f12b x:0 0 0 2024-03-24T00:49:12.416 INFO:tasks.workunit.client.1.smithi088.stdout:8/999: symlink d9/dc/d11/d24/d29/d36/l154 0 2024-03-24T00:49:12.417 INFO:tasks.workunit.client.0.smithi038.stdout:8/963: write d0/d15/f94 [3402126,56765] 0 2024-03-24T00:49:12.417 INFO:tasks.workunit.client.0.smithi038.stdout:8/964: chown d0/d45/d37/f13a 0 1 2024-03-24T00:49:12.427 INFO:tasks.workunit.client.0.smithi038.stdout:5/869: sync 2024-03-24T00:49:12.427 INFO:tasks.workunit.client.0.smithi038.stdout:1/983: sync 2024-03-24T00:49:12.427 INFO:tasks.workunit.client.0.smithi038.stdout:6/985: sync 2024-03-24T00:49:12.427 INFO:tasks.workunit.client.0.smithi038.stdout:6/986: truncate d1/d37/d68/d145/fdd 671378 0 2024-03-24T00:49:12.428 INFO:tasks.workunit.client.0.smithi038.stdout:6/987: creat d1/da/d61/da2/db1/f149 x:0 0 0 2024-03-24T00:49:12.428 INFO:tasks.workunit.client.0.smithi038.stdout:6/988: chown d1/d37/d84/df5/l142 392541147 1 2024-03-24T00:49:12.428 INFO:tasks.workunit.client.0.smithi038.stdout:6/989: getdents d1/d37/d84/df5/d113/d13c/d13d 0 2024-03-24T00:49:12.429 INFO:tasks.workunit.client.0.smithi038.stdout:5/870: symlink d6/d19/d20/d70/l11c 0 2024-03-24T00:49:12.432 INFO:tasks.workunit.client.0.smithi038.stdout:1/984: rename da/dd/d2c/d5a/d101/de8/d108/dc4/c133 to da/dd/d7a/d82/d98/d9e/c143 0 2024-03-24T00:49:12.434 INFO:tasks.workunit.client.0.smithi038.stdout:6/990: symlink d1/da/de6/l14a 0 2024-03-24T00:49:12.434 INFO:tasks.workunit.client.0.smithi038.stdout:6/991: dread - d1/da/d1b/d25/d49/fda zero size 2024-03-24T00:49:12.436 INFO:tasks.workunit.client.1.smithi088.stdout:2/933: read db/dd/fcf [1754080,116752] 0 2024-03-24T00:49:12.437 INFO:tasks.workunit.client.0.smithi038.stdout:1/985: symlink da/d3c/d86/d111/l144 0 2024-03-24T00:49:12.437 INFO:tasks.workunit.client.1.smithi088.stdout:3/991: dwrite d3/de/d6f/d85/d90/ff7 [0,4194304] 0 2024-03-24T00:49:12.439 INFO:tasks.workunit.client.0.smithi038.stdout:6/992: creat d1/da/d1b/dfd/d143/f14b x:0 0 0 2024-03-24T00:49:12.439 INFO:tasks.workunit.client.0.smithi038.stdout:6/993: readlink d1/d37/d68/d145/l111 0 2024-03-24T00:49:12.440 INFO:tasks.workunit.client.0.smithi038.stdout:8/965: write d0/d1d/d27/f8e [3432092,4344] 0 2024-03-24T00:49:12.440 INFO:tasks.workunit.client.0.smithi038.stdout:6/994: read d1/da/d1b/fb3 [691832,19576] 0 2024-03-24T00:49:12.442 INFO:tasks.workunit.client.1.smithi088.stdout:3/992: mkdir d3/de/d6f/dcc/d13c 0 2024-03-24T00:49:12.442 INFO:tasks.workunit.client.0.smithi038.stdout:1/986: getdents da/dd/d2c/d5a/d101/dda 0 2024-03-24T00:49:12.444 INFO:tasks.workunit.client.1.smithi088.stdout:3/993: creat d3/d6/d27/d3c/d80/f13d x:0 0 0 2024-03-24T00:49:12.444 INFO:tasks.workunit.client.1.smithi088.stdout:3/994: truncate d3/de/d6f/d85/d49/d70/d96/fe3 4456292 0 2024-03-24T00:49:12.444 INFO:tasks.workunit.client.1.smithi088.stdout:3/995: chown d3/de/d6f/f7a 24 1 2024-03-24T00:49:12.444 INFO:tasks.workunit.client.1.smithi088.stdout:3/996: chown d3/d6/d19/d2a/l23 1 1 2024-03-24T00:49:12.444 INFO:tasks.workunit.client.1.smithi088.stdout:3/997: truncate d3/d6/d27/d3c/d80/f13d 765390 0 2024-03-24T00:49:12.445 INFO:tasks.workunit.client.0.smithi038.stdout:6/995: mkdir d1/d37/d84/df5/d113/d13c/d14c 0 2024-03-24T00:49:12.445 INFO:tasks.workunit.client.0.smithi038.stdout:6/996: dread - d1/da/d61/d79/fba zero size 2024-03-24T00:49:12.446 INFO:tasks.workunit.client.0.smithi038.stdout:6/997: mkdir d1/d37/d68/d102/d14d 0 2024-03-24T00:49:12.448 INFO:tasks.workunit.client.1.smithi088.stdout:3/998: write d3/d6/d27/de9/fa2 [450467,106523] 0 2024-03-24T00:49:12.448 INFO:tasks.workunit.client.1.smithi088.stdout:3/999: fsync d3/d6/d27/d3c/fa8 0 2024-03-24T00:49:12.453 INFO:tasks.workunit.client.1.smithi088.stdout:7/932: dwrite d3/d32/d34/d9c/d4b/d74/d79/d7f/f80 [0,4194304] 0 2024-03-24T00:49:12.454 INFO:tasks.workunit.client.1.smithi088.stdout:7/933: fdatasync d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/f11f 0 2024-03-24T00:49:12.454 INFO:tasks.workunit.client.1.smithi088.stdout:7/934: dread - d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/f11f zero size 2024-03-24T00:49:12.456 INFO:tasks.workunit.client.0.smithi038.stdout:5/871: write d6/d19/d20/d70/da3/fab [11229848,69937] 0 2024-03-24T00:49:12.457 INFO:tasks.workunit.client.0.smithi038.stdout:8/966: dread d0/d1d/d27/d4d/f7b [0,4194304] 0 2024-03-24T00:49:12.458 INFO:tasks.workunit.client.0.smithi038.stdout:5/872: creat d6/d19/d20/d70/da3/f11d x:0 0 0 2024-03-24T00:49:12.458 INFO:tasks.workunit.client.0.smithi038.stdout:5/873: chown d6/d1e/d97 0 1 2024-03-24T00:49:12.458 INFO:tasks.workunit.client.0.smithi038.stdout:5/874: dread - d6/d19/d20/d64/d6b/fc2 zero size 2024-03-24T00:49:12.459 INFO:tasks.workunit.client.0.smithi038.stdout:5/875: mknod d6/d19/d20/d64/d6b/d77/c11e 0 2024-03-24T00:49:12.464 INFO:tasks.workunit.client.0.smithi038.stdout:5/876: truncate d6/d19/d20/d42/f72 1968825 0 2024-03-24T00:49:12.467 INFO:tasks.workunit.client.0.smithi038.stdout:1/987: dread da/dd/d2c/d41/f65 [0,4194304] 0 2024-03-24T00:49:12.473 INFO:tasks.workunit.client.0.smithi038.stdout:8/967: dread d0/d1d/d20/d79/f104 [0,4194304] 0 2024-03-24T00:49:12.473 INFO:tasks.workunit.client.0.smithi038.stdout:8/968: getdents d0/d45/d37/d5f/db7/d62/dfa/dc8/d11c 0 2024-03-24T00:49:12.473 INFO:tasks.workunit.client.0.smithi038.stdout:8/969: dread - d0/d45/d37/d5f/db7/d62/fd9 zero size 2024-03-24T00:49:12.475 INFO:tasks.workunit.client.0.smithi038.stdout:8/970: unlink d0/d1d/d27/d4d/daf/cf0 0 2024-03-24T00:49:12.479 INFO:tasks.workunit.client.0.smithi038.stdout:8/971: symlink d0/d45/db8/l13d 0 2024-03-24T00:49:12.479 INFO:tasks.workunit.client.0.smithi038.stdout:8/972: chown d0/d45/c9c 7608 1 2024-03-24T00:49:12.480 INFO:tasks.workunit.client.0.smithi038.stdout:8/973: fdatasync d0/d45/d37/d5f/d68/f85 0 2024-03-24T00:49:12.480 INFO:tasks.workunit.client.0.smithi038.stdout:8/974: mknod d0/d1d/d20/d79/d102/c13e 0 2024-03-24T00:49:12.481 INFO:tasks.workunit.client.0.smithi038.stdout:8/975: truncate d0/d45/d66/f40 2976900 0 2024-03-24T00:49:12.481 INFO:tasks.workunit.client.0.smithi038.stdout:8/976: chown d0/d1d/d27/d4d/fec 177503958 1 2024-03-24T00:49:12.482 INFO:tasks.workunit.client.0.smithi038.stdout:8/977: dread - d0/d45/d66/da6/fe8 zero size 2024-03-24T00:49:12.483 INFO:tasks.workunit.client.0.smithi038.stdout:8/978: link d0/d45/d37/f13a d0/d1d/d27/d6b/f13f 0 2024-03-24T00:49:12.484 INFO:tasks.workunit.client.0.smithi038.stdout:8/979: write d0/d1d/d27/d6b/dfc/d100/f122 [142049,8736] 0 2024-03-24T00:49:12.490 INFO:tasks.workunit.client.0.smithi038.stdout:0/888: sync 2024-03-24T00:49:12.490 INFO:tasks.workunit.client.0.smithi038.stdout:7/880: sync 2024-03-24T00:49:12.490 INFO:tasks.workunit.client.0.smithi038.stdout:3/979: sync 2024-03-24T00:49:12.497 INFO:tasks.workunit.client.0.smithi038.stdout:7/881: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/d141 0 2024-03-24T00:49:12.497 INFO:tasks.workunit.client.0.smithi038.stdout:7/882: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fc7 0 2024-03-24T00:49:12.498 INFO:tasks.workunit.client.0.smithi038.stdout:7/883: readlink d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/l95 0 2024-03-24T00:49:12.498 INFO:tasks.workunit.client.0.smithi038.stdout:7/884: write d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/d10a/f10f [649762,19634] 0 2024-03-24T00:49:12.498 INFO:tasks.workunit.client.0.smithi038.stdout:3/980: mknod d3/d30/d5d/d7c/db0/c150 0 2024-03-24T00:49:12.498 INFO:tasks.workunit.client.0.smithi038.stdout:3/981: chown d3/d30/d5d/d6a/laf 130215344 1 2024-03-24T00:49:12.499 INFO:tasks.workunit.client.0.smithi038.stdout:8/980: read d0/d1d/d20/f58 [2508674,37569] 0 2024-03-24T00:49:12.499 INFO:tasks.workunit.client.0.smithi038.stdout:8/981: creat d0/d45/d37/d5f/db7/d62/dfa/dc8/f140 x:0 0 0 2024-03-24T00:49:12.500 INFO:tasks.workunit.client.0.smithi038.stdout:8/982: write d0/d45/d37/f48 [3487536,102215] 0 2024-03-24T00:49:12.505 INFO:tasks.workunit.client.0.smithi038.stdout:3/982: symlink d3/d30/d70/l151 0 2024-03-24T00:49:12.508 INFO:tasks.workunit.client.1.smithi088.stdout:2/934: dwrite db/dd/dc9/d100/f10e [0,4194304] 0 2024-03-24T00:49:12.508 INFO:tasks.workunit.client.1.smithi088.stdout:2/935: write db/dd/d96/d99/fca [1990261,7854] 0 2024-03-24T00:49:12.508 INFO:tasks.workunit.client.1.smithi088.stdout:2/936: creat db/f12c x:0 0 0 2024-03-24T00:49:12.515 INFO:tasks.workunit.client.1.smithi088.stdout:2/937: mknod db/dd/d96/d4e/d8f/dad/df3/c12d 0 2024-03-24T00:49:12.515 INFO:tasks.workunit.client.1.smithi088.stdout:2/938: dread - db/dd/d96/d99/db2/f10b zero size 2024-03-24T00:49:12.522 INFO:tasks.workunit.client.1.smithi088.stdout:7/935: dwrite d3/d31/f70 [0,4194304] 0 2024-03-24T00:49:12.522 INFO:tasks.workunit.client.1.smithi088.stdout:7/936: creat d3/d32/d34/d9c/d4b/ddc/f124 x:0 0 0 2024-03-24T00:49:12.523 INFO:tasks.workunit.client.0.smithi038.stdout:1/988: dread da/f16 [0,4194304] 0 2024-03-24T00:49:12.523 INFO:tasks.workunit.client.0.smithi038.stdout:7/885: read d2/d9/f59 [617650,117365] 0 2024-03-24T00:49:12.524 INFO:tasks.workunit.client.0.smithi038.stdout:5/877: dwrite d6/d1e/f63 [0,4194304] 0 2024-03-24T00:49:12.524 INFO:tasks.workunit.client.0.smithi038.stdout:6/998: dwrite d1/d37/f100 [0,4194304] 0 2024-03-24T00:49:12.524 INFO:tasks.workunit.client.0.smithi038.stdout:5/878: fsync d6/d19/d20/d42/f91 0 2024-03-24T00:49:12.526 INFO:tasks.workunit.client.0.smithi038.stdout:1/989: chown da/dd/d2c/d41/lb2 7395036 1 2024-03-24T00:49:12.526 INFO:tasks.workunit.client.0.smithi038.stdout:1/990: read - da/dd/d7a/d82/f109 zero size 2024-03-24T00:49:12.531 INFO:tasks.workunit.client.0.smithi038.stdout:0/889: dwrite d0/d12/d92/fb2 [0,4194304] 0 2024-03-24T00:49:12.535 INFO:tasks.workunit.client.0.smithi038.stdout:7/886: creat d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/dcc/f142 x:0 0 0 2024-03-24T00:49:12.538 INFO:tasks.workunit.client.0.smithi038.stdout:1/991: mknod da/d3c/d86/dd7/c145 0 2024-03-24T00:49:12.539 INFO:tasks.workunit.client.0.smithi038.stdout:1/992: chown da/dd/l1b 0 1 2024-03-24T00:49:12.542 INFO:tasks.workunit.client.1.smithi088.stdout:2/939: rename db/dd/d96/d4e/d8f/dad to db/dd/d96/d12e 0 2024-03-24T00:49:12.542 INFO:tasks.workunit.client.1.smithi088.stdout:2/940: write db/dd/d96/d99/db2/d88/f117 [308404,69441] 0 2024-03-24T00:49:12.549 INFO:tasks.workunit.client.0.smithi038.stdout:0/890: mkdir d0/d12/d15/d58/d12b 0 2024-03-24T00:49:12.552 INFO:tasks.workunit.client.0.smithi038.stdout:1/993: symlink da/dd/d7a/d82/d98/d9e/l146 0 2024-03-24T00:49:12.555 INFO:tasks.workunit.client.0.smithi038.stdout:0/891: rename d0/d12/d15/d58/d64/da7/dbb/dd4/dea/f125 to d0/d12/f12c 0 2024-03-24T00:49:12.567 INFO:tasks.workunit.client.1.smithi088.stdout:7/937: truncate d3/ff 4105994 0 2024-03-24T00:49:12.567 INFO:tasks.workunit.client.1.smithi088.stdout:7/938: readlink d3/d31/l4e 0 2024-03-24T00:49:12.567 INFO:tasks.workunit.client.1.smithi088.stdout:7/939: chown d3/dbf/le0 7578396 1 2024-03-24T00:49:12.567 INFO:tasks.workunit.client.1.smithi088.stdout:9/989: sync 2024-03-24T00:49:12.567 INFO:tasks.workunit.client.1.smithi088.stdout:1/994: sync 2024-03-24T00:49:12.569 INFO:tasks.workunit.client.1.smithi088.stdout:7/940: symlink d3/d32/d34/d9c/d4b/d73/l125 0 2024-03-24T00:49:12.576 INFO:tasks.workunit.client.1.smithi088.stdout:7/941: rename d3/f24 to d3/d32/d34/d9c/d4b/d74/f126 0 2024-03-24T00:49:12.583 INFO:tasks.workunit.client.0.smithi038.stdout:0/892: dread d0/d12/d42/d9d/d28/f39 [0,4194304] 0 2024-03-24T00:49:12.586 INFO:tasks.workunit.client.0.smithi038.stdout:0/893: rename d0/d12/d15/d1a/la5 to d0/d12/d42/d9d/d28/d118/l12d 0 2024-03-24T00:49:12.586 INFO:tasks.workunit.client.0.smithi038.stdout:0/894: write d0/d12/d42/d9d/d28/f48 [2955101,35827] 0 2024-03-24T00:49:12.592 INFO:tasks.workunit.client.0.smithi038.stdout:0/895: symlink d0/d14/de4/d100/l12e 0 2024-03-24T00:49:12.592 INFO:tasks.workunit.client.0.smithi038.stdout:0/896: write d0/d12/d15/d58/d64/da7/dbb/dcd/f10b [794739,126291] 0 2024-03-24T00:49:12.592 INFO:tasks.workunit.client.0.smithi038.stdout:0/897: stat d0/d1b/d2c/f86 0 2024-03-24T00:49:12.592 INFO:tasks.workunit.client.0.smithi038.stdout:8/983: dwrite d0/d45/d37/d5f/d68/f91 [0,4194304] 0 2024-03-24T00:49:12.593 INFO:tasks.workunit.client.0.smithi038.stdout:8/984: write d0/d1d/d20/d79/fcd [3805289,116201] 0 2024-03-24T00:49:12.596 INFO:tasks.workunit.client.0.smithi038.stdout:0/898: link d0/d1b/d2c/c4e d0/d12/d15/d58/c12f 0 2024-03-24T00:49:12.606 INFO:tasks.workunit.client.0.smithi038.stdout:6/999: dread d1/d37/d68/d145/fe2 [0,4194304] 0 2024-03-24T00:49:12.611 INFO:tasks.workunit.client.0.smithi038.stdout:0/899: rmdir d0/d14/de4 39 2024-03-24T00:49:12.611 INFO:tasks.workunit.client.0.smithi038.stdout:0/900: write d0/d14/d84/fd2 [1313341,114246] 0 2024-03-24T00:49:12.611 INFO:tasks.workunit.client.0.smithi038.stdout:0/901: truncate d0/d12/d9a/f9c 1961546 0 2024-03-24T00:49:12.612 INFO:tasks.workunit.client.0.smithi038.stdout:0/902: chown d0/d12/f68 3304121 1 2024-03-24T00:49:12.613 INFO:tasks.workunit.client.0.smithi038.stdout:0/903: mknod d0/d12/d15/d58/d64/c130 0 2024-03-24T00:49:12.613 INFO:tasks.workunit.client.0.smithi038.stdout:0/904: write d0/d14/d88/db8/f116 [716771,6515] 0 2024-03-24T00:49:12.616 INFO:tasks.workunit.client.0.smithi038.stdout:3/983: dwrite d3/d30/d70/d90/f14e [0,4194304] 0 2024-03-24T00:49:12.616 INFO:tasks.workunit.client.0.smithi038.stdout:3/984: chown d3/d9/dc/da6/db5/l8e 6924 1 2024-03-24T00:49:12.617 INFO:tasks.workunit.client.0.smithi038.stdout:3/985: write d3/d5a/fc8 [196866,1416] 0 2024-03-24T00:49:12.624 INFO:tasks.workunit.client.1.smithi088.stdout:7/942: dread d3/d32/d76/f81 [0,4194304] 0 2024-03-24T00:49:12.624 INFO:tasks.workunit.client.1.smithi088.stdout:7/943: stat d3/d32/d34/d9c/d4b/da7/dc5/f107 0 2024-03-24T00:49:12.625 INFO:tasks.workunit.client.1.smithi088.stdout:1/995: dwrite d6/d23/f29 [4194304,4194304] 0 2024-03-24T00:49:12.626 INFO:tasks.workunit.client.0.smithi038.stdout:1/994: dwrite f7 [0,4194304] 0 2024-03-24T00:49:12.627 INFO:tasks.workunit.client.0.smithi038.stdout:1/995: chown da/d3c/l1d 3952 1 2024-03-24T00:49:12.627 INFO:tasks.workunit.client.1.smithi088.stdout:7/944: rename d3/d32/d34/d9c/d4b/da7/dc5/d101/l10e to d3/d32/d34/d9c/d4b/d73/l127 0 2024-03-24T00:49:12.627 INFO:tasks.workunit.client.1.smithi088.stdout:7/945: write d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/fb6 [288537,70624] 0 2024-03-24T00:49:12.629 INFO:tasks.workunit.client.1.smithi088.stdout:1/996: mkdir d6/d27/d67/d88/d149 0 2024-03-24T00:49:12.631 INFO:tasks.workunit.client.1.smithi088.stdout:1/997: getdents d6/d6e/dfc/d108/d5b/d93 0 2024-03-24T00:49:12.633 INFO:tasks.workunit.client.1.smithi088.stdout:1/998: creat d6/d6e/dfc/d108/d5b/d5d/d61/d7d/d137/f14a x:0 0 0 2024-03-24T00:49:12.634 INFO:tasks.workunit.client.0.smithi038.stdout:0/905: dread d0/d12/d42/d44/f93 [0,4194304] 0 2024-03-24T00:49:12.634 INFO:tasks.workunit.client.0.smithi038.stdout:0/906: creat d0/d12/d15/d58/d64/da7/dbb/dcd/f131 x:0 0 0 2024-03-24T00:49:12.635 INFO:tasks.workunit.client.1.smithi088.stdout:1/999: truncate d6/d23/f34 666871 0 2024-03-24T00:49:12.637 INFO:tasks.workunit.client.0.smithi038.stdout:0/907: mkdir d0/d12/d42/d83/d132 0 2024-03-24T00:49:12.637 INFO:tasks.workunit.client.0.smithi038.stdout:2/847: sync 2024-03-24T00:49:12.637 INFO:tasks.workunit.client.0.smithi038.stdout:4/879: sync 2024-03-24T00:49:12.641 INFO:tasks.workunit.client.0.smithi038.stdout:8/985: dwrite d0/d1d/d20/d79/f104 [0,4194304] 0 2024-03-24T00:49:12.643 INFO:tasks.workunit.client.0.smithi038.stdout:4/880: creat da/d12/d37/f12e x:0 0 0 2024-03-24T00:49:12.648 INFO:tasks.workunit.client.1.smithi088.stdout:7/946: dread d3/d32/d34/d9c/d4b/f5e [0,4194304] 0 2024-03-24T00:49:12.652 INFO:tasks.workunit.client.0.smithi038.stdout:2/848: read d4/d18/d38/fc0 [1977945,40974] 0 2024-03-24T00:49:12.652 INFO:tasks.workunit.client.0.smithi038.stdout:8/986: unlink d0/d45/d66/f40 0 2024-03-24T00:49:12.652 INFO:tasks.workunit.client.0.smithi038.stdout:8/987: write d0/f76 [2766907,115233] 0 2024-03-24T00:49:12.652 INFO:tasks.workunit.client.0.smithi038.stdout:2/849: write d4/d18/d38/f82 [4815456,128719] 0 2024-03-24T00:49:12.652 INFO:tasks.workunit.client.0.smithi038.stdout:2/850: chown d4/d27/d8f/ffc 22646289 1 2024-03-24T00:49:12.654 INFO:tasks.workunit.client.0.smithi038.stdout:8/988: symlink d0/d45/d66/d6d/d106/d117/l141 0 2024-03-24T00:49:12.658 INFO:tasks.workunit.client.0.smithi038.stdout:4/881: getdents da/d12/d24/d5f/d8a/da4 0 2024-03-24T00:49:12.658 INFO:tasks.workunit.client.0.smithi038.stdout:4/882: chown da/d12/d24/d68 2 1 2024-03-24T00:49:12.660 INFO:tasks.workunit.client.0.smithi038.stdout:4/883: symlink da/d12/d24/d68/d6c/dce/l12f 0 2024-03-24T00:49:12.660 INFO:tasks.workunit.client.0.smithi038.stdout:4/884: rename da to da/d12/d24/d68/d130 22 2024-03-24T00:49:12.663 INFO:tasks.workunit.client.0.smithi038.stdout:0/908: dread d0/d14/fcf [0,4194304] 0 2024-03-24T00:49:12.663 INFO:tasks.workunit.client.0.smithi038.stdout:3/986: dwrite d3/d5a/fc5 [0,4194304] 0 2024-03-24T00:49:12.667 INFO:tasks.workunit.client.0.smithi038.stdout:0/909: creat d0/d12/f133 x:0 0 0 2024-03-24T00:49:12.667 INFO:tasks.workunit.client.0.smithi038.stdout:0/910: fdatasync d0/daf/fca 0 2024-03-24T00:49:12.668 INFO:tasks.workunit.client.0.smithi038.stdout:8/989: dread d0/d1d/d20/f60 [0,4194304] 0 2024-03-24T00:49:12.668 INFO:tasks.workunit.client.0.smithi038.stdout:0/911: unlink d0/d12/d42/d9d/d28/f48 0 2024-03-24T00:49:12.669 INFO:tasks.workunit.client.0.smithi038.stdout:8/990: truncate d0/d10/f92 4851046 0 2024-03-24T00:49:12.673 INFO:tasks.workunit.client.0.smithi038.stdout:3/987: dread d3/d9/dc/da6/db5/d19/d82/ffb [0,4194304] 0 2024-03-24T00:49:12.673 INFO:tasks.workunit.client.0.smithi038.stdout:8/991: rename d0/d1d/d27/d4d/cde to d0/d45/d37/c142 0 2024-03-24T00:49:12.673 INFO:tasks.workunit.client.0.smithi038.stdout:3/988: truncate d3/d9/dc/da6/db5/d19/d82/fba 899935 0 2024-03-24T00:49:12.673 INFO:tasks.workunit.client.0.smithi038.stdout:3/989: chown d3/d9/dc/da6 1663686 1 2024-03-24T00:49:12.674 INFO:tasks.workunit.client.0.smithi038.stdout:8/992: symlink d0/d1d/d27/d4d/daf/dea/l143 0 2024-03-24T00:49:12.676 INFO:tasks.workunit.client.0.smithi038.stdout:8/993: creat d0/d1d/d8c/d136/f144 x:0 0 0 2024-03-24T00:49:12.676 INFO:tasks.workunit.client.0.smithi038.stdout:8/994: chown d0/d1d/d20/f58 203 1 2024-03-24T00:49:12.676 INFO:tasks.workunit.client.0.smithi038.stdout:8/995: truncate d0/d1d/d27/d6b/da4/f11b 961578 0 2024-03-24T00:49:12.677 INFO:tasks.workunit.client.0.smithi038.stdout:3/990: mknod d3/d9/dc/da6/db5/d19/da4/c152 0 2024-03-24T00:49:12.679 INFO:tasks.workunit.client.0.smithi038.stdout:3/991: symlink d3/d5a/l153 0 2024-03-24T00:49:12.680 INFO:tasks.workunit.client.0.smithi038.stdout:3/992: rename d3/d9/dc/da6/db5/d19/d82/d89/ld6 to d3/d2a/l154 0 2024-03-24T00:49:12.686 INFO:tasks.workunit.client.1.smithi088.stdout:2/941: sync 2024-03-24T00:49:12.688 INFO:tasks.workunit.client.1.smithi088.stdout:2/942: mknod db/dd/d96/d4e/d8f/c12f 0 2024-03-24T00:49:12.696 INFO:tasks.workunit.client.1.smithi088.stdout:2/943: write db/dd/d96/d99/db2/ff5 [705917,72641] 0 2024-03-24T00:49:12.696 INFO:tasks.workunit.client.1.smithi088.stdout:2/944: chown db/dd/d96/lf1 7 1 2024-03-24T00:49:12.697 INFO:tasks.workunit.client.0.smithi038.stdout:8/996: dread d0/d45/f89 [0,4194304] 0 2024-03-24T00:49:12.697 INFO:tasks.workunit.client.0.smithi038.stdout:8/997: dread - d0/d45/d66/da6/fe8 zero size 2024-03-24T00:49:12.697 INFO:tasks.workunit.client.0.smithi038.stdout:8/998: dread - d0/d1d/d27/d4d/fd7 zero size 2024-03-24T00:49:12.698 INFO:tasks.workunit.client.0.smithi038.stdout:8/999: creat d0/d45/d66/d107/d127/f145 x:0 0 0 2024-03-24T00:49:12.699 INFO:tasks.workunit.client.0.smithi038.stdout:4/885: dwrite da/d12/d37/fe6 [4194304,4194304] 0 2024-03-24T00:49:12.700 INFO:tasks.workunit.client.0.smithi038.stdout:4/886: unlink da/d38/fa3 0 2024-03-24T00:49:12.701 INFO:tasks.workunit.client.1.smithi088.stdout:2/945: dread db/dd/d96/d99/db2/fb4 [0,4194304] 0 2024-03-24T00:49:12.701 INFO:tasks.workunit.client.1.smithi088.stdout:2/946: fsync db/dd/d96/d4e/d8f/fe4 0 2024-03-24T00:49:12.701 INFO:tasks.workunit.client.1.smithi088.stdout:2/947: readlink db/d21/lae 0 2024-03-24T00:49:12.701 INFO:tasks.workunit.client.1.smithi088.stdout:2/948: fdatasync db/dd/d96/f119 0 2024-03-24T00:49:12.701 INFO:tasks.workunit.client.0.smithi038.stdout:4/887: symlink da/d12/d24/d5f/l131 0 2024-03-24T00:49:12.704 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:12 smithi088 ceph-mon[36610]: pgmap v42: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 219 MiB/s rd, 247 MiB/s wr, 380 op/s 2024-03-24T00:49:12.704 INFO:tasks.workunit.client.0.smithi038.stdout:4/888: creat da/d38/f132 x:0 0 0 2024-03-24T00:49:12.704 INFO:tasks.workunit.client.0.smithi038.stdout:4/889: read - da/d12/d24/d68/d6c/d7c/f118 zero size 2024-03-24T00:49:12.705 INFO:tasks.workunit.client.1.smithi088.stdout:2/949: link db/dd4/c127 db/d21/c130 0 2024-03-24T00:49:12.705 INFO:tasks.workunit.client.1.smithi088.stdout:2/950: chown db/dd/d96/d99/db2/d123 520105 1 2024-03-24T00:49:12.705 INFO:tasks.workunit.client.0.smithi038.stdout:4/890: creat da/d12/d29/f133 x:0 0 0 2024-03-24T00:49:12.706 INFO:tasks.workunit.client.0.smithi038.stdout:4/891: readlink da/d12/d24/d5f/d8a/db6/dd1/ld6 0 2024-03-24T00:49:12.727 INFO:tasks.workunit.client.0.smithi038.stdout:4/892: write da/d12/d24/d68/f93 [2728808,91868] 0 2024-03-24T00:49:12.727 INFO:tasks.workunit.client.0.smithi038.stdout:4/893: chown da/d12/f5c 85034 1 2024-03-24T00:49:12.727 INFO:tasks.workunit.client.0.smithi038.stdout:4/894: getdents da/d12 0 2024-03-24T00:49:12.728 INFO:tasks.workunit.client.0.smithi038.stdout:4/895: fsync da/d12/d24/d5f/df8/f103 0 2024-03-24T00:49:12.752 INFO:tasks.workunit.client.1.smithi088.stdout:2/951: dwrite db/d21/d73/fc5 [0,4194304] 0 2024-03-24T00:49:12.752 INFO:tasks.workunit.client.0.smithi038.stdout:0/912: dwrite d0/d12/d92/fa2 [0,4194304] 0 2024-03-24T00:49:12.752 INFO:tasks.workunit.client.0.smithi038.stdout:3/993: dwrite d3/d9/dc/df/fdd [0,4194304] 0 2024-03-24T00:49:12.754 INFO:tasks.workunit.client.1.smithi088.stdout:2/952: rmdir db/d21/d73 39 2024-03-24T00:49:12.756 INFO:tasks.workunit.client.1.smithi088.stdout:2/953: rename db/dd/d96/d99/dc3/c10a to db/dd/d96/d99/db2/ddc/c131 0 2024-03-24T00:49:12.756 INFO:tasks.workunit.client.1.smithi088.stdout:2/954: chown db/dd/d96/d105 5128 1 2024-03-24T00:49:12.758 INFO:tasks.workunit.client.1.smithi088.stdout:7/947: sync 2024-03-24T00:49:12.758 INFO:tasks.workunit.client.1.smithi088.stdout:9/990: sync 2024-03-24T00:49:12.758 INFO:tasks.workunit.client.1.smithi088.stdout:9/991: chown d1/d10 60447590 1 2024-03-24T00:49:12.758 INFO:tasks.workunit.client.1.smithi088.stdout:9/992: write d1/d10/d29/d6d/f13b [866134,36299] 0 2024-03-24T00:49:12.759 INFO:tasks.workunit.client.0.smithi038.stdout:0/913: dread d0/d12/d15/d58/d64/ffa [0,4194304] 0 2024-03-24T00:49:12.760 INFO:tasks.workunit.client.1.smithi088.stdout:2/955: creat db/d21/d35/dcb/f132 x:0 0 0 2024-03-24T00:49:12.763 INFO:tasks.workunit.client.1.smithi088.stdout:9/993: rename d1/d10/d29/def/d136 to d1/d22/d141 0 2024-03-24T00:49:12.763 INFO:tasks.workunit.client.1.smithi088.stdout:9/994: chown d1/d22/d27/c31 2129069497 1 2024-03-24T00:49:12.764 INFO:tasks.workunit.client.0.smithi038.stdout:0/914: symlink d0/d12/d15/d58/d64/l134 0 2024-03-24T00:49:12.766 INFO:tasks.workunit.client.1.smithi088.stdout:9/995: creat d1/d10/d29/db0/f142 x:0 0 0 2024-03-24T00:49:12.767 INFO:tasks.workunit.client.0.smithi038.stdout:3/994: write d3/d9/f16 [1571880,4308] 0 2024-03-24T00:49:12.767 INFO:tasks.workunit.client.0.smithi038.stdout:0/915: mknod d0/d14/d84/d119/c135 0 2024-03-24T00:49:12.769 INFO:tasks.workunit.client.0.smithi038.stdout:3/995: mkdir d3/d9/d77/dd4/d155 0 2024-03-24T00:49:12.771 INFO:tasks.workunit.client.0.smithi038.stdout:3/996: mknod d3/d9/dc/da6/db5/d19/dd2/c156 0 2024-03-24T00:49:12.772 INFO:tasks.workunit.client.1.smithi088.stdout:9/996: getdents d1/d10/d77/dc0 0 2024-03-24T00:49:12.772 INFO:tasks.workunit.client.1.smithi088.stdout:9/997: creat d1/d10/d29/db0/f143 x:0 0 0 2024-03-24T00:49:12.772 INFO:tasks.workunit.client.1.smithi088.stdout:9/998: write d1/d10/d9b/dab/d7a/fd9 [1881212,118628] 0 2024-03-24T00:49:12.774 INFO:tasks.workunit.client.0.smithi038.stdout:4/896: dwrite da/d38/fd3 [0,4194304] 0 2024-03-24T00:49:12.776 INFO:tasks.workunit.client.0.smithi038.stdout:4/897: mkdir da/d12/d37/dd/d15/d6d/d8f/df0/d121/d134 0 2024-03-24T00:49:12.777 INFO:tasks.workunit.client.0.smithi038.stdout:3/997: dread d3/d9/f96 [0,4194304] 0 2024-03-24T00:49:12.780 INFO:tasks.workunit.client.0.smithi038.stdout:4/898: mknod da/d12/d37/dd/d15/d6d/d8f/df0/d121/c135 0 2024-03-24T00:49:12.781 INFO:tasks.workunit.client.0.smithi038.stdout:0/916: rename d0/d12/d15/d58 to d0/d14/d84/d119/d136 0 2024-03-24T00:49:12.781 INFO:tasks.workunit.client.0.smithi038.stdout:0/917: dread - d0/d14/d88/db8/f124 zero size 2024-03-24T00:49:12.788 INFO:tasks.workunit.client.0.smithi038.stdout:0/918: link d0/d12/d15/d1a/c6a d0/c137 0 2024-03-24T00:49:12.788 INFO:tasks.workunit.client.0.smithi038.stdout:0/919: fsync d0/d12/d42/d44/f93 0 2024-03-24T00:49:12.789 INFO:tasks.workunit.client.0.smithi038.stdout:0/920: truncate d0/d14/d84/d119/d136/d64/f7d 4179971 0 2024-03-24T00:49:12.789 INFO:tasks.workunit.client.0.smithi038.stdout:0/921: dread - d0/d14/d88/db8/fed zero size 2024-03-24T00:49:12.790 INFO:tasks.workunit.client.0.smithi038.stdout:4/899: dread da/d12/d24/d5f/f75 [0,4194304] 0 2024-03-24T00:49:12.791 INFO:tasks.workunit.client.0.smithi038.stdout:4/900: write da/d12/d37/dd/f8d [796396,119012] 0 2024-03-24T00:49:12.793 INFO:tasks.workunit.client.0.smithi038.stdout:4/901: rename da/d12/d24/d68/d6c/dce to da/d12/d29/d10d/d136 0 2024-03-24T00:49:12.797 INFO:tasks.workunit.client.0.smithi038.stdout:4/902: mknod da/d12/d37/dd/c137 0 2024-03-24T00:49:12.798 INFO:tasks.workunit.client.0.smithi038.stdout:4/903: mknod da/d12/d24/d5f/d8a/db6/dd1/c138 0 2024-03-24T00:49:12.800 INFO:tasks.workunit.client.0.smithi038.stdout:4/904: unlink da/d12/c52 0 2024-03-24T00:49:12.818 INFO:tasks.workunit.client.1.smithi088.stdout:2/956: dwrite db/d21/d35/d49/f71 [0,4194304] 0 2024-03-24T00:49:12.823 INFO:tasks.workunit.client.1.smithi088.stdout:2/957: rename db/d21/d73 to db/dd/d96/d99/d133 0 2024-03-24T00:49:12.823 INFO:tasks.workunit.client.1.smithi088.stdout:2/958: truncate db/dd/d96/d99/db2/ff2 581663 0 2024-03-24T00:49:12.832 INFO:tasks.workunit.client.0.smithi038.stdout:0/922: dread d0/d1b/d2c/f3b [0,4194304] 0 2024-03-24T00:49:12.832 INFO:tasks.workunit.client.0.smithi038.stdout:4/905: write da/d12/d37/dd/f95 [159557,22756] 0 2024-03-24T00:49:12.832 INFO:tasks.workunit.client.0.smithi038.stdout:4/906: stat da/d12/d24/d68/d6c/d7c 0 2024-03-24T00:49:12.832 INFO:tasks.workunit.client.0.smithi038.stdout:4/907: creat da/d12/d29/d5b/d73/db4/f139 x:0 0 0 2024-03-24T00:49:12.832 INFO:tasks.workunit.client.0.smithi038.stdout:4/908: truncate da/f1e 1382357 0 2024-03-24T00:49:12.833 INFO:tasks.workunit.client.0.smithi038.stdout:4/909: chown da/d12/d37/dd/l20 19302 1 2024-03-24T00:49:12.833 INFO:tasks.workunit.client.0.smithi038.stdout:0/923: mkdir d0/d14/de4/d100/d138 0 2024-03-24T00:49:12.838 INFO:tasks.workunit.client.0.smithi038.stdout:4/910: rmdir da/d12/d24/d5f/d108 0 2024-03-24T00:49:12.839 INFO:tasks.workunit.client.0.smithi038.stdout:3/998: dwrite d3/d9/dc/da6/db5/d19/d82/d89/da8/f100 [8388608,4194304] 0 2024-03-24T00:49:12.839 INFO:tasks.workunit.client.0.smithi038.stdout:0/924: mkdir d0/d12/d42/d5e/d11f/d139 0 2024-03-24T00:49:12.840 INFO:tasks.workunit.client.0.smithi038.stdout:3/999: creat d3/d9/dc/d51/f157 x:0 0 0 2024-03-24T00:49:12.840 INFO:tasks.workunit.client.1.smithi088.stdout:2/959: dread db/d21/f44 [0,4194304] 0 2024-03-24T00:49:12.841 INFO:tasks.workunit.client.0.smithi038.stdout:0/925: truncate d0/d1b/f7c 1279632 0 2024-03-24T00:49:12.844 INFO:tasks.workunit.client.0.smithi038.stdout:0/926: mknod d0/d12/d42/d9d/d28/c13a 0 2024-03-24T00:49:12.844 INFO:tasks.workunit.client.0.smithi038.stdout:0/927: truncate d0/d14/de4/f11a 130795 0 2024-03-24T00:49:12.852 INFO:tasks.workunit.client.0.smithi038.stdout:4/911: dread da/d12/d37/dd/f8d [0,4194304] 0 2024-03-24T00:49:12.853 INFO:tasks.workunit.client.0.smithi038.stdout:4/912: truncate da/f7d 3388955 0 2024-03-24T00:49:12.853 INFO:tasks.workunit.client.0.smithi038.stdout:4/913: chown da/d12/d24/d5f/d9a/f92 721514735 1 2024-03-24T00:49:12.853 INFO:tasks.workunit.client.0.smithi038.stdout:4/914: readlink da/d12/d37/dd/d15/d6d/d8f/l128 0 2024-03-24T00:49:12.853 INFO:tasks.workunit.client.0.smithi038.stdout:4/915: fsync da/d12/d37/d83/dac/fad 0 2024-03-24T00:49:12.854 INFO:tasks.workunit.client.1.smithi088.stdout:2/960: write db/dd/d96/d99/db2/f65 [738562,110134] 0 2024-03-24T00:49:12.858 INFO:tasks.workunit.client.1.smithi088.stdout:2/961: symlink db/d21/d35/d49/l134 0 2024-03-24T00:49:12.859 INFO:tasks.workunit.client.0.smithi038.stdout:4/916: creat da/d12/d37/d83/f13a x:0 0 0 2024-03-24T00:49:12.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:12 smithi038 ceph-mon[29271]: pgmap v42: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 219 MiB/s rd, 247 MiB/s wr, 380 op/s 2024-03-24T00:49:12.861 INFO:tasks.workunit.client.0.smithi038.stdout:4/917: mknod da/d12/d29/d5b/d73/db4/dd8/c13b 0 2024-03-24T00:49:12.862 INFO:tasks.workunit.client.1.smithi088.stdout:2/962: truncate db/d21/fb6 495222 0 2024-03-24T00:49:12.863 INFO:tasks.workunit.client.0.smithi038.stdout:0/928: read d0/d14/fb5 [1710695,25214] 0 2024-03-24T00:49:12.863 INFO:tasks.workunit.client.0.smithi038.stdout:0/929: chown d0/d14/d84/d119/d136/lf8 544927 1 2024-03-24T00:49:12.864 INFO:tasks.workunit.client.0.smithi038.stdout:4/918: rename da/d12/d24/d5f/d8a/db6/dd1/c138 to da/d12/c13c 0 2024-03-24T00:49:12.866 INFO:tasks.workunit.client.1.smithi088.stdout:2/963: creat db/dd/d96/d4e/d8f/f135 x:0 0 0 2024-03-24T00:49:12.869 INFO:tasks.workunit.client.0.smithi038.stdout:4/919: link da/d12/d37/dd/d15/d4c/d70/ff5 da/d12/d29/d5b/d73/db4/dd8/f13d 0 2024-03-24T00:49:12.871 INFO:tasks.workunit.client.1.smithi088.stdout:2/964: truncate db/dd/dc9/d100/f10e 2112130 0 2024-03-24T00:49:12.881 INFO:tasks.workunit.client.1.smithi088.stdout:9/999: sync 2024-03-24T00:49:12.881 INFO:tasks.workunit.client.1.smithi088.stdout:7/948: sync 2024-03-24T00:49:12.882 INFO:tasks.workunit.client.0.smithi038.stdout:4/920: read da/d27/f69 [1162409,110622] 0 2024-03-24T00:49:12.882 INFO:tasks.workunit.client.0.smithi038.stdout:4/921: fdatasync da/d12/d37/f99 0 2024-03-24T00:49:12.884 INFO:tasks.workunit.client.0.smithi038.stdout:4/922: truncate da/d12/d37/dd/d15/f17 278253 0 2024-03-24T00:49:12.917 INFO:tasks.workunit.client.0.smithi038.stdout:4/923: dread da/d12/d37/dd/f21 [4194304,4194304] 0 2024-03-24T00:49:12.919 INFO:tasks.workunit.client.0.smithi038.stdout:0/930: dwrite d0/d14/d84/d119/d136/d64/da7/dbb/dcd/f131 [0,4194304] 0 2024-03-24T00:49:12.919 INFO:tasks.workunit.client.0.smithi038.stdout:4/924: creat da/d12/d37/dd/d15/d6d/d8f/df0/d121/f13e x:0 0 0 2024-03-24T00:49:12.921 INFO:tasks.workunit.client.0.smithi038.stdout:0/931: rmdir d0/d14/d84/d119/d136/d64/da7/dbb/dd4 39 2024-03-24T00:49:12.922 INFO:tasks.workunit.client.0.smithi038.stdout:4/925: truncate da/d12/d37/fe6 2327837 0 2024-03-24T00:49:12.922 INFO:tasks.workunit.client.0.smithi038.stdout:4/926: fdatasync da/d12/d37/dd/d15/d6d/d8f/f12c 0 2024-03-24T00:49:12.924 INFO:tasks.workunit.client.0.smithi038.stdout:0/932: link d0/d14/d84/d119/d136/d64/da7/dbb/dd4/ce6 d0/d12/d42/d5e/d11f/d139/c13b 0 2024-03-24T00:49:12.924 INFO:tasks.workunit.client.0.smithi038.stdout:0/933: chown d0/f2 1692 1 2024-03-24T00:49:12.925 INFO:tasks.workunit.client.0.smithi038.stdout:4/927: link da/d12/d37/dd/d15/d6d/d8f/fbb da/d12/d37/dd/d15/d4c/f13f 0 2024-03-24T00:49:12.928 INFO:tasks.workunit.client.0.smithi038.stdout:4/928: creat da/d12/d29/d5b/d73/f140 x:0 0 0 2024-03-24T00:49:12.930 INFO:tasks.workunit.client.0.smithi038.stdout:7/887: sync 2024-03-24T00:49:12.930 INFO:tasks.workunit.client.0.smithi038.stdout:5/879: sync 2024-03-24T00:49:12.930 INFO:tasks.workunit.client.0.smithi038.stdout:5/880: fdatasync d6/d60/f9a 0 2024-03-24T00:49:12.931 INFO:tasks.workunit.client.0.smithi038.stdout:1/996: sync 2024-03-24T00:49:12.931 INFO:tasks.workunit.client.0.smithi038.stdout:2/851: sync 2024-03-24T00:49:12.931 INFO:tasks.workunit.client.0.smithi038.stdout:2/852: rename d4 to d4/da3/d118/de8/d124 22 2024-03-24T00:49:12.932 INFO:tasks.workunit.client.0.smithi038.stdout:2/853: stat d4/d55/d88/lc1 0 2024-03-24T00:49:12.932 INFO:tasks.workunit.client.0.smithi038.stdout:2/854: chown d4/d18/d38/d63/f122 28 1 2024-03-24T00:49:12.932 INFO:tasks.workunit.client.0.smithi038.stdout:2/855: dread - d4/d27/d2f/d30/df8/f107 zero size 2024-03-24T00:49:12.932 INFO:tasks.workunit.client.0.smithi038.stdout:1/997: getdents da/dd/d7a/d82 0 2024-03-24T00:49:12.932 INFO:tasks.workunit.client.0.smithi038.stdout:1/998: dread - da/dd/d22/dab/f139 zero size 2024-03-24T00:49:12.932 INFO:tasks.workunit.client.0.smithi038.stdout:4/929: mknod da/d12/d24/c141 0 2024-03-24T00:49:12.942 INFO:tasks.workunit.client.1.smithi088.stdout:7/949: dwrite d3/fe5 [0,4194304] 0 2024-03-24T00:49:12.949 INFO:tasks.workunit.client.0.smithi038.stdout:7/888: creat d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/f143 x:0 0 0 2024-03-24T00:49:12.949 INFO:tasks.workunit.client.0.smithi038.stdout:4/930: mkdir da/d12/d37/dd/d15/d142 0 2024-03-24T00:49:12.949 INFO:tasks.workunit.client.0.smithi038.stdout:5/881: creat d6/d19/d20/d42/d43/d4c/d6d/f11f x:0 0 0 2024-03-24T00:49:12.949 INFO:tasks.workunit.client.0.smithi038.stdout:2/856: truncate d4/d27/f91 4180761 0 2024-03-24T00:49:12.949 INFO:tasks.workunit.client.0.smithi038.stdout:5/882: write d6/f9 [3185183,86659] 0 2024-03-24T00:49:12.950 INFO:tasks.workunit.client.0.smithi038.stdout:0/934: dread d0/d14/d84/d119/d136/d64/da7/dbb/dd4/dea/fbe [0,4194304] 0 2024-03-24T00:49:12.950 INFO:tasks.workunit.client.0.smithi038.stdout:2/857: write d4/da3/d118/d11f/f10c [466067,107918] 0 2024-03-24T00:49:12.950 INFO:tasks.workunit.client.0.smithi038.stdout:0/935: chown d0/d1b/l56 14 1 2024-03-24T00:49:12.950 INFO:tasks.workunit.client.0.smithi038.stdout:0/936: write d0/d14/fb5 [280442,114181] 0 2024-03-24T00:49:12.950 INFO:tasks.workunit.client.0.smithi038.stdout:4/931: mkdir da/d12/d24/d68/d6c/d7c/d143 0 2024-03-24T00:49:12.950 INFO:tasks.workunit.client.0.smithi038.stdout:4/932: readlink da/d12/l36 0 2024-03-24T00:49:12.950 INFO:tasks.workunit.client.0.smithi038.stdout:2/858: rename d4/f66 to d4/d18/f125 0 2024-03-24T00:49:12.952 INFO:tasks.workunit.client.0.smithi038.stdout:0/937: creat d0/d12/d42/d5e/d11f/f13c x:0 0 0 2024-03-24T00:49:12.952 INFO:tasks.workunit.client.0.smithi038.stdout:0/938: chown d0/d14/de4 485820 1 2024-03-24T00:49:12.952 INFO:tasks.workunit.client.0.smithi038.stdout:0/939: chown d0/d12/d42/d83/fb1 100782372 1 2024-03-24T00:49:12.952 INFO:tasks.workunit.client.0.smithi038.stdout:0/940: readlink d0/d14/d84/d119/d136/d64/da7/dbb/dd4/ldc 0 2024-03-24T00:49:12.952 INFO:tasks.workunit.client.0.smithi038.stdout:0/941: chown d0/d1b/f87 53103134 1 2024-03-24T00:49:12.955 INFO:tasks.workunit.client.0.smithi038.stdout:4/933: mknod da/d12/d37/dd/d15/df1/c144 0 2024-03-24T00:49:12.956 INFO:tasks.workunit.client.0.smithi038.stdout:2/859: mkdir d4/d27/d54/dab/d126 0 2024-03-24T00:49:12.963 INFO:tasks.workunit.client.0.smithi038.stdout:4/934: rename da/d12/d37/d83/f13a to da/d12/d24/d68/d6c/da6/f145 0 2024-03-24T00:49:12.964 INFO:tasks.workunit.client.0.smithi038.stdout:2/860: dread d4/d27/d54/d53/fba [0,4194304] 0 2024-03-24T00:49:12.968 INFO:tasks.workunit.client.1.smithi088.stdout:7/950: write d3/d32/d34/d9c/d4b/d74/d79/fa1 [904890,49472] 0 2024-03-24T00:49:12.970 INFO:tasks.workunit.client.0.smithi038.stdout:7/889: dread d2/d9/da/dc/f48 [0,4194304] 0 2024-03-24T00:49:12.970 INFO:tasks.workunit.client.0.smithi038.stdout:4/935: unlink da/d12/d24/d5f/d9a/f119 0 2024-03-24T00:49:12.973 INFO:tasks.workunit.client.0.smithi038.stdout:7/890: creat d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/f144 x:0 0 0 2024-03-24T00:49:12.975 INFO:tasks.workunit.client.0.smithi038.stdout:4/936: rename da/d12/d29/d5b/ce2 to da/d12/d24/d68/d6c/d7c/c146 0 2024-03-24T00:49:12.977 INFO:tasks.workunit.client.0.smithi038.stdout:4/937: mkdir da/d12/d24/d147 0 2024-03-24T00:49:12.979 INFO:tasks.workunit.client.0.smithi038.stdout:4/938: unlink da/d12/f3f 0 2024-03-24T00:49:12.982 INFO:tasks.workunit.client.0.smithi038.stdout:4/939: link da/d12/d37/d83/dac/cb0 da/d12/d29/d5b/d73/c148 0 2024-03-24T00:49:12.984 INFO:tasks.workunit.client.0.smithi038.stdout:2/861: write d4/d55/fb2 [2384028,80792] 0 2024-03-24T00:49:12.987 INFO:tasks.workunit.client.0.smithi038.stdout:2/862: link d4/d27/d54/d67/lbd d4/d18/d5b/df2/l127 0 2024-03-24T00:49:12.987 INFO:tasks.workunit.client.0.smithi038.stdout:2/863: chown d4/d55/lcb 142 1 2024-03-24T00:49:12.987 INFO:tasks.workunit.client.0.smithi038.stdout:2/864: chown d4/d18/d26/c35 3280586 1 2024-03-24T00:49:12.987 INFO:tasks.workunit.client.0.smithi038.stdout:2/865: chown d4/d18/f11c 8871424 1 2024-03-24T00:49:12.987 INFO:tasks.workunit.client.0.smithi038.stdout:2/866: write d4/d18/d38/f57 [4233092,52375] 0 2024-03-24T00:49:12.993 INFO:tasks.workunit.client.0.smithi038.stdout:2/867: rename d4/da3/d118/l4d to d4/d27/d54/d53/de2/l128 0 2024-03-24T00:49:12.993 INFO:tasks.workunit.client.0.smithi038.stdout:2/868: chown d4/d27/d2f/ld6 167247614 1 2024-03-24T00:49:12.993 INFO:tasks.workunit.client.0.smithi038.stdout:2/869: fdatasync d4/d18/d26/f62 0 2024-03-24T00:49:12.993 INFO:tasks.workunit.client.0.smithi038.stdout:2/870: dread - d4/d18/d5b/f113 zero size 2024-03-24T00:49:12.995 INFO:tasks.workunit.client.0.smithi038.stdout:2/871: mknod d4/d27/d54/d67/c129 0 2024-03-24T00:49:13.000 INFO:tasks.workunit.client.0.smithi038.stdout:2/872: mkdir d4/d18/d26/d12a 0 2024-03-24T00:49:13.002 INFO:tasks.workunit.client.0.smithi038.stdout:2/873: rmdir d4/d18/d26/d117 39 2024-03-24T00:49:13.005 INFO:tasks.workunit.client.0.smithi038.stdout:2/874: getdents d4/d27/d2f 0 2024-03-24T00:49:13.007 INFO:tasks.workunit.client.1.smithi088.stdout:2/965: sync 2024-03-24T00:49:13.007 INFO:tasks.workunit.client.1.smithi088.stdout:2/966: write db/dd/d96/d99/f10c [213349,73211] 0 2024-03-24T00:49:13.008 INFO:tasks.workunit.client.0.smithi038.stdout:1/999: dwrite f7 [0,4194304] 0 2024-03-24T00:49:13.013 INFO:tasks.workunit.client.0.smithi038.stdout:2/875: dread d4/fdd [0,4194304] 0 2024-03-24T00:49:13.013 INFO:tasks.workunit.client.0.smithi038.stdout:2/876: readlink d4/dc7/lc8 0 2024-03-24T00:49:13.027 INFO:tasks.workunit.client.0.smithi038.stdout:5/883: dwrite d6/d1e/d97/db0/fe0 [0,4194304] 0 2024-03-24T00:49:13.035 INFO:tasks.workunit.client.0.smithi038.stdout:0/942: dwrite d0/d14/de4/f11a [0,4194304] 0 2024-03-24T00:49:13.036 INFO:tasks.workunit.client.1.smithi088.stdout:7/951: dwrite d3/d31/f6e [0,4194304] 0 2024-03-24T00:49:13.036 INFO:tasks.workunit.client.1.smithi088.stdout:7/952: chown d3/d32/d34/d9c/d4b/f6c 24608557 1 2024-03-24T00:49:13.036 INFO:tasks.workunit.client.1.smithi088.stdout:7/953: write d3/d32/d34/d9c/d4b/d74/d79/d7f/d89/f11f [710163,54019] 0 2024-03-24T00:49:13.042 INFO:tasks.workunit.client.1.smithi088.stdout:7/954: mkdir d3/d32/d34/d9c/d4b/d73/dab/dc3/d128 0 2024-03-24T00:49:13.044 INFO:tasks.workunit.client.0.smithi038.stdout:7/891: dwrite d2/d2c/d9a/fd0 [0,4194304] 0 2024-03-24T00:49:13.044 INFO:tasks.workunit.client.0.smithi038.stdout:7/892: creat d2/d2c/d9a/f145 x:0 0 0 2024-03-24T00:49:13.050 INFO:tasks.workunit.client.0.smithi038.stdout:7/893: creat d2/d9/d16/dfc/f146 x:0 0 0 2024-03-24T00:49:13.051 INFO:tasks.workunit.client.0.smithi038.stdout:7/894: link d2/d9/da/d23/d2b/d5b/d72/d92/c6a d2/d9/da/d23/d2b/d5b/d72/d92/d5d/c147 0 2024-03-24T00:49:13.053 INFO:tasks.workunit.client.0.smithi038.stdout:5/884: write d6/f18 [1423117,81911] 0 2024-03-24T00:49:13.055 INFO:tasks.workunit.client.0.smithi038.stdout:5/885: mkdir d6/d60/d120 0 2024-03-24T00:49:13.055 INFO:tasks.workunit.client.0.smithi038.stdout:5/886: fdatasync d6/d16/d10f/fdd 0 2024-03-24T00:49:13.056 INFO:tasks.workunit.client.0.smithi038.stdout:5/887: mkdir d6/d16/d67/d121 0 2024-03-24T00:49:13.056 INFO:tasks.workunit.client.0.smithi038.stdout:5/888: fdatasync d6/d16/d10f/fdd 0 2024-03-24T00:49:13.058 INFO:tasks.workunit.client.0.smithi038.stdout:5/889: rename d6/d16/d21/d2d/da6 to d6/d1e/d109/d122 0 2024-03-24T00:49:13.062 INFO:tasks.workunit.client.0.smithi038.stdout:5/890: rmdir d6/d19 39 2024-03-24T00:49:13.062 INFO:tasks.workunit.client.0.smithi038.stdout:5/891: mknod d6/d1e/d97/c123 0 2024-03-24T00:49:13.062 INFO:tasks.workunit.client.0.smithi038.stdout:5/892: getdents d6/d19/d20/d42 0 2024-03-24T00:49:13.076 INFO:tasks.workunit.client.1.smithi088.stdout:7/955: dread d3/d32/d34/d9c/d4b/f6c [0,4194304] 0 2024-03-24T00:49:13.076 INFO:tasks.workunit.client.1.smithi088.stdout:7/956: fdatasync d3/d32/d34/d9c/d4b/ddc/f124 0 2024-03-24T00:49:13.077 INFO:tasks.workunit.client.1.smithi088.stdout:7/957: truncate d3/d32/d34/d9c/d4b/d73/dab/fb3 2545860 0 2024-03-24T00:49:13.078 INFO:tasks.workunit.client.1.smithi088.stdout:7/958: symlink d3/dbf/l129 0 2024-03-24T00:49:13.081 INFO:tasks.workunit.client.0.smithi038.stdout:5/893: dread d6/d19/d20/d42/d43/d4c/f74 [0,4194304] 0 2024-03-24T00:49:13.081 INFO:tasks.workunit.client.0.smithi038.stdout:4/940: dwrite da/d12/d37/dd/f95 [0,4194304] 0 2024-03-24T00:49:13.081 INFO:tasks.workunit.client.1.smithi088.stdout:2/967: dwrite db/dd/fba [0,4194304] 0 2024-03-24T00:49:13.082 INFO:tasks.workunit.client.0.smithi038.stdout:7/895: write d2/d9/da/d23/d2b/d5b/d72/d92/f29 [2306731,66951] 0 2024-03-24T00:49:13.082 INFO:tasks.workunit.client.0.smithi038.stdout:2/877: dwrite d4/d27/d54/d53/f64 [0,4194304] 0 2024-03-24T00:49:13.082 INFO:tasks.workunit.client.0.smithi038.stdout:7/896: fdatasync d2/d2c/f32 0 2024-03-24T00:49:13.084 INFO:tasks.workunit.client.1.smithi088.stdout:2/968: creat db/dd/d96/d99/dc3/f136 x:0 0 0 2024-03-24T00:49:13.088 INFO:tasks.workunit.client.0.smithi038.stdout:4/941: unlink da/d12/d24/d68/d6c/d7c/ccd 0 2024-03-24T00:49:13.089 INFO:tasks.workunit.client.0.smithi038.stdout:4/942: chown da/f1d 5 1 2024-03-24T00:49:13.089 INFO:tasks.workunit.client.0.smithi038.stdout:4/943: chown da/d12/d37/l7b 5 1 2024-03-24T00:49:13.092 INFO:tasks.workunit.client.0.smithi038.stdout:7/897: mkdir d2/d2c/d9a/dcb/d132/d148 0 2024-03-24T00:49:13.094 INFO:tasks.workunit.client.0.smithi038.stdout:4/944: mknod da/d12/d37/dd/d15/d6d/d8f/df0/d121/c149 0 2024-03-24T00:49:13.094 INFO:tasks.workunit.client.0.smithi038.stdout:4/945: fsync da/fa2 0 2024-03-24T00:49:13.095 INFO:tasks.workunit.client.0.smithi038.stdout:4/946: creat da/d12/d37/d83/dac/dcf/f14a x:0 0 0 2024-03-24T00:49:13.098 INFO:tasks.workunit.client.1.smithi088.stdout:2/969: mknod db/d21/c137 0 2024-03-24T00:49:13.107 INFO:tasks.workunit.client.0.smithi038.stdout:7/898: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d5d/c149 0 2024-03-24T00:49:13.114 INFO:tasks.workunit.client.0.smithi038.stdout:5/894: dread d6/d16/d67/df6/f114 [0,4194304] 0 2024-03-24T00:49:13.114 INFO:tasks.workunit.client.0.smithi038.stdout:5/895: chown d6/d19/d20/d64/d6b/d77/f116 6991 1 2024-03-24T00:49:13.118 INFO:tasks.workunit.client.1.smithi088.stdout:2/970: write db/d21/f33 [2840562,118379] 0 2024-03-24T00:49:13.118 INFO:tasks.workunit.client.1.smithi088.stdout:2/971: readlink db/dd/d96/d99/dc3/ld1 0 2024-03-24T00:49:13.125 INFO:tasks.workunit.client.0.smithi038.stdout:0/943: dwrite d0/d14/d88/db8/f116 [0,4194304] 0 2024-03-24T00:49:13.129 INFO:tasks.workunit.client.0.smithi038.stdout:7/899: write d2/d9/da/d23/d2b/d5b/d72/d92/d75/fd2 [2358901,63393] 0 2024-03-24T00:49:13.132 INFO:tasks.workunit.client.0.smithi038.stdout:0/944: symlink d0/d14/de4/d100/d138/l13d 0 2024-03-24T00:49:13.133 INFO:tasks.workunit.client.0.smithi038.stdout:5/896: dread d6/f9 [0,4194304] 0 2024-03-24T00:49:13.136 INFO:tasks.workunit.client.0.smithi038.stdout:7/900: rename d2/d9/da/dc/d8a/db8/cdb to d2/d9/da/d23/d2b/d5b/d72/d92/d130/c14a 0 2024-03-24T00:49:13.136 INFO:tasks.workunit.client.0.smithi038.stdout:7/901: write d2/d2c/f74 [4764909,2075] 0 2024-03-24T00:49:13.136 INFO:tasks.workunit.client.1.smithi088.stdout:2/972: dread db/dd/d96/d4e/fa0 [0,4194304] 0 2024-03-24T00:49:13.137 INFO:tasks.workunit.client.1.smithi088.stdout:2/973: chown db/dd/c47 15 1 2024-03-24T00:49:13.137 INFO:tasks.workunit.client.1.smithi088.stdout:2/974: chown db/dd 6981717 1 2024-03-24T00:49:13.137 INFO:tasks.workunit.client.1.smithi088.stdout:2/975: truncate db/d21/d35/f9e 4286179 0 2024-03-24T00:49:13.137 INFO:tasks.workunit.client.0.smithi038.stdout:0/945: link d0/d1b/c12a d0/d12/d42/d9d/d28/d118/c13e 0 2024-03-24T00:49:13.138 INFO:tasks.workunit.client.0.smithi038.stdout:5/897: rmdir d6/d19/d20/d64 39 2024-03-24T00:49:13.138 INFO:tasks.workunit.client.0.smithi038.stdout:5/898: chown d6/d19/d20/d42/d43/d68/c7d 1483592 1 2024-03-24T00:49:13.138 INFO:tasks.workunit.client.0.smithi038.stdout:2/878: dwrite d4/d27/d54/d53/f95 [0,4194304] 0 2024-03-24T00:49:13.139 INFO:tasks.workunit.client.1.smithi088.stdout:2/976: unlink db/d21/c130 0 2024-03-24T00:49:13.139 INFO:tasks.workunit.client.1.smithi088.stdout:7/959: dwrite d3/d32/d34/d9c/d4b/f60 [0,4194304] 0 2024-03-24T00:49:13.140 INFO:tasks.workunit.client.1.smithi088.stdout:7/960: write d3/d32/d34/d9c/d4b/f4c [4426631,36557] 0 2024-03-24T00:49:13.140 INFO:tasks.workunit.client.1.smithi088.stdout:7/961: fdatasync d3/d32/d76/df3/d103/f105 0 2024-03-24T00:49:13.141 INFO:tasks.workunit.client.1.smithi088.stdout:7/962: fsync d3/d32/d34/d9c/f61 0 2024-03-24T00:49:13.141 INFO:tasks.workunit.client.0.smithi038.stdout:2/879: write d4/d27/d8f/d9e/fa2 [342757,117131] 0 2024-03-24T00:49:13.141 INFO:tasks.workunit.client.0.smithi038.stdout:7/902: link d2/d9/da/d23/d2b/d5b/d72/d92/d33/cdc d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/d10b/c14b 0 2024-03-24T00:49:13.142 INFO:tasks.workunit.client.0.smithi038.stdout:0/946: mknod d0/d12/d42/d83/dfe/c13f 0 2024-03-24T00:49:13.144 INFO:tasks.workunit.client.0.smithi038.stdout:5/899: symlink d6/d19/d20/d42/l124 0 2024-03-24T00:49:13.144 INFO:tasks.workunit.client.1.smithi088.stdout:2/977: truncate db/f55 386659 0 2024-03-24T00:49:13.144 INFO:tasks.workunit.client.1.smithi088.stdout:2/978: chown db/dd/d96/d99/dfa 160331459 1 2024-03-24T00:49:13.147 INFO:tasks.workunit.client.0.smithi038.stdout:2/880: unlink d4/da3/d118/d7c/c108 0 2024-03-24T00:49:13.149 INFO:tasks.workunit.client.1.smithi088.stdout:2/979: unlink db/dd/d96/d4e/d8f/d9c/c107 0 2024-03-24T00:49:13.150 INFO:tasks.workunit.client.0.smithi038.stdout:2/881: mkdir d4/d18/d26/d117/d12b 0 2024-03-24T00:49:13.150 INFO:tasks.workunit.client.0.smithi038.stdout:2/882: chown d4/d27/d54/d67/fce 48678 1 2024-03-24T00:49:13.150 INFO:tasks.workunit.client.0.smithi038.stdout:2/883: fdatasync d4/d27/f2a 0 2024-03-24T00:49:13.151 INFO:tasks.workunit.client.0.smithi038.stdout:2/884: mknod d4/d18/d26/d37/c12c 0 2024-03-24T00:49:13.153 INFO:tasks.workunit.client.1.smithi088.stdout:2/980: link db/dd/d96/d4e/d8f/fa7 db/dd/d96/d4e/f138 0 2024-03-24T00:49:13.153 INFO:tasks.workunit.client.1.smithi088.stdout:2/981: truncate db/d21/d35/f10d 780993 0 2024-03-24T00:49:13.154 INFO:tasks.workunit.client.1.smithi088.stdout:7/963: dread d3/d32/d34/d9c/fae [0,4194304] 0 2024-03-24T00:49:13.154 INFO:tasks.workunit.client.1.smithi088.stdout:2/982: rename db/dd/c66 to db/dd/c139 0 2024-03-24T00:49:13.162 INFO:tasks.workunit.client.0.smithi038.stdout:7/903: dread d2/d9/da/d23/d2b/d5b/d72/d92/d98/f9f [0,4194304] 0 2024-03-24T00:49:13.164 INFO:tasks.workunit.client.1.smithi088.stdout:7/964: rmdir d3/d32/d34/d9c/d4b 39 2024-03-24T00:49:13.164 INFO:tasks.workunit.client.1.smithi088.stdout:7/965: readlink d3/d32/d34/d9c/d4b/d73/dab/dc3/l10f 0 2024-03-24T00:49:13.169 INFO:tasks.workunit.client.0.smithi038.stdout:0/947: dread d0/d14/de4/feb [0,4194304] 0 2024-03-24T00:49:13.169 INFO:tasks.workunit.client.0.smithi038.stdout:0/948: chown d0/d12/d42/d5e/db4/fcb 1020743 1 2024-03-24T00:49:13.170 INFO:tasks.workunit.client.1.smithi088.stdout:7/966: getdents d3/d32/d34 0 2024-03-24T00:49:13.173 INFO:tasks.workunit.client.0.smithi038.stdout:5/900: dread d6/d19/d20/d42/d43/d4c/f9c [0,4194304] 0 2024-03-24T00:49:13.174 INFO:tasks.workunit.client.0.smithi038.stdout:5/901: chown d6/d19/da1/ffe 1147 1 2024-03-24T00:49:13.175 INFO:tasks.workunit.client.1.smithi088.stdout:7/967: getdents d3/d32/d34/d9c/d4b/d74/d79/dea 0 2024-03-24T00:49:13.179 INFO:tasks.workunit.client.1.smithi088.stdout:7/968: rename d3/d31/f83 to d3/dbf/f12a 0 2024-03-24T00:49:13.179 INFO:tasks.workunit.client.0.smithi038.stdout:5/902: symlink d6/d19/d20/d70/l125 0 2024-03-24T00:49:13.179 INFO:tasks.workunit.client.0.smithi038.stdout:5/903: truncate d6/d19/d20/d33/d6f/faf 633165 0 2024-03-24T00:49:13.179 INFO:tasks.workunit.client.0.smithi038.stdout:5/904: truncate d6/d16/d10f/d92/f105 320625 0 2024-03-24T00:49:13.179 INFO:tasks.workunit.client.0.smithi038.stdout:5/905: read d6/f26 [1734110,102706] 0 2024-03-24T00:49:13.183 INFO:tasks.workunit.client.1.smithi088.stdout:2/983: dread db/dc/f2b [0,4194304] 0 2024-03-24T00:49:13.183 INFO:tasks.workunit.client.1.smithi088.stdout:2/984: write db/d21/f74 [3434362,62722] 0 2024-03-24T00:49:13.203 INFO:tasks.workunit.client.1.smithi088.stdout:2/985: dread db/dd/dc9/d100/f10e [0,4194304] 0 2024-03-24T00:49:13.207 INFO:tasks.workunit.client.0.smithi038.stdout:7/904: dwrite d2/d9/da/d9b/ff9 [0,4194304] 0 2024-03-24T00:49:13.213 INFO:tasks.workunit.client.0.smithi038.stdout:7/905: write d2/d9/da/dc/da8/dc5/fe1 [453538,15292] 0 2024-03-24T00:49:13.214 INFO:tasks.workunit.client.1.smithi088.stdout:2/986: dread db/dd/f121 [0,4194304] 0 2024-03-24T00:49:13.214 INFO:tasks.workunit.client.1.smithi088.stdout:2/987: chown db/d21/d35/f7d 39367 1 2024-03-24T00:49:13.214 INFO:tasks.workunit.client.1.smithi088.stdout:2/988: chown db/d21/d35/d49/f71 217393 1 2024-03-24T00:49:13.214 INFO:tasks.workunit.client.1.smithi088.stdout:2/989: fdatasync db/dd/d96/d99/db2/d88/f117 0 2024-03-24T00:49:13.215 INFO:tasks.workunit.client.1.smithi088.stdout:2/990: mkdir db/d21/dea/d13a 0 2024-03-24T00:49:13.215 INFO:tasks.workunit.client.1.smithi088.stdout:2/991: chown db/d21/ce1 0 1 2024-03-24T00:49:13.233 INFO:tasks.workunit.client.1.smithi088.stdout:2/992: dread db/d21/d35/d49/f8c [0,4194304] 0 2024-03-24T00:49:13.237 INFO:tasks.workunit.client.1.smithi088.stdout:7/969: dwrite d3/f2d [0,4194304] 0 2024-03-24T00:49:13.259 INFO:tasks.workunit.client.1.smithi088.stdout:7/970: dread d3/d32/d34/d9c/f6f [0,4194304] 0 2024-03-24T00:49:13.264 INFO:tasks.workunit.client.1.smithi088.stdout:7/971: rename d3/d32/d34/c9f to d3/d32/d76/d8f/c12b 0 2024-03-24T00:49:13.270 INFO:tasks.workunit.client.0.smithi038.stdout:0/949: dwrite d0/d12/d15/d1a/fff [0,4194304] 0 2024-03-24T00:49:13.277 INFO:tasks.workunit.client.0.smithi038.stdout:0/950: unlink d0/d12/d15/d1a/fff 0 2024-03-24T00:49:13.280 INFO:tasks.workunit.client.0.smithi038.stdout:0/951: creat d0/d14/d84/d119/d136/d64/da7/dbb/f140 x:0 0 0 2024-03-24T00:49:13.281 INFO:tasks.workunit.client.0.smithi038.stdout:0/952: rename d0/d14/de4/d100 to d0/d14/d84/d119/d141 0 2024-03-24T00:49:13.282 INFO:tasks.workunit.client.0.smithi038.stdout:0/953: truncate d0/d12/d15/f22 2854125 0 2024-03-24T00:49:13.291 INFO:tasks.workunit.client.0.smithi038.stdout:0/954: dread d0/f129 [0,4194304] 0 2024-03-24T00:49:13.292 INFO:tasks.workunit.client.0.smithi038.stdout:0/955: mkdir d0/d12/d42/d83/d142 0 2024-03-24T00:49:13.296 INFO:tasks.workunit.client.0.smithi038.stdout:5/906: dwrite d6/d19/d20/d42/d43/d68/dc8/fd4 [0,4194304] 0 2024-03-24T00:49:13.300 INFO:tasks.workunit.client.0.smithi038.stdout:7/906: dwrite d2/d9/da/f73 [4194304,4194304] 0 2024-03-24T00:49:13.346 INFO:tasks.workunit.client.1.smithi088.stdout:2/993: sync 2024-03-24T00:49:13.373 INFO:tasks.workunit.client.1.smithi088.stdout:2/994: dread db/d21/f44 [0,4194304] 0 2024-03-24T00:49:13.374 INFO:tasks.workunit.client.1.smithi088.stdout:2/995: creat db/dd/d96/d12e/df3/f13b x:0 0 0 2024-03-24T00:49:13.375 INFO:tasks.workunit.client.1.smithi088.stdout:2/996: mknod db/dd4/d122/c13c 0 2024-03-24T00:49:13.376 INFO:tasks.workunit.client.1.smithi088.stdout:2/997: creat db/d21/d35/d11e/f13d x:0 0 0 2024-03-24T00:49:13.376 INFO:tasks.workunit.client.1.smithi088.stdout:2/998: creat db/dd/d96/d4e/d8f/d9c/db0/f13e x:0 0 0 2024-03-24T00:49:13.376 INFO:tasks.workunit.client.1.smithi088.stdout:2/999: creat db/dd/dc9/f13f x:0 0 0 2024-03-24T00:49:13.380 INFO:tasks.workunit.client.0.smithi038.stdout:5/907: dwrite d6/f18 [4194304,4194304] 0 2024-03-24T00:49:13.380 INFO:tasks.workunit.client.0.smithi038.stdout:5/908: fsync d6/d19/d20/d42/d43/d4c/d6d/f11f 0 2024-03-24T00:49:13.381 INFO:tasks.workunit.client.0.smithi038.stdout:5/909: truncate d6/d19/d20/d70/da3/f11d 871766 0 2024-03-24T00:49:13.381 INFO:tasks.workunit.client.0.smithi038.stdout:0/956: dwrite d0/d14/d84/d119/d136/d64/ffa [0,4194304] 0 2024-03-24T00:49:13.383 INFO:tasks.workunit.client.0.smithi038.stdout:0/957: mknod d0/d1b/df1/d10d/c143 0 2024-03-24T00:49:13.396 INFO:tasks.workunit.client.0.smithi038.stdout:0/958: write d0/d12/d15/d1a/f98 [1985549,47724] 0 2024-03-24T00:49:13.399 INFO:tasks.workunit.client.0.smithi038.stdout:0/959: mkdir d0/d14/d84/d119/d141/d138/d144 0 2024-03-24T00:49:13.446 INFO:tasks.workunit.client.1.smithi088.stdout:7/972: sync 2024-03-24T00:49:13.447 INFO:tasks.workunit.client.1.smithi088.stdout:7/973: write d3/d32/d34/d9c/d4b/d73/fbe [3926988,126198] 0 2024-03-24T00:49:13.447 INFO:tasks.workunit.client.0.smithi038.stdout:0/960: dwrite d0/d12/d15/d1a/fc0 [0,4194304] 0 2024-03-24T00:49:13.447 INFO:tasks.workunit.client.0.smithi038.stdout:0/961: chown d0/d12/d42/d44/lb7 108901042 1 2024-03-24T00:49:13.447 INFO:tasks.workunit.client.0.smithi038.stdout:5/910: dwrite d6/d16/d21/d2d/fd2 [0,4194304] 0 2024-03-24T00:49:13.449 INFO:tasks.workunit.client.1.smithi088.stdout:7/974: symlink d3/d32/d76/l12c 0 2024-03-24T00:49:13.449 INFO:tasks.workunit.client.1.smithi088.stdout:7/975: read - d3/d32/d34/d9c/d4b/d74/d79/d7f/ffb zero size 2024-03-24T00:49:13.449 INFO:tasks.workunit.client.1.smithi088.stdout:7/976: fsync d3/d32/d76/df3/d103/f10a 0 2024-03-24T00:49:13.449 INFO:tasks.workunit.client.1.smithi088.stdout:7/977: dread - d3/d32/d34/d9c/d4b/da7/fc9 zero size 2024-03-24T00:49:13.451 INFO:tasks.workunit.client.0.smithi038.stdout:0/962: creat d0/d12/d15/f145 x:0 0 0 2024-03-24T00:49:13.452 INFO:tasks.workunit.client.0.smithi038.stdout:0/963: write d0/d14/d84/d119/d136/d64/da7/dbb/dd4/ff4 [11903,110796] 0 2024-03-24T00:49:13.452 INFO:tasks.workunit.client.0.smithi038.stdout:5/911: mkdir d6/d19/d20/d42/d43/d126 0 2024-03-24T00:49:13.454 INFO:tasks.workunit.client.1.smithi088.stdout:7/978: rmdir d3/d32/d34/d9c/d4b/d73/dab 39 2024-03-24T00:49:13.458 INFO:tasks.workunit.client.0.smithi038.stdout:0/964: rename d0/c137 to d0/d12/d42/d9d/d28/c146 0 2024-03-24T00:49:13.460 INFO:tasks.workunit.client.1.smithi088.stdout:7/979: creat d3/d32/d34/d9c/d4b/da7/dc5/d101/d11e/f12d x:0 0 0 2024-03-24T00:49:13.463 INFO:tasks.workunit.client.0.smithi038.stdout:0/965: write d0/d12/d15/d1a/fcc [315192,83559] 0 2024-03-24T00:49:13.465 INFO:tasks.workunit.client.1.smithi088.stdout:7/980: rename d3/d31/le9 to d3/d32/d34/d9c/d4b/ddc/l12e 0 2024-03-24T00:49:13.467 INFO:tasks.workunit.client.0.smithi038.stdout:0/966: mkdir d0/d14/d84/d119/d136/d64/da7/dbb/d147 0 2024-03-24T00:49:13.469 INFO:tasks.workunit.client.0.smithi038.stdout:0/967: symlink d0/d14/de4/l148 0 2024-03-24T00:49:13.469 INFO:tasks.workunit.client.0.smithi038.stdout:0/968: chown d0/d12/d42/d44/lb7 1818563 1 2024-03-24T00:49:13.471 INFO:tasks.workunit.client.0.smithi038.stdout:0/969: rename d0/c53 to d0/d12/c149 0 2024-03-24T00:49:13.473 INFO:tasks.workunit.client.0.smithi038.stdout:0/970: getdents d0/d14/d84/d119/d136/d64/da7/dbb/dcd 0 2024-03-24T00:49:13.473 INFO:tasks.workunit.client.0.smithi038.stdout:0/971: write d0/d14/d88/f97 [4466967,83632] 0 2024-03-24T00:49:13.473 INFO:tasks.workunit.client.0.smithi038.stdout:0/972: write d0/d12/d15/d1a/fcc [2087582,32989] 0 2024-03-24T00:49:13.477 INFO:tasks.workunit.client.0.smithi038.stdout:0/973: rename d0/d12/d42/d5e/d11f/f13c to d0/d14/de4/f14a 0 2024-03-24T00:49:13.480 INFO:tasks.workunit.client.0.smithi038.stdout:0/974: symlink d0/d14/d84/d119/d141/d138/d144/l14b 0 2024-03-24T00:49:13.483 INFO:tasks.workunit.client.0.smithi038.stdout:0/975: creat d0/d12/d42/d83/dfe/f14c x:0 0 0 2024-03-24T00:49:13.485 INFO:tasks.workunit.client.1.smithi088.stdout:7/981: read d3/d32/d34/d9c/d4b/d73/f117 [2884154,10284] 0 2024-03-24T00:49:13.485 INFO:tasks.workunit.client.0.smithi038.stdout:0/976: creat d0/d12/d42/d9d/f14d x:0 0 0 2024-03-24T00:49:13.487 INFO:tasks.workunit.client.1.smithi088.stdout:7/982: creat d3/d32/d34/d9c/d4b/f12f x:0 0 0 2024-03-24T00:49:13.490 INFO:tasks.workunit.client.1.smithi088.stdout:7/983: unlink d3/d31/f7a 0 2024-03-24T00:49:13.493 INFO:tasks.workunit.client.1.smithi088.stdout:7/984: creat d3/d32/d34/d9c/d4b/da7/dc5/f130 x:0 0 0 2024-03-24T00:49:13.497 INFO:tasks.workunit.client.1.smithi088.stdout:7/985: unlink d3/cb 0 2024-03-24T00:49:13.497 INFO:tasks.workunit.client.1.smithi088.stdout:7/986: fsync d3/fe5 0 2024-03-24T00:49:13.497 INFO:tasks.workunit.client.1.smithi088.stdout:7/987: dread - d3/d32/d34/d9c/d4b/da7/dc5/f122 zero size 2024-03-24T00:49:13.515 INFO:tasks.workunit.client.0.smithi038.stdout:0/977: read d0/d12/d92/fd6 [3998177,16007] 0 2024-03-24T00:49:13.515 INFO:tasks.workunit.client.0.smithi038.stdout:0/978: write d0/d12/d42/d83/dfe/f10f [572134,54700] 0 2024-03-24T00:49:13.518 INFO:tasks.workunit.client.0.smithi038.stdout:0/979: mknod d0/d12/d15/c14e 0 2024-03-24T00:49:13.520 INFO:tasks.workunit.client.0.smithi038.stdout:0/980: mkdir d0/d14/d84/d119/d136/d64/da7/dbb/d14f 0 2024-03-24T00:49:13.524 INFO:tasks.workunit.client.1.smithi088.stdout:7/988: getdents d3/d32/d34/d9c/d4b 0 2024-03-24T00:49:13.524 INFO:tasks.workunit.client.1.smithi088.stdout:7/989: chown d3/d32/d76/df3/ff6 9 1 2024-03-24T00:49:13.533 INFO:tasks.workunit.client.0.smithi038.stdout:0/981: dread d0/d14/d84/d119/d136/d64/f108 [0,4194304] 0 2024-03-24T00:49:13.535 INFO:tasks.workunit.client.0.smithi038.stdout:0/982: symlink d0/d12/d42/d5e/d11f/l150 0 2024-03-24T00:49:13.538 INFO:tasks.workunit.client.0.smithi038.stdout:0/983: unlink d0/d12/d42/d83/cbd 0 2024-03-24T00:49:13.541 INFO:tasks.workunit.client.0.smithi038.stdout:0/984: symlink d0/d14/d84/d119/d136/d64/l151 0 2024-03-24T00:49:13.541 INFO:tasks.workunit.client.0.smithi038.stdout:0/985: read d0/d14/d84/d119/d136/d64/da7/dbb/dd4/dea/f6f [7914,26721] 0 2024-03-24T00:49:13.554 INFO:tasks.workunit.client.1.smithi088.stdout:7/990: write d3/f21 [8890452,126195] 0 2024-03-24T00:49:13.560 INFO:tasks.workunit.client.1.smithi088.stdout:7/991: link d3/d32/d34/d9c/d4b/d73/dab/dc3/c113 d3/d32/d34/d9c/d4b/da7/dc5/d101/c131 0 2024-03-24T00:49:13.562 INFO:tasks.workunit.client.1.smithi088.stdout:7/992: symlink d3/d32/d34/d7c/l132 0 2024-03-24T00:49:13.567 INFO:tasks.workunit.client.1.smithi088.stdout:7/993: rmdir d3/d31/d56 39 2024-03-24T00:49:13.567 INFO:tasks.workunit.client.1.smithi088.stdout:7/994: stat d3/d32/d34/d9c/d4b/d73/ld9 0 2024-03-24T00:49:13.571 INFO:tasks.workunit.client.1.smithi088.stdout:7/995: creat d3/d32/d34/d9c/d4b/d74/d79/d7f/f133 x:0 0 0 2024-03-24T00:49:13.576 INFO:tasks.workunit.client.0.smithi038.stdout:0/986: dread d0/d1b/d2c/f86 [0,4194304] 0 2024-03-24T00:49:13.576 INFO:tasks.workunit.client.0.smithi038.stdout:0/987: write d0/d14/d84/d119/d136/d64/da7/dbb/dd4/dea/f8d [2631324,40348] 0 2024-03-24T00:49:13.586 INFO:tasks.workunit.client.1.smithi088.stdout:7/996: dread d3/f51 [0,4194304] 0 2024-03-24T00:49:13.591 INFO:tasks.workunit.client.1.smithi088.stdout:7/997: link d3/d32/d34/f3b d3/d32/f134 0 2024-03-24T00:49:13.607 INFO:tasks.workunit.client.0.smithi038.stdout:7/907: dwrite d2/d9/da/d23/def/f123 [0,4194304] 0 2024-03-24T00:49:13.617 INFO:tasks.workunit.client.0.smithi038.stdout:7/908: symlink d2/d9/da/d9b/l14c 0 2024-03-24T00:49:13.623 INFO:tasks.workunit.client.0.smithi038.stdout:7/909: dread - d2/d2c/f56 zero size 2024-03-24T00:49:13.632 INFO:tasks.workunit.client.0.smithi038.stdout:7/910: rmdir d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe 39 2024-03-24T00:49:13.634 INFO:tasks.workunit.client.1.smithi088.stdout:7/998: write d3/d32/d34/d9c/d4b/d74/f82 [1275845,94398] 0 2024-03-24T00:49:13.634 INFO:tasks.workunit.client.1.smithi088.stdout:7/999: write d3/d32/d34/d9c/d4b/d74/d79/d7f/fb7 [1401687,130711] 0 2024-03-24T00:49:13.643 INFO:tasks.workunit.client.1.smithi088.stderr:+ rm -rf -- ./tmp.K9rthFc5O1 2024-03-24T00:49:13.656 INFO:tasks.workunit.client.0.smithi038.stdout:2/885: sync 2024-03-24T00:49:13.656 INFO:tasks.workunit.client.0.smithi038.stdout:4/947: sync 2024-03-24T00:49:13.660 INFO:tasks.workunit.client.0.smithi038.stdout:2/886: unlink d4/da3/fb7 0 2024-03-24T00:49:13.661 INFO:tasks.workunit.client.0.smithi038.stdout:4/948: unlink da/d12/d37/dd/d15/d6d/d8f/df0/f124 0 2024-03-24T00:49:13.664 INFO:tasks.workunit.client.0.smithi038.stdout:0/988: dwrite d0/d12/d42/d83/fe3 [0,4194304] 0 2024-03-24T00:49:13.665 INFO:tasks.workunit.client.0.smithi038.stdout:2/887: mknod d4/d27/d54/d53/de2/c12d 0 2024-03-24T00:49:13.665 INFO:tasks.workunit.client.0.smithi038.stdout:2/888: truncate d4/d18/d26/f8c 210410 0 2024-03-24T00:49:13.674 INFO:tasks.workunit.client.0.smithi038.stdout:0/989: rename d0/d14/d84/d119/d136/fe7 to d0/d14/de4/f152 0 2024-03-24T00:49:13.677 INFO:tasks.workunit.client.0.smithi038.stdout:4/949: getdents da/d12/d29 0 2024-03-24T00:49:13.692 INFO:tasks.workunit.client.0.smithi038.stdout:0/990: creat d0/d14/d84/d119/d141/d138/d144/f153 x:0 0 0 2024-03-24T00:49:13.692 INFO:tasks.workunit.client.0.smithi038.stdout:0/991: write d0/d12/d15/d1a/fcc [2663080,75613] 0 2024-03-24T00:49:13.709 INFO:tasks.workunit.client.0.smithi038.stdout:4/950: getdents da/d12/d24 0 2024-03-24T00:49:13.709 INFO:tasks.workunit.client.0.smithi038.stdout:4/951: chown da/d12/d24/d68/d6c/da6/lc1 1135 1 2024-03-24T00:49:13.744 INFO:tasks.workunit.client.0.smithi038.stdout:2/889: dwrite d4/d18/d5b/f113 [0,4194304] 0 2024-03-24T00:49:13.751 INFO:tasks.workunit.client.0.smithi038.stdout:2/890: creat d4/da3/d118/d7c/d103/f12e x:0 0 0 2024-03-24T00:49:13.751 INFO:tasks.workunit.client.0.smithi038.stdout:2/891: truncate d4/da3/d118/f78 196810 0 2024-03-24T00:49:13.752 INFO:tasks.workunit.client.0.smithi038.stdout:2/892: chown d4/d18/d26/f4e 496622 1 2024-03-24T00:49:13.755 INFO:tasks.workunit.client.0.smithi038.stdout:2/893: creat d4/d27/d54/dab/d104/f12f x:0 0 0 2024-03-24T00:49:13.755 INFO:tasks.workunit.client.0.smithi038.stdout:2/894: creat d4/d18/f130 x:0 0 0 2024-03-24T00:49:13.782 INFO:tasks.workunit.client.0.smithi038.stdout:0/992: dwrite d0/d14/d88/db8/ff2 [0,4194304] 0 2024-03-24T00:49:13.782 INFO:tasks.workunit.client.0.smithi038.stdout:0/993: chown d0/d12/d42/d44/f11e 5158056 1 2024-03-24T00:49:13.786 INFO:tasks.workunit.client.0.smithi038.stdout:0/994: mknod d0/d14/d84/d119/d136/d12b/c154 0 2024-03-24T00:49:13.787 INFO:tasks.workunit.client.0.smithi038.stdout:0/995: truncate d0/d14/fb5 4644871 0 2024-03-24T00:49:13.791 INFO:tasks.workunit.client.0.smithi038.stdout:0/996: unlink d0/d14/d84/d119/d136/d64/da7/dbb/dd4/dea/f8d 0 2024-03-24T00:49:13.791 INFO:tasks.workunit.client.0.smithi038.stdout:0/997: write d0/d12/d42/d5e/ff0 [1054444,25503] 0 2024-03-24T00:49:13.793 INFO:tasks.workunit.client.0.smithi038.stdout:2/895: dread d4/d18/f7b [0,4194304] 0 2024-03-24T00:49:13.793 INFO:tasks.workunit.client.0.smithi038.stdout:2/896: chown d4/d27/d54/d67/f87 0 1 2024-03-24T00:49:13.806 INFO:tasks.workunit.client.0.smithi038.stdout:4/952: dwrite da/d38/f114 [0,4194304] 0 2024-03-24T00:49:13.807 INFO:tasks.workunit.client.0.smithi038.stdout:4/953: creat da/d12/d24/d5f/df8/f14b x:0 0 0 2024-03-24T00:49:13.812 INFO:tasks.workunit.client.0.smithi038.stdout:4/954: rename da/d12/d37/dd/d15/d4c/d70/lb8 to da/l14c 0 2024-03-24T00:49:13.814 INFO:tasks.workunit.client.0.smithi038.stdout:4/955: symlink da/d12/d37/dd/d15/l14d 0 2024-03-24T00:49:13.814 INFO:tasks.workunit.client.0.smithi038.stdout:4/956: chown da/d12/d29/c49 380 1 2024-03-24T00:49:13.818 INFO:tasks.workunit.client.0.smithi038.stdout:4/957: rename da/d38/f76 to da/d12/d24/d5f/d9a/f14e 0 2024-03-24T00:49:13.818 INFO:tasks.workunit.client.0.smithi038.stdout:4/958: write da/d12/d29/d5b/d73/db4/fc3 [445642,17286] 0 2024-03-24T00:49:13.819 INFO:tasks.workunit.client.0.smithi038.stdout:0/998: dread d0/d1b/f20 [0,4194304] 0 2024-03-24T00:49:13.819 INFO:tasks.workunit.client.0.smithi038.stdout:0/999: creat d0/d12/d42/d83/f155 x:0 0 0 2024-03-24T00:49:13.820 INFO:tasks.workunit.client.0.smithi038.stdout:4/959: symlink da/d12/d37/dd/d15/d142/l14f 0 2024-03-24T00:49:13.820 INFO:tasks.workunit.client.0.smithi038.stdout:4/960: creat da/d12/d37/dd/d15/d6d/d8f/f150 x:0 0 0 2024-03-24T00:49:13.824 INFO:tasks.workunit.client.0.smithi038.stdout:4/961: rename da/d12/d24/d5f/f7e to da/d12/d37/dd/d15/d6d/d8f/df0/f151 0 2024-03-24T00:49:13.825 INFO:tasks.workunit.client.0.smithi038.stdout:4/962: stat da/d12/d24/c59 0 2024-03-24T00:49:13.825 INFO:tasks.workunit.client.0.smithi038.stdout:5/912: sync 2024-03-24T00:49:13.838 INFO:tasks.workunit.client.0.smithi038.stdout:4/963: dread da/d12/d37/dd/d15/d4c/f53 [0,4194304] 0 2024-03-24T00:49:13.838 INFO:tasks.workunit.client.0.smithi038.stdout:4/964: truncate da/d12/d24/d68/d6c/ff4 630262 0 2024-03-24T00:49:13.841 INFO:tasks.workunit.client.0.smithi038.stdout:5/913: dread d6/d16/d14/f8e [0,4194304] 0 2024-03-24T00:49:13.842 INFO:tasks.workunit.client.0.smithi038.stdout:4/965: creat da/d12/d24/d5f/d8a/da4/f152 x:0 0 0 2024-03-24T00:49:13.842 INFO:tasks.workunit.client.0.smithi038.stdout:4/966: truncate da/d12/d24/d5f/d8a/db6/dd1/f10e 614735 0 2024-03-24T00:49:13.845 INFO:tasks.workunit.client.0.smithi038.stdout:2/897: dwrite d4/d27/d54/d53/de2/f9a [0,4194304] 0 2024-03-24T00:49:13.849 INFO:tasks.workunit.client.0.smithi038.stdout:5/914: rename d6/d16/d14 to d6/d16/d21/d2d/d127 0 2024-03-24T00:49:13.911 INFO:tasks.workunit.client.0.smithi038.stdout:7/911: sync 2024-03-24T00:49:13.955 INFO:tasks.workunit.client.0.smithi038.stdout:7/912: creat d2/d9/da/d23/d2b/d5b/d72/d92/d75/d12a/f14d x:0 0 0 2024-03-24T00:49:13.955 INFO:tasks.workunit.client.0.smithi038.stdout:7/913: chown d2/d9/c11 49317966 1 2024-03-24T00:49:13.956 INFO:tasks.workunit.client.0.smithi038.stdout:7/914: stat d2/d9/da/d23/c6e 0 2024-03-24T00:49:13.958 INFO:tasks.workunit.client.0.smithi038.stdout:4/967: rename da/d12/d37/dd/d15/d6d/d8f to da/d12/d29/d5b/d73/db4/de3/d153 0 2024-03-24T00:49:13.958 INFO:tasks.workunit.client.0.smithi038.stdout:4/968: chown da/d12/d29/cee 60398 1 2024-03-24T00:49:13.965 INFO:tasks.workunit.client.0.smithi038.stdout:7/915: mkdir d2/d2c/d9a/deb/d12f/d14e 0 2024-03-24T00:49:13.967 INFO:tasks.workunit.client.0.smithi038.stdout:5/915: rename d6/d23/f3b to d6/d19/d20/d70/f128 0 2024-03-24T00:49:13.970 INFO:tasks.workunit.client.0.smithi038.stdout:4/969: mknod da/d12/d24/d147/c154 0 2024-03-24T00:49:13.975 INFO:tasks.workunit.client.0.smithi038.stdout:7/916: rename d2/d9/da/d23/c7a to d2/d9/d16/dfc/c14f 0 2024-03-24T00:49:13.978 INFO:tasks.workunit.client.0.smithi038.stdout:5/916: rename d6/d19/d20/d42/d43/d4c/f9d to d6/d19/d20/d33/f129 0 2024-03-24T00:49:13.980 INFO:tasks.workunit.client.0.smithi038.stdout:7/917: link d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fca d2/d9/f150 0 2024-03-24T00:49:13.981 INFO:tasks.workunit.client.0.smithi038.stdout:7/918: creat d2/d9/d16/dfc/f151 x:0 0 0 2024-03-24T00:49:13.986 INFO:tasks.workunit.client.0.smithi038.stdout:7/919: creat d2/d9/da/dc/d8a/d105/f152 x:0 0 0 2024-03-24T00:49:13.988 INFO:tasks.workunit.client.0.smithi038.stdout:7/920: truncate d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/f70 2051671 0 2024-03-24T00:49:14.036 INFO:tasks.workunit.client.0.smithi038.stdout:4/970: dwrite da/d12/d37/f99 [0,4194304] 0 2024-03-24T00:49:14.043 INFO:tasks.workunit.client.0.smithi038.stdout:4/971: mkdir da/d12/d24/d5f/d8a/da4/d155 0 2024-03-24T00:49:14.061 INFO:tasks.workunit.client.0.smithi038.stdout:7/921: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fd5 [0,4194304] 0 2024-03-24T00:49:14.064 INFO:tasks.workunit.client.0.smithi038.stdout:7/922: mknod d2/d9/d16/c153 0 2024-03-24T00:49:14.070 INFO:tasks.workunit.client.0.smithi038.stdout:7/923: link d2/d9/c11 d2/d9/da/dc/d93/c154 0 2024-03-24T00:49:14.073 INFO:tasks.workunit.client.0.smithi038.stdout:7/924: stat d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/l11a 0 2024-03-24T00:49:14.077 INFO:tasks.workunit.client.0.smithi038.stdout:7/925: symlink d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/l155 0 2024-03-24T00:49:14.078 INFO:tasks.workunit.client.0.smithi038.stdout:7/926: chown d2/d9/d16/dfc/f12c 10524 1 2024-03-24T00:49:14.080 INFO:tasks.workunit.client.0.smithi038.stdout:7/927: symlink d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/d112/l156 0 2024-03-24T00:49:14.082 INFO:tasks.workunit.client.0.smithi038.stdout:7/928: rename d2/d2c/fd7 to d2/d9/f157 0 2024-03-24T00:49:14.092 INFO:tasks.workunit.client.0.smithi038.stdout:4/972: dwrite da/d12/d24/d5f/d8a/db6/dd1/f10e [0,4194304] 0 2024-03-24T00:49:14.092 INFO:tasks.workunit.client.0.smithi038.stdout:4/973: write da/d12/d37/dd/d15/df1/f11b [617243,122884] 0 2024-03-24T00:49:14.092 INFO:tasks.workunit.client.0.smithi038.stdout:4/974: dread - da/d12/d24/f12a zero size 2024-03-24T00:49:14.097 INFO:tasks.workunit.client.0.smithi038.stdout:4/975: mknod da/d12/d37/dd/d15/d4c/d70/c156 0 2024-03-24T00:49:14.097 INFO:tasks.workunit.client.0.smithi038.stdout:4/976: stat da/d38/f3c 0 2024-03-24T00:49:14.101 INFO:tasks.workunit.client.0.smithi038.stdout:4/977: rmdir da/d38 39 2024-03-24T00:49:14.103 INFO:tasks.workunit.client.0.smithi038.stdout:4/978: rename da/d38/l11e to da/d12/d37/d83/dac/dcf/l157 0 2024-03-24T00:49:14.105 INFO:tasks.workunit.client.0.smithi038.stdout:4/979: creat da/d12/d29/d10d/d136/f158 x:0 0 0 2024-03-24T00:49:14.170 INFO:tasks.workunit.client.0.smithi038.stdout:4/980: dwrite da/d12/d29/d5b/d73/db4/f139 [0,4194304] 0 2024-03-24T00:49:14.175 INFO:tasks.workunit.client.0.smithi038.stdout:4/981: symlink da/d38/l159 0 2024-03-24T00:49:14.179 INFO:tasks.workunit.client.0.smithi038.stdout:4/982: symlink da/d12/d37/dd/l15a 0 2024-03-24T00:49:14.179 INFO:tasks.workunit.client.0.smithi038.stdout:4/983: fdatasync da/d12/d24/f26 0 2024-03-24T00:49:14.239 INFO:tasks.workunit.client.0.smithi038.stdout:4/984: dwrite da/d12/d29/d5b/d73/db4/ff3 [0,4194304] 0 2024-03-24T00:49:14.239 INFO:tasks.workunit.client.0.smithi038.stdout:4/985: fsync da/d12/d29/d5b/d73/db4/f139 0 2024-03-24T00:49:14.243 INFO:tasks.workunit.client.0.smithi038.stdout:2/898: sync 2024-03-24T00:49:14.247 INFO:tasks.workunit.client.0.smithi038.stdout:2/899: rmdir d4/d18/d26/d37/d69/d83 39 2024-03-24T00:49:14.247 INFO:tasks.workunit.client.0.smithi038.stdout:2/900: fdatasync d4/d18/f22 0 2024-03-24T00:49:14.250 INFO:tasks.workunit.client.0.smithi038.stdout:2/901: creat d4/da3/d118/d11f/f131 x:0 0 0 2024-03-24T00:49:14.251 INFO:tasks.workunit.client.0.smithi038.stdout:2/902: chown d4/d76/fb3 0 1 2024-03-24T00:49:14.275 INFO:tasks.workunit.client.0.smithi038.stdout:2/903: write d4/d18/d26/d37/d69/d83/fd3 [1903887,96027] 0 2024-03-24T00:49:14.283 INFO:tasks.workunit.client.0.smithi038.stdout:2/904: rmdir d4/d18/d26/d117 39 2024-03-24T00:49:14.318 INFO:tasks.workunit.client.0.smithi038.stdout:4/986: dwrite da/d12/d37/dd/f34 [0,4194304] 0 2024-03-24T00:49:14.318 INFO:tasks.workunit.client.0.smithi038.stdout:4/987: truncate da/d12/d37/d83/dac/dcf/f14a 632166 0 2024-03-24T00:49:14.324 INFO:tasks.workunit.client.0.smithi038.stdout:4/988: symlink da/d12/d29/d5b/d73/db4/de3/l15b 0 2024-03-24T00:49:14.342 INFO:tasks.workunit.client.0.smithi038.stdout:2/905: dwrite d4/d27/d54/d67/f87 [0,4194304] 0 2024-03-24T00:49:14.344 INFO:tasks.workunit.client.0.smithi038.stdout:4/989: dread da/d12/d37/dd/f2b [0,4194304] 0 2024-03-24T00:49:14.345 INFO:tasks.workunit.client.0.smithi038.stdout:4/990: write da/d12/d29/fb1 [296072,90912] 0 2024-03-24T00:49:14.345 INFO:tasks.workunit.client.0.smithi038.stdout:4/991: dread - da/d38/f132 zero size 2024-03-24T00:49:14.345 INFO:tasks.workunit.client.0.smithi038.stdout:4/992: dread - da/d12/d24/fc0 zero size 2024-03-24T00:49:14.360 INFO:tasks.workunit.client.0.smithi038.stdout:2/906: dread d4/dc7/ff4 [0,4194304] 0 2024-03-24T00:49:14.365 INFO:tasks.workunit.client.0.smithi038.stdout:2/907: mknod d4/d18/d38/c132 0 2024-03-24T00:49:14.368 INFO:tasks.workunit.client.0.smithi038.stdout:4/993: dread da/d38/f61 [0,4194304] 0 2024-03-24T00:49:14.371 INFO:tasks.workunit.client.0.smithi038.stdout:2/908: rmdir d4/d27/d8f 39 2024-03-24T00:49:14.371 INFO:tasks.workunit.client.0.smithi038.stdout:7/929: sync 2024-03-24T00:49:14.371 INFO:tasks.workunit.client.0.smithi038.stdout:5/917: sync 2024-03-24T00:49:14.373 INFO:tasks.workunit.client.0.smithi038.stdout:4/994: link da/d12/d29/d5b/d73/db4/de3/d153/l128 da/d12/d24/d5f/d8a/da4/d155/l15c 0 2024-03-24T00:49:14.373 INFO:tasks.workunit.client.0.smithi038.stdout:4/995: stat da/d12/d24 0 2024-03-24T00:49:14.374 INFO:tasks.workunit.client.0.smithi038.stdout:7/930: getdents d2/d9/d16 0 2024-03-24T00:49:14.377 INFO:tasks.workunit.client.0.smithi038.stdout:4/996: symlink da/d12/d29/d10d/d136/l15d 0 2024-03-24T00:49:14.381 INFO:tasks.workunit.client.0.smithi038.stdout:7/931: creat d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/f158 x:0 0 0 2024-03-24T00:49:14.388 INFO:tasks.workunit.client.0.smithi038.stdout:7/932: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d5d/dcd/d159 0 2024-03-24T00:49:14.388 INFO:tasks.workunit.client.0.smithi038.stdout:4/997: mkdir da/d38/d15e 0 2024-03-24T00:49:14.388 INFO:tasks.workunit.client.0.smithi038.stdout:4/998: chown da/fa2 127636 1 2024-03-24T00:49:14.398 INFO:tasks.workunit.client.0.smithi038.stdout:5/918: write d6/d19/d20/fcf [867617,87059] 0 2024-03-24T00:49:14.404 INFO:tasks.workunit.client.0.smithi038.stdout:5/919: symlink d6/d19/d20/d42/d43/d48/l12a 0 2024-03-24T00:49:14.405 INFO:tasks.workunit.client.0.smithi038.stdout:5/920: write d6/d16/d21/d2d/d127/f8e [3165197,76936] 0 2024-03-24T00:49:14.407 INFO:tasks.workunit.client.0.smithi038.stdout:5/921: symlink d6/d16/d67/d121/l12b 0 2024-03-24T00:49:14.419 INFO:tasks.workunit.client.0.smithi038.stdout:4/999: dread da/d12/d24/d5f/d8a/fc4 [0,4194304] 0 2024-03-24T00:49:14.420 INFO:tasks.workunit.client.0.smithi038.stdout:7/933: dread d2/f4 [0,4194304] 0 2024-03-24T00:49:14.422 INFO:tasks.workunit.client.0.smithi038.stdout:7/934: rmdir d2/d9/d16 39 2024-03-24T00:49:14.427 INFO:tasks.workunit.client.0.smithi038.stdout:7/935: mkdir d2/d9/da/dc/d4a/d15a 0 2024-03-24T00:49:14.429 INFO:tasks.workunit.client.0.smithi038.stdout:5/922: write d6/d19/d20/d33/d6f/faf [194257,105569] 0 2024-03-24T00:49:14.429 INFO:tasks.workunit.client.0.smithi038.stdout:5/923: fdatasync d6/d16/d67/df6/f101 0 2024-03-24T00:49:14.430 INFO:tasks.workunit.client.0.smithi038.stdout:7/936: truncate d2/d9/da/d23/d2b/d5b/d72/d92/d5d/f9d 2023749 0 2024-03-24T00:49:14.430 INFO:tasks.workunit.client.0.smithi038.stdout:7/937: write d2/d9/f54 [8960870,105473] 0 2024-03-24T00:49:14.432 INFO:tasks.workunit.client.0.smithi038.stdout:5/924: mkdir d6/d19/d12c 0 2024-03-24T00:49:14.433 INFO:tasks.workunit.client.0.smithi038.stdout:7/938: creat d2/d9/da/d23/d2b/d5b/d72/d92/f15b x:0 0 0 2024-03-24T00:49:14.440 INFO:tasks.workunit.client.0.smithi038.stdout:5/925: rename d6/ce to d6/d1e/d97/dd1/c12d 0 2024-03-24T00:49:14.440 INFO:tasks.workunit.client.0.smithi038.stdout:5/926: write d6/d1e/d97/db0/f10d [343546,97321] 0 2024-03-24T00:49:14.440 INFO:tasks.workunit.client.0.smithi038.stdout:7/939: rmdir d2/d9/da/d23/d2b/d5b/d72/d92/d5d/dcd/d159 0 2024-03-24T00:49:14.440 INFO:tasks.workunit.client.0.smithi038.stdout:7/940: fsync d2/d9/d16/f77 0 2024-03-24T00:49:14.440 INFO:tasks.workunit.client.0.smithi038.stdout:7/941: dread - d2/d2c/d9a/dcb/d132/f140 zero size 2024-03-24T00:49:14.452 INFO:tasks.workunit.client.0.smithi038.stdout:2/909: dwrite d4/d18/da1/ff1 [0,4194304] 0 2024-03-24T00:49:14.466 INFO:tasks.workunit.client.0.smithi038.stdout:2/910: symlink d4/da3/d118/d7c/l133 0 2024-03-24T00:49:14.467 INFO:tasks.workunit.client.0.smithi038.stdout:2/911: mknod d4/d27/c134 0 2024-03-24T00:49:14.472 INFO:tasks.workunit.client.0.smithi038.stdout:2/912: creat d4/d18/da1/f135 x:0 0 0 2024-03-24T00:49:14.478 INFO:tasks.workunit.client.0.smithi038.stdout:2/913: creat d4/da3/d118/f136 x:0 0 0 2024-03-24T00:49:14.504 INFO:tasks.workunit.client.0.smithi038.stdout:2/914: dread d4/d27/d54/d53/de2/f9a [0,4194304] 0 2024-03-24T00:49:14.507 INFO:tasks.workunit.client.0.smithi038.stdout:2/915: link d4/d18/f2e d4/d27/d54/d67/f137 0 2024-03-24T00:49:14.509 INFO:tasks.workunit.client.0.smithi038.stdout:2/916: fdatasync d4/da3/ff6 0 2024-03-24T00:49:14.509 INFO:tasks.workunit.client.0.smithi038.stdout:2/917: creat d4/d18/d38/d63/f138 x:0 0 0 2024-03-24T00:49:14.512 INFO:tasks.workunit.client.0.smithi038.stdout:2/918: mknod d4/d27/d8f/c139 0 2024-03-24T00:49:14.519 INFO:tasks.workunit.client.0.smithi038.stdout:5/927: dwrite d6/d16/d67/f117 [0,4194304] 0 2024-03-24T00:49:14.520 INFO:tasks.workunit.client.0.smithi038.stdout:5/928: write d6/d19/d20/d42/fd5 [5018622,35597] 0 2024-03-24T00:49:14.539 INFO:tasks.workunit.client.0.smithi038.stdout:7/942: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f97 [0,4194304] 0 2024-03-24T00:49:14.561 INFO:tasks.workunit.client.0.smithi038.stdout:7/943: dread d2/d9/da/f47 [0,4194304] 0 2024-03-24T00:49:14.568 INFO:tasks.workunit.client.0.smithi038.stdout:7/944: rmdir d2/d9/da/d9b 39 2024-03-24T00:49:14.568 INFO:tasks.workunit.client.0.smithi038.stdout:7/945: stat d2/d9/da/d23/d2b/d5b/d72/d92/d5d 0 2024-03-24T00:49:14.584 INFO:tasks.workunit.client.0.smithi038.stdout:7/946: dread d2/d2c/f74 [0,4194304] 0 2024-03-24T00:49:14.584 INFO:tasks.workunit.client.0.smithi038.stdout:7/947: fsync d2/d9/da/d9b/ff9 0 2024-03-24T00:49:14.585 INFO:tasks.workunit.client.0.smithi038.stdout:7/948: stat d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/c61 0 2024-03-24T00:49:14.608 INFO:tasks.workunit.client.0.smithi038.stdout:7/949: dread d2/d9/da/d23/d2b/d5b/d72/d92/d5d/fda [0,4194304] 0 2024-03-24T00:49:14.608 INFO:tasks.workunit.client.0.smithi038.stdout:7/950: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/d75/ffd 0 2024-03-24T00:49:14.609 INFO:tasks.workunit.client.0.smithi038.stdout:7/951: dread - d2/d9/d16/dfc/f151 zero size 2024-03-24T00:49:14.609 INFO:tasks.workunit.client.0.smithi038.stdout:7/952: creat d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/dcc/f15c x:0 0 0 2024-03-24T00:49:14.609 INFO:tasks.workunit.client.0.smithi038.stdout:7/953: chown d2/d9/da/d23/d2b/d5b/cf7 1 1 2024-03-24T00:49:14.614 INFO:tasks.workunit.client.0.smithi038.stdout:7/954: creat d2/d9/da/d23/d2b/d5b/f15d x:0 0 0 2024-03-24T00:49:14.615 INFO:tasks.workunit.client.0.smithi038.stdout:5/929: dwrite d6/d19/d20/d42/fd5 [4194304,4194304] 0 2024-03-24T00:49:14.618 INFO:tasks.workunit.client.0.smithi038.stdout:7/955: symlink d2/d9/da/d23/d2b/d5b/d72/d92/d130/l15e 0 2024-03-24T00:49:14.622 INFO:tasks.workunit.client.0.smithi038.stdout:5/930: creat d6/d16/d67/d121/f12e x:0 0 0 2024-03-24T00:49:14.622 INFO:tasks.workunit.client.0.smithi038.stdout:5/931: write d6/d16/d10f/f81 [4507647,18485] 0 2024-03-24T00:49:14.624 INFO:tasks.workunit.client.0.smithi038.stdout:7/956: mkdir d2/d2c/d9a/dcb/d132/d15f 0 2024-03-24T00:49:14.632 INFO:tasks.workunit.client.0.smithi038.stdout:7/957: creat d2/d9/da/d9b/f160 x:0 0 0 2024-03-24T00:49:14.632 INFO:tasks.workunit.client.0.smithi038.stdout:7/958: write d2/d9/d16/dfc/f119 [914655,13942] 0 2024-03-24T00:49:14.634 INFO:tasks.workunit.client.0.smithi038.stdout:5/932: rmdir d6/d19/d20/d42/d79 39 2024-03-24T00:49:14.636 INFO:tasks.workunit.client.0.smithi038.stdout:7/959: creat d2/d2c/d9a/deb/d12f/d14e/f161 x:0 0 0 2024-03-24T00:49:14.641 INFO:tasks.workunit.client.0.smithi038.stdout:5/933: rename d6/d19/d20/d33/fdb to d6/d19/d20/d64/d6b/de8/f12f 0 2024-03-24T00:49:14.650 INFO:tasks.workunit.client.0.smithi038.stdout:7/960: rename d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/dcc/lf3 to d2/d9/da/dc/d4a/l162 0 2024-03-24T00:49:14.705 INFO:tasks.workunit.client.0.smithi038.stdout:7/961: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/f29 [0,4194304] 0 2024-03-24T00:49:14.718 INFO:tasks.workunit.client.0.smithi038.stdout:5/934: dwrite d6/d19/d20/d42/f72 [0,4194304] 0 2024-03-24T00:49:14.742 INFO:tasks.workunit.client.0.smithi038.stdout:5/935: dread d6/d19/d20/d42/d43/d4c/f62 [0,4194304] 0 2024-03-24T00:49:14.744 INFO:tasks.workunit.client.0.smithi038.stdout:5/936: creat d6/d16/d21/d2d/d127/f130 x:0 0 0 2024-03-24T00:49:14.746 INFO:tasks.workunit.client.0.smithi038.stdout:2/919: sync 2024-03-24T00:49:14.751 INFO:tasks.workunit.client.0.smithi038.stdout:2/920: mkdir d4/da3/d118/d7c/d13a 0 2024-03-24T00:49:14.761 INFO:tasks.workunit.client.0.smithi038.stdout:2/921: dread d4/d18/d26/f62 [4194304,4194304] 0 2024-03-24T00:49:14.764 INFO:tasks.workunit.client.0.smithi038.stdout:2/922: getdents d4/da3 0 2024-03-24T00:49:14.764 INFO:tasks.workunit.client.0.smithi038.stdout:2/923: fsync d4/dc7/ff4 0 2024-03-24T00:49:14.768 INFO:tasks.workunit.client.0.smithi038.stdout:2/924: creat d4/d18/d26/d12a/f13b x:0 0 0 2024-03-24T00:49:14.792 INFO:tasks.workunit.client.0.smithi038.stdout:7/962: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fc7 [0,4194304] 0 2024-03-24T00:49:14.811 INFO:tasks.workunit.client.0.smithi038.stdout:5/937: dwrite d6/d19/d20/d42/fd5 [4194304,4194304] 0 2024-03-24T00:49:14.813 INFO:tasks.workunit.client.0.smithi038.stdout:5/938: mkdir d6/d16/d21/d2d/d127/d131 0 2024-03-24T00:49:14.813 INFO:tasks.workunit.client.0.smithi038.stdout:5/939: dread - d6/d16/d67/df6/f101 zero size 2024-03-24T00:49:14.813 INFO:tasks.workunit.client.0.smithi038.stdout:5/940: chown d6/d1e/d109 15342530 1 2024-03-24T00:49:14.816 INFO:tasks.workunit.client.0.smithi038.stdout:5/941: link d6/d19/d20/d64/d6b/d77/f100 d6/d19/d20/d42/d43/d87/f132 0 2024-03-24T00:49:14.822 INFO:tasks.workunit.client.0.smithi038.stdout:5/942: creat d6/d16/d10f/d92/f133 x:0 0 0 2024-03-24T00:49:14.825 INFO:tasks.workunit.client.0.smithi038.stdout:5/943: creat d6/d19/d20/f134 x:0 0 0 2024-03-24T00:49:14.843 INFO:tasks.workunit.client.0.smithi038.stdout:7/963: write d2/d9/da/d23/d2b/d5b/f110 [1467984,127471] 0 2024-03-24T00:49:14.848 INFO:tasks.workunit.client.0.smithi038.stdout:5/944: dread d6/d16/d10f/f81 [0,4194304] 0 2024-03-24T00:49:14.855 INFO:tasks.workunit.client.0.smithi038.stdout:7/964: dread d2/d9/da/f1d [0,4194304] 0 2024-03-24T00:49:14.857 INFO:tasks.workunit.client.0.smithi038.stdout:2/925: dwrite d4/d27/d54/d53/fa5 [0,4194304] 0 2024-03-24T00:49:14.861 INFO:tasks.workunit.client.0.smithi038.stdout:2/926: mkdir d4/d27/d8f/d9e/d13c 0 2024-03-24T00:49:14.861 INFO:tasks.workunit.client.0.smithi038.stdout:2/927: write d4/d27/d2f/f3c [1339477,49401] 0 2024-03-24T00:49:14.889 INFO:tasks.workunit.client.0.smithi038.stdout:2/928: dread d4/da3/d118/f78 [0,4194304] 0 2024-03-24T00:49:14.889 INFO:tasks.workunit.client.0.smithi038.stdout:2/929: getdents d4/d27/d8f/d9e 0 2024-03-24T00:49:14.891 INFO:tasks.workunit.client.0.smithi038.stdout:2/930: creat d4/d27/d54/d67/d111/f13d x:0 0 0 2024-03-24T00:49:14.891 INFO:tasks.workunit.client.0.smithi038.stdout:2/931: write d4/d18/d38/f57 [4495249,28937] 0 2024-03-24T00:49:14.898 INFO:tasks.workunit.client.0.smithi038.stdout:5/945: dwrite d6/d19/d20/d33/d6f/faf [0,4194304] 0 2024-03-24T00:49:14.898 INFO:tasks.workunit.client.0.smithi038.stdout:5/946: stat d6/d1e/f63 0 2024-03-24T00:49:14.898 INFO:tasks.workunit.client.0.smithi038.stdout:2/932: rename d4/d18/d38/d63 to d4/da3/d118/d10a/d13e 0 2024-03-24T00:49:14.903 INFO:tasks.workunit.client.0.smithi038.stdout:2/933: mkdir d4/d27/d2f/d13f 0 2024-03-24T00:49:14.903 INFO:tasks.workunit.client.0.smithi038.stdout:5/947: rmdir d6/d19/d20/d42/d43/d68/dc8/de7 39 2024-03-24T00:49:14.903 INFO:tasks.workunit.client.0.smithi038.stdout:5/948: read - d6/d19/d20/d64/d6b/d77/f116 zero size 2024-03-24T00:49:14.914 INFO:tasks.workunit.client.0.smithi038.stdout:2/934: write d4/d76/fb1 [1216906,124693] 0 2024-03-24T00:49:14.916 INFO:tasks.workunit.client.0.smithi038.stdout:2/935: creat d4/d27/d54/d53/f140 x:0 0 0 2024-03-24T00:49:14.923 INFO:tasks.workunit.client.0.smithi038.stdout:5/949: dread d6/d19/d20/d42/d43/d68/f8b [0,4194304] 0 2024-03-24T00:49:14.923 INFO:tasks.workunit.client.0.smithi038.stdout:5/950: chown d6/d19/d20/d33/ff7 1 1 2024-03-24T00:49:14.923 INFO:tasks.workunit.client.0.smithi038.stdout:5/951: dread - d6/d1e/fcb zero size 2024-03-24T00:49:14.923 INFO:tasks.workunit.client.0.smithi038.stdout:5/952: fdatasync d6/d19/d20/d42/d43/d68/f8b 0 2024-03-24T00:49:14.923 INFO:tasks.workunit.client.0.smithi038.stdout:5/953: chown d6/d19/d20/d33/f107 468307462 1 2024-03-24T00:49:14.925 INFO:tasks.workunit.client.0.smithi038.stdout:5/954: creat d6/d19/d20/d42/d43/d68/dc8/f135 x:0 0 0 2024-03-24T00:49:14.925 INFO:tasks.workunit.client.0.smithi038.stdout:5/955: chown d6/d19/d20/d42/d43/d68 29 1 2024-03-24T00:49:14.927 INFO:tasks.workunit.client.0.smithi038.stdout:5/956: unlink d6/d19/d20/d42/d43/l99 0 2024-03-24T00:49:14.928 INFO:tasks.workunit.client.0.smithi038.stdout:5/957: mkdir d6/d16/d21/d136 0 2024-03-24T00:49:14.972 INFO:tasks.workunit.client.0.smithi038.stdout:5/958: write d6/d19/d20/d42/d43/d4c/f62 [1939718,61902] 0 2024-03-24T00:49:14.972 INFO:tasks.workunit.client.0.smithi038.stdout:5/959: stat d6/d16/d67/df6/f101 0 2024-03-24T00:49:14.972 INFO:tasks.workunit.client.0.smithi038.stdout:5/960: chown d6/d19/d20/d42/d43/d68 0 1 2024-03-24T00:49:14.972 INFO:tasks.workunit.client.0.smithi038.stdout:5/961: readlink d6/d60/lbd 0 2024-03-24T00:49:14.974 INFO:tasks.workunit.client.0.smithi038.stdout:2/936: dwrite d4/da3/d118/d7c/d103/f12e [0,4194304] 0 2024-03-24T00:49:14.974 INFO:tasks.workunit.client.0.smithi038.stdout:5/962: symlink d6/d16/d10f/d92/l137 0 2024-03-24T00:49:14.974 INFO:tasks.workunit.client.0.smithi038.stdout:5/963: chown d6/d16/d67/cda 774 1 2024-03-24T00:49:14.978 INFO:tasks.workunit.client.0.smithi038.stdout:5/964: creat d6/d1e/f138 x:0 0 0 2024-03-24T00:49:14.982 INFO:tasks.workunit.client.0.smithi038.stdout:5/965: creat d6/d19/d20/d42/d43/d4c/d111/f139 x:0 0 0 2024-03-24T00:49:14.982 INFO:tasks.workunit.client.0.smithi038.stdout:5/966: dread - d6/d19/d20/d42/d43/d68/dc8/f135 zero size 2024-03-24T00:49:14.990 INFO:tasks.workunit.client.0.smithi038.stdout:5/967: rename d6/d60/feb to d6/d19/d20/d42/d43/d68/dc8/de7/f13a 0 2024-03-24T00:49:14.990 INFO:tasks.workunit.client.0.smithi038.stdout:5/968: write d6/d1e/f138 [465472,47670] 0 2024-03-24T00:49:14.998 INFO:tasks.workunit.client.0.smithi038.stdout:5/969: unlink d6/d19/d20/d64/d6b/d77/cbc 0 2024-03-24T00:49:15.017 INFO:tasks.workunit.client.0.smithi038.stdout:5/970: write d6/d60/fca [2687766,24480] 0 2024-03-24T00:49:15.021 INFO:tasks.workunit.client.0.smithi038.stdout:5/971: link d6/d19/d20/d64/d6b/d77/f100 d6/d16/f13b 0 2024-03-24T00:49:15.022 INFO:tasks.workunit.client.0.smithi038.stdout:5/972: write d6/d19/d20/d33/f107 [51011,36325] 0 2024-03-24T00:49:15.023 INFO:tasks.workunit.client.0.smithi038.stdout:5/973: mknod d6/d16/d10f/d92/c13c 0 2024-03-24T00:49:15.023 INFO:tasks.workunit.client.0.smithi038.stdout:5/974: dread - d6/d19/d20/d70/ff3 zero size 2024-03-24T00:49:15.023 INFO:tasks.workunit.client.0.smithi038.stdout:5/975: write d6/d19/d20/d42/d43/d68/dc8/de7/f113 [824577,51246] 0 2024-03-24T00:49:15.023 INFO:tasks.workunit.client.0.smithi038.stdout:5/976: chown d6/d19/d20/d64/d6b/c108 3300158 1 2024-03-24T00:49:15.026 INFO:tasks.workunit.client.0.smithi038.stdout:5/977: getdents d6/d19/d20/d42/d43/d84 0 2024-03-24T00:49:15.030 INFO:tasks.workunit.client.0.smithi038.stdout:5/978: rmdir d6/d19/d20/d70/da3 39 2024-03-24T00:49:15.040 INFO:tasks.workunit.client.0.smithi038.stdout:2/937: dwrite d4/d27/d54/d53/f95 [4194304,4194304] 0 2024-03-24T00:49:15.040 INFO:tasks.workunit.client.0.smithi038.stdout:2/938: readlink d4/d27/d54/d53/de2/l128 0 2024-03-24T00:49:15.046 INFO:tasks.workunit.client.0.smithi038.stdout:2/939: symlink d4/d18/da1/l141 0 2024-03-24T00:49:15.077 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:14 smithi038 ceph-mon[29271]: pgmap v43: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 146 MiB/s rd, 163 MiB/s wr, 247 op/s 2024-03-24T00:49:15.086 INFO:tasks.workunit.client.0.smithi038.stdout:2/940: dwrite d4/d18/d26/d37/fd8 [4194304,4194304] 0 2024-03-24T00:49:15.106 INFO:tasks.workunit.client.0.smithi038.stdout:2/941: dread d4/d18/d38/fc0 [0,4194304] 0 2024-03-24T00:49:15.106 INFO:tasks.workunit.client.0.smithi038.stdout:2/942: fsync d4/d27/d54/d53/fba 0 2024-03-24T00:49:15.123 INFO:tasks.workunit.client.0.smithi038.stdout:2/943: mkdir d4/d102/d142 0 2024-03-24T00:49:15.123 INFO:tasks.workunit.client.0.smithi038.stdout:2/944: truncate d4/d18/da1/fa4 411106 0 2024-03-24T00:49:15.127 INFO:tasks.workunit.client.0.smithi038.stdout:2/945: truncate d4/d18/d26/f6d 2099475 0 2024-03-24T00:49:15.130 INFO:tasks.workunit.client.0.smithi038.stdout:2/946: link d4/d27/d54/d53/fa5 d4/d18/d26/d37/d69/da6/d121/f143 0 2024-03-24T00:49:15.130 INFO:tasks.workunit.client.0.smithi038.stdout:2/947: creat d4/d18/d26/d37/de0/f144 x:0 0 0 2024-03-24T00:49:15.135 INFO:tasks.workunit.client.0.smithi038.stdout:2/948: rename d4/d27/d54/d67/c90 to d4/d18/d5b/c145 0 2024-03-24T00:49:15.135 INFO:tasks.workunit.client.0.smithi038.stdout:2/949: chown d4/f19 125996 1 2024-03-24T00:49:15.135 INFO:tasks.workunit.client.0.smithi038.stdout:2/950: chown d4/d18/f7b 3558733 1 2024-03-24T00:49:15.138 INFO:tasks.workunit.client.0.smithi038.stdout:2/951: getdents d4/da3/d118/d10a/d13e 0 2024-03-24T00:49:15.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:14 smithi088 ceph-mon[36610]: pgmap v43: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 146 MiB/s rd, 163 MiB/s wr, 247 op/s 2024-03-24T00:49:15.278 INFO:tasks.workunit.client.0.smithi038.stdout:7/965: sync 2024-03-24T00:49:15.302 INFO:tasks.workunit.client.0.smithi038.stdout:7/966: write d2/d2c/d9a/fd0 [2305269,115150] 0 2024-03-24T00:49:15.382 INFO:tasks.workunit.client.0.smithi038.stdout:7/967: dwrite d2/d9/da/dc/d93/f116 [0,4194304] 0 2024-03-24T00:49:15.383 INFO:tasks.workunit.client.0.smithi038.stdout:7/968: fdatasync d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/f66 0 2024-03-24T00:49:15.390 INFO:tasks.workunit.client.0.smithi038.stdout:7/969: rename d2/d9/da/d23/d2b/d5b/d72/d92/d130/l15e to d2/d2c/l163 0 2024-03-24T00:49:15.392 INFO:tasks.workunit.client.0.smithi038.stdout:7/970: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d98/c164 0 2024-03-24T00:49:15.394 INFO:tasks.workunit.client.0.smithi038.stdout:7/971: unlink d2/d9/da/dc/d8a/fa5 0 2024-03-24T00:49:15.395 INFO:tasks.workunit.client.0.smithi038.stdout:7/972: mknod d2/d9/da/d23/d2b/d5b/d72/d92/d75/c165 0 2024-03-24T00:49:15.396 INFO:tasks.workunit.client.0.smithi038.stdout:5/979: sync 2024-03-24T00:49:15.396 INFO:tasks.workunit.client.0.smithi038.stdout:2/952: sync 2024-03-24T00:49:15.398 INFO:tasks.workunit.client.0.smithi038.stdout:2/953: creat d4/da3/d118/de8/d110/f146 x:0 0 0 2024-03-24T00:49:15.423 INFO:tasks.workunit.client.0.smithi038.stdout:7/973: write d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/d112/fc1 [328292,34038] 0 2024-03-24T00:49:15.425 INFO:tasks.workunit.client.0.smithi038.stdout:7/974: creat d2/d2c/d9a/deb/d12f/d133/f166 x:0 0 0 2024-03-24T00:49:15.427 INFO:tasks.workunit.client.0.smithi038.stdout:5/980: dread d6/d19/d20/d42/d43/d68/f8b [4194304,4194304] 0 2024-03-24T00:49:15.431 INFO:tasks.workunit.client.0.smithi038.stdout:5/981: rename d6/d60/fca to d6/f13d 0 2024-03-24T00:49:15.443 INFO:tasks.workunit.client.0.smithi038.stdout:7/975: write d2/d9/da/f36 [1020952,65717] 0 2024-03-24T00:49:15.444 INFO:tasks.workunit.client.0.smithi038.stdout:7/976: readlink d2/d9/da/d23/d2b/d5b/d72/d92/d5d/la6 0 2024-03-24T00:49:15.446 INFO:tasks.workunit.client.0.smithi038.stdout:7/977: chown d2/d9/da/d23/d2b/d5b/d72/d92/d5d/l79 4210 1 2024-03-24T00:49:15.446 INFO:tasks.workunit.client.0.smithi038.stdout:7/978: chown d2/d9/da/d23/d2b/d5b/d72/d92/d33/f27 1567226965 1 2024-03-24T00:49:15.448 INFO:tasks.workunit.client.0.smithi038.stdout:7/979: symlink d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/l167 0 2024-03-24T00:49:15.448 INFO:tasks.workunit.client.0.smithi038.stdout:7/980: chown d2/d9/da/d23/d2b/d5b/d72/d92/d98/dab/fe0 7391 1 2024-03-24T00:49:15.448 INFO:tasks.workunit.client.0.smithi038.stdout:7/981: chown d2/d9/da/dc/da8/dc5/fe3 565 1 2024-03-24T00:49:15.448 INFO:tasks.workunit.client.0.smithi038.stdout:7/982: creat d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/f168 x:0 0 0 2024-03-24T00:49:15.449 INFO:tasks.workunit.client.0.smithi038.stdout:7/983: rmdir d2/d2c/d9a/d109 39 2024-03-24T00:49:15.452 INFO:tasks.workunit.client.0.smithi038.stdout:7/984: mknod d2/d9/da/d23/d2b/d5b/d72/d92/c169 0 2024-03-24T00:49:15.454 INFO:tasks.workunit.client.0.smithi038.stdout:7/985: mkdir d2/d9/da/d23/d2b/d5b/d72/d92/d33/d37/dfe/d141/d16a 0 2024-03-24T00:49:15.454 INFO:tasks.workunit.client.0.smithi038.stdout:7/986: dread - d2/d9/d16/f128 zero size 2024-03-24T00:49:15.457 INFO:tasks.workunit.client.0.smithi038.stdout:2/954: dwrite d4/d18/d26/f4e [8388608,4194304] 0 2024-03-24T00:49:15.457 INFO:tasks.workunit.client.0.smithi038.stdout:7/987: symlink d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/l16b 0 2024-03-24T00:49:15.457 INFO:tasks.workunit.client.0.smithi038.stdout:7/988: creat d2/d9/da/d23/d2b/d5b/f16c x:0 0 0 2024-03-24T00:49:15.458 INFO:tasks.workunit.client.0.smithi038.stdout:2/955: rename d4/d27/d54/d53/l10e to d4/d27/d54/d67/d111/l147 0 2024-03-24T00:49:15.461 INFO:tasks.workunit.client.0.smithi038.stdout:7/989: mkdir d2/d2c/d9a/dcb/d132/d16d 0 2024-03-24T00:49:15.463 INFO:tasks.workunit.client.0.smithi038.stdout:7/990: creat d2/d9/da/dc/d8a/f16e x:0 0 0 2024-03-24T00:49:15.466 INFO:tasks.workunit.client.0.smithi038.stdout:7/991: creat d2/d9/da/d23/d2b/d5b/d72/d92/d5d/dcd/f16f x:0 0 0 2024-03-24T00:49:15.471 INFO:tasks.workunit.client.0.smithi038.stdout:2/956: dread d4/d27/d2f/f3c [0,4194304] 0 2024-03-24T00:49:15.471 INFO:tasks.workunit.client.0.smithi038.stdout:2/957: readlink d4/d18/lfe 0 2024-03-24T00:49:15.471 INFO:tasks.workunit.client.0.smithi038.stdout:2/958: readlink d4/d18/d26/d37/l52 0 2024-03-24T00:49:15.474 INFO:tasks.workunit.client.0.smithi038.stdout:2/959: creat d4/da3/d118/d10a/d13e/f148 x:0 0 0 2024-03-24T00:49:15.474 INFO:tasks.workunit.client.0.smithi038.stdout:2/960: chown d4/d27/d54/d67/d111/l147 459230 1 2024-03-24T00:49:15.478 INFO:tasks.workunit.client.0.smithi038.stdout:2/961: mkdir d4/d27/d2f/d149 0 2024-03-24T00:49:15.478 INFO:tasks.workunit.client.0.smithi038.stdout:2/962: fdatasync d4/d27/f106 0 2024-03-24T00:49:15.478 INFO:tasks.workunit.client.0.smithi038.stdout:2/963: dread - d4/da3/d118/f136 zero size 2024-03-24T00:49:15.485 INFO:tasks.workunit.client.0.smithi038.stdout:7/992: write d2/d9/da/d23/d2b/d5b/d72/d92/d33/f31 [1548704,48156] 0 2024-03-24T00:49:15.489 INFO:tasks.workunit.client.0.smithi038.stdout:2/964: rename d4/d27/d54/dab/d104 to d4/da3/d118/d10a/d13e/d14a 0 2024-03-24T00:49:15.506 INFO:tasks.workunit.client.0.smithi038.stdout:5/982: dwrite d6/d23/f32 [0,4194304] 0 2024-03-24T00:49:15.509 INFO:tasks.workunit.client.0.smithi038.stdout:5/983: creat d6/d19/d20/d70/f13e x:0 0 0 2024-03-24T00:49:15.532 INFO:tasks.workunit.client.0.smithi038.stdout:5/984: dread d6/d19/d20/d42/d43/d87/f132 [0,4194304] 0 2024-03-24T00:49:15.559 INFO:tasks.workunit.client.0.smithi038.stdout:5/985: read d6/d19/d20/d42/fd5 [7599769,49691] 0 2024-03-24T00:49:15.561 INFO:tasks.workunit.client.0.smithi038.stdout:5/986: creat d6/d16/d21/d2d/d127/d131/f13f x:0 0 0 2024-03-24T00:49:15.565 INFO:tasks.workunit.client.0.smithi038.stdout:5/987: creat d6/d19/d12c/f140 x:0 0 0 2024-03-24T00:49:15.567 INFO:tasks.workunit.client.0.smithi038.stdout:5/988: creat d6/d19/d20/d42/d43/d84/f141 x:0 0 0 2024-03-24T00:49:15.567 INFO:tasks.workunit.client.0.smithi038.stdout:5/989: chown d6/d19/d20/d42/d43/d126 1 1 2024-03-24T00:49:15.569 INFO:tasks.workunit.client.0.smithi038.stdout:5/990: symlink d6/d19/d20/d33/l142 0 2024-03-24T00:49:15.581 INFO:tasks.workunit.client.0.smithi038.stdout:7/993: dwrite d2/d9/d16/f128 [0,4194304] 0 2024-03-24T00:49:15.585 INFO:tasks.workunit.client.0.smithi038.stdout:7/994: creat d2/d9/da/d23/def/df0/f170 x:0 0 0 2024-03-24T00:49:15.587 INFO:tasks.workunit.client.0.smithi038.stdout:7/995: mknod d2/d9/da/d23/d2b/d5b/c171 0 2024-03-24T00:49:15.593 INFO:tasks.workunit.client.0.smithi038.stdout:7/996: rmdir d2/d2c/d9a/dcb/d132 39 2024-03-24T00:49:15.598 INFO:tasks.workunit.client.0.smithi038.stdout:2/965: dwrite d4/da3/d118/d11f/f10c [0,4194304] 0 2024-03-24T00:49:15.642 INFO:tasks.workunit.client.0.smithi038.stdout:7/997: dwrite d2/d9/da/d23/d2b/d5b/d72/d92/d75/d7d/db4/fd6 [0,4194304] 0 2024-03-24T00:49:15.647 INFO:tasks.workunit.client.0.smithi038.stdout:5/991: dwrite d6/f9 [0,4194304] 0 2024-03-24T00:49:15.650 INFO:tasks.workunit.client.0.smithi038.stdout:5/992: rename d6/d16/d21/d2d/d127/f130 to d6/d16/f143 0 2024-03-24T00:49:15.654 INFO:tasks.workunit.client.0.smithi038.stdout:5/993: mknod d6/d19/d20/d42/d43/d68/dc8/df5/c144 0 2024-03-24T00:49:15.654 INFO:tasks.workunit.client.0.smithi038.stdout:5/994: chown d6/d60 3 1 2024-03-24T00:49:15.656 INFO:tasks.workunit.client.0.smithi038.stdout:5/995: creat d6/d16/f145 x:0 0 0 2024-03-24T00:49:15.664 INFO:tasks.workunit.client.0.smithi038.stdout:7/998: write d2/d9/fa7 [3244723,116493] 0 2024-03-24T00:49:15.664 INFO:tasks.workunit.client.0.smithi038.stdout:7/999: dread - d2/d9/da/f3f zero size 2024-03-24T00:49:15.712 INFO:tasks.workunit.client.0.smithi038.stdout:5/996: dwrite d6/d16/f13b [0,4194304] 0 2024-03-24T00:49:15.714 INFO:tasks.workunit.client.0.smithi038.stdout:5/997: creat d6/f146 x:0 0 0 2024-03-24T00:49:15.715 INFO:tasks.workunit.client.0.smithi038.stdout:5/998: symlink d6/d19/d20/d42/l147 0 2024-03-24T00:49:15.717 INFO:tasks.workunit.client.0.smithi038.stdout:5/999: truncate d6/d19/d20/d42/d43/d68/f8b 12035982 0 2024-03-24T00:49:15.970 INFO:tasks.workunit.client.0.smithi038.stdout:2/966: sync 2024-03-24T00:49:15.972 INFO:tasks.workunit.client.0.smithi038.stdout:2/967: rename d4/d27/d2f/f3c to d4/d27/d2f/d30/f14b 0 2024-03-24T00:49:15.974 INFO:tasks.workunit.client.0.smithi038.stdout:2/968: creat d4/d27/d54/d53/de2/f14c x:0 0 0 2024-03-24T00:49:15.976 INFO:tasks.workunit.client.0.smithi038.stdout:2/969: getdents d4/d27/d54/dab 0 2024-03-24T00:49:15.976 INFO:tasks.workunit.client.0.smithi038.stdout:2/970: write d4/da3/d118/de8/d110/f146 [472633,98882] 0 2024-03-24T00:49:16.003 INFO:tasks.workunit.client.0.smithi038.stdout:2/971: write d4/d18/f28 [3383587,26574] 0 2024-03-24T00:49:16.004 INFO:tasks.workunit.client.0.smithi038.stdout:2/972: rmdir d4/d18/d26/d37 39 2024-03-24T00:49:16.007 INFO:tasks.workunit.client.0.smithi038.stdout:2/973: creat d4/da3/d118/de8/f14d x:0 0 0 2024-03-24T00:49:16.007 INFO:tasks.workunit.client.0.smithi038.stdout:2/974: getdents d4/d102/d142 0 2024-03-24T00:49:16.011 INFO:tasks.workunit.client.0.smithi038.stdout:2/975: creat d4/d18/d26/d37/d69/f14e x:0 0 0 2024-03-24T00:49:16.024 INFO:tasks.workunit.client.0.smithi038.stdout:2/976: dread d4/d76/fb1 [0,4194304] 0 2024-03-24T00:49:16.067 INFO:tasks.workunit.client.0.smithi038.stdout:2/977: dread d4/d18/f28 [0,4194304] 0 2024-03-24T00:49:16.067 INFO:tasks.workunit.client.0.smithi038.stdout:2/978: chown d4/d18/d26/d37/d69/da6 1017757 1 2024-03-24T00:49:16.069 INFO:tasks.workunit.client.0.smithi038.stdout:2/979: creat d4/d18/d26/d37/f14f x:0 0 0 2024-03-24T00:49:16.307 INFO:tasks.workunit.client.0.smithi038.stdout:2/980: sync 2024-03-24T00:49:16.320 INFO:tasks.workunit.client.0.smithi038.stdout:2/981: symlink d4/d18/d26/d117/l150 0 2024-03-24T00:49:16.340 INFO:tasks.workunit.client.0.smithi038.stdout:2/982: dread d4/d18/d5b/f113 [0,4194304] 0 2024-03-24T00:49:16.340 INFO:tasks.workunit.client.0.smithi038.stdout:2/983: read d4/d27/f94 [2904990,60929] 0 2024-03-24T00:49:16.345 INFO:tasks.workunit.client.0.smithi038.stdout:2/984: creat d4/d27/d2f/d13f/f151 x:0 0 0 2024-03-24T00:49:16.371 INFO:tasks.workunit.client.0.smithi038.stdout:2/985: dread d4/da3/d118/d9b/fa7 [0,4194304] 0 2024-03-24T00:49:16.461 INFO:tasks.workunit.client.0.smithi038.stdout:2/986: dwrite d4/d18/f125 [4194304,4194304] 0 2024-03-24T00:49:16.461 INFO:tasks.workunit.client.0.smithi038.stdout:2/987: fsync d4/d18/fe1 0 2024-03-24T00:49:16.621 INFO:tasks.workunit.client.0.smithi038.stdout:2/988: rmdir d4/da3/d118/d10a 39 2024-03-24T00:49:16.621 INFO:tasks.workunit.client.0.smithi038.stdout:2/989: dread - d4/d27/d54/feb zero size 2024-03-24T00:49:16.621 INFO:tasks.workunit.client.0.smithi038.stdout:2/990: write d4/fdd [1396240,115351] 0 2024-03-24T00:49:16.641 INFO:tasks.workunit.client.0.smithi038.stdout:2/991: dread d4/d18/d26/f2c [0,4194304] 0 2024-03-24T00:49:16.652 INFO:tasks.workunit.client.0.smithi038.stdout:2/992: dread d4/da3/d118/de8/d110/f146 [0,4194304] 0 2024-03-24T00:49:16.652 INFO:tasks.workunit.client.0.smithi038.stdout:2/993: dread - d4/da3/d118/d10a/d13e/f148 zero size 2024-03-24T00:49:16.652 INFO:tasks.workunit.client.0.smithi038.stdout:2/994: truncate d4/d18/d5b/fe7 1521458 0 2024-03-24T00:49:16.652 INFO:tasks.workunit.client.0.smithi038.stdout:2/995: chown d4/d55/l120 0 1 2024-03-24T00:49:16.652 INFO:tasks.workunit.client.0.smithi038.stdout:2/996: chown d4/da3/d118/d10a/d13e/f138 48817 1 2024-03-24T00:49:16.652 INFO:tasks.workunit.client.0.smithi038.stdout:2/997: readlink d4/d27/d54/d67/l119 0 2024-03-24T00:49:16.755 INFO:tasks.workunit.client.0.smithi038.stdout:2/998: write d4/d18/da1/fa4 [278852,75087] 0 2024-03-24T00:49:16.757 INFO:tasks.workunit.client.0.smithi038.stdout:2/999: mkdir d4/da3/d118/d7c/d152 0 2024-03-24T00:49:16.765 INFO:tasks.workunit.client.0.smithi038.stderr:+ rm -rf -- ./tmp.zG281EyAMe 2024-03-24T00:49:17.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:16 smithi038 ceph-mon[29271]: pgmap v44: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 220 MiB/s rd, 239 MiB/s wr, 377 op/s 2024-03-24T00:49:17.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:16 smithi088 ceph-mon[36610]: pgmap v44: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 220 MiB/s rd, 239 MiB/s wr, 377 op/s 2024-03-24T00:49:19.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:18 smithi038 ceph-mon[29271]: pgmap v45: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 145 MiB/s rd, 158 MiB/s wr, 251 op/s 2024-03-24T00:49:19.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:18 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:19.125 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:18 smithi088 ceph-mon[36610]: pgmap v45: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 145 MiB/s rd, 158 MiB/s wr, 251 op/s 2024-03-24T00:49:19.125 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:18 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:49:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:49:20.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:20.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:49:20.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mgr fail", "who": "smithi088.njctea"}]: dispatch 2024-03-24T00:49:20.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mgr fail", "who": "smithi088.njctea"}]: dispatch 2024-03-24T00:49:20.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: osdmap e49: 6 total, 6 up, 6 in 2024-03-24T00:49:20.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd='[{"prefix": "mgr fail", "who": "smithi088.njctea"}]': finished 2024-03-24T00:49:20.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: mgrmap e27: smithi038.huhioz(active, starting, since 0.0122242s) 2024-03-24T00:49:20.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:49:20.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:49:20.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:49:20.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi038.huhioz", "id": "smithi038.huhioz"}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T00:49:20.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: Manager daemon smithi038.huhioz is now available 2024-03-24T00:49:20.113 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:49:20.113 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:49:20.113 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:49:20.113 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:49:20.113 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/trash_purge_schedule"}]: dispatch 2024-03-24T00:49:20.150 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:20.150 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:49:20.150 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:49:20.150 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' 2024-03-24T00:49:20.150 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:49:20.150 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.24351 172.21.15.88:0/2409183175' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mgr fail", "who": "smithi088.njctea"}]: dispatch 2024-03-24T00:49:20.150 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd=[{"prefix": "mgr fail", "who": "smithi088.njctea"}]: dispatch 2024-03-24T00:49:20.150 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: osdmap e49: 6 total, 6 up, 6 in 2024-03-24T00:49:20.150 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.24351 ' entity='mgr.smithi088.njctea' cmd='[{"prefix": "mgr fail", "who": "smithi088.njctea"}]': finished 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: mgrmap e27: smithi038.huhioz(active, starting, since 0.0122242s) 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi038.huhioz", "id": "smithi038.huhioz"}]: dispatch 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:49:20.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:49:20.152 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:49:20.152 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T00:49:20.152 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T00:49:20.152 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T00:49:20.152 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: Manager daemon smithi038.huhioz is now available 2024-03-24T00:49:20.152 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:49:20.152 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:49:20.152 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:49:20.152 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:49:20.152 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/trash_purge_schedule"}]: dispatch 2024-03-24T00:49:21.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:21 smithi088 ceph-mon[36610]: mgrmap e28: smithi038.huhioz(active, since 1.02158s) 2024-03-24T00:49:21.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:21 smithi088 ceph-mon[36610]: pgmap v3: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-03-24T00:49:21.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:21 smithi038 ceph-mon[29271]: mgrmap e28: smithi038.huhioz(active, since 1.02158s) 2024-03-24T00:49:21.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:21 smithi038 ceph-mon[29271]: pgmap v3: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-03-24T00:49:22.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:22 smithi038 ceph-mon[29271]: [24/Mar/2024:00:49:20] ENGINE Bus STARTING 2024-03-24T00:49:22.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:22 smithi038 ceph-mon[29271]: [24/Mar/2024:00:49:20] ENGINE Serving on http://172.21.15.38:8765 2024-03-24T00:49:22.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:22 smithi038 ceph-mon[29271]: [24/Mar/2024:00:49:21] ENGINE Serving on https://172.21.15.38:7150 2024-03-24T00:49:22.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:22 smithi038 ceph-mon[29271]: [24/Mar/2024:00:49:21] ENGINE Bus STARTED 2024-03-24T00:49:22.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:22 smithi038 ceph-mon[29271]: pgmap v4: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-03-24T00:49:22.403 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:22 smithi088 ceph-mon[36610]: [24/Mar/2024:00:49:20] ENGINE Bus STARTING 2024-03-24T00:49:22.404 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:22 smithi088 ceph-mon[36610]: [24/Mar/2024:00:49:20] ENGINE Serving on http://172.21.15.38:8765 2024-03-24T00:49:22.404 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:22 smithi088 ceph-mon[36610]: [24/Mar/2024:00:49:21] ENGINE Serving on https://172.21.15.38:7150 2024-03-24T00:49:22.404 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:22 smithi088 ceph-mon[36610]: [24/Mar/2024:00:49:21] ENGINE Bus STARTED 2024-03-24T00:49:22.404 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:22 smithi088 ceph-mon[36610]: pgmap v4: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-03-24T00:49:23.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:23 smithi088 ceph-mon[36610]: mgrmap e29: smithi038.huhioz(active, since 3s) 2024-03-24T00:49:23.437 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:23 smithi038 ceph-mon[29271]: mgrmap e29: smithi038.huhioz(active, since 3s) 2024-03-24T00:49:24.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:24 smithi038 ceph-mon[29271]: pgmap v5: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-03-24T00:49:24.517 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:24 smithi088 ceph-mon[36610]: pgmap v5: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-03-24T00:49:25.419 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:25 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:25.419 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:25 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:25.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:25 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:25.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:25 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:26.595 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:26 smithi088 ceph-mon[36610]: pgmap v6: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 821 KiB/s rd, 856 KiB/s wr, 101 op/s 2024-03-24T00:49:26.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:26 smithi038 ceph-mon[29271]: pgmap v6: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 821 KiB/s rd, 856 KiB/s wr, 101 op/s 2024-03-24T00:49:27.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:27 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:27.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:27 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:27.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:27 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:27.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:27 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:27.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:27 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:27.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:27 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:27.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:27 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:27.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:27 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:28.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:28 smithi038 ceph-mon[29271]: pgmap v7: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 1.0 MiB/s rd, 985 KiB/s wr, 140 op/s 2024-03-24T00:49:28.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:28 smithi088 ceph-mon[36610]: pgmap v7: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 1.0 MiB/s rd, 985 KiB/s wr, 140 op/s 2024-03-24T00:49:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:29 smithi088 ceph-mon[36610]: pgmap v8: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 829 KiB/s rd, 766 KiB/s wr, 109 op/s 2024-03-24T00:49:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:29 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:29 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:29 smithi088 ceph-mon[36610]: Standby manager daemon smithi088.njctea started 2024-03-24T00:49:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:29 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/2003702387' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/crt"}]: dispatch 2024-03-24T00:49:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:29 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/2003702387' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:49:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:29 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/2003702387' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/key"}]: dispatch 2024-03-24T00:49:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:29 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/2003702387' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:49:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:29 smithi038 ceph-mon[29271]: pgmap v8: 65 pgs: 65 active+clean; 2.6 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 829 KiB/s rd, 766 KiB/s wr, 109 op/s 2024-03-24T00:49:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:29 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:29 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:29 smithi038 ceph-mon[29271]: Standby manager daemon smithi088.njctea started 2024-03-24T00:49:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:29 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/2003702387' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/crt"}]: dispatch 2024-03-24T00:49:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:29 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/2003702387' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:49:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:29 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/2003702387' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/key"}]: dispatch 2024-03-24T00:49:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:29 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/2003702387' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:49:31.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:31 smithi038 ceph-mon[29271]: mgrmap e30: smithi038.huhioz(active, since 11s), standbys: smithi088.njctea 2024-03-24T00:49:31.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:31 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi088.njctea", "id": "smithi088.njctea"}]: dispatch 2024-03-24T00:49:31.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:31 smithi038 ceph-mon[29271]: pgmap v9: 65 pgs: 65 active+clean; 1.4 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-03-24T00:49:31.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:31 smithi088 ceph-mon[36610]: mgrmap e30: smithi038.huhioz(active, since 11s), standbys: smithi088.njctea 2024-03-24T00:49:31.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:31 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi088.njctea", "id": "smithi088.njctea"}]: dispatch 2024-03-24T00:49:31.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:31 smithi088 ceph-mon[36610]: pgmap v9: 65 pgs: 65 active+clean; 1.4 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-03-24T00:49:33.098 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-24T00:49:33.099 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-24T00:49:34.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:34 smithi038 ceph-mon[29271]: pgmap v10: 65 pgs: 65 active+clean; 1.4 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-03-24T00:49:34.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:34 smithi088 ceph-mon[36610]: pgmap v10: 65 pgs: 65 active+clean; 1.4 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-03-24T00:49:35.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:49:35.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:49:36.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:36 smithi038 ceph-mon[29271]: pgmap v11: 65 pgs: 65 active+clean; 1.1 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-03-24T00:49:36.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:36 smithi088 ceph-mon[36610]: pgmap v11: 65 pgs: 65 active+clean; 1.1 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 163 op/s 2024-03-24T00:49:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.762+0000 7fa330c00700 1 -- 172.21.15.38:0/3141146411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa32c0ff350 msgr2=0x7fa32c101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.762+0000 7fa330c00700 1 --2- 172.21.15.38:0/3141146411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa32c0ff350 0x7fa32c101780 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009910 tx=0x7fa31c009c20 comp rx=0 tx=0).stop 2024-03-24T00:49:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.762+0000 7fa330c00700 1 -- 172.21.15.38:0/3141146411 shutdown_connections 2024-03-24T00:49:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.762+0000 7fa330c00700 1 --2- 172.21.15.38:0/3141146411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa32c101cc0 0x7fa32c104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.762+0000 7fa330c00700 1 --2- 172.21.15.38:0/3141146411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa32c0ff350 0x7fa32c101780 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:36.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.762+0000 7fa330c00700 1 -- 172.21.15.38:0/3141146411 >> 172.21.15.38:0/3141146411 conn(0x7fa32c0faf70 msgr2=0x7fa32c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.763+0000 7fa330c00700 1 -- 172.21.15.38:0/3141146411 shutdown_connections 2024-03-24T00:49:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.763+0000 7fa330c00700 1 -- 172.21.15.38:0/3141146411 wait complete. 2024-03-24T00:49:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.764+0000 7fa330c00700 1 Processor -- start 2024-03-24T00:49:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.764+0000 7fa330c00700 1 -- start start 2024-03-24T00:49:36.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.765+0000 7fa330c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa32c0ff350 0x7fa32c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:36.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.765+0000 7fa330c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa32c101cc0 0x7fa32c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:36.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.765+0000 7fa330c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c19d5e0 con 0x7fa32c0ff350 2024-03-24T00:49:36.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.765+0000 7fa330c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c19d720 con 0x7fa32c101cc0 2024-03-24T00:49:36.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.765+0000 7fa329e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa32c0ff350 0x7fa32c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:36.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.765+0000 7fa329e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa32c0ff350 0x7fa32c19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48484/0 (socket says 172.21.15.38:48484) 2024-03-24T00:49:36.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.765+0000 7fa329e00700 1 -- 172.21.15.38:0/4033325186 learned_addr learned my addr 172.21.15.38:0/4033325186 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:49:36.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.765+0000 7fa329400700 1 --2- 172.21.15.38:0/4033325186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa32c101cc0 0x7fa32c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:36.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.766+0000 7fa329e00700 1 -- 172.21.15.38:0/4033325186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa32c101cc0 msgr2=0x7fa32c19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:36.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.766+0000 7fa329e00700 1 --2- 172.21.15.38:0/4033325186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa32c101cc0 0x7fa32c19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:36.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.766+0000 7fa329e00700 1 -- 172.21.15.38:0/4033325186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c0095f0 con 0x7fa32c0ff350 2024-03-24T00:49:36.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.766+0000 7fa329e00700 1 --2- 172.21.15.38:0/4033325186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa32c0ff350 0x7fa32c19ca80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa31c005b40 tx=0x7fa31c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:36.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.766+0000 7fa322a00700 1 -- 172.21.15.38:0/4033325186 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31c01d070 con 0x7fa32c0ff350 2024-03-24T00:49:36.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.766+0000 7fa322a00700 1 -- 172.21.15.38:0/4033325186 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa31c0050d0 con 0x7fa32c0ff350 2024-03-24T00:49:36.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.766+0000 7fa322a00700 1 -- 172.21.15.38:0/4033325186 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31c0226a0 con 0x7fa32c0ff350 2024-03-24T00:49:36.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.767+0000 7fa330c00700 1 -- 172.21.15.38:0/4033325186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c1a2120 con 0x7fa32c0ff350 2024-03-24T00:49:36.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.767+0000 7fa330c00700 1 -- 172.21.15.38:0/4033325186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c1a2690 con 0x7fa32c0ff350 2024-03-24T00:49:36.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.768+0000 7fa330c00700 1 -- 172.21.15.38:0/4033325186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c04ea80 con 0x7fa32c0ff350 2024-03-24T00:49:36.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.770+0000 7fa322a00700 1 -- 172.21.15.38:0/4033325186 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 91796+0+0 (secure 0 0 0) 0x7fa31c003680 con 0x7fa32c0ff350 2024-03-24T00:49:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.771+0000 7fa322a00700 1 --2- 172.21.15.38:0/4033325186 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa31006dd50 0x7fa310070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.771+0000 7fa322a00700 1 -- 172.21.15.38:0/4033325186 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa31c08e9b0 con 0x7fa32c0ff350 2024-03-24T00:49:36.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.772+0000 7fa329400700 1 --2- 172.21.15.38:0/4033325186 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa31006dd50 0x7fa310070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.772+0000 7fa329400700 1 --2- 172.21.15.38:0/4033325186 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa31006dd50 0x7fa310070210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa318009bf0 tx=0x7fa318009260 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:36.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:36.777+0000 7fa322a00700 1 -- 172.21.15.38:0/4033325186 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fa31c05bc90 con 0x7fa32c0ff350 2024-03-24T00:49:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.210+0000 7fa330c00700 1 -- 172.21.15.38:0/4033325186 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa32c103c40 con 0x7fa31006dd50 2024-03-24T00:49:37.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.212+0000 7fa322a00700 1 -- 172.21.15.38:0/4033325186 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fa32c103c40 con 0x7fa31006dd50 2024-03-24T00:49:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.216+0000 7fa330c00700 1 -- 172.21.15.38:0/4033325186 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa31006dd50 msgr2=0x7fa310070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.216+0000 7fa330c00700 1 --2- 172.21.15.38:0/4033325186 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa31006dd50 0x7fa310070210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa318009bf0 tx=0x7fa318009260 comp rx=0 tx=0).stop 2024-03-24T00:49:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.216+0000 7fa330c00700 1 -- 172.21.15.38:0/4033325186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa32c0ff350 msgr2=0x7fa32c19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.216+0000 7fa330c00700 1 --2- 172.21.15.38:0/4033325186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa32c0ff350 0x7fa32c19ca80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa31c005b40 tx=0x7fa31c00f740 comp rx=0 tx=0).stop 2024-03-24T00:49:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.217+0000 7fa330c00700 1 -- 172.21.15.38:0/4033325186 shutdown_connections 2024-03-24T00:49:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.217+0000 7fa330c00700 1 --2- 172.21.15.38:0/4033325186 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa31006dd50 0x7fa310070210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.217+0000 7fa330c00700 1 --2- 172.21.15.38:0/4033325186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa32c0ff350 0x7fa32c19ca80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.217+0000 7fa330c00700 1 --2- 172.21.15.38:0/4033325186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa32c101cc0 0x7fa32c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.217+0000 7fa330c00700 1 -- 172.21.15.38:0/4033325186 >> 172.21.15.38:0/4033325186 conn(0x7fa32c0faf70 msgr2=0x7fa32c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.218+0000 7fa330c00700 1 -- 172.21.15.38:0/4033325186 shutdown_connections 2024-03-24T00:49:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.218+0000 7fa330c00700 1 -- 172.21.15.38:0/4033325186 wait complete. 2024-03-24T00:49:37.244 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:49:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.435+0000 7f263b800700 1 -- 172.21.15.38:0/853786438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26340737b0 msgr2=0x7f2634073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.435+0000 7f263b800700 1 --2- 172.21.15.38:0/853786438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26340737b0 0x7f2634073c30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2620009910 tx=0x7f2620009c20 comp rx=0 tx=0).stop 2024-03-24T00:49:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.435+0000 7f263b800700 1 -- 172.21.15.38:0/853786438 shutdown_connections 2024-03-24T00:49:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.435+0000 7f263b800700 1 --2- 172.21.15.38:0/853786438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26340737b0 0x7f2634073c30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.435+0000 7f263b800700 1 --2- 172.21.15.38:0/853786438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2634074d80 0x7f26340731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.435+0000 7f263b800700 1 -- 172.21.15.38:0/853786438 >> 172.21.15.38:0/853786438 conn(0x7f26340fc0a0 msgr2=0x7f26340fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.435+0000 7f263b800700 1 -- 172.21.15.38:0/853786438 shutdown_connections 2024-03-24T00:49:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.435+0000 7f263b800700 1 -- 172.21.15.38:0/853786438 wait complete. 2024-03-24T00:49:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.437+0000 7f263b800700 1 Processor -- start 2024-03-24T00:49:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.437+0000 7f263b800700 1 -- start start 2024-03-24T00:49:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.438+0000 7f263b800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26340737b0 0x7f26341986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.438+0000 7f263b800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634074d80 0x7f2634198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.438+0000 7f263b800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2634199210 con 0x7f2634074d80 2024-03-24T00:49:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.438+0000 7f263b800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2634199350 con 0x7f26340737b0 2024-03-24T00:49:37.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.438+0000 7f2633e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634074d80 0x7f2634198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:37.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.438+0000 7f2633e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634074d80 0x7f2634198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48514/0 (socket says 172.21.15.38:48514) 2024-03-24T00:49:37.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.438+0000 7f2633e00700 1 -- 172.21.15.38:0/3123504181 learned_addr learned my addr 172.21.15.38:0/3123504181 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:49:37.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.438+0000 7f2638e00700 1 --2- 172.21.15.38:0/3123504181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26340737b0 0x7f26341986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:37.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.439+0000 7f2638e00700 1 -- 172.21.15.38:0/3123504181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634074d80 msgr2=0x7f2634198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:37.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.439+0000 7f2638e00700 1 --2- 172.21.15.38:0/3123504181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634074d80 0x7f2634198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:37.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.439+0000 7f2638e00700 1 -- 172.21.15.38:0/3123504181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26200095f0 con 0x7f26340737b0 2024-03-24T00:49:37.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.439+0000 7f2638e00700 1 --2- 172.21.15.38:0/3123504181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26340737b0 0x7f26341986b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f262800d8d0 tx=0x7f262800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:37.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.440+0000 7f2631600700 1 -- 172.21.15.38:0/3123504181 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2628009900 con 0x7f26340737b0 2024-03-24T00:49:37.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.440+0000 7f263b800700 1 -- 172.21.15.38:0/3123504181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263419ddb0 con 0x7f26340737b0 2024-03-24T00:49:37.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.440+0000 7f263b800700 1 -- 172.21.15.38:0/3123504181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263419e2f0 con 0x7f26340737b0 2024-03-24T00:49:37.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.440+0000 7f2631600700 1 -- 172.21.15.38:0/3123504181 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2628010460 con 0x7f26340737b0 2024-03-24T00:49:37.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.440+0000 7f2631600700 1 -- 172.21.15.38:0/3123504181 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f262800b5d0 con 0x7f26340737b0 2024-03-24T00:49:37.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.442+0000 7f263b800700 1 -- 172.21.15.38:0/3123504181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2614005320 con 0x7f26340737b0 2024-03-24T00:49:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.443+0000 7f2631600700 1 -- 172.21.15.38:0/3123504181 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) v1 ==== 91796+0+0 (secure 0 0 0) 0x7f2628009a60 con 0x7f26340737b0 2024-03-24T00:49:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.445+0000 7f2631600700 1 --2- 172.21.15.38:0/3123504181 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f261c06dd50 0x7f261c070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.445+0000 7f2631600700 1 -- 172.21.15.38:0/3123504181 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f262808d790 con 0x7f26340737b0 2024-03-24T00:49:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.445+0000 7f2633e00700 1 --2- 172.21.15.38:0/3123504181 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f261c06dd50 0x7f261c070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.446+0000 7f2633e00700 1 --2- 172.21.15.38:0/3123504181 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f261c06dd50 0x7f261c070210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f262000b5c0 tx=0x7f2620005a90 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:37.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.460+0000 7f2631600700 1 -- 172.21.15.38:0/3123504181 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f262805a6c0 con 0x7f26340737b0 2024-03-24T00:49:37.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.810+0000 7f263b800700 1 -- 172.21.15.38:0/3123504181 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2614000bf0 con 0x7f261c06dd50 2024-03-24T00:49:37.810 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:37 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:37.811 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:37 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:37.811 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:37 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:49:37.811 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:37 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:49:37.811 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:37 smithi038 ceph-mon[29271]: Updating smithi038:/etc/ceph/ceph.conf 2024-03-24T00:49:37.811 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:37 smithi038 ceph-mon[29271]: Updating smithi088:/etc/ceph/ceph.conf 2024-03-24T00:49:37.811 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:37 smithi038 ceph-mon[29271]: pgmap v12: 65 pgs: 65 active+clean; 654 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-03-24T00:49:37.811 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:37 smithi038 ceph-mon[29271]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:49:37.811 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:37 smithi038 ceph-mon[29271]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:37.811 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:37 smithi038 ceph-mon[29271]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:49:37.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.812+0000 7f2631600700 1 -- 172.21.15.38:0/3123504181 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f2614000bf0 con 0x7f261c06dd50 2024-03-24T00:49:37.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.814+0000 7f263b800700 1 -- 172.21.15.38:0/3123504181 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f261c06dd50 msgr2=0x7f261c070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:37.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.814+0000 7f263b800700 1 --2- 172.21.15.38:0/3123504181 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f261c06dd50 0x7f261c070210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f262000b5c0 tx=0x7f2620005a90 comp rx=0 tx=0).stop 2024-03-24T00:49:37.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.814+0000 7f263b800700 1 -- 172.21.15.38:0/3123504181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26340737b0 msgr2=0x7f26341986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:37.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.814+0000 7f263b800700 1 --2- 172.21.15.38:0/3123504181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26340737b0 0x7f26341986b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f262800d8d0 tx=0x7f262800dc90 comp rx=0 tx=0).stop 2024-03-24T00:49:37.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.814+0000 7f263b800700 1 -- 172.21.15.38:0/3123504181 shutdown_connections 2024-03-24T00:49:37.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.814+0000 7f263b800700 1 --2- 172.21.15.38:0/3123504181 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f261c06dd50 0x7f261c070210 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:37.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.814+0000 7f263b800700 1 --2- 172.21.15.38:0/3123504181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26340737b0 0x7f26341986b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:37.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.814+0000 7f263b800700 1 --2- 172.21.15.38:0/3123504181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2634074d80 0x7f2634198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:37.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.814+0000 7f263b800700 1 -- 172.21.15.38:0/3123504181 >> 172.21.15.38:0/3123504181 conn(0x7f26340fc0a0 msgr2=0x7f2634106d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:37.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.814+0000 7f263b800700 1 -- 172.21.15.38:0/3123504181 shutdown_connections 2024-03-24T00:49:37.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:37.814+0000 7f263b800700 1 -- 172.21.15.38:0/3123504181 wait complete. 2024-03-24T00:49:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:37 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:37 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:37 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:49:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:37 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:49:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:37 smithi088 ceph-mon[36610]: Updating smithi038:/etc/ceph/ceph.conf 2024-03-24T00:49:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:37 smithi088 ceph-mon[36610]: Updating smithi088:/etc/ceph/ceph.conf 2024-03-24T00:49:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:37 smithi088 ceph-mon[36610]: pgmap v12: 65 pgs: 65 active+clean; 654 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-03-24T00:49:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:37 smithi088 ceph-mon[36610]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:49:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:37 smithi088 ceph-mon[36610]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:37 smithi088 ceph-mon[36610]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:49:38.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.016+0000 7ff225600700 1 -- 172.21.15.38:0/4125838982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff220074d80 msgr2=0x7ff2200731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:38.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.016+0000 7ff225600700 1 --2- 172.21.15.38:0/4125838982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff220074d80 0x7ff2200731e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff210009910 tx=0x7ff210009c20 comp rx=0 tx=0).stop 2024-03-24T00:49:38.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.017+0000 7ff225600700 1 -- 172.21.15.38:0/4125838982 shutdown_connections 2024-03-24T00:49:38.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.017+0000 7ff225600700 1 --2- 172.21.15.38:0/4125838982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2200737b0 0x7ff220073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:38.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.017+0000 7ff225600700 1 --2- 172.21.15.38:0/4125838982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff220074d80 0x7ff2200731e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:38.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.017+0000 7ff225600700 1 -- 172.21.15.38:0/4125838982 >> 172.21.15.38:0/4125838982 conn(0x7ff2200fc090 msgr2=0x7ff2200fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:38.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.017+0000 7ff225600700 1 -- 172.21.15.38:0/4125838982 shutdown_connections 2024-03-24T00:49:38.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.017+0000 7ff225600700 1 -- 172.21.15.38:0/4125838982 wait complete. 2024-03-24T00:49:38.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.018+0000 7ff225600700 1 Processor -- start 2024-03-24T00:49:38.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.018+0000 7ff225600700 1 -- start start 2024-03-24T00:49:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.019+0000 7ff225600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2200737b0 0x7ff220198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.019+0000 7ff225600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff220074d80 0x7ff220198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.019+0000 7ff225600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff220199270 con 0x7ff220074d80 2024-03-24T00:49:38.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.019+0000 7ff225600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2201993b0 con 0x7ff2200737b0 2024-03-24T00:49:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.019+0000 7ff21e000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff220074d80 0x7ff220198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.019+0000 7ff21e000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff220074d80 0x7ff220198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48538/0 (socket says 172.21.15.38:48538) 2024-03-24T00:49:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.019+0000 7ff21ea00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2200737b0 0x7ff220198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.019+0000 7ff21ea00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2200737b0 0x7ff220198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:34714/0 (socket says 172.21.15.38:34714) 2024-03-24T00:49:38.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.020+0000 7ff21e000700 1 -- 172.21.15.38:0/2114966598 learned_addr learned my addr 172.21.15.38:0/2114966598 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:49:38.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.021+0000 7ff21e000700 1 -- 172.21.15.38:0/2114966598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2200737b0 msgr2=0x7ff220198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:38.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.021+0000 7ff21e000700 1 --2- 172.21.15.38:0/2114966598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2200737b0 0x7ff220198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:38.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.021+0000 7ff21e000700 1 -- 172.21.15.38:0/2114966598 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2100095f0 con 0x7ff220074d80 2024-03-24T00:49:38.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.021+0000 7ff21e000700 1 --2- 172.21.15.38:0/2114966598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff220074d80 0x7ff220198c50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00d8d0 tx=0x7ff20c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:38.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.021+0000 7ff217400700 1 -- 172.21.15.38:0/2114966598 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff20c0098a0 con 0x7ff220074d80 2024-03-24T00:49:38.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.021+0000 7ff217400700 1 -- 172.21.15.38:0/2114966598 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff20c010460 con 0x7ff220074d80 2024-03-24T00:49:38.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.021+0000 7ff225600700 1 -- 172.21.15.38:0/2114966598 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22019ddc0 con 0x7ff220074d80 2024-03-24T00:49:38.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.021+0000 7ff217400700 1 -- 172.21.15.38:0/2114966598 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff20c00f5f0 con 0x7ff220074d80 2024-03-24T00:49:38.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.022+0000 7ff225600700 1 -- 172.21.15.38:0/2114966598 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22006a850 con 0x7ff220074d80 2024-03-24T00:49:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.024+0000 7ff225600700 1 -- 172.21.15.38:0/2114966598 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff220192840 con 0x7ff220074d80 2024-03-24T00:49:38.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.025+0000 7ff217400700 1 -- 172.21.15.38:0/2114966598 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 91796+0+0 (secure 0 0 0) 0x7ff20c009a00 con 0x7ff220074d80 2024-03-24T00:49:38.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.026+0000 7ff217400700 1 --2- 172.21.15.38:0/2114966598 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7ff20407edb0 0x7ff204081270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:38.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.026+0000 7ff217400700 1 -- 172.21.15.38:0/2114966598 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff20c08d940 con 0x7ff220074d80 2024-03-24T00:49:38.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.027+0000 7ff21ea00700 1 --2- 172.21.15.38:0/2114966598 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7ff20407edb0 0x7ff204081270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:38.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.028+0000 7ff21ea00700 1 --2- 172.21.15.38:0/2114966598 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7ff20407edb0 0x7ff204081270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff210005090 tx=0x7ff210005fd0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.035+0000 7ff217400700 1 -- 172.21.15.38:0/2114966598 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7ff20c05ab80 con 0x7ff220074d80 2024-03-24T00:49:38.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.441+0000 7ff225600700 1 -- 172.21.15.38:0/2114966598 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2200611f0 con 0x7ff20407edb0 2024-03-24T00:49:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.457+0000 7ff217400700 1 -- 172.21.15.38:0/2114966598 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff2200611f0 con 0x7ff20407edb0 2024-03-24T00:49:38.458 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:49:38.458 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (10m) 11s ago 11m 14.1M - 0.25.0 c8568f914cd2 9f1243fbcb3e 2024-03-24T00:49:38.458 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (12m) 11s ago 12m 8036k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:49:38.458 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (10m) 14s ago 10m 7830k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:49:38.458 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (12m) 11s ago 12m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:49:38.458 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (10m) 14s ago 10m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (9m) 11s ago 11m 72.9M - 9.4.7 954c08fa6188 e6aa268d69e8 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (4m) 11s ago 4m 151M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (4m) 11s ago 4m 228M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (4m) 14s ago 4m 24.2M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (4m) 14s ago 4m 22.4M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (51s) 11s ago 13m 474M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (117s) 14s ago 10m 74.6M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b21fee18bc67 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (13m) 11s ago 14m 53.8M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (10m) 14s ago 10m 41.7M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (12m) 11s ago 12m 8116k - 1.5.0 0da6a335fe13 c34cc077c93f 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (10m) 14s ago 10m 7968k - 1.5.0 0da6a335fe13 189e840ee2f1 2024-03-24T00:49:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (9m) 11s ago 9m 203M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:49:38.460 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (8m) 11s ago 8m 218M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:49:38.460 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (8m) 11s ago 8m 177M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:49:38.460 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (7m) 14s ago 7m 243M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:49:38.460 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (7m) 14s ago 7m 208M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:49:38.460 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (6m) 14s ago 6m 213M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:49:38.460 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (69s) 11s ago 11m 48.7M - 2.43.0 a07b618ecd1d 4cf217b32ca6 2024-03-24T00:49:38.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.460+0000 7ff225600700 1 -- 172.21.15.38:0/2114966598 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7ff20407edb0 msgr2=0x7ff204081270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:38.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.460+0000 7ff225600700 1 --2- 172.21.15.38:0/2114966598 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7ff20407edb0 0x7ff204081270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff210005090 tx=0x7ff210005fd0 comp rx=0 tx=0).stop 2024-03-24T00:49:38.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.460+0000 7ff225600700 1 -- 172.21.15.38:0/2114966598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff220074d80 msgr2=0x7ff220198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:38.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.460+0000 7ff225600700 1 --2- 172.21.15.38:0/2114966598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff220074d80 0x7ff220198c50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00d8d0 tx=0x7ff20c00dbe0 comp rx=0 tx=0).stop 2024-03-24T00:49:38.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.461+0000 7ff225600700 1 -- 172.21.15.38:0/2114966598 shutdown_connections 2024-03-24T00:49:38.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.461+0000 7ff225600700 1 --2- 172.21.15.38:0/2114966598 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7ff20407edb0 0x7ff204081270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:38.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.461+0000 7ff225600700 1 --2- 172.21.15.38:0/2114966598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2200737b0 0x7ff220198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:38.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.461+0000 7ff225600700 1 --2- 172.21.15.38:0/2114966598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff220074d80 0x7ff220198c50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:38.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.461+0000 7ff225600700 1 -- 172.21.15.38:0/2114966598 >> 172.21.15.38:0/2114966598 conn(0x7ff2200fc090 msgr2=0x7ff220106dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:38.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.461+0000 7ff225600700 1 -- 172.21.15.38:0/2114966598 shutdown_connections 2024-03-24T00:49:38.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.461+0000 7ff225600700 1 -- 172.21.15.38:0/2114966598 wait complete. 2024-03-24T00:49:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.660+0000 7f31f4800700 1 -- 172.21.15.38:0/3520114067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31ec101560 msgr2=0x7f31ec103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.660+0000 7f31f4800700 1 --2- 172.21.15.38:0/3520114067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31ec101560 0x7f31ec103950 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009910 tx=0x7f31e0009c20 comp rx=0 tx=0).stop 2024-03-24T00:49:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.660+0000 7f31f4800700 1 -- 172.21.15.38:0/3520114067 shutdown_connections 2024-03-24T00:49:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.660+0000 7f31f4800700 1 --2- 172.21.15.38:0/3520114067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec103e90 0x7f31ec106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.660+0000 7f31f4800700 1 --2- 172.21.15.38:0/3520114067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31ec101560 0x7f31ec103950 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.660+0000 7f31f4800700 1 -- 172.21.15.38:0/3520114067 >> 172.21.15.38:0/3520114067 conn(0x7f31ec0faf70 msgr2=0x7f31ec0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.660+0000 7f31f4800700 1 -- 172.21.15.38:0/3520114067 shutdown_connections 2024-03-24T00:49:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.661+0000 7f31f4800700 1 -- 172.21.15.38:0/3520114067 wait complete. 2024-03-24T00:49:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.662+0000 7f31f4800700 1 Processor -- start 2024-03-24T00:49:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.662+0000 7f31f4800700 1 -- start start 2024-03-24T00:49:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.663+0000 7f31f4800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec101560 0x7f31ec196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.663+0000 7f31f4800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31ec103e90 0x7f31ec1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.663+0000 7f31f4800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec196ff0 con 0x7f31ec103e90 2024-03-24T00:49:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.663+0000 7f31f4800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec197130 con 0x7f31ec101560 2024-03-24T00:49:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.663+0000 7f31f1e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec101560 0x7f31ec196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.663+0000 7f31f1e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec101560 0x7f31ec196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:34726/0 (socket says 172.21.15.38:34726) 2024-03-24T00:49:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.663+0000 7f31f1e00700 1 -- 172.21.15.38:0/1008386444 learned_addr learned my addr 172.21.15.38:0/1008386444 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:49:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.663+0000 7f31f1400700 1 --2- 172.21.15.38:0/1008386444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31ec103e90 0x7f31ec1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.664+0000 7f31f1400700 1 -- 172.21.15.38:0/1008386444 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec101560 msgr2=0x7f31ec196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.664+0000 7f31f1400700 1 --2- 172.21.15.38:0/1008386444 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec101560 0x7f31ec196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.664+0000 7f31f1400700 1 -- 172.21.15.38:0/1008386444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e00095f0 con 0x7f31ec103e90 2024-03-24T00:49:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.664+0000 7f31f1400700 1 --2- 172.21.15.38:0/1008386444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31ec103e90 0x7f31ec1969d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00e970 tx=0x7f31dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.664+0000 7f31e6a00700 1 -- 172.21.15.38:0/1008386444 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31dc00c9d0 con 0x7f31ec103e90 2024-03-24T00:49:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.665+0000 7f31e6a00700 1 -- 172.21.15.38:0/1008386444 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31dc010430 con 0x7f31ec103e90 2024-03-24T00:49:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.665+0000 7f31f4800700 1 -- 172.21.15.38:0/1008386444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31ec19bb90 con 0x7f31ec103e90 2024-03-24T00:49:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.665+0000 7f31f4800700 1 -- 172.21.15.38:0/1008386444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ec19c160 con 0x7f31ec103e90 2024-03-24T00:49:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.665+0000 7f31e6a00700 1 -- 172.21.15.38:0/1008386444 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31dc018630 con 0x7f31ec103e90 2024-03-24T00:49:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.666+0000 7f31f4800700 1 -- 172.21.15.38:0/1008386444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d0005320 con 0x7f31ec103e90 2024-03-24T00:49:38.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.672+0000 7f31e6a00700 1 -- 172.21.15.38:0/1008386444 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 91796+0+0 (secure 0 0 0) 0x7f31dc00cb30 con 0x7f31ec103e90 2024-03-24T00:49:38.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.673+0000 7f31e6a00700 1 --2- 172.21.15.38:0/1008386444 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f31d406dd50 0x7f31d4070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:38.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.673+0000 7f31e6a00700 1 -- 172.21.15.38:0/1008386444 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f31dc014070 con 0x7f31ec103e90 2024-03-24T00:49:38.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.673+0000 7f31f1e00700 1 --2- 172.21.15.38:0/1008386444 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f31d406dd50 0x7f31d4070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:38.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.674+0000 7f31f1e00700 1 --2- 172.21.15.38:0/1008386444 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f31d406dd50 0x7f31d4070210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009fd0 tx=0x7f31e0005fb0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:38.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:38.675+0000 7f31e6a00700 1 -- 172.21.15.38:0/1008386444 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f31dc05b620 con 0x7f31ec103e90 2024-03-24T00:49:38.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:38 smithi038 ceph-mon[29271]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:38.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:38 smithi038 ceph-mon[29271]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:49:38.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:38 smithi038 ceph-mon[29271]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:49:38.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:38 smithi088 ceph-mon[36610]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:38.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:38 smithi088 ceph-mon[36610]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:49:38.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:38 smithi088 ceph-mon[36610]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:49:39.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.144+0000 7f31f4800700 1 -- 172.21.15.38:0/1008386444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f31d0005cc0 con 0x7f31ec103e90 2024-03-24T00:49:39.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.145+0000 7f31e6a00700 1 -- 172.21.15.38:0/1008386444 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f31dc05b1b0 con 0x7f31ec103e90 2024-03-24T00:49:39.145 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:49:39.145 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:49:39.145 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:49:39.145 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:49:39.145 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:49:39.145 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:49:39.145 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:49:39.146 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:49:39.146 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:49:39.146 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:49:39.146 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:49:39.146 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:49:39.146 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:49:39.146 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:49:39.146 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 12, 2024-03-24T00:49:39.146 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:49:39.146 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:49:39.146 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:49:39.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.149+0000 7f31f4800700 1 -- 172.21.15.38:0/1008386444 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f31d406dd50 msgr2=0x7f31d4070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:39.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.149+0000 7f31f4800700 1 --2- 172.21.15.38:0/1008386444 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f31d406dd50 0x7f31d4070210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009fd0 tx=0x7f31e0005fb0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.149+0000 7f31f4800700 1 -- 172.21.15.38:0/1008386444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31ec103e90 msgr2=0x7f31ec1969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:39.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.149+0000 7f31f4800700 1 --2- 172.21.15.38:0/1008386444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31ec103e90 0x7f31ec1969d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00e970 tx=0x7f31dc00ec80 comp rx=0 tx=0).stop 2024-03-24T00:49:39.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.149+0000 7f31f4800700 1 -- 172.21.15.38:0/1008386444 shutdown_connections 2024-03-24T00:49:39.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.149+0000 7f31f4800700 1 --2- 172.21.15.38:0/1008386444 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f31d406dd50 0x7f31d4070210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.149+0000 7f31f4800700 1 --2- 172.21.15.38:0/1008386444 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ec101560 0x7f31ec196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.149+0000 7f31f4800700 1 --2- 172.21.15.38:0/1008386444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31ec103e90 0x7f31ec1969d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.149+0000 7f31f4800700 1 -- 172.21.15.38:0/1008386444 >> 172.21.15.38:0/1008386444 conn(0x7f31ec0faf70 msgr2=0x7f31ec0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:39.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.149+0000 7f31f4800700 1 -- 172.21.15.38:0/1008386444 shutdown_connections 2024-03-24T00:49:39.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.149+0000 7f31f4800700 1 -- 172.21.15.38:0/1008386444 wait complete. 2024-03-24T00:49:39.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.325+0000 7fcbeea00700 1 -- 172.21.15.38:0/3464783205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe8108300 msgr2=0x7fcbe8108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.325+0000 7fcbeea00700 1 --2- 172.21.15.38:0/3464783205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe8108300 0x7fcbe8108780 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009910 tx=0x7fcbd0009c20 comp rx=0 tx=0).stop 2024-03-24T00:49:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.325+0000 7fcbeea00700 1 -- 172.21.15.38:0/3464783205 shutdown_connections 2024-03-24T00:49:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.325+0000 7fcbeea00700 1 --2- 172.21.15.38:0/3464783205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe8108300 0x7fcbe8108780 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.325+0000 7fcbeea00700 1 --2- 172.21.15.38:0/3464783205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbe8107130 0x7fcbe8107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.325+0000 7fcbeea00700 1 -- 172.21.15.38:0/3464783205 >> 172.21.15.38:0/3464783205 conn(0x7fcbe80762d0 msgr2=0x7fcbe8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.325+0000 7fcbeea00700 1 -- 172.21.15.38:0/3464783205 shutdown_connections 2024-03-24T00:49:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.325+0000 7fcbeea00700 1 -- 172.21.15.38:0/3464783205 wait complete. 2024-03-24T00:49:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.326+0000 7fcbeea00700 1 Processor -- start 2024-03-24T00:49:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.327+0000 7fcbeea00700 1 -- start start 2024-03-24T00:49:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.327+0000 7fcbeea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe8107130 0x7fcbe819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.327+0000 7fcbeea00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbe8108300 0x7fcbe819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.327+0000 7fcbeea00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe819d5d0 con 0x7fcbe8107130 2024-03-24T00:49:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.327+0000 7fcbeea00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe819d710 con 0x7fcbe8108300 2024-03-24T00:49:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.327+0000 7fcbe7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe8107130 0x7fcbe819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.327+0000 7fcbe7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe8107130 0x7fcbe819ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48574/0 (socket says 172.21.15.38:48574) 2024-03-24T00:49:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.327+0000 7fcbe7e00700 1 -- 172.21.15.38:0/3994920260 learned_addr learned my addr 172.21.15.38:0/3994920260 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:49:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.327+0000 7fcbdf400700 1 --2- 172.21.15.38:0/3994920260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbe8108300 0x7fcbe819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.328+0000 7fcbe7e00700 1 -- 172.21.15.38:0/3994920260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbe8108300 msgr2=0x7fcbe819cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.328+0000 7fcbe7e00700 1 --2- 172.21.15.38:0/3994920260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbe8108300 0x7fcbe819cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.328+0000 7fcbe7e00700 1 -- 172.21.15.38:0/3994920260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd00095f0 con 0x7fcbe8107130 2024-03-24T00:49:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.328+0000 7fcbe7e00700 1 --2- 172.21.15.38:0/3994920260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe8107130 0x7fcbe819ca70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800ef10 tx=0x7fcbd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.328+0000 7fcbe5600700 1 -- 172.21.15.38:0/3994920260 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbd800c9f0 con 0x7fcbe8107130 2024-03-24T00:49:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.328+0000 7fcbe5600700 1 -- 172.21.15.38:0/3994920260 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbd800f040 con 0x7fcbe8107130 2024-03-24T00:49:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.328+0000 7fcbe5600700 1 -- 172.21.15.38:0/3994920260 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbd80146b0 con 0x7fcbe8107130 2024-03-24T00:49:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.328+0000 7fcbeea00700 1 -- 172.21.15.38:0/3994920260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe81a2110 con 0x7fcbe8107130 2024-03-24T00:49:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.329+0000 7fcbeea00700 1 -- 172.21.15.38:0/3994920260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe81a26e0 con 0x7fcbe8107130 2024-03-24T00:49:39.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.330+0000 7fcbeea00700 1 -- 172.21.15.38:0/3994920260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe8196ca0 con 0x7fcbe8107130 2024-03-24T00:49:39.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.330+0000 7fcbe5600700 1 -- 172.21.15.38:0/3994920260 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 91796+0+0 (secure 0 0 0) 0x7fcbd800cb50 con 0x7fcbe8107130 2024-03-24T00:49:39.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.331+0000 7fcbe5600700 1 --2- 172.21.15.38:0/3994920260 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcbcc06dd50 0x7fcbcc070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:49:39.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.331+0000 7fcbe5600700 1 -- 172.21.15.38:0/3994920260 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcbd808dd20 con 0x7fcbe8107130 2024-03-24T00:49:39.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.332+0000 7fcbdf400700 1 --2- 172.21.15.38:0/3994920260 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcbcc06dd50 0x7fcbcc070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:49:39.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.332+0000 7fcbdf400700 1 --2- 172.21.15.38:0/3994920260 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcbcc06dd50 0x7fcbcc070210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000b6f0 tx=0x7fcbd0005fb0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:49:39.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.339+0000 7fcbe5600700 1 -- 172.21.15.38:0/3994920260 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fcbd805b000 con 0x7fcbe8107130 2024-03-24T00:49:39.706 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-24T00:49:39.706 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-24T00:49:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.718+0000 7fcbeea00700 1 -- 172.21.15.38:0/3994920260 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbe80611f0 con 0x7fcbcc06dd50 2024-03-24T00:49:39.722 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:49:39.722 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:49:39.723 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:49:39.723 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-24T00:49:39.723 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:49:39.723 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-03-24T00:49:39.723 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:49:39.723 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "2/2 daemons upgraded", 2024-03-24T00:49:39.723 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "", 2024-03-24T00:49:39.723 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:49:39.723 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:49:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.720+0000 7fcbe5600700 1 -- 172.21.15.38:0/3994920260 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fcbe80611f0 con 0x7fcbcc06dd50 2024-03-24T00:49:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.723+0000 7fcbeea00700 1 -- 172.21.15.38:0/3994920260 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcbcc06dd50 msgr2=0x7fcbcc070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.723+0000 7fcbeea00700 1 --2- 172.21.15.38:0/3994920260 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcbcc06dd50 0x7fcbcc070210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000b6f0 tx=0x7fcbd0005fb0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.723+0000 7fcbeea00700 1 -- 172.21.15.38:0/3994920260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe8107130 msgr2=0x7fcbe819ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:49:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.723+0000 7fcbeea00700 1 --2- 172.21.15.38:0/3994920260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe8107130 0x7fcbe819ca70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800ef10 tx=0x7fcbd800c3a0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.723+0000 7fcbeea00700 1 -- 172.21.15.38:0/3994920260 shutdown_connections 2024-03-24T00:49:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.723+0000 7fcbeea00700 1 --2- 172.21.15.38:0/3994920260 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcbcc06dd50 0x7fcbcc070210 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.723+0000 7fcbeea00700 1 --2- 172.21.15.38:0/3994920260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe8107130 0x7fcbe819ca70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.723+0000 7fcbeea00700 1 --2- 172.21.15.38:0/3994920260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbe8108300 0x7fcbe819cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:49:39.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.723+0000 7fcbeea00700 1 -- 172.21.15.38:0/3994920260 >> 172.21.15.38:0/3994920260 conn(0x7fcbe80762d0 msgr2=0x7fcbe810b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:49:39.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.723+0000 7fcbeea00700 1 -- 172.21.15.38:0/3994920260 shutdown_connections 2024-03-24T00:49:39.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:49:39.723+0000 7fcbeea00700 1 -- 172.21.15.38:0/3994920260 wait complete. 2024-03-24T00:49:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:39 smithi038 ceph-mon[29271]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:39 smithi038 ceph-mon[29271]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:49:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:39 smithi038 ceph-mon[29271]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:49:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:39 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1008386444' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:49:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:39 smithi038 ceph-mon[29271]: pgmap v13: 65 pgs: 65 active+clean; 654 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1010 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-03-24T00:49:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:40.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:40.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:39 smithi088 ceph-mon[36610]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:40.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:39 smithi088 ceph-mon[36610]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:49:40.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:39 smithi088 ceph-mon[36610]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:49:40.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:39 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1008386444' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:49:40.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:39 smithi088 ceph-mon[36610]: pgmap v13: 65 pgs: 65 active+clean; 654 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1010 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-03-24T00:49:40.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:40.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:40.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:40.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:40.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:41.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:40 smithi088 ceph-mon[36610]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:41.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:40 smithi088 ceph-mon[36610]: Reconfiguring prometheus.smithi038 (dependencies changed)... 2024-03-24T00:49:41.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:40 smithi088 ceph-mon[36610]: Reconfiguring daemon prometheus.smithi038 on smithi038 2024-03-24T00:49:41.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:40 smithi038 ceph-mon[29271]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:49:41.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:40 smithi038 ceph-mon[29271]: Reconfiguring prometheus.smithi038 (dependencies changed)... 2024-03-24T00:49:41.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:40 smithi038 ceph-mon[29271]: Reconfiguring daemon prometheus.smithi038 on smithi038 2024-03-24T00:49:42.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:41 smithi038 ceph-mon[29271]: pgmap v14: 65 pgs: 65 active+clean; 437 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-03-24T00:49:42.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:41 smithi088 ceph-mon[36610]: pgmap v14: 65 pgs: 65 active+clean; 437 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-03-24T00:49:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:44 smithi038 ceph-mon[29271]: pgmap v15: 65 pgs: 65 active+clean; 437 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-03-24T00:49:44.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:44 smithi088 ceph-mon[36610]: pgmap v15: 65 pgs: 65 active+clean; 437 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-03-24T00:49:46.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:46 smithi038 ceph-mon[29271]: pgmap v16: 65 pgs: 65 active+clean; 384 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-03-24T00:49:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:46 smithi088 ceph-mon[36610]: pgmap v16: 65 pgs: 65 active+clean; 384 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-03-24T00:49:48.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:48 smithi038 ceph-mon[29271]: pgmap v17: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-03-24T00:49:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:48 smithi088 ceph-mon[36610]: pgmap v17: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-03-24T00:49:50.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:49 smithi038 ceph-mon[29271]: pgmap v18: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-03-24T00:49:50.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:49 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:49:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:49 smithi088 ceph-mon[36610]: pgmap v18: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-03-24T00:49:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:49 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:49:52.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:52 smithi088 ceph-mon[36610]: pgmap v19: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-03-24T00:49:52.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:52 smithi038 ceph-mon[29271]: pgmap v19: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-03-24T00:49:54.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:54 smithi038 ceph-mon[29271]: pgmap v20: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-03-24T00:49:54.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:54 smithi088 ceph-mon[36610]: pgmap v20: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-03-24T00:49:56.511 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:56 smithi088 ceph-mon[36610]: pgmap v21: 65 pgs: 65 active+clean; 327 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 228 op/s 2024-03-24T00:49:56.511 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:56 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:56.511 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:56 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:56.511 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:56 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:49:56.511 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:56 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:49:56.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:56 smithi038 ceph-mon[29271]: pgmap v21: 65 pgs: 65 active+clean; 327 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 228 op/s 2024-03-24T00:49:56.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:56 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:56.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:56 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:49:56.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:56 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:49:56.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:56 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:49:57.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:57 smithi088 ceph-mon[36610]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:49:57.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:57 smithi038 ceph-mon[29271]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:49:58.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:58 smithi038 ceph-mon[29271]: pgmap v22: 65 pgs: 65 active+clean; 328 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 242 op/s 2024-03-24T00:49:58.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:58 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:49:58.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:58 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:49:58.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:58 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:49:58.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:58 smithi088 ceph-mon[36610]: pgmap v22: 65 pgs: 65 active+clean; 328 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 242 op/s 2024-03-24T00:49:58.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:58 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:49:58.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:58 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:49:58.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:58 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:49:59.604 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:59 smithi088 ceph-mon[36610]: Upgrade: Updating mgr.smithi088.njctea 2024-03-24T00:49:59.604 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:49:59 smithi088 ceph-mon[36610]: Deploying daemon mgr.smithi088.njctea on smithi088 2024-03-24T00:49:59.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:59 smithi038 ceph-mon[29271]: Upgrade: Updating mgr.smithi088.njctea 2024-03-24T00:49:59.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:49:59 smithi038 ceph-mon[29271]: Deploying daemon mgr.smithi088.njctea on smithi088 2024-03-24T00:50:00.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:00 smithi038 ceph-mon[29271]: pgmap v23: 65 pgs: 65 active+clean; 328 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-03-24T00:50:00.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:00 smithi038 ceph-mon[29271]: overall HEALTH_OK 2024-03-24T00:50:00.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:00 smithi088 ceph-mon[36610]: pgmap v23: 65 pgs: 65 active+clean; 328 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-03-24T00:50:00.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:00 smithi088 ceph-mon[36610]: overall HEALTH_OK 2024-03-24T00:50:02.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:02 smithi038 ceph-mon[29271]: pgmap v24: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 300 op/s 2024-03-24T00:50:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:02 smithi088 ceph-mon[36610]: pgmap v24: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 300 op/s 2024-03-24T00:50:04.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:04 smithi038 ceph-mon[29271]: pgmap v25: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 255 op/s 2024-03-24T00:50:04.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:04 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:04.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:04 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:04.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:04 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:50:04.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:04 smithi088 ceph-mon[36610]: pgmap v25: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 255 op/s 2024-03-24T00:50:04.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:04 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:04.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:04 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:04.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:04 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:50:05.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:05 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:50:05.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:05 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:50:06.537 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:06 smithi088 ceph-mon[36610]: pgmap v26: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 385 op/s 2024-03-24T00:50:06.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:06 smithi038 ceph-mon[29271]: pgmap v26: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 385 op/s 2024-03-24T00:50:08.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:08 smithi038 ceph-mon[29271]: pgmap v27: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 312 op/s 2024-03-24T00:50:08.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:08 smithi088 ceph-mon[36610]: pgmap v27: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 312 op/s 2024-03-24T00:50:09.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.956+0000 7f06b0400700 1 -- 172.21.15.38:0/3026257234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a8075760 msgr2=0x7f06a8075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:09.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.956+0000 7f06b0400700 1 --2- 172.21.15.38:0/3026257234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a8075760 0x7f06a8075b80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f069c009960 tx=0x7f069c009c70 comp rx=0 tx=0).stop 2024-03-24T00:50:09.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.957+0000 7f06b0400700 1 -- 172.21.15.38:0/3026257234 shutdown_connections 2024-03-24T00:50:09.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.957+0000 7f06b0400700 1 --2- 172.21.15.38:0/3026257234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06a80769b0 0x7f06a8076e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:09.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.957+0000 7f06b0400700 1 --2- 172.21.15.38:0/3026257234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a8075760 0x7f06a8075b80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:09.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.957+0000 7f06b0400700 1 -- 172.21.15.38:0/3026257234 >> 172.21.15.38:0/3026257234 conn(0x7f06a80fe2a0 msgr2=0x7f06a81006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:09.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.957+0000 7f06b0400700 1 -- 172.21.15.38:0/3026257234 shutdown_connections 2024-03-24T00:50:09.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.957+0000 7f06b0400700 1 -- 172.21.15.38:0/3026257234 wait complete. 2024-03-24T00:50:09.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.959+0000 7f06b0400700 1 Processor -- start 2024-03-24T00:50:09.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.959+0000 7f06b0400700 1 -- start start 2024-03-24T00:50:09.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.959+0000 7f06b0400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06a8075760 0x7f06a819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:09.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.959+0000 7f06b0400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a80769b0 0x7f06a819d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:09.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.959+0000 7f06b0400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a819d6d0 con 0x7f06a80769b0 2024-03-24T00:50:09.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.959+0000 7f06b0400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a819d810 con 0x7f06a8075760 2024-03-24T00:50:09.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.960+0000 7f06ad000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a80769b0 0x7f06a819d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:09.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.960+0000 7f06ad000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a80769b0 0x7f06a819d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42270/0 (socket says 172.21.15.38:42270) 2024-03-24T00:50:09.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.960+0000 7f06ad000700 1 -- 172.21.15.38:0/1783290657 learned_addr learned my addr 172.21.15.38:0/1783290657 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:50:09.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.960+0000 7f06ada00700 1 --2- 172.21.15.38:0/1783290657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06a8075760 0x7f06a819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:09.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.961+0000 7f06ad000700 1 -- 172.21.15.38:0/1783290657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06a8075760 msgr2=0x7f06a819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:09.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.961+0000 7f06ad000700 1 --2- 172.21.15.38:0/1783290657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06a8075760 0x7f06a819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:09.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.961+0000 7f06ad000700 1 -- 172.21.15.38:0/1783290657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0095f0 con 0x7f06a80769b0 2024-03-24T00:50:09.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.961+0000 7f06ad000700 1 --2- 172.21.15.38:0/1783290657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a80769b0 0x7f06a819d0b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f069400c8a0 tx=0x7f069400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:09.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.962+0000 7f06a6000700 1 -- 172.21.15.38:0/1783290657 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06940078e0 con 0x7f06a80769b0 2024-03-24T00:50:09.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.962+0000 7f06a6000700 1 -- 172.21.15.38:0/1783290657 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f069400f450 con 0x7f06a80769b0 2024-03-24T00:50:09.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.962+0000 7f06a6000700 1 -- 172.21.15.38:0/1783290657 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f069400e5e0 con 0x7f06a80769b0 2024-03-24T00:50:09.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.963+0000 7f06b0400700 1 -- 172.21.15.38:0/1783290657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a81a2220 con 0x7f06a80769b0 2024-03-24T00:50:09.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.963+0000 7f06b0400700 1 -- 172.21.15.38:0/1783290657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a81a2770 con 0x7f06a80769b0 2024-03-24T00:50:09.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.964+0000 7f06b0400700 1 -- 172.21.15.38:0/1783290657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a8196c70 con 0x7f06a80769b0 2024-03-24T00:50:09.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.966+0000 7f06a6000700 1 -- 172.21.15.38:0/1783290657 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 91796+0+0 (secure 0 0 0) 0x7f069400f5c0 con 0x7f06a80769b0 2024-03-24T00:50:09.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.967+0000 7f06a6000700 1 --2- 172.21.15.38:0/1783290657 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f069006dd50 0x7f0690070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:09.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.968+0000 7f06a6000700 1 -- 172.21.15.38:0/1783290657 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f069408d470 con 0x7f06a80769b0 2024-03-24T00:50:09.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.968+0000 7f06ada00700 1 --2- 172.21.15.38:0/1783290657 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f069006dd50 0x7f0690070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:09.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.970+0000 7f06ada00700 1 --2- 172.21.15.38:0/1783290657 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f069006dd50 0x7f0690070210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f069c006010 tx=0x7f069c0058e0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:09.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:09.974+0000 7f06a6000700 1 -- 172.21.15.38:0/1783290657 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f069405a6b0 con 0x7f06a80769b0 2024-03-24T00:50:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:09 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:09 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:09 smithi038 ceph-mon[29271]: pgmap v28: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1019 KiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-03-24T00:50:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:09 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:09 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:09 smithi088 ceph-mon[36610]: pgmap v28: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1019 KiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-03-24T00:50:10.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.437+0000 7f06b0400700 1 -- 172.21.15.38:0/1783290657 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06a80611f0 con 0x7f069006dd50 2024-03-24T00:50:10.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.439+0000 7f06a6000700 1 -- 172.21.15.38:0/1783290657 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f06a80611f0 con 0x7f069006dd50 2024-03-24T00:50:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.443+0000 7f06b0400700 1 -- 172.21.15.38:0/1783290657 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f069006dd50 msgr2=0x7f0690070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.443+0000 7f06b0400700 1 --2- 172.21.15.38:0/1783290657 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f069006dd50 0x7f0690070210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f069c006010 tx=0x7f069c0058e0 comp rx=0 tx=0).stop 2024-03-24T00:50:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.443+0000 7f06b0400700 1 -- 172.21.15.38:0/1783290657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a80769b0 msgr2=0x7f06a819d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.443+0000 7f06b0400700 1 --2- 172.21.15.38:0/1783290657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a80769b0 0x7f06a819d0b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f069400c8a0 tx=0x7f069400cbb0 comp rx=0 tx=0).stop 2024-03-24T00:50:10.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.443+0000 7f06b0400700 1 -- 172.21.15.38:0/1783290657 shutdown_connections 2024-03-24T00:50:10.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.443+0000 7f06b0400700 1 --2- 172.21.15.38:0/1783290657 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f069006dd50 0x7f0690070210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:10.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.443+0000 7f06b0400700 1 --2- 172.21.15.38:0/1783290657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06a8075760 0x7f06a819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:10.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.443+0000 7f06b0400700 1 --2- 172.21.15.38:0/1783290657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06a80769b0 0x7f06a819d0b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:10.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.443+0000 7f06b0400700 1 -- 172.21.15.38:0/1783290657 >> 172.21.15.38:0/1783290657 conn(0x7f06a80fe2a0 msgr2=0x7f06a810d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:10.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.444+0000 7f06b0400700 1 -- 172.21.15.38:0/1783290657 shutdown_connections 2024-03-24T00:50:10.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.444+0000 7f06b0400700 1 -- 172.21.15.38:0/1783290657 wait complete. 2024-03-24T00:50:10.467 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:50:10.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.607+0000 7f9be2e00700 1 -- 172.21.15.38:0/2847704114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc101cd0 msgr2=0x7f9bdc102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:10.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.607+0000 7f9be2e00700 1 --2- 172.21.15.38:0/2847704114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc102130 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009910 tx=0x7f9bc8009c20 comp rx=0 tx=0).stop 2024-03-24T00:50:10.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.607+0000 7f9be2e00700 1 -- 172.21.15.38:0/2847704114 shutdown_connections 2024-03-24T00:50:10.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.607+0000 7f9be2e00700 1 --2- 172.21.15.38:0/2847704114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc102130 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:10.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.607+0000 7f9be2e00700 1 --2- 172.21.15.38:0/2847704114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bdc100ad0 0x7f9bdc100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:10.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.607+0000 7f9be2e00700 1 -- 172.21.15.38:0/2847704114 >> 172.21.15.38:0/2847704114 conn(0x7f9bdc0fc090 msgr2=0x7f9bdc0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:10.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.607+0000 7f9be2e00700 1 -- 172.21.15.38:0/2847704114 shutdown_connections 2024-03-24T00:50:10.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.607+0000 7f9be2e00700 1 -- 172.21.15.38:0/2847704114 wait complete. 2024-03-24T00:50:10.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.609+0000 7f9be2e00700 1 Processor -- start 2024-03-24T00:50:10.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.609+0000 7f9be2e00700 1 -- start start 2024-03-24T00:50:10.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.609+0000 7f9be2e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bdc100ad0 0x7f9bdc196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:10.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.609+0000 7f9be2e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:10.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.609+0000 7f9be2e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc197080 con 0x7f9bdc101cd0 2024-03-24T00:50:10.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.609+0000 7f9be2e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc1971c0 con 0x7f9bdc100ad0 2024-03-24T00:50:10.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.610+0000 7f9bdb400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:10.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.610+0000 7f9bdb400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42296/0 (socket says 172.21.15.38:42296) 2024-03-24T00:50:10.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.610+0000 7f9bdb400700 1 -- 172.21.15.38:0/137183381 learned_addr learned my addr 172.21.15.38:0/137183381 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:50:10.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.610+0000 7f9bdbe00700 1 --2- 172.21.15.38:0/137183381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bdc100ad0 0x7f9bdc196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:10.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.610+0000 7f9bdb400700 1 -- 172.21.15.38:0/137183381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bdc100ad0 msgr2=0x7f9bdc196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:10.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.610+0000 7f9bdb400700 1 --2- 172.21.15.38:0/137183381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bdc100ad0 0x7f9bdc196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:10.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.610+0000 7f9bdb400700 1 -- 172.21.15.38:0/137183381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc80095f0 con 0x7f9bdc101cd0 2024-03-24T00:50:10.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.610+0000 7f9bdb400700 1 --2- 172.21.15.38:0/137183381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc196a60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005b40 tx=0x7f9bc800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:10.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.611+0000 7f9bd8c00700 1 -- 172.21.15.38:0/137183381 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc801d070 con 0x7f9bdc101cd0 2024-03-24T00:50:10.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.611+0000 7f9bd8c00700 1 -- 172.21.15.38:0/137183381 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc80050d0 con 0x7f9bdc101cd0 2024-03-24T00:50:10.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.611+0000 7f9be2e00700 1 -- 172.21.15.38:0/137183381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bdc19bbc0 con 0x7f9bdc101cd0 2024-03-24T00:50:10.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.611+0000 7f9bd8c00700 1 -- 172.21.15.38:0/137183381 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc80226a0 con 0x7f9bdc101cd0 2024-03-24T00:50:10.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.611+0000 7f9be2e00700 1 -- 172.21.15.38:0/137183381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bdc19c130 con 0x7f9bdc101cd0 2024-03-24T00:50:10.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.612+0000 7f9be2e00700 1 -- 172.21.15.38:0/137183381 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc005320 con 0x7f9bdc101cd0 2024-03-24T00:50:10.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.613+0000 7f9bd8c00700 1 -- 172.21.15.38:0/137183381 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 91796+0+0 (secure 0 0 0) 0x7f9bc8003680 con 0x7f9bdc101cd0 2024-03-24T00:50:10.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.614+0000 7f9bd8c00700 1 --2- 172.21.15.38:0/137183381 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9bc006dd50 0x7f9bc0070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:10.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.614+0000 7f9bd8c00700 1 -- 172.21.15.38:0/137183381 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9bc808efe0 con 0x7f9bdc101cd0 2024-03-24T00:50:10.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.615+0000 7f9bdbe00700 1 --2- 172.21.15.38:0/137183381 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9bc006dd50 0x7f9bc0070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:10.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.615+0000 7f9bdbe00700 1 --2- 172.21.15.38:0/137183381 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9bc006dd50 0x7f9bc0070210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc101b30 tx=0x7f9bcc009260 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.622+0000 7f9bd8c00700 1 -- 172.21.15.38:0/137183381 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f9bc805c200 con 0x7f9bdc101cd0 2024-03-24T00:50:10.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.979+0000 7f9be2e00700 1 -- 172.21.15.38:0/137183381 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bbc000bf0 con 0x7f9bc006dd50 2024-03-24T00:50:10.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.981+0000 7f9bd8c00700 1 -- 172.21.15.38:0/137183381 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f9bbc000bf0 con 0x7f9bc006dd50 2024-03-24T00:50:10.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.985+0000 7f9be2e00700 1 -- 172.21.15.38:0/137183381 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9bc006dd50 msgr2=0x7f9bc0070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:10.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.985+0000 7f9be2e00700 1 --2- 172.21.15.38:0/137183381 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9bc006dd50 0x7f9bc0070210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc101b30 tx=0x7f9bcc009260 comp rx=0 tx=0).stop 2024-03-24T00:50:10.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.985+0000 7f9be2e00700 1 -- 172.21.15.38:0/137183381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc101cd0 msgr2=0x7f9bdc196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:10.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.985+0000 7f9be2e00700 1 --2- 172.21.15.38:0/137183381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc196a60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005b40 tx=0x7f9bc800f740 comp rx=0 tx=0).stop 2024-03-24T00:50:10.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.986+0000 7f9be2e00700 1 -- 172.21.15.38:0/137183381 shutdown_connections 2024-03-24T00:50:10.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.986+0000 7f9be2e00700 1 --2- 172.21.15.38:0/137183381 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9bc006dd50 0x7f9bc0070210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.986+0000 7f9be2e00700 1 --2- 172.21.15.38:0/137183381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bdc100ad0 0x7f9bdc196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.986+0000 7f9be2e00700 1 --2- 172.21.15.38:0/137183381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc196a60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.986+0000 7f9be2e00700 1 -- 172.21.15.38:0/137183381 >> 172.21.15.38:0/137183381 conn(0x7f9bdc0fc090 msgr2=0x7f9bdc105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.986+0000 7f9be2e00700 1 -- 172.21.15.38:0/137183381 shutdown_connections 2024-03-24T00:50:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:10.987+0000 7f9be2e00700 1 -- 172.21.15.38:0/137183381 wait complete. 2024-03-24T00:50:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.177+0000 7f2daae00700 1 -- 172.21.15.38:0/3881671972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2da4103f00 msgr2=0x7f2da4104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.177+0000 7f2daae00700 1 --2- 172.21.15.38:0/3881671972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2da4103f00 0x7f2da4104360 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009910 tx=0x7f2d90009c20 comp rx=0 tx=0).stop 2024-03-24T00:50:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.177+0000 7f2daae00700 1 -- 172.21.15.38:0/3881671972 shutdown_connections 2024-03-24T00:50:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.177+0000 7f2daae00700 1 --2- 172.21.15.38:0/3881671972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2da4103f00 0x7f2da4104360 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.177+0000 7f2daae00700 1 --2- 172.21.15.38:0/3881671972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da4102d00 0x7f2da4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.177+0000 7f2daae00700 1 -- 172.21.15.38:0/3881671972 >> 172.21.15.38:0/3881671972 conn(0x7f2da40fe2a0 msgr2=0x7f2da41006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.178+0000 7f2daae00700 1 -- 172.21.15.38:0/3881671972 shutdown_connections 2024-03-24T00:50:11.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.178+0000 7f2daae00700 1 -- 172.21.15.38:0/3881671972 wait complete. 2024-03-24T00:50:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.179+0000 7f2daae00700 1 Processor -- start 2024-03-24T00:50:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.179+0000 7f2daae00700 1 -- start start 2024-03-24T00:50:11.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.180+0000 7f2daae00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2da4102d00 0x7f2da41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.180+0000 7f2daae00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da4103f00 0x7f2da4198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.180+0000 7f2daae00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da4199220 con 0x7f2da4102d00 2024-03-24T00:50:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.180+0000 7f2daae00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da4199360 con 0x7f2da4103f00 2024-03-24T00:50:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.180+0000 7f2da3e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2da4102d00 0x7f2da41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.180+0000 7f2da3e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2da4102d00 0x7f2da41986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42308/0 (socket says 172.21.15.38:42308) 2024-03-24T00:50:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.180+0000 7f2da3e00700 1 -- 172.21.15.38:0/3799441333 learned_addr learned my addr 172.21.15.38:0/3799441333 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:50:11.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.180+0000 7f2da3400700 1 --2- 172.21.15.38:0/3799441333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da4103f00 0x7f2da4198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.181+0000 7f2da3e00700 1 -- 172.21.15.38:0/3799441333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da4103f00 msgr2=0x7f2da4198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.181+0000 7f2da3e00700 1 --2- 172.21.15.38:0/3799441333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da4103f00 0x7f2da4198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.181+0000 7f2da3e00700 1 -- 172.21.15.38:0/3799441333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d900095f0 con 0x7f2da4102d00 2024-03-24T00:50:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.181+0000 7f2da3e00700 1 --2- 172.21.15.38:0/3799441333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2da4102d00 0x7f2da41986c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400d900 tx=0x7f2d9400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.182+0000 7f2da0c00700 1 -- 172.21.15.38:0/3799441333 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d940041f0 con 0x7f2da4102d00 2024-03-24T00:50:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.182+0000 7f2da0c00700 1 -- 172.21.15.38:0/3799441333 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d94004350 con 0x7f2da4102d00 2024-03-24T00:50:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.182+0000 7f2daae00700 1 -- 172.21.15.38:0/3799441333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da419ddc0 con 0x7f2da4102d00 2024-03-24T00:50:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.182+0000 7f2da0c00700 1 -- 172.21.15.38:0/3799441333 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d94003cf0 con 0x7f2da4102d00 2024-03-24T00:50:11.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.182+0000 7f2daae00700 1 -- 172.21.15.38:0/3799441333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da419e310 con 0x7f2da4102d00 2024-03-24T00:50:11.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.184+0000 7f2daae00700 1 -- 172.21.15.38:0/3799441333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da4192880 con 0x7f2da4102d00 2024-03-24T00:50:11.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.185+0000 7f2da0c00700 1 -- 172.21.15.38:0/3799441333 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 91796+0+0 (secure 0 0 0) 0x7f2d94009730 con 0x7f2da4102d00 2024-03-24T00:50:11.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.186+0000 7f2da0c00700 1 --2- 172.21.15.38:0/3799441333 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f2d8806dda0 0x7f2d88070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:11.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.187+0000 7f2da0c00700 1 -- 172.21.15.38:0/3799441333 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2d9408d7a0 con 0x7f2da4102d00 2024-03-24T00:50:11.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.187+0000 7f2da3400700 1 --2- 172.21.15.38:0/3799441333 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f2d8806dda0 0x7f2d88070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:11.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.188+0000 7f2da3400700 1 --2- 172.21.15.38:0/3799441333 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f2d8806dda0 0x7f2d88070260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d90006010 tx=0x7f2d9000b540 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:11.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.192+0000 7f2da0c00700 1 -- 172.21.15.38:0/3799441333 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f2d9405ab20 con 0x7f2da4102d00 2024-03-24T00:50:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.617+0000 7f2daae00700 1 -- 172.21.15.38:0/3799441333 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2da40611f0 con 0x7f2d8806dda0 2024-03-24T00:50:11.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.656+0000 7f2da0c00700 1 -- 172.21.15.38:0/3799441333 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2da40611f0 con 0x7f2d8806dda0 2024-03-24T00:50:11.657 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:50:11.657 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (10m) 44s ago 12m 14.1M - 0.25.0 c8568f914cd2 9f1243fbcb3e 2024-03-24T00:50:11.657 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (12m) 44s ago 12m 8036k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:50:11.657 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (11m) 2s ago 11m 7894k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:50:11.657 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (12m) 44s ago 12m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:50:11.657 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (11m) 2s ago 11m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:50:11.657 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (10m) 44s ago 12m 72.9M - 9.4.7 954c08fa6188 e6aa268d69e8 2024-03-24T00:50:11.657 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (5m) 44s ago 5m 151M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:50:11.657 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (5m) 44s ago 5m 228M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:50:11.657 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (5m) 2s ago 5m 24.2M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:50:11.657 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (5m) 2s ago 5m 23.4M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (84s) 44s ago 14m 474M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (8s) 2s ago 11m 69.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (14m) 44s ago 14m 53.8M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (11m) 2s ago 11m 43.3M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (12m) 44s ago 12m 8116k - 1.5.0 0da6a335fe13 c34cc077c93f 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (11m) 2s ago 11m 8120k - 1.5.0 0da6a335fe13 189e840ee2f1 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (10m) 44s ago 10m 203M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (9m) 44s ago 9m 218M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (8m) 44s ago 8m 177M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (8m) 2s ago 8m 240M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (7m) 2s ago 7m 229M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (7m) 2s ago 7m 200M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:50:11.658 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 starting - - - - 2024-03-24T00:50:11.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.660+0000 7f2daae00700 1 -- 172.21.15.38:0/3799441333 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f2d8806dda0 msgr2=0x7f2d88070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:11.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.660+0000 7f2daae00700 1 --2- 172.21.15.38:0/3799441333 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f2d8806dda0 0x7f2d88070260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d90006010 tx=0x7f2d9000b540 comp rx=0 tx=0).stop 2024-03-24T00:50:11.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.660+0000 7f2daae00700 1 -- 172.21.15.38:0/3799441333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2da4102d00 msgr2=0x7f2da41986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:11.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.660+0000 7f2daae00700 1 --2- 172.21.15.38:0/3799441333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2da4102d00 0x7f2da41986c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400d900 tx=0x7f2d9400dc10 comp rx=0 tx=0).stop 2024-03-24T00:50:11.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.661+0000 7f2daae00700 1 -- 172.21.15.38:0/3799441333 shutdown_connections 2024-03-24T00:50:11.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.661+0000 7f2daae00700 1 --2- 172.21.15.38:0/3799441333 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f2d8806dda0 0x7f2d88070260 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:11.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.661+0000 7f2daae00700 1 --2- 172.21.15.38:0/3799441333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2da4102d00 0x7f2da41986c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:11.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.661+0000 7f2daae00700 1 --2- 172.21.15.38:0/3799441333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da4103f00 0x7f2da4198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:11.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.661+0000 7f2daae00700 1 -- 172.21.15.38:0/3799441333 >> 172.21.15.38:0/3799441333 conn(0x7f2da40fe2a0 msgr2=0x7f2da4107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:11.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.662+0000 7f2daae00700 1 -- 172.21.15.38:0/3799441333 shutdown_connections 2024-03-24T00:50:11.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.662+0000 7f2daae00700 1 -- 172.21.15.38:0/3799441333 wait complete. 2024-03-24T00:50:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.916+0000 7faab0800700 1 -- 172.21.15.38:0/2572359871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa8101560 msgr2=0x7faaa8103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.916+0000 7faab0800700 1 --2- 172.21.15.38:0/2572359871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa8101560 0x7faaa8103950 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009870 tx=0x7faa9c009b80 comp rx=0 tx=0).stop 2024-03-24T00:50:11.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.917+0000 7faab0800700 1 -- 172.21.15.38:0/2572359871 shutdown_connections 2024-03-24T00:50:11.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.917+0000 7faab0800700 1 --2- 172.21.15.38:0/2572359871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8103e90 0x7faaa8106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:11.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.917+0000 7faab0800700 1 --2- 172.21.15.38:0/2572359871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa8101560 0x7faaa8103950 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:11.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.918+0000 7faab0800700 1 -- 172.21.15.38:0/2572359871 >> 172.21.15.38:0/2572359871 conn(0x7faaa80faf70 msgr2=0x7faaa80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:11.918 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:11 smithi038 ceph-mon[29271]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:11.918 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:11 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:11.919 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:11 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:11.919 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:11 smithi038 ceph-mon[29271]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:11.919 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:11 smithi038 ceph-mon[29271]: pgmap v29: 65 pgs: 65 active+clean; 340 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 583 op/s 2024-03-24T00:50:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.919+0000 7faab0800700 1 -- 172.21.15.38:0/2572359871 shutdown_connections 2024-03-24T00:50:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.920+0000 7faab0800700 1 -- 172.21.15.38:0/2572359871 wait complete. 2024-03-24T00:50:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.923+0000 7faab0800700 1 Processor -- start 2024-03-24T00:50:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.923+0000 7faab0800700 1 -- start start 2024-03-24T00:50:11.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.924+0000 7faab0800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8101560 0x7faaa81942c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:11.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.924+0000 7faab0800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa8103e90 0x7faaa8194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:11.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.924+0000 7faab0800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8194e20 con 0x7faaa8101560 2024-03-24T00:50:11.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.924+0000 7faab0800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8194f60 con 0x7faaa8103e90 2024-03-24T00:50:11.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.926+0000 7faaade00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8101560 0x7faaa81942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:11.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.926+0000 7faaade00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8101560 0x7faaa81942c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42328/0 (socket says 172.21.15.38:42328) 2024-03-24T00:50:11.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.926+0000 7faaade00700 1 -- 172.21.15.38:0/1802365538 learned_addr learned my addr 172.21.15.38:0/1802365538 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:50:11.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.927+0000 7faaade00700 1 -- 172.21.15.38:0/1802365538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa8103e90 msgr2=0x7faaa8194800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:11.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.927+0000 7faaad400700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa8103e90 0x7faaa8194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:11.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.927+0000 7faaade00700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa8103e90 0x7faaa8194800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:11.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.927+0000 7faaade00700 1 -- 172.21.15.38:0/1802365538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c009520 con 0x7faaa8101560 2024-03-24T00:50:11.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.929+0000 7faaad400700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa8103e90 0x7faaa8194800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:50:11.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.929+0000 7faaade00700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8101560 0x7faaa81942c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009230 tx=0x7faa9c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:11.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.931+0000 7faaa2a00700 1 -- 172.21.15.38:0/1802365538 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa9c01c070 con 0x7faaa8101560 2024-03-24T00:50:11.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.932+0000 7faaa2a00700 1 -- 172.21.15.38:0/1802365538 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa9c00b4e0 con 0x7faaa8101560 2024-03-24T00:50:11.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.932+0000 7faaa2a00700 1 -- 172.21.15.38:0/1802365538 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa9c022e50 con 0x7faaa8101560 2024-03-24T00:50:11.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.932+0000 7faab0800700 1 -- 172.21.15.38:0/1802365538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa8199960 con 0x7faaa8101560 2024-03-24T00:50:11.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.932+0000 7faab0800700 1 -- 172.21.15.38:0/1802365538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa8199e80 con 0x7faaa8101560 2024-03-24T00:50:11.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.934+0000 7faab0800700 1 -- 172.21.15.38:0/1802365538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa804ea80 con 0x7faaa8101560 2024-03-24T00:50:11.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.941+0000 7faaa2a00700 1 -- 172.21.15.38:0/1802365538 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 91796+0+0 (secure 0 0 0) 0x7faa9c020070 con 0x7faaa8101560 2024-03-24T00:50:11.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.943+0000 7faaa2a00700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa9006dda0 0x7faa90070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:11.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.943+0000 7faaa2a00700 1 -- 172.21.15.38:0/1802365538 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faa9c08f2a0 con 0x7faaa8101560 2024-03-24T00:50:11.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.944+0000 7faaad400700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa9006dda0 0x7faa90070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:11.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.945+0000 7faaad400700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa9006dda0 0x7faa90070260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faa98005fd0 tx=0x7faa9800f040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:11.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:11.950+0000 7faaa2a00700 1 -- 172.21.15.38:0/1802365538 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7faa9c05c6d0 con 0x7faaa8101560 2024-03-24T00:50:12.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:11 smithi088 ceph-mon[36610]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:12.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:11 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:12.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:11 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:12.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:11 smithi088 ceph-mon[36610]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:12.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:11 smithi088 ceph-mon[36610]: pgmap v29: 65 pgs: 65 active+clean; 340 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 583 op/s 2024-03-24T00:50:12.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.535+0000 7faab0800700 1 -- 172.21.15.38:0/1802365538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faaa819a220 con 0x7faaa8101560 2024-03-24T00:50:12.536 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:50:12.536 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:50:12.536 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:50:12.536 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:50:12.536 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:50:12.536 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:50:12.536 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 12, 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:50:12.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.536+0000 7faaa2a00700 1 -- 172.21.15.38:0/1802365538 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7faa9c0272f0 con 0x7faaa8101560 2024-03-24T00:50:12.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.539+0000 7faab0800700 1 -- 172.21.15.38:0/1802365538 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa9006dda0 msgr2=0x7faa90070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:12.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.539+0000 7faab0800700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa9006dda0 0x7faa90070260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faa98005fd0 tx=0x7faa9800f040 comp rx=0 tx=0).stop 2024-03-24T00:50:12.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.539+0000 7faab0800700 1 -- 172.21.15.38:0/1802365538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8101560 msgr2=0x7faaa81942c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:12.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.539+0000 7faab0800700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8101560 0x7faaa81942c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009230 tx=0x7faa9c00f740 comp rx=0 tx=0).stop 2024-03-24T00:50:12.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.539+0000 7faab0800700 1 -- 172.21.15.38:0/1802365538 shutdown_connections 2024-03-24T00:50:12.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.539+0000 7faab0800700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa9006dda0 0x7faa90070260 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:12.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.539+0000 7faab0800700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa8101560 0x7faaa81942c0 secure :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009230 tx=0x7faa9c00f740 comp rx=0 tx=0).stop 2024-03-24T00:50:12.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.539+0000 7faab0800700 1 --2- 172.21.15.38:0/1802365538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa8103e90 0x7faaa8194800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:12.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.539+0000 7faab0800700 1 -- 172.21.15.38:0/1802365538 >> 172.21.15.38:0/1802365538 conn(0x7faaa80faf70 msgr2=0x7faaa80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:12.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.539+0000 7faab0800700 1 -- 172.21.15.38:0/1802365538 shutdown_connections 2024-03-24T00:50:12.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.539+0000 7faab0800700 1 -- 172.21.15.38:0/1802365538 wait complete. 2024-03-24T00:50:12.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.739+0000 7fd1a3200700 1 -- 172.21.15.38:0/1883453530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c101560 msgr2=0x7fd19c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:12.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.739+0000 7fd1a3200700 1 --2- 172.21.15.38:0/1883453530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c101560 0x7fd19c103950 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009910 tx=0x7fd18c009c20 comp rx=0 tx=0).stop 2024-03-24T00:50:12.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.739+0000 7fd1a3200700 1 -- 172.21.15.38:0/1883453530 shutdown_connections 2024-03-24T00:50:12.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.739+0000 7fd1a3200700 1 --2- 172.21.15.38:0/1883453530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c103e90 0x7fd19c106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:12.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.739+0000 7fd1a3200700 1 --2- 172.21.15.38:0/1883453530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c101560 0x7fd19c103950 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:12.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.739+0000 7fd1a3200700 1 -- 172.21.15.38:0/1883453530 >> 172.21.15.38:0/1883453530 conn(0x7fd19c0faf70 msgr2=0x7fd19c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:12.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.739+0000 7fd1a3200700 1 -- 172.21.15.38:0/1883453530 shutdown_connections 2024-03-24T00:50:12.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.739+0000 7fd1a3200700 1 -- 172.21.15.38:0/1883453530 wait complete. 2024-03-24T00:50:12.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.740+0000 7fd1a3200700 1 Processor -- start 2024-03-24T00:50:12.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.741+0000 7fd1a3200700 1 -- start start 2024-03-24T00:50:12.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.741+0000 7fd1a3200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101560 0x7fd19c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:12.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.741+0000 7fd1a3200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c103e90 0x7fd19c19d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:12.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.741+0000 7fd1a3200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19d660 con 0x7fd19c103e90 2024-03-24T00:50:12.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.741+0000 7fd1a3200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19d7a0 con 0x7fd19c101560 2024-03-24T00:50:12.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.742+0000 7fd19be00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101560 0x7fd19c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:12.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.742+0000 7fd19be00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101560 0x7fd19c19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:40544/0 (socket says 172.21.15.38:40544) 2024-03-24T00:50:12.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.742+0000 7fd19be00700 1 -- 172.21.15.38:0/2283557107 learned_addr learned my addr 172.21.15.38:0/2283557107 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:50:12.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.742+0000 7fd19b400700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c103e90 0x7fd19c19d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:12.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.743+0000 7fd19be00700 1 -- 172.21.15.38:0/2283557107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c103e90 msgr2=0x7fd19c19d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:12.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.743+0000 7fd19be00700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c103e90 0x7fd19c19d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:12.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.743+0000 7fd19be00700 1 -- 172.21.15.38:0/2283557107 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0095f0 con 0x7fd19c101560 2024-03-24T00:50:12.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.743+0000 7fd19be00700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101560 0x7fd19c19cb00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00b650 tx=0x7fd18c00f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:12.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.743+0000 7fd198c00700 1 -- 172.21.15.38:0/2283557107 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18c01d070 con 0x7fd19c101560 2024-03-24T00:50:12.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.743+0000 7fd198c00700 1 -- 172.21.15.38:0/2283557107 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd18c00fe70 con 0x7fd19c101560 2024-03-24T00:50:12.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.744+0000 7fd19b400700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c103e90 0x7fd19c19d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:50:12.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.744+0000 7fd198c00700 1 -- 172.21.15.38:0/2283557107 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18c017630 con 0x7fd19c101560 2024-03-24T00:50:12.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.744+0000 7fd1a3200700 1 -- 172.21.15.38:0/2283557107 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c1a21a0 con 0x7fd19c101560 2024-03-24T00:50:12.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.744+0000 7fd1a3200700 1 -- 172.21.15.38:0/2283557107 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c1a2650 con 0x7fd19c101560 2024-03-24T00:50:12.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.746+0000 7fd1a3200700 1 -- 172.21.15.38:0/2283557107 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c04ea80 con 0x7fd19c101560 2024-03-24T00:50:12.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.748+0000 7fd198c00700 1 -- 172.21.15.38:0/2283557107 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) v1 ==== 91796+0+0 (secure 0 0 0) 0x7fd18c003860 con 0x7fd19c101560 2024-03-24T00:50:12.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.748+0000 7fd198c00700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd18006dd50 0x7fd180070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:12.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.748+0000 7fd198c00700 1 -- 172.21.15.38:0/2283557107 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd18c003e40 con 0x7fd19c101560 2024-03-24T00:50:12.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.749+0000 7fd19b400700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd18006dd50 0x7fd180070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:12.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.750+0000 7fd19b400700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd18006dd50 0x7fd180070210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd188009be0 tx=0x7fd188009260 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:12.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:12.753+0000 7fd198c00700 1 -- 172.21.15.38:0/2283557107 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fd18c05c380 con 0x7fd19c101560 2024-03-24T00:50:13.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.022+0000 7fd1a3200700 1 -- 172.21.15.38:0/2283557107 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd19c1a2920 con 0x7fd18006dd50 2024-03-24T00:50:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.024+0000 7fd198c00700 1 -- 172.21.15.38:0/2283557107 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fd19c1a2920 con 0x7fd18006dd50 2024-03-24T00:50:13.024 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:50:13.024 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:50:13.024 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:50:13.024 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-24T00:50:13.024 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:50:13.024 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-03-24T00:50:13.024 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:50:13.025 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "2/2 daemons upgraded", 2024-03-24T00:50:13.025 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mgr daemons", 2024-03-24T00:50:13.025 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:50:13.025 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:50:13.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.027+0000 7fd1a3200700 1 -- 172.21.15.38:0/2283557107 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd18006dd50 msgr2=0x7fd180070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:13.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.027+0000 7fd1a3200700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd18006dd50 0x7fd180070210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd188009be0 tx=0x7fd188009260 comp rx=0 tx=0).stop 2024-03-24T00:50:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.027+0000 7fd1a3200700 1 -- 172.21.15.38:0/2283557107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101560 msgr2=0x7fd19c19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.027+0000 7fd1a3200700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101560 0x7fd19c19cb00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00b650 tx=0x7fd18c00f510 comp rx=0 tx=0).stop 2024-03-24T00:50:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.027+0000 7fd1a3200700 1 -- 172.21.15.38:0/2283557107 shutdown_connections 2024-03-24T00:50:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.027+0000 7fd1a3200700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd18006dd50 0x7fd180070210 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.027+0000 7fd1a3200700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101560 0x7fd19c19cb00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.027+0000 7fd1a3200700 1 --2- 172.21.15.38:0/2283557107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c103e90 0x7fd19c19d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.027+0000 7fd1a3200700 1 -- 172.21.15.38:0/2283557107 >> 172.21.15.38:0/2283557107 conn(0x7fd19c0faf70 msgr2=0x7fd19c0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.027+0000 7fd1a3200700 1 -- 172.21.15.38:0/2283557107 shutdown_connections 2024-03-24T00:50:13.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:13.028+0000 7fd1a3200700 1 -- 172.21.15.38:0/2283557107 wait complete. 2024-03-24T00:50:13.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:12 smithi088 ceph-mon[36610]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:13.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:12 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1802365538' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:12 smithi038 ceph-mon[29271]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:13.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:12 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1802365538' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:14.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:13 smithi088 ceph-mon[36610]: from='client.24489 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:14.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:13 smithi088 ceph-mon[36610]: pgmap v30: 65 pgs: 65 active+clean; 340 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 450 op/s 2024-03-24T00:50:14.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:13 smithi038 ceph-mon[29271]: from='client.24489 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:14.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:13 smithi038 ceph-mon[29271]: pgmap v30: 65 pgs: 65 active+clean; 340 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 450 op/s 2024-03-24T00:50:14.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:14 smithi088 ceph-mon[36610]: Standby manager daemon smithi088.njctea restarted 2024-03-24T00:50:14.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:14 smithi088 ceph-mon[36610]: Standby manager daemon smithi088.njctea started 2024-03-24T00:50:14.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:14 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/1201305858' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/crt"}]: dispatch 2024-03-24T00:50:14.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:14 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/1201305858' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:50:14.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:14 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/1201305858' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/key"}]: dispatch 2024-03-24T00:50:14.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:14 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/1201305858' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:50:15.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:14 smithi038 ceph-mon[29271]: Standby manager daemon smithi088.njctea restarted 2024-03-24T00:50:15.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:14 smithi038 ceph-mon[29271]: Standby manager daemon smithi088.njctea started 2024-03-24T00:50:15.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:14 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/1201305858' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/crt"}]: dispatch 2024-03-24T00:50:15.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:14 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/1201305858' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:50:15.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:14 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/1201305858' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/key"}]: dispatch 2024-03-24T00:50:15.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:14 smithi038 ceph-mon[29271]: from='mgr.? 172.21.15.88:0/1201305858' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:50:16.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:15 smithi088 ceph-mon[36610]: mgrmap e31: smithi038.huhioz(active, since 55s), standbys: smithi088.njctea 2024-03-24T00:50:16.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:15 smithi088 ceph-mon[36610]: pgmap v31: 65 pgs: 65 active+clean; 337 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 953 op/s 2024-03-24T00:50:16.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:15 smithi038 ceph-mon[29271]: mgrmap e31: smithi038.huhioz(active, since 55s), standbys: smithi088.njctea 2024-03-24T00:50:16.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:15 smithi038 ceph-mon[29271]: pgmap v31: 65 pgs: 65 active+clean; 337 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 953 op/s 2024-03-24T00:50:17.275 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:17 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:17.275 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:17 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:17.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:17 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:17.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:17 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: pgmap v32: 65 pgs: 65 active+clean; 333 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 893 op/s 2024-03-24T00:50:18.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi038.huhioz"}]: dispatch 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi038.huhioz"}]': finished 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi088.njctea"}]: dispatch 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi088.njctea"}]': finished 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: pgmap v32: 65 pgs: 65 active+clean; 333 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 893 op/s 2024-03-24T00:50:18.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:50:18.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:50:18.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:50:18.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi038.huhioz"}]: dispatch 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi038.huhioz"}]': finished 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi088.njctea"}]: dispatch 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi088.njctea"}]': finished 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:18.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:18.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:19.302 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:19 smithi038 ceph-mon[29271]: Upgrade: Setting container_image for all mgr 2024-03-24T00:50:19.302 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:19 smithi038 ceph-mon[29271]: Upgrade: Setting container_image for all rgw 2024-03-24T00:50:19.303 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:19 smithi038 ceph-mon[29271]: Upgrade: Setting container_image for all rbd-mirror 2024-03-24T00:50:19.303 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:19 smithi038 ceph-mon[29271]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-24T00:50:19.303 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:19 smithi038 ceph-mon[29271]: Upgrade: Setting container_image for all iscsi 2024-03-24T00:50:19.303 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:19 smithi038 ceph-mon[29271]: Upgrade: Setting container_image for all nfs 2024-03-24T00:50:19.303 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:19 smithi038 ceph-mon[29271]: Upgrade: Setting container_image for all nvmeof 2024-03-24T00:50:19.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:19 smithi088 ceph-mon[36610]: Upgrade: Setting container_image for all mgr 2024-03-24T00:50:19.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:19 smithi088 ceph-mon[36610]: Upgrade: Setting container_image for all rgw 2024-03-24T00:50:19.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:19 smithi088 ceph-mon[36610]: Upgrade: Setting container_image for all rbd-mirror 2024-03-24T00:50:19.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:19 smithi088 ceph-mon[36610]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-24T00:50:19.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:19 smithi088 ceph-mon[36610]: Upgrade: Setting container_image for all iscsi 2024-03-24T00:50:19.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:19 smithi088 ceph-mon[36610]: Upgrade: Setting container_image for all nfs 2024-03-24T00:50:19.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:19 smithi088 ceph-mon[36610]: Upgrade: Setting container_image for all nvmeof 2024-03-24T00:50:20.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:20 smithi038 ceph-mon[29271]: pgmap v33: 65 pgs: 65 active+clean; 333 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 893 op/s 2024-03-24T00:50:20.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:20 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:50:20.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:20 smithi088 ceph-mon[36610]: pgmap v33: 65 pgs: 65 active+clean; 333 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 893 op/s 2024-03-24T00:50:20.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:20 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:50:22.317 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:22 smithi038 ceph-mon[29271]: Upgrade: Updating node-exporter.smithi038 (1/2) 2024-03-24T00:50:22.317 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:22 smithi038 ceph-mon[29271]: Deploying daemon node-exporter.smithi038 on smithi038 2024-03-24T00:50:22.317 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:22 smithi038 ceph-mon[29271]: pgmap v34: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 1.33k op/s 2024-03-24T00:50:22.536 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:22 smithi088 ceph-mon[36610]: Upgrade: Updating node-exporter.smithi038 (1/2) 2024-03-24T00:50:22.536 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:22 smithi088 ceph-mon[36610]: Deploying daemon node-exporter.smithi038 on smithi038 2024-03-24T00:50:22.536 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:22 smithi088 ceph-mon[36610]: pgmap v34: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 1.33k op/s 2024-03-24T00:50:24.591 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:24 smithi038 ceph-mon[29271]: pgmap v35: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 991 KiB/s rd, 1.3 MiB/s wr, 1.01k op/s 2024-03-24T00:50:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:24 smithi088 ceph-mon[36610]: pgmap v35: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 991 KiB/s rd, 1.3 MiB/s wr, 1.01k op/s 2024-03-24T00:50:26.499 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:26 smithi088 ceph-mon[36610]: pgmap v36: 65 pgs: 65 active+clean; 310 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 1.23k op/s 2024-03-24T00:50:26.500 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:26 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:26.500 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:26 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:26.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:26 smithi038 ceph-mon[29271]: pgmap v36: 65 pgs: 65 active+clean; 310 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 1.23k op/s 2024-03-24T00:50:26.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:26 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:26.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:26 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:28.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:28 smithi038 ceph-mon[29271]: pgmap v37: 65 pgs: 65 active+clean; 305 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 823 op/s 2024-03-24T00:50:28.643 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:28 smithi088 ceph-mon[36610]: pgmap v37: 65 pgs: 65 active+clean; 305 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 823 op/s 2024-03-24T00:50:29.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:29 smithi088 ceph-mon[36610]: Upgrade: Updating node-exporter.smithi088 (2/2) 2024-03-24T00:50:29.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:29 smithi088 ceph-mon[36610]: Deploying daemon node-exporter.smithi088 on smithi088 2024-03-24T00:50:29.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:29 smithi038 ceph-mon[29271]: Upgrade: Updating node-exporter.smithi088 (2/2) 2024-03-24T00:50:29.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:29 smithi038 ceph-mon[29271]: Deploying daemon node-exporter.smithi088 on smithi088 2024-03-24T00:50:30.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:30 smithi088 ceph-mon[36610]: pgmap v38: 65 pgs: 65 active+clean; 305 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 751 op/s 2024-03-24T00:50:30.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:30 smithi038 ceph-mon[29271]: pgmap v38: 65 pgs: 65 active+clean; 305 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 751 op/s 2024-03-24T00:50:32.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:32 smithi038 ceph-mon[29271]: pgmap v39: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 906 op/s 2024-03-24T00:50:32.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:32 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:32.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:32 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:32.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:32 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:50:32.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:32 smithi088 ceph-mon[36610]: pgmap v39: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 906 op/s 2024-03-24T00:50:32.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:32 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:32.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:32 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:32.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:32 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:50:34.532 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:34 smithi088 ceph-mon[36610]: pgmap v40: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 466 op/s 2024-03-24T00:50:34.593 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:34 smithi038 ceph-mon[29271]: pgmap v40: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 466 op/s 2024-03-24T00:50:35.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:50:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:50:36.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:36 smithi038 ceph-mon[29271]: pgmap v41: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 575 op/s 2024-03-24T00:50:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:36 smithi088 ceph-mon[36610]: pgmap v41: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 575 op/s 2024-03-24T00:50:37.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:37 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:37.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:37 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:37.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:37 smithi038 ceph-mon[29271]: pgmap v42: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 392 op/s 2024-03-24T00:50:37.880 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:37 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:37.880 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:37 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:37.880 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:37 smithi088 ceph-mon[36610]: pgmap v42: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 392 op/s 2024-03-24T00:50:39.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:39.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:39.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:39 smithi038 ceph-mon[29271]: pgmap v43: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 298 op/s 2024-03-24T00:50:39.905 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:39.905 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:39.905 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:39 smithi088 ceph-mon[36610]: pgmap v43: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 298 op/s 2024-03-24T00:50:42.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:42 smithi088 ceph-mon[36610]: pgmap v44: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 427 op/s 2024-03-24T00:50:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:42 smithi038 ceph-mon[29271]: pgmap v44: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 427 op/s 2024-03-24T00:50:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.240+0000 7fca94c00700 1 -- 172.21.15.38:0/2134566247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca90100ad0 msgr2=0x7fca90100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.240+0000 7fca94c00700 1 --2- 172.21.15.38:0/2134566247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca90100ad0 0x7fca90100ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fca80009910 tx=0x7fca80009c20 comp rx=0 tx=0).stop 2024-03-24T00:50:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.240+0000 7fca94c00700 1 -- 172.21.15.38:0/2134566247 shutdown_connections 2024-03-24T00:50:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.240+0000 7fca94c00700 1 --2- 172.21.15.38:0/2134566247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca90101cd0 0x7fca90102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.240+0000 7fca94c00700 1 --2- 172.21.15.38:0/2134566247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca90100ad0 0x7fca90100ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.240+0000 7fca94c00700 1 -- 172.21.15.38:0/2134566247 >> 172.21.15.38:0/2134566247 conn(0x7fca900fc090 msgr2=0x7fca900fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.241+0000 7fca94c00700 1 -- 172.21.15.38:0/2134566247 shutdown_connections 2024-03-24T00:50:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.241+0000 7fca94c00700 1 -- 172.21.15.38:0/2134566247 wait complete. 2024-03-24T00:50:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.242+0000 7fca94c00700 1 Processor -- start 2024-03-24T00:50:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.242+0000 7fca94c00700 1 -- start start 2024-03-24T00:50:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.243+0000 7fca94c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca90100ad0 0x7fca90074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.243+0000 7fca94c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca90101cd0 0x7fca900731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.243+0000 7fca94c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca900736e0 con 0x7fca90100ad0 2024-03-24T00:50:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.243+0000 7fca94c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90073820 con 0x7fca90101cd0 2024-03-24T00:50:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.243+0000 7fca8de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca90100ad0 0x7fca90074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.243+0000 7fca8de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca90100ad0 0x7fca90074ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59962/0 (socket says 172.21.15.38:59962) 2024-03-24T00:50:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.243+0000 7fca8de00700 1 -- 172.21.15.38:0/624592663 learned_addr learned my addr 172.21.15.38:0/624592663 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:50:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.243+0000 7fca8d400700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca90101cd0 0x7fca900731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.244+0000 7fca8de00700 1 -- 172.21.15.38:0/624592663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca90101cd0 msgr2=0x7fca900731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.244+0000 7fca8de00700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca90101cd0 0x7fca900731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.244+0000 7fca8de00700 1 -- 172.21.15.38:0/624592663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800095f0 con 0x7fca90100ad0 2024-03-24T00:50:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.244+0000 7fca8d400700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca90101cd0 0x7fca900731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T00:50:43.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.244+0000 7fca8de00700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca90100ad0 0x7fca90074ab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fca8000bd70 tx=0x7fca8000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:43.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.245+0000 7fca86a00700 1 -- 172.21.15.38:0/624592663 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca8001d070 con 0x7fca90100ad0 2024-03-24T00:50:43.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.245+0000 7fca86a00700 1 -- 172.21.15.38:0/624592663 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca8000f460 con 0x7fca90100ad0 2024-03-24T00:50:43.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.245+0000 7fca86a00700 1 -- 172.21.15.38:0/624592663 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca80021640 con 0x7fca90100ad0 2024-03-24T00:50:43.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.245+0000 7fca94c00700 1 -- 172.21.15.38:0/624592663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca90073aa0 con 0x7fca90100ad0 2024-03-24T00:50:43.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.245+0000 7fca94c00700 1 -- 172.21.15.38:0/624592663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca90074010 con 0x7fca90100ad0 2024-03-24T00:50:43.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.247+0000 7fca94c00700 1 -- 172.21.15.38:0/624592663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca901906d0 con 0x7fca90100ad0 2024-03-24T00:50:43.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.247+0000 7fca86a00700 1 -- 172.21.15.38:0/624592663 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 91796+0+0 (secure 0 0 0) 0x7fca8000fab0 con 0x7fca90100ad0 2024-03-24T00:50:43.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.248+0000 7fca86a00700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fca7406dda0 0x7fca74070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:43.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.248+0000 7fca86a00700 1 -- 172.21.15.38:0/624592663 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fca8008ea80 con 0x7fca90100ad0 2024-03-24T00:50:43.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.249+0000 7fca8d400700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fca7406dda0 0x7fca74070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.251+0000 7fca8d400700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fca7406dda0 0x7fca74070260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009de0 tx=0x7fca7c009190 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:43.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.256+0000 7fca86a00700 1 -- 172.21.15.38:0/624592663 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fca8005be00 con 0x7fca90100ad0 2024-03-24T00:50:43.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.607+0000 7fca94c00700 1 -- 172.21.15.38:0/624592663 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca90074010 con 0x7fca7406dda0 2024-03-24T00:50:43.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.609+0000 7fca86a00700 1 -- 172.21.15.38:0/624592663 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fca90074010 con 0x7fca7406dda0 2024-03-24T00:50:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.613+0000 7fca94c00700 1 -- 172.21.15.38:0/624592663 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fca7406dda0 msgr2=0x7fca74070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.613+0000 7fca94c00700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fca7406dda0 0x7fca74070260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009de0 tx=0x7fca7c009190 comp rx=0 tx=0).stop 2024-03-24T00:50:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.613+0000 7fca94c00700 1 -- 172.21.15.38:0/624592663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca90100ad0 msgr2=0x7fca90074ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.613+0000 7fca94c00700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca90100ad0 0x7fca90074ab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fca8000bd70 tx=0x7fca8000bda0 comp rx=0 tx=0).stop 2024-03-24T00:50:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.613+0000 7fca94c00700 1 -- 172.21.15.38:0/624592663 shutdown_connections 2024-03-24T00:50:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.613+0000 7fca94c00700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fca7406dda0 0x7fca74070260 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.613+0000 7fca94c00700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca90100ad0 0x7fca90074ab0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.613+0000 7fca94c00700 1 --2- 172.21.15.38:0/624592663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca90101cd0 0x7fca900731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:43.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.613+0000 7fca94c00700 1 -- 172.21.15.38:0/624592663 >> 172.21.15.38:0/624592663 conn(0x7fca900fc090 msgr2=0x7fca90105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:43.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.614+0000 7fca94c00700 1 -- 172.21.15.38:0/624592663 shutdown_connections 2024-03-24T00:50:43.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.614+0000 7fca94c00700 1 -- 172.21.15.38:0/624592663 wait complete. 2024-03-24T00:50:43.631 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:50:43.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.803+0000 7fd215a00700 1 -- 172.21.15.38:0/3641504995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd210103e90 msgr2=0x7fd210106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:43.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.803+0000 7fd215a00700 1 --2- 172.21.15.38:0/3641504995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd210103e90 0x7fd210106280 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009910 tx=0x7fd1fc009c20 comp rx=0 tx=0).stop 2024-03-24T00:50:43.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.804+0000 7fd215a00700 1 -- 172.21.15.38:0/3641504995 shutdown_connections 2024-03-24T00:50:43.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.804+0000 7fd215a00700 1 --2- 172.21.15.38:0/3641504995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd210103e90 0x7fd210106280 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:43.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.804+0000 7fd215a00700 1 --2- 172.21.15.38:0/3641504995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd210101560 0x7fd210103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:43.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.804+0000 7fd215a00700 1 -- 172.21.15.38:0/3641504995 >> 172.21.15.38:0/3641504995 conn(0x7fd2100faf70 msgr2=0x7fd2100fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:43.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.804+0000 7fd215a00700 1 -- 172.21.15.38:0/3641504995 shutdown_connections 2024-03-24T00:50:43.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.804+0000 7fd215a00700 1 -- 172.21.15.38:0/3641504995 wait complete. 2024-03-24T00:50:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.805+0000 7fd215a00700 1 Processor -- start 2024-03-24T00:50:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.806+0000 7fd215a00700 1 -- start start 2024-03-24T00:50:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.806+0000 7fd215a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd210101560 0x7fd210198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.806+0000 7fd215a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd210103e90 0x7fd210198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:43.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.806+0000 7fd215a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd210199270 con 0x7fd210101560 2024-03-24T00:50:43.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.806+0000 7fd215a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2101993b0 con 0x7fd210103e90 2024-03-24T00:50:43.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.807+0000 7fd20ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd210101560 0x7fd210198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:43.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.807+0000 7fd20ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd210101560 0x7fd210198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59980/0 (socket says 172.21.15.38:59980) 2024-03-24T00:50:43.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.807+0000 7fd20ea00700 1 -- 172.21.15.38:0/1751398001 learned_addr learned my addr 172.21.15.38:0/1751398001 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:50:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.808+0000 7fd20e000700 1 --2- 172.21.15.38:0/1751398001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd210103e90 0x7fd210198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.808+0000 7fd20ea00700 1 -- 172.21.15.38:0/1751398001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd210103e90 msgr2=0x7fd210198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.808+0000 7fd20ea00700 1 --2- 172.21.15.38:0/1751398001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd210103e90 0x7fd210198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.808+0000 7fd20ea00700 1 -- 172.21.15.38:0/1751398001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc0095f0 con 0x7fd210101560 2024-03-24T00:50:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.809+0000 7fd20ea00700 1 --2- 172.21.15.38:0/1751398001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd210101560 0x7fd210198710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd20000d900 tx=0x7fd20000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.809+0000 7fd207400700 1 -- 172.21.15.38:0/1751398001 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd200004a00 con 0x7fd210101560 2024-03-24T00:50:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.809+0000 7fd207400700 1 -- 172.21.15.38:0/1751398001 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2000054e0 con 0x7fd210101560 2024-03-24T00:50:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.809+0000 7fd207400700 1 -- 172.21.15.38:0/1751398001 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd200009d60 con 0x7fd210101560 2024-03-24T00:50:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.810+0000 7fd215a00700 1 -- 172.21.15.38:0/1751398001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21019de10 con 0x7fd210101560 2024-03-24T00:50:43.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.810+0000 7fd215a00700 1 -- 172.21.15.38:0/1751398001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21019e350 con 0x7fd210101560 2024-03-24T00:50:43.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.813+0000 7fd215a00700 1 -- 172.21.15.38:0/1751398001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21004ea80 con 0x7fd210101560 2024-03-24T00:50:43.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.814+0000 7fd207400700 1 -- 172.21.15.38:0/1751398001 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 91796+0+0 (secure 0 0 0) 0x7fd200005020 con 0x7fd210101560 2024-03-24T00:50:43.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.817+0000 7fd207400700 1 --2- 172.21.15.38:0/1751398001 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd1f406dd50 0x7fd1f4070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:43.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.818+0000 7fd207400700 1 -- 172.21.15.38:0/1751398001 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd20008da10 con 0x7fd210101560 2024-03-24T00:50:43.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.818+0000 7fd20e000700 1 --2- 172.21.15.38:0/1751398001 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd1f406dd50 0x7fd1f4070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:43.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.821+0000 7fd20e000700 1 --2- 172.21.15.38:0/1751398001 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd1f406dd50 0x7fd1f4070210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00b5c0 tx=0x7fd1fc005160 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:43.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:43.824+0000 7fd207400700 1 -- 172.21.15.38:0/1751398001 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fd20005ac50 con 0x7fd210101560 2024-03-24T00:50:44.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.246+0000 7fd215a00700 1 -- 172.21.15.38:0/1751398001 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2100ffc80 con 0x7fd1f406dd50 2024-03-24T00:50:44.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.247+0000 7fd207400700 1 -- 172.21.15.38:0/1751398001 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fd2100ffc80 con 0x7fd1f406dd50 2024-03-24T00:50:44.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.250+0000 7fd215a00700 1 -- 172.21.15.38:0/1751398001 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd1f406dd50 msgr2=0x7fd1f4070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:44.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.250+0000 7fd215a00700 1 --2- 172.21.15.38:0/1751398001 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd1f406dd50 0x7fd1f4070210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00b5c0 tx=0x7fd1fc005160 comp rx=0 tx=0).stop 2024-03-24T00:50:44.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.250+0000 7fd215a00700 1 -- 172.21.15.38:0/1751398001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd210101560 msgr2=0x7fd210198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:44.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.250+0000 7fd215a00700 1 --2- 172.21.15.38:0/1751398001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd210101560 0x7fd210198710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd20000d900 tx=0x7fd20000dcc0 comp rx=0 tx=0).stop 2024-03-24T00:50:44.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.250+0000 7fd215a00700 1 -- 172.21.15.38:0/1751398001 shutdown_connections 2024-03-24T00:50:44.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.250+0000 7fd215a00700 1 --2- 172.21.15.38:0/1751398001 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd1f406dd50 0x7fd1f4070210 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:44.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.250+0000 7fd215a00700 1 --2- 172.21.15.38:0/1751398001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd210101560 0x7fd210198710 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:44.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.250+0000 7fd215a00700 1 --2- 172.21.15.38:0/1751398001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd210103e90 0x7fd210198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:44.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.250+0000 7fd215a00700 1 -- 172.21.15.38:0/1751398001 >> 172.21.15.38:0/1751398001 conn(0x7fd2100faf70 msgr2=0x7fd2100ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:44.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.251+0000 7fd215a00700 1 -- 172.21.15.38:0/1751398001 shutdown_connections 2024-03-24T00:50:44.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.251+0000 7fd215a00700 1 -- 172.21.15.38:0/1751398001 wait complete. 2024-03-24T00:50:44.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.449+0000 7f93db400700 1 -- 172.21.15.38:0/4170244495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93d41082f0 msgr2=0x7f93d4108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:44.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.449+0000 7f93db400700 1 --2- 172.21.15.38:0/4170244495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93d41082f0 0x7f93d4108770 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009910 tx=0x7f93c4009c20 comp rx=0 tx=0).stop 2024-03-24T00:50:44.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.449+0000 7f93db400700 1 -- 172.21.15.38:0/4170244495 shutdown_connections 2024-03-24T00:50:44.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.449+0000 7f93db400700 1 --2- 172.21.15.38:0/4170244495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93d41082f0 0x7f93d4108770 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.449+0000 7f93db400700 1 --2- 172.21.15.38:0/4170244495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d4072970 0x7f93d4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.449+0000 7f93db400700 1 -- 172.21.15.38:0/4170244495 >> 172.21.15.38:0/4170244495 conn(0x7f93d406da40 msgr2=0x7f93d406fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.449+0000 7f93db400700 1 -- 172.21.15.38:0/4170244495 shutdown_connections 2024-03-24T00:50:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.449+0000 7f93db400700 1 -- 172.21.15.38:0/4170244495 wait complete. 2024-03-24T00:50:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.450+0000 7f93db400700 1 Processor -- start 2024-03-24T00:50:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.450+0000 7f93db400700 1 -- start start 2024-03-24T00:50:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.450+0000 7f93db400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d4072970 0x7f93d410a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.450+0000 7f93db400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93d41082f0 0x7f93d4109080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.450+0000 7f93db400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d41095c0 con 0x7f93d41082f0 2024-03-24T00:50:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.450+0000 7f93db400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d4109700 con 0x7f93d4072970 2024-03-24T00:50:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.452+0000 7f93d3e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93d41082f0 0x7f93d4109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.452+0000 7f93d3e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93d41082f0 0x7f93d4109080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59992/0 (socket says 172.21.15.38:59992) 2024-03-24T00:50:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.452+0000 7f93d3e00700 1 -- 172.21.15.38:0/2548748445 learned_addr learned my addr 172.21.15.38:0/2548748445 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:50:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.452+0000 7f93d8a00700 1 --2- 172.21.15.38:0/2548748445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d4072970 0x7f93d410a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:44.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.453+0000 7f93d3e00700 1 -- 172.21.15.38:0/2548748445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d4072970 msgr2=0x7f93d410a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:44.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.453+0000 7f93d3e00700 1 --2- 172.21.15.38:0/2548748445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d4072970 0x7f93d410a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:44.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.453+0000 7f93d3e00700 1 -- 172.21.15.38:0/2548748445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40095f0 con 0x7f93d41082f0 2024-03-24T00:50:44.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.454+0000 7f93d3e00700 1 --2- 172.21.15.38:0/2548748445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93d41082f0 0x7f93d4109080 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f93c400bda0 tx=0x7f93c400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:44.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.454+0000 7f93d1600700 1 -- 172.21.15.38:0/2548748445 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93c401d070 con 0x7f93d41082f0 2024-03-24T00:50:44.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.455+0000 7f93db400700 1 -- 172.21.15.38:0/2548748445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d4109980 con 0x7f93d41082f0 2024-03-24T00:50:44.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.455+0000 7f93db400700 1 -- 172.21.15.38:0/2548748445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d4109ef0 con 0x7f93d41082f0 2024-03-24T00:50:44.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.457+0000 7f93db400700 1 -- 172.21.15.38:0/2548748445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d4198e70 con 0x7f93d41082f0 2024-03-24T00:50:44.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.457+0000 7f93d1600700 1 -- 172.21.15.38:0/2548748445 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93c400f460 con 0x7f93d41082f0 2024-03-24T00:50:44.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.457+0000 7f93d1600700 1 -- 172.21.15.38:0/2548748445 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93c4021620 con 0x7f93d41082f0 2024-03-24T00:50:44.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.460+0000 7f93d1600700 1 -- 172.21.15.38:0/2548748445 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 91796+0+0 (secure 0 0 0) 0x7f93c4003a60 con 0x7f93d41082f0 2024-03-24T00:50:44.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.462+0000 7f93d1600700 1 --2- 172.21.15.38:0/2548748445 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f93b806dd50 0x7f93b8070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:44.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.463+0000 7f93d1600700 1 -- 172.21.15.38:0/2548748445 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f93c408f270 con 0x7f93d41082f0 2024-03-24T00:50:44.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.466+0000 7f93d8a00700 1 --2- 172.21.15.38:0/2548748445 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f93b806dd50 0x7f93b8070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:44.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.467+0000 7f93d8a00700 1 --2- 172.21.15.38:0/2548748445 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f93b806dd50 0x7f93b8070210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f93cc0060b0 tx=0x7f93cc007b80 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:44.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.471+0000 7f93d1600700 1 -- 172.21.15.38:0/2548748445 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f93c405c5f0 con 0x7f93d41082f0 2024-03-24T00:50:44.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:44 smithi038 ceph-mon[29271]: pgmap v45: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 272 op/s 2024-03-24T00:50:44.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:44 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:44.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:44 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:44.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:44 smithi088 ceph-mon[36610]: pgmap v45: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 272 op/s 2024-03-24T00:50:44.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:44 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:44.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:44 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:44.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.848+0000 7f93db400700 1 -- 172.21.15.38:0/2548748445 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f93d4109ef0 con 0x7f93b806dd50 2024-03-24T00:50:44.873 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:50:44.873 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (11m) 28s ago 13m 14.6M - 0.25.0 c8568f914cd2 9f1243fbcb3e 2024-03-24T00:50:44.873 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (13m) 28s ago 13m 8108k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:50:44.873 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (11m) 8s ago 11m 7922k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:50:44.873 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (13m) 28s ago 13m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:50:44.873 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (11m) 8s ago 11m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:50:44.873 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (10m) 28s ago 12m 74.1M - 9.4.7 954c08fa6188 e6aa268d69e8 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (5m) 28s ago 5m 149M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (5m) 28s ago 5m 225M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (5m) 8s ago 5m 24.4M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (5m) 8s ago 5m 24.3M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (117s) 28s ago 14m 494M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (41s) 8s ago 11m 447M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (15m) 28s ago 15m 54.1M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (11m) 8s ago 11m 44.5M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 starting - - - - 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (13s) 8s ago 11m 6753k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (10m) 28s ago 10m 200M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (10m) 28s ago 10m 218M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (9m) 28s ago 9m 176M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:50:44.874 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (8m) 8s ago 8m 274M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:50:44.875 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (8m) 8s ago 8m 258M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:50:44.875 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (7m) 8s ago 7m 222M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:50:44.875 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (50s) 28s ago 12m 47.5M - 2.43.0 a07b618ecd1d 5985f95c5f6c 2024-03-24T00:50:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.871+0000 7f93d1600700 1 -- 172.21.15.38:0/2548748445 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f93d4109ef0 con 0x7f93b806dd50 2024-03-24T00:50:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.875+0000 7f93db400700 1 -- 172.21.15.38:0/2548748445 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f93b806dd50 msgr2=0x7f93b8070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.875+0000 7f93db400700 1 --2- 172.21.15.38:0/2548748445 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f93b806dd50 0x7f93b8070210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f93cc0060b0 tx=0x7f93cc007b80 comp rx=0 tx=0).stop 2024-03-24T00:50:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.876+0000 7f93db400700 1 -- 172.21.15.38:0/2548748445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93d41082f0 msgr2=0x7f93d4109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.876+0000 7f93db400700 1 --2- 172.21.15.38:0/2548748445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93d41082f0 0x7f93d4109080 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f93c400bda0 tx=0x7f93c400be80 comp rx=0 tx=0).stop 2024-03-24T00:50:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.876+0000 7f93db400700 1 -- 172.21.15.38:0/2548748445 shutdown_connections 2024-03-24T00:50:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.876+0000 7f93db400700 1 --2- 172.21.15.38:0/2548748445 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f93b806dd50 0x7f93b8070210 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f93cc0060b0 tx=0x7f93cc007b80 comp rx=0 tx=0).stop 2024-03-24T00:50:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.876+0000 7f93db400700 1 --2- 172.21.15.38:0/2548748445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d4072970 0x7f93d410a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.876+0000 7f93db400700 1 --2- 172.21.15.38:0/2548748445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93d41082f0 0x7f93d4109080 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f93c400bda0 tx=0x7f93c400be80 comp rx=0 tx=0).stop 2024-03-24T00:50:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.876+0000 7f93db400700 1 -- 172.21.15.38:0/2548748445 >> 172.21.15.38:0/2548748445 conn(0x7f93d406da40 msgr2=0x7f93d410d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.877+0000 7f93db400700 1 -- 172.21.15.38:0/2548748445 shutdown_connections 2024-03-24T00:50:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:44.878+0000 7f93db400700 1 -- 172.21.15.38:0/2548748445 wait complete. 2024-03-24T00:50:45.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.101+0000 7fbe38a00700 1 -- 172.21.15.38:0/557157420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe340ffcf0 msgr2=0x7fbe34100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:45.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.101+0000 7fbe38a00700 1 --2- 172.21.15.38:0/557157420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe340ffcf0 0x7fbe34100170 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400b0a0 tx=0x7fbe2400b3b0 comp rx=0 tx=0).stop 2024-03-24T00:50:45.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.102+0000 7fbe38a00700 1 -- 172.21.15.38:0/557157420 shutdown_connections 2024-03-24T00:50:45.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.102+0000 7fbe38a00700 1 --2- 172.21.15.38:0/557157420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe340ffcf0 0x7fbe34100170 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:45.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.102+0000 7fbe38a00700 1 --2- 172.21.15.38:0/557157420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe340ff390 0x7fbe340ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:45.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.102+0000 7fbe38a00700 1 -- 172.21.15.38:0/557157420 >> 172.21.15.38:0/557157420 conn(0x7fbe340faf70 msgr2=0x7fbe340fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:45.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.103+0000 7fbe38a00700 1 -- 172.21.15.38:0/557157420 shutdown_connections 2024-03-24T00:50:45.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.103+0000 7fbe38a00700 1 -- 172.21.15.38:0/557157420 wait complete. 2024-03-24T00:50:45.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.105+0000 7fbe38a00700 1 Processor -- start 2024-03-24T00:50:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.105+0000 7fbe38a00700 1 -- start start 2024-03-24T00:50:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.105+0000 7fbe38a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe340ff390 0x7fbe34198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.105+0000 7fbe38a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe340ffcf0 0x7fbe34198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.105+0000 7fbe38a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe34199280 con 0x7fbe340ffcf0 2024-03-24T00:50:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.105+0000 7fbe38a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe341993c0 con 0x7fbe340ff390 2024-03-24T00:50:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.106+0000 7fbe31e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe340ff390 0x7fbe34198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.106+0000 7fbe31e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe340ff390 0x7fbe34198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:45002/0 (socket says 172.21.15.38:45002) 2024-03-24T00:50:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.106+0000 7fbe31e00700 1 -- 172.21.15.38:0/111681070 learned_addr learned my addr 172.21.15.38:0/111681070 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:50:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.107+0000 7fbe31400700 1 --2- 172.21.15.38:0/111681070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe340ffcf0 0x7fbe34198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.107+0000 7fbe31e00700 1 -- 172.21.15.38:0/111681070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe340ffcf0 msgr2=0x7fbe34198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.107+0000 7fbe31e00700 1 --2- 172.21.15.38:0/111681070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe340ffcf0 0x7fbe34198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.107+0000 7fbe31e00700 1 -- 172.21.15.38:0/111681070 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe24009d00 con 0x7fbe340ff390 2024-03-24T00:50:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.107+0000 7fbe31e00700 1 --2- 172.21.15.38:0/111681070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe340ff390 0x7fbe34198720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00d8d0 tx=0x7fbe2c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.108+0000 7fbe2aa00700 1 -- 172.21.15.38:0/111681070 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe2c009940 con 0x7fbe340ff390 2024-03-24T00:50:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.108+0000 7fbe38a00700 1 -- 172.21.15.38:0/111681070 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3419ddd0 con 0x7fbe340ff390 2024-03-24T00:50:45.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.108+0000 7fbe38a00700 1 -- 172.21.15.38:0/111681070 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3419e2d0 con 0x7fbe340ff390 2024-03-24T00:50:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.108+0000 7fbe2aa00700 1 -- 172.21.15.38:0/111681070 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe2c010460 con 0x7fbe340ff390 2024-03-24T00:50:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.108+0000 7fbe2aa00700 1 -- 172.21.15.38:0/111681070 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe2c00f5f0 con 0x7fbe340ff390 2024-03-24T00:50:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.110+0000 7fbe38a00700 1 -- 172.21.15.38:0/111681070 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe10005320 con 0x7fbe340ff390 2024-03-24T00:50:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.111+0000 7fbe2aa00700 1 -- 172.21.15.38:0/111681070 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91796+0+0 (secure 0 0 0) 0x7fbe2c010ad0 con 0x7fbe340ff390 2024-03-24T00:50:45.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.112+0000 7fbe2aa00700 1 --2- 172.21.15.38:0/111681070 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe1806dd50 0x7fbe18070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:45.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.112+0000 7fbe2aa00700 1 -- 172.21.15.38:0/111681070 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbe2c08da00 con 0x7fbe340ff390 2024-03-24T00:50:45.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.113+0000 7fbe31400700 1 --2- 172.21.15.38:0/111681070 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe1806dd50 0x7fbe18070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:45.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.113+0000 7fbe31400700 1 --2- 172.21.15.38:0/111681070 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe1806dd50 0x7fbe18070210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000f80 tx=0x7fbe2400b7b0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:45.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.127+0000 7fbe2aa00700 1 -- 172.21.15.38:0/111681070 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fbe2c05ad70 con 0x7fbe340ff390 2024-03-24T00:50:45.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:45 smithi038 ceph-mon[29271]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:45.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:45 smithi038 ceph-mon[29271]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:45.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.593+0000 7fbe38a00700 1 -- 172.21.15.38:0/111681070 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbe10005cc0 con 0x7fbe340ff390 2024-03-24T00:50:45.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.594+0000 7fbe2aa00700 1 -- 172.21.15.38:0/111681070 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7fbe2c05a900 con 0x7fbe340ff390 2024-03-24T00:50:45.595 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:50:45.596 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 12, 2024-03-24T00:50:45.597 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:50:45.597 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:50:45.597 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:50:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.598+0000 7fbe38a00700 1 -- 172.21.15.38:0/111681070 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe1806dd50 msgr2=0x7fbe18070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.598+0000 7fbe38a00700 1 --2- 172.21.15.38:0/111681070 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe1806dd50 0x7fbe18070210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000f80 tx=0x7fbe2400b7b0 comp rx=0 tx=0).stop 2024-03-24T00:50:45.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.598+0000 7fbe38a00700 1 -- 172.21.15.38:0/111681070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe340ff390 msgr2=0x7fbe34198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:45.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.598+0000 7fbe38a00700 1 --2- 172.21.15.38:0/111681070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe340ff390 0x7fbe34198720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00d8d0 tx=0x7fbe2c00dc90 comp rx=0 tx=0).stop 2024-03-24T00:50:45.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.600+0000 7fbe38a00700 1 -- 172.21.15.38:0/111681070 shutdown_connections 2024-03-24T00:50:45.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.600+0000 7fbe38a00700 1 --2- 172.21.15.38:0/111681070 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe1806dd50 0x7fbe18070210 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:45.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.600+0000 7fbe38a00700 1 --2- 172.21.15.38:0/111681070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe340ff390 0x7fbe34198720 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:45.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.600+0000 7fbe38a00700 1 --2- 172.21.15.38:0/111681070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe340ffcf0 0x7fbe34198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:45.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.600+0000 7fbe38a00700 1 -- 172.21.15.38:0/111681070 >> 172.21.15.38:0/111681070 conn(0x7fbe340faf70 msgr2=0x7fbe34107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:45.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.600+0000 7fbe38a00700 1 -- 172.21.15.38:0/111681070 shutdown_connections 2024-03-24T00:50:45.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.600+0000 7fbe38a00700 1 -- 172.21.15.38:0/111681070 wait complete. 2024-03-24T00:50:45.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:45 smithi088 ceph-mon[36610]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:45.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:45 smithi088 ceph-mon[36610]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:45.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.822+0000 7f5235000700 1 -- 172.21.15.38:0/226576102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5230107130 msgr2=0x7f5230107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:45.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.822+0000 7f5235000700 1 --2- 172.21.15.38:0/226576102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5230107130 0x7f5230107550 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5220009910 tx=0x7f5220009c20 comp rx=0 tx=0).stop 2024-03-24T00:50:45.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.822+0000 7f5235000700 1 -- 172.21.15.38:0/226576102 shutdown_connections 2024-03-24T00:50:45.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.822+0000 7f5235000700 1 --2- 172.21.15.38:0/226576102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5230108300 0x7f5230108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:45.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.823+0000 7f5235000700 1 --2- 172.21.15.38:0/226576102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5230107130 0x7f5230107550 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:45.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.823+0000 7f5235000700 1 -- 172.21.15.38:0/226576102 >> 172.21.15.38:0/226576102 conn(0x7f52300762d0 msgr2=0x7f5230078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:45.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.823+0000 7f5235000700 1 -- 172.21.15.38:0/226576102 shutdown_connections 2024-03-24T00:50:45.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.823+0000 7f5235000700 1 -- 172.21.15.38:0/226576102 wait complete. 2024-03-24T00:50:45.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.826+0000 7f5235000700 1 Processor -- start 2024-03-24T00:50:45.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.827+0000 7f5235000700 1 -- start start 2024-03-24T00:50:45.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.828+0000 7f5235000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5230107130 0x7f523019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:45.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.828+0000 7f5235000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5230108300 0x7f523019d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:45.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.829+0000 7f5235000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523019d7a0 con 0x7f5230108300 2024-03-24T00:50:45.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.829+0000 7f5235000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52301a2100 con 0x7f5230107130 2024-03-24T00:50:45.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.829+0000 7f522de00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5230107130 0x7f523019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:45.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.830+0000 7f522d400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5230108300 0x7f523019d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:45.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.830+0000 7f522de00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5230107130 0x7f523019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:45014/0 (socket says 172.21.15.38:45014) 2024-03-24T00:50:45.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.830+0000 7f522de00700 1 -- 172.21.15.38:0/3764446092 learned_addr learned my addr 172.21.15.38:0/3764446092 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:50:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.831+0000 7f522de00700 1 -- 172.21.15.38:0/3764446092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5230108300 msgr2=0x7f523019d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.831+0000 7f522de00700 1 --2- 172.21.15.38:0/3764446092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5230108300 0x7f523019d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.831+0000 7f522de00700 1 -- 172.21.15.38:0/3764446092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52200095f0 con 0x7f5230107130 2024-03-24T00:50:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.831+0000 7f522d400700 1 --2- 172.21.15.38:0/3764446092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5230108300 0x7f523019d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:50:45.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.831+0000 7f522de00700 1 --2- 172.21.15.38:0/3764446092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5230107130 0x7f523019cc40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5220000c00 tx=0x7f522000be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:45.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.831+0000 7f5226a00700 1 -- 172.21.15.38:0/3764446092 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f522001d070 con 0x7f5230107130 2024-03-24T00:50:45.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.832+0000 7f5235000700 1 -- 172.21.15.38:0/3764446092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52301a22a0 con 0x7f5230107130 2024-03-24T00:50:45.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.832+0000 7f5235000700 1 -- 172.21.15.38:0/3764446092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52301a2790 con 0x7f5230107130 2024-03-24T00:50:45.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.833+0000 7f5226a00700 1 -- 172.21.15.38:0/3764446092 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5220022470 con 0x7f5230107130 2024-03-24T00:50:45.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.833+0000 7f5226a00700 1 -- 172.21.15.38:0/3764446092 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f522000f690 con 0x7f5230107130 2024-03-24T00:50:45.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.833+0000 7f521be00700 1 -- 172.21.15.38:0/3764446092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5210005320 con 0x7f5230107130 2024-03-24T00:50:45.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.835+0000 7f5226a00700 1 -- 172.21.15.38:0/3764446092 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91796+0+0 (secure 0 0 0) 0x7f522000f8b0 con 0x7f5230107130 2024-03-24T00:50:45.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.838+0000 7f5226a00700 1 --2- 172.21.15.38:0/3764446092 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f521406dda0 0x7f5214070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:50:45.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.838+0000 7f5226a00700 1 -- 172.21.15.38:0/3764446092 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f522008f780 con 0x7f5230107130 2024-03-24T00:50:45.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.839+0000 7f522d400700 1 --2- 172.21.15.38:0/3764446092 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f521406dda0 0x7f5214070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:50:45.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.841+0000 7f5226a00700 1 -- 172.21.15.38:0/3764446092 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f52200591a0 con 0x7f5230107130 2024-03-24T00:50:45.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:45.841+0000 7f522d400700 1 --2- 172.21.15.38:0/3764446092 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f521406dda0 0x7f5214070260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f521c006fd0 tx=0x7f521c008040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:50:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.167+0000 7f521be00700 1 -- 172.21.15.38:0/3764446092 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5210000bf0 con 0x7f521406dda0 2024-03-24T00:50:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.169+0000 7f5226a00700 1 -- 172.21.15.38:0/3764446092 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f5210000bf0 con 0x7f521406dda0 2024-03-24T00:50:46.171 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:50:46.171 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:50:46.171 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:50:46.171 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-24T00:50:46.171 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:50:46.171 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-03-24T00:50:46.171 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:50:46.171 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "2/2 daemons upgraded", 2024-03-24T00:50:46.171 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading node-exporter daemons", 2024-03-24T00:50:46.171 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:50:46.171 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:50:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.174+0000 7f521be00700 1 -- 172.21.15.38:0/3764446092 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f521406dda0 msgr2=0x7f5214070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.174+0000 7f521be00700 1 --2- 172.21.15.38:0/3764446092 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f521406dda0 0x7f5214070260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f521c006fd0 tx=0x7f521c008040 comp rx=0 tx=0).stop 2024-03-24T00:50:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.174+0000 7f521be00700 1 -- 172.21.15.38:0/3764446092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5230107130 msgr2=0x7f523019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:50:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.174+0000 7f521be00700 1 --2- 172.21.15.38:0/3764446092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5230107130 0x7f523019cc40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5220000c00 tx=0x7f522000be20 comp rx=0 tx=0).stop 2024-03-24T00:50:46.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.175+0000 7f521be00700 1 -- 172.21.15.38:0/3764446092 shutdown_connections 2024-03-24T00:50:46.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.175+0000 7f521be00700 1 --2- 172.21.15.38:0/3764446092 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f521406dda0 0x7f5214070260 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:46.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.175+0000 7f521be00700 1 --2- 172.21.15.38:0/3764446092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5230107130 0x7f523019cc40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:46.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.175+0000 7f521be00700 1 --2- 172.21.15.38:0/3764446092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5230108300 0x7f523019d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:50:46.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.175+0000 7f521be00700 1 -- 172.21.15.38:0/3764446092 >> 172.21.15.38:0/3764446092 conn(0x7f52300762d0 msgr2=0x7f523010b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:50:46.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.176+0000 7f521be00700 1 -- 172.21.15.38:0/3764446092 shutdown_connections 2024-03-24T00:50:46.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:50:46.176+0000 7f521be00700 1 -- 172.21.15.38:0/3764446092 wait complete. 2024-03-24T00:50:46.488 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:46 smithi038 ceph-mon[29271]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:46.488 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:46 smithi038 ceph-mon[29271]: pgmap v46: 65 pgs: 65 active+clean; 261 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 340 op/s 2024-03-24T00:50:46.488 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:46 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/111681070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:46 smithi088 ceph-mon[36610]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:46 smithi088 ceph-mon[36610]: pgmap v46: 65 pgs: 65 active+clean; 261 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 340 op/s 2024-03-24T00:50:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:46 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/111681070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:47.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:47 smithi038 ceph-mon[29271]: from='client.24503 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:47.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:47 smithi088 ceph-mon[36610]: from='client.24503 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:50:48.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:48 smithi038 ceph-mon[29271]: pgmap v47: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 290 op/s 2024-03-24T00:50:48.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:48 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:48.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:48 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:48 smithi088 ceph-mon[36610]: pgmap v47: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 290 op/s 2024-03-24T00:50:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:48 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:48 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:50.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:49 smithi038 ceph-mon[29271]: pgmap v48: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 994 KiB/s rd, 1.0 MiB/s wr, 256 op/s 2024-03-24T00:50:50.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:49 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:50:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:49 smithi088 ceph-mon[36610]: pgmap v48: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 994 KiB/s rd, 1.0 MiB/s wr, 256 op/s 2024-03-24T00:50:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:49 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:50:51.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:50 smithi038 ceph-mon[29271]: mgrmap e32: smithi038.huhioz(active, since 90s), standbys: smithi088.njctea 2024-03-24T00:50:51.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:50 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:51.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:50 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:50 smithi088 ceph-mon[36610]: mgrmap e32: smithi038.huhioz(active, since 90s), standbys: smithi088.njctea 2024-03-24T00:50:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:50 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:50 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:51.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:51 smithi038 ceph-mon[29271]: pgmap v49: 65 pgs: 65 active+clean; 263 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 801 op/s 2024-03-24T00:50:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:51 smithi088 ceph-mon[36610]: pgmap v49: 65 pgs: 65 active+clean; 263 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 801 op/s 2024-03-24T00:50:54.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:54 smithi038 ceph-mon[29271]: pgmap v50: 65 pgs: 65 active+clean; 263 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 978 KiB/s rd, 1.2 MiB/s wr, 672 op/s 2024-03-24T00:50:54.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:54 smithi088 ceph-mon[36610]: pgmap v50: 65 pgs: 65 active+clean; 263 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 978 KiB/s rd, 1.2 MiB/s wr, 672 op/s 2024-03-24T00:50:56.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:56 smithi038 ceph-mon[29271]: pgmap v51: 65 pgs: 65 active+clean; 251 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 765 op/s 2024-03-24T00:50:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:56 smithi088 ceph-mon[36610]: pgmap v51: 65 pgs: 65 active+clean; 251 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 765 op/s 2024-03-24T00:50:58.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:58 smithi038 ceph-mon[29271]: pgmap v52: 65 pgs: 65 active+clean; 252 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 771 op/s 2024-03-24T00:50:58.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:58 smithi088 ceph-mon[36610]: pgmap v52: 65 pgs: 65 active+clean; 252 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 771 op/s 2024-03-24T00:50:59.796 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:59.796 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:59.796 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:50:59.796 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:50:59.796 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:59.796 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:50:59.796 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.796 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.797 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.797 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.797 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.797 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.797 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.797 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.797 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:50:59 smithi038 ceph-mon[29271]: pgmap v53: 65 pgs: 65 active+clean; 252 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 711 op/s 2024-03-24T00:50:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:50:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:50:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:50:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:50:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:50:59.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:50:59 smithi088 ceph-mon[36610]: pgmap v53: 65 pgs: 65 active+clean; 252 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 711 op/s 2024-03-24T00:51:02.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:02 smithi038 ceph-mon[29271]: pgmap v54: 65 pgs: 65 active+clean; 242 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 1.29k op/s 2024-03-24T00:51:02.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:02 smithi038 ceph-mon[29271]: Upgrade: Updating prometheus.smithi038 2024-03-24T00:51:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:02 smithi088 ceph-mon[36610]: pgmap v54: 65 pgs: 65 active+clean; 242 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 1.29k op/s 2024-03-24T00:51:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:02 smithi088 ceph-mon[36610]: Upgrade: Updating prometheus.smithi038 2024-03-24T00:51:03.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:03 smithi038 ceph-mon[29271]: Deploying daemon prometheus.smithi038 on smithi038 2024-03-24T00:51:03.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:03 smithi088 ceph-mon[36610]: Deploying daemon prometheus.smithi038 on smithi038 2024-03-24T00:51:04.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:04 smithi038 ceph-mon[29271]: pgmap v55: 65 pgs: 65 active+clean; 242 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 747 op/s 2024-03-24T00:51:04.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:04 smithi088 ceph-mon[36610]: pgmap v55: 65 pgs: 65 active+clean; 242 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 747 op/s 2024-03-24T00:51:05.278 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:05 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:51:05.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:05 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:51:06.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:06 smithi038 ceph-mon[29271]: pgmap v56: 65 pgs: 65 active+clean; 232 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 939 op/s 2024-03-24T00:51:06.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:06 smithi088 ceph-mon[36610]: pgmap v56: 65 pgs: 65 active+clean; 232 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 939 op/s 2024-03-24T00:51:08.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:08 smithi038 ceph-mon[29271]: pgmap v57: 65 pgs: 65 active+clean; 234 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 937 op/s 2024-03-24T00:51:08.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:08 smithi088 ceph-mon[36610]: pgmap v57: 65 pgs: 65 active+clean; 234 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 937 op/s 2024-03-24T00:51:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:09 smithi038 ceph-mon[29271]: pgmap v58: 65 pgs: 65 active+clean; 234 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 961 KiB/s rd, 1.1 MiB/s wr, 863 op/s 2024-03-24T00:51:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:09 smithi088 ceph-mon[36610]: pgmap v58: 65 pgs: 65 active+clean; 234 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 961 KiB/s rd, 1.1 MiB/s wr, 863 op/s 2024-03-24T00:51:11.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:11 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:11.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:11 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:11.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:11 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:51:11.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:11 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:11.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:11 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:11.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:11 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:51:12.227 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-24T00:51:12.227 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-24T00:51:12.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:12 smithi038 ceph-mon[29271]: pgmap v59: 65 pgs: 65 active+clean; 226 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 1.14k op/s 2024-03-24T00:51:12.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:12 smithi088 ceph-mon[36610]: pgmap v59: 65 pgs: 65 active+clean; 226 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 1.14k op/s 2024-03-24T00:51:13.361 DEBUG:teuthology.parallel:result is None 2024-03-24T00:51:14.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:14 smithi038 ceph-mon[29271]: pgmap v60: 65 pgs: 65 active+clean; 226 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 555 op/s 2024-03-24T00:51:14.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:14 smithi088 ceph-mon[36610]: pgmap v60: 65 pgs: 65 active+clean; 226 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 555 op/s 2024-03-24T00:51:15.039 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-24T00:51:15.039 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-24T00:51:16.259 DEBUG:teuthology.parallel:result is None 2024-03-24T00:51:16.259 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-24T00:51:16.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.343+0000 7fa2f0e00700 1 -- 172.21.15.38:0/2291554618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec100ad0 msgr2=0x7fa2ec100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:16.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.343+0000 7fa2f0e00700 1 --2- 172.21.15.38:0/2291554618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec100ad0 0x7fa2ec100ef0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009910 tx=0x7fa2dc009c20 comp rx=0 tx=0).stop 2024-03-24T00:51:16.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.343+0000 7fa2f0e00700 1 -- 172.21.15.38:0/2291554618 shutdown_connections 2024-03-24T00:51:16.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.343+0000 7fa2f0e00700 1 --2- 172.21.15.38:0/2291554618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ec101cd0 0x7fa2ec102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:16.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.343+0000 7fa2f0e00700 1 --2- 172.21.15.38:0/2291554618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec100ad0 0x7fa2ec100ef0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:16.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.343+0000 7fa2f0e00700 1 -- 172.21.15.38:0/2291554618 >> 172.21.15.38:0/2291554618 conn(0x7fa2ec0fc090 msgr2=0x7fa2ec0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:16.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.343+0000 7fa2f0e00700 1 -- 172.21.15.38:0/2291554618 shutdown_connections 2024-03-24T00:51:16.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.344+0000 7fa2f0e00700 1 -- 172.21.15.38:0/2291554618 wait complete. 2024-03-24T00:51:16.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.345+0000 7fa2f0e00700 1 Processor -- start 2024-03-24T00:51:16.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.345+0000 7fa2f0e00700 1 -- start start 2024-03-24T00:51:16.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.346+0000 7fa2f0e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec100ad0 0x7fa2ec1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:16.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.346+0000 7fa2f0e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ec101cd0 0x7fa2ec196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:16.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.346+0000 7fa2f0e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec197050 con 0x7fa2ec100ad0 2024-03-24T00:51:16.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.346+0000 7fa2f0e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec197190 con 0x7fa2ec101cd0 2024-03-24T00:51:16.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.346+0000 7fa2e9e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec100ad0 0x7fa2ec1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:16.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.347+0000 7fa2e9e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec100ad0 0x7fa2ec1964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52818/0 (socket says 172.21.15.38:52818) 2024-03-24T00:51:16.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.347+0000 7fa2e9e00700 1 -- 172.21.15.38:0/4091714559 learned_addr learned my addr 172.21.15.38:0/4091714559 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:51:16.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.347+0000 7fa2e9400700 1 --2- 172.21.15.38:0/4091714559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ec101cd0 0x7fa2ec196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:16.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.347+0000 7fa2e9e00700 1 -- 172.21.15.38:0/4091714559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ec101cd0 msgr2=0x7fa2ec196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:16.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.347+0000 7fa2e9e00700 1 --2- 172.21.15.38:0/4091714559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ec101cd0 0x7fa2ec196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:16.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.347+0000 7fa2e9e00700 1 -- 172.21.15.38:0/4091714559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc0095f0 con 0x7fa2ec100ad0 2024-03-24T00:51:16.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.348+0000 7fa2e9e00700 1 --2- 172.21.15.38:0/4091714559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec100ad0 0x7fa2ec1964f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00bd70 tx=0x7fa2dc00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:16.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.348+0000 7fa2e2a00700 1 -- 172.21.15.38:0/4091714559 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2dc01d070 con 0x7fa2ec100ad0 2024-03-24T00:51:16.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.348+0000 7fa2e2a00700 1 -- 172.21.15.38:0/4091714559 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2dc00f460 con 0x7fa2ec100ad0 2024-03-24T00:51:16.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.348+0000 7fa2e2a00700 1 -- 172.21.15.38:0/4091714559 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2dc021640 con 0x7fa2ec100ad0 2024-03-24T00:51:16.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.348+0000 7fa2f0e00700 1 -- 172.21.15.38:0/4091714559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec072fe0 con 0x7fa2ec100ad0 2024-03-24T00:51:16.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.348+0000 7fa2f0e00700 1 -- 172.21.15.38:0/4091714559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec073500 con 0x7fa2ec100ad0 2024-03-24T00:51:16.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.350+0000 7fa2f0e00700 1 -- 172.21.15.38:0/4091714559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec190650 con 0x7fa2ec100ad0 2024-03-24T00:51:16.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.351+0000 7fa2e2a00700 1 -- 172.21.15.38:0/4091714559 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7fa2dc00fad0 con 0x7fa2ec100ad0 2024-03-24T00:51:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.352+0000 7fa2e2a00700 1 --2- 172.21.15.38:0/4091714559 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa2d006dc30 0x7fa2d00700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.353+0000 7fa2e2a00700 1 -- 172.21.15.38:0/4091714559 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa2dc08eb80 con 0x7fa2ec100ad0 2024-03-24T00:51:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.353+0000 7fa2e9400700 1 --2- 172.21.15.38:0/4091714559 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa2d006dc30 0x7fa2d00700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:16.355 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-24T00:51:16.355 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-24T00:51:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.354+0000 7fa2e9400700 1 --2- 172.21.15.38:0/4091714559 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa2d006dc30 0x7fa2d00700f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8005950 tx=0x7fa2d800a300 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:16.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.358+0000 7fa2e2a00700 1 -- 172.21.15.38:0/4091714559 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fa2dc05c0a0 con 0x7fa2ec100ad0 2024-03-24T00:51:16.453 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-24T00:51:16.453 DEBUG:teuthology.parallel:result is None 2024-03-24T00:51:16.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:16 smithi038 ceph-mon[29271]: pgmap v61: 65 pgs: 65 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 665 op/s 2024-03-24T00:51:16.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.623+0000 7fa2f0e00700 1 -- 172.21.15.38:0/4091714559 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2ec0611f0 con 0x7fa2d006dc30 2024-03-24T00:51:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.625+0000 7fa2e2a00700 1 -- 172.21.15.38:0/4091714559 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fa2ec0611f0 con 0x7fa2d006dc30 2024-03-24T00:51:16.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.628+0000 7fa2f0e00700 1 -- 172.21.15.38:0/4091714559 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa2d006dc30 msgr2=0x7fa2d00700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:16.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.628+0000 7fa2f0e00700 1 --2- 172.21.15.38:0/4091714559 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa2d006dc30 0x7fa2d00700f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8005950 tx=0x7fa2d800a300 comp rx=0 tx=0).stop 2024-03-24T00:51:16.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.628+0000 7fa2f0e00700 1 -- 172.21.15.38:0/4091714559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec100ad0 msgr2=0x7fa2ec1964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:16.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.628+0000 7fa2f0e00700 1 --2- 172.21.15.38:0/4091714559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec100ad0 0x7fa2ec1964f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00bd70 tx=0x7fa2dc00bda0 comp rx=0 tx=0).stop 2024-03-24T00:51:16.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.628+0000 7fa2f0e00700 1 -- 172.21.15.38:0/4091714559 shutdown_connections 2024-03-24T00:51:16.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.628+0000 7fa2f0e00700 1 --2- 172.21.15.38:0/4091714559 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fa2d006dc30 0x7fa2d00700f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:16.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.628+0000 7fa2f0e00700 1 --2- 172.21.15.38:0/4091714559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ec100ad0 0x7fa2ec1964f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:16.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.628+0000 7fa2f0e00700 1 --2- 172.21.15.38:0/4091714559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2ec101cd0 0x7fa2ec196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:16.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.628+0000 7fa2f0e00700 1 -- 172.21.15.38:0/4091714559 >> 172.21.15.38:0/4091714559 conn(0x7fa2ec0fc090 msgr2=0x7fa2ec1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:16.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.629+0000 7fa2f0e00700 1 -- 172.21.15.38:0/4091714559 shutdown_connections 2024-03-24T00:51:16.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.629+0000 7fa2f0e00700 1 -- 172.21.15.38:0/4091714559 wait complete. 2024-03-24T00:51:16.646 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:51:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:16 smithi088 ceph-mon[36610]: pgmap v61: 65 pgs: 65 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 665 op/s 2024-03-24T00:51:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.792+0000 7f48ce000700 1 -- 172.21.15.38:0/2013589087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c81013c0 msgr2=0x7f48c81037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.792+0000 7f48ce000700 1 --2- 172.21.15.38:0/2013589087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c81013c0 0x7f48c81037b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009910 tx=0x7f48bc009c20 comp rx=0 tx=0).stop 2024-03-24T00:51:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.792+0000 7f48ce000700 1 -- 172.21.15.38:0/2013589087 shutdown_connections 2024-03-24T00:51:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.792+0000 7f48ce000700 1 --2- 172.21.15.38:0/2013589087 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48c8103cf0 0x7f48c81060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.792+0000 7f48ce000700 1 --2- 172.21.15.38:0/2013589087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c81013c0 0x7f48c81037b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.792+0000 7f48ce000700 1 -- 172.21.15.38:0/2013589087 >> 172.21.15.38:0/2013589087 conn(0x7f48c80fadf0 msgr2=0x7f48c80fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.792+0000 7f48ce000700 1 -- 172.21.15.38:0/2013589087 shutdown_connections 2024-03-24T00:51:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.792+0000 7f48ce000700 1 -- 172.21.15.38:0/2013589087 wait complete. 2024-03-24T00:51:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.793+0000 7f48ce000700 1 Processor -- start 2024-03-24T00:51:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.793+0000 7f48ce000700 1 -- start start 2024-03-24T00:51:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.793+0000 7f48ce000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c81013c0 0x7f48c8100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.793+0000 7f48ce000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48c8103cf0 0x7f48c80ff370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.794+0000 7f48ce000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80ff8b0 con 0x7f48c81013c0 2024-03-24T00:51:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.794+0000 7f48ce000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80ff9f0 con 0x7f48c8103cf0 2024-03-24T00:51:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.794+0000 7f48ccc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c81013c0 0x7f48c8100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.794+0000 7f48ccc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c81013c0 0x7f48c8100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52840/0 (socket says 172.21.15.38:52840) 2024-03-24T00:51:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.794+0000 7f48ccc00700 1 -- 172.21.15.38:0/1110158764 learned_addr learned my addr 172.21.15.38:0/1110158764 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:51:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.794+0000 7f48c7e00700 1 --2- 172.21.15.38:0/1110158764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48c8103cf0 0x7f48c80ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.794+0000 7f48ccc00700 1 -- 172.21.15.38:0/1110158764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48c8103cf0 msgr2=0x7f48c80ff370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.794+0000 7f48ccc00700 1 --2- 172.21.15.38:0/1110158764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48c8103cf0 0x7f48c80ff370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.794+0000 7f48ccc00700 1 -- 172.21.15.38:0/1110158764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc0095f0 con 0x7f48c81013c0 2024-03-24T00:51:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.795+0000 7f48ccc00700 1 --2- 172.21.15.38:0/1110158764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c81013c0 0x7f48c8100cd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00bd70 tx=0x7f48bc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.795+0000 7f48c5600700 1 -- 172.21.15.38:0/1110158764 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48bc01d070 con 0x7f48c81013c0 2024-03-24T00:51:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.795+0000 7f48c5600700 1 -- 172.21.15.38:0/1110158764 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48bc00f460 con 0x7f48c81013c0 2024-03-24T00:51:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.795+0000 7f48c5600700 1 -- 172.21.15.38:0/1110158764 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48bc021620 con 0x7f48c81013c0 2024-03-24T00:51:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.795+0000 7f48ce000700 1 -- 172.21.15.38:0/1110158764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c80ffc70 con 0x7f48c81013c0 2024-03-24T00:51:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.795+0000 7f48ce000700 1 -- 172.21.15.38:0/1110158764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c81001e0 con 0x7f48c81013c0 2024-03-24T00:51:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.796+0000 7f48ce000700 1 -- 172.21.15.38:0/1110158764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c804f2d0 con 0x7f48c81013c0 2024-03-24T00:51:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.797+0000 7f48c5600700 1 -- 172.21.15.38:0/1110158764 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f48bc00fa70 con 0x7f48c81013c0 2024-03-24T00:51:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.798+0000 7f48c5600700 1 --2- 172.21.15.38:0/1110158764 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f48b006dc30 0x7f48b00700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.798+0000 7f48c5600700 1 -- 172.21.15.38:0/1110158764 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f48bc08e9a0 con 0x7f48c81013c0 2024-03-24T00:51:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.799+0000 7f48c7e00700 1 --2- 172.21.15.38:0/1110158764 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f48b006dc30 0x7f48b00700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:16.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.799+0000 7f48c7e00700 1 --2- 172.21.15.38:0/1110158764 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f48b006dc30 0x7f48b00700f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f48b4005950 tx=0x7f48b4005cf0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:16.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:16.803+0000 7f48c5600700 1 -- 172.21.15.38:0/1110158764 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f48bc05be00 con 0x7f48c81013c0 2024-03-24T00:51:17.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.063+0000 7f48ce000700 1 -- 172.21.15.38:0/1110158764 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48c819e380 con 0x7f48b006dc30 2024-03-24T00:51:17.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.064+0000 7f48c5600700 1 -- 172.21.15.38:0/1110158764 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f48c819e380 con 0x7f48b006dc30 2024-03-24T00:51:17.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.068+0000 7f48ce000700 1 -- 172.21.15.38:0/1110158764 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f48b006dc30 msgr2=0x7f48b00700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:17.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.068+0000 7f48ce000700 1 --2- 172.21.15.38:0/1110158764 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f48b006dc30 0x7f48b00700f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f48b4005950 tx=0x7f48b4005cf0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.068+0000 7f48ce000700 1 -- 172.21.15.38:0/1110158764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c81013c0 msgr2=0x7f48c8100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:17.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.068+0000 7f48ce000700 1 --2- 172.21.15.38:0/1110158764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c81013c0 0x7f48c8100cd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00bd70 tx=0x7f48bc00be50 comp rx=0 tx=0).stop 2024-03-24T00:51:17.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.069+0000 7f48ce000700 1 -- 172.21.15.38:0/1110158764 shutdown_connections 2024-03-24T00:51:17.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.069+0000 7f48ce000700 1 --2- 172.21.15.38:0/1110158764 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f48b006dc30 0x7f48b00700f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.069+0000 7f48ce000700 1 --2- 172.21.15.38:0/1110158764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c81013c0 0x7f48c8100cd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.069+0000 7f48ce000700 1 --2- 172.21.15.38:0/1110158764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48c8103cf0 0x7f48c80ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.069+0000 7f48ce000700 1 -- 172.21.15.38:0/1110158764 >> 172.21.15.38:0/1110158764 conn(0x7f48c80fadf0 msgr2=0x7f48c80fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:17.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.069+0000 7f48ce000700 1 -- 172.21.15.38:0/1110158764 shutdown_connections 2024-03-24T00:51:17.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.069+0000 7f48ce000700 1 -- 172.21.15.38:0/1110158764 wait complete. 2024-03-24T00:51:17.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.221+0000 7f1a4b200700 1 -- 172.21.15.38:0/4103533485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a4c101430 msgr2=0x7f1a4c103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:17.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.221+0000 7f1a4b200700 1 --2- 172.21.15.38:0/4103533485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a4c101430 0x7f1a4c103820 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009910 tx=0x7f1a3c009c20 comp rx=0 tx=0).stop 2024-03-24T00:51:17.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.221+0000 7f1a4b200700 1 -- 172.21.15.38:0/4103533485 shutdown_connections 2024-03-24T00:51:17.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.221+0000 7f1a4b200700 1 --2- 172.21.15.38:0/4103533485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4c103d60 0x7f1a4c106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.221+0000 7f1a4b200700 1 --2- 172.21.15.38:0/4103533485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a4c101430 0x7f1a4c103820 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.221+0000 7f1a4b200700 1 -- 172.21.15.38:0/4103533485 >> 172.21.15.38:0/4103533485 conn(0x7f1a4c0fade0 msgr2=0x7f1a4c0fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:17.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.222+0000 7f1a4b200700 1 -- 172.21.15.38:0/4103533485 shutdown_connections 2024-03-24T00:51:17.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.222+0000 7f1a4b200700 1 -- 172.21.15.38:0/4103533485 wait complete. 2024-03-24T00:51:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.223+0000 7f1a4b200700 1 Processor -- start 2024-03-24T00:51:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.223+0000 7f1a4b200700 1 -- start start 2024-03-24T00:51:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.223+0000 7f1a4b200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4c101430 0x7f1a4c100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.223+0000 7f1a4b200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a4c103d60 0x7f1a4c0ff3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.223+0000 7f1a4b200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0ff920 con 0x7f1a4c103d60 2024-03-24T00:51:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.223+0000 7f1a4b200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0ffa60 con 0x7f1a4c101430 2024-03-24T00:51:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.224+0000 7f1a49400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a4c103d60 0x7f1a4c0ff3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.224+0000 7f1a49400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a4c103d60 0x7f1a4c0ff3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52854/0 (socket says 172.21.15.38:52854) 2024-03-24T00:51:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.224+0000 7f1a49400700 1 -- 172.21.15.38:0/2252949285 learned_addr learned my addr 172.21.15.38:0/2252949285 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:51:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.224+0000 7f1a49e00700 1 --2- 172.21.15.38:0/2252949285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4c101430 0x7f1a4c100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:17.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.224+0000 7f1a49400700 1 -- 172.21.15.38:0/2252949285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4c101430 msgr2=0x7f1a4c100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.224+0000 7f1a49400700 1 --2- 172.21.15.38:0/2252949285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4c101430 0x7f1a4c100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.224+0000 7f1a49400700 1 -- 172.21.15.38:0/2252949285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c0095f0 con 0x7f1a4c103d60 2024-03-24T00:51:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.225+0000 7f1a49400700 1 --2- 172.21.15.38:0/2252949285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a4c103d60 0x7f1a4c0ff3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800b4b0 tx=0x7f1a3800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.225+0000 7f1a42a00700 1 -- 172.21.15.38:0/2252949285 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a38011800 con 0x7f1a4c103d60 2024-03-24T00:51:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.225+0000 7f1a4b200700 1 -- 172.21.15.38:0/2252949285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c0ffd40 con 0x7f1a4c103d60 2024-03-24T00:51:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.225+0000 7f1a42a00700 1 -- 172.21.15.38:0/2252949285 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a38011e20 con 0x7f1a4c103d60 2024-03-24T00:51:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.225+0000 7f1a42a00700 1 -- 172.21.15.38:0/2252949285 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a3800f2e0 con 0x7f1a4c103d60 2024-03-24T00:51:17.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.225+0000 7f1a4b200700 1 -- 172.21.15.38:0/2252949285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c19e4a0 con 0x7f1a4c103d60 2024-03-24T00:51:17.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.227+0000 7f1a4b200700 1 -- 172.21.15.38:0/2252949285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4c04f2d0 con 0x7f1a4c103d60 2024-03-24T00:51:17.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.228+0000 7f1a42a00700 1 -- 172.21.15.38:0/2252949285 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f1a38010380 con 0x7f1a4c103d60 2024-03-24T00:51:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.229+0000 7f1a42a00700 1 --2- 172.21.15.38:0/2252949285 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f1a3006dc30 0x7f1a300700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.229+0000 7f1a42a00700 1 -- 172.21.15.38:0/2252949285 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1a3808d2f0 con 0x7f1a4c103d60 2024-03-24T00:51:17.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.230+0000 7f1a49e00700 1 --2- 172.21.15.38:0/2252949285 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f1a3006dc30 0x7f1a300700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:17.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.230+0000 7f1a49e00700 1 --2- 172.21.15.38:0/2252949285 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f1a3006dc30 0x7f1a300700f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c006010 tx=0x7f1a3c005180 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:17.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.234+0000 7f1a42a00700 1 -- 172.21.15.38:0/2252949285 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f1a3805a800 con 0x7f1a4c103d60 2024-03-24T00:51:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.475+0000 7f1a4b200700 1 -- 172.21.15.38:0/2252949285 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a4c19e630 con 0x7f1a3006dc30 2024-03-24T00:51:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.493+0000 7f1a42a00700 1 -- 172.21.15.38:0/2252949285 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1a4c19e630 con 0x7f1a3006dc30 2024-03-24T00:51:17.494 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:51:17.494 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (11m) 29s ago 13m 14.6M - 0.25.0 c8568f914cd2 9f1243fbcb3e 2024-03-24T00:51:17.494 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (13m) 29s ago 13m 8140k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:51:17.494 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (12m) 40s ago 12m 7922k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:51:17.494 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (13m) 29s ago 13m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:51:17.494 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (12m) 40s ago 12m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:51:17.494 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (11m) 29s ago 13m 75.2M - 9.4.7 954c08fa6188 e6aa268d69e8 2024-03-24T00:51:17.494 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (6m) 29s ago 6m 136M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:51:17.495 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (6m) 29s ago 6m 208M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:51:17.495 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (6m) 40s ago 6m 24.4M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:51:17.495 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (6m) 40s ago 6m 24.3M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:51:17.495 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (2m) 29s ago 15m 496M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:51:17.495 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (74s) 40s ago 12m 447M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:51:17.495 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (15m) 29s ago 15m 54.8M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:51:17.495 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (12m) 40s ago 12m 44.5M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:51:17.495 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (52s) 29s ago 13m 6727k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:51:17.495 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (46s) 40s ago 12m 6753k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:51:17.495 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (11m) 29s ago 11m 221M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:51:17.496 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (10m) 29s ago 10m 244M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:51:17.496 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (9m) 29s ago 9m 195M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:51:17.496 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (9m) 40s ago 9m 274M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:51:17.496 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (8m) 40s ago 8m 258M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:51:17.496 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (8m) 40s ago 8m 222M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:51:17.496 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 starting - - - - 2024-03-24T00:51:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.496+0000 7f1a4b200700 1 -- 172.21.15.38:0/2252949285 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f1a3006dc30 msgr2=0x7f1a300700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.496+0000 7f1a4b200700 1 --2- 172.21.15.38:0/2252949285 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f1a3006dc30 0x7f1a300700f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c006010 tx=0x7f1a3c005180 comp rx=0 tx=0).stop 2024-03-24T00:51:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.496+0000 7f1a4b200700 1 -- 172.21.15.38:0/2252949285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a4c103d60 msgr2=0x7f1a4c0ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.496+0000 7f1a4b200700 1 --2- 172.21.15.38:0/2252949285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a4c103d60 0x7f1a4c0ff3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800b4b0 tx=0x7f1a3800b7c0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.496+0000 7f1a4b200700 1 -- 172.21.15.38:0/2252949285 shutdown_connections 2024-03-24T00:51:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.496+0000 7f1a4b200700 1 --2- 172.21.15.38:0/2252949285 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f1a3006dc30 0x7f1a300700f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.496+0000 7f1a4b200700 1 --2- 172.21.15.38:0/2252949285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4c101430 0x7f1a4c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.496+0000 7f1a4b200700 1 --2- 172.21.15.38:0/2252949285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a4c103d60 0x7f1a4c0ff3e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.496+0000 7f1a4b200700 1 -- 172.21.15.38:0/2252949285 >> 172.21.15.38:0/2252949285 conn(0x7f1a4c0fade0 msgr2=0x7f1a4c0fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:17.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.497+0000 7f1a4b200700 1 -- 172.21.15.38:0/2252949285 shutdown_connections 2024-03-24T00:51:17.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.497+0000 7f1a4b200700 1 -- 172.21.15.38:0/2252949285 wait complete. 2024-03-24T00:51:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.654+0000 7f459fe00700 1 -- 172.21.15.38:0/3094645965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4598101560 msgr2=0x7f4598103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.654+0000 7f459fe00700 1 --2- 172.21.15.38:0/3094645965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4598101560 0x7f4598103950 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f458c009910 tx=0x7f458c009c20 comp rx=0 tx=0).stop 2024-03-24T00:51:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.654+0000 7f459fe00700 1 -- 172.21.15.38:0/3094645965 shutdown_connections 2024-03-24T00:51:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.654+0000 7f459fe00700 1 --2- 172.21.15.38:0/3094645965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4598103e90 0x7f4598106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.654+0000 7f459fe00700 1 --2- 172.21.15.38:0/3094645965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4598101560 0x7f4598103950 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.654+0000 7f459fe00700 1 -- 172.21.15.38:0/3094645965 >> 172.21.15.38:0/3094645965 conn(0x7f45980faf70 msgr2=0x7f45980fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.655+0000 7f459fe00700 1 -- 172.21.15.38:0/3094645965 shutdown_connections 2024-03-24T00:51:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.655+0000 7f459fe00700 1 -- 172.21.15.38:0/3094645965 wait complete. 2024-03-24T00:51:17.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.656+0000 7f459fe00700 1 Processor -- start 2024-03-24T00:51:17.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.656+0000 7f459fe00700 1 -- start start 2024-03-24T00:51:17.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.656+0000 7f459fe00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4598101560 0x7f4598100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:17.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.656+0000 7f459fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4598103e90 0x7f45980ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:17.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.656+0000 7f459fe00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4598101400 con 0x7f4598103e90 2024-03-24T00:51:17.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.656+0000 7f459fe00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45980ffa50 con 0x7f4598101560 2024-03-24T00:51:17.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.657+0000 7f459ca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4598103e90 0x7f45980ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:17.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.657+0000 7f459ca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4598103e90 0x7f45980ff510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52872/0 (socket says 172.21.15.38:52872) 2024-03-24T00:51:17.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.657+0000 7f459ca00700 1 -- 172.21.15.38:0/989700849 learned_addr learned my addr 172.21.15.38:0/989700849 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:51:17.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.657+0000 7f459d400700 1 --2- 172.21.15.38:0/989700849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4598101560 0x7f4598100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:17.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.657+0000 7f459ca00700 1 -- 172.21.15.38:0/989700849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4598101560 msgr2=0x7f4598100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:17.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.657+0000 7f459ca00700 1 --2- 172.21.15.38:0/989700849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4598101560 0x7f4598100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:17.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.657+0000 7f459ca00700 1 -- 172.21.15.38:0/989700849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c0095f0 con 0x7f4598103e90 2024-03-24T00:51:17.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.658+0000 7f459ca00700 1 --2- 172.21.15.38:0/989700849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4598103e90 0x7f45980ff510 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f458400c930 tx=0x7f458400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:17.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.658+0000 7f4596000700 1 -- 172.21.15.38:0/989700849 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4584007ad0 con 0x7f4598103e90 2024-03-24T00:51:17.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.658+0000 7f4596000700 1 -- 172.21.15.38:0/989700849 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f458400ce80 con 0x7f4598103e90 2024-03-24T00:51:17.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.658+0000 7f4596000700 1 -- 172.21.15.38:0/989700849 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4584018630 con 0x7f4598103e90 2024-03-24T00:51:17.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.658+0000 7f459fe00700 1 -- 172.21.15.38:0/989700849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45980ffd30 con 0x7f4598103e90 2024-03-24T00:51:17.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.658+0000 7f459fe00700 1 -- 172.21.15.38:0/989700849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4598100300 con 0x7f4598103e90 2024-03-24T00:51:17.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.660+0000 7f459fe00700 1 -- 172.21.15.38:0/989700849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459804ea80 con 0x7f4598103e90 2024-03-24T00:51:17.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.660+0000 7f4596000700 1 -- 172.21.15.38:0/989700849 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f4584007c30 con 0x7f4598103e90 2024-03-24T00:51:17.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.661+0000 7f4596000700 1 --2- 172.21.15.38:0/989700849 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f458006dc30 0x7f45800700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:17.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.662+0000 7f4596000700 1 -- 172.21.15.38:0/989700849 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f458408d870 con 0x7f4598103e90 2024-03-24T00:51:17.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.662+0000 7f459d400700 1 --2- 172.21.15.38:0/989700849 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f458006dc30 0x7f45800700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:17.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.663+0000 7f459d400700 1 --2- 172.21.15.38:0/989700849 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f458006dc30 0x7f45800700f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f458c006010 tx=0x7f458c00b540 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:17.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:17.666+0000 7f4596000700 1 -- 172.21.15.38:0/989700849 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f458405ac50 con 0x7f4598103e90 2024-03-24T00:51:18.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.010+0000 7f459fe00700 1 -- 172.21.15.38:0/989700849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f459819e640 con 0x7f4598103e90 2024-03-24T00:51:18.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.011+0000 7f4596000700 1 -- 172.21.15.38:0/989700849 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f458405a7e0 con 0x7f4598103e90 2024-03-24T00:51:18.011 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:51:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:51:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:51:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:51:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:51:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 12, 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:51:18.012 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:51:18.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.014+0000 7f459fe00700 1 -- 172.21.15.38:0/989700849 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f458006dc30 msgr2=0x7f45800700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:18.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.014+0000 7f459fe00700 1 --2- 172.21.15.38:0/989700849 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f458006dc30 0x7f45800700f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f458c006010 tx=0x7f458c00b540 comp rx=0 tx=0).stop 2024-03-24T00:51:18.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.014+0000 7f459fe00700 1 -- 172.21.15.38:0/989700849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4598103e90 msgr2=0x7f45980ff510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:18.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.014+0000 7f459fe00700 1 --2- 172.21.15.38:0/989700849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4598103e90 0x7f45980ff510 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f458400c930 tx=0x7f458400cc40 comp rx=0 tx=0).stop 2024-03-24T00:51:18.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.014+0000 7f459fe00700 1 -- 172.21.15.38:0/989700849 shutdown_connections 2024-03-24T00:51:18.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.014+0000 7f459fe00700 1 --2- 172.21.15.38:0/989700849 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f458006dc30 0x7f45800700f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:18.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.014+0000 7f459fe00700 1 --2- 172.21.15.38:0/989700849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4598101560 0x7f4598100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:18.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.014+0000 7f459fe00700 1 --2- 172.21.15.38:0/989700849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4598103e90 0x7f45980ff510 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:18.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.014+0000 7f459fe00700 1 -- 172.21.15.38:0/989700849 >> 172.21.15.38:0/989700849 conn(0x7f45980faf70 msgr2=0x7f45980fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:18.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.014+0000 7f459fe00700 1 -- 172.21.15.38:0/989700849 shutdown_connections 2024-03-24T00:51:18.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.014+0000 7f459fe00700 1 -- 172.21.15.38:0/989700849 wait complete. 2024-03-24T00:51:18.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.181+0000 7f3c73400700 1 -- 172.21.15.38:0/3454516567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c6c10a300 msgr2=0x7f3c6c10c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:18.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.181+0000 7f3c73400700 1 --2- 172.21.15.38:0/3454516567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c6c10a300 0x7f3c6c10c790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009910 tx=0x7f3c58009c20 comp rx=0 tx=0).stop 2024-03-24T00:51:18.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.181+0000 7f3c73400700 1 -- 172.21.15.38:0/3454516567 shutdown_connections 2024-03-24T00:51:18.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.181+0000 7f3c73400700 1 --2- 172.21.15.38:0/3454516567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c6c10a300 0x7f3c6c10c790 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:18.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.181+0000 7f3c73400700 1 --2- 172.21.15.38:0/3454516567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c6c072ac0 0x7f3c6c109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:18.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.181+0000 7f3c73400700 1 -- 172.21.15.38:0/3454516567 >> 172.21.15.38:0/3454516567 conn(0x7f3c6c06da40 msgr2=0x7f3c6c06fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:18.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.182+0000 7f3c73400700 1 -- 172.21.15.38:0/3454516567 shutdown_connections 2024-03-24T00:51:18.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.182+0000 7f3c73400700 1 -- 172.21.15.38:0/3454516567 wait complete. 2024-03-24T00:51:18.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.183+0000 7f3c73400700 1 Processor -- start 2024-03-24T00:51:18.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.184+0000 7f3c73400700 1 -- start start 2024-03-24T00:51:18.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.184+0000 7f3c73400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c6c072ac0 0x7f3c6c1a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:18.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.184+0000 7f3c73400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c6c10a300 0x7f3c6c1a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:18.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.184+0000 7f3c73400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c1a5e30 con 0x7f3c6c10a300 2024-03-24T00:51:18.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.184+0000 7f3c73400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c1a5f70 con 0x7f3c6c072ac0 2024-03-24T00:51:18.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.185+0000 7f3c6be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c6c10a300 0x7f3c6c1a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:18.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.185+0000 7f3c6be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c6c10a300 0x7f3c6c1a5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52882/0 (socket says 172.21.15.38:52882) 2024-03-24T00:51:18.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.185+0000 7f3c70a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c6c072ac0 0x7f3c6c1a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:18.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.185+0000 7f3c6be00700 1 -- 172.21.15.38:0/2367564558 learned_addr learned my addr 172.21.15.38:0/2367564558 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:51:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.186+0000 7f3c6be00700 1 -- 172.21.15.38:0/2367564558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c6c072ac0 msgr2=0x7f3c6c1a52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.186+0000 7f3c6be00700 1 --2- 172.21.15.38:0/2367564558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c6c072ac0 0x7f3c6c1a52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.186+0000 7f3c6be00700 1 -- 172.21.15.38:0/2367564558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c580095f0 con 0x7f3c6c10a300 2024-03-24T00:51:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.186+0000 7f3c6be00700 1 --2- 172.21.15.38:0/2367564558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c6c10a300 0x7f3c6c1a5810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800b800 tx=0x7f3c5800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.187+0000 7f3c69600700 1 -- 172.21.15.38:0/2367564558 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c5801d070 con 0x7f3c6c10a300 2024-03-24T00:51:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.187+0000 7f3c69600700 1 -- 172.21.15.38:0/2367564558 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c580052a0 con 0x7f3c6c10a300 2024-03-24T00:51:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.187+0000 7f3c69600700 1 -- 172.21.15.38:0/2367564558 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c58017400 con 0x7f3c6c10a300 2024-03-24T00:51:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.187+0000 7f3c73400700 1 -- 172.21.15.38:0/2367564558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c1aa920 con 0x7f3c6c10a300 2024-03-24T00:51:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.187+0000 7f3c73400700 1 -- 172.21.15.38:0/2367564558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c1aae90 con 0x7f3c6c10a300 2024-03-24T00:51:18.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.189+0000 7f3c73400700 1 -- 172.21.15.38:0/2367564558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4c005320 con 0x7f3c6c10a300 2024-03-24T00:51:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.191+0000 7f3c69600700 1 -- 172.21.15.38:0/2367564558 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f3c58017670 con 0x7f3c6c10a300 2024-03-24T00:51:18.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.192+0000 7f3c69600700 1 --2- 172.21.15.38:0/2367564558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f3c5406dc80 0x7f3c54070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:18.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.193+0000 7f3c69600700 1 -- 172.21.15.38:0/2367564558 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3c5808eac0 con 0x7f3c6c10a300 2024-03-24T00:51:18.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.194+0000 7f3c70a00700 1 --2- 172.21.15.38:0/2367564558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f3c5406dc80 0x7f3c54070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:18.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.195+0000 7f3c70a00700 1 --2- 172.21.15.38:0/2367564558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f3c5406dc80 0x7f3c54070140 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009c60 tx=0x7f3c60009260 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.199+0000 7f3c69600700 1 -- 172.21.15.38:0/2367564558 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f3c5805bf20 con 0x7f3c6c10a300 2024-03-24T00:51:18.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:18 smithi038 ceph-mon[29271]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:18.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:18 smithi038 ceph-mon[29271]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:18.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:18 smithi038 ceph-mon[29271]: pgmap v62: 65 pgs: 65 active+clean; 213 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 517 op/s 2024-03-24T00:51:18.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:18 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/989700849' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:18.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:18.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:18 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:18.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.491+0000 7f3c73400700 1 -- 172.21.15.38:0/2367564558 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c4c000bf0 con 0x7f3c5406dc80 2024-03-24T00:51:18.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.493+0000 7f3c69600700 1 -- 172.21.15.38:0/2367564558 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f3c4c000bf0 con 0x7f3c5406dc80 2024-03-24T00:51:18.494 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:51:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:51:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:51:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-24T00:51:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:51:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-03-24T00:51:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:51:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "2/2 daemons upgraded", 2024-03-24T00:51:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading prometheus daemons", 2024-03-24T00:51:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:51:18.495 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:51:18.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.496+0000 7f3c73400700 1 -- 172.21.15.38:0/2367564558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f3c5406dc80 msgr2=0x7f3c54070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:18.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.496+0000 7f3c73400700 1 --2- 172.21.15.38:0/2367564558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f3c5406dc80 0x7f3c54070140 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009c60 tx=0x7f3c60009260 comp rx=0 tx=0).stop 2024-03-24T00:51:18.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.496+0000 7f3c73400700 1 -- 172.21.15.38:0/2367564558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c6c10a300 msgr2=0x7f3c6c1a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:18.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.496+0000 7f3c73400700 1 --2- 172.21.15.38:0/2367564558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c6c10a300 0x7f3c6c1a5810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800b800 tx=0x7f3c5800b610 comp rx=0 tx=0).stop 2024-03-24T00:51:18.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.496+0000 7f3c73400700 1 -- 172.21.15.38:0/2367564558 shutdown_connections 2024-03-24T00:51:18.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.496+0000 7f3c73400700 1 --2- 172.21.15.38:0/2367564558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f3c5406dc80 0x7f3c54070140 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:18.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.496+0000 7f3c73400700 1 --2- 172.21.15.38:0/2367564558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c6c072ac0 0x7f3c6c1a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:18.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.496+0000 7f3c73400700 1 --2- 172.21.15.38:0/2367564558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c6c10a300 0x7f3c6c1a5810 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:18.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.496+0000 7f3c73400700 1 -- 172.21.15.38:0/2367564558 >> 172.21.15.38:0/2367564558 conn(0x7f3c6c06da40 msgr2=0x7f3c6c10b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:18.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.497+0000 7f3c73400700 1 -- 172.21.15.38:0/2367564558 shutdown_connections 2024-03-24T00:51:18.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:18.497+0000 7f3c73400700 1 -- 172.21.15.38:0/2367564558 wait complete. 2024-03-24T00:51:18.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:18 smithi088 ceph-mon[36610]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:18.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:18 smithi088 ceph-mon[36610]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:18.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:18 smithi088 ceph-mon[36610]: pgmap v62: 65 pgs: 65 active+clean; 213 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 517 op/s 2024-03-24T00:51:18.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:18 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/989700849' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:18.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:18.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:18 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:19 smithi038 ceph-mon[29271]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:19.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:19 smithi088 ceph-mon[36610]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:20.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:20 smithi038 ceph-mon[29271]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:20.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:20 smithi038 ceph-mon[29271]: pgmap v63: 65 pgs: 65 active+clean; 213 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 427 op/s 2024-03-24T00:51:20.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:20 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:51:20.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:20 smithi088 ceph-mon[36610]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:20.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:20 smithi088 ceph-mon[36610]: pgmap v63: 65 pgs: 65 active+clean; 213 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 427 op/s 2024-03-24T00:51:20.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:20 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:51:21.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:21 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:21.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:21 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:21.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:21 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:21.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:21 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:22.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:22 smithi038 ceph-mon[29271]: pgmap v64: 65 pgs: 65 active+clean; 202 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 458 op/s 2024-03-24T00:51:22.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:22 smithi088 ceph-mon[36610]: pgmap v64: 65 pgs: 65 active+clean; 202 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 458 op/s 2024-03-24T00:51:24.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:24 smithi038 ceph-mon[29271]: pgmap v65: 65 pgs: 65 active+clean; 202 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 630 KiB/s rd, 560 KiB/s wr, 185 op/s 2024-03-24T00:51:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:24 smithi088 ceph-mon[36610]: pgmap v65: 65 pgs: 65 active+clean; 202 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 630 KiB/s rd, 560 KiB/s wr, 185 op/s 2024-03-24T00:51:26.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:26 smithi038 ceph-mon[29271]: pgmap v66: 65 pgs: 65 active+clean; 203 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 638 KiB/s rd, 567 KiB/s wr, 186 op/s 2024-03-24T00:51:26.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:26 smithi088 ceph-mon[36610]: pgmap v66: 65 pgs: 65 active+clean; 203 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 638 KiB/s rd, 567 KiB/s wr, 186 op/s 2024-03-24T00:51:28.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: pgmap v67: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 33 KiB/s rd, 33 KiB/s wr, 77 op/s 2024-03-24T00:51:28.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:28.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:28.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:51:28.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:51:28.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:28.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:51:28.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:51:28.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:28 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: pgmap v67: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 33 KiB/s rd, 33 KiB/s wr, 77 op/s 2024-03-24T00:51:28.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:28.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:28.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:51:28.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:51:28.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:28.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:51:28.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:51:28.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:28.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:28 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:29.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:29 smithi088 ceph-mon[36610]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:51:29.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:29 smithi088 ceph-mon[36610]: pgmap v68: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 20 KiB/s wr, 33 op/s 2024-03-24T00:51:29.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:29 smithi038 ceph-mon[29271]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T00:51:29.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:29 smithi038 ceph-mon[29271]: pgmap v68: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 20 KiB/s wr, 33 op/s 2024-03-24T00:51:31.104 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:30 smithi038 ceph-mon[29271]: Upgrade: Updating alertmanager.smithi038 2024-03-24T00:51:31.104 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:30 smithi038 ceph-mon[29271]: Deploying daemon alertmanager.smithi038 on smithi038 2024-03-24T00:51:31.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:30 smithi088 ceph-mon[36610]: Upgrade: Updating alertmanager.smithi038 2024-03-24T00:51:31.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:30 smithi088 ceph-mon[36610]: Deploying daemon alertmanager.smithi038 on smithi038 2024-03-24T00:51:32.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:31 smithi038 ceph-mon[29271]: pgmap v69: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 20 KiB/s wr, 35 op/s 2024-03-24T00:51:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:31 smithi088 ceph-mon[36610]: pgmap v69: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 20 KiB/s wr, 35 op/s 2024-03-24T00:51:34.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:34 smithi038 ceph-mon[29271]: pgmap v70: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8.9 KiB/s rd, 7.6 KiB/s wr, 4 op/s 2024-03-24T00:51:34.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:34 smithi088 ceph-mon[36610]: pgmap v70: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8.9 KiB/s rd, 7.6 KiB/s wr, 4 op/s 2024-03-24T00:51:35.594 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:51:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:51:36.579 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:36 smithi038 ceph-mon[29271]: pgmap v71: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 9.7 KiB/s rd, 7.6 KiB/s wr, 5 op/s 2024-03-24T00:51:36.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:36 smithi088 ceph-mon[36610]: pgmap v71: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 9.7 KiB/s rd, 7.6 KiB/s wr, 5 op/s 2024-03-24T00:51:38.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:38 smithi038 ceph-mon[29271]: pgmap v72: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-24T00:51:38.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:38 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:38.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:38 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:38.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:38 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:51:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:38 smithi088 ceph-mon[36610]: pgmap v72: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-24T00:51:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:38 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:38 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:38 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:51:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:39 smithi038 ceph-mon[29271]: pgmap v73: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-03-24T00:51:40.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:39 smithi088 ceph-mon[36610]: pgmap v73: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-03-24T00:51:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:42 smithi038 ceph-mon[29271]: pgmap v74: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-24T00:51:42.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:42 smithi088 ceph-mon[36610]: pgmap v74: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-24T00:51:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:44 smithi038 ceph-mon[29271]: pgmap v75: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:51:44.625 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:44 smithi088 ceph-mon[36610]: pgmap v75: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:51:46.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:46 smithi038 ceph-mon[29271]: pgmap v76: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:51:46.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:46 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:46.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:46 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:46 smithi088 ceph-mon[36610]: pgmap v76: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:51:46.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:46 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:46.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:46 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:48.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:48 smithi038 ceph-mon[29271]: pgmap v77: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:51:48.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:48 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:48.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:48 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.673+0000 7f0b6f600700 1 -- 172.21.15.38:0/756428464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b68100be0 msgr2=0x7f0b68101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.673+0000 7f0b6f600700 1 --2- 172.21.15.38:0/756428464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b68100be0 0x7f0b68101040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009910 tx=0x7f0b54009c20 comp rx=0 tx=0).stop 2024-03-24T00:51:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.673+0000 7f0b6f600700 1 -- 172.21.15.38:0/756428464 shutdown_connections 2024-03-24T00:51:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.673+0000 7f0b6f600700 1 --2- 172.21.15.38:0/756428464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b68100be0 0x7f0b68101040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.673+0000 7f0b6f600700 1 --2- 172.21.15.38:0/756428464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b680ff9e0 0x7f0b680ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.673+0000 7f0b6f600700 1 -- 172.21.15.38:0/756428464 >> 172.21.15.38:0/756428464 conn(0x7f0b680faf80 msgr2=0x7f0b680fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.673+0000 7f0b6f600700 1 -- 172.21.15.38:0/756428464 shutdown_connections 2024-03-24T00:51:48.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.673+0000 7f0b6f600700 1 -- 172.21.15.38:0/756428464 wait complete. 2024-03-24T00:51:48.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.675+0000 7f0b6f600700 1 Processor -- start 2024-03-24T00:51:48.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.675+0000 7f0b6f600700 1 -- start start 2024-03-24T00:51:48.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.676+0000 7f0b6f600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b680ff9e0 0x7f0b6819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:48.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.676+0000 7f0b6f600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b68100be0 0x7f0b6819d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:48.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.676+0000 7f0b6f600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6819d680 con 0x7f0b680ff9e0 2024-03-24T00:51:48.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.676+0000 7f0b6f600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6819d7c0 con 0x7f0b68100be0 2024-03-24T00:51:48.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.676+0000 7f0b6cc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b680ff9e0 0x7f0b6819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:48.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.676+0000 7f0b6cc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b680ff9e0 0x7f0b6819cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44152/0 (socket says 172.21.15.38:44152) 2024-03-24T00:51:48.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.676+0000 7f0b6cc00700 1 -- 172.21.15.38:0/1507878993 learned_addr learned my addr 172.21.15.38:0/1507878993 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:51:48.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.677+0000 7f0b67e00700 1 --2- 172.21.15.38:0/1507878993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b68100be0 0x7f0b6819d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:48.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.677+0000 7f0b6cc00700 1 -- 172.21.15.38:0/1507878993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b68100be0 msgr2=0x7f0b6819d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:48.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.677+0000 7f0b6cc00700 1 --2- 172.21.15.38:0/1507878993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b68100be0 0x7f0b6819d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:48.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.677+0000 7f0b6cc00700 1 -- 172.21.15.38:0/1507878993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540095f0 con 0x7f0b680ff9e0 2024-03-24T00:51:48.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.678+0000 7f0b6cc00700 1 --2- 172.21.15.38:0/1507878993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b680ff9e0 0x7f0b6819cb20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00e8f0 tx=0x7f0b5c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:48.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.678+0000 7f0b65600700 1 -- 172.21.15.38:0/1507878993 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b5c009790 con 0x7f0b680ff9e0 2024-03-24T00:51:48.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.678+0000 7f0b65600700 1 -- 172.21.15.38:0/1507878993 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b5c00c810 con 0x7f0b680ff9e0 2024-03-24T00:51:48.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.678+0000 7f0b6f600700 1 -- 172.21.15.38:0/1507878993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b681a2220 con 0x7f0b680ff9e0 2024-03-24T00:51:48.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.678+0000 7f0b65600700 1 -- 172.21.15.38:0/1507878993 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b5c003d20 con 0x7f0b680ff9e0 2024-03-24T00:51:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:48 smithi088 ceph-mon[36610]: pgmap v77: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:51:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:48 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:48 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:48.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.678+0000 7f0b6f600700 1 -- 172.21.15.38:0/1507878993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b681a27f0 con 0x7f0b680ff9e0 2024-03-24T00:51:48.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.680+0000 7f0b6f600700 1 -- 172.21.15.38:0/1507878993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6804ea80 con 0x7f0b680ff9e0 2024-03-24T00:51:48.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.681+0000 7f0b65600700 1 -- 172.21.15.38:0/1507878993 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f0b5c0041f0 con 0x7f0b680ff9e0 2024-03-24T00:51:48.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.682+0000 7f0b65600700 1 --2- 172.21.15.38:0/1507878993 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f0b5006dc20 0x7f0b500700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:48.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.682+0000 7f0b65600700 1 -- 172.21.15.38:0/1507878993 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0b5c014070 con 0x7f0b680ff9e0 2024-03-24T00:51:48.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.682+0000 7f0b67e00700 1 --2- 172.21.15.38:0/1507878993 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f0b5006dc20 0x7f0b500700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:48.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.683+0000 7f0b67e00700 1 --2- 172.21.15.38:0/1507878993 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f0b5006dc20 0x7f0b500700e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400b800 tx=0x7f0b54009f70 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:48.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.687+0000 7f0b65600700 1 -- 172.21.15.38:0/1507878993 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f0b5c05af20 con 0x7f0b680ff9e0 2024-03-24T00:51:48.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.968+0000 7f0b6f600700 1 -- 172.21.15.38:0/1507878993 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b680696d0 con 0x7f0b5006dc20 2024-03-24T00:51:48.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.969+0000 7f0b65600700 1 -- 172.21.15.38:0/1507878993 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f0b680696d0 con 0x7f0b5006dc20 2024-03-24T00:51:48.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.972+0000 7f0b6f600700 1 -- 172.21.15.38:0/1507878993 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f0b5006dc20 msgr2=0x7f0b500700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:48.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.972+0000 7f0b6f600700 1 --2- 172.21.15.38:0/1507878993 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f0b5006dc20 0x7f0b500700e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400b800 tx=0x7f0b54009f70 comp rx=0 tx=0).stop 2024-03-24T00:51:48.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.973+0000 7f0b6f600700 1 -- 172.21.15.38:0/1507878993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b680ff9e0 msgr2=0x7f0b6819cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:48.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.973+0000 7f0b6f600700 1 --2- 172.21.15.38:0/1507878993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b680ff9e0 0x7f0b6819cb20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00e8f0 tx=0x7f0b5c00ec00 comp rx=0 tx=0).stop 2024-03-24T00:51:48.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.973+0000 7f0b6f600700 1 -- 172.21.15.38:0/1507878993 shutdown_connections 2024-03-24T00:51:48.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.973+0000 7f0b6f600700 1 --2- 172.21.15.38:0/1507878993 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f0b5006dc20 0x7f0b500700e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:48.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.973+0000 7f0b6f600700 1 --2- 172.21.15.38:0/1507878993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b680ff9e0 0x7f0b6819cb20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:48.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.973+0000 7f0b6f600700 1 --2- 172.21.15.38:0/1507878993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b68100be0 0x7f0b6819d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:48.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.973+0000 7f0b6f600700 1 -- 172.21.15.38:0/1507878993 >> 172.21.15.38:0/1507878993 conn(0x7f0b680faf80 msgr2=0x7f0b68068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:48.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.973+0000 7f0b6f600700 1 -- 172.21.15.38:0/1507878993 shutdown_connections 2024-03-24T00:51:48.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:48.973+0000 7f0b6f600700 1 -- 172.21.15.38:0/1507878993 wait complete. 2024-03-24T00:51:48.990 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:51:49.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.126+0000 7f5810400700 1 -- 172.21.15.38:0/378166461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5808101ce0 msgr2=0x7f5808102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:49.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.126+0000 7f5810400700 1 --2- 172.21.15.38:0/378166461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5808101ce0 0x7f5808102140 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009910 tx=0x7f57f4009c20 comp rx=0 tx=0).stop 2024-03-24T00:51:49.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.126+0000 7f5810400700 1 -- 172.21.15.38:0/378166461 shutdown_connections 2024-03-24T00:51:49.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.126+0000 7f5810400700 1 --2- 172.21.15.38:0/378166461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5808101ce0 0x7f5808102140 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.126+0000 7f5810400700 1 --2- 172.21.15.38:0/378166461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5808100ae0 0x7f5808100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.126+0000 7f5810400700 1 -- 172.21.15.38:0/378166461 >> 172.21.15.38:0/378166461 conn(0x7f58080fc0a0 msgr2=0x7f58080fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:49.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.126+0000 7f5810400700 1 -- 172.21.15.38:0/378166461 shutdown_connections 2024-03-24T00:51:49.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.126+0000 7f5810400700 1 -- 172.21.15.38:0/378166461 wait complete. 2024-03-24T00:51:49.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.127+0000 7f5810400700 1 Processor -- start 2024-03-24T00:51:49.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.127+0000 7f5810400700 1 -- start start 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.128+0000 7f5810400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5808100ae0 0x7f58081965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.128+0000 7f5810400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5808101ce0 0x7f5808196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.128+0000 7f5810400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5808197130 con 0x7f5808101ce0 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.128+0000 7f5810400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5808197270 con 0x7f5808100ae0 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.128+0000 7f580d000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5808101ce0 0x7f5808196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.128+0000 7f580d000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5808101ce0 0x7f5808196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44166/0 (socket says 172.21.15.38:44166) 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.128+0000 7f580d000700 1 -- 172.21.15.38:0/1780766156 learned_addr learned my addr 172.21.15.38:0/1780766156 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.128+0000 7f580da00700 1 --2- 172.21.15.38:0/1780766156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5808100ae0 0x7f58081965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.128+0000 7f580d000700 1 -- 172.21.15.38:0/1780766156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5808100ae0 msgr2=0x7f58081965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.128+0000 7f580d000700 1 --2- 172.21.15.38:0/1780766156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5808100ae0 0x7f58081965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.128+0000 7f580d000700 1 -- 172.21.15.38:0/1780766156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f40095f0 con 0x7f5808101ce0 2024-03-24T00:51:49.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.129+0000 7f580d000700 1 --2- 172.21.15.38:0/1780766156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5808101ce0 0x7f5808196b10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f57f400bd70 tx=0x7f57f400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:49.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.129+0000 7f5806000700 1 -- 172.21.15.38:0/1780766156 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57f401d070 con 0x7f5808101ce0 2024-03-24T00:51:49.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.129+0000 7f5806000700 1 -- 172.21.15.38:0/1780766156 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57f400f460 con 0x7f5808101ce0 2024-03-24T00:51:49.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.129+0000 7f5810400700 1 -- 172.21.15.38:0/1780766156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580819bc70 con 0x7f5808101ce0 2024-03-24T00:51:49.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.129+0000 7f5806000700 1 -- 172.21.15.38:0/1780766156 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57f4021620 con 0x7f5808101ce0 2024-03-24T00:51:49.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.129+0000 7f5810400700 1 -- 172.21.15.38:0/1780766156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580819c1e0 con 0x7f5808101ce0 2024-03-24T00:51:49.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.130+0000 7f5810400700 1 -- 172.21.15.38:0/1780766156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5808190700 con 0x7f5808101ce0 2024-03-24T00:51:49.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.131+0000 7f5806000700 1 -- 172.21.15.38:0/1780766156 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f57f400fa70 con 0x7f5808101ce0 2024-03-24T00:51:49.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.132+0000 7f5806000700 1 --2- 172.21.15.38:0/1780766156 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f57f006dc30 0x7f57f00700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:49.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.132+0000 7f5806000700 1 -- 172.21.15.38:0/1780766156 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f57f408ebd0 con 0x7f5808101ce0 2024-03-24T00:51:49.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.133+0000 7f580da00700 1 --2- 172.21.15.38:0/1780766156 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f57f006dc30 0x7f57f00700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:49.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.133+0000 7f580da00700 1 --2- 172.21.15.38:0/1780766156 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f57f006dc30 0x7f57f00700f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f57fc006fd0 tx=0x7f57fc008040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:49.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.137+0000 7f5806000700 1 -- 172.21.15.38:0/1780766156 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f57f405bfb0 con 0x7f5808101ce0 2024-03-24T00:51:49.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.400+0000 7f5810400700 1 -- 172.21.15.38:0/1780766156 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58080611f0 con 0x7f57f006dc30 2024-03-24T00:51:49.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.402+0000 7f5806000700 1 -- 172.21.15.38:0/1780766156 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f58080611f0 con 0x7f57f006dc30 2024-03-24T00:51:49.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.404+0000 7f5810400700 1 -- 172.21.15.38:0/1780766156 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f57f006dc30 msgr2=0x7f57f00700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:49.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.404+0000 7f5810400700 1 --2- 172.21.15.38:0/1780766156 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f57f006dc30 0x7f57f00700f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f57fc006fd0 tx=0x7f57fc008040 comp rx=0 tx=0).stop 2024-03-24T00:51:49.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.404+0000 7f5810400700 1 -- 172.21.15.38:0/1780766156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5808101ce0 msgr2=0x7f5808196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:49.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.404+0000 7f5810400700 1 --2- 172.21.15.38:0/1780766156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5808101ce0 0x7f5808196b10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f57f400bd70 tx=0x7f57f400be50 comp rx=0 tx=0).stop 2024-03-24T00:51:49.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.404+0000 7f5810400700 1 -- 172.21.15.38:0/1780766156 shutdown_connections 2024-03-24T00:51:49.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.405+0000 7f5810400700 1 --2- 172.21.15.38:0/1780766156 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f57f006dc30 0x7f57f00700f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.405+0000 7f5810400700 1 --2- 172.21.15.38:0/1780766156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5808100ae0 0x7f58081965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.405+0000 7f5810400700 1 --2- 172.21.15.38:0/1780766156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5808101ce0 0x7f5808196b10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.405+0000 7f5810400700 1 -- 172.21.15.38:0/1780766156 >> 172.21.15.38:0/1780766156 conn(0x7f58080fc0a0 msgr2=0x7f5808105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:49.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.405+0000 7f5810400700 1 -- 172.21.15.38:0/1780766156 shutdown_connections 2024-03-24T00:51:49.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.405+0000 7f5810400700 1 -- 172.21.15.38:0/1780766156 wait complete. 2024-03-24T00:51:49.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.565+0000 7f928fe00700 1 -- 172.21.15.38:0/1432763517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9288101560 msgr2=0x7f9288103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:49.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.565+0000 7f928fe00700 1 --2- 172.21.15.38:0/1432763517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9288101560 0x7f9288103950 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f927c009910 tx=0x7f927c009c20 comp rx=0 tx=0).stop 2024-03-24T00:51:49.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.565+0000 7f928fe00700 1 -- 172.21.15.38:0/1432763517 shutdown_connections 2024-03-24T00:51:49.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.565+0000 7f928fe00700 1 --2- 172.21.15.38:0/1432763517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9288103e90 0x7f9288106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.565+0000 7f928fe00700 1 --2- 172.21.15.38:0/1432763517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9288101560 0x7f9288103950 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.565+0000 7f928fe00700 1 -- 172.21.15.38:0/1432763517 >> 172.21.15.38:0/1432763517 conn(0x7f92880faf70 msgr2=0x7f92880fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:49.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.565+0000 7f928fe00700 1 -- 172.21.15.38:0/1432763517 shutdown_connections 2024-03-24T00:51:49.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.565+0000 7f928fe00700 1 -- 172.21.15.38:0/1432763517 wait complete. 2024-03-24T00:51:49.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.567+0000 7f928fe00700 1 Processor -- start 2024-03-24T00:51:49.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.567+0000 7f928fe00700 1 -- start start 2024-03-24T00:51:49.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.567+0000 7f928fe00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9288101560 0x7f92881965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:49.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.567+0000 7f928fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9288103e90 0x7f9288196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:49.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.567+0000 7f928fe00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288197140 con 0x7f9288103e90 2024-03-24T00:51:49.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.567+0000 7f928fe00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288197280 con 0x7f9288101560 2024-03-24T00:51:49.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.568+0000 7f928ca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9288103e90 0x7f9288196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:49.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.568+0000 7f928ca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9288103e90 0x7f9288196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44182/0 (socket says 172.21.15.38:44182) 2024-03-24T00:51:49.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.568+0000 7f928ca00700 1 -- 172.21.15.38:0/1294581986 learned_addr learned my addr 172.21.15.38:0/1294581986 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:51:49.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.568+0000 7f928d400700 1 --2- 172.21.15.38:0/1294581986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9288101560 0x7f92881965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:49.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.568+0000 7f928ca00700 1 -- 172.21.15.38:0/1294581986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9288101560 msgr2=0x7f92881965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:49.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.568+0000 7f928ca00700 1 --2- 172.21.15.38:0/1294581986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9288101560 0x7f92881965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.568+0000 7f928ca00700 1 -- 172.21.15.38:0/1294581986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c0095f0 con 0x7f9288103e90 2024-03-24T00:51:49.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.569+0000 7f928ca00700 1 --2- 172.21.15.38:0/1294581986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9288103e90 0x7f9288196b20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f927400b4b0 tx=0x7f927400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:49.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.569+0000 7f9286000700 1 -- 172.21.15.38:0/1294581986 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9274011800 con 0x7f9288103e90 2024-03-24T00:51:49.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.569+0000 7f9286000700 1 -- 172.21.15.38:0/1294581986 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9274011e20 con 0x7f9288103e90 2024-03-24T00:51:49.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.569+0000 7f928fe00700 1 -- 172.21.15.38:0/1294581986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928819bc90 con 0x7f9288103e90 2024-03-24T00:51:49.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.569+0000 7f9286000700 1 -- 172.21.15.38:0/1294581986 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927400f2e0 con 0x7f9288103e90 2024-03-24T00:51:49.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.569+0000 7f928fe00700 1 -- 172.21.15.38:0/1294581986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928819c260 con 0x7f9288103e90 2024-03-24T00:51:49.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.570+0000 7f928fe00700 1 -- 172.21.15.38:0/1294581986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928804ea80 con 0x7f9288103e90 2024-03-24T00:51:49.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.571+0000 7f9286000700 1 -- 172.21.15.38:0/1294581986 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f9274010380 con 0x7f9288103e90 2024-03-24T00:51:49.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.572+0000 7f9286000700 1 --2- 172.21.15.38:0/1294581986 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f927006dc30 0x7f92700700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:49.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.572+0000 7f9286000700 1 -- 172.21.15.38:0/1294581986 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f927408d2a0 con 0x7f9288103e90 2024-03-24T00:51:49.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.573+0000 7f928d400700 1 --2- 172.21.15.38:0/1294581986 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f927006dc30 0x7f92700700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:49.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.573+0000 7f928d400700 1 --2- 172.21.15.38:0/1294581986 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f927006dc30 0x7f92700700f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f927c00b5c0 tx=0x7f927c005160 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:49.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.577+0000 7f9286000700 1 -- 172.21.15.38:0/1294581986 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f927405a7a0 con 0x7f9288103e90 2024-03-24T00:51:49.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.823+0000 7f928fe00700 1 -- 172.21.15.38:0/1294581986 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f928819c500 con 0x7f927006dc30 2024-03-24T00:51:49.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.839+0000 7f9286000700 1 -- 172.21.15.38:0/1294581986 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f928819c500 con 0x7f927006dc30 2024-03-24T00:51:49.840 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (12s) 4s ago 14m 14.0M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (14m) 4s ago 14m 8242k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (13m) 73s ago 13m 7922k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (14m) 4s ago 14m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (12m) 73s ago 12m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (11m) 4s ago 13m 75.3M - 9.4.7 954c08fa6188 e6aa268d69e8 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (6m) 4s ago 6m 98.3M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (6m) 4s ago 6m 151M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (6m) 73s ago 6m 24.4M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (6m) 73s ago 6m 24.3M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (3m) 4s ago 15m 500M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (106s) 73s ago 12m 447M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:51:49.841 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (16m) 4s ago 16m 56.6M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:51:49.842 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (12m) 73s ago 12m 44.5M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:51:49.842 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (84s) 4s ago 14m 7721k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:51:49.842 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (78s) 73s ago 12m 6753k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:51:49.842 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (11m) 4s ago 11m 255M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:51:49.842 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (11m) 4s ago 11m 269M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:51:49.842 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (10m) 4s ago 10m 225M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:51:49.842 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (9m) 73s ago 9m 274M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:51:49.842 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (9m) 73s ago 9m 258M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:51:49.842 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (8m) 73s ago 8m 222M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:51:49.842 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (40s) 4s ago 13m 53.3M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:51:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.842+0000 7f928fe00700 1 -- 172.21.15.38:0/1294581986 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f927006dc30 msgr2=0x7f92700700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.842+0000 7f928fe00700 1 --2- 172.21.15.38:0/1294581986 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f927006dc30 0x7f92700700f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f927c00b5c0 tx=0x7f927c005160 comp rx=0 tx=0).stop 2024-03-24T00:51:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.842+0000 7f928fe00700 1 -- 172.21.15.38:0/1294581986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9288103e90 msgr2=0x7f9288196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.842+0000 7f928fe00700 1 --2- 172.21.15.38:0/1294581986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9288103e90 0x7f9288196b20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f927400b4b0 tx=0x7f927400b870 comp rx=0 tx=0).stop 2024-03-24T00:51:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.842+0000 7f928fe00700 1 -- 172.21.15.38:0/1294581986 shutdown_connections 2024-03-24T00:51:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.842+0000 7f928fe00700 1 --2- 172.21.15.38:0/1294581986 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f927006dc30 0x7f92700700f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.842+0000 7f928fe00700 1 --2- 172.21.15.38:0/1294581986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9288101560 0x7f92881965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.842+0000 7f928fe00700 1 --2- 172.21.15.38:0/1294581986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9288103e90 0x7f9288196b20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.842+0000 7f928fe00700 1 -- 172.21.15.38:0/1294581986 >> 172.21.15.38:0/1294581986 conn(0x7f92880faf70 msgr2=0x7f92880fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:49.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.842+0000 7f928fe00700 1 -- 172.21.15.38:0/1294581986 shutdown_connections 2024-03-24T00:51:49.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.842+0000 7f928fe00700 1 -- 172.21.15.38:0/1294581986 wait complete. 2024-03-24T00:51:49.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.991+0000 7f9551200700 1 -- 172.21.15.38:0/143914037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f954c101cd0 msgr2=0x7f954c102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:49.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.991+0000 7f9551200700 1 --2- 172.21.15.38:0/143914037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f954c101cd0 0x7f954c102130 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9538009910 tx=0x7f9538009c20 comp rx=0 tx=0).stop 2024-03-24T00:51:49.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.991+0000 7f9551200700 1 -- 172.21.15.38:0/143914037 shutdown_connections 2024-03-24T00:51:49.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.991+0000 7f9551200700 1 --2- 172.21.15.38:0/143914037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f954c101cd0 0x7f954c102130 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.991+0000 7f9551200700 1 --2- 172.21.15.38:0/143914037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f954c100ad0 0x7f954c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.991+0000 7f9551200700 1 -- 172.21.15.38:0/143914037 >> 172.21.15.38:0/143914037 conn(0x7f954c0fc090 msgr2=0x7f954c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:49.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.992+0000 7f9551200700 1 -- 172.21.15.38:0/143914037 shutdown_connections 2024-03-24T00:51:49.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.992+0000 7f9551200700 1 -- 172.21.15.38:0/143914037 wait complete. 2024-03-24T00:51:49.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.993+0000 7f9551200700 1 Processor -- start 2024-03-24T00:51:49.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.993+0000 7f9551200700 1 -- start start 2024-03-24T00:51:49.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.994+0000 7f9551200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f954c100ad0 0x7f954c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:49.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.994+0000 7f9551200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f954c101cd0 0x7f954c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:49.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.994+0000 7f9551200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c199270 con 0x7f954c101cd0 2024-03-24T00:51:49.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.994+0000 7f9551200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c1993b0 con 0x7f954c100ad0 2024-03-24T00:51:49.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.994+0000 7f9549400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f954c101cd0 0x7f954c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:49.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.994+0000 7f9549400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f954c101cd0 0x7f954c198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44208/0 (socket says 172.21.15.38:44208) 2024-03-24T00:51:49.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.994+0000 7f9549400700 1 -- 172.21.15.38:0/1684834699 learned_addr learned my addr 172.21.15.38:0/1684834699 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:51:49.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.994+0000 7f9549e00700 1 --2- 172.21.15.38:0/1684834699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f954c100ad0 0x7f954c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:49.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.994+0000 7f9549400700 1 -- 172.21.15.38:0/1684834699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f954c100ad0 msgr2=0x7f954c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:49.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.994+0000 7f9549400700 1 --2- 172.21.15.38:0/1684834699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f954c100ad0 0x7f954c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:49.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.994+0000 7f9549400700 1 -- 172.21.15.38:0/1684834699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95380095f0 con 0x7f954c101cd0 2024-03-24T00:51:49.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.995+0000 7f9549400700 1 --2- 172.21.15.38:0/1684834699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f954c101cd0 0x7f954c198c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f953800bd70 tx=0x7f953800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.995+0000 7f9542a00700 1 -- 172.21.15.38:0/1684834699 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f953801d070 con 0x7f954c101cd0 2024-03-24T00:51:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.995+0000 7f9542a00700 1 -- 172.21.15.38:0/1684834699 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f953800f460 con 0x7f954c101cd0 2024-03-24T00:51:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.995+0000 7f9542a00700 1 -- 172.21.15.38:0/1684834699 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9538021620 con 0x7f954c101cd0 2024-03-24T00:51:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.995+0000 7f9551200700 1 -- 172.21.15.38:0/1684834699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c19ddb0 con 0x7f954c101cd0 2024-03-24T00:51:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.995+0000 7f9551200700 1 -- 172.21.15.38:0/1684834699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c19e2d0 con 0x7f954c101cd0 2024-03-24T00:51:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.997+0000 7f9551200700 1 -- 172.21.15.38:0/1684834699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954c106120 con 0x7f954c101cd0 2024-03-24T00:51:49.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.998+0000 7f9542a00700 1 -- 172.21.15.38:0/1684834699 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f953800fa70 con 0x7f954c101cd0 2024-03-24T00:51:50.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:49.999+0000 7f9542a00700 1 --2- 172.21.15.38:0/1684834699 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f953006dc30 0x7f95300700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:50.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.000+0000 7f9542a00700 1 -- 172.21.15.38:0/1684834699 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f953808ea40 con 0x7f954c101cd0 2024-03-24T00:51:50.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.000+0000 7f9549e00700 1 --2- 172.21.15.38:0/1684834699 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f953006dc30 0x7f95300700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:50.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.001+0000 7f9549e00700 1 --2- 172.21.15.38:0/1684834699 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f953006dc30 0x7f95300700f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f953c005fd0 tx=0x7f953c005dd0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:50.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.004+0000 7f9542a00700 1 -- 172.21.15.38:0/1684834699 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f953805bf40 con 0x7f954c101cd0 2024-03-24T00:51:50.108 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:49 smithi038 ceph-mon[29271]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:50.108 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:49 smithi038 ceph-mon[29271]: pgmap v78: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:51:50.108 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:49 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:51:50.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:49 smithi088 ceph-mon[36610]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:50.182 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:49 smithi088 ceph-mon[36610]: pgmap v78: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:51:50.182 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:49 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:51:50.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.327+0000 7f9551200700 1 -- 172.21.15.38:0/1684834699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f954c02d120 con 0x7f954c101cd0 2024-03-24T00:51:50.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.328+0000 7f9542a00700 1 -- 172.21.15.38:0/1684834699 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f953805bad0 con 0x7f954c101cd0 2024-03-24T00:51:50.330 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:51:50.330 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:51:50.330 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:51:50.330 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:51:50.330 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:51:50.330 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:51:50.330 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:51:50.330 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:51:50.330 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:51:50.330 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:51:50.330 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:51:50.331 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:51:50.331 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:51:50.331 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:51:50.331 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 12, 2024-03-24T00:51:50.331 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:51:50.331 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:51:50.331 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:51:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.331+0000 7f9551200700 1 -- 172.21.15.38:0/1684834699 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f953006dc30 msgr2=0x7f95300700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.331+0000 7f9551200700 1 --2- 172.21.15.38:0/1684834699 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f953006dc30 0x7f95300700f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f953c005fd0 tx=0x7f953c005dd0 comp rx=0 tx=0).stop 2024-03-24T00:51:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.331+0000 7f9551200700 1 -- 172.21.15.38:0/1684834699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f954c101cd0 msgr2=0x7f954c198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.331+0000 7f9551200700 1 --2- 172.21.15.38:0/1684834699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f954c101cd0 0x7f954c198c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f953800bd70 tx=0x7f953800be50 comp rx=0 tx=0).stop 2024-03-24T00:51:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.331+0000 7f9551200700 1 -- 172.21.15.38:0/1684834699 shutdown_connections 2024-03-24T00:51:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.331+0000 7f9551200700 1 --2- 172.21.15.38:0/1684834699 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f953006dc30 0x7f95300700f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.331+0000 7f9551200700 1 --2- 172.21.15.38:0/1684834699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f954c100ad0 0x7f954c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.332+0000 7f9551200700 1 --2- 172.21.15.38:0/1684834699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f954c101cd0 0x7f954c198c50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.332+0000 7f9551200700 1 -- 172.21.15.38:0/1684834699 >> 172.21.15.38:0/1684834699 conn(0x7f954c0fc090 msgr2=0x7f954c103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.332+0000 7f9551200700 1 -- 172.21.15.38:0/1684834699 shutdown_connections 2024-03-24T00:51:50.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.332+0000 7f9551200700 1 -- 172.21.15.38:0/1684834699 wait complete. 2024-03-24T00:51:50.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.492+0000 7fd0ffe00700 1 -- 172.21.15.38:0/4077290632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f8101560 msgr2=0x7fd0f8103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:50.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.492+0000 7fd0ffe00700 1 --2- 172.21.15.38:0/4077290632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f8101560 0x7fd0f8103950 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009910 tx=0x7fd0ec009c20 comp rx=0 tx=0).stop 2024-03-24T00:51:50.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.492+0000 7fd0ffe00700 1 -- 172.21.15.38:0/4077290632 shutdown_connections 2024-03-24T00:51:50.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.492+0000 7fd0ffe00700 1 --2- 172.21.15.38:0/4077290632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0f8103e90 0x7fd0f8106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:50.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.492+0000 7fd0ffe00700 1 --2- 172.21.15.38:0/4077290632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f8101560 0x7fd0f8103950 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:50.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.492+0000 7fd0ffe00700 1 -- 172.21.15.38:0/4077290632 >> 172.21.15.38:0/4077290632 conn(0x7fd0f80faf70 msgr2=0x7fd0f80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:50.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.492+0000 7fd0ffe00700 1 -- 172.21.15.38:0/4077290632 shutdown_connections 2024-03-24T00:51:50.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.492+0000 7fd0ffe00700 1 -- 172.21.15.38:0/4077290632 wait complete. 2024-03-24T00:51:50.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.493+0000 7fd0ffe00700 1 Processor -- start 2024-03-24T00:51:50.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.494+0000 7fd0ffe00700 1 -- start start 2024-03-24T00:51:50.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.494+0000 7fd0ffe00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0f8101560 0x7fd0f81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:50.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.494+0000 7fd0ffe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f8103e90 0x7fd0f8198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:50.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.494+0000 7fd0ffe00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8199220 con 0x7fd0f8103e90 2024-03-24T00:51:50.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.494+0000 7fd0ffe00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8199360 con 0x7fd0f8101560 2024-03-24T00:51:50.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.494+0000 7fd0fca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f8103e90 0x7fd0f8198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:50.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.494+0000 7fd0fca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f8103e90 0x7fd0f8198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44224/0 (socket says 172.21.15.38:44224) 2024-03-24T00:51:50.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.494+0000 7fd0fca00700 1 -- 172.21.15.38:0/2976226370 learned_addr learned my addr 172.21.15.38:0/2976226370 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:51:50.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.495+0000 7fd0fd400700 1 --2- 172.21.15.38:0/2976226370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0f8101560 0x7fd0f81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:50.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.495+0000 7fd0fca00700 1 -- 172.21.15.38:0/2976226370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0f8101560 msgr2=0x7fd0f81986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:50.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.495+0000 7fd0fca00700 1 --2- 172.21.15.38:0/2976226370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0f8101560 0x7fd0f81986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:50.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.495+0000 7fd0fca00700 1 -- 172.21.15.38:0/2976226370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec0095f0 con 0x7fd0f8103e90 2024-03-24T00:51:50.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.495+0000 7fd0fca00700 1 --2- 172.21.15.38:0/2976226370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f8103e90 0x7fd0f8198c00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400d8d0 tx=0x7fd0e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:50.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.495+0000 7fd0f6000700 1 -- 172.21.15.38:0/2976226370 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0e4009940 con 0x7fd0f8103e90 2024-03-24T00:51:50.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.496+0000 7fd0f6000700 1 -- 172.21.15.38:0/2976226370 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0e4010460 con 0x7fd0f8103e90 2024-03-24T00:51:50.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.496+0000 7fd0ffe00700 1 -- 172.21.15.38:0/2976226370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f819ddc0 con 0x7fd0f8103e90 2024-03-24T00:51:50.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.496+0000 7fd0f6000700 1 -- 172.21.15.38:0/2976226370 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0e400f5f0 con 0x7fd0f8103e90 2024-03-24T00:51:50.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.496+0000 7fd0ffe00700 1 -- 172.21.15.38:0/2976226370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f819e360 con 0x7fd0f8103e90 2024-03-24T00:51:50.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.497+0000 7fd0ffe00700 1 -- 172.21.15.38:0/2976226370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f804ea80 con 0x7fd0f8103e90 2024-03-24T00:51:50.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.498+0000 7fd0f6000700 1 -- 172.21.15.38:0/2976226370 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7fd0e40105d0 con 0x7fd0f8103e90 2024-03-24T00:51:50.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.499+0000 7fd0f6000700 1 --2- 172.21.15.38:0/2976226370 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd0e006dc20 0x7fd0e00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:51:50.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.499+0000 7fd0f6000700 1 -- 172.21.15.38:0/2976226370 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd0e408d760 con 0x7fd0f8103e90 2024-03-24T00:51:50.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.499+0000 7fd0fd400700 1 --2- 172.21.15.38:0/2976226370 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd0e006dc20 0x7fd0e00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:51:50.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.500+0000 7fd0fd400700 1 --2- 172.21.15.38:0/2976226370 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd0e006dc20 0x7fd0e00700e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec006010 tx=0x7fd0ec00b580 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:51:50.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.503+0000 7fd0f6000700 1 -- 172.21.15.38:0/2976226370 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fd0e405abc0 con 0x7fd0f8103e90 2024-03-24T00:51:50.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.778+0000 7fd0ffe00700 1 -- 172.21.15.38:0/2976226370 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0f80ffd20 con 0x7fd0e006dc20 2024-03-24T00:51:50.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.781+0000 7fd0f6000700 1 -- 172.21.15.38:0/2976226370 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fd0f80ffd20 con 0x7fd0e006dc20 2024-03-24T00:51:50.782 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:51:50.783 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:51:50.783 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:51:50.783 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-24T00:51:50.783 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:51:50.783 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-03-24T00:51:50.783 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:51:50.783 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "2/2 daemons upgraded", 2024-03-24T00:51:50.783 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading alertmanager daemons", 2024-03-24T00:51:50.783 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:51:50.783 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:51:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.786+0000 7fd0ffe00700 1 -- 172.21.15.38:0/2976226370 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd0e006dc20 msgr2=0x7fd0e00700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.786+0000 7fd0ffe00700 1 --2- 172.21.15.38:0/2976226370 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd0e006dc20 0x7fd0e00700e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec006010 tx=0x7fd0ec00b580 comp rx=0 tx=0).stop 2024-03-24T00:51:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.786+0000 7fd0ffe00700 1 -- 172.21.15.38:0/2976226370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f8103e90 msgr2=0x7fd0f8198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:51:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.786+0000 7fd0ffe00700 1 --2- 172.21.15.38:0/2976226370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f8103e90 0x7fd0f8198c00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400d8d0 tx=0x7fd0e400dc90 comp rx=0 tx=0).stop 2024-03-24T00:51:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.786+0000 7fd0ffe00700 1 -- 172.21.15.38:0/2976226370 shutdown_connections 2024-03-24T00:51:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.786+0000 7fd0ffe00700 1 --2- 172.21.15.38:0/2976226370 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd0e006dc20 0x7fd0e00700e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.786+0000 7fd0ffe00700 1 --2- 172.21.15.38:0/2976226370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0f8101560 0x7fd0f81986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.786+0000 7fd0ffe00700 1 --2- 172.21.15.38:0/2976226370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f8103e90 0x7fd0f8198c00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:51:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.786+0000 7fd0ffe00700 1 -- 172.21.15.38:0/2976226370 >> 172.21.15.38:0/2976226370 conn(0x7fd0f80faf70 msgr2=0x7fd0f80ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:51:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.786+0000 7fd0ffe00700 1 -- 172.21.15.38:0/2976226370 shutdown_connections 2024-03-24T00:51:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:51:50.786+0000 7fd0ffe00700 1 -- 172.21.15.38:0/2976226370 wait complete. 2024-03-24T00:51:51.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:50 smithi038 ceph-mon[29271]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:51.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:50 smithi038 ceph-mon[29271]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:51.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:50 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1684834699' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:50 smithi088 ceph-mon[36610]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:50 smithi088 ceph-mon[36610]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:50 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1684834699' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:52.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:51 smithi038 ceph-mon[29271]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:52.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:51 smithi038 ceph-mon[29271]: pgmap v79: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:51:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:51 smithi088 ceph-mon[36610]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:51:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:51 smithi088 ceph-mon[36610]: pgmap v79: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:51:54.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: pgmap v80: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:51:54.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:54.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:54.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:51:54.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:51:54.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:54.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T00:51:54.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:51:54.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:54 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: pgmap v80: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:51:54.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:54.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:54.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:51:54.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:51:54.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:51:54.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T00:51:54.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:51:54.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:54.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:54 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:51:55.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:55 smithi038 ceph-mon[29271]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T00:51:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:55 smithi088 ceph-mon[36610]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T00:51:56.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:56 smithi038 ceph-mon[29271]: pgmap v81: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:51:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:56 smithi088 ceph-mon[36610]: pgmap v81: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:51:57.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:57 smithi038 ceph-mon[29271]: Upgrade: Updating grafana.smithi038 2024-03-24T00:51:57.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:57 smithi038 ceph-mon[29271]: Deploying daemon grafana.smithi038 on smithi038 2024-03-24T00:51:57.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:57 smithi088 ceph-mon[36610]: Upgrade: Updating grafana.smithi038 2024-03-24T00:51:57.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:57 smithi088 ceph-mon[36610]: Deploying daemon grafana.smithi038 on smithi038 2024-03-24T00:51:58.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:58 smithi038 ceph-mon[29271]: pgmap v82: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:51:58.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:58 smithi088 ceph-mon[36610]: pgmap v82: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:00.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:51:59 smithi088 ceph-mon[36610]: pgmap v83: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:00.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:51:59 smithi038 ceph-mon[29271]: pgmap v83: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:02.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:02 smithi038 ceph-mon[29271]: pgmap v84: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:02 smithi088 ceph-mon[36610]: pgmap v84: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:04.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:04 smithi038 ceph-mon[29271]: pgmap v85: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:04.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:04 smithi088 ceph-mon[36610]: pgmap v85: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:05.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:05 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:52:05.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:05 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:52:06.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:06 smithi038 ceph-mon[29271]: pgmap v86: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:06.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:06 smithi088 ceph-mon[36610]: pgmap v86: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:08.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:08 smithi038 ceph-mon[29271]: pgmap v87: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:08.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:08 smithi088 ceph-mon[36610]: pgmap v87: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:09 smithi038 ceph-mon[29271]: pgmap v88: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:09 smithi088 ceph-mon[36610]: pgmap v88: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:12.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:12 smithi038 ceph-mon[29271]: pgmap v89: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:12.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:12 smithi088 ceph-mon[36610]: pgmap v89: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:14.345 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:14 smithi038 ceph-mon[29271]: pgmap v90: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:14.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:14 smithi088 ceph-mon[36610]: pgmap v90: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:16.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:16 smithi038 ceph-mon[29271]: pgmap v91: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:16 smithi088 ceph-mon[36610]: pgmap v91: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:18.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:18 smithi038 ceph-mon[29271]: pgmap v92: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:18.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:18 smithi088 ceph-mon[36610]: pgmap v92: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:20.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:20.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:20.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:52:20.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:19 smithi088 ceph-mon[36610]: pgmap v93: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:20.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:52:20.187 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:20.187 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:20.187 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:52:20.187 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:19 smithi038 ceph-mon[29271]: pgmap v93: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:20.187 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:52:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.967+0000 7efc7d600700 1 -- 172.21.15.38:0/3027324208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc78107130 msgr2=0x7efc78107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.967+0000 7efc7d600700 1 --2- 172.21.15.38:0/3027324208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc78107130 0x7efc78107550 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efc68009910 tx=0x7efc68009c20 comp rx=0 tx=0).stop 2024-03-24T00:52:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.967+0000 7efc7d600700 1 -- 172.21.15.38:0/3027324208 shutdown_connections 2024-03-24T00:52:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.967+0000 7efc7d600700 1 --2- 172.21.15.38:0/3027324208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc78108300 0x7efc78108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.967+0000 7efc7d600700 1 --2- 172.21.15.38:0/3027324208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc78107130 0x7efc78107550 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.967+0000 7efc7d600700 1 -- 172.21.15.38:0/3027324208 >> 172.21.15.38:0/3027324208 conn(0x7efc780762d0 msgr2=0x7efc78078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.967+0000 7efc7d600700 1 -- 172.21.15.38:0/3027324208 shutdown_connections 2024-03-24T00:52:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.967+0000 7efc7d600700 1 -- 172.21.15.38:0/3027324208 wait complete. 2024-03-24T00:52:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.968+0000 7efc7d600700 1 Processor -- start 2024-03-24T00:52:20.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.968+0000 7efc7d600700 1 -- start start 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.968+0000 7efc7d600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc78108300 0x7efc7819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.968+0000 7efc7d600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7819d230 0x7efc781a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.968+0000 7efc7d600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7819d6b0 con 0x7efc7819d230 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.968+0000 7efc7d600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7819d820 con 0x7efc78108300 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.969+0000 7efc76000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7819d230 0x7efc781a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.969+0000 7efc76000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7819d230 0x7efc781a2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57696/0 (socket says 172.21.15.38:57696) 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.969+0000 7efc76000700 1 -- 172.21.15.38:0/3285266038 learned_addr learned my addr 172.21.15.38:0/3285266038 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.969+0000 7efc76a00700 1 --2- 172.21.15.38:0/3285266038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc78108300 0x7efc7819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.969+0000 7efc76000700 1 -- 172.21.15.38:0/3285266038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc78108300 msgr2=0x7efc7819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.969+0000 7efc76000700 1 --2- 172.21.15.38:0/3285266038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc78108300 0x7efc7819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.969+0000 7efc76000700 1 -- 172.21.15.38:0/3285266038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc680095f0 con 0x7efc7819d230 2024-03-24T00:52:20.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.970+0000 7efc76000700 1 --2- 172.21.15.38:0/3285266038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7819d230 0x7efc781a2240 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efc6400c930 tx=0x7efc6400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:20.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.970+0000 7efc6f400700 1 -- 172.21.15.38:0/3285266038 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc64007ad0 con 0x7efc7819d230 2024-03-24T00:52:20.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.970+0000 7efc6f400700 1 -- 172.21.15.38:0/3285266038 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc64007c30 con 0x7efc7819d230 2024-03-24T00:52:20.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.970+0000 7efc7d600700 1 -- 172.21.15.38:0/3285266038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc781a2780 con 0x7efc7819d230 2024-03-24T00:52:20.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.970+0000 7efc6f400700 1 -- 172.21.15.38:0/3285266038 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc64018610 con 0x7efc7819d230 2024-03-24T00:52:20.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.970+0000 7efc7d600700 1 -- 172.21.15.38:0/3285266038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc781a2d00 con 0x7efc7819d230 2024-03-24T00:52:20.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.971+0000 7efc7d600700 1 -- 172.21.15.38:0/3285266038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc78196ca0 con 0x7efc7819d230 2024-03-24T00:52:20.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.973+0000 7efc6f400700 1 -- 172.21.15.38:0/3285266038 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7efc6400f450 con 0x7efc7819d230 2024-03-24T00:52:20.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.974+0000 7efc6f400700 1 --2- 172.21.15.38:0/3285266038 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7efc5c06dc30 0x7efc5c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:20.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.974+0000 7efc6f400700 1 -- 172.21.15.38:0/3285266038 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efc6408dad0 con 0x7efc7819d230 2024-03-24T00:52:20.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.974+0000 7efc76a00700 1 --2- 172.21.15.38:0/3285266038 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7efc5c06dc30 0x7efc5c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:20.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.975+0000 7efc76a00700 1 --2- 172.21.15.38:0/3285266038 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7efc5c06dc30 0x7efc5c0700f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efc78077790 tx=0x7efc68005fb0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:20.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:20.979+0000 7efc6f400700 1 -- 172.21.15.38:0/3285266038 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7efc64057380 con 0x7efc7819d230 2024-03-24T00:52:21.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.251+0000 7efc7d600700 1 -- 172.21.15.38:0/3285266038 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc780611f0 con 0x7efc5c06dc30 2024-03-24T00:52:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.253+0000 7efc6f400700 1 -- 172.21.15.38:0/3285266038 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7efc780611f0 con 0x7efc5c06dc30 2024-03-24T00:52:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.256+0000 7efc7d600700 1 -- 172.21.15.38:0/3285266038 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7efc5c06dc30 msgr2=0x7efc5c0700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.256+0000 7efc7d600700 1 --2- 172.21.15.38:0/3285266038 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7efc5c06dc30 0x7efc5c0700f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efc78077790 tx=0x7efc68005fb0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.256+0000 7efc7d600700 1 -- 172.21.15.38:0/3285266038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7819d230 msgr2=0x7efc781a2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.256+0000 7efc7d600700 1 --2- 172.21.15.38:0/3285266038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7819d230 0x7efc781a2240 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efc6400c930 tx=0x7efc6400ccf0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.256+0000 7efc7d600700 1 -- 172.21.15.38:0/3285266038 shutdown_connections 2024-03-24T00:52:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.256+0000 7efc7d600700 1 --2- 172.21.15.38:0/3285266038 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7efc5c06dc30 0x7efc5c0700f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.256+0000 7efc7d600700 1 --2- 172.21.15.38:0/3285266038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc78108300 0x7efc7819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.256+0000 7efc7d600700 1 --2- 172.21.15.38:0/3285266038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7819d230 0x7efc781a2240 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.256+0000 7efc7d600700 1 -- 172.21.15.38:0/3285266038 >> 172.21.15.38:0/3285266038 conn(0x7efc780762d0 msgr2=0x7efc7810b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.256+0000 7efc7d600700 1 -- 172.21.15.38:0/3285266038 shutdown_connections 2024-03-24T00:52:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.256+0000 7efc7d600700 1 -- 172.21.15.38:0/3285266038 wait complete. 2024-03-24T00:52:21.273 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:52:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.417+0000 7f9985600700 1 -- 172.21.15.38:0/2975237863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9980103f00 msgr2=0x7f9980104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.417+0000 7f9985600700 1 --2- 172.21.15.38:0/2975237863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9980103f00 0x7f9980104360 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f996c009910 tx=0x7f996c009c20 comp rx=0 tx=0).stop 2024-03-24T00:52:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.418+0000 7f9985600700 1 -- 172.21.15.38:0/2975237863 shutdown_connections 2024-03-24T00:52:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.418+0000 7f9985600700 1 --2- 172.21.15.38:0/2975237863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9980103f00 0x7f9980104360 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.418+0000 7f9985600700 1 --2- 172.21.15.38:0/2975237863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9980102d00 0x7f9980103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.418+0000 7f9985600700 1 -- 172.21.15.38:0/2975237863 >> 172.21.15.38:0/2975237863 conn(0x7f99800fe2c0 msgr2=0x7f99801006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.418+0000 7f9985600700 1 -- 172.21.15.38:0/2975237863 shutdown_connections 2024-03-24T00:52:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.418+0000 7f9985600700 1 -- 172.21.15.38:0/2975237863 wait complete. 2024-03-24T00:52:21.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.419+0000 7f9985600700 1 Processor -- start 2024-03-24T00:52:21.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.419+0000 7f9985600700 1 -- start start 2024-03-24T00:52:21.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.419+0000 7f9985600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9980102d00 0x7f99801986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.419+0000 7f9985600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9980103f00 0x7f9980198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.419+0000 7f9985600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99801991b0 con 0x7f9980102d00 2024-03-24T00:52:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.419+0000 7f9985600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99801992f0 con 0x7f9980103f00 2024-03-24T00:52:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.420+0000 7f997ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9980102d00 0x7f99801986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.420+0000 7f997ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9980102d00 0x7f99801986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57702/0 (socket says 172.21.15.38:57702) 2024-03-24T00:52:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.420+0000 7f997ea00700 1 -- 172.21.15.38:0/1744102823 learned_addr learned my addr 172.21.15.38:0/1744102823 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:52:21.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.420+0000 7f997e000700 1 --2- 172.21.15.38:0/1744102823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9980103f00 0x7f9980198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:21.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.420+0000 7f997ea00700 1 -- 172.21.15.38:0/1744102823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9980103f00 msgr2=0x7f9980198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:21.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.420+0000 7f997ea00700 1 --2- 172.21.15.38:0/1744102823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9980103f00 0x7f9980198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.420+0000 7f997ea00700 1 -- 172.21.15.38:0/1744102823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c0095f0 con 0x7f9980102d00 2024-03-24T00:52:21.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.420+0000 7f997ea00700 1 --2- 172.21.15.38:0/1744102823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9980102d00 0x7f99801986e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f997000c910 tx=0x7f997000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:21.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.421+0000 7f9977400700 1 -- 172.21.15.38:0/1744102823 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99700041f0 con 0x7f9980102d00 2024-03-24T00:52:21.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.421+0000 7f9977400700 1 -- 172.21.15.38:0/1744102823 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9970007dc0 con 0x7f9980102d00 2024-03-24T00:52:21.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.421+0000 7f9985600700 1 -- 172.21.15.38:0/1744102823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998019dd60 con 0x7f9980102d00 2024-03-24T00:52:21.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.421+0000 7f9977400700 1 -- 172.21.15.38:0/1744102823 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9970003e70 con 0x7f9980102d00 2024-03-24T00:52:21.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.421+0000 7f9985600700 1 -- 172.21.15.38:0/1744102823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998019e2a0 con 0x7f9980102d00 2024-03-24T00:52:21.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.422+0000 7f9985600700 1 -- 172.21.15.38:0/1744102823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998004eaf0 con 0x7f9980102d00 2024-03-24T00:52:21.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.423+0000 7f9977400700 1 -- 172.21.15.38:0/1744102823 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f9970004350 con 0x7f9980102d00 2024-03-24T00:52:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.424+0000 7f9977400700 1 --2- 172.21.15.38:0/1744102823 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f996406dbc0 0x7f9964070080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.424+0000 7f9977400700 1 -- 172.21.15.38:0/1744102823 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f997008d600 con 0x7f9980102d00 2024-03-24T00:52:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.425+0000 7f997e000700 1 --2- 172.21.15.38:0/1744102823 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f996406dbc0 0x7f9964070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.426+0000 7f997e000700 1 --2- 172.21.15.38:0/1744102823 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f996406dbc0 0x7f9964070080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f996c00b5c0 tx=0x7f996c005160 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.430+0000 7f9977400700 1 -- 172.21.15.38:0/1744102823 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f998004eaf0 con 0x7f9980102d00 2024-03-24T00:52:21.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.699+0000 7f9985600700 1 -- 172.21.15.38:0/1744102823 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f998004ea80 con 0x7f996406dbc0 2024-03-24T00:52:21.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.701+0000 7f9977400700 1 -- 172.21.15.38:0/1744102823 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f998004ea80 con 0x7f996406dbc0 2024-03-24T00:52:21.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.704+0000 7f9985600700 1 -- 172.21.15.38:0/1744102823 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f996406dbc0 msgr2=0x7f9964070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:21.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.704+0000 7f9985600700 1 --2- 172.21.15.38:0/1744102823 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f996406dbc0 0x7f9964070080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f996c00b5c0 tx=0x7f996c005160 comp rx=0 tx=0).stop 2024-03-24T00:52:21.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.704+0000 7f9985600700 1 -- 172.21.15.38:0/1744102823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9980102d00 msgr2=0x7f99801986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:21.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.704+0000 7f9985600700 1 --2- 172.21.15.38:0/1744102823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9980102d00 0x7f99801986e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f997000c910 tx=0x7f997000ccd0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.704+0000 7f9985600700 1 -- 172.21.15.38:0/1744102823 shutdown_connections 2024-03-24T00:52:21.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.704+0000 7f9985600700 1 --2- 172.21.15.38:0/1744102823 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f996406dbc0 0x7f9964070080 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.704+0000 7f9985600700 1 --2- 172.21.15.38:0/1744102823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9980102d00 0x7f99801986e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.704+0000 7f9985600700 1 --2- 172.21.15.38:0/1744102823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9980103f00 0x7f9980198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.704+0000 7f9985600700 1 -- 172.21.15.38:0/1744102823 >> 172.21.15.38:0/1744102823 conn(0x7f99800fe2c0 msgr2=0x7f9980107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:21.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.705+0000 7f9985600700 1 -- 172.21.15.38:0/1744102823 shutdown_connections 2024-03-24T00:52:21.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.705+0000 7f9985600700 1 -- 172.21.15.38:0/1744102823 wait complete. 2024-03-24T00:52:21.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.864+0000 7f36d2000700 1 -- 172.21.15.38:0/639997940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36cc073810 msgr2=0x7f36cc073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:21.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.864+0000 7f36d2000700 1 --2- 172.21.15.38:0/639997940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36cc073810 0x7f36cc073c90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009910 tx=0x7f36b8009c20 comp rx=0 tx=0).stop 2024-03-24T00:52:21.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.864+0000 7f36d2000700 1 -- 172.21.15.38:0/639997940 shutdown_connections 2024-03-24T00:52:21.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.864+0000 7f36d2000700 1 --2- 172.21.15.38:0/639997940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36cc073810 0x7f36cc073c90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.864+0000 7f36d2000700 1 --2- 172.21.15.38:0/639997940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36cc074de0 0x7f36cc073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.864+0000 7f36d2000700 1 -- 172.21.15.38:0/639997940 >> 172.21.15.38:0/639997940 conn(0x7f36cc0fc090 msgr2=0x7f36cc0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:21.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.864+0000 7f36d2000700 1 -- 172.21.15.38:0/639997940 shutdown_connections 2024-03-24T00:52:21.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.864+0000 7f36d2000700 1 -- 172.21.15.38:0/639997940 wait complete. 2024-03-24T00:52:21.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.865+0000 7f36d2000700 1 Processor -- start 2024-03-24T00:52:21.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.866+0000 7f36d2000700 1 -- start start 2024-03-24T00:52:21.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.866+0000 7f36d2000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36cc073810 0x7f36cc1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:21.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.866+0000 7f36d2000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36cc074de0 0x7f36cc198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:21.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.866+0000 7f36d2000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc199250 con 0x7f36cc073810 2024-03-24T00:52:21.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.866+0000 7f36d2000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc199390 con 0x7f36cc074de0 2024-03-24T00:52:21.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.866+0000 7f36cb400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36cc073810 0x7f36cc1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:21.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.866+0000 7f36cb400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36cc073810 0x7f36cc1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57718/0 (socket says 172.21.15.38:57718) 2024-03-24T00:52:21.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.866+0000 7f36cb400700 1 -- 172.21.15.38:0/2462405475 learned_addr learned my addr 172.21.15.38:0/2462405475 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:52:21.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.867+0000 7f36caa00700 1 --2- 172.21.15.38:0/2462405475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36cc074de0 0x7f36cc198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:21.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.867+0000 7f36cb400700 1 -- 172.21.15.38:0/2462405475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36cc074de0 msgr2=0x7f36cc198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:21.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.867+0000 7f36cb400700 1 --2- 172.21.15.38:0/2462405475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36cc074de0 0x7f36cc198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:21.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.867+0000 7f36cb400700 1 -- 172.21.15.38:0/2462405475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b80095f0 con 0x7f36cc073810 2024-03-24T00:52:21.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.867+0000 7f36cb400700 1 --2- 172.21.15.38:0/2462405475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36cc073810 0x7f36cc1986f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00d900 tx=0x7f36bc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:21.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.868+0000 7f36c3e00700 1 -- 172.21.15.38:0/2462405475 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36bc009900 con 0x7f36cc073810 2024-03-24T00:52:21.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.868+0000 7f36c3e00700 1 -- 172.21.15.38:0/2462405475 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36bc010460 con 0x7f36cc073810 2024-03-24T00:52:21.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.868+0000 7f36c3e00700 1 -- 172.21.15.38:0/2462405475 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36bc00f5f0 con 0x7f36cc073810 2024-03-24T00:52:21.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.868+0000 7f36d2000700 1 -- 172.21.15.38:0/2462405475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36cc19ddf0 con 0x7f36cc073810 2024-03-24T00:52:21.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.868+0000 7f36d2000700 1 -- 172.21.15.38:0/2462405475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36cc19e2f0 con 0x7f36cc073810 2024-03-24T00:52:21.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.869+0000 7f36d2000700 1 -- 172.21.15.38:0/2462405475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36cc192880 con 0x7f36cc073810 2024-03-24T00:52:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.871+0000 7f36c3e00700 1 -- 172.21.15.38:0/2462405475 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f36bc0105d0 con 0x7f36cc073810 2024-03-24T00:52:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.872+0000 7f36c3e00700 1 --2- 172.21.15.38:0/2462405475 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f36b006dc30 0x7f36b00700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.872+0000 7f36c3e00700 1 -- 172.21.15.38:0/2462405475 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f36bc08d2f0 con 0x7f36cc073810 2024-03-24T00:52:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.872+0000 7f36caa00700 1 --2- 172.21.15.38:0/2462405475 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f36b006dc30 0x7f36b00700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.873+0000 7f36caa00700 1 --2- 172.21.15.38:0/2462405475 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f36b006dc30 0x7f36b00700f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f36b8005090 tx=0x7f36b800b540 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:21.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:21.878+0000 7f36c3e00700 1 -- 172.21.15.38:0/2462405475 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f36bc05a750 con 0x7f36cc073810 2024-03-24T00:52:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.126+0000 7f36d2000700 1 -- 172.21.15.38:0/2462405475 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f36cc0611f0 con 0x7f36b006dc30 2024-03-24T00:52:22.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.142+0000 7f36c3e00700 1 -- 172.21.15.38:0/2462405475 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f36cc0611f0 con 0x7f36b006dc30 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (44s) 36s ago 14m 14.0M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (15m) 36s ago 15m 8242k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (13m) 105s ago 13m 7922k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (15m) 36s ago 14m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (13m) 105s ago 13m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 starting - - - - 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (7m) 36s ago 7m 98.3M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (7m) 36s ago 7m 151M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (7m) 105s ago 7m 24.4M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (7m) 105s ago 7m 24.3M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (3m) 36s ago 16m 500M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (2m) 105s ago 13m 447M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:52:22.144 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (16m) 36s ago 16m 56.6M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:52:22.145 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (13m) 105s ago 13m 44.5M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:52:22.145 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (117s) 36s ago 14m 7721k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:52:22.145 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (111s) 105s ago 13m 6753k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:52:22.145 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (12m) 36s ago 12m 255M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:52:22.145 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (11m) 36s ago 11m 269M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:52:22.145 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (10m) 36s ago 10m 225M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:52:22.145 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (10m) 105s ago 10m 274M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:52:22.145 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (9m) 105s ago 9m 258M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:52:22.145 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (9m) 105s ago 9m 222M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:52:22.145 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (72s) 36s ago 14m 53.3M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:52:22.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.145+0000 7f36d2000700 1 -- 172.21.15.38:0/2462405475 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f36b006dc30 msgr2=0x7f36b00700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.145+0000 7f36d2000700 1 --2- 172.21.15.38:0/2462405475 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f36b006dc30 0x7f36b00700f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f36b8005090 tx=0x7f36b800b540 comp rx=0 tx=0).stop 2024-03-24T00:52:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.145+0000 7f36d2000700 1 -- 172.21.15.38:0/2462405475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36cc073810 msgr2=0x7f36cc1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.145+0000 7f36d2000700 1 --2- 172.21.15.38:0/2462405475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36cc073810 0x7f36cc1986f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00d900 tx=0x7f36bc00dc10 comp rx=0 tx=0).stop 2024-03-24T00:52:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.145+0000 7f36d2000700 1 -- 172.21.15.38:0/2462405475 shutdown_connections 2024-03-24T00:52:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.145+0000 7f36d2000700 1 --2- 172.21.15.38:0/2462405475 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f36b006dc30 0x7f36b00700f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.145+0000 7f36d2000700 1 --2- 172.21.15.38:0/2462405475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36cc073810 0x7f36cc1986f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.145+0000 7f36d2000700 1 --2- 172.21.15.38:0/2462405475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36cc074de0 0x7f36cc198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.145+0000 7f36d2000700 1 -- 172.21.15.38:0/2462405475 >> 172.21.15.38:0/2462405475 conn(0x7f36cc0fc090 msgr2=0x7f36cc106c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.146+0000 7f36d2000700 1 -- 172.21.15.38:0/2462405475 shutdown_connections 2024-03-24T00:52:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.146+0000 7f36d2000700 1 -- 172.21.15.38:0/2462405475 wait complete. 2024-03-24T00:52:22.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.320+0000 7f223c400700 1 -- 172.21.15.38:0/1895317622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234103e90 msgr2=0x7f2234106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:22.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.320+0000 7f223c400700 1 --2- 172.21.15.38:0/1895317622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234103e90 0x7f2234106280 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2220009910 tx=0x7f2220009c20 comp rx=0 tx=0).stop 2024-03-24T00:52:22.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.320+0000 7f223c400700 1 -- 172.21.15.38:0/1895317622 shutdown_connections 2024-03-24T00:52:22.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.320+0000 7f223c400700 1 --2- 172.21.15.38:0/1895317622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234103e90 0x7f2234106280 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.320+0000 7f223c400700 1 --2- 172.21.15.38:0/1895317622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2234101560 0x7f2234103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.320+0000 7f223c400700 1 -- 172.21.15.38:0/1895317622 >> 172.21.15.38:0/1895317622 conn(0x7f22340faf70 msgr2=0x7f22340fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:22.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.320+0000 7f223c400700 1 -- 172.21.15.38:0/1895317622 shutdown_connections 2024-03-24T00:52:22.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.320+0000 7f223c400700 1 -- 172.21.15.38:0/1895317622 wait complete. 2024-03-24T00:52:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.321+0000 7f223c400700 1 Processor -- start 2024-03-24T00:52:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.321+0000 7f223c400700 1 -- start start 2024-03-24T00:52:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f223c400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2234101560 0x7f2234100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f223c400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234103e90 0x7f22340ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f223c400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340ffa50 con 0x7f2234103e90 2024-03-24T00:52:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f223c400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340ffb90 con 0x7f2234101560 2024-03-24T00:52:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f2239000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234103e90 0x7f22340ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f2239000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234103e90 0x7f22340ff510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57738/0 (socket says 172.21.15.38:57738) 2024-03-24T00:52:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f2239000700 1 -- 172.21.15.38:0/212932385 learned_addr learned my addr 172.21.15.38:0/212932385 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:52:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f2239a00700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2234101560 0x7f2234100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f2239000700 1 -- 172.21.15.38:0/212932385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2234101560 msgr2=0x7f2234100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f2239000700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2234101560 0x7f2234100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f2239000700 1 -- 172.21.15.38:0/212932385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22200095f0 con 0x7f2234103e90 2024-03-24T00:52:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.322+0000 7f2239a00700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2234101560 0x7f2234100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:52:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.323+0000 7f2239000700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234103e90 0x7f22340ff510 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2220006010 tx=0x7f222000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.323+0000 7f2232000700 1 -- 172.21.15.38:0/212932385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f222001d070 con 0x7f2234103e90 2024-03-24T00:52:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.323+0000 7f2232000700 1 -- 172.21.15.38:0/212932385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f222000f460 con 0x7f2234103e90 2024-03-24T00:52:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.323+0000 7f2232000700 1 -- 172.21.15.38:0/212932385 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2220021640 con 0x7f2234103e90 2024-03-24T00:52:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.323+0000 7f223c400700 1 -- 172.21.15.38:0/212932385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22340ffe10 con 0x7f2234103e90 2024-03-24T00:52:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.323+0000 7f223c400700 1 -- 172.21.15.38:0/212932385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22341002e0 con 0x7f2234103e90 2024-03-24T00:52:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.324+0000 7f223c400700 1 -- 172.21.15.38:0/212932385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223404ea80 con 0x7f2234103e90 2024-03-24T00:52:22.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.327+0000 7f2232000700 1 -- 172.21.15.38:0/212932385 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f222000fad0 con 0x7f2234103e90 2024-03-24T00:52:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.328+0000 7f2232000700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f221c06dc30 0x7f221c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.328+0000 7f2232000700 1 -- 172.21.15.38:0/212932385 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f222008ef70 con 0x7f2234103e90 2024-03-24T00:52:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.328+0000 7f2239a00700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f221c06dc30 0x7f221c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.329+0000 7f2239a00700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f221c06dc30 0x7f221c0700f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2228005fd0 tx=0x7f2228005dc0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.330+0000 7f2232000700 1 -- 172.21.15.38:0/212932385 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f222005c470 con 0x7f2234103e90 2024-03-24T00:52:22.430 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:22 smithi038 ceph-mon[29271]: pgmap v94: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:22.430 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:22 smithi038 ceph-mon[29271]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:52:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.652+0000 7f223c400700 1 -- 172.21.15.38:0/212932385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f223419e5c0 con 0x7f2234103e90 2024-03-24T00:52:22.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.653+0000 7f2232000700 1 -- 172.21.15.38:0/212932385 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f222005c000 con 0x7f2234103e90 2024-03-24T00:52:22.654 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:52:22.654 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:52:22.654 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:52:22.654 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:52:22.654 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 12, 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:52:22.655 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:52:22.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.655+0000 7f223c400700 1 -- 172.21.15.38:0/212932385 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f221c06dc30 msgr2=0x7f221c0700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:22.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.655+0000 7f223c400700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f221c06dc30 0x7f221c0700f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2228005fd0 tx=0x7f2228005dc0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.655+0000 7f223c400700 1 -- 172.21.15.38:0/212932385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234103e90 msgr2=0x7f22340ff510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:22.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.655+0000 7f223c400700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234103e90 0x7f22340ff510 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2220006010 tx=0x7f222000be20 comp rx=0 tx=0).stop 2024-03-24T00:52:22.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.656+0000 7f223c400700 1 -- 172.21.15.38:0/212932385 shutdown_connections 2024-03-24T00:52:22.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.656+0000 7f223c400700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f221c06dc30 0x7f221c0700f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.656+0000 7f223c400700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2234101560 0x7f2234100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.656+0000 7f223c400700 1 --2- 172.21.15.38:0/212932385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234103e90 0x7f22340ff510 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.656+0000 7f223c400700 1 -- 172.21.15.38:0/212932385 >> 172.21.15.38:0/212932385 conn(0x7f22340faf70 msgr2=0x7f22340fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:22.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.656+0000 7f223c400700 1 -- 172.21.15.38:0/212932385 shutdown_connections 2024-03-24T00:52:22.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.656+0000 7f223c400700 1 -- 172.21.15.38:0/212932385 wait complete. 2024-03-24T00:52:22.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:22 smithi088 ceph-mon[36610]: pgmap v94: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:22.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:22 smithi088 ceph-mon[36610]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:52:22.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.821+0000 7fdcb6c00700 1 -- 172.21.15.38:0/1455366615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb0074de0 msgr2=0x7fdcb0073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:22.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.821+0000 7fdcb6c00700 1 --2- 172.21.15.38:0/1455366615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb0074de0 0x7fdcb0073240 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009910 tx=0x7fdca0009c20 comp rx=0 tx=0).stop 2024-03-24T00:52:22.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.822+0000 7fdcb6c00700 1 -- 172.21.15.38:0/1455366615 shutdown_connections 2024-03-24T00:52:22.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.822+0000 7fdcb6c00700 1 --2- 172.21.15.38:0/1455366615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcb0073810 0x7fdcb0073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.822+0000 7fdcb6c00700 1 --2- 172.21.15.38:0/1455366615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb0074de0 0x7fdcb0073240 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.822+0000 7fdcb6c00700 1 -- 172.21.15.38:0/1455366615 >> 172.21.15.38:0/1455366615 conn(0x7fdcb00fc090 msgr2=0x7fdcb00fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:22.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.822+0000 7fdcb6c00700 1 -- 172.21.15.38:0/1455366615 shutdown_connections 2024-03-24T00:52:22.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.822+0000 7fdcb6c00700 1 -- 172.21.15.38:0/1455366615 wait complete. 2024-03-24T00:52:22.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.823+0000 7fdcb6c00700 1 Processor -- start 2024-03-24T00:52:22.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.824+0000 7fdcb6c00700 1 -- start start 2024-03-24T00:52:22.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.824+0000 7fdcb6c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb0073810 0x7fdcb019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:22.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.824+0000 7fdcb6c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcb0074de0 0x7fdcb019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:22.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.824+0000 7fdcb6c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb019d5d0 con 0x7fdcb0073810 2024-03-24T00:52:22.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.824+0000 7fdcb6c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb019d710 con 0x7fdcb0074de0 2024-03-24T00:52:22.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.824+0000 7fdcafe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb0073810 0x7fdcb019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:22.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.824+0000 7fdcafe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb0073810 0x7fdcb019cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57750/0 (socket says 172.21.15.38:57750) 2024-03-24T00:52:22.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.824+0000 7fdcafe00700 1 -- 172.21.15.38:0/3661223497 learned_addr learned my addr 172.21.15.38:0/3661223497 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:52:22.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.825+0000 7fdcaf400700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcb0074de0 0x7fdcb019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:22.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.825+0000 7fdcafe00700 1 -- 172.21.15.38:0/3661223497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcb0074de0 msgr2=0x7fdcb019d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:22.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.825+0000 7fdcafe00700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcb0074de0 0x7fdcb019d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:22.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.825+0000 7fdcafe00700 1 -- 172.21.15.38:0/3661223497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca00095f0 con 0x7fdcb0073810 2024-03-24T00:52:22.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.825+0000 7fdcaf400700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcb0074de0 0x7fdcb019d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:52:22.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.826+0000 7fdcafe00700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb0073810 0x7fdcb019cb00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdca000bd70 tx=0x7fdca000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:22.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.826+0000 7fdcacc00700 1 -- 172.21.15.38:0/3661223497 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdca001d070 con 0x7fdcb0073810 2024-03-24T00:52:22.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.826+0000 7fdcacc00700 1 -- 172.21.15.38:0/3661223497 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdca000f460 con 0x7fdcb0073810 2024-03-24T00:52:22.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.826+0000 7fdcb6c00700 1 -- 172.21.15.38:0/3661223497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb01a2120 con 0x7fdcb0073810 2024-03-24T00:52:22.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.826+0000 7fdcb6c00700 1 -- 172.21.15.38:0/3661223497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb01a2660 con 0x7fdcb0073810 2024-03-24T00:52:22.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.827+0000 7fdcacc00700 1 -- 172.21.15.38:0/3661223497 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdca0021620 con 0x7fdcb0073810 2024-03-24T00:52:22.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.828+0000 7fdcb6c00700 1 -- 172.21.15.38:0/3661223497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb0196c60 con 0x7fdcb0073810 2024-03-24T00:52:22.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.830+0000 7fdcacc00700 1 -- 172.21.15.38:0/3661223497 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7fdca000fa70 con 0x7fdcb0073810 2024-03-24T00:52:22.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.831+0000 7fdcacc00700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fdc9406dc30 0x7fdc940700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:22.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.832+0000 7fdcacc00700 1 -- 172.21.15.38:0/3661223497 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fdca008eb50 con 0x7fdcb0073810 2024-03-24T00:52:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.832+0000 7fdcaf400700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fdc9406dc30 0x7fdc940700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.833+0000 7fdcaf400700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fdc9406dc30 0x7fdc940700f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0074b10 tx=0x7fdc9c00a680 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:22.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:22.836+0000 7fdcacc00700 1 -- 172.21.15.38:0/3661223497 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fdca005bf30 con 0x7fdcb0073810 2024-03-24T00:52:23.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.089+0000 7fdcb6c00700 1 -- 172.21.15.38:0/3661223497 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcb00611f0 con 0x7fdc9406dc30 2024-03-24T00:52:23.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.091+0000 7fdcacc00700 1 -- 172.21.15.38:0/3661223497 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fdcb00611f0 con 0x7fdc9406dc30 2024-03-24T00:52:23.093 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:52:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:52:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:52:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-24T00:52:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:52:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-03-24T00:52:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:52:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "2/2 daemons upgraded", 2024-03-24T00:52:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading grafana daemons", 2024-03-24T00:52:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:52:23.094 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:52:23.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.094+0000 7fdcb6c00700 1 -- 172.21.15.38:0/3661223497 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fdc9406dc30 msgr2=0x7fdc940700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:23.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.094+0000 7fdcb6c00700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fdc9406dc30 0x7fdc940700f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0074b10 tx=0x7fdc9c00a680 comp rx=0 tx=0).stop 2024-03-24T00:52:23.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.094+0000 7fdcb6c00700 1 -- 172.21.15.38:0/3661223497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb0073810 msgr2=0x7fdcb019cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:23.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.094+0000 7fdcb6c00700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb0073810 0x7fdcb019cb00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdca000bd70 tx=0x7fdca000be50 comp rx=0 tx=0).stop 2024-03-24T00:52:23.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.095+0000 7fdcb6c00700 1 -- 172.21.15.38:0/3661223497 shutdown_connections 2024-03-24T00:52:23.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.095+0000 7fdcb6c00700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fdc9406dc30 0x7fdc940700f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:23.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.095+0000 7fdcb6c00700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcb0073810 0x7fdcb019cb00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:23.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.095+0000 7fdcb6c00700 1 --2- 172.21.15.38:0/3661223497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcb0074de0 0x7fdcb019d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:23.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.095+0000 7fdcb6c00700 1 -- 172.21.15.38:0/3661223497 >> 172.21.15.38:0/3661223497 conn(0x7fdcb00fc090 msgr2=0x7fdcb0102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:23.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.095+0000 7fdcb6c00700 1 -- 172.21.15.38:0/3661223497 shutdown_connections 2024-03-24T00:52:23.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:23.095+0000 7fdcb6c00700 1 -- 172.21.15.38:0/3661223497 wait complete. 2024-03-24T00:52:23.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:23 smithi038 ceph-mon[29271]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:52:23.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:23 smithi038 ceph-mon[29271]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:52:23.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:23 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/212932385' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:23.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:23 smithi088 ceph-mon[36610]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:52:23.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:23 smithi088 ceph-mon[36610]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:52:23.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:23 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/212932385' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:24.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:24 smithi038 ceph-mon[29271]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:52:24.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:24 smithi038 ceph-mon[29271]: pgmap v95: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:24 smithi088 ceph-mon[36610]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:52:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:24 smithi088 ceph-mon[36610]: pgmap v95: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:26.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:26 smithi038 ceph-mon[29271]: pgmap v96: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:26.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:26 smithi088 ceph-mon[36610]: pgmap v96: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:27 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:27 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:27 smithi088 ceph-mon[36610]: pgmap v97: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:27.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:27 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:27.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:27 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:27.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:27 smithi038 ceph-mon[29271]: pgmap v97: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:29.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:29 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:29.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:29 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:29.853 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:29 smithi038 ceph-mon[29271]: pgmap v98: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:29 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:29 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:29 smithi088 ceph-mon[36610]: pgmap v98: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:32.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:32 smithi038 ceph-mon[29271]: pgmap v99: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:32.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:32 smithi088 ceph-mon[36610]: pgmap v99: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:34.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:34 smithi038 ceph-mon[29271]: pgmap v100: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:34.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:34 smithi088 ceph-mon[36610]: pgmap v100: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:35.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:52:35.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:35.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:35.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-24T00:52:35.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-24T00:52:35.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-24T00:52:35.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-24T00:52:35.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-24T00:52:35.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-24T00:52:35.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-24T00:52:35.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-24T00:52:35.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-24T00:52:35.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-24T00:52:35.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.613 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-24T00:52:35.614 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:52:35.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:52:35.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:52:35.615 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:35.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:52:35.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:35.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:35.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:52:35.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.683 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.683 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.683 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:35.683 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:35.683 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-24T00:52:35.683 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-24T00:52:35.683 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-24T00:52:35.684 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:52:35.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:36.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:36 smithi038 ceph-mon[29271]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T00:52:36.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:36 smithi038 ceph-mon[29271]: Upgrade: Finalizing container_image settings 2024-03-24T00:52:36.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:36 smithi038 ceph-mon[29271]: Upgrade: Complete! 2024-03-24T00:52:36.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:36 smithi038 ceph-mon[29271]: pgmap v101: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:36.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:36 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:52:36.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:36 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:52:36.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:36 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:52:36.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:36 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:36.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:36 smithi088 ceph-mon[36610]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T00:52:36.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:36 smithi088 ceph-mon[36610]: Upgrade: Finalizing container_image settings 2024-03-24T00:52:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:36 smithi088 ceph-mon[36610]: Upgrade: Complete! 2024-03-24T00:52:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:36 smithi088 ceph-mon[36610]: pgmap v101: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:36 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:52:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:36 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:52:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:36 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:52:36.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:36 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:38.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:38 smithi038 ceph-mon[29271]: pgmap v102: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:38 smithi088 ceph-mon[36610]: pgmap v102: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:40.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:40 smithi038 ceph-mon[29271]: pgmap v103: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:40.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:40 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:40.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:40 smithi088 ceph-mon[36610]: pgmap v103: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:40.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:40 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:52:42.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:42 smithi088 ceph-mon[36610]: pgmap v104: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:42 smithi038 ceph-mon[29271]: pgmap v104: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:44 smithi038 ceph-mon[29271]: pgmap v105: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:44.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:44 smithi088 ceph-mon[36610]: pgmap v105: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:46.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:46 smithi038 ceph-mon[29271]: pgmap v106: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:46 smithi088 ceph-mon[36610]: pgmap v106: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:48.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:48 smithi038 ceph-mon[29271]: pgmap v107: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:48 smithi088 ceph-mon[36610]: pgmap v107: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:49 smithi088 ceph-mon[36610]: pgmap v108: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:49 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:52:50.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:49 smithi038 ceph-mon[29271]: pgmap v108: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:50.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:49 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:52:52.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:52 smithi038 ceph-mon[29271]: pgmap v109: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:52.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:52 smithi088 ceph-mon[36610]: pgmap v109: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:53.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.293+0000 7f6736000700 1 -- 172.21.15.38:0/2294828774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6730103f00 msgr2=0x7f6730104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:53.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.293+0000 7f6736000700 1 --2- 172.21.15.38:0/2294828774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6730103f00 0x7f6730104360 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6718009910 tx=0x7f6718009c20 comp rx=0 tx=0).stop 2024-03-24T00:52:53.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.294+0000 7f6736000700 1 -- 172.21.15.38:0/2294828774 shutdown_connections 2024-03-24T00:52:53.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.294+0000 7f6736000700 1 --2- 172.21.15.38:0/2294828774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6730103f00 0x7f6730104360 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:53.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.294+0000 7f6736000700 1 --2- 172.21.15.38:0/2294828774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6730102d00 0x7f6730103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:53.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.294+0000 7f6736000700 1 -- 172.21.15.38:0/2294828774 >> 172.21.15.38:0/2294828774 conn(0x7f67300fe2a0 msgr2=0x7f67301006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:53.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.294+0000 7f6736000700 1 -- 172.21.15.38:0/2294828774 shutdown_connections 2024-03-24T00:52:53.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.294+0000 7f6736000700 1 -- 172.21.15.38:0/2294828774 wait complete. 2024-03-24T00:52:53.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.295+0000 7f6736000700 1 Processor -- start 2024-03-24T00:52:53.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.295+0000 7f6736000700 1 -- start start 2024-03-24T00:52:53.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.296+0000 7f6736000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6730102d00 0x7f6730198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:53.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.296+0000 7f6736000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6730103f00 0x7f6730198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:53.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.296+0000 7f6736000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6730199270 con 0x7f6730102d00 2024-03-24T00:52:53.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.296+0000 7f6736000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67301993b0 con 0x7f6730103f00 2024-03-24T00:52:53.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.296+0000 7f672f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6730102d00 0x7f6730198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:53.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.296+0000 7f672f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6730102d00 0x7f6730198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41682/0 (socket says 172.21.15.38:41682) 2024-03-24T00:52:53.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.296+0000 7f672f400700 1 -- 172.21.15.38:0/3970792931 learned_addr learned my addr 172.21.15.38:0/3970792931 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:52:53.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.296+0000 7f6726a00700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6730103f00 0x7f6730198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:53.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.297+0000 7f672f400700 1 -- 172.21.15.38:0/3970792931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6730103f00 msgr2=0x7f6730198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:53.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.297+0000 7f672f400700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6730103f00 0x7f6730198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:53.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.297+0000 7f672f400700 1 -- 172.21.15.38:0/3970792931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67180095f0 con 0x7f6730102d00 2024-03-24T00:52:53.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.297+0000 7f6726a00700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6730103f00 0x7f6730198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:52:53.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.297+0000 7f672f400700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6730102d00 0x7f6730198710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6720007e90 tx=0x7f672000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:53.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.297+0000 7f672cc00700 1 -- 172.21.15.38:0/3970792931 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6720017070 con 0x7f6730102d00 2024-03-24T00:52:53.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.297+0000 7f672cc00700 1 -- 172.21.15.38:0/3970792931 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f672000f040 con 0x7f6730102d00 2024-03-24T00:52:53.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.297+0000 7f672cc00700 1 -- 172.21.15.38:0/3970792931 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67200048f0 con 0x7f6730102d00 2024-03-24T00:52:53.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.297+0000 7f6736000700 1 -- 172.21.15.38:0/3970792931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673019ddc0 con 0x7f6730102d00 2024-03-24T00:52:53.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.298+0000 7f6736000700 1 -- 172.21.15.38:0/3970792931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673019e310 con 0x7f6730102d00 2024-03-24T00:52:53.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.299+0000 7f6736000700 1 -- 172.21.15.38:0/3970792931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6730192880 con 0x7f6730102d00 2024-03-24T00:52:53.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.300+0000 7f672cc00700 1 -- 172.21.15.38:0/3970792931 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f6720004120 con 0x7f6730102d00 2024-03-24T00:52:53.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.301+0000 7f672cc00700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f671406dc30 0x7f67140700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:53.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.301+0000 7f672cc00700 1 -- 172.21.15.38:0/3970792931 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f672008d1d0 con 0x7f6730102d00 2024-03-24T00:52:53.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.301+0000 7f6726a00700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f671406dc30 0x7f67140700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:53.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.302+0000 7f6726a00700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f671406dc30 0x7f67140700f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f671800b5c0 tx=0x7f6718005160 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:53.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.306+0000 7f672cc00700 1 -- 172.21.15.38:0/3970792931 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f672005a630 con 0x7f6730102d00 2024-03-24T00:52:53.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.593+0000 7f6736000700 1 -- 172.21.15.38:0/3970792931 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67300611f0 con 0x7f671406dc30 2024-03-24T00:52:53.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.596+0000 7f672cc00700 1 -- 172.21.15.38:0/3970792931 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f67300611f0 con 0x7f671406dc30 2024-03-24T00:52:53.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.599+0000 7f6736000700 1 -- 172.21.15.38:0/3970792931 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f671406dc30 msgr2=0x7f67140700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:53.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.599+0000 7f6736000700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f671406dc30 0x7f67140700f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f671800b5c0 tx=0x7f6718005160 comp rx=0 tx=0).stop 2024-03-24T00:52:53.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.599+0000 7f6736000700 1 -- 172.21.15.38:0/3970792931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6730102d00 msgr2=0x7f6730198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:53.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.599+0000 7f6736000700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6730102d00 0x7f6730198710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6720007e90 tx=0x7f672000d3b0 comp rx=0 tx=0).stop 2024-03-24T00:52:53.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.599+0000 7f6736000700 1 -- 172.21.15.38:0/3970792931 shutdown_connections 2024-03-24T00:52:53.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.599+0000 7f6736000700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f671406dc30 0x7f67140700f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:53.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.599+0000 7f6736000700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6730102d00 0x7f6730198710 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:53.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.599+0000 7f6736000700 1 --2- 172.21.15.38:0/3970792931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6730103f00 0x7f6730198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:53.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.599+0000 7f6736000700 1 -- 172.21.15.38:0/3970792931 >> 172.21.15.38:0/3970792931 conn(0x7f67300fe2a0 msgr2=0x7f6730107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:53.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.599+0000 7f6736000700 1 -- 172.21.15.38:0/3970792931 shutdown_connections 2024-03-24T00:52:53.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:53.599+0000 7f6736000700 1 -- 172.21.15.38:0/3970792931 wait complete. 2024-03-24T00:52:54.374 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-03-24T00:52:54.513 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:54 smithi038 ceph-mon[29271]: pgmap v110: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:54.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:54 smithi088 ceph-mon[36610]: pgmap v110: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:54.845 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:52:55.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:55 smithi038 ceph-mon[29271]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:52:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:55 smithi088 ceph-mon[36610]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:52:56.462 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:56 smithi038 ceph-mon[29271]: pgmap v111: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:56 smithi088 ceph-mon[36610]: pgmap v111: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:52:56.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.943+0000 7f67d6c00700 1 -- 172.21.15.38:0/2847984767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67d0073a70 msgr2=0x7f67d0111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:56.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.943+0000 7f67d6c00700 1 --2- 172.21.15.38:0/2847984767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67d0073a70 0x7f67d0111540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009910 tx=0x7f67bc009c20 comp rx=0 tx=0).stop 2024-03-24T00:52:56.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.943+0000 7f67d6c00700 1 -- 172.21.15.38:0/2847984767 shutdown_connections 2024-03-24T00:52:56.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.943+0000 7f67d6c00700 1 --2- 172.21.15.38:0/2847984767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67d0073a70 0x7f67d0111540 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:56.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.943+0000 7f67d6c00700 1 --2- 172.21.15.38:0/2847984767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0073150 0x7f67d0073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:56.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.943+0000 7f67d6c00700 1 -- 172.21.15.38:0/2847984767 >> 172.21.15.38:0/2847984767 conn(0x7f67d00fc580 msgr2=0x7f67d00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:56.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.943+0000 7f67d6c00700 1 -- 172.21.15.38:0/2847984767 shutdown_connections 2024-03-24T00:52:56.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.943+0000 7f67d6c00700 1 -- 172.21.15.38:0/2847984767 wait complete. 2024-03-24T00:52:56.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.944+0000 7f67d6c00700 1 Processor -- start 2024-03-24T00:52:56.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.945+0000 7f67d6c00700 1 -- start start 2024-03-24T00:52:56.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.945+0000 7f67d6c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0073150 0x7f67d019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:56.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.945+0000 7f67d6c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67d0073a70 0x7f67d019d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:56.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.945+0000 7f67d6c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d019dc00 con 0x7f67d0073a70 2024-03-24T00:52:56.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.945+0000 7f67d6c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d01a18d0 con 0x7f67d0073150 2024-03-24T00:52:56.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.945+0000 7f67cf400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67d0073a70 0x7f67d019d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:56.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.945+0000 7f67cf400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67d0073a70 0x7f67d019d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41696/0 (socket says 172.21.15.38:41696) 2024-03-24T00:52:56.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.945+0000 7f67cf400700 1 -- 172.21.15.38:0/2547655155 learned_addr learned my addr 172.21.15.38:0/2547655155 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:52:56.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.946+0000 7f67cfe00700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0073150 0x7f67d019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:56.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.946+0000 7f67cf400700 1 -- 172.21.15.38:0/2547655155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0073150 msgr2=0x7f67d019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:56.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.946+0000 7f67cf400700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0073150 0x7f67d019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:56.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.946+0000 7f67cf400700 1 -- 172.21.15.38:0/2547655155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc0095f0 con 0x7f67d0073a70 2024-03-24T00:52:56.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.946+0000 7f67cfe00700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0073150 0x7f67d019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:52:56.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.946+0000 7f67cf400700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67d0073a70 0x7f67d019d520 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0092e0 tx=0x7f67bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:56.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.946+0000 7f67ccc00700 1 -- 172.21.15.38:0/2547655155 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67bc01d070 con 0x7f67d0073a70 2024-03-24T00:52:56.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.946+0000 7f67ccc00700 1 -- 172.21.15.38:0/2547655155 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67bc00f460 con 0x7f67d0073a70 2024-03-24T00:52:56.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.946+0000 7f67d6c00700 1 -- 172.21.15.38:0/2547655155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d01a1bb0 con 0x7f67d0073a70 2024-03-24T00:52:56.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.946+0000 7f67ccc00700 1 -- 172.21.15.38:0/2547655155 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67bc017630 con 0x7f67d0073a70 2024-03-24T00:52:56.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.947+0000 7f67d6c00700 1 -- 172.21.15.38:0/2547655155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d01a2100 con 0x7f67d0073a70 2024-03-24T00:52:56.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.948+0000 7f67d6c00700 1 -- 172.21.15.38:0/2547655155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d0074890 con 0x7f67d0073a70 2024-03-24T00:52:56.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.949+0000 7f67ccc00700 1 -- 172.21.15.38:0/2547655155 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f67bc00f5d0 con 0x7f67d0073a70 2024-03-24T00:52:56.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.950+0000 7f67ccc00700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f67b406dc30 0x7f67b40700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:52:56.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.950+0000 7f67ccc00700 1 -- 172.21.15.38:0/2547655155 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f67bc08ed50 con 0x7f67d0073a70 2024-03-24T00:52:56.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.950+0000 7f67cfe00700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f67b406dc30 0x7f67b40700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:52:56.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.951+0000 7f67cfe00700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f67b406dc30 0x7f67b40700f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67c0005fd0 tx=0x7f67c0005dc0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:52:56.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:56.955+0000 7f67ccc00700 1 -- 172.21.15.38:0/2547655155 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f67bc058600 con 0x7f67d0073a70 2024-03-24T00:52:57.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.278+0000 7f67d6c00700 1 -- 172.21.15.38:0/2547655155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f67d01a23e0 con 0x7f67d0073a70 2024-03-24T00:52:57.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.278+0000 7f67ccc00700 1 -- 172.21.15.38:0/2547655155 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f67bc05bcc0 con 0x7f67d0073a70 2024-03-24T00:52:57.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.281+0000 7f67d6c00700 1 -- 172.21.15.38:0/2547655155 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f67b406dc30 msgr2=0x7f67b40700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.281+0000 7f67d6c00700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f67b406dc30 0x7f67b40700f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67c0005fd0 tx=0x7f67c0005dc0 comp rx=0 tx=0).stop 2024-03-24T00:52:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.281+0000 7f67d6c00700 1 -- 172.21.15.38:0/2547655155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67d0073a70 msgr2=0x7f67d019d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:52:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.281+0000 7f67d6c00700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67d0073a70 0x7f67d019d520 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0092e0 tx=0x7f67bc00be20 comp rx=0 tx=0).stop 2024-03-24T00:52:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.281+0000 7f67d6c00700 1 -- 172.21.15.38:0/2547655155 shutdown_connections 2024-03-24T00:52:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.281+0000 7f67d6c00700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f67b406dc30 0x7f67b40700f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.281+0000 7f67d6c00700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0073150 0x7f67d019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.281+0000 7f67d6c00700 1 --2- 172.21.15.38:0/2547655155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67d0073a70 0x7f67d019d520 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:52:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.281+0000 7f67d6c00700 1 -- 172.21.15.38:0/2547655155 >> 172.21.15.38:0/2547655155 conn(0x7f67d00fc580 msgr2=0x7f67d01007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:52:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.282+0000 7f67d6c00700 1 -- 172.21.15.38:0/2547655155 shutdown_connections 2024-03-24T00:52:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:52:57.282+0000 7f67d6c00700 1 -- 172.21.15.38:0/2547655155 wait complete. 2024-03-24T00:52:57.297 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:52:57.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:57 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/2547655155' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:57.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:57 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2547655155' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:52:58.085 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-03-24T00:52:58.516 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:52:58.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:58 smithi038 ceph-mon[29271]: pgmap v112: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:52:58.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:58 smithi088 ceph-mon[36610]: pgmap v112: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:00.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:52:59 smithi088 ceph-mon[36610]: pgmap v113: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:00.234 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:52:59 smithi038 ceph-mon[29271]: pgmap v113: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:00.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.775+0000 7fd254a00700 1 -- 172.21.15.38:0/934847628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2501008d0 msgr2=0x7fd250100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:00.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.775+0000 7fd254a00700 1 --2- 172.21.15.38:0/934847628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2501008d0 0x7fd250100d50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd238009910 tx=0x7fd238009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:00.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.775+0000 7fd254a00700 1 -- 172.21.15.38:0/934847628 shutdown_connections 2024-03-24T00:53:00.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.775+0000 7fd254a00700 1 --2- 172.21.15.38:0/934847628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2501008d0 0x7fd250100d50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:00.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.775+0000 7fd254a00700 1 --2- 172.21.15.38:0/934847628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2501068d0 0x7fd250106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:00.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.775+0000 7fd254a00700 1 -- 172.21.15.38:0/934847628 >> 172.21.15.38:0/934847628 conn(0x7fd2500fc400 msgr2=0x7fd2500fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:00.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.776+0000 7fd254a00700 1 -- 172.21.15.38:0/934847628 shutdown_connections 2024-03-24T00:53:00.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.776+0000 7fd254a00700 1 -- 172.21.15.38:0/934847628 wait complete. 2024-03-24T00:53:00.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.777+0000 7fd254a00700 1 Processor -- start 2024-03-24T00:53:00.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.777+0000 7fd254a00700 1 -- start start 2024-03-24T00:53:00.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.777+0000 7fd254a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2501008d0 0x7fd25019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.777+0000 7fd254a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2501068d0 0x7fd25019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.777+0000 7fd254a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25019cfe0 con 0x7fd2501008d0 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.777+0000 7fd254a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd250196650 con 0x7fd2501068d0 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.778+0000 7fd24f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2501008d0 0x7fd25019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.778+0000 7fd24f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2501008d0 0x7fd25019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41712/0 (socket says 172.21.15.38:41712) 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.778+0000 7fd24f400700 1 -- 172.21.15.38:0/1914238151 learned_addr learned my addr 172.21.15.38:0/1914238151 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.778+0000 7fd247e00700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2501068d0 0x7fd25019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.778+0000 7fd24f400700 1 -- 172.21.15.38:0/1914238151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2501068d0 msgr2=0x7fd25019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.778+0000 7fd24f400700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2501068d0 0x7fd25019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.778+0000 7fd24f400700 1 -- 172.21.15.38:0/1914238151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380095f0 con 0x7fd2501008d0 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.778+0000 7fd247e00700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2501068d0 0x7fd25019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:53:00.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.779+0000 7fd24f400700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2501008d0 0x7fd25019c510 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c930 tx=0x7fd24000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:00.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.779+0000 7fd24cc00700 1 -- 172.21.15.38:0/1914238151 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd240007ad0 con 0x7fd2501008d0 2024-03-24T00:53:00.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.779+0000 7fd24cc00700 1 -- 172.21.15.38:0/1914238151 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd240007c30 con 0x7fd2501008d0 2024-03-24T00:53:00.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.779+0000 7fd24cc00700 1 -- 172.21.15.38:0/1914238151 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd240018610 con 0x7fd2501008d0 2024-03-24T00:53:00.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.779+0000 7fd254a00700 1 -- 172.21.15.38:0/1914238151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd250196930 con 0x7fd2501008d0 2024-03-24T00:53:00.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.779+0000 7fd254a00700 1 -- 172.21.15.38:0/1914238151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd250196e80 con 0x7fd2501008d0 2024-03-24T00:53:00.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.781+0000 7fd254a00700 1 -- 172.21.15.38:0/1914238151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd250108ef0 con 0x7fd2501008d0 2024-03-24T00:53:00.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.782+0000 7fd24cc00700 1 -- 172.21.15.38:0/1914238151 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7fd24000f450 con 0x7fd2501008d0 2024-03-24T00:53:00.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.783+0000 7fd24cc00700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd23406dc20 0x7fd2340700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:00.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.783+0000 7fd24cc00700 1 -- 172.21.15.38:0/1914238151 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd24008da00 con 0x7fd2501008d0 2024-03-24T00:53:00.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.783+0000 7fd247e00700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd23406dc20 0x7fd2340700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:00.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.784+0000 7fd247e00700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd23406dc20 0x7fd2340700e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd250197d60 tx=0x7fd23801a040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:00.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:00.789+0000 7fd24cc00700 1 -- 172.21.15.38:0/1914238151 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fd24005ae60 con 0x7fd2501008d0 2024-03-24T00:53:01.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.125+0000 7fd254a00700 1 -- 172.21.15.38:0/1914238151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd250197b70 con 0x7fd2501008d0 2024-03-24T00:53:01.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.127+0000 7fd24cc00700 1 -- 172.21.15.38:0/1914238151 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7fd24005a9f0 con 0x7fd2501008d0 2024-03-24T00:53:01.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.130+0000 7fd254a00700 1 -- 172.21.15.38:0/1914238151 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd23406dc20 msgr2=0x7fd2340700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:01.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.130+0000 7fd254a00700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd23406dc20 0x7fd2340700e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd250197d60 tx=0x7fd23801a040 comp rx=0 tx=0).stop 2024-03-24T00:53:01.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.130+0000 7fd254a00700 1 -- 172.21.15.38:0/1914238151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2501008d0 msgr2=0x7fd25019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:01.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.130+0000 7fd254a00700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2501008d0 0x7fd25019c510 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c930 tx=0x7fd24000ccf0 comp rx=0 tx=0).stop 2024-03-24T00:53:01.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.131+0000 7fd254a00700 1 -- 172.21.15.38:0/1914238151 shutdown_connections 2024-03-24T00:53:01.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.131+0000 7fd254a00700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd23406dc20 0x7fd2340700e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:01.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.131+0000 7fd254a00700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2501008d0 0x7fd25019c510 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:01.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.131+0000 7fd254a00700 1 --2- 172.21.15.38:0/1914238151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2501068d0 0x7fd25019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:01.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.131+0000 7fd254a00700 1 -- 172.21.15.38:0/1914238151 >> 172.21.15.38:0/1914238151 conn(0x7fd2500fc400 msgr2=0x7fd2500fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:01.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.131+0000 7fd254a00700 1 -- 172.21.15.38:0/1914238151 shutdown_connections 2024-03-24T00:53:01.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:01.131+0000 7fd254a00700 1 -- 172.21.15.38:0/1914238151 wait complete. 2024-03-24T00:53:01.148 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)" 2024-03-24T00:53:01.566 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:01 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1914238151' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:53:01.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:01 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1914238151' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:53:01.978 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-03-24T00:53:02.417 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:53:02.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:02 smithi088 ceph-mon[36610]: pgmap v114: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:02.573 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:02 smithi038 ceph-mon[29271]: pgmap v114: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:04.292 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:04 smithi038 ceph-mon[29271]: pgmap v115: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:04.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.593+0000 7f92f0c00700 1 -- 172.21.15.38:0/387084862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92ec0739f0 msgr2=0x7f92ec10cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:04.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.593+0000 7f92f0c00700 1 --2- 172.21.15.38:0/387084862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92ec0739f0 0x7f92ec10cdf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009910 tx=0x7f92d4009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.594+0000 7f92f0c00700 1 -- 172.21.15.38:0/387084862 shutdown_connections 2024-03-24T00:53:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.594+0000 7f92f0c00700 1 --2- 172.21.15.38:0/387084862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92ec0739f0 0x7f92ec10cdf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.594+0000 7f92f0c00700 1 --2- 172.21.15.38:0/387084862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92ec0730d0 0x7f92ec0734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.594+0000 7f92f0c00700 1 -- 172.21.15.38:0/387084862 >> 172.21.15.38:0/387084862 conn(0x7f92ec0fc580 msgr2=0x7f92ec0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.594+0000 7f92f0c00700 1 -- 172.21.15.38:0/387084862 shutdown_connections 2024-03-24T00:53:04.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.594+0000 7f92f0c00700 1 -- 172.21.15.38:0/387084862 wait complete. 2024-03-24T00:53:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.595+0000 7f92f0c00700 1 Processor -- start 2024-03-24T00:53:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.595+0000 7f92f0c00700 1 -- start start 2024-03-24T00:53:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.596+0000 7f92f0c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92ec0730d0 0x7f92ec19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.596+0000 7f92f0c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92ec0739f0 0x7f92ec19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.596+0000 7f92f0c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec19f420 con 0x7f92ec0739f0 2024-03-24T00:53:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.596+0000 7f92f0c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec198a00 con 0x7f92ec0730d0 2024-03-24T00:53:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.596+0000 7f92e1400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92ec0739f0 0x7f92ec19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:04.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.596+0000 7f92e1400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92ec0739f0 0x7f92ec19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44170/0 (socket says 172.21.15.38:44170) 2024-03-24T00:53:04.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.596+0000 7f92e1400700 1 -- 172.21.15.38:0/1949860558 learned_addr learned my addr 172.21.15.38:0/1949860558 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:04.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.596+0000 7f92e9e00700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92ec0730d0 0x7f92ec19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:04.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.597+0000 7f92e1400700 1 -- 172.21.15.38:0/1949860558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92ec0730d0 msgr2=0x7f92ec19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:04.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.597+0000 7f92e1400700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92ec0730d0 0x7f92ec19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:04.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.597+0000 7f92e1400700 1 -- 172.21.15.38:0/1949860558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40095f0 con 0x7f92ec0739f0 2024-03-24T00:53:04.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.597+0000 7f92e9e00700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92ec0730d0 0x7f92ec19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:53:04.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.597+0000 7f92e1400700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92ec0739f0 0x7f92ec19ee00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f92d40092e0 tx=0x7f92d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:04.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.598+0000 7f92e3400700 1 -- 172.21.15.38:0/1949860558 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92d401d070 con 0x7f92ec0739f0 2024-03-24T00:53:04.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.598+0000 7f92e3400700 1 -- 172.21.15.38:0/1949860558 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92d40056e0 con 0x7f92ec0739f0 2024-03-24T00:53:04.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.598+0000 7f92f0c00700 1 -- 172.21.15.38:0/1949860558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ec198c80 con 0x7f92ec0739f0 2024-03-24T00:53:04.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.598+0000 7f92f0c00700 1 -- 172.21.15.38:0/1949860558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec199170 con 0x7f92ec0739f0 2024-03-24T00:53:04.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.598+0000 7f92e3400700 1 -- 172.21.15.38:0/1949860558 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92d400fe80 con 0x7f92ec0739f0 2024-03-24T00:53:04.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.599+0000 7f92f0c00700 1 -- 172.21.15.38:0/1949860558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ec107700 con 0x7f92ec0739f0 2024-03-24T00:53:04.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.600+0000 7f92e3400700 1 -- 172.21.15.38:0/1949860558 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f92d400f6f0 con 0x7f92ec0739f0 2024-03-24T00:53:04.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.601+0000 7f92e3400700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f92d006dc30 0x7f92d00700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:04.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.601+0000 7f92e3400700 1 -- 172.21.15.38:0/1949860558 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f92d408ec60 con 0x7f92ec0739f0 2024-03-24T00:53:04.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.601+0000 7f92e9e00700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f92d006dc30 0x7f92d00700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:04.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.602+0000 7f92e9e00700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f92d006dc30 0x7f92d00700f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009dd0 tx=0x7f92dc009040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:04.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.611+0000 7f92e3400700 1 -- 172.21.15.38:0/1949860558 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f92d405c040 con 0x7f92ec0739f0 2024-03-24T00:53:04.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:04 smithi088 ceph-mon[36610]: pgmap v115: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:04.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.924+0000 7f92f0c00700 1 -- 172.21.15.38:0/1949860558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f92ec074810 con 0x7f92ec0739f0 2024-03-24T00:53:04.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.926+0000 7f92e3400700 1 -- 172.21.15.38:0/1949860558 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f92d405bbd0 con 0x7f92ec0739f0 2024-03-24T00:53:04.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.928+0000 7f92f0c00700 1 -- 172.21.15.38:0/1949860558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f92d006dc30 msgr2=0x7f92d00700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:04.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.928+0000 7f92f0c00700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f92d006dc30 0x7f92d00700f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009dd0 tx=0x7f92dc009040 comp rx=0 tx=0).stop 2024-03-24T00:53:04.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.928+0000 7f92f0c00700 1 -- 172.21.15.38:0/1949860558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92ec0739f0 msgr2=0x7f92ec19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:04.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.928+0000 7f92f0c00700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92ec0739f0 0x7f92ec19ee00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f92d40092e0 tx=0x7f92d400be20 comp rx=0 tx=0).stop 2024-03-24T00:53:04.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.928+0000 7f92f0c00700 1 -- 172.21.15.38:0/1949860558 shutdown_connections 2024-03-24T00:53:04.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.928+0000 7f92f0c00700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f92d006dc30 0x7f92d00700f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:04.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.928+0000 7f92f0c00700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92ec0730d0 0x7f92ec19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:04.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.928+0000 7f92f0c00700 1 --2- 172.21.15.38:0/1949860558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92ec0739f0 0x7f92ec19ee00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:04.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.928+0000 7f92f0c00700 1 -- 172.21.15.38:0/1949860558 >> 172.21.15.38:0/1949860558 conn(0x7f92ec0fc580 msgr2=0x7f92ec1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:04.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.929+0000 7f92f0c00700 1 -- 172.21.15.38:0/1949860558 shutdown_connections 2024-03-24T00:53:04.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:04.929+0000 7f92f0c00700 1 -- 172.21.15.38:0/1949860558 wait complete. 2024-03-24T00:53:04.943 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:53:05.570 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:05 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:53:05.571 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:05 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1949860558' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:53:05.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:05 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:53:05.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:05 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1949860558' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:53:05.914 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-03-24T00:53:06.350 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:53:06.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:06 smithi038 ceph-mon[29271]: pgmap v116: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:06.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:06 smithi088 ceph-mon[36610]: pgmap v116: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.445+0000 7fade2c00700 1 -- 172.21.15.38:0/790365555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faddc1038a0 msgr2=0x7faddc1078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.445+0000 7fade2c00700 1 --2- 172.21.15.38:0/790365555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faddc1038a0 0x7faddc1078f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009910 tx=0x7fadc8009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.445+0000 7fade2c00700 1 -- 172.21.15.38:0/790365555 shutdown_connections 2024-03-24T00:53:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.445+0000 7fade2c00700 1 --2- 172.21.15.38:0/790365555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faddc1038a0 0x7faddc1078f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.445+0000 7fade2c00700 1 --2- 172.21.15.38:0/790365555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faddc102ef0 0x7faddc1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.445+0000 7fade2c00700 1 -- 172.21.15.38:0/790365555 >> 172.21.15.38:0/790365555 conn(0x7faddc0fe780 msgr2=0x7faddc100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.445+0000 7fade2c00700 1 -- 172.21.15.38:0/790365555 shutdown_connections 2024-03-24T00:53:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.445+0000 7fade2c00700 1 -- 172.21.15.38:0/790365555 wait complete. 2024-03-24T00:53:08.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.446+0000 7fade2c00700 1 Processor -- start 2024-03-24T00:53:08.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.446+0000 7fade2c00700 1 -- start start 2024-03-24T00:53:08.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.446+0000 7fade2c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faddc102ef0 0x7faddc19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:08.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.446+0000 7fade2c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faddc1038a0 0x7faddc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:08.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.447+0000 7fade2c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc19f530 con 0x7faddc102ef0 2024-03-24T00:53:08.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.447+0000 7fade2c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc198b10 con 0x7faddc1038a0 2024-03-24T00:53:08.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.447+0000 7faddbe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faddc102ef0 0x7faddc19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:08.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.447+0000 7faddbe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faddc102ef0 0x7faddc19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44180/0 (socket says 172.21.15.38:44180) 2024-03-24T00:53:08.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.447+0000 7faddbe00700 1 -- 172.21.15.38:0/604684223 learned_addr learned my addr 172.21.15.38:0/604684223 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:08.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.447+0000 7faddb400700 1 --2- 172.21.15.38:0/604684223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faddc1038a0 0x7faddc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:08.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.447+0000 7faddbe00700 1 -- 172.21.15.38:0/604684223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faddc1038a0 msgr2=0x7faddc19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:08.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.447+0000 7faddbe00700 1 --2- 172.21.15.38:0/604684223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faddc1038a0 0x7faddc19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:08.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.447+0000 7faddbe00700 1 -- 172.21.15.38:0/604684223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc80095f0 con 0x7faddc102ef0 2024-03-24T00:53:08.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.448+0000 7faddbe00700 1 --2- 172.21.15.38:0/604684223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faddc102ef0 0x7faddc19e9d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00b4b0 tx=0x7fadcc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:08.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.448+0000 7fadd8c00700 1 -- 172.21.15.38:0/604684223 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadcc011800 con 0x7faddc102ef0 2024-03-24T00:53:08.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.448+0000 7fadd8c00700 1 -- 172.21.15.38:0/604684223 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadcc011e20 con 0x7faddc102ef0 2024-03-24T00:53:08.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.448+0000 7fade2c00700 1 -- 172.21.15.38:0/604684223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc198df0 con 0x7faddc102ef0 2024-03-24T00:53:08.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.448+0000 7fadd8c00700 1 -- 172.21.15.38:0/604684223 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadcc00f2e0 con 0x7faddc102ef0 2024-03-24T00:53:08.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.448+0000 7fade2c00700 1 -- 172.21.15.38:0/604684223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faddc199340 con 0x7faddc102ef0 2024-03-24T00:53:08.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.449+0000 7fade2c00700 1 -- 172.21.15.38:0/604684223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faddc10b220 con 0x7faddc102ef0 2024-03-24T00:53:08.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.450+0000 7fadd8c00700 1 -- 172.21.15.38:0/604684223 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7fadcc010380 con 0x7faddc102ef0 2024-03-24T00:53:08.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.451+0000 7fadd8c00700 1 --2- 172.21.15.38:0/604684223 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fadc006dc20 0x7fadc00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:08.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.451+0000 7fadd8c00700 1 -- 172.21.15.38:0/604684223 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fadcc08d3a0 con 0x7faddc102ef0 2024-03-24T00:53:08.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.452+0000 7faddb400700 1 --2- 172.21.15.38:0/604684223 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fadc006dc20 0x7fadc00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:08.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.452+0000 7faddb400700 1 --2- 172.21.15.38:0/604684223 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fadc006dc20 0x7fadc00700e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fadc80059d0 tx=0x7fadc8005960 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.456+0000 7fadd8c00700 1 -- 172.21.15.38:0/604684223 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fadcc05a8b0 con 0x7faddc102ef0 2024-03-24T00:53:08.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:08 smithi038 ceph-mon[29271]: pgmap v117: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:08.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:08 smithi088 ceph-mon[36610]: pgmap v117: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:08.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:08.711+0000 7fade2c00700 1 -- 172.21.15.38:0/604684223 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "target": ["mon-mgr", ""]}) v1 -- 0x7faddc199fd0 con 0x7fadc006dc20 2024-03-24T00:53:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:09 smithi088 ceph-mon[36610]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:09 smithi088 ceph-mon[36610]: pgmap v118: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:10.334 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:09 smithi038 ceph-mon[29271]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:10.334 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:09 smithi038 ceph-mon[29271]: pgmap v118: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:11.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.539+0000 7fadd8c00700 1 -- 172.21.15.38:0/604684223 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5740 (secure 0 0 0) 0x7faddc199fd0 con 0x7fadc006dc20 2024-03-24T00:53:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.542+0000 7fade2c00700 1 -- 172.21.15.38:0/604684223 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fadc006dc20 msgr2=0x7fadc00700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.542+0000 7fade2c00700 1 --2- 172.21.15.38:0/604684223 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fadc006dc20 0x7fadc00700e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fadc80059d0 tx=0x7fadc8005960 comp rx=0 tx=0).stop 2024-03-24T00:53:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.542+0000 7fade2c00700 1 -- 172.21.15.38:0/604684223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faddc102ef0 msgr2=0x7faddc19e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.542+0000 7fade2c00700 1 --2- 172.21.15.38:0/604684223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faddc102ef0 0x7faddc19e9d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00b4b0 tx=0x7fadcc00b870 comp rx=0 tx=0).stop 2024-03-24T00:53:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.543+0000 7fade2c00700 1 -- 172.21.15.38:0/604684223 shutdown_connections 2024-03-24T00:53:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.543+0000 7fade2c00700 1 --2- 172.21.15.38:0/604684223 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fadc006dc20 0x7fadc00700e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.543+0000 7fade2c00700 1 --2- 172.21.15.38:0/604684223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faddc102ef0 0x7faddc19e9d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.543+0000 7fade2c00700 1 --2- 172.21.15.38:0/604684223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faddc1038a0 0x7faddc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:11.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.543+0000 7fade2c00700 1 -- 172.21.15.38:0/604684223 >> 172.21.15.38:0/604684223 conn(0x7faddc0fe780 msgr2=0x7faddc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:11.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.543+0000 7fade2c00700 1 -- 172.21.15.38:0/604684223 shutdown_connections 2024-03-24T00:53:11.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:11.543+0000 7fade2c00700 1 -- 172.21.15.38:0/604684223 wait complete. 2024-03-24T00:53:11.561 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:53:12.323 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph orch ps' 2024-03-24T00:53:12.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:12 smithi038 ceph-mon[29271]: pgmap v119: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:12.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:12 smithi088 ceph-mon[36610]: pgmap v119: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:12.760 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:53:14.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:14 smithi038 ceph-mon[29271]: pgmap v120: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:14.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:14 smithi088 ceph-mon[36610]: pgmap v120: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:14.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.894+0000 7faa9b200700 1 -- 172.21.15.38:0/341135886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94103880 msgr2=0x7faa941078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:14.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.894+0000 7faa9b200700 1 --2- 172.21.15.38:0/341135886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94103880 0x7faa941078d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faa80009910 tx=0x7faa80009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:14.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.894+0000 7faa9b200700 1 -- 172.21.15.38:0/341135886 shutdown_connections 2024-03-24T00:53:14.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.894+0000 7faa9b200700 1 --2- 172.21.15.38:0/341135886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94103880 0x7faa941078d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:14.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.894+0000 7faa9b200700 1 --2- 172.21.15.38:0/341135886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa94102ed0 0x7faa941032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:14.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.894+0000 7faa9b200700 1 -- 172.21.15.38:0/341135886 >> 172.21.15.38:0/341135886 conn(0x7faa940fe780 msgr2=0x7faa94100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:14.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.895+0000 7faa9b200700 1 -- 172.21.15.38:0/341135886 shutdown_connections 2024-03-24T00:53:14.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.895+0000 7faa9b200700 1 -- 172.21.15.38:0/341135886 wait complete. 2024-03-24T00:53:14.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.896+0000 7faa9b200700 1 Processor -- start 2024-03-24T00:53:14.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.896+0000 7faa9b200700 1 -- start start 2024-03-24T00:53:14.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.896+0000 7faa9b200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa94102ed0 0x7faa94198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:14.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.896+0000 7faa9b200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94103880 0x7faa94199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:14.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.896+0000 7faa9b200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa941998d0 con 0x7faa94103880 2024-03-24T00:53:14.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.896+0000 7faa9b200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9419c540 con 0x7faa94102ed0 2024-03-24T00:53:14.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.896+0000 7faa93400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94103880 0x7faa94199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:14.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.897+0000 7faa93400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94103880 0x7faa94199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47184/0 (socket says 172.21.15.38:47184) 2024-03-24T00:53:14.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.897+0000 7faa93400700 1 -- 172.21.15.38:0/3167202558 learned_addr learned my addr 172.21.15.38:0/3167202558 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:14.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.897+0000 7faa93e00700 1 --2- 172.21.15.38:0/3167202558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa94102ed0 0x7faa94198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:14.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.897+0000 7faa93400700 1 -- 172.21.15.38:0/3167202558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa94102ed0 msgr2=0x7faa94198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:14.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.897+0000 7faa93400700 1 --2- 172.21.15.38:0/3167202558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa94102ed0 0x7faa94198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:14.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.897+0000 7faa93400700 1 -- 172.21.15.38:0/3167202558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa800095f0 con 0x7faa94103880 2024-03-24T00:53:14.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.897+0000 7faa93400700 1 --2- 172.21.15.38:0/3167202558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94103880 0x7faa94199240 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faa80005b40 tx=0x7faa8000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:14.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.898+0000 7faa90c00700 1 -- 172.21.15.38:0/3167202558 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa8001d070 con 0x7faa94103880 2024-03-24T00:53:14.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.898+0000 7faa90c00700 1 -- 172.21.15.38:0/3167202558 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa8000f460 con 0x7faa94103880 2024-03-24T00:53:14.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.898+0000 7faa9b200700 1 -- 172.21.15.38:0/3167202558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9419c7c0 con 0x7faa94103880 2024-03-24T00:53:14.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.898+0000 7faa9b200700 1 -- 172.21.15.38:0/3167202558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9419ccb0 con 0x7faa94103880 2024-03-24T00:53:14.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.898+0000 7faa90c00700 1 -- 172.21.15.38:0/3167202558 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa800225f0 con 0x7faa94103880 2024-03-24T00:53:14.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.899+0000 7faa9b200700 1 -- 172.21.15.38:0/3167202558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9410b220 con 0x7faa94103880 2024-03-24T00:53:14.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.900+0000 7faa90c00700 1 -- 172.21.15.38:0/3167202558 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7faa80003a60 con 0x7faa94103880 2024-03-24T00:53:14.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.901+0000 7faa90c00700 1 --2- 172.21.15.38:0/3167202558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa7806dc30 0x7faa780700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:14.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.901+0000 7faa90c00700 1 -- 172.21.15.38:0/3167202558 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faa8008ee60 con 0x7faa94103880 2024-03-24T00:53:14.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.902+0000 7faa93e00700 1 --2- 172.21.15.38:0/3167202558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa7806dc30 0x7faa780700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:14.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.902+0000 7faa93e00700 1 --2- 172.21.15.38:0/3167202558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa7806dc30 0x7faa780700f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa84006fd0 tx=0x7faa84008040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:14.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:14.908+0000 7faa90c00700 1 -- 172.21.15.38:0/3167202558 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7faa80058710 con 0x7faa94103880 2024-03-24T00:53:15.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.146+0000 7faa9b200700 1 -- 172.21.15.38:0/3167202558 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faa9419a040 con 0x7faa7806dc30 2024-03-24T00:53:15.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.182+0000 7faa90c00700 1 -- 172.21.15.38:0/3167202558 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7faa9419a040 con 0x7faa7806dc30 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (97s) 48s ago 15m 14.1M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (15m) 48s ago 15m 8296k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (14m) 2m ago 14m 7922k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (15m) 48s ago 15m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (14m) 2m ago 14m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (56s) 48s ago 15m 52.4M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (8m) 48s ago 8m 97.8M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (8m) 48s ago 8m 151M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (8m) 2m ago 8m 24.4M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (8m) 2m ago 8m 24.3M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (4m) 48s ago 17m 502M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:53:15.184 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (3m) 2m ago 14m 447M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:53:15.185 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (17m) 48s ago 17m 53.1M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:53:15.185 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (14m) 2m ago 14m 44.5M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:53:15.185 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (2m) 48s ago 15m 7482k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:53:15.185 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (2m) 2m ago 14m 6753k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:53:15.185 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (13m) 48s ago 13m 255M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:53:15.185 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (12m) 48s ago 12m 270M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:53:15.185 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (11m) 48s ago 11m 226M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:53:15.185 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (11m) 2m ago 11m 274M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:53:15.185 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (10m) 2m ago 10m 258M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:53:15.185 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (10m) 2m ago 10m 222M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:53:15.185 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (2m) 48s ago 14m 55.6M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:53:15.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.186+0000 7faa9b200700 1 -- 172.21.15.38:0/3167202558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa7806dc30 msgr2=0x7faa780700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:15.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.186+0000 7faa9b200700 1 --2- 172.21.15.38:0/3167202558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa7806dc30 0x7faa780700f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa84006fd0 tx=0x7faa84008040 comp rx=0 tx=0).stop 2024-03-24T00:53:15.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.186+0000 7faa9b200700 1 -- 172.21.15.38:0/3167202558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94103880 msgr2=0x7faa94199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:15.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.186+0000 7faa9b200700 1 --2- 172.21.15.38:0/3167202558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94103880 0x7faa94199240 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faa80005b40 tx=0x7faa8000be20 comp rx=0 tx=0).stop 2024-03-24T00:53:15.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.186+0000 7faa9b200700 1 -- 172.21.15.38:0/3167202558 shutdown_connections 2024-03-24T00:53:15.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.186+0000 7faa9b200700 1 --2- 172.21.15.38:0/3167202558 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7faa7806dc30 0x7faa780700f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:15.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.186+0000 7faa9b200700 1 --2- 172.21.15.38:0/3167202558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa94102ed0 0x7faa94198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:15.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.186+0000 7faa9b200700 1 --2- 172.21.15.38:0/3167202558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa94103880 0x7faa94199240 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:15.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.186+0000 7faa9b200700 1 -- 172.21.15.38:0/3167202558 >> 172.21.15.38:0/3167202558 conn(0x7faa940fe780 msgr2=0x7faa940ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:15.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.187+0000 7faa9b200700 1 -- 172.21.15.38:0/3167202558 shutdown_connections 2024-03-24T00:53:15.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:15.187+0000 7faa9b200700 1 -- 172.21.15.38:0/3167202558 wait complete. 2024-03-24T00:53:15.923 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-24T00:53:15.924 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:53:15.924 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-03-24T00:53:16.372 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:53:16.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:16 smithi038 ceph-mon[29271]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:16.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:16 smithi038 ceph-mon[29271]: pgmap v121: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:16 smithi088 ceph-mon[36610]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:16 smithi088 ceph-mon[36610]: pgmap v121: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.533+0000 7fc48f800700 1 -- 172.21.15.38:0/2329243076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc488106a70 msgr2=0x7fc488106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.533+0000 7fc48f800700 1 --2- 172.21.15.38:0/2329243076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc488106a70 0x7fc488106e50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc47800b0a0 tx=0x7fc47800b3b0 comp rx=0 tx=0).stop 2024-03-24T00:53:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.533+0000 7fc48f800700 1 -- 172.21.15.38:0/2329243076 shutdown_connections 2024-03-24T00:53:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.533+0000 7fc48f800700 1 --2- 172.21.15.38:0/2329243076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc488100a70 0x7fc488100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.533+0000 7fc48f800700 1 --2- 172.21.15.38:0/2329243076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc488106a70 0x7fc488106e50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.533+0000 7fc48f800700 1 -- 172.21.15.38:0/2329243076 >> 172.21.15.38:0/2329243076 conn(0x7fc4880fc5a0 msgr2=0x7fc4880fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.534+0000 7fc48f800700 1 -- 172.21.15.38:0/2329243076 shutdown_connections 2024-03-24T00:53:18.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.534+0000 7fc48f800700 1 -- 172.21.15.38:0/2329243076 wait complete. 2024-03-24T00:53:18.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.535+0000 7fc48f800700 1 Processor -- start 2024-03-24T00:53:18.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.535+0000 7fc48f800700 1 -- start start 2024-03-24T00:53:18.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.535+0000 7fc48f800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc488100a70 0x7fc48819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:18.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.535+0000 7fc48f800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc488106a70 0x7fc48819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:18.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.535+0000 7fc48f800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881989a0 con 0x7fc488100a70 2024-03-24T00:53:18.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.535+0000 7fc48f800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc488198b10 con 0x7fc488106a70 2024-03-24T00:53:18.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.536+0000 7fc48ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc488100a70 0x7fc48819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:18.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.536+0000 7fc48ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc488100a70 0x7fc48819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47198/0 (socket says 172.21.15.38:47198) 2024-03-24T00:53:18.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.536+0000 7fc48ce00700 1 -- 172.21.15.38:0/1240065372 learned_addr learned my addr 172.21.15.38:0/1240065372 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:18.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.536+0000 7fc47fe00700 1 --2- 172.21.15.38:0/1240065372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc488106a70 0x7fc48819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:18.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.536+0000 7fc48ce00700 1 -- 172.21.15.38:0/1240065372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc488106a70 msgr2=0x7fc48819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:18.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.536+0000 7fc48ce00700 1 --2- 172.21.15.38:0/1240065372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc488106a70 0x7fc48819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:18.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.536+0000 7fc48ce00700 1 -- 172.21.15.38:0/1240065372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc478009d00 con 0x7fc488100a70 2024-03-24T00:53:18.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.537+0000 7fc48ce00700 1 --2- 172.21.15.38:0/1240065372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc488100a70 0x7fc48819e8f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc478000f80 tx=0x7fc478007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:18.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.537+0000 7fc486000700 1 -- 172.21.15.38:0/1240065372 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc47800e040 con 0x7fc488100a70 2024-03-24T00:53:18.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.537+0000 7fc486000700 1 -- 172.21.15.38:0/1240065372 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc478003c10 con 0x7fc488100a70 2024-03-24T00:53:18.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.537+0000 7fc486000700 1 -- 172.21.15.38:0/1240065372 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc47801d6b0 con 0x7fc488100a70 2024-03-24T00:53:18.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.537+0000 7fc48f800700 1 -- 172.21.15.38:0/1240065372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc488198d90 con 0x7fc488100a70 2024-03-24T00:53:18.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.537+0000 7fc48f800700 1 -- 172.21.15.38:0/1240065372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc488199280 con 0x7fc488100a70 2024-03-24T00:53:18.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.538+0000 7fc48f800700 1 -- 172.21.15.38:0/1240065372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc488104d20 con 0x7fc488100a70 2024-03-24T00:53:18.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.539+0000 7fc486000700 1 -- 172.21.15.38:0/1240065372 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7fc478004040 con 0x7fc488100a70 2024-03-24T00:53:18.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.541+0000 7fc486000700 1 --2- 172.21.15.38:0/1240065372 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fc46c06dc20 0x7fc46c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:18.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.541+0000 7fc486000700 1 -- 172.21.15.38:0/1240065372 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc47808ef90 con 0x7fc488100a70 2024-03-24T00:53:18.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.541+0000 7fc47fe00700 1 --2- 172.21.15.38:0/1240065372 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fc46c06dc20 0x7fc46c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:18.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.542+0000 7fc47fe00700 1 --2- 172.21.15.38:0/1240065372 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fc46c06dc20 0x7fc46c0700e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc470005950 tx=0x7fc4700058e0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:18.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.545+0000 7fc486000700 1 -- 172.21.15.38:0/1240065372 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fc4780588c0 con 0x7fc488100a70 2024-03-24T00:53:18.573 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:18 smithi038 ceph-mon[29271]: pgmap v122: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:18.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:18 smithi088 ceph-mon[36610]: pgmap v122: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:18.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.808+0000 7fc48f800700 1 -- 172.21.15.38:0/1240065372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fc488199e80 con 0x7fc488100a70 2024-03-24T00:53:18.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.819+0000 7fc486000700 1 -- 172.21.15.38:0/1240065372 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc4780170c0 con 0x7fc488100a70 2024-03-24T00:53:18.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.822+0000 7fc48f800700 1 -- 172.21.15.38:0/1240065372 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fc46c06dc20 msgr2=0x7fc46c0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.822+0000 7fc48f800700 1 --2- 172.21.15.38:0/1240065372 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fc46c06dc20 0x7fc46c0700e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc470005950 tx=0x7fc4700058e0 comp rx=0 tx=0).stop 2024-03-24T00:53:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.822+0000 7fc48f800700 1 -- 172.21.15.38:0/1240065372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc488100a70 msgr2=0x7fc48819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.822+0000 7fc48f800700 1 --2- 172.21.15.38:0/1240065372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc488100a70 0x7fc48819e8f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc478000f80 tx=0x7fc478007bd0 comp rx=0 tx=0).stop 2024-03-24T00:53:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.823+0000 7fc48f800700 1 -- 172.21.15.38:0/1240065372 shutdown_connections 2024-03-24T00:53:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.823+0000 7fc48f800700 1 --2- 172.21.15.38:0/1240065372 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fc46c06dc20 0x7fc46c0700e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.823+0000 7fc48f800700 1 --2- 172.21.15.38:0/1240065372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc488100a70 0x7fc48819e8f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.823+0000 7fc48f800700 1 --2- 172.21.15.38:0/1240065372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc488106a70 0x7fc48819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:18.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.823+0000 7fc48f800700 1 -- 172.21.15.38:0/1240065372 >> 172.21.15.38:0/1240065372 conn(0x7fc4880fc5a0 msgr2=0x7fc488073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:18.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.823+0000 7fc48f800700 1 -- 172.21.15.38:0/1240065372 shutdown_connections 2024-03-24T00:53:18.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:18.823+0000 7fc48f800700 1 -- 172.21.15.38:0/1240065372 wait complete. 2024-03-24T00:53:19.731 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-24T00:53:19.731 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:53:19.732 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-24T00:53:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:19 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1240065372' entity='client.admin' 2024-03-24T00:53:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:53:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:19 smithi038 ceph-mon[29271]: pgmap v123: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:19 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:53:20.171 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:53:20.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:19 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1240065372' entity='client.admin' 2024-03-24T00:53:20.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:53:20.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:19 smithi088 ceph-mon[36610]: pgmap v123: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:20.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:19 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:53:21.078 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:20 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:53:21.078 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:20 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:53:21.078 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:20 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:21.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:20 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:53:21.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:20 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:53:21.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:20 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:22.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:21 smithi038 ceph-mon[29271]: pgmap v124: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:22.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:21 smithi088 ceph-mon[36610]: pgmap v124: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.356+0000 7f069d000700 1 -- 172.21.15.38:0/1727464585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0698107310 msgr2=0x7f06981076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.356+0000 7f069d000700 1 --2- 172.21.15.38:0/1727464585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0698107310 0x7f06981076f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0688009910 tx=0x7f0688009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:22.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.356+0000 7f069d000700 1 -- 172.21.15.38:0/1727464585 shutdown_connections 2024-03-24T00:53:22.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.356+0000 7f069d000700 1 --2- 172.21.15.38:0/1727464585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0698107cc0 0x7f069810bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:22.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.356+0000 7f069d000700 1 --2- 172.21.15.38:0/1727464585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0698107310 0x7f06981076f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:22.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.356+0000 7f069d000700 1 -- 172.21.15.38:0/1727464585 >> 172.21.15.38:0/1727464585 conn(0x7f06980768e0 msgr2=0x7f0698078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:22.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.357+0000 7f069d000700 1 -- 172.21.15.38:0/1727464585 shutdown_connections 2024-03-24T00:53:22.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.357+0000 7f069d000700 1 -- 172.21.15.38:0/1727464585 wait complete. 2024-03-24T00:53:22.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.358+0000 7f069d000700 1 Processor -- start 2024-03-24T00:53:22.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.358+0000 7f069d000700 1 -- start start 2024-03-24T00:53:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.359+0000 7f069d000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0698107cc0 0x7f069819d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.359+0000 7f069d000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f069819d770 0x7f06981a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.359+0000 7f069d000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069819dd90 con 0x7f069819d770 2024-03-24T00:53:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.359+0000 7f069d000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069819df00 con 0x7f0698107cc0 2024-03-24T00:53:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.359+0000 7f068d400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f069819d770 0x7f06981a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.359+0000 7f068d400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f069819d770 0x7f06981a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47218/0 (socket says 172.21.15.38:47218) 2024-03-24T00:53:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.359+0000 7f068d400700 1 -- 172.21.15.38:0/134471988 learned_addr learned my addr 172.21.15.38:0/134471988 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.360+0000 7f0695e00700 1 --2- 172.21.15.38:0/134471988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0698107cc0 0x7f069819d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.360+0000 7f068d400700 1 -- 172.21.15.38:0/134471988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0698107cc0 msgr2=0x7f069819d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.360+0000 7f068d400700 1 --2- 172.21.15.38:0/134471988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0698107cc0 0x7f069819d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.360+0000 7f068d400700 1 -- 172.21.15.38:0/134471988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06880095f0 con 0x7f069819d770 2024-03-24T00:53:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.360+0000 7f068d400700 1 --2- 172.21.15.38:0/134471988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f069819d770 0x7f06981a1b20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f068000d900 tx=0x7f068000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.361+0000 7f068f400700 1 -- 172.21.15.38:0/134471988 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0680009900 con 0x7f069819d770 2024-03-24T00:53:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.361+0000 7f068f400700 1 -- 172.21.15.38:0/134471988 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0680010460 con 0x7f069819d770 2024-03-24T00:53:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.361+0000 7f068f400700 1 -- 172.21.15.38:0/134471988 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f068000f5f0 con 0x7f069819d770 2024-03-24T00:53:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.361+0000 7f069d000700 1 -- 172.21.15.38:0/134471988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06981a2120 con 0x7f069819d770 2024-03-24T00:53:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.362+0000 7f069d000700 1 -- 172.21.15.38:0/134471988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06981a2670 con 0x7f069819d770 2024-03-24T00:53:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.363+0000 7f069d000700 1 -- 172.21.15.38:0/134471988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06981087f0 con 0x7f069819d770 2024-03-24T00:53:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.364+0000 7f068f400700 1 -- 172.21.15.38:0/134471988 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f06800105d0 con 0x7f069819d770 2024-03-24T00:53:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.365+0000 7f068f400700 1 --2- 172.21.15.38:0/134471988 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f067c06dc30 0x7f067c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.365+0000 7f068f400700 1 -- 172.21.15.38:0/134471988 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f068008cca0 con 0x7f069819d770 2024-03-24T00:53:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.365+0000 7f0695e00700 1 --2- 172.21.15.38:0/134471988 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f067c06dc30 0x7f067c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.366+0000 7f0695e00700 1 --2- 172.21.15.38:0/134471988 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f067c06dc30 0x7f067c0700f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0688005b40 tx=0x7f068801a040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.370+0000 7f068f400700 1 -- 172.21.15.38:0/134471988 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f0680056650 con 0x7f069819d770 2024-03-24T00:53:22.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.607+0000 7f069d000700 1 -- 172.21.15.38:0/134471988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f06981a2910 con 0x7f069819d770 2024-03-24T00:53:22.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.608+0000 7f068f400700 1 -- 172.21.15.38:0/134471988 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7f0680059d10 con 0x7f069819d770 2024-03-24T00:53:22.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.611+0000 7f069d000700 1 -- 172.21.15.38:0/134471988 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f067c06dc30 msgr2=0x7f067c0700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:22.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.611+0000 7f069d000700 1 --2- 172.21.15.38:0/134471988 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f067c06dc30 0x7f067c0700f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0688005b40 tx=0x7f068801a040 comp rx=0 tx=0).stop 2024-03-24T00:53:22.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.611+0000 7f069d000700 1 -- 172.21.15.38:0/134471988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f069819d770 msgr2=0x7f06981a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:22.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.611+0000 7f069d000700 1 --2- 172.21.15.38:0/134471988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f069819d770 0x7f06981a1b20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f068000d900 tx=0x7f068000dc10 comp rx=0 tx=0).stop 2024-03-24T00:53:22.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.611+0000 7f069d000700 1 -- 172.21.15.38:0/134471988 shutdown_connections 2024-03-24T00:53:22.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.611+0000 7f069d000700 1 --2- 172.21.15.38:0/134471988 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f067c06dc30 0x7f067c0700f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:22.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.611+0000 7f069d000700 1 --2- 172.21.15.38:0/134471988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0698107cc0 0x7f069819d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:22.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.611+0000 7f069d000700 1 --2- 172.21.15.38:0/134471988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f069819d770 0x7f06981a1b20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:22.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.611+0000 7f069d000700 1 -- 172.21.15.38:0/134471988 >> 172.21.15.38:0/134471988 conn(0x7f06980768e0 msgr2=0x7f0698077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:22.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.611+0000 7f069d000700 1 -- 172.21.15.38:0/134471988 shutdown_connections 2024-03-24T00:53:22.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:22.612+0000 7f069d000700 1 -- 172.21.15.38:0/134471988 wait complete. 2024-03-24T00:53:23.500 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-24T00:53:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:53:24.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:24 smithi038 ceph-mon[29271]: pgmap v125: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:24 smithi088 ceph-mon[36610]: pgmap v125: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.938+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2513256275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b68107310 msgr2=0x7f8b681076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.938+0000 7f8b6ce00700 1 --2- 172.21.15.38:0/2513256275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b68107310 0x7f8b681076f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009910 tx=0x7f8b58009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.939+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2513256275 shutdown_connections 2024-03-24T00:53:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.939+0000 7f8b6ce00700 1 --2- 172.21.15.38:0/2513256275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b68107cc0 0x7f8b6810bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.939+0000 7f8b6ce00700 1 --2- 172.21.15.38:0/2513256275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b68107310 0x7f8b681076f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.939+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2513256275 >> 172.21.15.38:0/2513256275 conn(0x7f8b680768e0 msgr2=0x7f8b68078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:25.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.939+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2513256275 shutdown_connections 2024-03-24T00:53:25.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.939+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2513256275 wait complete. 2024-03-24T00:53:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.940+0000 7f8b6ce00700 1 Processor -- start 2024-03-24T00:53:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.940+0000 7f8b6ce00700 1 -- start start 2024-03-24T00:53:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.940+0000 7f8b6ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b68107310 0x7f8b6819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.940+0000 7f8b6ce00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b68107cc0 0x7f8b6819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.940+0000 7f8b6ce00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6819dc30 con 0x7f8b68107310 2024-03-24T00:53:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.940+0000 7f8b6ce00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b681a1900 con 0x7f8b68107cc0 2024-03-24T00:53:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.941+0000 7f8b65e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b68107310 0x7f8b6819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.941+0000 7f8b65e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b68107310 0x7f8b6819d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58034/0 (socket says 172.21.15.38:58034) 2024-03-24T00:53:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.941+0000 7f8b65e00700 1 -- 172.21.15.38:0/2323164418 learned_addr learned my addr 172.21.15.38:0/2323164418 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:25.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.941+0000 7f8b65400700 1 --2- 172.21.15.38:0/2323164418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b68107cc0 0x7f8b6819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:25.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.941+0000 7f8b65e00700 1 -- 172.21.15.38:0/2323164418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b68107cc0 msgr2=0x7f8b6819d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:25.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.941+0000 7f8b65e00700 1 --2- 172.21.15.38:0/2323164418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b68107cc0 0x7f8b6819d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:25.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.941+0000 7f8b65e00700 1 -- 172.21.15.38:0/2323164418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580095f0 con 0x7f8b68107310 2024-03-24T00:53:25.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.941+0000 7f8b65e00700 1 --2- 172.21.15.38:0/2323164418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b68107310 0x7f8b6819d010 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005f50 tx=0x7f8b5800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:25.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.942+0000 7f8b5ea00700 1 -- 172.21.15.38:0/2323164418 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5801d070 con 0x7f8b68107310 2024-03-24T00:53:25.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.942+0000 7f8b5ea00700 1 -- 172.21.15.38:0/2323164418 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b580052a0 con 0x7f8b68107310 2024-03-24T00:53:25.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.942+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2323164418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b681a1b80 con 0x7f8b68107310 2024-03-24T00:53:25.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.942+0000 7f8b5ea00700 1 -- 172.21.15.38:0/2323164418 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b58017400 con 0x7f8b68107310 2024-03-24T00:53:25.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.942+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2323164418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b681a2070 con 0x7f8b68107310 2024-03-24T00:53:25.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.943+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2323164418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6810f6b0 con 0x7f8b68107310 2024-03-24T00:53:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.944+0000 7f8b5ea00700 1 -- 172.21.15.38:0/2323164418 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f8b5800f7a0 con 0x7f8b68107310 2024-03-24T00:53:25.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.945+0000 7f8b5ea00700 1 --2- 172.21.15.38:0/2323164418 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f8b4c06dc30 0x7f8b4c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:25.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.945+0000 7f8b5ea00700 1 -- 172.21.15.38:0/2323164418 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8b5808ed00 con 0x7f8b68107310 2024-03-24T00:53:25.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.945+0000 7f8b65400700 1 --2- 172.21.15.38:0/2323164418 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f8b4c06dc30 0x7f8b4c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:25.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.946+0000 7f8b65400700 1 --2- 172.21.15.38:0/2323164418 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f8b4c06dc30 0x7f8b4c0700f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b6819e570 tx=0x7f8b54009260 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:25.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:25.950+0000 7f8b5ea00700 1 -- 172.21.15.38:0/2323164418 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f8b580585b0 con 0x7f8b68107310 2024-03-24T00:53:26.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.181+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2323164418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f8b681087f0 con 0x7f8b68107310 2024-03-24T00:53:26.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.182+0000 7f8b5ea00700 1 -- 172.21.15.38:0/2323164418 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7f8b5805bc70 con 0x7f8b68107310 2024-03-24T00:53:26.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.185+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2323164418 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f8b4c06dc30 msgr2=0x7f8b4c0700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:26.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.185+0000 7f8b6ce00700 1 --2- 172.21.15.38:0/2323164418 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f8b4c06dc30 0x7f8b4c0700f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b6819e570 tx=0x7f8b54009260 comp rx=0 tx=0).stop 2024-03-24T00:53:26.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.185+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2323164418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b68107310 msgr2=0x7f8b6819d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:26.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.185+0000 7f8b6ce00700 1 --2- 172.21.15.38:0/2323164418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b68107310 0x7f8b6819d010 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005f50 tx=0x7f8b5800b650 comp rx=0 tx=0).stop 2024-03-24T00:53:26.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.185+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2323164418 shutdown_connections 2024-03-24T00:53:26.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.185+0000 7f8b6ce00700 1 --2- 172.21.15.38:0/2323164418 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f8b4c06dc30 0x7f8b4c0700f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:26.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.185+0000 7f8b6ce00700 1 --2- 172.21.15.38:0/2323164418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b68107310 0x7f8b6819d010 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:26.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.186+0000 7f8b6ce00700 1 --2- 172.21.15.38:0/2323164418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b68107cc0 0x7f8b6819d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:26.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.186+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2323164418 >> 172.21.15.38:0/2323164418 conn(0x7f8b680768e0 msgr2=0x7f8b68076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:26.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.186+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2323164418 shutdown_connections 2024-03-24T00:53:26.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:26.186+0000 7f8b6ce00700 1 -- 172.21.15.38:0/2323164418 wait complete. 2024-03-24T00:53:26.575 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:26 smithi038 ceph-mon[29271]: pgmap v126: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:26.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:26 smithi088 ceph-mon[36610]: pgmap v126: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:27.066 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-24T00:53:27.526 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:53:28.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:28 smithi038 ceph-mon[29271]: pgmap v127: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:28.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:28 smithi088 ceph-mon[36610]: pgmap v127: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.576+0000 7f08e0c00700 1 -- 172.21.15.38:0/2388355282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc107cc0 msgr2=0x7f08dc10bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.576+0000 7f08e0c00700 1 --2- 172.21.15.38:0/2388355282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc107cc0 0x7f08dc10bd10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009910 tx=0x7f08c8009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.576+0000 7f08e0c00700 1 -- 172.21.15.38:0/2388355282 shutdown_connections 2024-03-24T00:53:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.576+0000 7f08e0c00700 1 --2- 172.21.15.38:0/2388355282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc107cc0 0x7f08dc10bd10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.576+0000 7f08e0c00700 1 --2- 172.21.15.38:0/2388355282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08dc107310 0x7f08dc1076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.576+0000 7f08e0c00700 1 -- 172.21.15.38:0/2388355282 >> 172.21.15.38:0/2388355282 conn(0x7f08dc0768e0 msgr2=0x7f08dc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.577+0000 7f08e0c00700 1 -- 172.21.15.38:0/2388355282 shutdown_connections 2024-03-24T00:53:29.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.577+0000 7f08e0c00700 1 -- 172.21.15.38:0/2388355282 wait complete. 2024-03-24T00:53:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.578+0000 7f08e0c00700 1 Processor -- start 2024-03-24T00:53:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.578+0000 7f08e0c00700 1 -- start start 2024-03-24T00:53:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.578+0000 7f08e0c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc107310 0x7f08dc19d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.578+0000 7f08e0c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08dc107cc0 0x7f08dc19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.578+0000 7f08e0c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc19dcc0 con 0x7f08dc107310 2024-03-24T00:53:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.578+0000 7f08e0c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc1a1990 con 0x7f08dc107cc0 2024-03-24T00:53:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.579+0000 7f08d9e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc107310 0x7f08dc19d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.579+0000 7f08d9e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc107310 0x7f08dc19d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58048/0 (socket says 172.21.15.38:58048) 2024-03-24T00:53:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.579+0000 7f08d9e00700 1 -- 172.21.15.38:0/2043546851 learned_addr learned my addr 172.21.15.38:0/2043546851 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.579+0000 7f08d9400700 1 --2- 172.21.15.38:0/2043546851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08dc107cc0 0x7f08dc19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.579+0000 7f08d9e00700 1 -- 172.21.15.38:0/2043546851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08dc107cc0 msgr2=0x7f08dc19d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.579+0000 7f08d9e00700 1 --2- 172.21.15.38:0/2043546851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08dc107cc0 0x7f08dc19d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.579+0000 7f08d9e00700 1 -- 172.21.15.38:0/2043546851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c80095f0 con 0x7f08dc107310 2024-03-24T00:53:29.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.580+0000 7f08d9e00700 1 --2- 172.21.15.38:0/2043546851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc107310 0x7f08dc19d0a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00ada0 tx=0x7f08cc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.580+0000 7f08d2a00700 1 -- 172.21.15.38:0/2043546851 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08cc00f070 con 0x7f08dc107310 2024-03-24T00:53:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.580+0000 7f08d2a00700 1 -- 172.21.15.38:0/2043546851 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08cc007bf0 con 0x7f08dc107310 2024-03-24T00:53:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.580+0000 7f08d2a00700 1 -- 172.21.15.38:0/2043546851 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08cc005690 con 0x7f08dc107310 2024-03-24T00:53:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.580+0000 7f08e0c00700 1 -- 172.21.15.38:0/2043546851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc1a1c70 con 0x7f08dc107310 2024-03-24T00:53:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.580+0000 7f08e0c00700 1 -- 172.21.15.38:0/2043546851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc1a21c0 con 0x7f08dc107310 2024-03-24T00:53:29.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.581+0000 7f08e0c00700 1 -- 172.21.15.38:0/2043546851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc10f710 con 0x7f08dc107310 2024-03-24T00:53:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.582+0000 7f08d2a00700 1 -- 172.21.15.38:0/2043546851 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f08cc005870 con 0x7f08dc107310 2024-03-24T00:53:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.583+0000 7f08d2a00700 1 --2- 172.21.15.38:0/2043546851 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f08c006dc30 0x7f08c00700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.584+0000 7f08d2a00700 1 -- 172.21.15.38:0/2043546851 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f08cc08d1c0 con 0x7f08dc107310 2024-03-24T00:53:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.584+0000 7f08d9400700 1 --2- 172.21.15.38:0/2043546851 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f08c006dc30 0x7f08c00700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:29.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.585+0000 7f08d9400700 1 --2- 172.21.15.38:0/2043546851 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f08c006dc30 0x7f08c00700f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f08dc19e600 tx=0x7f08c801a040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:29.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.589+0000 7f08d2a00700 1 -- 172.21.15.38:0/2043546851 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f08cc056af0 con 0x7f08dc107310 2024-03-24T00:53:29.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.828+0000 7f08e0c00700 1 -- 172.21.15.38:0/2043546851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f08dc19e420 con 0x7f08dc107310 2024-03-24T00:53:29.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.829+0000 7f08d2a00700 1 -- 172.21.15.38:0/2043546851 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7f08cc05a1b0 con 0x7f08dc107310 2024-03-24T00:53:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.832+0000 7f08e0c00700 1 -- 172.21.15.38:0/2043546851 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f08c006dc30 msgr2=0x7f08c00700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.832+0000 7f08e0c00700 1 --2- 172.21.15.38:0/2043546851 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f08c006dc30 0x7f08c00700f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f08dc19e600 tx=0x7f08c801a040 comp rx=0 tx=0).stop 2024-03-24T00:53:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.832+0000 7f08e0c00700 1 -- 172.21.15.38:0/2043546851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc107310 msgr2=0x7f08dc19d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.832+0000 7f08e0c00700 1 --2- 172.21.15.38:0/2043546851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc107310 0x7f08dc19d0a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00ada0 tx=0x7f08cc00c330 comp rx=0 tx=0).stop 2024-03-24T00:53:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.832+0000 7f08e0c00700 1 -- 172.21.15.38:0/2043546851 shutdown_connections 2024-03-24T00:53:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.832+0000 7f08e0c00700 1 --2- 172.21.15.38:0/2043546851 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f08c006dc30 0x7f08c00700f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.832+0000 7f08e0c00700 1 --2- 172.21.15.38:0/2043546851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc107310 0x7f08dc19d0a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.832+0000 7f08e0c00700 1 --2- 172.21.15.38:0/2043546851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08dc107cc0 0x7f08dc19d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.832+0000 7f08e0c00700 1 -- 172.21.15.38:0/2043546851 >> 172.21.15.38:0/2043546851 conn(0x7f08dc0768e0 msgr2=0x7f08dc076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.832+0000 7f08e0c00700 1 -- 172.21.15.38:0/2043546851 shutdown_connections 2024-03-24T00:53:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:29.833+0000 7f08e0c00700 1 -- 172.21.15.38:0/2043546851 wait complete. 2024-03-24T00:53:30.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:29 smithi038 ceph-mon[29271]: pgmap v128: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:29 smithi088 ceph-mon[36610]: pgmap v128: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:30.592 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-24T00:53:31.058 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:53:32.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:32 smithi038 ceph-mon[29271]: pgmap v129: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:32.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:32 smithi088 ceph-mon[36610]: pgmap v129: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:33.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.217+0000 7f50b9c00700 1 -- 172.21.15.38:0/2812812389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50b4107310 msgr2=0x7f50b41076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:33.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.217+0000 7f50b9c00700 1 --2- 172.21.15.38:0/2812812389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50b4107310 0x7f50b41076f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009910 tx=0x7f50a4009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:33.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.217+0000 7f50b9c00700 1 -- 172.21.15.38:0/2812812389 shutdown_connections 2024-03-24T00:53:33.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.217+0000 7f50b9c00700 1 --2- 172.21.15.38:0/2812812389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b4107cc0 0x7f50b410bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:33.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.217+0000 7f50b9c00700 1 --2- 172.21.15.38:0/2812812389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50b4107310 0x7f50b41076f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:33.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.217+0000 7f50b9c00700 1 -- 172.21.15.38:0/2812812389 >> 172.21.15.38:0/2812812389 conn(0x7f50b40768e0 msgr2=0x7f50b4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:33.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.217+0000 7f50b9c00700 1 -- 172.21.15.38:0/2812812389 shutdown_connections 2024-03-24T00:53:33.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.217+0000 7f50b9c00700 1 -- 172.21.15.38:0/2812812389 wait complete. 2024-03-24T00:53:33.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.218+0000 7f50b9c00700 1 Processor -- start 2024-03-24T00:53:33.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.218+0000 7f50b9c00700 1 -- start start 2024-03-24T00:53:33.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.219+0000 7f50b9c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b4107310 0x7f50b419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:33.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.219+0000 7f50b9c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50b4107cc0 0x7f50b419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:33.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.219+0000 7f50b9c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b419dc50 con 0x7f50b4107cc0 2024-03-24T00:53:33.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.219+0000 7f50b9c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41a18d0 con 0x7f50b4107310 2024-03-24T00:53:33.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.219+0000 7f50b2000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50b4107cc0 0x7f50b419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:33.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.219+0000 7f50b2000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50b4107cc0 0x7f50b419d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44466/0 (socket says 172.21.15.38:44466) 2024-03-24T00:53:33.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.219+0000 7f50b2000700 1 -- 172.21.15.38:0/2475764311 learned_addr learned my addr 172.21.15.38:0/2475764311 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.219+0000 7f50b2a00700 1 --2- 172.21.15.38:0/2475764311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b4107310 0x7f50b419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.220+0000 7f50b2000700 1 -- 172.21.15.38:0/2475764311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b4107310 msgr2=0x7f50b419d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.220+0000 7f50b2000700 1 --2- 172.21.15.38:0/2475764311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b4107310 0x7f50b419d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.220+0000 7f50b2000700 1 -- 172.21.15.38:0/2475764311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a40095f0 con 0x7f50b4107cc0 2024-03-24T00:53:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.220+0000 7f50b2000700 1 --2- 172.21.15.38:0/2475764311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50b4107cc0 0x7f50b419d570 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f50a000c8f0 tx=0x7f50a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.220+0000 7f50ab400700 1 -- 172.21.15.38:0/2475764311 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a0007a50 con 0x7f50b4107cc0 2024-03-24T00:53:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.220+0000 7f50ab400700 1 -- 172.21.15.38:0/2475764311 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50a0007bb0 con 0x7f50b4107cc0 2024-03-24T00:53:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.220+0000 7f50ab400700 1 -- 172.21.15.38:0/2475764311 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a0018630 con 0x7f50b4107cc0 2024-03-24T00:53:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.220+0000 7f50b9c00700 1 -- 172.21.15.38:0/2475764311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b41a1bb0 con 0x7f50b4107cc0 2024-03-24T00:53:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.220+0000 7f50b9c00700 1 -- 172.21.15.38:0/2475764311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b41a2100 con 0x7f50b4107cc0 2024-03-24T00:53:33.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.222+0000 7f50b9c00700 1 -- 172.21.15.38:0/2475764311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b410f6b0 con 0x7f50b4107cc0 2024-03-24T00:53:33.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.223+0000 7f50ab400700 1 -- 172.21.15.38:0/2475764311 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f50a0007d20 con 0x7f50b4107cc0 2024-03-24T00:53:33.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.223+0000 7f50ab400700 1 --2- 172.21.15.38:0/2475764311 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f509806dc20 0x7f50980700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:33.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.224+0000 7f50ab400700 1 -- 172.21.15.38:0/2475764311 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f50a008d780 con 0x7f50b4107cc0 2024-03-24T00:53:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.224+0000 7f50b2a00700 1 --2- 172.21.15.38:0/2475764311 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f509806dc20 0x7f50980700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.225+0000 7f50b2a00700 1 --2- 172.21.15.38:0/2475764311 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f509806dc20 0x7f50980700e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50a4005f50 tx=0x7f50a401a040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.228+0000 7f50ab400700 1 -- 172.21.15.38:0/2475764311 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f50a00570b0 con 0x7f50b4107cc0 2024-03-24T00:53:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.476+0000 7f50b9c00700 1 -- 172.21.15.38:0/2475764311 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "target": ["mon-mgr", ""]}) v1 -- 0x7f50b419e360 con 0x7f509806dc20 2024-03-24T00:53:33.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.490+0000 7f50ab400700 1 -- 172.21.15.38:0/2475764311 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f50b419e360 con 0x7f509806dc20 2024-03-24T00:53:33.492 INFO:teuthology.orchestra.run.smithi038.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:53:33.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.493+0000 7f50b9c00700 1 -- 172.21.15.38:0/2475764311 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f509806dc20 msgr2=0x7f50980700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:33.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.493+0000 7f50b9c00700 1 --2- 172.21.15.38:0/2475764311 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f509806dc20 0x7f50980700e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50a4005f50 tx=0x7f50a401a040 comp rx=0 tx=0).stop 2024-03-24T00:53:33.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.493+0000 7f50b9c00700 1 -- 172.21.15.38:0/2475764311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50b4107cc0 msgr2=0x7f50b419d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:33.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.493+0000 7f50b9c00700 1 --2- 172.21.15.38:0/2475764311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50b4107cc0 0x7f50b419d570 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f50a000c8f0 tx=0x7f50a000ccb0 comp rx=0 tx=0).stop 2024-03-24T00:53:33.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.493+0000 7f50b9c00700 1 -- 172.21.15.38:0/2475764311 shutdown_connections 2024-03-24T00:53:33.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.493+0000 7f50b9c00700 1 --2- 172.21.15.38:0/2475764311 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f509806dc20 0x7f50980700e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:33.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.493+0000 7f50b9c00700 1 --2- 172.21.15.38:0/2475764311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50b4107310 0x7f50b419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:33.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.493+0000 7f50b9c00700 1 --2- 172.21.15.38:0/2475764311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50b4107cc0 0x7f50b419d570 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:33.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.493+0000 7f50b9c00700 1 -- 172.21.15.38:0/2475764311 >> 172.21.15.38:0/2475764311 conn(0x7f50b40768e0 msgr2=0x7f50b4076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:33.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.494+0000 7f50b9c00700 1 -- 172.21.15.38:0/2475764311 shutdown_connections 2024-03-24T00:53:33.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:33.494+0000 7f50b9c00700 1 -- 172.21.15.38:0/2475764311 wait complete. 2024-03-24T00:53:34.363 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-24T00:53:34.363 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T00:53:34.364 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-24T00:53:34.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:34 smithi038 ceph-mon[29271]: pgmap v130: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:34.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:34 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:34.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:34 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:53:34.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:34 smithi088 ceph-mon[36610]: pgmap v130: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:34.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:34 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:34.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:34 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:53:34.781 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T00:53:35.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:35 smithi038 ceph-mon[29271]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:35.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:35 smithi038 ceph-mon[29271]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:53:35.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:53:35.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:53:35.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:53:35.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:35 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:35 smithi088 ceph-mon[36610]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:35 smithi088 ceph-mon[36610]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:53:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:53:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:53:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:53:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:35 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:36.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:36 smithi038 ceph-mon[29271]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:53:36.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:36 smithi038 ceph-mon[29271]: pgmap v131: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:36.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:36 smithi088 ceph-mon[36610]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-24T00:53:36.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:36 smithi088 ceph-mon[36610]: pgmap v131: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:53:37.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.078+0000 7f5634a00700 1 -- 172.21.15.38:0/2540322193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5630102ad0 msgr2=0x7f5630102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:37.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.078+0000 7f5634a00700 1 --2- 172.21.15.38:0/2540322193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5630102ad0 0x7f5630102f50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f561c009910 tx=0x7f561c009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:37.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.078+0000 7f5634a00700 1 -- 172.21.15.38:0/2540322193 shutdown_connections 2024-03-24T00:53:37.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.078+0000 7f5634a00700 1 --2- 172.21.15.38:0/2540322193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5630102ad0 0x7f5630102f50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.078+0000 7f5634a00700 1 --2- 172.21.15.38:0/2540322193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5630108b60 0x7f5630108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.078+0000 7f5634a00700 1 -- 172.21.15.38:0/2540322193 >> 172.21.15.38:0/2540322193 conn(0x7f56300fe5e0 msgr2=0x7f5630100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:37.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.078+0000 7f5634a00700 1 -- 172.21.15.38:0/2540322193 shutdown_connections 2024-03-24T00:53:37.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.078+0000 7f5634a00700 1 -- 172.21.15.38:0/2540322193 wait complete. 2024-03-24T00:53:37.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.080+0000 7f5634a00700 1 Processor -- start 2024-03-24T00:53:37.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.080+0000 7f5634a00700 1 -- start start 2024-03-24T00:53:37.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.080+0000 7f5634a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5630102ad0 0x7f563019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:37.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.080+0000 7f5634a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5630108b60 0x7f563019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.080+0000 7f5634a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563019f2e0 con 0x7f5630102ad0 2024-03-24T00:53:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.080+0000 7f5634a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5630198850 con 0x7f5630108b60 2024-03-24T00:53:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.080+0000 7f562e800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5630102ad0 0x7f563019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.080+0000 7f562e800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5630102ad0 0x7f563019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44474/0 (socket says 172.21.15.38:44474) 2024-03-24T00:53:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.080+0000 7f562e800700 1 -- 172.21.15.38:0/1786107530 learned_addr learned my addr 172.21.15.38:0/1786107530 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.081+0000 7f562de00700 1 --2- 172.21.15.38:0/1786107530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5630108b60 0x7f563019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.081+0000 7f562e800700 1 -- 172.21.15.38:0/1786107530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5630108b60 msgr2=0x7f563019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.081+0000 7f562e800700 1 --2- 172.21.15.38:0/1786107530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5630108b60 0x7f563019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.081+0000 7f562e800700 1 -- 172.21.15.38:0/1786107530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c0095f0 con 0x7f5630102ad0 2024-03-24T00:53:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.081+0000 7f562e800700 1 --2- 172.21.15.38:0/1786107530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5630102ad0 0x7f563019e710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f562000d8d0 tx=0x7f562000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.082+0000 7f5627400700 1 -- 172.21.15.38:0/1786107530 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5620009940 con 0x7f5630102ad0 2024-03-24T00:53:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.082+0000 7f5627400700 1 -- 172.21.15.38:0/1786107530 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5620010460 con 0x7f5630102ad0 2024-03-24T00:53:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.082+0000 7f5634a00700 1 -- 172.21.15.38:0/1786107530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5630198b30 con 0x7f5630102ad0 2024-03-24T00:53:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.082+0000 7f5627400700 1 -- 172.21.15.38:0/1786107530 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f562000f5f0 con 0x7f5630102ad0 2024-03-24T00:53:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.082+0000 7f5634a00700 1 -- 172.21.15.38:0/1786107530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5630199080 con 0x7f5630102ad0 2024-03-24T00:53:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.083+0000 7f5634a00700 1 -- 172.21.15.38:0/1786107530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563010b0f0 con 0x7f5630102ad0 2024-03-24T00:53:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.084+0000 7f5627400700 1 -- 172.21.15.38:0/1786107530 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f56200105d0 con 0x7f5630102ad0 2024-03-24T00:53:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.086+0000 7f5627400700 1 --2- 172.21.15.38:0/1786107530 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f561406dc30 0x7f56140700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.086+0000 7f5627400700 1 -- 172.21.15.38:0/1786107530 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f562008d780 con 0x7f5630102ad0 2024-03-24T00:53:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.086+0000 7f562de00700 1 --2- 172.21.15.38:0/1786107530 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f561406dc30 0x7f56140700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.087+0000 7f562de00700 1 --2- 172.21.15.38:0/1786107530 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f561406dc30 0x7f56140700f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f561c005b40 tx=0x7f561c01a040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.091+0000 7f5627400700 1 -- 172.21.15.38:0/1786107530 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f562005abe0 con 0x7f5630102ad0 2024-03-24T00:53:37.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.344+0000 7f5634a00700 1 -- 172.21.15.38:0/1786107530 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5630199d80 con 0x7f561406dc30 2024-03-24T00:53:37.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.347+0000 7f5627400700 1 -- 172.21.15.38:0/1786107530 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f5630199d80 con 0x7f561406dc30 2024-03-24T00:53:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.349+0000 7f5634a00700 1 -- 172.21.15.38:0/1786107530 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f561406dc30 msgr2=0x7f56140700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.349+0000 7f5634a00700 1 --2- 172.21.15.38:0/1786107530 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f561406dc30 0x7f56140700f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f561c005b40 tx=0x7f561c01a040 comp rx=0 tx=0).stop 2024-03-24T00:53:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.349+0000 7f5634a00700 1 -- 172.21.15.38:0/1786107530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5630102ad0 msgr2=0x7f563019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.349+0000 7f5634a00700 1 --2- 172.21.15.38:0/1786107530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5630102ad0 0x7f563019e710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f562000d8d0 tx=0x7f562000dc90 comp rx=0 tx=0).stop 2024-03-24T00:53:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.350+0000 7f5634a00700 1 -- 172.21.15.38:0/1786107530 shutdown_connections 2024-03-24T00:53:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.350+0000 7f5634a00700 1 --2- 172.21.15.38:0/1786107530 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f561406dc30 0x7f56140700f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.350+0000 7f5634a00700 1 --2- 172.21.15.38:0/1786107530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5630102ad0 0x7f563019e710 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.350+0000 7f5634a00700 1 --2- 172.21.15.38:0/1786107530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5630108b60 0x7f563019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.350+0000 7f5634a00700 1 -- 172.21.15.38:0/1786107530 >> 172.21.15.38:0/1786107530 conn(0x7f56300fe5e0 msgr2=0x7f56300ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:37.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.350+0000 7f5634a00700 1 -- 172.21.15.38:0/1786107530 shutdown_connections 2024-03-24T00:53:37.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.350+0000 7f5634a00700 1 -- 172.21.15.38:0/1786107530 wait complete. 2024-03-24T00:53:37.368 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:53:37.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.525+0000 7f2813a00700 1 -- 172.21.15.38:0/3521506856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c106a70 msgr2=0x7f280c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:37.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.525+0000 7f2813a00700 1 --2- 172.21.15.38:0/3521506856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c106a70 0x7f280c106e50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009910 tx=0x7f27fc009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:37.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.525+0000 7f2813a00700 1 -- 172.21.15.38:0/3521506856 shutdown_connections 2024-03-24T00:53:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.525+0000 7f2813a00700 1 --2- 172.21.15.38:0/3521506856 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f280c100a70 0x7f280c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.525+0000 7f2813a00700 1 --2- 172.21.15.38:0/3521506856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c106a70 0x7f280c106e50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.525+0000 7f2813a00700 1 -- 172.21.15.38:0/3521506856 >> 172.21.15.38:0/3521506856 conn(0x7f280c0fc580 msgr2=0x7f280c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.525+0000 7f2813a00700 1 -- 172.21.15.38:0/3521506856 shutdown_connections 2024-03-24T00:53:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.525+0000 7f2813a00700 1 -- 172.21.15.38:0/3521506856 wait complete. 2024-03-24T00:53:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.526+0000 7f2813a00700 1 Processor -- start 2024-03-24T00:53:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.526+0000 7f2813a00700 1 -- start start 2024-03-24T00:53:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.527+0000 7f2813a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f280c100a70 0x7f280c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.527+0000 7f2813a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c106a70 0x7f280c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.527+0000 7f2813a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c1989a0 con 0x7f280c106a70 2024-03-24T00:53:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.527+0000 7f2813a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c198b10 con 0x7f280c100a70 2024-03-24T00:53:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.527+0000 7f2803e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c106a70 0x7f280c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.527+0000 7f2803e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c106a70 0x7f280c19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44482/0 (socket says 172.21.15.38:44482) 2024-03-24T00:53:37.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.527+0000 7f2803e00700 1 -- 172.21.15.38:0/441992182 learned_addr learned my addr 172.21.15.38:0/441992182 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:37.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.527+0000 7f2811000700 1 --2- 172.21.15.38:0/441992182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f280c100a70 0x7f280c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:37.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.527+0000 7f2803e00700 1 -- 172.21.15.38:0/441992182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f280c100a70 msgr2=0x7f280c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:37.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.527+0000 7f2803e00700 1 --2- 172.21.15.38:0/441992182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f280c100a70 0x7f280c19e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.527+0000 7f2803e00700 1 -- 172.21.15.38:0/441992182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc0095f0 con 0x7f280c106a70 2024-03-24T00:53:37.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.528+0000 7f2803e00700 1 --2- 172.21.15.38:0/441992182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c106a70 0x7f280c19ee30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f27f400d8d0 tx=0x7f27f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:37.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.528+0000 7f280a000700 1 -- 172.21.15.38:0/441992182 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27f4009940 con 0x7f280c106a70 2024-03-24T00:53:37.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.528+0000 7f280a000700 1 -- 172.21.15.38:0/441992182 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27f4010460 con 0x7f280c106a70 2024-03-24T00:53:37.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.528+0000 7f2813a00700 1 -- 172.21.15.38:0/441992182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c198df0 con 0x7f280c106a70 2024-03-24T00:53:37.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.528+0000 7f280a000700 1 -- 172.21.15.38:0/441992182 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27f400f5f0 con 0x7f280c106a70 2024-03-24T00:53:37.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.528+0000 7f2813a00700 1 -- 172.21.15.38:0/441992182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c199340 con 0x7f280c106a70 2024-03-24T00:53:37.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.530+0000 7f2813a00700 1 -- 172.21.15.38:0/441992182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280c104d20 con 0x7f280c106a70 2024-03-24T00:53:37.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.531+0000 7f280a000700 1 -- 172.21.15.38:0/441992182 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f27f40105d0 con 0x7f280c106a70 2024-03-24T00:53:37.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.531+0000 7f280a000700 1 --2- 172.21.15.38:0/441992182 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f27f006dc20 0x7f27f00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:37.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.532+0000 7f280a000700 1 -- 172.21.15.38:0/441992182 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f27f408d6e0 con 0x7f280c106a70 2024-03-24T00:53:37.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.532+0000 7f2811000700 1 --2- 172.21.15.38:0/441992182 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f27f006dc20 0x7f27f00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:37.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.533+0000 7f2811000700 1 --2- 172.21.15.38:0/441992182 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f27f006dc20 0x7f27f00700e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00b5c0 tx=0x7f27fc005960 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:37.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.537+0000 7f280a000700 1 -- 172.21.15.38:0/441992182 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f27f40570b0 con 0x7f280c106a70 2024-03-24T00:53:37.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.812+0000 7f2813a00700 1 -- 172.21.15.38:0/441992182 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f280c199e80 con 0x7f27f006dc20 2024-03-24T00:53:37.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.814+0000 7f280a000700 1 -- 172.21.15.38:0/441992182 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f280c199e80 con 0x7f27f006dc20 2024-03-24T00:53:37.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.816+0000 7f2813a00700 1 -- 172.21.15.38:0/441992182 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f27f006dc20 msgr2=0x7f27f00700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:37.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.816+0000 7f2813a00700 1 --2- 172.21.15.38:0/441992182 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f27f006dc20 0x7f27f00700e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00b5c0 tx=0x7f27fc005960 comp rx=0 tx=0).stop 2024-03-24T00:53:37.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.816+0000 7f2813a00700 1 -- 172.21.15.38:0/441992182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c106a70 msgr2=0x7f280c19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:37.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.816+0000 7f2813a00700 1 --2- 172.21.15.38:0/441992182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c106a70 0x7f280c19ee30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f27f400d8d0 tx=0x7f27f400dc90 comp rx=0 tx=0).stop 2024-03-24T00:53:37.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.817+0000 7f2813a00700 1 -- 172.21.15.38:0/441992182 shutdown_connections 2024-03-24T00:53:37.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.817+0000 7f2813a00700 1 --2- 172.21.15.38:0/441992182 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f27f006dc20 0x7f27f00700e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.817+0000 7f2813a00700 1 --2- 172.21.15.38:0/441992182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f280c100a70 0x7f280c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.817+0000 7f2813a00700 1 --2- 172.21.15.38:0/441992182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c106a70 0x7f280c19ee30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.817+0000 7f2813a00700 1 -- 172.21.15.38:0/441992182 >> 172.21.15.38:0/441992182 conn(0x7f280c0fc580 msgr2=0x7f280c073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:37.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.817+0000 7f2813a00700 1 -- 172.21.15.38:0/441992182 shutdown_connections 2024-03-24T00:53:37.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.817+0000 7f2813a00700 1 -- 172.21.15.38:0/441992182 wait complete. 2024-03-24T00:53:37.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.978+0000 7f9d24c00700 1 -- 172.21.15.38:0/4060805734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d200731e0 msgr2=0x7f9d200735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:37.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.978+0000 7f9d24c00700 1 --2- 172.21.15.38:0/4060805734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d200731e0 0x7f9d200735c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009910 tx=0x7f9d10009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:37.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.978+0000 7f9d24c00700 1 -- 172.21.15.38:0/4060805734 shutdown_connections 2024-03-24T00:53:37.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.978+0000 7f9d24c00700 1 --2- 172.21.15.38:0/4060805734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d20073b00 0x7f9d2010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.978+0000 7f9d24c00700 1 --2- 172.21.15.38:0/4060805734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d200731e0 0x7f9d200735c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.978+0000 7f9d24c00700 1 -- 172.21.15.38:0/4060805734 >> 172.21.15.38:0/4060805734 conn(0x7f9d200fc3f0 msgr2=0x7f9d200fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:37.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.978+0000 7f9d24c00700 1 -- 172.21.15.38:0/4060805734 shutdown_connections 2024-03-24T00:53:37.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.978+0000 7f9d24c00700 1 -- 172.21.15.38:0/4060805734 wait complete. 2024-03-24T00:53:37.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.979+0000 7f9d24c00700 1 Processor -- start 2024-03-24T00:53:37.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.979+0000 7f9d24c00700 1 -- start start 2024-03-24T00:53:37.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.980+0000 7f9d24c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d200731e0 0x7f9d20198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:37.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.980+0000 7f9d24c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d20073b00 0x7f9d20198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:37.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.980+0000 7f9d24c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d20199600 con 0x7f9d200731e0 2024-03-24T00:53:37.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.980+0000 7f9d24c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2019d350 con 0x7f9d20073b00 2024-03-24T00:53:37.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.981+0000 7f9d1f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d200731e0 0x7f9d20198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:37.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.981+0000 7f9d1ea00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d20073b00 0x7f9d20198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:37.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.981+0000 7f9d1ea00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d20073b00 0x7f9d20198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:52910/0 (socket says 172.21.15.38:52910) 2024-03-24T00:53:37.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.981+0000 7f9d1ea00700 1 -- 172.21.15.38:0/3744550979 learned_addr learned my addr 172.21.15.38:0/3744550979 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:37.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.981+0000 7f9d1f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d200731e0 0x7f9d20198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44492/0 (socket says 172.21.15.38:44492) 2024-03-24T00:53:37.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.981+0000 7f9d1f400700 1 -- 172.21.15.38:0/3744550979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d20073b00 msgr2=0x7f9d20198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:37.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.981+0000 7f9d1f400700 1 --2- 172.21.15.38:0/3744550979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d20073b00 0x7f9d20198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:37.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.981+0000 7f9d1f400700 1 -- 172.21.15.38:0/3744550979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100095f0 con 0x7f9d200731e0 2024-03-24T00:53:37.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.982+0000 7f9d1f400700 1 --2- 172.21.15.38:0/3744550979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d200731e0 0x7f9d20198a30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9d100098e0 tx=0x7f9d1000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:37.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.982+0000 7f9d17e00700 1 -- 172.21.15.38:0/3744550979 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d1001d070 con 0x7f9d200731e0 2024-03-24T00:53:37.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.982+0000 7f9d24c00700 1 -- 172.21.15.38:0/3744550979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2019d5d0 con 0x7f9d200731e0 2024-03-24T00:53:37.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.982+0000 7f9d17e00700 1 -- 172.21.15.38:0/3744550979 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d10003d10 con 0x7f9d200731e0 2024-03-24T00:53:37.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.982+0000 7f9d17e00700 1 -- 172.21.15.38:0/3744550979 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d1000fd60 con 0x7f9d200731e0 2024-03-24T00:53:37.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.983+0000 7f9d24c00700 1 -- 172.21.15.38:0/3744550979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2019dac0 con 0x7f9d200731e0 2024-03-24T00:53:37.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.984+0000 7f9d24c00700 1 -- 172.21.15.38:0/3744550979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d20074890 con 0x7f9d200731e0 2024-03-24T00:53:37.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.985+0000 7f9d17e00700 1 -- 172.21.15.38:0/3744550979 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f9d1000f560 con 0x7f9d200731e0 2024-03-24T00:53:37.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.986+0000 7f9d17e00700 1 --2- 172.21.15.38:0/3744550979 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9d0406dc80 0x7f9d04070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:37.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.986+0000 7f9d17e00700 1 -- 172.21.15.38:0/3744550979 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9d1008ebf0 con 0x7f9d200731e0 2024-03-24T00:53:37.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.987+0000 7f9d1ea00700 1 --2- 172.21.15.38:0/3744550979 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9d0406dc80 0x7f9d04070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:37.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.988+0000 7f9d1ea00700 1 --2- 172.21.15.38:0/3744550979 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9d0406dc80 0x7f9d04070140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d20199f90 tx=0x7f9d0c00b500 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:37.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:37.993+0000 7f9d17e00700 1 -- 172.21.15.38:0/3744550979 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f9d1005bfd0 con 0x7f9d200731e0 2024-03-24T00:53:38.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.232+0000 7f9d24c00700 1 -- 172.21.15.38:0/3744550979 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d201074d0 con 0x7f9d0406dc80 2024-03-24T00:53:38.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.249+0000 7f9d17e00700 1 -- 172.21.15.38:0/3744550979 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9d201074d0 con 0x7f9d0406dc80 2024-03-24T00:53:38.252 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:53:38.252 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (2m) 71s ago 15m 14.1M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:53:38.252 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (16m) 71s ago 16m 8296k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:53:38.252 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (14m) 3m ago 14m 7922k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:53:38.252 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (16m) 71s ago 16m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:53:38.252 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (14m) 3m ago 14m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:53:38.252 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (79s) 71s ago 15m 52.4M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:53:38.252 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (8m) 71s ago 8m 97.8M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:53:38.252 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (8m) 71s ago 8m 151M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (8m) 3m ago 8m 24.4M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (8m) 3m ago 8m 24.3M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (4m) 71s ago 17m 502M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (3m) 3m ago 14m 447M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (17m) 71s ago 18m 53.1M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 da6fd287d5c7 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (14m) 3m ago 14m 44.5M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (3m) 71s ago 16m 7482k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (3m) 3m ago 14m 6753k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (13m) 71s ago 13m 255M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (12m) 71s ago 12m 270M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (12m) 71s ago 12m 226M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (11m) 3m ago 11m 274M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (11m) 3m ago 11m 258M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:53:38.253 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (10m) 3m ago 10m 222M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:53:38.254 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (2m) 71s ago 15m 55.6M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:53:38.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.253+0000 7f9d24c00700 1 -- 172.21.15.38:0/3744550979 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9d0406dc80 msgr2=0x7f9d04070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:38.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.253+0000 7f9d24c00700 1 --2- 172.21.15.38:0/3744550979 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9d0406dc80 0x7f9d04070140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d20199f90 tx=0x7f9d0c00b500 comp rx=0 tx=0).stop 2024-03-24T00:53:38.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.253+0000 7f9d24c00700 1 -- 172.21.15.38:0/3744550979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d200731e0 msgr2=0x7f9d20198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.253+0000 7f9d24c00700 1 --2- 172.21.15.38:0/3744550979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d200731e0 0x7f9d20198a30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9d100098e0 tx=0x7f9d1000bdc0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.253+0000 7f9d24c00700 1 -- 172.21.15.38:0/3744550979 shutdown_connections 2024-03-24T00:53:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.253+0000 7f9d24c00700 1 --2- 172.21.15.38:0/3744550979 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f9d0406dc80 0x7f9d04070140 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.253+0000 7f9d24c00700 1 --2- 172.21.15.38:0/3744550979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d200731e0 0x7f9d20198a30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.253+0000 7f9d24c00700 1 --2- 172.21.15.38:0/3744550979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d20073b00 0x7f9d20198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.253+0000 7f9d24c00700 1 -- 172.21.15.38:0/3744550979 >> 172.21.15.38:0/3744550979 conn(0x7f9d200fc3f0 msgr2=0x7f9d201050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.253+0000 7f9d24c00700 1 -- 172.21.15.38:0/3744550979 shutdown_connections 2024-03-24T00:53:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.253+0000 7f9d24c00700 1 -- 172.21.15.38:0/3744550979 wait complete. 2024-03-24T00:53:38.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.416+0000 7fd734400700 1 -- 172.21.15.38:0/2032120050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c106a70 msgr2=0x7fd72c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:38.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.416+0000 7fd734400700 1 --2- 172.21.15.38:0/2032120050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c106a70 0x7fd72c106e50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd72000b0a0 tx=0x7fd72000b3b0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.416+0000 7fd734400700 1 -- 172.21.15.38:0/2032120050 shutdown_connections 2024-03-24T00:53:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.416+0000 7fd734400700 1 --2- 172.21.15.38:0/2032120050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c100a70 0x7fd72c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.417+0000 7fd734400700 1 --2- 172.21.15.38:0/2032120050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c106a70 0x7fd72c106e50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.417+0000 7fd734400700 1 -- 172.21.15.38:0/2032120050 >> 172.21.15.38:0/2032120050 conn(0x7fd72c0fc5a0 msgr2=0x7fd72c0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.417+0000 7fd734400700 1 -- 172.21.15.38:0/2032120050 shutdown_connections 2024-03-24T00:53:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.417+0000 7fd734400700 1 -- 172.21.15.38:0/2032120050 wait complete. 2024-03-24T00:53:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.418+0000 7fd734400700 1 Processor -- start 2024-03-24T00:53:38.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.418+0000 7fd734400700 1 -- start start 2024-03-24T00:53:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.418+0000 7fd734400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c100a70 0x7fd72c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.418+0000 7fd734400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c106a70 0x7fd72c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.418+0000 7fd734400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c073b30 con 0x7fd72c106a70 2024-03-24T00:53:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.418+0000 7fd734400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c073c70 con 0x7fd72c100a70 2024-03-24T00:53:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.419+0000 7fd731000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c106a70 0x7fd72c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.419+0000 7fd731000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c106a70 0x7fd72c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44516/0 (socket says 172.21.15.38:44516) 2024-03-24T00:53:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.419+0000 7fd731000700 1 -- 172.21.15.38:0/1305726738 learned_addr learned my addr 172.21.15.38:0/1305726738 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.419+0000 7fd731a00700 1 --2- 172.21.15.38:0/1305726738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c100a70 0x7fd72c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.419+0000 7fd731000700 1 -- 172.21.15.38:0/1305726738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c100a70 msgr2=0x7fd72c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.419+0000 7fd731000700 1 --2- 172.21.15.38:0/1305726738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c100a70 0x7fd72c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.419+0000 7fd731000700 1 -- 172.21.15.38:0/1305726738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd720009d00 con 0x7fd72c106a70 2024-03-24T00:53:38.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.420+0000 7fd731000700 1 --2- 172.21.15.38:0/1305726738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c106a70 0x7fd72c0735f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd71800c8a0 tx=0x7fd71800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:38.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.420+0000 7fd72a000700 1 -- 172.21.15.38:0/1305726738 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7180078e0 con 0x7fd72c106a70 2024-03-24T00:53:38.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.420+0000 7fd72a000700 1 -- 172.21.15.38:0/1305726738 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd71800f450 con 0x7fd72c106a70 2024-03-24T00:53:38.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.420+0000 7fd72a000700 1 -- 172.21.15.38:0/1305726738 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71800e5e0 con 0x7fd72c106a70 2024-03-24T00:53:38.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.420+0000 7fd734400700 1 -- 172.21.15.38:0/1305726738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72c1a2c10 con 0x7fd72c106a70 2024-03-24T00:53:38.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.420+0000 7fd734400700 1 -- 172.21.15.38:0/1305726738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72c1a2f70 con 0x7fd72c106a70 2024-03-24T00:53:38.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.421+0000 7fd734400700 1 -- 172.21.15.38:0/1305726738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72c109070 con 0x7fd72c106a70 2024-03-24T00:53:38.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.422+0000 7fd72a000700 1 -- 172.21.15.38:0/1305726738 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7fd71800f5c0 con 0x7fd72c106a70 2024-03-24T00:53:38.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.423+0000 7fd72a000700 1 --2- 172.21.15.38:0/1305726738 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd71406dc30 0x7fd7140700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:38.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.423+0000 7fd72a000700 1 -- 172.21.15.38:0/1305726738 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd71808d6a0 con 0x7fd72c106a70 2024-03-24T00:53:38.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.424+0000 7fd731a00700 1 --2- 172.21.15.38:0/1305726738 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd71406dc30 0x7fd7140700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:38.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.424+0000 7fd731a00700 1 --2- 172.21.15.38:0/1305726738 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd71406dc30 0x7fd7140700f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd720000f80 tx=0x7fd7200097e0 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:38.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.428+0000 7fd72a000700 1 -- 172.21.15.38:0/1305726738 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fd718057070 con 0x7fd72c106a70 2024-03-24T00:53:38.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:38 smithi088 ceph-mon[36610]: pgmap v132: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:38.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:38 smithi088 ceph-mon[36610]: from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:38.564 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:38 smithi038 ceph-mon[29271]: pgmap v132: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:38.564 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:38 smithi038 ceph-mon[29271]: from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:38.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.758+0000 7fd734400700 1 -- 172.21.15.38:0/1305726738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd72c073e30 con 0x7fd72c106a70 2024-03-24T00:53:38.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.759+0000 7fd72a000700 1 -- 172.21.15.38:0/1305726738 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7fd71805a730 con 0x7fd72c106a70 2024-03-24T00:53:38.760 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:53:38.760 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:53:38.760 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2 2024-03-24T00:53:38.760 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:53:38.760 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 12, 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:53:38.761 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:53:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.761+0000 7fd734400700 1 -- 172.21.15.38:0/1305726738 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd71406dc30 msgr2=0x7fd7140700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.761+0000 7fd734400700 1 --2- 172.21.15.38:0/1305726738 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd71406dc30 0x7fd7140700f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd720000f80 tx=0x7fd7200097e0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.761+0000 7fd734400700 1 -- 172.21.15.38:0/1305726738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c106a70 msgr2=0x7fd72c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.761+0000 7fd734400700 1 --2- 172.21.15.38:0/1305726738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c106a70 0x7fd72c0735f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd71800c8a0 tx=0x7fd71800cbb0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.762+0000 7fd734400700 1 -- 172.21.15.38:0/1305726738 shutdown_connections 2024-03-24T00:53:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.762+0000 7fd734400700 1 --2- 172.21.15.38:0/1305726738 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fd71406dc30 0x7fd7140700f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.762+0000 7fd734400700 1 --2- 172.21.15.38:0/1305726738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72c100a70 0x7fd72c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.762+0000 7fd734400700 1 --2- 172.21.15.38:0/1305726738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd72c106a70 0x7fd72c0735f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.762+0000 7fd734400700 1 -- 172.21.15.38:0/1305726738 >> 172.21.15.38:0/1305726738 conn(0x7fd72c0fc5a0 msgr2=0x7fd72c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.762+0000 7fd734400700 1 -- 172.21.15.38:0/1305726738 shutdown_connections 2024-03-24T00:53:38.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.762+0000 7fd734400700 1 -- 172.21.15.38:0/1305726738 wait complete. 2024-03-24T00:53:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.927+0000 7fcc9fe00700 1 -- 172.21.15.38:0/2271382175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc98107310 msgr2=0x7fcc981076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.927+0000 7fcc9fe00700 1 --2- 172.21.15.38:0/2271382175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc98107310 0x7fcc981076f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009910 tx=0x7fcc8c009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.927+0000 7fcc9fe00700 1 -- 172.21.15.38:0/2271382175 shutdown_connections 2024-03-24T00:53:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.927+0000 7fcc9fe00700 1 --2- 172.21.15.38:0/2271382175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc98107cc0 0x7fcc9810bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.927+0000 7fcc9fe00700 1 --2- 172.21.15.38:0/2271382175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc98107310 0x7fcc981076f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.927+0000 7fcc9fe00700 1 -- 172.21.15.38:0/2271382175 >> 172.21.15.38:0/2271382175 conn(0x7fcc980768e0 msgr2=0x7fcc98078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.928+0000 7fcc9fe00700 1 -- 172.21.15.38:0/2271382175 shutdown_connections 2024-03-24T00:53:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.928+0000 7fcc9fe00700 1 -- 172.21.15.38:0/2271382175 wait complete. 2024-03-24T00:53:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.929+0000 7fcc9fe00700 1 Processor -- start 2024-03-24T00:53:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.929+0000 7fcc9fe00700 1 -- start start 2024-03-24T00:53:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.929+0000 7fcc9fe00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc98107310 0x7fcc9819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.929+0000 7fcc9fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc98107cc0 0x7fcc9819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.929+0000 7fcc9fe00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9819dc50 con 0x7fcc98107cc0 2024-03-24T00:53:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.929+0000 7fcc9fe00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981a18d0 con 0x7fcc98107310 2024-03-24T00:53:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.929+0000 7fcc9ca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc98107cc0 0x7fcc9819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.930+0000 7fcc9ca00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc98107cc0 0x7fcc9819d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44542/0 (socket says 172.21.15.38:44542) 2024-03-24T00:53:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.930+0000 7fcc9ca00700 1 -- 172.21.15.38:0/1616246957 learned_addr learned my addr 172.21.15.38:0/1616246957 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:38.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.930+0000 7fcc9d400700 1 --2- 172.21.15.38:0/1616246957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc98107310 0x7fcc9819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:38.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.930+0000 7fcc9ca00700 1 -- 172.21.15.38:0/1616246957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc98107310 msgr2=0x7fcc9819d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:38.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.930+0000 7fcc9ca00700 1 --2- 172.21.15.38:0/1616246957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc98107310 0x7fcc9819d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:38.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.930+0000 7fcc9ca00700 1 -- 172.21.15.38:0/1616246957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c0095f0 con 0x7fcc98107cc0 2024-03-24T00:53:38.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.930+0000 7fcc9ca00700 1 --2- 172.21.15.38:0/1616246957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc98107cc0 0x7fcc9819d570 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400d900 tx=0x7fcc8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:38.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.931+0000 7fcc96000700 1 -- 172.21.15.38:0/1616246957 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc84009900 con 0x7fcc98107cc0 2024-03-24T00:53:38.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.931+0000 7fcc96000700 1 -- 172.21.15.38:0/1616246957 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc84010460 con 0x7fcc98107cc0 2024-03-24T00:53:38.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.931+0000 7fcc9fe00700 1 -- 172.21.15.38:0/1616246957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc981a1bb0 con 0x7fcc98107cc0 2024-03-24T00:53:38.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.931+0000 7fcc96000700 1 -- 172.21.15.38:0/1616246957 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc8400f5f0 con 0x7fcc98107cc0 2024-03-24T00:53:38.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.931+0000 7fcc9fe00700 1 -- 172.21.15.38:0/1616246957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc981a2100 con 0x7fcc98107cc0 2024-03-24T00:53:38.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.932+0000 7fcc9fe00700 1 -- 172.21.15.38:0/1616246957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9810f6b0 con 0x7fcc98107cc0 2024-03-24T00:53:38.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.933+0000 7fcc96000700 1 -- 172.21.15.38:0/1616246957 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7fcc840105d0 con 0x7fcc98107cc0 2024-03-24T00:53:38.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.934+0000 7fcc96000700 1 --2- 172.21.15.38:0/1616246957 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcc8006dc30 0x7fcc800700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:38.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.934+0000 7fcc96000700 1 -- 172.21.15.38:0/1616246957 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcc8408d610 con 0x7fcc98107cc0 2024-03-24T00:53:38.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.935+0000 7fcc9d400700 1 --2- 172.21.15.38:0/1616246957 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcc8006dc30 0x7fcc800700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:38.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.935+0000 7fcc9d400700 1 --2- 172.21.15.38:0/1616246957 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcc8006dc30 0x7fcc800700f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00b5c0 tx=0x7fcc8c01a040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:38.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:38.939+0000 7fcc96000700 1 -- 172.21.15.38:0/1616246957 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fcc84056f40 con 0x7fcc98107cc0 2024-03-24T00:53:39.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.224+0000 7fcc9fe00700 1 -- 172.21.15.38:0/1616246957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcc9819e2f0 con 0x7fcc98107cc0 2024-03-24T00:53:39.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.225+0000 7fcc96000700 1 -- 172.21.15.38:0/1616246957 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1868 (secure 0 0 0) 0x7fcc84017760 con 0x7fcc98107cc0 2024-03-24T00:53:39.227 INFO:teuthology.orchestra.run.smithi038.stdout:e20 2024-03-24T00:53:39.227 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:53:39.227 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:53:39.227 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:53:39.227 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:53:39.227 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:53:39.227 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:53:39.227 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 20 2024-03-24T00:53:39.227 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:53:39.227 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:53:39.227 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:53:39.228 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:53:39.229 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:53:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.229+0000 7fcc9fe00700 1 -- 172.21.15.38:0/1616246957 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcc8006dc30 msgr2=0x7fcc800700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.229+0000 7fcc9fe00700 1 --2- 172.21.15.38:0/1616246957 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcc8006dc30 0x7fcc800700f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00b5c0 tx=0x7fcc8c01a040 comp rx=0 tx=0).stop 2024-03-24T00:53:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.229+0000 7fcc9fe00700 1 -- 172.21.15.38:0/1616246957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc98107cc0 msgr2=0x7fcc9819d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.229+0000 7fcc9fe00700 1 --2- 172.21.15.38:0/1616246957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc98107cc0 0x7fcc9819d570 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400d900 tx=0x7fcc8400dcc0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.230+0000 7fcc9fe00700 1 -- 172.21.15.38:0/1616246957 shutdown_connections 2024-03-24T00:53:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.230+0000 7fcc9fe00700 1 --2- 172.21.15.38:0/1616246957 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fcc8006dc30 0x7fcc800700f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.230+0000 7fcc9fe00700 1 --2- 172.21.15.38:0/1616246957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc98107310 0x7fcc9819d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.230+0000 7fcc9fe00700 1 --2- 172.21.15.38:0/1616246957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc98107cc0 0x7fcc9819d570 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.230+0000 7fcc9fe00700 1 -- 172.21.15.38:0/1616246957 >> 172.21.15.38:0/1616246957 conn(0x7fcc980768e0 msgr2=0x7fcc98076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.230+0000 7fcc9fe00700 1 -- 172.21.15.38:0/1616246957 shutdown_connections 2024-03-24T00:53:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.230+0000 7fcc9fe00700 1 -- 172.21.15.38:0/1616246957 wait complete. 2024-03-24T00:53:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:53:39.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.409+0000 7f9289600700 1 -- 172.21.15.38:0/3908669470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9284100a70 msgr2=0x7f9284100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:39.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.409+0000 7f9289600700 1 --2- 172.21.15.38:0/3908669470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9284100a70 0x7f9284100ef0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9270009910 tx=0x7f9270009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:39.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.410+0000 7f9289600700 1 -- 172.21.15.38:0/3908669470 shutdown_connections 2024-03-24T00:53:39.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.410+0000 7f9289600700 1 --2- 172.21.15.38:0/3908669470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9284100a70 0x7f9284100ef0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.410+0000 7f9289600700 1 --2- 172.21.15.38:0/3908669470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9284106a70 0x7f9284106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.410+0000 7f9289600700 1 -- 172.21.15.38:0/3908669470 >> 172.21.15.38:0/3908669470 conn(0x7f92840fc580 msgr2=0x7f92840fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:39.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.410+0000 7f9289600700 1 -- 172.21.15.38:0/3908669470 shutdown_connections 2024-03-24T00:53:39.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.410+0000 7f9289600700 1 -- 172.21.15.38:0/3908669470 wait complete. 2024-03-24T00:53:39.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.411+0000 7f9289600700 1 Processor -- start 2024-03-24T00:53:39.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.412+0000 7f9289600700 1 -- start start 2024-03-24T00:53:39.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.412+0000 7f9289600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9284100a70 0x7f928419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:39.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.412+0000 7f9289600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9284106a70 0x7f928419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:39.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.412+0000 7f9289600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928419f350 con 0x7f9284100a70 2024-03-24T00:53:39.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.412+0000 7f9289600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92841989c0 con 0x7f9284106a70 2024-03-24T00:53:39.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.412+0000 7f9282a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9284100a70 0x7f928419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:39.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.413+0000 7f9282a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9284100a70 0x7f928419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44560/0 (socket says 172.21.15.38:44560) 2024-03-24T00:53:39.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.413+0000 7f9282a00700 1 -- 172.21.15.38:0/1930931247 learned_addr learned my addr 172.21.15.38:0/1930931247 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:39.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.413+0000 7f9282000700 1 --2- 172.21.15.38:0/1930931247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9284106a70 0x7f928419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:39.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.413+0000 7f9282a00700 1 -- 172.21.15.38:0/1930931247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9284106a70 msgr2=0x7f928419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:39.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.413+0000 7f9282a00700 1 --2- 172.21.15.38:0/1930931247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9284106a70 0x7f928419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.413+0000 7f9282a00700 1 -- 172.21.15.38:0/1930931247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92700095f0 con 0x7f9284100a70 2024-03-24T00:53:39.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.413+0000 7f9282a00700 1 --2- 172.21.15.38:0/1930931247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9284100a70 0x7f928419e880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f927400c930 tx=0x7f927400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:39.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.414+0000 7f927b400700 1 -- 172.21.15.38:0/1930931247 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9274007ad0 con 0x7f9284100a70 2024-03-24T00:53:39.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.414+0000 7f927b400700 1 -- 172.21.15.38:0/1930931247 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9274007c30 con 0x7f9284100a70 2024-03-24T00:53:39.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.414+0000 7f9289600700 1 -- 172.21.15.38:0/1930931247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9284198ca0 con 0x7f9284100a70 2024-03-24T00:53:39.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.414+0000 7f927b400700 1 -- 172.21.15.38:0/1930931247 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9274018610 con 0x7f9284100a70 2024-03-24T00:53:39.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.414+0000 7f9289600700 1 -- 172.21.15.38:0/1930931247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92841991f0 con 0x7f9284100a70 2024-03-24T00:53:39.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.415+0000 7f9289600700 1 -- 172.21.15.38:0/1930931247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9284104d20 con 0x7f9284100a70 2024-03-24T00:53:39.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.416+0000 7f927b400700 1 -- 172.21.15.38:0/1930931247 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7f927400f450 con 0x7f9284100a70 2024-03-24T00:53:39.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.418+0000 7f927b400700 1 --2- 172.21.15.38:0/1930931247 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f926806dc30 0x7f92680700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:39.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.418+0000 7f927b400700 1 -- 172.21.15.38:0/1930931247 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f927408da00 con 0x7f9284100a70 2024-03-24T00:53:39.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.418+0000 7f9282000700 1 --2- 172.21.15.38:0/1930931247 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f926806dc30 0x7f92680700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:39.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.419+0000 7f9282000700 1 --2- 172.21.15.38:0/1930931247 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f926806dc30 0x7f92680700f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9270005f50 tx=0x7f927001a040 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:39.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.424+0000 7f927b400700 1 -- 172.21.15.38:0/1930931247 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f927405ae60 con 0x7f9284100a70 2024-03-24T00:53:39.596 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:39 smithi038 ceph-mon[29271]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:39.596 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:39 smithi038 ceph-mon[29271]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:39.597 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:39.597 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:53:39.597 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:53:39.597 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:39.597 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:39 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T00:53:39.597 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:39 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1305726738' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:53:39.597 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:39 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/1616246957' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:53:39.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:39 smithi088 ceph-mon[36610]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:39.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:39 smithi088 ceph-mon[36610]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:53:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:53:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:39 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T00:53:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:39 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1305726738' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:53:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:39 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1616246957' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:53:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.716+0000 7f9289600700 1 -- 172.21.15.38:0/1930931247 --> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9284199ef0 con 0x7f926806dc30 2024-03-24T00:53:39.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.718+0000 7f927b400700 1 -- 172.21.15.38:0/1930931247 <== mgr.14664 v2:172.21.15.38:6800/839555621 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9284199ef0 con 0x7f926806dc30 2024-03-24T00:53:39.719 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:53:39.719 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:53:39.720 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:53:39.720 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:53:39.720 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:53:39.720 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-03-24T00:53:39.720 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:53:39.720 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "2/23 daemons upgraded", 2024-03-24T00:53:39.720 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mon daemons", 2024-03-24T00:53:39.720 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:53:39.720 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:53:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.720+0000 7f9289600700 1 -- 172.21.15.38:0/1930931247 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f926806dc30 msgr2=0x7f92680700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.720+0000 7f9289600700 1 --2- 172.21.15.38:0/1930931247 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f926806dc30 0x7f92680700f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9270005f50 tx=0x7f927001a040 comp rx=0 tx=0).stop 2024-03-24T00:53:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.720+0000 7f9289600700 1 -- 172.21.15.38:0/1930931247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9284100a70 msgr2=0x7f928419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.720+0000 7f9289600700 1 --2- 172.21.15.38:0/1930931247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9284100a70 0x7f928419e880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f927400c930 tx=0x7f927400ccf0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.721+0000 7f9289600700 1 -- 172.21.15.38:0/1930931247 shutdown_connections 2024-03-24T00:53:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.721+0000 7f9289600700 1 --2- 172.21.15.38:0/1930931247 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7f926806dc30 0x7f92680700f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.721+0000 7f9289600700 1 --2- 172.21.15.38:0/1930931247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9284100a70 0x7f928419e880 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.721+0000 7f9289600700 1 --2- 172.21.15.38:0/1930931247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9284106a70 0x7f928419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.721+0000 7f9289600700 1 -- 172.21.15.38:0/1930931247 >> 172.21.15.38:0/1930931247 conn(0x7f92840fc580 msgr2=0x7f92840731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.721+0000 7f9289600700 1 -- 172.21.15.38:0/1930931247 shutdown_connections 2024-03-24T00:53:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.721+0000 7f9289600700 1 -- 172.21.15.38:0/1930931247 wait complete. 2024-03-24T00:53:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 -- 172.21.15.38:0/719133825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe44100a70 msgr2=0x7fbe44100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 --2- 172.21.15.38:0/719133825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe44100a70 0x7fbe44100ef0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009910 tx=0x7fbe30009c20 comp rx=0 tx=0).stop 2024-03-24T00:53:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 -- 172.21.15.38:0/719133825 shutdown_connections 2024-03-24T00:53:39.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 --2- 172.21.15.38:0/719133825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe44100a70 0x7fbe44100ef0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 --2- 172.21.15.38:0/719133825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe44106a70 0x7fbe44106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 -- 172.21.15.38:0/719133825 >> 172.21.15.38:0/719133825 conn(0x7fbe440fc5a0 msgr2=0x7fbe440fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 -- 172.21.15.38:0/719133825 shutdown_connections 2024-03-24T00:53:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 -- 172.21.15.38:0/719133825 wait complete. 2024-03-24T00:53:39.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 Processor -- start 2024-03-24T00:53:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 -- start start 2024-03-24T00:53:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe44100a70 0x7fbe4419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe44106a70 0x7fbe4419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44198a20 con 0x7fbe44106a70 2024-03-24T00:53:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.894+0000 7fbe4a000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44198b90 con 0x7fbe44100a70 2024-03-24T00:53:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.895+0000 7fbe42a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe44106a70 0x7fbe4419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.895+0000 7fbe42a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe44106a70 0x7fbe4419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44576/0 (socket says 172.21.15.38:44576) 2024-03-24T00:53:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.895+0000 7fbe42a00700 1 -- 172.21.15.38:0/491372397 learned_addr learned my addr 172.21.15.38:0/491372397 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:53:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.895+0000 7fbe43400700 1 --2- 172.21.15.38:0/491372397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe44100a70 0x7fbe4419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:39.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.895+0000 7fbe42a00700 1 -- 172.21.15.38:0/491372397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe44100a70 msgr2=0x7fbe4419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.895+0000 7fbe42a00700 1 --2- 172.21.15.38:0/491372397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe44100a70 0x7fbe4419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.895+0000 7fbe42a00700 1 -- 172.21.15.38:0/491372397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300095f0 con 0x7fbe44106a70 2024-03-24T00:53:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.896+0000 7fbe42a00700 1 --2- 172.21.15.38:0/491372397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe44106a70 0x7fbe4419eeb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbe300092e0 tx=0x7fbe3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.896+0000 7fbe3be00700 1 -- 172.21.15.38:0/491372397 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe3001d070 con 0x7fbe44106a70 2024-03-24T00:53:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.896+0000 7fbe3be00700 1 -- 172.21.15.38:0/491372397 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe3000f460 con 0x7fbe44106a70 2024-03-24T00:53:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.896+0000 7fbe3be00700 1 -- 172.21.15.38:0/491372397 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe30017630 con 0x7fbe44106a70 2024-03-24T00:53:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.896+0000 7fbe4a000700 1 -- 172.21.15.38:0/491372397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe44198e10 con 0x7fbe44106a70 2024-03-24T00:53:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.896+0000 7fbe4a000700 1 -- 172.21.15.38:0/491372397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe44199300 con 0x7fbe44106a70 2024-03-24T00:53:39.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.897+0000 7fbe4a000700 1 -- 172.21.15.38:0/491372397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe44104d20 con 0x7fbe44106a70 2024-03-24T00:53:39.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.899+0000 7fbe3be00700 1 -- 172.21.15.38:0/491372397 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 32) v1 ==== 91694+0+0 (secure 0 0 0) 0x7fbe3000f5d0 con 0x7fbe44106a70 2024-03-24T00:53:39.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.899+0000 7fbe3be00700 1 --2- 172.21.15.38:0/491372397 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe2806dc30 0x7fbe280700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:53:39.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.900+0000 7fbe3be00700 1 -- 172.21.15.38:0/491372397 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbe3008ec30 con 0x7fbe44106a70 2024-03-24T00:53:39.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.900+0000 7fbe43400700 1 --2- 172.21.15.38:0/491372397 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe2806dc30 0x7fbe280700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:53:39.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.901+0000 7fbe43400700 1 --2- 172.21.15.38:0/491372397 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe2806dc30 0x7fbe280700f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe34005fd0 tx=0x7fbe3400a560 comp rx=0 tx=0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:53:39.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:39.905+0000 7fbe3be00700 1 -- 172.21.15.38:0/491372397 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fbe30058600 con 0x7fbe44106a70 2024-03-24T00:53:40.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.233+0000 7fbe4a000700 1 -- 172.21.15.38:0/491372397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbe4404eab0 con 0x7fbe44106a70 2024-03-24T00:53:40.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.233+0000 7fbe3be00700 1 -- 172.21.15.38:0/491372397 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fbe3005bcc0 con 0x7fbe44106a70 2024-03-24T00:53:40.235 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_OK 2024-03-24T00:53:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.236+0000 7fbe4a000700 1 -- 172.21.15.38:0/491372397 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe2806dc30 msgr2=0x7fbe280700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.236+0000 7fbe4a000700 1 --2- 172.21.15.38:0/491372397 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe2806dc30 0x7fbe280700f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe34005fd0 tx=0x7fbe3400a560 comp rx=0 tx=0).stop 2024-03-24T00:53:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.236+0000 7fbe4a000700 1 -- 172.21.15.38:0/491372397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe44106a70 msgr2=0x7fbe4419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:53:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.236+0000 7fbe4a000700 1 --2- 172.21.15.38:0/491372397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe44106a70 0x7fbe4419eeb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbe300092e0 tx=0x7fbe3000be20 comp rx=0 tx=0).stop 2024-03-24T00:53:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.237+0000 7fbe4a000700 1 -- 172.21.15.38:0/491372397 shutdown_connections 2024-03-24T00:53:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.237+0000 7fbe4a000700 1 --2- 172.21.15.38:0/491372397 >> [v2:172.21.15.38:6800/839555621,v1:172.21.15.38:6801/839555621] conn(0x7fbe2806dc30 0x7fbe280700f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.237+0000 7fbe4a000700 1 --2- 172.21.15.38:0/491372397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe44100a70 0x7fbe4419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.237+0000 7fbe4a000700 1 --2- 172.21.15.38:0/491372397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe44106a70 0x7fbe4419eeb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:53:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.237+0000 7fbe4a000700 1 -- 172.21.15.38:0/491372397 >> 172.21.15.38:0/491372397 conn(0x7fbe440fc5a0 msgr2=0x7fbe440731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:53:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.237+0000 7fbe4a000700 1 -- 172.21.15.38:0/491372397 shutdown_connections 2024-03-24T00:53:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:53:40.237+0000 7fbe4a000700 1 -- 172.21.15.38:0/491372397 wait complete. 2024-03-24T00:53:40.607 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:40 smithi038 ceph-mon[29271]: Upgrade: Target is version 19.0.0-2145-g3be9a5c9 (squid) 2024-03-24T00:53:40.607 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:40 smithi038 ceph-mon[29271]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc'] 2024-03-24T00:53:40.607 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:40 smithi038 ceph-mon[29271]: Upgrade: Setting container_image for all mgr 2024-03-24T00:53:40.607 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:40 smithi038 ceph-mon[29271]: pgmap v133: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:40.607 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:40 smithi038 ceph-mon[29271]: from='client.? 172.21.15.38:0/491372397' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:53:40.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:40 smithi088 ceph-mon[36610]: Upgrade: Target is version 19.0.0-2145-g3be9a5c9 (squid) 2024-03-24T00:53:40.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:40 smithi088 ceph-mon[36610]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc'] 2024-03-24T00:53:40.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:40 smithi088 ceph-mon[36610]: Upgrade: Setting container_image for all mgr 2024-03-24T00:53:40.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:40 smithi088 ceph-mon[36610]: pgmap v133: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:53:40.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:40 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/491372397' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:53:41.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:41 smithi038 ceph-mon[29271]: from='client.14890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:41.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:41 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:41.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:41 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:53:41.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:41 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:53:41.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:41 smithi038 ceph-mon[29271]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:53:41.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:41 smithi088 ceph-mon[36610]: from='client.14890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:53:41.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:41 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' 2024-03-24T00:53:41.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:41 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:53:41.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:41 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:53:41.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:41 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:53:42.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:42 smithi088 ceph-mon[36610]: Upgrade: Updating mon.smithi038 2024-03-24T00:53:42.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:42 smithi088 ceph-mon[36610]: Deploying daemon mon.smithi038 on smithi038 2024-03-24T00:53:42.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:42 smithi088 ceph-mon[36610]: pgmap v134: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:53:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:42 smithi038 ceph-mon[29271]: Upgrade: Updating mon.smithi038 2024-03-24T00:53:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:42 smithi038 ceph-mon[29271]: Deploying daemon mon.smithi038 on smithi038 2024-03-24T00:53:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:42 smithi038 ceph-mon[29271]: pgmap v134: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:53:44.103 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:43 smithi038 systemd[1]: Stopping Ceph mon.smithi038 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:53:44.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:44 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[29267]: 2024-03-24T00:53:44.100+0000 7f3829e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi038 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T00:53:44.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:44 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[29267]: 2024-03-24T00:53:44.100+0000 7f3829e00700 -1 mon.smithi038@0(leader) e2 *** Got Signal Terminated *** 2024-03-24T00:53:44.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:44 smithi038 podman[111937]: 2024-03-24 00:53:44.152844756 +0000 UTC m=+0.178822898 container died da6fd287d5c70413a7a6c4a6a5b88ada5e5bfbdd2979586ca60ba69901ecfb71 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, maintainer=Guillaume Abrioux , RELEASE=reef-738c2fd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-24T00:53:44.708 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:44 smithi038 podman[111937]: 2024-03-24 00:53:44.68393764 +0000 UTC m=+0.709915794 container cleanup da6fd287d5c70413a7a6c4a6a5b88ada5e5bfbdd2979586ca60ba69901ecfb71 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, RELEASE=reef-738c2fd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:53:44.708 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:44 smithi038 bash[111937]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038 2024-03-24T00:53:45.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:44 smithi038 podman[111959]: 2024-03-24 00:53:44.926934191 +0000 UTC m=+0.760729401 container remove da6fd287d5c70413a7a6c4a6a5b88ada5e5bfbdd2979586ca60ba69901ecfb71 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=reef-738c2fd, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-24T00:53:45.823 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:45 smithi038 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi038.service: Deactivated successfully. 2024-03-24T00:53:45.823 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:45 smithi038 systemd[1]: Stopped Ceph mon.smithi038 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:53:45.823 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:45 smithi038 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi038.service: Consumed 33.588s CPU time. 2024-03-24T00:53:46.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:45 smithi038 systemd[1]: Starting Ceph mon.smithi038 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:53:46.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 podman[112090]: 2024-03-24 00:53:46.086498291 +0000 UTC m=+0.033665503 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:53:46.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 podman[112090]: 2024-03-24 00:53:46.211374301 +0000 UTC m=+0.158541534 container create ba098af28413976f02407c0b6e9dab98a3c86c9ce6492c392fb9daff638f289b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-24T00:53:46.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 podman[112090]: 2024-03-24 00:53:46.490426448 +0000 UTC m=+0.437593688 container init ba098af28413976f02407c0b6e9dab98a3c86c9ce6492c392fb9daff638f289b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T00:53:46.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 podman[112090]: 2024-03-24 00:53:46.501993328 +0000 UTC m=+0.449160547 container start ba098af28413976f02407c0b6e9dab98a3c86c9ce6492c392fb9daff638f289b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-24T00:53:46.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: pidfile_write: ignore empty --pid-file 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: load: jerasure load: lrc 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: RocksDB version: 7.9.2 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Git sha 0 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: DB SUMMARY 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: DB Session ID: HUX8I7PWGS6LP1EIE197 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: CURRENT file: CURRENT 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1732 Bytes 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi038/store.db dir, Total Num: 1, files: 000035.sst 2024-03-24T00:53:46.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi038/store.db: 000033.log size: 1005080 ; 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.error_if_exists: 0 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.create_if_missing: 0 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.paranoid_checks: 1 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.env: 0x563ae2f64b20 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.info_log: 0x563ae5695840 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.statistics: (nil) 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.use_fsync: 0 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_log_file_size: 0 2024-03-24T00:53:46.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T00:53:46.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T00:53:46.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T00:53:46.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T00:53:46.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.allow_fallocate: 1 2024-03-24T00:53:46.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T00:53:46.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T00:53:46.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.use_direct_reads: 0 2024-03-24T00:53:46.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T00:53:46.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.db_log_dir: 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.wal_dir: 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.write_buffer_manager: 0x563ae56681e0 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T00:53:46.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.unordered_write: 0 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.row_cache: None 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.wal_filter: None 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T00:53:46.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.two_write_queues: 0 2024-03-24T00:53:46.867 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T00:53:46.867 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.wal_compression: 0 2024-03-24T00:53:46.867 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.atomic_flush: 0 2024-03-24T00:53:46.867 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T00:53:46.867 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T00:53:46.868 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T00:53:46.868 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.log_readahead_size: 0 2024-03-24T00:53:46.868 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T00:53:46.868 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T00:53:46.868 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T00:53:46.868 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T00:53:46.868 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T00:53:46.868 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_background_jobs: 2 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_background_compactions: -1 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_subcompactions: 1 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T00:53:46.869 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_open_files: -1 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_background_flushes: -1 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Compression algorithms supported: 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: kZSTD supported: 0 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: kXpressCompression supported: 0 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: kZlibCompression supported: 1 2024-03-24T00:53:46.870 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: kSnappyCompression supported: 1 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: kLZ4Compression supported: 1 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: kBZip2Compression supported: 0 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi038/store.db/MANIFEST-000015 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.merge_operator: 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_filter: None 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T00:53:46.871 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563ae5695940) 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: cache_index_and_filter_blocks: 1 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: pin_top_level_index_and_filter: 1 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: index_type: 0 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: data_block_index_type: 0 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: index_shortening: 1 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: checksum: 4 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: no_block_cache: 0 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_cache: 0x563ae569b090 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_cache_name: BinnedLRUCache 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_cache_options: 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: capacity : 536870912 2024-03-24T00:53:46.872 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: num_shard_bits : 4 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: strict_capacity_limit : 0 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: high_pri_pool_ratio: 0.000 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_cache_compressed: (nil) 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: persistent_cache: (nil) 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_size: 4096 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_size_deviation: 10 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_restart_interval: 16 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: index_block_restart_interval: 1 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: metadata_block_size: 4096 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: partition_filters: 0 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: use_delta_encoding: 1 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: filter_policy: bloomfilter 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: whole_key_filtering: 1 2024-03-24T00:53:46.873 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: verify_compression: 0 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: read_amp_bytes_per_bit: 0 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: format_version: 5 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: enable_index_compression: 1 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: block_align: 0 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: max_auto_readahead_size: 262144 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: prepopulate_block_cache: 0 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: initial_auto_readahead_size: 8192 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compression: NoCompression 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T00:53:46.874 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T00:53:46.875 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.num_levels: 7 2024-03-24T00:53:46.875 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T00:53:46.876 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T00:53:46.877 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T00:53:46.878 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.table_properties_collectors: 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.inplace_update_support: 0 2024-03-24T00:53:46.879 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.bloom_locality: 0 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.max_successive_merges: 0 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.ttl: 2592000 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T00:53:46.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.enable_blob_files: false 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.min_blob_size: 0 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi038/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 37, last_sequence is 16518, log_number is 33,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 33 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 33 2024-03-24T00:53:46.881 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f3a5330f-515b-487e-976f-10635dfa202b 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711241626559452, "job": 1, "event": "recovery_started", "wal_files": [33]} 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #33 mode 2 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711241626566849, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 38, "file_size": 934138, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16519, "largest_seqno": 17886, "table_properties": {"data_size": 928435, "index_size": 3280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1413, "raw_key_size": 12663, "raw_average_key_size": 23, "raw_value_size": 917042, "raw_average_value_size": 1704, "num_data_blocks": 149, "num_entries": 538, "num_filter_entries": 538, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711241626, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f3a5330f-515b-487e-976f-10635dfa202b", "db_session_id": "HUX8I7PWGS6LP1EIE197", "orig_file_number": 38, "seqno_to_time_mapping": "N/A"}} 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711241626567016, "job": 1, "event": "recovery_finished"} 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/version_set.cc:5047] Creating manifest 40 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi038/store.db/000033.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563ae578c000 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: DB pointer 0x563ae5778000 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: ** DB Stats ** 2024-03-24T00:53:46.882 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: ** Compaction Stats [default] ** 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: L0 1/0 912.24 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 216.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: L6 1/0 7.84 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Sum 2/0 8.73 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 216.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-24T00:53:46.883 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 216.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: ** Compaction Stats [default] ** 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 216.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T00:53:46.884 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Cumulative compaction: 0.00 GB write, 55.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Interval compaction: 0.00 GB write, 55.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Block cache BinnedLRUCache@0x563ae569b090#2 capacity: 512.00 MB usage: 4.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.2e-05 secs_since: 0 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.53 KB,0.000292063%) IndexBlock(1,3.33 KB,0.000634789%) Misc(1,0.00 KB,0%) 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: starting mon.smithi038 rank 0 at public addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] at bind addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi038 fsid 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: mon.smithi038@-1(???) e2 preinit fsid 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: mon.smithi038@-1(???).mds e20 new map 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: mon.smithi038@-1(???).mds e20 print_map 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: e20 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:53:46.885 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: legacy client fscid: 1 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: Filesystem 'cephfs' (1) 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: fs_name cephfs 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: epoch 20 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: tableserver 0 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: root 0 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: session_timeout 60 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: session_autoclose 300 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: max_file_size 1099511627776 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: max_xattr_size 65536 2024-03-24T00:53:46.886 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: required_client_features {} 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: last_failure 0 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: last_failure_osd_epoch 47 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: max_mds 2 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: in 0,1 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: up {0=14462,1=24271} 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: failed 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: damaged 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: stopped 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: data_pools [3] 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: metadata_pool 2 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: inline_data disabled 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: balancer 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: bal_rank_mask -1 2024-03-24T00:53:46.887 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: standby_count_wanted 1 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: qdb_cluster leader: 0 members: 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: [mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: [mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: [mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: [mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout: 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: mon.smithi038@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: mon.smithi038@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: mon.smithi038@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: mon.smithi038@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 ceph-mon[112108]: mon.smithi038@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 bash[112090]: ba098af28413976f02407c0b6e9dab98a3c86c9ce6492c392fb9daff638f289b 2024-03-24T00:53:46.888 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:46 smithi038 systemd[1]: Started Ceph mon.smithi038 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:53:47.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:53:47.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:53:47.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: pgmap v136: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:53:47.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: mon.smithi038 calling monitor election 2024-03-24T00:53:47.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: mon.smithi038 is new leader, mons smithi038,smithi088 in quorum (ranks 0,1) 2024-03-24T00:53:47.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: monmap e2: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T00:53:47.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:active} 2 up:standby-replay 2024-03-24T00:53:47.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: osdmap e49: 6 total, 6 up, 6 in 2024-03-24T00:53:47.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: mgrmap e32: smithi038.huhioz(active, since 4m), standbys: smithi088.njctea 2024-03-24T00:53:47.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: overall HEALTH_OK 2024-03-24T00:53:47.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: from='mgr.14664 ' entity='' 2024-03-24T00:53:47.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:47 smithi038 ceph-mon[112108]: mgrmap e33: smithi038.huhioz(active, since 4m), standbys: smithi088.njctea 2024-03-24T00:53:47.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:53:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: from='mgr.14664 172.21.15.38:0/1110976585' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:53:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: pgmap v136: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:53:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: mon.smithi038 calling monitor election 2024-03-24T00:53:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: mon.smithi038 is new leader, mons smithi038,smithi088 in quorum (ranks 0,1) 2024-03-24T00:53:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: monmap e2: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T00:53:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:active} 2 up:standby-replay 2024-03-24T00:53:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: osdmap e49: 6 total, 6 up, 6 in 2024-03-24T00:53:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: mgrmap e32: smithi038.huhioz(active, since 4m), standbys: smithi088.njctea 2024-03-24T00:53:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: overall HEALTH_OK 2024-03-24T00:53:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: from='mgr.14664 ' entity='' 2024-03-24T00:53:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:47 smithi088 ceph-mon[36610]: mgrmap e33: smithi038.huhioz(active, since 4m), standbys: smithi088.njctea 2024-03-24T00:53:56.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:56 smithi038 ceph-mon[112108]: Standby manager daemon smithi088.njctea restarted 2024-03-24T00:53:56.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:56 smithi038 ceph-mon[112108]: Standby manager daemon smithi088.njctea started 2024-03-24T00:53:56.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:56 smithi038 ceph-mon[112108]: from='mgr.? 172.21.15.88:0/107772406' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/crt"}]: dispatch 2024-03-24T00:53:56.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:56 smithi038 ceph-mon[112108]: from='mgr.? 172.21.15.88:0/107772406' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:53:56.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:56 smithi038 ceph-mon[112108]: from='mgr.? 172.21.15.88:0/107772406' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/key"}]: dispatch 2024-03-24T00:53:56.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:56 smithi038 ceph-mon[112108]: from='mgr.? 172.21.15.88:0/107772406' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:53:56.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:56 smithi088 ceph-mon[36610]: Standby manager daemon smithi088.njctea restarted 2024-03-24T00:53:56.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:56 smithi088 ceph-mon[36610]: Standby manager daemon smithi088.njctea started 2024-03-24T00:53:56.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:56 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/107772406' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/crt"}]: dispatch 2024-03-24T00:53:56.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:56 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/107772406' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T00:53:56.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:56 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/107772406' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.njctea/key"}]: dispatch 2024-03-24T00:53:56.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:56 smithi088 ceph-mon[36610]: from='mgr.? 172.21.15.88:0/107772406' entity='mgr.smithi088.njctea' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T00:53:57.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: mgrmap e34: smithi038.huhioz(active, since 4m), standbys: smithi088.njctea 2024-03-24T00:53:57.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: Active manager daemon smithi038.huhioz restarted 2024-03-24T00:53:57.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: Activating manager daemon smithi038.huhioz 2024-03-24T00:53:57.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: osdmap e50: 6 total, 6 up, 6 in 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: mgrmap e35: smithi038.huhioz(active, starting, since 0.0107581s), standbys: smithi088.njctea 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi038.huhioz", "id": "smithi038.huhioz"}]: dispatch 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi088.njctea", "id": "smithi088.njctea"}]: dispatch 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:53:57.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:53:57.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:53:57.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:53:57.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T00:53:57.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T00:53:57.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T00:53:57.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: Manager daemon smithi038.huhioz is now available 2024-03-24T00:53:57.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:53:57.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:53:57.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:53:57.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:53:57.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:57 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/trash_purge_schedule"}]: dispatch 2024-03-24T00:53:57.567 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: mgrmap e34: smithi038.huhioz(active, since 4m), standbys: smithi088.njctea 2024-03-24T00:53:57.567 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: Active manager daemon smithi038.huhioz restarted 2024-03-24T00:53:57.567 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: Activating manager daemon smithi038.huhioz 2024-03-24T00:53:57.567 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: osdmap e50: 6 total, 6 up, 6 in 2024-03-24T00:53:57.567 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: mgrmap e35: smithi038.huhioz(active, starting, since 0.0107581s), standbys: smithi088.njctea 2024-03-24T00:53:57.567 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:53:57.567 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:53:57.567 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:53:57.567 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi038.huhioz", "id": "smithi038.huhioz"}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr metadata", "who": "smithi088.njctea", "id": "smithi088.njctea"}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T00:53:57.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: Manager daemon smithi038.huhioz is now available 2024-03-24T00:53:57.569 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:53:57.569 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:53:57.569 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:53:57.569 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/mirror_snapshot_schedule"}]: dispatch 2024-03-24T00:53:57.569 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.huhioz/trash_purge_schedule"}]: dispatch 2024-03-24T00:53:58.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:58 smithi088 ceph-mon[36610]: [24/Mar/2024:00:53:57] ENGINE Bus STARTING 2024-03-24T00:53:58.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:58 smithi088 ceph-mon[36610]: [24/Mar/2024:00:53:57] ENGINE Serving on http://172.21.15.38:8765 2024-03-24T00:53:58.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:58 smithi088 ceph-mon[36610]: [24/Mar/2024:00:53:57] ENGINE Serving on https://172.21.15.38:7150 2024-03-24T00:53:58.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:58 smithi088 ceph-mon[36610]: [24/Mar/2024:00:53:57] ENGINE Bus STARTED 2024-03-24T00:53:58.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:58 smithi088 ceph-mon[36610]: mgrmap e36: smithi038.huhioz(active, since 1.02411s), standbys: smithi088.njctea 2024-03-24T00:53:58.439 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:58 smithi038 ceph-mon[112108]: [24/Mar/2024:00:53:57] ENGINE Bus STARTING 2024-03-24T00:53:58.439 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:58 smithi038 ceph-mon[112108]: [24/Mar/2024:00:53:57] ENGINE Serving on http://172.21.15.38:8765 2024-03-24T00:53:58.439 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:58 smithi038 ceph-mon[112108]: [24/Mar/2024:00:53:57] ENGINE Serving on https://172.21.15.38:7150 2024-03-24T00:53:58.439 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:58 smithi038 ceph-mon[112108]: [24/Mar/2024:00:53:57] ENGINE Bus STARTED 2024-03-24T00:53:58.439 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:58 smithi038 ceph-mon[112108]: mgrmap e36: smithi038.huhioz(active, since 1.02411s), standbys: smithi088.njctea 2024-03-24T00:53:59.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:53:59 smithi088 ceph-mon[36610]: pgmap v3: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-24T00:53:59.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:53:59 smithi038 ceph-mon[112108]: pgmap v3: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-24T00:54:00.424 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:00 smithi038 ceph-mon[112108]: pgmap v4: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-24T00:54:00.425 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:00 smithi038 ceph-mon[112108]: mgrmap e37: smithi038.huhioz(active, since 2s), standbys: smithi088.njctea 2024-03-24T00:54:00.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:00 smithi088 ceph-mon[36610]: pgmap v4: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-24T00:54:00.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:00 smithi088 ceph-mon[36610]: mgrmap e37: smithi038.huhioz(active, since 2s), standbys: smithi088.njctea 2024-03-24T00:54:02.447 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:02 smithi038 ceph-mon[112108]: pgmap v5: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-24T00:54:02.447 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:02 smithi038 ceph-mon[112108]: mgrmap e38: smithi038.huhioz(active, since 4s), standbys: smithi088.njctea 2024-03-24T00:54:02.447 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:02 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:02.447 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:02 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:02 smithi088 ceph-mon[36610]: pgmap v5: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-24T00:54:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:02 smithi088 ceph-mon[36610]: mgrmap e38: smithi038.huhioz(active, since 4s), standbys: smithi088.njctea 2024-03-24T00:54:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:02 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:02 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:04.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:04 smithi038 ceph-mon[112108]: pgmap v6: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-24T00:54:04.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:04 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:04.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:04 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:04.636 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:04 smithi088 ceph-mon[36610]: pgmap v6: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-24T00:54:04.636 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:04 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:04.636 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:04 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:05.698 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:05 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:05.698 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:05 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:05.699 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:05 smithi088 ceph-mon[36610]: pgmap v7: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-24T00:54:05.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:05.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:05.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:05 smithi038 ceph-mon[112108]: pgmap v7: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-24T00:54:07.856 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:07 smithi038 ceph-mon[112108]: pgmap v8: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-24T00:54:07.857 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:07 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:07.857 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:07 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:07.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:07 smithi088 ceph-mon[36610]: pgmap v8: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-24T00:54:07.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:07 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:07.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:07 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:09.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:09 smithi038 ceph-mon[112108]: pgmap v9: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-24T00:54:09.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:09 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:09.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:09 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:09.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:09 smithi088 ceph-mon[36610]: pgmap v9: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-24T00:54:09.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:09 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:09.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:09 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.415+0000 7f77a4000700 1 -- 172.21.15.38:0/953810764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f779c100a70 msgr2=0x7f779c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.415+0000 7f77a4000700 1 --2- 172.21.15.38:0/953810764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f779c100a70 0x7f779c100ef0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7788009910 tx=0x7f7788009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.415+0000 7f77a4000700 1 -- 172.21.15.38:0/953810764 shutdown_connections 2024-03-24T00:54:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.415+0000 7f77a4000700 1 --2- 172.21.15.38:0/953810764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f779c100a70 0x7f779c100ef0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.415+0000 7f77a4000700 1 --2- 172.21.15.38:0/953810764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f779c106a70 0x7f779c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.415+0000 7f77a4000700 1 -- 172.21.15.38:0/953810764 >> 172.21.15.38:0/953810764 conn(0x7f779c0fc580 msgr2=0x7f779c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.415+0000 7f77a4000700 1 -- 172.21.15.38:0/953810764 shutdown_connections 2024-03-24T00:54:10.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.415+0000 7f77a4000700 1 -- 172.21.15.38:0/953810764 wait complete. 2024-03-24T00:54:10.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.417+0000 7f77a4000700 1 Processor -- start 2024-03-24T00:54:10.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.417+0000 7f77a4000700 1 -- start start 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.417+0000 7f77a4000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f779c100a70 0x7f779c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.418+0000 7f77a4000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f779c106a70 0x7f779c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.418+0000 7f77a4000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c19f3f0 con 0x7f779c100a70 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.418+0000 7f77a4000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c198a20 con 0x7f779c106a70 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.418+0000 7f77a1600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f779c100a70 0x7f779c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.418+0000 7f77a1600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f779c100a70 0x7f779c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58794/0 (socket says 172.21.15.38:58794) 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.418+0000 7f77a1600700 1 -- 172.21.15.38:0/2815847065 learned_addr learned my addr 172.21.15.38:0/2815847065 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.418+0000 7f77a0c00700 1 --2- 172.21.15.38:0/2815847065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f779c106a70 0x7f779c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.419+0000 7f77a1600700 1 -- 172.21.15.38:0/2815847065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f779c106a70 msgr2=0x7f779c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.419+0000 7f77a1600700 1 --2- 172.21.15.38:0/2815847065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f779c106a70 0x7f779c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.419+0000 7f77a1600700 1 -- 172.21.15.38:0/2815847065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77880095f0 con 0x7f779c100a70 2024-03-24T00:54:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.419+0000 7f77a1600700 1 --2- 172.21.15.38:0/2815847065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f779c100a70 0x7f779c19e890 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f779000d8d0 tx=0x7f779000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.419+0000 7f779a000700 1 -- 172.21.15.38:0/2815847065 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7790009940 con 0x7f779c100a70 2024-03-24T00:54:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.419+0000 7f77a4000700 1 -- 172.21.15.38:0/2815847065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c198d00 con 0x7f779c100a70 2024-03-24T00:54:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.419+0000 7f779a000700 1 -- 172.21.15.38:0/2815847065 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7790010460 con 0x7f779c100a70 2024-03-24T00:54:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.419+0000 7f779a000700 1 -- 172.21.15.38:0/2815847065 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f779000f5f0 con 0x7f779c100a70 2024-03-24T00:54:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.419+0000 7f77a4000700 1 -- 172.21.15.38:0/2815847065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c199250 con 0x7f779c100a70 2024-03-24T00:54:10.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.421+0000 7f77a4000700 1 -- 172.21.15.38:0/2815847065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779c104d20 con 0x7f779c100a70 2024-03-24T00:54:10.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.422+0000 7f779a000700 1 -- 172.21.15.38:0/2815847065 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f77900105d0 con 0x7f779c100a70 2024-03-24T00:54:10.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.423+0000 7f779a000700 1 --2- 172.21.15.38:0/2815847065 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f778406e0e0 0x7f77840705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:10.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.423+0000 7f779a000700 1 -- 172.21.15.38:0/2815847065 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f779008dfa0 con 0x7f779c100a70 2024-03-24T00:54:10.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.423+0000 7f77a0c00700 1 --2- 172.21.15.38:0/2815847065 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f778406e0e0 0x7f77840705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:10.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.424+0000 7f77a0c00700 1 --2- 172.21.15.38:0/2815847065 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f778406e0e0 0x7f77840705a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f77880059d0 tx=0x7f778801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.433+0000 7f779a000700 1 -- 172.21.15.38:0/2815847065 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f779005ad80 con 0x7f779c100a70 2024-03-24T00:54:10.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.688+0000 7f77a4000700 1 -- 172.21.15.38:0/2815847065 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f779c199e90 con 0x7f778406e0e0 2024-03-24T00:54:10.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.690+0000 7f779a000700 1 -- 172.21.15.38:0/2815847065 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f779c199e90 con 0x7f778406e0e0 2024-03-24T00:54:10.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.693+0000 7f77a4000700 1 -- 172.21.15.38:0/2815847065 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f778406e0e0 msgr2=0x7f77840705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:10.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.693+0000 7f77a4000700 1 --2- 172.21.15.38:0/2815847065 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f778406e0e0 0x7f77840705a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f77880059d0 tx=0x7f778801a040 comp rx=0 tx=0).stop 2024-03-24T00:54:10.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.693+0000 7f77a4000700 1 -- 172.21.15.38:0/2815847065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f779c100a70 msgr2=0x7f779c19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:10.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.693+0000 7f77a4000700 1 --2- 172.21.15.38:0/2815847065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f779c100a70 0x7f779c19e890 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f779000d8d0 tx=0x7f779000dc90 comp rx=0 tx=0).stop 2024-03-24T00:54:10.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.693+0000 7f77a4000700 1 -- 172.21.15.38:0/2815847065 shutdown_connections 2024-03-24T00:54:10.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.693+0000 7f77a4000700 1 --2- 172.21.15.38:0/2815847065 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f778406e0e0 0x7f77840705a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:10.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.693+0000 7f77a4000700 1 --2- 172.21.15.38:0/2815847065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f779c100a70 0x7f779c19e890 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:10.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.693+0000 7f77a4000700 1 --2- 172.21.15.38:0/2815847065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f779c106a70 0x7f779c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:10.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.693+0000 7f77a4000700 1 -- 172.21.15.38:0/2815847065 >> 172.21.15.38:0/2815847065 conn(0x7f779c0fc580 msgr2=0x7f779c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:10.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.694+0000 7f77a4000700 1 -- 172.21.15.38:0/2815847065 shutdown_connections 2024-03-24T00:54:10.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.694+0000 7f77a4000700 1 -- 172.21.15.38:0/2815847065 wait complete. 2024-03-24T00:54:10.716 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:54:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.899+0000 7f7f83400700 1 -- 172.21.15.38:0/3539835935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f7c101c10 msgr2=0x7f7f7c101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.900+0000 7f7f83400700 1 --2- 172.21.15.38:0/3539835935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f7c101c10 0x7f7f7c101ff0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009910 tx=0x7f7f70009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.900+0000 7f7f83400700 1 -- 172.21.15.38:0/3539835935 shutdown_connections 2024-03-24T00:54:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.900+0000 7f7f83400700 1 --2- 172.21.15.38:0/3539835935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f7c102530 0x7f7f7c10aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.900+0000 7f7f83400700 1 --2- 172.21.15.38:0/3539835935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f7c101c10 0x7f7f7c101ff0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.900+0000 7f7f83400700 1 -- 172.21.15.38:0/3539835935 >> 172.21.15.38:0/3539835935 conn(0x7f7f7c076640 msgr2=0x7f7f7c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:10.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.900+0000 7f7f83400700 1 -- 172.21.15.38:0/3539835935 shutdown_connections 2024-03-24T00:54:10.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.900+0000 7f7f83400700 1 -- 172.21.15.38:0/3539835935 wait complete. 2024-03-24T00:54:10.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.901+0000 7f7f83400700 1 Processor -- start 2024-03-24T00:54:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.902+0000 7f7f83400700 1 -- start start 2024-03-24T00:54:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.902+0000 7f7f83400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f7c101c10 0x7f7f7c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.902+0000 7f7f83400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f7c102530 0x7f7f7c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.902+0000 7f7f83400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c19f520 con 0x7f7f7c101c10 2024-03-24T00:54:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.902+0000 7f7f83400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c198a90 con 0x7f7f7c102530 2024-03-24T00:54:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.903+0000 7f7f80a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f7c101c10 0x7f7f7c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.903+0000 7f7f80a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f7c101c10 0x7f7f7c19e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58808/0 (socket says 172.21.15.38:58808) 2024-03-24T00:54:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.903+0000 7f7f80a00700 1 -- 172.21.15.38:0/4067322100 learned_addr learned my addr 172.21.15.38:0/4067322100 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.903+0000 7f7f7be00700 1 --2- 172.21.15.38:0/4067322100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f7c102530 0x7f7f7c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.903+0000 7f7f80a00700 1 -- 172.21.15.38:0/4067322100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f7c102530 msgr2=0x7f7f7c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.903+0000 7f7f80a00700 1 --2- 172.21.15.38:0/4067322100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f7c102530 0x7f7f7c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.903+0000 7f7f80a00700 1 -- 172.21.15.38:0/4067322100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700095f0 con 0x7f7f7c101c10 2024-03-24T00:54:10.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.904+0000 7f7f80a00700 1 --2- 172.21.15.38:0/4067322100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f7c101c10 0x7f7f7c19e950 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7f700092e0 tx=0x7f7f7000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:10.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.904+0000 7f7f79600700 1 -- 172.21.15.38:0/4067322100 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f7001d070 con 0x7f7f7c101c10 2024-03-24T00:54:10.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.904+0000 7f7f79600700 1 -- 172.21.15.38:0/4067322100 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f700056e0 con 0x7f7f7c101c10 2024-03-24T00:54:10.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.904+0000 7f7f79600700 1 -- 172.21.15.38:0/4067322100 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f7000fe80 con 0x7f7f7c101c10 2024-03-24T00:54:10.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.904+0000 7f7f83400700 1 -- 172.21.15.38:0/4067322100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c198d10 con 0x7f7f7c101c10 2024-03-24T00:54:10.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.904+0000 7f7f83400700 1 -- 172.21.15.38:0/4067322100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c199200 con 0x7f7f7c101c10 2024-03-24T00:54:10.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.906+0000 7f7f83400700 1 -- 172.21.15.38:0/4067322100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c068d60 con 0x7f7f7c101c10 2024-03-24T00:54:10.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.907+0000 7f7f79600700 1 -- 172.21.15.38:0/4067322100 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f7f7000f6f0 con 0x7f7f7c101c10 2024-03-24T00:54:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.909+0000 7f7f79600700 1 --2- 172.21.15.38:0/4067322100 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7f6406e120 0x7f7f640705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.909+0000 7f7f79600700 1 -- 172.21.15.38:0/4067322100 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7f7008f2c0 con 0x7f7f7c101c10 2024-03-24T00:54:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.909+0000 7f7f7be00700 1 --2- 172.21.15.38:0/4067322100 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7f6406e120 0x7f7f640705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.910+0000 7f7f7be00700 1 --2- 172.21.15.38:0/4067322100 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7f6406e120 0x7f7f640705e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7f68005950 tx=0x7f7f680058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:10.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:10.918+0000 7f7f79600700 1 -- 172.21.15.38:0/4067322100 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f7005c100 con 0x7f7f7c101c10 2024-03-24T00:54:11.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.186+0000 7f7f83400700 1 -- 172.21.15.38:0/4067322100 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f7c199470 con 0x7f7f6406e120 2024-03-24T00:54:11.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.189+0000 7f7f79600700 1 -- 172.21.15.38:0/4067322100 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f7f7c199470 con 0x7f7f6406e120 2024-03-24T00:54:11.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.191+0000 7f7f83400700 1 -- 172.21.15.38:0/4067322100 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7f6406e120 msgr2=0x7f7f640705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:11.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.191+0000 7f7f83400700 1 --2- 172.21.15.38:0/4067322100 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7f6406e120 0x7f7f640705e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7f68005950 tx=0x7f7f680058e0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.191+0000 7f7f83400700 1 -- 172.21.15.38:0/4067322100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f7c101c10 msgr2=0x7f7f7c19e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:11.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.191+0000 7f7f83400700 1 --2- 172.21.15.38:0/4067322100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f7c101c10 0x7f7f7c19e950 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7f700092e0 tx=0x7f7f7000be20 comp rx=0 tx=0).stop 2024-03-24T00:54:11.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.192+0000 7f7f83400700 1 -- 172.21.15.38:0/4067322100 shutdown_connections 2024-03-24T00:54:11.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.192+0000 7f7f83400700 1 --2- 172.21.15.38:0/4067322100 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7f6406e120 0x7f7f640705e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.192+0000 7f7f83400700 1 --2- 172.21.15.38:0/4067322100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f7c101c10 0x7f7f7c19e950 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.192+0000 7f7f83400700 1 --2- 172.21.15.38:0/4067322100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f7c102530 0x7f7f7c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.192+0000 7f7f83400700 1 -- 172.21.15.38:0/4067322100 >> 172.21.15.38:0/4067322100 conn(0x7f7f7c076640 msgr2=0x7f7f7c10a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:11.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.192+0000 7f7f83400700 1 -- 172.21.15.38:0/4067322100 shutdown_connections 2024-03-24T00:54:11.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.192+0000 7f7f83400700 1 -- 172.21.15.38:0/4067322100 wait complete. 2024-03-24T00:54:11.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.345+0000 7ff7a5800700 1 -- 172.21.15.38:0/2126629374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7a0106a70 msgr2=0x7ff7a0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.345+0000 7ff7a5800700 1 --2- 172.21.15.38:0/2126629374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7a0106a70 0x7ff7a0106e50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff790009910 tx=0x7ff790009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.345+0000 7ff7a5800700 1 -- 172.21.15.38:0/2126629374 shutdown_connections 2024-03-24T00:54:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.345+0000 7ff7a5800700 1 --2- 172.21.15.38:0/2126629374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a0100a70 0x7ff7a0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.345+0000 7ff7a5800700 1 --2- 172.21.15.38:0/2126629374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7a0106a70 0x7ff7a0106e50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.345+0000 7ff7a5800700 1 -- 172.21.15.38:0/2126629374 >> 172.21.15.38:0/2126629374 conn(0x7ff7a00fc580 msgr2=0x7ff7a00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.345+0000 7ff7a5800700 1 -- 172.21.15.38:0/2126629374 shutdown_connections 2024-03-24T00:54:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.345+0000 7ff7a5800700 1 -- 172.21.15.38:0/2126629374 wait complete. 2024-03-24T00:54:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.346+0000 7ff7a5800700 1 Processor -- start 2024-03-24T00:54:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.347+0000 7ff7a5800700 1 -- start start 2024-03-24T00:54:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.347+0000 7ff7a5800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7a0100a70 0x7ff7a019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.347+0000 7ff7a5800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a0106a70 0x7ff7a019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.347+0000 7ff7a5800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019f440 con 0x7ff7a0100a70 2024-03-24T00:54:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.347+0000 7ff7a5800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a0198a20 con 0x7ff7a0106a70 2024-03-24T00:54:11.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.347+0000 7ff79ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7a0100a70 0x7ff7a019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:11.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.348+0000 7ff79ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7a0100a70 0x7ff7a019e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58828/0 (socket says 172.21.15.38:58828) 2024-03-24T00:54:11.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.348+0000 7ff79ea00700 1 -- 172.21.15.38:0/3907306081 learned_addr learned my addr 172.21.15.38:0/3907306081 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:11.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.348+0000 7ff79e000700 1 --2- 172.21.15.38:0/3907306081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a0106a70 0x7ff7a019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:11.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.348+0000 7ff79ea00700 1 -- 172.21.15.38:0/3907306081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a0106a70 msgr2=0x7ff7a019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:11.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.348+0000 7ff79ea00700 1 --2- 172.21.15.38:0/3907306081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a0106a70 0x7ff7a019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.348+0000 7ff79ea00700 1 -- 172.21.15.38:0/3907306081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900095f0 con 0x7ff7a0100a70 2024-03-24T00:54:11.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.348+0000 7ff79ea00700 1 --2- 172.21.15.38:0/3907306081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7a0100a70 0x7ff7a019e8e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff7900092e0 tx=0x7ff79000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:11.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.349+0000 7ff797400700 1 -- 172.21.15.38:0/3907306081 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff79001d070 con 0x7ff7a0100a70 2024-03-24T00:54:11.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.349+0000 7ff797400700 1 -- 172.21.15.38:0/3907306081 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7900056e0 con 0x7ff7a0100a70 2024-03-24T00:54:11.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.349+0000 7ff797400700 1 -- 172.21.15.38:0/3907306081 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff79000fe80 con 0x7ff7a0100a70 2024-03-24T00:54:11.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.349+0000 7ff7a5800700 1 -- 172.21.15.38:0/3907306081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a0198ca0 con 0x7ff7a0100a70 2024-03-24T00:54:11.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.349+0000 7ff7a5800700 1 -- 172.21.15.38:0/3907306081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a0199190 con 0x7ff7a0100a70 2024-03-24T00:54:11.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.350+0000 7ff7a5800700 1 -- 172.21.15.38:0/3907306081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a0104d20 con 0x7ff7a0100a70 2024-03-24T00:54:11.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.352+0000 7ff797400700 1 -- 172.21.15.38:0/3907306081 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7ff79000f6f0 con 0x7ff7a0100a70 2024-03-24T00:54:11.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.353+0000 7ff797400700 1 --2- 172.21.15.38:0/3907306081 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff78406e0d0 0x7ff784070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:11.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.354+0000 7ff797400700 1 -- 172.21.15.38:0/3907306081 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff79008f3f0 con 0x7ff7a0100a70 2024-03-24T00:54:11.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.354+0000 7ff79e000700 1 --2- 172.21.15.38:0/3907306081 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff78406e0d0 0x7ff784070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:11.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.354+0000 7ff79e000700 1 --2- 172.21.15.38:0/3907306081 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff78406e0d0 0x7ff784070590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff78c005950 tx=0x7ff78c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:11.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.359+0000 7ff797400700 1 -- 172.21.15.38:0/3907306081 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff79005c1b0 con 0x7ff7a0100a70 2024-03-24T00:54:11.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.601+0000 7ff7a5800700 1 -- 172.21.15.38:0/3907306081 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7a0199ee0 con 0x7ff78406e0d0 2024-03-24T00:54:11.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.614+0000 7ff797400700 1 -- 172.21.15.38:0/3907306081 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff7a0199ee0 con 0x7ff78406e0d0 2024-03-24T00:54:11.615 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:54:11.615 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (2m) 7s ago 16m 14.4M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:54:11.615 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (16m) 7s ago 16m 8426k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:54:11.615 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (15m) 10s ago 15m 8228k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:54:11.615 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (16m) 7s ago 16m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:54:11.615 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (15m) 10s ago 15m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:54:11.615 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (113s) 7s ago 16m 75.2M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (9m) 7s ago 9m 97.5M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (9m) 7s ago 9m 151M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (9m) 10s ago 9m 24.5M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (9m) 10s ago 9m 24.8M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (5m) 7s ago 18m 478M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (4m) 10s ago 15m 447M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (25s) 7s ago 18m 29.0M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (15m) 10s ago 15m 49.6M 2048M 18.2.2-952-g738c2fd6 68aff9aff3b2 7850aeed066f 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (3m) 7s ago 16m 7684k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (3m) 10s ago 15m 7424k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (14m) 7s ago 14m 258M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (13m) 7s ago 13m 272M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (12m) 7s ago 12m 228M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (12m) 10s ago 12m 303M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:54:11.616 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (11m) 10s ago 11m 283M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:54:11.617 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (11m) 10s ago 11m 252M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:54:11.617 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (3m) 7s ago 15m 66.3M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:54:11.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.616+0000 7ff7a5800700 1 -- 172.21.15.38:0/3907306081 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff78406e0d0 msgr2=0x7ff784070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.616+0000 7ff7a5800700 1 --2- 172.21.15.38:0/3907306081 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff78406e0d0 0x7ff784070590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff78c005950 tx=0x7ff78c0058e0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.616+0000 7ff7a5800700 1 -- 172.21.15.38:0/3907306081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7a0100a70 msgr2=0x7ff7a019e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.616+0000 7ff7a5800700 1 --2- 172.21.15.38:0/3907306081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7a0100a70 0x7ff7a019e8e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff7900092e0 tx=0x7ff79000be20 comp rx=0 tx=0).stop 2024-03-24T00:54:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.616+0000 7ff7a5800700 1 -- 172.21.15.38:0/3907306081 shutdown_connections 2024-03-24T00:54:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.616+0000 7ff7a5800700 1 --2- 172.21.15.38:0/3907306081 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff78406e0d0 0x7ff784070590 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.616+0000 7ff7a5800700 1 --2- 172.21.15.38:0/3907306081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7a0100a70 0x7ff7a019e8e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.616+0000 7ff7a5800700 1 --2- 172.21.15.38:0/3907306081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a0106a70 0x7ff7a019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.616+0000 7ff7a5800700 1 -- 172.21.15.38:0/3907306081 >> 172.21.15.38:0/3907306081 conn(0x7ff7a00fc580 msgr2=0x7ff7a00fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.617+0000 7ff7a5800700 1 -- 172.21.15.38:0/3907306081 shutdown_connections 2024-03-24T00:54:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.617+0000 7ff7a5800700 1 -- 172.21.15.38:0/3907306081 wait complete. 2024-03-24T00:54:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.771+0000 7f87fdc00700 1 -- 172.21.15.38:0/2603222389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f8073b00 msgr2=0x7f87f810cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.771+0000 7f87fdc00700 1 --2- 172.21.15.38:0/2603222389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f8073b00 0x7f87f810cbe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009910 tx=0x7f87e4009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.771+0000 7f87fdc00700 1 -- 172.21.15.38:0/2603222389 shutdown_connections 2024-03-24T00:54:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.771+0000 7f87fdc00700 1 --2- 172.21.15.38:0/2603222389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f8073b00 0x7f87f810cbe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.771+0000 7f87fdc00700 1 --2- 172.21.15.38:0/2603222389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87f80731e0 0x7f87f80735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.771+0000 7f87fdc00700 1 -- 172.21.15.38:0/2603222389 >> 172.21.15.38:0/2603222389 conn(0x7f87f80fc3f0 msgr2=0x7f87f80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.771+0000 7f87fdc00700 1 -- 172.21.15.38:0/2603222389 shutdown_connections 2024-03-24T00:54:11.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.771+0000 7f87fdc00700 1 -- 172.21.15.38:0/2603222389 wait complete. 2024-03-24T00:54:11.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.772+0000 7f87fdc00700 1 Processor -- start 2024-03-24T00:54:11.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.773+0000 7f87fdc00700 1 -- start start 2024-03-24T00:54:11.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.773+0000 7f87fdc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f80731e0 0x7f87f819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:11.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.773+0000 7f87fdc00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87f8073b00 0x7f87f819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:11.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.773+0000 7f87fdc00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f819f2e0 con 0x7f87f80731e0 2024-03-24T00:54:11.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.773+0000 7f87fdc00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f81988a0 con 0x7f87f8073b00 2024-03-24T00:54:11.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.773+0000 7f87f7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f80731e0 0x7f87f819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:11.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.773+0000 7f87f7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f80731e0 0x7f87f819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58848/0 (socket says 172.21.15.38:58848) 2024-03-24T00:54:11.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.773+0000 7f87f7e00700 1 -- 172.21.15.38:0/1018891827 learned_addr learned my addr 172.21.15.38:0/1018891827 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:11.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.774+0000 7f87f7400700 1 --2- 172.21.15.38:0/1018891827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87f8073b00 0x7f87f819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.774+0000 7f87f7e00700 1 -- 172.21.15.38:0/1018891827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87f8073b00 msgr2=0x7f87f819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.774+0000 7f87f7e00700 1 --2- 172.21.15.38:0/1018891827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87f8073b00 0x7f87f819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.774+0000 7f87f7e00700 1 -- 172.21.15.38:0/1018891827 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e40095f0 con 0x7f87f80731e0 2024-03-24T00:54:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.774+0000 7f87f7e00700 1 --2- 172.21.15.38:0/1018891827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f80731e0 0x7f87f819e710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f87e800d8d0 tx=0x7f87e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.774+0000 7f87f4c00700 1 -- 172.21.15.38:0/1018891827 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e8009940 con 0x7f87f80731e0 2024-03-24T00:54:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.775+0000 7f87fdc00700 1 -- 172.21.15.38:0/1018891827 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f8198b80 con 0x7f87f80731e0 2024-03-24T00:54:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.775+0000 7f87f4c00700 1 -- 172.21.15.38:0/1018891827 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87e8010460 con 0x7f87f80731e0 2024-03-24T00:54:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.775+0000 7f87f4c00700 1 -- 172.21.15.38:0/1018891827 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e800f5f0 con 0x7f87f80731e0 2024-03-24T00:54:11.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.775+0000 7f87fdc00700 1 -- 172.21.15.38:0/1018891827 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f81990d0 con 0x7f87f80731e0 2024-03-24T00:54:11.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.776+0000 7f87fdc00700 1 -- 172.21.15.38:0/1018891827 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f810a380 con 0x7f87f80731e0 2024-03-24T00:54:11.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.777+0000 7f87f4c00700 1 -- 172.21.15.38:0/1018891827 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f87e80105d0 con 0x7f87f80731e0 2024-03-24T00:54:11.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.778+0000 7f87f4c00700 1 --2- 172.21.15.38:0/1018891827 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f87dc06e0e0 0x7f87dc0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:11.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.778+0000 7f87f4c00700 1 -- 172.21.15.38:0/1018891827 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f87e808df90 con 0x7f87f80731e0 2024-03-24T00:54:11.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.778+0000 7f87f7400700 1 --2- 172.21.15.38:0/1018891827 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f87dc06e0e0 0x7f87dc0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:11.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.779+0000 7f87f7400700 1 --2- 172.21.15.38:0/1018891827 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f87dc06e0e0 0x7f87dc0705a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f87e40059d0 tx=0x7f87e401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:11.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:11.783+0000 7f87f4c00700 1 -- 172.21.15.38:0/1018891827 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f87e805ad70 con 0x7f87f80731e0 2024-03-24T00:54:11.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:11 smithi038 ceph-mon[112108]: pgmap v10: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-24T00:54:11.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:11 smithi038 ceph-mon[112108]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:11.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:11 smithi038 ceph-mon[112108]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:11.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:11 smithi088 ceph-mon[36610]: pgmap v10: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-24T00:54:11.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:11 smithi088 ceph-mon[36610]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:11.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:11 smithi088 ceph-mon[36610]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:12.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.101+0000 7f87fdc00700 1 -- 172.21.15.38:0/1018891827 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f87f8199d10 con 0x7f87f80731e0 2024-03-24T00:54:12.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.102+0000 7f87f4c00700 1 -- 172.21.15.38:0/1018891827 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f87e805a900 con 0x7f87f80731e0 2024-03-24T00:54:12.103 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 1, 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:54:12.104 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:54:12.105 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 11, 2024-03-24T00:54:12.105 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 3 2024-03-24T00:54:12.105 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:54:12.105 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:54:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.105+0000 7f87fdc00700 1 -- 172.21.15.38:0/1018891827 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f87dc06e0e0 msgr2=0x7f87dc0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.105+0000 7f87fdc00700 1 --2- 172.21.15.38:0/1018891827 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f87dc06e0e0 0x7f87dc0705a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f87e40059d0 tx=0x7f87e401a040 comp rx=0 tx=0).stop 2024-03-24T00:54:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.105+0000 7f87fdc00700 1 -- 172.21.15.38:0/1018891827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f80731e0 msgr2=0x7f87f819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.105+0000 7f87fdc00700 1 --2- 172.21.15.38:0/1018891827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f80731e0 0x7f87f819e710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f87e800d8d0 tx=0x7f87e800dc90 comp rx=0 tx=0).stop 2024-03-24T00:54:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.105+0000 7f87fdc00700 1 -- 172.21.15.38:0/1018891827 shutdown_connections 2024-03-24T00:54:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.105+0000 7f87fdc00700 1 --2- 172.21.15.38:0/1018891827 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f87dc06e0e0 0x7f87dc0705a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.105+0000 7f87fdc00700 1 --2- 172.21.15.38:0/1018891827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f80731e0 0x7f87f819e710 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.105+0000 7f87fdc00700 1 --2- 172.21.15.38:0/1018891827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87f8073b00 0x7f87f819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.105+0000 7f87fdc00700 1 -- 172.21.15.38:0/1018891827 >> 172.21.15.38:0/1018891827 conn(0x7f87f80fc3f0 msgr2=0x7f87f81074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:12.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.105+0000 7f87fdc00700 1 -- 172.21.15.38:0/1018891827 shutdown_connections 2024-03-24T00:54:12.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.105+0000 7f87fdc00700 1 -- 172.21.15.38:0/1018891827 wait complete. 2024-03-24T00:54:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.253+0000 7f8c0b200700 1 -- 172.21.15.38:0/1785544513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c04102530 msgr2=0x7f8c0410aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.253+0000 7f8c0b200700 1 --2- 172.21.15.38:0/1785544513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c04102530 0x7f8c0410aa20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009910 tx=0x7f8bf0009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.253+0000 7f8c0b200700 1 -- 172.21.15.38:0/1785544513 shutdown_connections 2024-03-24T00:54:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.253+0000 7f8c0b200700 1 --2- 172.21.15.38:0/1785544513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c04102530 0x7f8c0410aa20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.253+0000 7f8c0b200700 1 --2- 172.21.15.38:0/1785544513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c04101c10 0x7f8c04101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.253+0000 7f8c0b200700 1 -- 172.21.15.38:0/1785544513 >> 172.21.15.38:0/1785544513 conn(0x7f8c04076640 msgr2=0x7f8c040fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.253+0000 7f8c0b200700 1 -- 172.21.15.38:0/1785544513 shutdown_connections 2024-03-24T00:54:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.253+0000 7f8c0b200700 1 -- 172.21.15.38:0/1785544513 wait complete. 2024-03-24T00:54:12.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.254+0000 7f8c0b200700 1 Processor -- start 2024-03-24T00:54:12.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.255+0000 7f8c0b200700 1 -- start start 2024-03-24T00:54:12.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.255+0000 7f8c0b200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c04101c10 0x7f8c0419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:12.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.255+0000 7f8c0b200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c04102530 0x7f8c0419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:12.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.255+0000 7f8c0b200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419d2c0 con 0x7f8c04101c10 2024-03-24T00:54:12.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.255+0000 7f8c0b200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04196830 con 0x7f8c04102530 2024-03-24T00:54:12.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.255+0000 7f8c03e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c04101c10 0x7f8c0419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:12.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.255+0000 7f8c03e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c04101c10 0x7f8c0419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58870/0 (socket says 172.21.15.38:58870) 2024-03-24T00:54:12.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.255+0000 7f8c03e00700 1 -- 172.21.15.38:0/1884460653 learned_addr learned my addr 172.21.15.38:0/1884460653 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:12.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.256+0000 7f8c03400700 1 --2- 172.21.15.38:0/1884460653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c04102530 0x7f8c0419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:12.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.256+0000 7f8c03e00700 1 -- 172.21.15.38:0/1884460653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c04102530 msgr2=0x7f8c0419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:12.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.256+0000 7f8c03e00700 1 --2- 172.21.15.38:0/1884460653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c04102530 0x7f8c0419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.256+0000 7f8c03e00700 1 -- 172.21.15.38:0/1884460653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf00095f0 con 0x7f8c04101c10 2024-03-24T00:54:12.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.256+0000 7f8c03e00700 1 --2- 172.21.15.38:0/1884460653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c04101c10 0x7f8c0419c6f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400e920 tx=0x7f8bf400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:12.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.257+0000 7f8c00c00700 1 -- 172.21.15.38:0/1884460653 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf400c9d0 con 0x7f8c04101c10 2024-03-24T00:54:12.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.257+0000 7f8c00c00700 1 -- 172.21.15.38:0/1884460653 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bf4010430 con 0x7f8c04101c10 2024-03-24T00:54:12.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.257+0000 7f8c00c00700 1 -- 172.21.15.38:0/1884460653 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf4018630 con 0x7f8c04101c10 2024-03-24T00:54:12.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.257+0000 7f8c0b200700 1 -- 172.21.15.38:0/1884460653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c04196b10 con 0x7f8c04101c10 2024-03-24T00:54:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.257+0000 7f8c0b200700 1 -- 172.21.15.38:0/1884460653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c04197030 con 0x7f8c04101c10 2024-03-24T00:54:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.258+0000 7f8c0b200700 1 -- 172.21.15.38:0/1884460653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c04105330 con 0x7f8c04101c10 2024-03-24T00:54:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.259+0000 7f8c00c00700 1 -- 172.21.15.38:0/1884460653 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f8bf40105a0 con 0x7f8c04101c10 2024-03-24T00:54:12.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.260+0000 7f8c00c00700 1 --2- 172.21.15.38:0/1884460653 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8be80724e0 0x7f8be80749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:12.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.260+0000 7f8c00c00700 1 -- 172.21.15.38:0/1884460653 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8bf4014070 con 0x7f8c04101c10 2024-03-24T00:54:12.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.261+0000 7f8c03400700 1 --2- 172.21.15.38:0/1884460653 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8be80724e0 0x7f8be80749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:12.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.261+0000 7f8c03400700 1 --2- 172.21.15.38:0/1884460653 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8be80724e0 0x7f8be80749a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0005b40 tx=0x7f8bf0005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:12.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.267+0000 7f8c00c00700 1 -- 172.21.15.38:0/1884460653 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8bf4053740 con 0x7f8c04101c10 2024-03-24T00:54:12.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.578+0000 7f8c0b200700 1 -- 172.21.15.38:0/1884460653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8c04068d60 con 0x7f8c04101c10 2024-03-24T00:54:12.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.579+0000 7f8c00c00700 1 -- 172.21.15.38:0/1884460653 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1900 (secure 0 0 0) 0x7f8bf405b040 con 0x7f8c04101c10 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout:e20 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 20 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:54:12.581 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:54:12.582 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:54:12.583 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:54:12.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.583+0000 7f8c0b200700 1 -- 172.21.15.38:0/1884460653 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8be80724e0 msgr2=0x7f8be80749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:12.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.583+0000 7f8c0b200700 1 --2- 172.21.15.38:0/1884460653 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8be80724e0 0x7f8be80749a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0005b40 tx=0x7f8bf0005a90 comp rx=0 tx=0).stop 2024-03-24T00:54:12.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.583+0000 7f8c0b200700 1 -- 172.21.15.38:0/1884460653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c04101c10 msgr2=0x7f8c0419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:12.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.583+0000 7f8c0b200700 1 --2- 172.21.15.38:0/1884460653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c04101c10 0x7f8c0419c6f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400e920 tx=0x7f8bf400ec30 comp rx=0 tx=0).stop 2024-03-24T00:54:12.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.583+0000 7f8c0b200700 1 -- 172.21.15.38:0/1884460653 shutdown_connections 2024-03-24T00:54:12.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.583+0000 7f8c0b200700 1 --2- 172.21.15.38:0/1884460653 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8be80724e0 0x7f8be80749a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.583+0000 7f8c0b200700 1 --2- 172.21.15.38:0/1884460653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c04101c10 0x7f8c0419c6f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.583+0000 7f8c0b200700 1 --2- 172.21.15.38:0/1884460653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c04102530 0x7f8c0419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.583+0000 7f8c0b200700 1 -- 172.21.15.38:0/1884460653 >> 172.21.15.38:0/1884460653 conn(0x7f8c04076640 msgr2=0x7f8c04102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:12.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.583+0000 7f8c0b200700 1 -- 172.21.15.38:0/1884460653 shutdown_connections 2024-03-24T00:54:12.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.583+0000 7f8c0b200700 1 -- 172.21.15.38:0/1884460653 wait complete. 2024-03-24T00:54:12.586 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:54:12.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.750+0000 7fd1a4800700 1 -- 172.21.15.38:0/90202356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c1025d0 msgr2=0x7fd19c10aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:12.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.750+0000 7fd1a4800700 1 --2- 172.21.15.38:0/90202356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c1025d0 0x7fd19c10aac0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009940 tx=0x7fd18c009c50 comp rx=0 tx=0).stop 2024-03-24T00:54:12.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.750+0000 7fd1a4800700 1 -- 172.21.15.38:0/90202356 shutdown_connections 2024-03-24T00:54:12.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.750+0000 7fd1a4800700 1 --2- 172.21.15.38:0/90202356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c1025d0 0x7fd19c10aac0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.750+0000 7fd1a4800700 1 --2- 172.21.15.38:0/90202356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101cb0 0x7fd19c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.750+0000 7fd1a4800700 1 -- 172.21.15.38:0/90202356 >> 172.21.15.38:0/90202356 conn(0x7fd19c0fb4c0 msgr2=0x7fd19c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:12.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.750+0000 7fd1a4800700 1 -- 172.21.15.38:0/90202356 shutdown_connections 2024-03-24T00:54:12.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.750+0000 7fd1a4800700 1 -- 172.21.15.38:0/90202356 wait complete. 2024-03-24T00:54:12.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.752+0000 7fd1a4800700 1 Processor -- start 2024-03-24T00:54:12.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.752+0000 7fd1a4800700 1 -- start start 2024-03-24T00:54:12.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.752+0000 7fd1a4800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101cb0 0x7fd19c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:12.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.752+0000 7fd1a4800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c1025d0 0x7fd19c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:12.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.752+0000 7fd1a4800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19f480 con 0x7fd19c1025d0 2024-03-24T00:54:12.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.752+0000 7fd1a4800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c198a60 con 0x7fd19c101cb0 2024-03-24T00:54:12.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.753+0000 7fd1a1e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101cb0 0x7fd19c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:12.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.753+0000 7fd1a1400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c1025d0 0x7fd19c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.753+0000 7fd1a1e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101cb0 0x7fd19c19e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.38:60360/0 (socket says 172.21.15.38:60360) 2024-03-24T00:54:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.753+0000 7fd1a1400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c1025d0 0x7fd19c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58890/0 (socket says 172.21.15.38:58890) 2024-03-24T00:54:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.753+0000 7fd1a1e00700 1 -- 172.21.15.38:0/3103676298 learned_addr learned my addr 172.21.15.38:0/3103676298 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.753+0000 7fd1a1e00700 1 -- 172.21.15.38:0/3103676298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c1025d0 msgr2=0x7fd19c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.753+0000 7fd1a1e00700 1 --2- 172.21.15.38:0/3103676298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c1025d0 0x7fd19c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.753+0000 7fd1a1e00700 1 -- 172.21.15.38:0/3103676298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0095f0 con 0x7fd19c101cb0 2024-03-24T00:54:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.753+0000 7fd1a1400700 1 --2- 172.21.15.38:0/3103676298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c1025d0 0x7fd19c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T00:54:12.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.754+0000 7fd1a1e00700 1 --2- 172.21.15.38:0/3103676298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101cb0 0x7fd19c19e920 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd19000e920 tx=0x7fd19000ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:12.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.754+0000 7fd196a00700 1 -- 172.21.15.38:0/3103676298 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd19000c9d0 con 0x7fd19c101cb0 2024-03-24T00:54:12.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.754+0000 7fd1a4800700 1 -- 172.21.15.38:0/3103676298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c198d40 con 0x7fd19c101cb0 2024-03-24T00:54:12.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.754+0000 7fd196a00700 1 -- 172.21.15.38:0/3103676298 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd190010430 con 0x7fd19c101cb0 2024-03-24T00:54:12.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.754+0000 7fd196a00700 1 -- 172.21.15.38:0/3103676298 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd190018630 con 0x7fd19c101cb0 2024-03-24T00:54:12.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.754+0000 7fd1a4800700 1 -- 172.21.15.38:0/3103676298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c199260 con 0x7fd19c101cb0 2024-03-24T00:54:12.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.755+0000 7fd1a4800700 1 -- 172.21.15.38:0/3103676298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c10a620 con 0x7fd19c101cb0 2024-03-24T00:54:12.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.757+0000 7fd196a00700 1 -- 172.21.15.38:0/3103676298 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91935+0+0 (secure 0 0 0) 0x7fd1900105a0 con 0x7fd19c101cb0 2024-03-24T00:54:12.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.758+0000 7fd196a00700 1 --2- 172.21.15.38:0/3103676298 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd18406e0d0 0x7fd184070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:12.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.758+0000 7fd196a00700 1 -- 172.21.15.38:0/3103676298 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd190014070 con 0x7fd19c101cb0 2024-03-24T00:54:12.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.758+0000 7fd1a1400700 1 --2- 172.21.15.38:0/3103676298 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd18406e0d0 0x7fd184070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:12.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.759+0000 7fd1a1400700 1 --2- 172.21.15.38:0/3103676298 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd18406e0d0 0x7fd184070590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009fa0 tx=0x7fd18c005ab0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:12.765+0000 7fd196a00700 1 -- 172.21.15.38:0/3103676298 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd190053740 con 0x7fd19c101cb0 2024-03-24T00:54:12.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:12 smithi038 ceph-mon[112108]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:12.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:54:12.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:12.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:12.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:12.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:54:12.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:12 smithi038 ceph-mon[112108]: Updating smithi038:/etc/ceph/ceph.conf 2024-03-24T00:54:12.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:12 smithi038 ceph-mon[112108]: Updating smithi088:/etc/ceph/ceph.conf 2024-03-24T00:54:12.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:12 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1018891827' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:54:12.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:12 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1884460653' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:54:12.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:12 smithi088 ceph-mon[36610]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:12.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:12 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:54:12.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:12 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:12.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:12 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:12.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:12 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:12.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:12 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:54:12.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:12 smithi088 ceph-mon[36610]: Updating smithi038:/etc/ceph/ceph.conf 2024-03-24T00:54:12.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:12 smithi088 ceph-mon[36610]: Updating smithi088:/etc/ceph/ceph.conf 2024-03-24T00:54:12.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:12 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1018891827' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:54:12.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:12 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/1884460653' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:54:13.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.018+0000 7fd1a4800700 1 -- 172.21.15.38:0/3103676298 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd19c1994d0 con 0x7fd18406e0d0 2024-03-24T00:54:13.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.020+0000 7fd196a00700 1 -- 172.21.15.38:0/3103676298 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fd19c1994d0 con 0x7fd18406e0d0 2024-03-24T00:54:13.021 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:54:13.021 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:54:13.021 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:54:13.021 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:54:13.021 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:54:13.021 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-03-24T00:54:13.021 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:54:13.021 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "3/23 daemons upgraded", 2024-03-24T00:54:13.021 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "", 2024-03-24T00:54:13.021 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:54:13.022 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:54:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.022+0000 7fd1a4800700 1 -- 172.21.15.38:0/3103676298 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd18406e0d0 msgr2=0x7fd184070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.022+0000 7fd1a4800700 1 --2- 172.21.15.38:0/3103676298 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd18406e0d0 0x7fd184070590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009fa0 tx=0x7fd18c005ab0 comp rx=0 tx=0).stop 2024-03-24T00:54:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.022+0000 7fd1a4800700 1 -- 172.21.15.38:0/3103676298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101cb0 msgr2=0x7fd19c19e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.022+0000 7fd1a4800700 1 --2- 172.21.15.38:0/3103676298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101cb0 0x7fd19c19e920 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd19000e920 tx=0x7fd19000ec30 comp rx=0 tx=0).stop 2024-03-24T00:54:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.023+0000 7fd1a4800700 1 -- 172.21.15.38:0/3103676298 shutdown_connections 2024-03-24T00:54:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.023+0000 7fd1a4800700 1 --2- 172.21.15.38:0/3103676298 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd18406e0d0 0x7fd184070590 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.023+0000 7fd1a4800700 1 --2- 172.21.15.38:0/3103676298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd19c101cb0 0x7fd19c19e920 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.023+0000 7fd1a4800700 1 --2- 172.21.15.38:0/3103676298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd19c1025d0 0x7fd19c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.023+0000 7fd1a4800700 1 -- 172.21.15.38:0/3103676298 >> 172.21.15.38:0/3103676298 conn(0x7fd19c0fb4c0 msgr2=0x7fd19c0fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.023+0000 7fd1a4800700 1 -- 172.21.15.38:0/3103676298 shutdown_connections 2024-03-24T00:54:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.023+0000 7fd1a4800700 1 -- 172.21.15.38:0/3103676298 wait complete. 2024-03-24T00:54:13.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.178+0000 7f671a000700 1 -- 172.21.15.38:0/3883160492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6714101c10 msgr2=0x7f6714101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:13.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.178+0000 7f671a000700 1 --2- 172.21.15.38:0/3883160492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6714101c10 0x7f6714101ff0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6704009910 tx=0x7f6704009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:13.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.178+0000 7f671a000700 1 -- 172.21.15.38:0/3883160492 shutdown_connections 2024-03-24T00:54:13.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.178+0000 7f671a000700 1 --2- 172.21.15.38:0/3883160492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6714102530 0x7f671410aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:13.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.178+0000 7f671a000700 1 --2- 172.21.15.38:0/3883160492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6714101c10 0x7f6714101ff0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:13.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.178+0000 7f671a000700 1 -- 172.21.15.38:0/3883160492 >> 172.21.15.38:0/3883160492 conn(0x7f6714076640 msgr2=0x7f67140fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:13.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.178+0000 7f671a000700 1 -- 172.21.15.38:0/3883160492 shutdown_connections 2024-03-24T00:54:13.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.178+0000 7f671a000700 1 -- 172.21.15.38:0/3883160492 wait complete. 2024-03-24T00:54:13.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.179+0000 7f671a000700 1 Processor -- start 2024-03-24T00:54:13.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.179+0000 7f671a000700 1 -- start start 2024-03-24T00:54:13.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.179+0000 7f671a000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6714101c10 0x7f671419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:13.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.179+0000 7f671a000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6714102530 0x7f671419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:13.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.179+0000 7f671a000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671419f3f0 con 0x7f6714101c10 2024-03-24T00:54:13.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.179+0000 7f671a000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67141989d0 con 0x7f6714102530 2024-03-24T00:54:13.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.180+0000 7f6713400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6714101c10 0x7f671419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:13.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.180+0000 7f6713400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6714101c10 0x7f671419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52334/0 (socket says 172.21.15.38:52334) 2024-03-24T00:54:13.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.180+0000 7f6713400700 1 -- 172.21.15.38:0/2440895478 learned_addr learned my addr 172.21.15.38:0/2440895478 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:13.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.180+0000 7f6712a00700 1 --2- 172.21.15.38:0/2440895478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6714102530 0x7f671419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:13.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.180+0000 7f6713400700 1 -- 172.21.15.38:0/2440895478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6714102530 msgr2=0x7f671419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:13.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.180+0000 7f6713400700 1 --2- 172.21.15.38:0/2440895478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6714102530 0x7f671419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:13.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.180+0000 7f6713400700 1 -- 172.21.15.38:0/2440895478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6700009520 con 0x7f6714101c10 2024-03-24T00:54:13.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.181+0000 7f6713400700 1 --2- 172.21.15.38:0/2440895478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6714101c10 0x7f671419e890 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6704009fd0 tx=0x7f6704005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:13.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.181+0000 7f670be00700 1 -- 172.21.15.38:0/2440895478 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f670401d070 con 0x7f6714101c10 2024-03-24T00:54:13.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.181+0000 7f670be00700 1 -- 172.21.15.38:0/2440895478 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67040052a0 con 0x7f6714101c10 2024-03-24T00:54:13.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.181+0000 7f671a000700 1 -- 172.21.15.38:0/2440895478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67040095f0 con 0x7f6714101c10 2024-03-24T00:54:13.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.181+0000 7f670be00700 1 -- 172.21.15.38:0/2440895478 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6704017400 con 0x7f6714101c10 2024-03-24T00:54:13.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.181+0000 7f671a000700 1 -- 172.21.15.38:0/2440895478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6714198fb0 con 0x7f6714101c10 2024-03-24T00:54:13.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.183+0000 7f671a000700 1 -- 172.21.15.38:0/2440895478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6714068d60 con 0x7f6714101c10 2024-03-24T00:54:13.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.184+0000 7f670be00700 1 -- 172.21.15.38:0/2440895478 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f670400f6f0 con 0x7f6714101c10 2024-03-24T00:54:13.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.185+0000 7f670be00700 1 --2- 172.21.15.38:0/2440895478 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f66f806e0e0 0x7f66f80705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:13.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.185+0000 7f670be00700 1 -- 172.21.15.38:0/2440895478 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f670408f220 con 0x7f6714101c10 2024-03-24T00:54:13.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.185+0000 7f6712a00700 1 --2- 172.21.15.38:0/2440895478 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f66f806e0e0 0x7f66f80705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:13.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.186+0000 7f6712a00700 1 --2- 172.21.15.38:0/2440895478 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f66f806e0e0 0x7f66f80705a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6714199b90 tx=0x7f6700009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:13.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.190+0000 7f670be00700 1 -- 172.21.15.38:0/2440895478 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f670405bfa0 con 0x7f6714101c10 2024-03-24T00:54:13.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.520+0000 7f671a000700 1 -- 172.21.15.38:0/2440895478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f671404ea80 con 0x7f6714101c10 2024-03-24T00:54:13.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.521+0000 7f670be00700 1 -- 172.21.15.38:0/2440895478 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f670405bb30 con 0x7f6714101c10 2024-03-24T00:54:13.522 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_OK 2024-03-24T00:54:13.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.523+0000 7f671a000700 1 -- 172.21.15.38:0/2440895478 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f66f806e0e0 msgr2=0x7f66f80705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:13.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.523+0000 7f671a000700 1 --2- 172.21.15.38:0/2440895478 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f66f806e0e0 0x7f66f80705a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6714199b90 tx=0x7f6700009260 comp rx=0 tx=0).stop 2024-03-24T00:54:13.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.523+0000 7f671a000700 1 -- 172.21.15.38:0/2440895478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6714101c10 msgr2=0x7f671419e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:13.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.523+0000 7f671a000700 1 --2- 172.21.15.38:0/2440895478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6714101c10 0x7f671419e890 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6704009fd0 tx=0x7f6704005ee0 comp rx=0 tx=0).stop 2024-03-24T00:54:13.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.523+0000 7f671a000700 1 -- 172.21.15.38:0/2440895478 shutdown_connections 2024-03-24T00:54:13.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.523+0000 7f671a000700 1 --2- 172.21.15.38:0/2440895478 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f66f806e0e0 0x7f66f80705a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:13.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.523+0000 7f671a000700 1 --2- 172.21.15.38:0/2440895478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6714101c10 0x7f671419e890 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:13.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.523+0000 7f671a000700 1 --2- 172.21.15.38:0/2440895478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6714102530 0x7f671419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:13.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.523+0000 7f671a000700 1 -- 172.21.15.38:0/2440895478 >> 172.21.15.38:0/2440895478 conn(0x7f6714076640 msgr2=0x7f6714100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:13.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.524+0000 7f671a000700 1 -- 172.21.15.38:0/2440895478 shutdown_connections 2024-03-24T00:54:13.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:13.524+0000 7f671a000700 1 -- 172.21.15.38:0/2440895478 wait complete. 2024-03-24T00:54:14.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:14 smithi038 ceph-mon[112108]: pgmap v11: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-24T00:54:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:14 smithi038 ceph-mon[112108]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:54:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:14 smithi038 ceph-mon[112108]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:54:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:14 smithi038 ceph-mon[112108]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:14 smithi038 ceph-mon[112108]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:54:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:14 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2440895478' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:54:14.412 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:14 smithi088 ceph-mon[36610]: pgmap v11: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-24T00:54:14.412 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:14 smithi088 ceph-mon[36610]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:54:14.412 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:14 smithi088 ceph-mon[36610]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T00:54:14.412 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:14 smithi088 ceph-mon[36610]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:14.412 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:14 smithi088 ceph-mon[36610]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:54:14.412 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:14 smithi088 ceph-mon[36610]: from='client.? 172.21.15.38:0/2440895478' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:54:15.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:15 smithi038 ceph-mon[112108]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:54:15.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:15 smithi038 ceph-mon[112108]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:54:15.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:15 smithi038 ceph-mon[112108]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:54:15.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:15 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:15.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:15 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:15.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:15 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:15.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:15 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:15.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:15 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:15.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:15 smithi088 ceph-mon[36610]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-03-24T00:54:15.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:15 smithi088 ceph-mon[36610]: Updating smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:54:15.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:15 smithi088 ceph-mon[36610]: Updating smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T00:54:15.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:15 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:15.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:15 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:15.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:15 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:15.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:15 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:15.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:15 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:16.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:16 smithi038 ceph-mon[112108]: pgmap v12: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-24T00:54:16.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:16 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:54:16.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:16 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:54:16.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:16 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T00:54:16.379 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:16 smithi088 ceph-mon[36610]: pgmap v12: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-24T00:54:16.379 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:16 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:54:16.379 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:16 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:54:16.379 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:16 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T00:54:18.644 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:18 smithi088 ceph-mon[36610]: pgmap v13: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:18.644 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:18 smithi088 ceph-mon[36610]: Upgrade: Updating mon.smithi088 2024-03-24T00:54:18.644 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:18 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:18.644 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:18 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:54:18.644 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:18 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:54:18.644 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:18 smithi088 ceph-mon[36610]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:18.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:18 smithi038 ceph-mon[112108]: pgmap v13: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:18.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:18 smithi038 ceph-mon[112108]: Upgrade: Updating mon.smithi088 2024-03-24T00:54:18.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:18 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:18.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:18 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:54:18.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:18 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:54:18.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:18 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:19.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:19 smithi088 ceph-mon[36610]: Deploying daemon mon.smithi088 on smithi088 2024-03-24T00:54:19.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:19 smithi038 ceph-mon[112108]: Deploying daemon mon.smithi088 on smithi088 2024-03-24T00:54:20.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:20 smithi038 ceph-mon[112108]: pgmap v14: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:20.658 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:20 smithi088 systemd[1]: Stopping Ceph mon.smithi088 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:54:20.658 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:20 smithi088 ceph-mon[36610]: pgmap v14: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:20.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:20 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088[36603]: 2024-03-24T00:54:20.656+0000 7f77be000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi088 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T00:54:20.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:20 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088[36603]: 2024-03-24T00:54:20.656+0000 7f77be000700 -1 mon.smithi088@1(peon) e2 *** Got Signal Terminated *** 2024-03-24T00:54:20.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:20 smithi088 podman[87191]: 2024-03-24 00:54:20.802654526 +0000 UTC m=+0.267425652 container died 7850aeed066fd515bc359d883e32a04103504f246ddb31f64d8d99cc40846533 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=reef-738c2fd) 2024-03-24T00:54:21.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:21 smithi088 podman[87191]: 2024-03-24 00:54:21.292808195 +0000 UTC m=+0.757579318 container cleanup 7850aeed066fd515bc359d883e32a04103504f246ddb31f64d8d99cc40846533 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=reef-738c2fd, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-24T00:54:21.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:21 smithi088 bash[87191]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088 2024-03-24T00:54:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:21 smithi088 podman[87206]: 2024-03-24 00:54:21.526387963 +0000 UTC m=+0.713884372 container remove 7850aeed066fd515bc359d883e32a04103504f246ddb31f64d8d99cc40846533 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, RELEASE=reef-738c2fd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-24T00:54:22.408 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:22 smithi088 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi088.service: Deactivated successfully. 2024-03-24T00:54:22.408 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:22 smithi088 systemd[1]: Stopped Ceph mon.smithi088 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:54:22.408 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:22 smithi088 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi088.service: Consumed 20.876s CPU time. 2024-03-24T00:54:22.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:22 smithi088 systemd[1]: Starting Ceph mon.smithi088 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:54:23.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:22 smithi088 podman[87332]: 2024-03-24 00:54:22.66495506 +0000 UTC m=+0.034231826 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:54:23.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:22 smithi088 podman[87332]: 2024-03-24 00:54:22.785426998 +0000 UTC m=+0.154703790 container create 3bace4fab2b0fb908fbcbd88a66e3edb7110cf4c2bc1a4d9e19c2056491af2e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-24T00:54:23.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 podman[87332]: 2024-03-24 00:54:23.091717072 +0000 UTC m=+0.460993888 container init 3bace4fab2b0fb908fbcbd88a66e3edb7110cf4c2bc1a4d9e19c2056491af2e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_BRANCH=HEAD) 2024-03-24T00:54:23.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 podman[87332]: 2024-03-24 00:54:23.102688812 +0000 UTC m=+0.471965588 container start 3bace4fab2b0fb908fbcbd88a66e3edb7110cf4c2bc1a4d9e19c2056491af2e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: pidfile_write: ignore empty --pid-file 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: load: jerasure load: lrc 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: RocksDB version: 7.9.2 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Git sha 0 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: DB SUMMARY 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: DB Session ID: BD8JC5TEWD3CV2KYE3RC 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: CURRENT file: CURRENT 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1626 Bytes 2024-03-24T00:54:23.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi088/store.db dir, Total Num: 1, files: 000030.sst 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi088/store.db: 000028.log size: 4544015 ; 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.error_if_exists: 0 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.create_if_missing: 0 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.paranoid_checks: 1 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.env: 0x560e7ccddb20 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.info_log: 0x560e7e53f840 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.statistics: (nil) 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.use_fsync: 0 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_log_file_size: 0 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T00:54:23.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T00:54:23.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T00:54:23.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T00:54:23.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.allow_fallocate: 1 2024-03-24T00:54:23.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T00:54:23.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T00:54:23.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.use_direct_reads: 0 2024-03-24T00:54:23.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T00:54:23.435 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T00:54:23.435 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.db_log_dir: 2024-03-24T00:54:23.435 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.wal_dir: 2024-03-24T00:54:23.435 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T00:54:23.435 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T00:54:23.435 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T00:54:23.435 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T00:54:23.435 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T00:54:23.435 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T00:54:23.435 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T00:54:23.435 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.write_buffer_manager: 0x560e7e5121e0 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.unordered_write: 0 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T00:54:23.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.row_cache: None 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.wal_filter: None 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.two_write_queues: 0 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.wal_compression: 0 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.atomic_flush: 0 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.log_readahead_size: 0 2024-03-24T00:54:23.437 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_background_jobs: 2 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_background_compactions: -1 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_subcompactions: 1 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T00:54:23.438 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_open_files: -1 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_background_flushes: -1 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Compression algorithms supported: 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T00:54:23.439 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: kZSTD supported: 0 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: kXpressCompression supported: 0 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: kZlibCompression supported: 1 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: kSnappyCompression supported: 1 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: kLZ4Compression supported: 1 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: kBZip2Compression supported: 0 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi088/store.db/MANIFEST-000010 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T00:54:23.440 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.merge_operator: 2024-03-24T00:54:23.441 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_filter: None 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560e7e53f940) 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: cache_index_and_filter_blocks: 1 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: pin_top_level_index_and_filter: 1 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_type: 0 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: data_block_index_type: 0 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_shortening: 1 2024-03-24T00:54:23.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: checksum: 4 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: no_block_cache: 0 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache: 0x560e7e545090 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_name: BinnedLRUCache 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_options: 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: capacity : 536870912 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: num_shard_bits : 4 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: strict_capacity_limit : 0 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: high_pri_pool_ratio: 0.000 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_compressed: (nil) 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: persistent_cache: (nil) 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_size: 4096 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_size_deviation: 10 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_restart_interval: 16 2024-03-24T00:54:23.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_block_restart_interval: 1 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: metadata_block_size: 4096 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: partition_filters: 0 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: use_delta_encoding: 1 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: filter_policy: bloomfilter 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: whole_key_filtering: 1 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: verify_compression: 0 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: read_amp_bytes_per_bit: 0 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: format_version: 5 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: enable_index_compression: 1 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_align: 0 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: max_auto_readahead_size: 262144 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: prepopulate_block_cache: 0 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: initial_auto_readahead_size: 8192 2024-03-24T00:54:23.444 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compression: NoCompression 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.num_levels: 7 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T00:54:23.445 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T00:54:23.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T00:54:23.447 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T00:54:23.448 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.table_properties_collectors: 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.inplace_update_support: 0 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.bloom_locality: 0 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.max_successive_merges: 0 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T00:54:23.449 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.ttl: 2592000 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.enable_blob_files: false 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.min_blob_size: 0 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T00:54:23.450 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi088/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 32, last_sequence is 16842, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 28 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c7c1ceb2-c7ae-4362-880a-60bb5eb2f13c 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711241663158023, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711241663176072, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2850842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16847, "largest_seqno": 18956, "table_properties": {"data_size": 2842708, "index_size": 5130, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1989, "raw_key_size": 18425, "raw_average_key_size": 23, "raw_value_size": 2825847, "raw_average_value_size": 3595, "num_data_blocks": 233, "num_entries": 786, "num_filter_entries": 786, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711241663, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c7c1ceb2-c7ae-4362-880a-60bb5eb2f13c", "db_session_id": "BD8JC5TEWD3CV2KYE3RC", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711241663176265, "job": 1, "event": "recovery_finished"} 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi088/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560e7e636000 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: DB pointer 0x560e7e622000 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T00:54:23.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** DB Stats ** 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** Compaction Stats [default] ** 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: L0 1/0 2.72 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 482.2 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: L6 1/0 7.84 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-24T00:54:23.452 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Sum 2/0 10.56 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 482.2 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 482.2 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** Compaction Stats [default] ** 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 482.2 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T00:54:23.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative compaction: 0.00 GB write, 111.74 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval compaction: 0.00 GB write, 111.74 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Block cache BinnedLRUCache@0x560e7e545090#2 capacity: 512.00 MB usage: 8.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.3e-05 secs_since: 0 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.03 KB,0.00038743%) IndexBlock(1,6.08 KB,0.00115931%) Misc(1,0.00 KB,0%) 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: starting mon.smithi088 rank 1 at public addrs [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] at bind addrs [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi088 fsid 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: mon.smithi088@-1(???) e2 preinit fsid 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: mon.smithi088@-1(???).mds e20 new map 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: mon.smithi088@-1(???).mds e20 print_map 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: e20 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:54:23.454 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: legacy client fscid: 1 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Filesystem 'cephfs' (1) 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: fs_name cephfs 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: epoch 20 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: tableserver 0 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: root 0 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: session_timeout 60 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: session_autoclose 300 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: max_file_size 1099511627776 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: max_xattr_size 65536 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: required_client_features {} 2024-03-24T00:54:23.455 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: last_failure 0 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: last_failure_osd_epoch 47 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: max_mds 2 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: in 0,1 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: up {0=14462,1=24271} 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: failed 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: damaged 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: stopped 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: data_pools [3] 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: metadata_pool 2 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: inline_data disabled 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: balancer 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: bal_rank_mask -1 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: standby_count_wanted 1 2024-03-24T00:54:23.456 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: qdb_cluster leader: 0 members: 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: [mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: [mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: [mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: [mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: mon.smithi088@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: mon.smithi088@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: mon.smithi088@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: mon.smithi088@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 ceph-mon[87351]: mon.smithi088@-1(???).paxosservice(auth 1..25) refresh upgraded, format 0 -> 3 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 bash[87332]: 3bace4fab2b0fb908fbcbd88a66e3edb7110cf4c2bc1a4d9e19c2056491af2e0 2024-03-24T00:54:23.457 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:23 smithi088 systemd[1]: Started Ceph mon.smithi088 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:54:25.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: mon.smithi038 calling monitor election 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: mon.smithi088 calling monitor election 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: mon.smithi038 is new leader, mons smithi038,smithi088 in quorum (ranks 0,1) 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: monmap e3: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:active} 2 up:standby-replay 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: osdmap e50: 6 total, 6 up, 6 in 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: mgrmap e38: smithi038.huhioz(active, since 27s), standbys: smithi088.njctea 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: overall HEALTH_OK 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:25.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: mon.smithi038 calling monitor election 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: mon.smithi088 calling monitor election 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: mon.smithi038 is new leader, mons smithi038,smithi088 in quorum (ranks 0,1) 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: monmap e3: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:active} 2 up:standby-replay 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: osdmap e50: 6 total, 6 up, 6 in 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: mgrmap e38: smithi038.huhioz(active, since 27s), standbys: smithi088.njctea 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: overall HEALTH_OK 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:25.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:54:26.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:26 smithi088 ceph-mon[87351]: pgmap v17: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:26.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:26 smithi038 ceph-mon[112108]: pgmap v17: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:27.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:54:27.666 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:54:28.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:28 smithi038 ceph-mon[112108]: pgmap v18: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:28.622 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:28 smithi088 ceph-mon[87351]: pgmap v18: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:30.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:29 smithi038 ceph-mon[112108]: pgmap v19: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:30.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:29 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:30.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:29 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:29 smithi088 ceph-mon[87351]: pgmap v19: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:29 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:29 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:32.077 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:31 smithi088 ceph-mon[87351]: pgmap v20: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:32.077 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:31 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:32.077 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:31 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:31 smithi038 ceph-mon[112108]: pgmap v20: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:31 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:31 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:33.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:33 smithi088 ceph-mon[87351]: pgmap v21: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:34.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:33 smithi038 ceph-mon[112108]: pgmap v21: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:36.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:35 smithi038 ceph-mon[112108]: pgmap v22: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:36.182 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:35 smithi088 ceph-mon[87351]: pgmap v22: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:37 smithi088 ceph-mon[87351]: pgmap v23: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:54:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:37 smithi088 ceph-mon[87351]: Reconfiguring mon.smithi038 (monmap changed)... 2024-03-24T00:54:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:54:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:54:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:37.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:37 smithi088 ceph-mon[87351]: Reconfiguring daemon mon.smithi038 on smithi038 2024-03-24T00:54:37.947 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:37 smithi038 ceph-mon[112108]: pgmap v23: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:37.947 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:37.947 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:37.947 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:37.947 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:54:37.947 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:37.947 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:37 smithi038 ceph-mon[112108]: Reconfiguring mon.smithi038 (monmap changed)... 2024-03-24T00:54:37.947 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:54:37.947 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:54:37.948 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:37.948 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:37 smithi038 ceph-mon[112108]: Reconfiguring daemon mon.smithi038 on smithi038 2024-03-24T00:54:39.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:39 smithi088 ceph-mon[87351]: pgmap v24: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:39.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:39 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:39.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:39 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:39 smithi038 ceph-mon[112108]: pgmap v24: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:39 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:39 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:40 smithi038 ceph-mon[112108]: Reconfiguring mgr.smithi038.huhioz (monmap changed)... 2024-03-24T00:54:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:40 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.huhioz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:54:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:40 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:54:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:40 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:40 smithi038 ceph-mon[112108]: Reconfiguring daemon mgr.smithi038.huhioz on smithi038 2024-03-24T00:54:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:40 smithi088 ceph-mon[87351]: Reconfiguring mgr.smithi038.huhioz (monmap changed)... 2024-03-24T00:54:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:40 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.huhioz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:54:40.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:40 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:54:40.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:40 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:40.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:40 smithi088 ceph-mon[87351]: Reconfiguring daemon mgr.smithi038.huhioz on smithi038 2024-03-24T00:54:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:41 smithi038 ceph-mon[112108]: pgmap v25: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:41.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:41 smithi088 ceph-mon[87351]: pgmap v25: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:54:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:42 smithi088 ceph-mon[87351]: Reconfiguring ceph-exporter.smithi038 (monmap changed)... 2024-03-24T00:54:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:54:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:54:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-24T00:54:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:42 smithi088 ceph-mon[87351]: Unable to update caps for client.ceph-exporter.smithi038 2024-03-24T00:54:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi038"}]: dispatch 2024-03-24T00:54:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:42.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:42 smithi088 ceph-mon[87351]: Reconfiguring daemon ceph-exporter.smithi038 on smithi038 2024-03-24T00:54:43.108 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:54:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:42 smithi038 ceph-mon[112108]: Reconfiguring ceph-exporter.smithi038 (monmap changed)... 2024-03-24T00:54:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:54:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:54:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-24T00:54:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:42 smithi038 ceph-mon[112108]: Unable to update caps for client.ceph-exporter.smithi038 2024-03-24T00:54:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi038"}]: dispatch 2024-03-24T00:54:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:43.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:42 smithi038 ceph-mon[112108]: Reconfiguring daemon ceph-exporter.smithi038 on smithi038 2024-03-24T00:54:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.698+0000 7f8218e00700 1 -- 172.21.15.38:0/3398847743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8214100a70 msgr2=0x7f8214100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.698+0000 7f8218e00700 1 --2- 172.21.15.38:0/3398847743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8214100a70 0x7f8214100ef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009910 tx=0x7f81fc009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.699+0000 7f8218e00700 1 -- 172.21.15.38:0/3398847743 shutdown_connections 2024-03-24T00:54:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.699+0000 7f8218e00700 1 --2- 172.21.15.38:0/3398847743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8214100a70 0x7f8214100ef0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:43.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.699+0000 7f8218e00700 1 --2- 172.21.15.38:0/3398847743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8214106a70 0x7f8214106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:43.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.699+0000 7f8218e00700 1 -- 172.21.15.38:0/3398847743 >> 172.21.15.38:0/3398847743 conn(0x7f82140fc5a0 msgr2=0x7f82140fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:43.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.699+0000 7f8218e00700 1 -- 172.21.15.38:0/3398847743 shutdown_connections 2024-03-24T00:54:43.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.699+0000 7f8218e00700 1 -- 172.21.15.38:0/3398847743 wait complete. 2024-03-24T00:54:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.700+0000 7f8218e00700 1 Processor -- start 2024-03-24T00:54:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.700+0000 7f8218e00700 1 -- start start 2024-03-24T00:54:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.700+0000 7f8218e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8214100a70 0x7f82140730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.700+0000 7f8218e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8214106a70 0x7f82140735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.700+0000 7f8218e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214073b30 con 0x7f8214100a70 2024-03-24T00:54:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.700+0000 7f8218e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214073c70 con 0x7f8214106a70 2024-03-24T00:54:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.701+0000 7f8211e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8214100a70 0x7f82140730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.701+0000 7f8211e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8214100a70 0x7f82140730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55014/0 (socket says 172.21.15.38:55014) 2024-03-24T00:54:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.701+0000 7f8211e00700 1 -- 172.21.15.38:0/923774126 learned_addr learned my addr 172.21.15.38:0/923774126 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.701+0000 7f820be00700 1 --2- 172.21.15.38:0/923774126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8214106a70 0x7f82140735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.701+0000 7f8211e00700 1 -- 172.21.15.38:0/923774126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8214106a70 msgr2=0x7f82140735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.701+0000 7f8211e00700 1 --2- 172.21.15.38:0/923774126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8214106a70 0x7f82140735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.701+0000 7f8211e00700 1 -- 172.21.15.38:0/923774126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc0095f0 con 0x7f8214100a70 2024-03-24T00:54:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.702+0000 7f8211e00700 1 --2- 172.21.15.38:0/923774126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8214100a70 0x7f82140730b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f820400e920 tx=0x7f820400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.702+0000 7f820aa00700 1 -- 172.21.15.38:0/923774126 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8204009790 con 0x7f8214100a70 2024-03-24T00:54:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.702+0000 7f8218e00700 1 -- 172.21.15.38:0/923774126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82141a2b90 con 0x7f8214100a70 2024-03-24T00:54:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.702+0000 7f820aa00700 1 -- 172.21.15.38:0/923774126 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f820400cba0 con 0x7f8214100a70 2024-03-24T00:54:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.702+0000 7f820aa00700 1 -- 172.21.15.38:0/923774126 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8204018650 con 0x7f8214100a70 2024-03-24T00:54:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.702+0000 7f8218e00700 1 -- 172.21.15.38:0/923774126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141a2ef0 con 0x7f8214100a70 2024-03-24T00:54:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.703+0000 7f8218e00700 1 -- 172.21.15.38:0/923774126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8214109090 con 0x7f8214100a70 2024-03-24T00:54:43.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.704+0000 7f820aa00700 1 -- 172.21.15.38:0/923774126 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f820400cd10 con 0x7f8214100a70 2024-03-24T00:54:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.705+0000 7f820aa00700 1 --2- 172.21.15.38:0/923774126 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f81f806e0d0 0x7f81f8070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.705+0000 7f820aa00700 1 -- 172.21.15.38:0/923774126 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8204014070 con 0x7f8214100a70 2024-03-24T00:54:43.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.706+0000 7f820be00700 1 --2- 172.21.15.38:0/923774126 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f81f806e0d0 0x7f81f8070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.707+0000 7f820be00700 1 --2- 172.21.15.38:0/923774126 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f81f806e0d0 0x7f81f8070590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00b760 tx=0x7f81fc009f90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.711+0000 7f820aa00700 1 -- 172.21.15.38:0/923774126 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8204053740 con 0x7f8214100a70 2024-03-24T00:54:43.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.964+0000 7f8218e00700 1 -- 172.21.15.38:0/923774126 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8214073e30 con 0x7f81f806e0d0 2024-03-24T00:54:43.965 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:43 smithi038 ceph-mon[112108]: pgmap v26: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:43.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.967+0000 7f820aa00700 1 -- 172.21.15.38:0/923774126 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f8214073e30 con 0x7f81f806e0d0 2024-03-24T00:54:43.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.970+0000 7f8218e00700 1 -- 172.21.15.38:0/923774126 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f81f806e0d0 msgr2=0x7f81f8070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:43.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.970+0000 7f8218e00700 1 --2- 172.21.15.38:0/923774126 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f81f806e0d0 0x7f81f8070590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00b760 tx=0x7f81fc009f90 comp rx=0 tx=0).stop 2024-03-24T00:54:43.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.970+0000 7f8218e00700 1 -- 172.21.15.38:0/923774126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8214100a70 msgr2=0x7f82140730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:43.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.970+0000 7f8218e00700 1 --2- 172.21.15.38:0/923774126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8214100a70 0x7f82140730b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f820400e920 tx=0x7f820400ece0 comp rx=0 tx=0).stop 2024-03-24T00:54:43.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.970+0000 7f8218e00700 1 -- 172.21.15.38:0/923774126 shutdown_connections 2024-03-24T00:54:43.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.970+0000 7f8218e00700 1 --2- 172.21.15.38:0/923774126 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f81f806e0d0 0x7f81f8070590 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:43.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.970+0000 7f8218e00700 1 --2- 172.21.15.38:0/923774126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8214100a70 0x7f82140730b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:43.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.970+0000 7f8218e00700 1 --2- 172.21.15.38:0/923774126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8214106a70 0x7f82140735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:43.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.970+0000 7f8218e00700 1 -- 172.21.15.38:0/923774126 >> 172.21.15.38:0/923774126 conn(0x7f82140fc5a0 msgr2=0x7f82140fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:43.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.970+0000 7f8218e00700 1 -- 172.21.15.38:0/923774126 shutdown_connections 2024-03-24T00:54:43.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:43.970+0000 7f8218e00700 1 -- 172.21.15.38:0/923774126 wait complete. 2024-03-24T00:54:43.987 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:54:44.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.129+0000 7feafd200700 1 -- 172.21.15.38:0/1372345512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf8107b90 msgr2=0x7feaf810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:44.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.129+0000 7feafd200700 1 --2- 172.21.15.38:0/1372345512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf8107b90 0x7feaf810bb30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feae4009910 tx=0x7feae4009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:44.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.129+0000 7feafd200700 1 -- 172.21.15.38:0/1372345512 shutdown_connections 2024-03-24T00:54:44.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.129+0000 7feafd200700 1 --2- 172.21.15.38:0/1372345512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf8107b90 0x7feaf810bb30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.129+0000 7feafd200700 1 --2- 172.21.15.38:0/1372345512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf81071e0 0x7feaf81075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.129+0000 7feafd200700 1 -- 172.21.15.38:0/1372345512 >> 172.21.15.38:0/1372345512 conn(0x7feaf8076700 msgr2=0x7feaf8078b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:44.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.130+0000 7feafd200700 1 -- 172.21.15.38:0/1372345512 shutdown_connections 2024-03-24T00:54:44.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.130+0000 7feafd200700 1 -- 172.21.15.38:0/1372345512 wait complete. 2024-03-24T00:54:44.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.131+0000 7feafd200700 1 Processor -- start 2024-03-24T00:54:44.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.131+0000 7feafd200700 1 -- start start 2024-03-24T00:54:44.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.131+0000 7feafd200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf81071e0 0x7feaf819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:44.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.131+0000 7feafd200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf8107b90 0x7feaf819d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:44.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.131+0000 7feafd200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf819da20 con 0x7feaf8107b90 2024-03-24T00:54:44.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.131+0000 7feafd200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf81a16f0 con 0x7feaf81071e0 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.132+0000 7feaf6a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf8107b90 0x7feaf819d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.132+0000 7feaf6a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf8107b90 0x7feaf819d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55036/0 (socket says 172.21.15.38:55036) 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.132+0000 7feaf6a00700 1 -- 172.21.15.38:0/2721050426 learned_addr learned my addr 172.21.15.38:0/2721050426 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.132+0000 7feaf7400700 1 --2- 172.21.15.38:0/2721050426 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf81071e0 0x7feaf819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.132+0000 7feaf6a00700 1 -- 172.21.15.38:0/2721050426 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf81071e0 msgr2=0x7feaf819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.132+0000 7feaf6a00700 1 --2- 172.21.15.38:0/2721050426 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf81071e0 0x7feaf819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.132+0000 7feaf6a00700 1 -- 172.21.15.38:0/2721050426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae40095f0 con 0x7feaf8107b90 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.132+0000 7feaf6a00700 1 --2- 172.21.15.38:0/2721050426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf8107b90 0x7feaf819d340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feae4000c00 tx=0x7feae400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.133+0000 7feaefe00700 1 -- 172.21.15.38:0/2721050426 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae401d070 con 0x7feaf8107b90 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.133+0000 7feaefe00700 1 -- 172.21.15.38:0/2721050426 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feae4005700 con 0x7feaf8107b90 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.133+0000 7feafd200700 1 -- 172.21.15.38:0/2721050426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf81a1970 con 0x7feaf8107b90 2024-03-24T00:54:44.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.133+0000 7feaefe00700 1 -- 172.21.15.38:0/2721050426 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae400fe50 con 0x7feaf8107b90 2024-03-24T00:54:44.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.133+0000 7feafd200700 1 -- 172.21.15.38:0/2721050426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf81a1e60 con 0x7feaf8107b90 2024-03-24T00:54:44.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.134+0000 7feafd200700 1 -- 172.21.15.38:0/2721050426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf810f530 con 0x7feaf8107b90 2024-03-24T00:54:44.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.135+0000 7feaefe00700 1 -- 172.21.15.38:0/2721050426 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7feae400f640 con 0x7feaf8107b90 2024-03-24T00:54:44.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.136+0000 7feaefe00700 1 --2- 172.21.15.38:0/2721050426 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7feadc06e0d0 0x7feadc070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:44.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.136+0000 7feaefe00700 1 -- 172.21.15.38:0/2721050426 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7feae408f320 con 0x7feaf8107b90 2024-03-24T00:54:44.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.136+0000 7feaf7400700 1 --2- 172.21.15.38:0/2721050426 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7feadc06e0d0 0x7feadc070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:44.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.137+0000 7feaf7400700 1 --2- 172.21.15.38:0/2721050426 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7feadc06e0d0 0x7feadc070590 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feae8005950 tx=0x7feae80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:44.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.140+0000 7feaefe00700 1 -- 172.21.15.38:0/2721050426 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feae4058650 con 0x7feaf8107b90 2024-03-24T00:54:44.381 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:43 smithi088 ceph-mon[87351]: pgmap v26: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.410+0000 7feafd200700 1 -- 172.21.15.38:0/2721050426 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feaf81a2080 con 0x7feadc06e0d0 2024-03-24T00:54:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.411+0000 7feaefe00700 1 -- 172.21.15.38:0/2721050426 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7feaf81a2080 con 0x7feadc06e0d0 2024-03-24T00:54:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.414+0000 7feafd200700 1 -- 172.21.15.38:0/2721050426 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7feadc06e0d0 msgr2=0x7feadc070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.414+0000 7feafd200700 1 --2- 172.21.15.38:0/2721050426 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7feadc06e0d0 0x7feadc070590 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feae8005950 tx=0x7feae80058e0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.414+0000 7feafd200700 1 -- 172.21.15.38:0/2721050426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf8107b90 msgr2=0x7feaf819d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.414+0000 7feafd200700 1 --2- 172.21.15.38:0/2721050426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf8107b90 0x7feaf819d340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feae4000c00 tx=0x7feae400be20 comp rx=0 tx=0).stop 2024-03-24T00:54:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.414+0000 7feafd200700 1 -- 172.21.15.38:0/2721050426 shutdown_connections 2024-03-24T00:54:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.414+0000 7feafd200700 1 --2- 172.21.15.38:0/2721050426 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7feadc06e0d0 0x7feadc070590 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.414+0000 7feafd200700 1 --2- 172.21.15.38:0/2721050426 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf81071e0 0x7feaf819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.414+0000 7feafd200700 1 --2- 172.21.15.38:0/2721050426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf8107b90 0x7feaf819d340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.415+0000 7feafd200700 1 -- 172.21.15.38:0/2721050426 >> 172.21.15.38:0/2721050426 conn(0x7feaf8076700 msgr2=0x7feaf8077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.415+0000 7feafd200700 1 -- 172.21.15.38:0/2721050426 shutdown_connections 2024-03-24T00:54:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.415+0000 7feafd200700 1 -- 172.21.15.38:0/2721050426 wait complete. 2024-03-24T00:54:44.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.565+0000 7fe844a00700 1 -- 172.21.15.38:0/4285645262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe840100aa0 msgr2=0x7fe840100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:44.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.565+0000 7fe844a00700 1 --2- 172.21.15.38:0/4285645262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe840100aa0 0x7fe840100f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe828009910 tx=0x7fe828009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:44.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.565+0000 7fe844a00700 1 -- 172.21.15.38:0/4285645262 shutdown_connections 2024-03-24T00:54:44.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.565+0000 7fe844a00700 1 --2- 172.21.15.38:0/4285645262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe840100aa0 0x7fe840100f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.565+0000 7fe844a00700 1 --2- 172.21.15.38:0/4285645262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe840106aa0 0x7fe840106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.565+0000 7fe844a00700 1 -- 172.21.15.38:0/4285645262 >> 172.21.15.38:0/4285645262 conn(0x7fe8400fc5d0 msgr2=0x7fe8400fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:44.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.565+0000 7fe844a00700 1 -- 172.21.15.38:0/4285645262 shutdown_connections 2024-03-24T00:54:44.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.566+0000 7fe844a00700 1 -- 172.21.15.38:0/4285645262 wait complete. 2024-03-24T00:54:44.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.566+0000 7fe844a00700 1 Processor -- start 2024-03-24T00:54:44.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.567+0000 7fe844a00700 1 -- start start 2024-03-24T00:54:44.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.567+0000 7fe844a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe840100aa0 0x7fe8400730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:44.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.567+0000 7fe844a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe840106aa0 0x7fe8400735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:44.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.567+0000 7fe844a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe840073b30 con 0x7fe840100aa0 2024-03-24T00:54:44.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.567+0000 7fe844a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe840073c70 con 0x7fe840106aa0 2024-03-24T00:54:44.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.567+0000 7fe83de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe840100aa0 0x7fe8400730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:44.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.567+0000 7fe83de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe840100aa0 0x7fe8400730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55050/0 (socket says 172.21.15.38:55050) 2024-03-24T00:54:44.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.567+0000 7fe83de00700 1 -- 172.21.15.38:0/4088492232 learned_addr learned my addr 172.21.15.38:0/4088492232 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:44.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.568+0000 7fe837e00700 1 --2- 172.21.15.38:0/4088492232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe840106aa0 0x7fe8400735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:44.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.568+0000 7fe83de00700 1 -- 172.21.15.38:0/4088492232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe840106aa0 msgr2=0x7fe8400735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:44.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.568+0000 7fe83de00700 1 --2- 172.21.15.38:0/4088492232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe840106aa0 0x7fe8400735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.568+0000 7fe83de00700 1 -- 172.21.15.38:0/4088492232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8280095f0 con 0x7fe840100aa0 2024-03-24T00:54:44.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.568+0000 7fe83de00700 1 --2- 172.21.15.38:0/4088492232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe840100aa0 0x7fe8400730b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe83000d8d0 tx=0x7fe83000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:44.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.568+0000 7fe836a00700 1 -- 172.21.15.38:0/4088492232 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe830009940 con 0x7fe840100aa0 2024-03-24T00:54:44.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.569+0000 7fe836a00700 1 -- 172.21.15.38:0/4088492232 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe830010460 con 0x7fe840100aa0 2024-03-24T00:54:44.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.569+0000 7fe844a00700 1 -- 172.21.15.38:0/4088492232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8401a2bc0 con 0x7fe840100aa0 2024-03-24T00:54:44.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.569+0000 7fe836a00700 1 -- 172.21.15.38:0/4088492232 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83000f5f0 con 0x7fe840100aa0 2024-03-24T00:54:44.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.569+0000 7fe844a00700 1 -- 172.21.15.38:0/4088492232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8401a2f20 con 0x7fe840100aa0 2024-03-24T00:54:44.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.570+0000 7fe844a00700 1 -- 172.21.15.38:0/4088492232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8401090c0 con 0x7fe840100aa0 2024-03-24T00:54:44.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.571+0000 7fe836a00700 1 -- 172.21.15.38:0/4088492232 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7fe8300105d0 con 0x7fe840100aa0 2024-03-24T00:54:44.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.572+0000 7fe836a00700 1 --2- 172.21.15.38:0/4088492232 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe82406e0d0 0x7fe824070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:44.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.572+0000 7fe836a00700 1 -- 172.21.15.38:0/4088492232 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe83008df20 con 0x7fe840100aa0 2024-03-24T00:54:44.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.572+0000 7fe837e00700 1 --2- 172.21.15.38:0/4088492232 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe82406e0d0 0x7fe824070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.573+0000 7fe837e00700 1 --2- 172.21.15.38:0/4088492232 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe82406e0d0 0x7fe824070590 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe828005b40 tx=0x7fe82801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:44.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.577+0000 7fe836a00700 1 -- 172.21.15.38:0/4088492232 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe8300571b0 con 0x7fe840100aa0 2024-03-24T00:54:44.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.807+0000 7fe844a00700 1 -- 172.21.15.38:0/4088492232 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe840073e30 con 0x7fe82406e0d0 2024-03-24T00:54:44.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.837+0000 7fe836a00700 1 -- 172.21.15.38:0/4088492232 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe840073e30 con 0x7fe82406e0d0 2024-03-24T00:54:44.838 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:54:44.838 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (3m) 40s ago 17m 14.4M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:54:44.838 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (17m) 40s ago 17m 8426k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:54:44.838 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (15m) 16s ago 15m 8270k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:54:44.838 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (17m) 40s ago 17m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:54:44.838 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (15m) 16s ago 15m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:54:44.838 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (2m) 40s ago 16m 75.2M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:54:44.838 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (9m) 40s ago 9m 97.5M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (9m) 40s ago 9m 151M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (9m) 16s ago 9m 24.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (9m) 16s ago 9m 24.8M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (5m) 40s ago 18m 478M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (4m) 16s ago 15m 448M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (58s) 40s ago 19m 29.0M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (22s) 16s ago 15m 20.4M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (4m) 40s ago 17m 7684k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (4m) 16s ago 15m 7868k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (14m) 40s ago 14m 258M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (14m) 40s ago 13m 272M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (13m) 40s ago 13m 228M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (12m) 16s ago 12m 303M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (12m) 16s ago 12m 283M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (11m) 16s ago 11m 252M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:54:44.839 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (3m) 40s ago 16m 66.3M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:54:44.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.840+0000 7fe844a00700 1 -- 172.21.15.38:0/4088492232 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe82406e0d0 msgr2=0x7fe824070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:44.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.840+0000 7fe844a00700 1 --2- 172.21.15.38:0/4088492232 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe82406e0d0 0x7fe824070590 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe828005b40 tx=0x7fe82801a040 comp rx=0 tx=0).stop 2024-03-24T00:54:44.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.840+0000 7fe844a00700 1 -- 172.21.15.38:0/4088492232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe840100aa0 msgr2=0x7fe8400730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:44.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.840+0000 7fe844a00700 1 --2- 172.21.15.38:0/4088492232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe840100aa0 0x7fe8400730b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe83000d8d0 tx=0x7fe83000dc90 comp rx=0 tx=0).stop 2024-03-24T00:54:44.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.841+0000 7fe844a00700 1 -- 172.21.15.38:0/4088492232 shutdown_connections 2024-03-24T00:54:44.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.841+0000 7fe844a00700 1 --2- 172.21.15.38:0/4088492232 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe82406e0d0 0x7fe824070590 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.841+0000 7fe844a00700 1 --2- 172.21.15.38:0/4088492232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe840100aa0 0x7fe8400730b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.841+0000 7fe844a00700 1 --2- 172.21.15.38:0/4088492232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe840106aa0 0x7fe8400735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.841+0000 7fe844a00700 1 -- 172.21.15.38:0/4088492232 >> 172.21.15.38:0/4088492232 conn(0x7fe8400fc5d0 msgr2=0x7fe8400fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:44.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.841+0000 7fe844a00700 1 -- 172.21.15.38:0/4088492232 shutdown_connections 2024-03-24T00:54:44.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.841+0000 7fe844a00700 1 -- 172.21.15.38:0/4088492232 wait complete. 2024-03-24T00:54:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.992+0000 7f54a6e00700 1 -- 172.21.15.38:0/620506437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a0100a70 msgr2=0x7f54a0100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.992+0000 7f54a6e00700 1 --2- 172.21.15.38:0/620506437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a0100a70 0x7f54a0100ef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f548c009910 tx=0x7f548c009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.992+0000 7f54a6e00700 1 -- 172.21.15.38:0/620506437 shutdown_connections 2024-03-24T00:54:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.992+0000 7f54a6e00700 1 --2- 172.21.15.38:0/620506437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a0100a70 0x7f54a0100ef0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.992+0000 7f54a6e00700 1 --2- 172.21.15.38:0/620506437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0106a70 0x7f54a0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.992+0000 7f54a6e00700 1 -- 172.21.15.38:0/620506437 >> 172.21.15.38:0/620506437 conn(0x7f54a00fc580 msgr2=0x7f54a00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.992+0000 7f54a6e00700 1 -- 172.21.15.38:0/620506437 shutdown_connections 2024-03-24T00:54:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.993+0000 7f54a6e00700 1 -- 172.21.15.38:0/620506437 wait complete. 2024-03-24T00:54:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.993+0000 7f54a6e00700 1 Processor -- start 2024-03-24T00:54:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.994+0000 7f54a6e00700 1 -- start start 2024-03-24T00:54:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.994+0000 7f54a6e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0100a70 0x7f54a019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.994+0000 7f54a6e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a0106a70 0x7f54a019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.994+0000 7f54a6e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a019d370 con 0x7f54a0106a70 2024-03-24T00:54:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.994+0000 7f54a6e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a0196950 con 0x7f54a0100a70 2024-03-24T00:54:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.994+0000 7f549f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a0106a70 0x7f54a019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.994+0000 7f549f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a0106a70 0x7f54a019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55060/0 (socket says 172.21.15.38:55060) 2024-03-24T00:54:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.994+0000 7f549f400700 1 -- 172.21.15.38:0/1497706347 learned_addr learned my addr 172.21.15.38:0/1497706347 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.995+0000 7f549fe00700 1 --2- 172.21.15.38:0/1497706347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0100a70 0x7f54a019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.995+0000 7f549f400700 1 -- 172.21.15.38:0/1497706347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0100a70 msgr2=0x7f54a019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.995+0000 7f549f400700 1 --2- 172.21.15.38:0/1497706347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0100a70 0x7f54a019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.995+0000 7f549f400700 1 -- 172.21.15.38:0/1497706347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c0095f0 con 0x7f54a0106a70 2024-03-24T00:54:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.995+0000 7f549f400700 1 --2- 172.21.15.38:0/1497706347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a0106a70 0x7f54a019cd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f548c00bd70 tx=0x7f548c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:44.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.996+0000 7f549cc00700 1 -- 172.21.15.38:0/1497706347 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548c01d070 con 0x7f54a0106a70 2024-03-24T00:54:44.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.996+0000 7f549cc00700 1 -- 172.21.15.38:0/1497706347 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f548c00f460 con 0x7f54a0106a70 2024-03-24T00:54:44.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.996+0000 7f549cc00700 1 -- 172.21.15.38:0/1497706347 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548c017610 con 0x7f54a0106a70 2024-03-24T00:54:44.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.996+0000 7f54a6e00700 1 -- 172.21.15.38:0/1497706347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a0196bd0 con 0x7f54a0106a70 2024-03-24T00:54:44.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.996+0000 7f54a6e00700 1 -- 172.21.15.38:0/1497706347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a01970c0 con 0x7f54a0106a70 2024-03-24T00:54:44.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.997+0000 7f54a6e00700 1 -- 172.21.15.38:0/1497706347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a0109090 con 0x7f54a0106a70 2024-03-24T00:54:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.998+0000 7f549cc00700 1 -- 172.21.15.38:0/1497706347 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f548c00f5d0 con 0x7f54a0106a70 2024-03-24T00:54:44.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.999+0000 7f549cc00700 1 --2- 172.21.15.38:0/1497706347 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f548406e120 0x7f54840705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:44.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:44.999+0000 7f549cc00700 1 -- 172.21.15.38:0/1497706347 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f548c08f450 con 0x7f54a0106a70 2024-03-24T00:54:45.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.000+0000 7f549fe00700 1 --2- 172.21.15.38:0/1497706347 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f548406e120 0x7f54840705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:45.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.000+0000 7f549fe00700 1 --2- 172.21.15.38:0/1497706347 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f548406e120 0x7f54840705e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5490006fd0 tx=0x7f5490008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.005+0000 7f549cc00700 1 -- 172.21.15.38:0/1497706347 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f548c05c2d0 con 0x7f54a0106a70 2024-03-24T00:54:45.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.342+0000 7f54a6e00700 1 -- 172.21.15.38:0/1497706347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f54a01021b0 con 0x7f54a0106a70 2024-03-24T00:54:45.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.343+0000 7f549cc00700 1 -- 172.21.15.38:0/1497706347 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+730 (secure 0 0 0) 0x7f548c05be60 con 0x7f54a0106a70 2024-03-24T00:54:45.343 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:54:45.344 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:54:45.345 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 10, 2024-03-24T00:54:45.345 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-24T00:54:45.345 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:54:45.345 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:54:45.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.346+0000 7f54a6e00700 1 -- 172.21.15.38:0/1497706347 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f548406e120 msgr2=0x7f54840705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:45.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.346+0000 7f54a6e00700 1 --2- 172.21.15.38:0/1497706347 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f548406e120 0x7f54840705e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5490006fd0 tx=0x7f5490008040 comp rx=0 tx=0).stop 2024-03-24T00:54:45.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.346+0000 7f54a6e00700 1 -- 172.21.15.38:0/1497706347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a0106a70 msgr2=0x7f54a019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:45.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.346+0000 7f54a6e00700 1 --2- 172.21.15.38:0/1497706347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a0106a70 0x7f54a019cd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f548c00bd70 tx=0x7f548c00be50 comp rx=0 tx=0).stop 2024-03-24T00:54:45.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.346+0000 7f54a6e00700 1 -- 172.21.15.38:0/1497706347 shutdown_connections 2024-03-24T00:54:45.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.346+0000 7f54a6e00700 1 --2- 172.21.15.38:0/1497706347 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f548406e120 0x7f54840705e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.346+0000 7f54a6e00700 1 --2- 172.21.15.38:0/1497706347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0100a70 0x7f54a019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.346+0000 7f54a6e00700 1 --2- 172.21.15.38:0/1497706347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54a0106a70 0x7f54a019cd50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.346+0000 7f54a6e00700 1 -- 172.21.15.38:0/1497706347 >> 172.21.15.38:0/1497706347 conn(0x7f54a00fc580 msgr2=0x7f54a0105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:45.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.346+0000 7f54a6e00700 1 -- 172.21.15.38:0/1497706347 shutdown_connections 2024-03-24T00:54:45.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.346+0000 7f54a6e00700 1 -- 172.21.15.38:0/1497706347 wait complete. 2024-03-24T00:54:45.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:45 smithi038 ceph-mon[112108]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:45.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:45 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:45.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:45 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:45.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:45 smithi038 ceph-mon[112108]: Reconfiguring crash.smithi038 (monmap changed)... 2024-03-24T00:54:45.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:45 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:54:45.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:45 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:45.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:45 smithi038 ceph-mon[112108]: Reconfiguring daemon crash.smithi038 on smithi038 2024-03-24T00:54:45.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.498+0000 7fdbb6c00700 1 -- 172.21.15.38:0/1430096530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb0102530 msgr2=0x7fdbb010aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:45.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.498+0000 7fdbb6c00700 1 --2- 172.21.15.38:0/1430096530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb0102530 0x7fdbb010aa20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009910 tx=0x7fdb9c009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:45.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.498+0000 7fdbb6c00700 1 -- 172.21.15.38:0/1430096530 shutdown_connections 2024-03-24T00:54:45.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.498+0000 7fdbb6c00700 1 --2- 172.21.15.38:0/1430096530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb0102530 0x7fdbb010aa20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.498+0000 7fdbb6c00700 1 --2- 172.21.15.38:0/1430096530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb0101c10 0x7fdbb0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.498+0000 7fdbb6c00700 1 -- 172.21.15.38:0/1430096530 >> 172.21.15.38:0/1430096530 conn(0x7fdbb0076640 msgr2=0x7fdbb00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:45.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.498+0000 7fdbb6c00700 1 -- 172.21.15.38:0/1430096530 shutdown_connections 2024-03-24T00:54:45.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.498+0000 7fdbb6c00700 1 -- 172.21.15.38:0/1430096530 wait complete. 2024-03-24T00:54:45.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.499+0000 7fdbb6c00700 1 Processor -- start 2024-03-24T00:54:45.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.499+0000 7fdbb6c00700 1 -- start start 2024-03-24T00:54:45.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.500+0000 7fdbb6c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb0101c10 0x7fdbb019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:45.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.500+0000 7fdbb6c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb0102530 0x7fdbb019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:45.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.500+0000 7fdbb6c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb019f5a0 con 0x7fdbb0101c10 2024-03-24T00:54:45.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.500+0000 7fdbb6c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb0198b80 con 0x7fdbb0102530 2024-03-24T00:54:45.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.500+0000 7fdbafe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb0101c10 0x7fdbb019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:45.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.500+0000 7fdbafe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb0101c10 0x7fdbb019ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55074/0 (socket says 172.21.15.38:55074) 2024-03-24T00:54:45.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.500+0000 7fdbafe00700 1 -- 172.21.15.38:0/3747130740 learned_addr learned my addr 172.21.15.38:0/3747130740 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:45.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.500+0000 7fdbaf400700 1 --2- 172.21.15.38:0/3747130740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb0102530 0x7fdbb019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.500+0000 7fdbafe00700 1 -- 172.21.15.38:0/3747130740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb0102530 msgr2=0x7fdbb019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.500+0000 7fdbafe00700 1 --2- 172.21.15.38:0/3747130740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb0102530 0x7fdbb019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.500+0000 7fdbafe00700 1 -- 172.21.15.38:0/3747130740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c0095f0 con 0x7fdbb0101c10 2024-03-24T00:54:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.501+0000 7fdbafe00700 1 --2- 172.21.15.38:0/3747130740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb0101c10 0x7fdbb019ea40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdba000b4b0 tx=0x7fdba000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.501+0000 7fdbacc00700 1 -- 172.21.15.38:0/3747130740 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba0011800 con 0x7fdbb0101c10 2024-03-24T00:54:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.501+0000 7fdbacc00700 1 -- 172.21.15.38:0/3747130740 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdba0011e20 con 0x7fdbb0101c10 2024-03-24T00:54:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.501+0000 7fdbacc00700 1 -- 172.21.15.38:0/3747130740 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba000f2e0 con 0x7fdbb0101c10 2024-03-24T00:54:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.501+0000 7fdbb6c00700 1 -- 172.21.15.38:0/3747130740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb0198e60 con 0x7fdbb0101c10 2024-03-24T00:54:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.501+0000 7fdbb6c00700 1 -- 172.21.15.38:0/3747130740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb01993b0 con 0x7fdbb0101c10 2024-03-24T00:54:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.502+0000 7fdbb6c00700 1 -- 172.21.15.38:0/3747130740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb010a580 con 0x7fdbb0101c10 2024-03-24T00:54:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.503+0000 7fdbacc00700 1 -- 172.21.15.38:0/3747130740 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7fdba0010380 con 0x7fdbb0101c10 2024-03-24T00:54:45.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.505+0000 7fdbacc00700 1 --2- 172.21.15.38:0/3747130740 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb9406e0d0 0x7fdb94070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:45.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.505+0000 7fdbacc00700 1 -- 172.21.15.38:0/3747130740 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fdba008da70 con 0x7fdbb0101c10 2024-03-24T00:54:45.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.505+0000 7fdbaf400700 1 --2- 172.21.15.38:0/3747130740 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb9406e0d0 0x7fdb94070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:45.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.506+0000 7fdbaf400700 1 --2- 172.21.15.38:0/3747130740 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb9406e0d0 0x7fdb94070590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c006010 tx=0x7fdb9c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.509+0000 7fdbacc00700 1 -- 172.21.15.38:0/3747130740 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdba005a960 con 0x7fdbb0101c10 2024-03-24T00:54:45.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:45 smithi088 ceph-mon[87351]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:45.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:45 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:45.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:45 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:45.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:45 smithi088 ceph-mon[87351]: Reconfiguring crash.smithi038 (monmap changed)... 2024-03-24T00:54:45.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:45 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:54:45.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:45 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:45.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:45 smithi088 ceph-mon[87351]: Reconfiguring daemon crash.smithi038 on smithi038 2024-03-24T00:54:45.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.775+0000 7fdbb6c00700 1 -- 172.21.15.38:0/3747130740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdbb0199fd0 con 0x7fdbb0101c10 2024-03-24T00:54:45.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.776+0000 7fdbacc00700 1 -- 172.21.15.38:0/3747130740 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1900 (secure 0 0 0) 0x7fdba005a4f0 con 0x7fdbb0101c10 2024-03-24T00:54:45.777 INFO:teuthology.orchestra.run.smithi038.stdout:e20 2024-03-24T00:54:45.777 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:54:45.777 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:54:45.777 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 20 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:54:45.778 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:54:45.779 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:54:45.780 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T00:54:45.780 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:45.780 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:45.780 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:45.780 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:54:45.780 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:54:45.780 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:54:45.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.780+0000 7fdbb6c00700 1 -- 172.21.15.38:0/3747130740 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb9406e0d0 msgr2=0x7fdb94070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:45.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.780+0000 7fdbb6c00700 1 --2- 172.21.15.38:0/3747130740 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb9406e0d0 0x7fdb94070590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c006010 tx=0x7fdb9c00b540 comp rx=0 tx=0).stop 2024-03-24T00:54:45.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.780+0000 7fdbb6c00700 1 -- 172.21.15.38:0/3747130740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb0101c10 msgr2=0x7fdbb019ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:45.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.780+0000 7fdbb6c00700 1 --2- 172.21.15.38:0/3747130740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb0101c10 0x7fdbb019ea40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdba000b4b0 tx=0x7fdba000b870 comp rx=0 tx=0).stop 2024-03-24T00:54:45.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.780+0000 7fdbb6c00700 1 -- 172.21.15.38:0/3747130740 shutdown_connections 2024-03-24T00:54:45.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.780+0000 7fdbb6c00700 1 --2- 172.21.15.38:0/3747130740 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb9406e0d0 0x7fdb94070590 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.780+0000 7fdbb6c00700 1 --2- 172.21.15.38:0/3747130740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb0101c10 0x7fdbb019ea40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.780+0000 7fdbb6c00700 1 --2- 172.21.15.38:0/3747130740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb0102530 0x7fdbb019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.780+0000 7fdbb6c00700 1 -- 172.21.15.38:0/3747130740 >> 172.21.15.38:0/3747130740 conn(0x7fdbb0076640 msgr2=0x7fdbb0100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:45.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.780+0000 7fdbb6c00700 1 -- 172.21.15.38:0/3747130740 shutdown_connections 2024-03-24T00:54:45.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.781+0000 7fdbb6c00700 1 -- 172.21.15.38:0/3747130740 wait complete. 2024-03-24T00:54:45.782 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:54:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.938+0000 7f84b5200700 1 -- 172.21.15.38:0/2267809635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b0102ef0 msgr2=0x7f84b01032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.938+0000 7f84b5200700 1 --2- 172.21.15.38:0/2267809635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b0102ef0 0x7f84b01032d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009910 tx=0x7f84a0009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.939+0000 7f84b5200700 1 -- 172.21.15.38:0/2267809635 shutdown_connections 2024-03-24T00:54:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.939+0000 7f84b5200700 1 --2- 172.21.15.38:0/2267809635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84b01038a0 0x7f84b01078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.939+0000 7f84b5200700 1 --2- 172.21.15.38:0/2267809635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b0102ef0 0x7f84b01032d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.939+0000 7f84b5200700 1 -- 172.21.15.38:0/2267809635 >> 172.21.15.38:0/2267809635 conn(0x7f84b00fe780 msgr2=0x7f84b0100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.939+0000 7f84b5200700 1 -- 172.21.15.38:0/2267809635 shutdown_connections 2024-03-24T00:54:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.939+0000 7f84b5200700 1 -- 172.21.15.38:0/2267809635 wait complete. 2024-03-24T00:54:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.940+0000 7f84b5200700 1 Processor -- start 2024-03-24T00:54:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.940+0000 7f84b5200700 1 -- start start 2024-03-24T00:54:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84b5200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b0102ef0 0x7f84b019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84b5200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84b01038a0 0x7f84b019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84b5200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b019f570 con 0x7f84b0102ef0 2024-03-24T00:54:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84b5200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0198ae0 con 0x7f84b01038a0 2024-03-24T00:54:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84ade00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b0102ef0 0x7f84b019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84ade00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b0102ef0 0x7f84b019e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55092/0 (socket says 172.21.15.38:55092) 2024-03-24T00:54:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84ade00700 1 -- 172.21.15.38:0/2188385483 learned_addr learned my addr 172.21.15.38:0/2188385483 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84ad400700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84b01038a0 0x7f84b019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84ade00700 1 -- 172.21.15.38:0/2188385483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84b01038a0 msgr2=0x7f84b019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84ade00700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84b01038a0 0x7f84b019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84ade00700 1 -- 172.21.15.38:0/2188385483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a00095f0 con 0x7f84b0102ef0 2024-03-24T00:54:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.941+0000 7f84ad400700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84b01038a0 0x7f84b019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:54:45.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.942+0000 7f84ade00700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b0102ef0 0x7f84b019e9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84a0006010 tx=0x7f84a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:45.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.942+0000 7f84a6a00700 1 -- 172.21.15.38:0/2188385483 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a001d070 con 0x7f84b0102ef0 2024-03-24T00:54:45.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.942+0000 7f84a6a00700 1 -- 172.21.15.38:0/2188385483 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84a000f460 con 0x7f84b0102ef0 2024-03-24T00:54:45.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.942+0000 7f84b5200700 1 -- 172.21.15.38:0/2188385483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b0198d60 con 0x7f84b0102ef0 2024-03-24T00:54:45.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.942+0000 7f84a6a00700 1 -- 172.21.15.38:0/2188385483 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a0017630 con 0x7f84b0102ef0 2024-03-24T00:54:45.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.942+0000 7f84b5200700 1 -- 172.21.15.38:0/2188385483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b0199250 con 0x7f84b0102ef0 2024-03-24T00:54:45.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.943+0000 7f84b5200700 1 -- 172.21.15.38:0/2188385483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b010b290 con 0x7f84b0102ef0 2024-03-24T00:54:45.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.944+0000 7f84a6a00700 1 -- 172.21.15.38:0/2188385483 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f84a000f5d0 con 0x7f84b0102ef0 2024-03-24T00:54:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.945+0000 7f84a6a00700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f849406e0e0 0x7f84940705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:45.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.946+0000 7f84a6a00700 1 -- 172.21.15.38:0/2188385483 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f84a008f440 con 0x7f84b0102ef0 2024-03-24T00:54:45.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.946+0000 7f84ad400700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f849406e0e0 0x7f84940705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:45.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.947+0000 7f84ad400700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f849406e0e0 0x7f84940705a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f849c005fd0 tx=0x7f849c005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:45.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:45.951+0000 7f84a6a00700 1 -- 172.21.15.38:0/2188385483 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f84a005c2e0 con 0x7f84b0102ef0 2024-03-24T00:54:46.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.207+0000 7f84b5200700 1 -- 172.21.15.38:0/2188385483 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84b019a010 con 0x7f849406e0e0 2024-03-24T00:54:46.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.209+0000 7f84a6a00700 1 -- 172.21.15.38:0/2188385483 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f84b019a010 con 0x7f849406e0e0 2024-03-24T00:54:46.210 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:54:46.210 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:54:46.210 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:54:46.210 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:54:46.210 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:54:46.210 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T00:54:46.210 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-03-24T00:54:46.210 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:54:46.211 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "4/23 daemons upgraded", 2024-03-24T00:54:46.211 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mon daemons", 2024-03-24T00:54:46.211 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:54:46.211 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:54:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.213+0000 7f84b5200700 1 -- 172.21.15.38:0/2188385483 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f849406e0e0 msgr2=0x7f84940705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.213+0000 7f84b5200700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f849406e0e0 0x7f84940705a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f849c005fd0 tx=0x7f849c005dc0 comp rx=0 tx=0).stop 2024-03-24T00:54:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.213+0000 7f84b5200700 1 -- 172.21.15.38:0/2188385483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b0102ef0 msgr2=0x7f84b019e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.213+0000 7f84b5200700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b0102ef0 0x7f84b019e9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84a0006010 tx=0x7f84a000be20 comp rx=0 tx=0).stop 2024-03-24T00:54:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.213+0000 7f84b5200700 1 -- 172.21.15.38:0/2188385483 shutdown_connections 2024-03-24T00:54:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.213+0000 7f84b5200700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f849406e0e0 0x7f84940705a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.213+0000 7f84b5200700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b0102ef0 0x7f84b019e9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.213+0000 7f84b5200700 1 --2- 172.21.15.38:0/2188385483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84b01038a0 0x7f84b019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:46.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.213+0000 7f84b5200700 1 -- 172.21.15.38:0/2188385483 >> 172.21.15.38:0/2188385483 conn(0x7f84b00fe780 msgr2=0x7f84b00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:46.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.214+0000 7f84b5200700 1 -- 172.21.15.38:0/2188385483 shutdown_connections 2024-03-24T00:54:46.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.214+0000 7f84b5200700 1 -- 172.21.15.38:0/2188385483 wait complete. 2024-03-24T00:54:46.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.382+0000 7f393a400700 1 -- 172.21.15.38:0/4128254002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3934106a70 msgr2=0x7f3934106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:46.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.382+0000 7f393a400700 1 --2- 172.21.15.38:0/4128254002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3934106a70 0x7f3934106e50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3924009910 tx=0x7f3924009c20 comp rx=0 tx=0).stop 2024-03-24T00:54:46.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.382+0000 7f393a400700 1 -- 172.21.15.38:0/4128254002 shutdown_connections 2024-03-24T00:54:46.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.382+0000 7f393a400700 1 --2- 172.21.15.38:0/4128254002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3934100a70 0x7f3934100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:46.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.382+0000 7f393a400700 1 --2- 172.21.15.38:0/4128254002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3934106a70 0x7f3934106e50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:46.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.382+0000 7f393a400700 1 -- 172.21.15.38:0/4128254002 >> 172.21.15.38:0/4128254002 conn(0x7f39340fc580 msgr2=0x7f39340fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:46.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.382+0000 7f393a400700 1 -- 172.21.15.38:0/4128254002 shutdown_connections 2024-03-24T00:54:46.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.383+0000 7f393a400700 1 -- 172.21.15.38:0/4128254002 wait complete. 2024-03-24T00:54:46.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.384+0000 7f393a400700 1 Processor -- start 2024-03-24T00:54:46.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.384+0000 7f393a400700 1 -- start start 2024-03-24T00:54:46.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.384+0000 7f393a400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3934100a70 0x7f393419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:46.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.384+0000 7f393a400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3934106a70 0x7f393419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:46.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.384+0000 7f393a400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393419f5a0 con 0x7f3934100a70 2024-03-24T00:54:46.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.384+0000 7f393a400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3934198b80 con 0x7f3934106a70 2024-03-24T00:54:46.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.385+0000 7f3933400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3934100a70 0x7f393419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:46.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.385+0000 7f3933400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3934100a70 0x7f393419ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55106/0 (socket says 172.21.15.38:55106) 2024-03-24T00:54:46.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.385+0000 7f3933400700 1 -- 172.21.15.38:0/2136548867 learned_addr learned my addr 172.21.15.38:0/2136548867 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:54:46.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.385+0000 7f392aa00700 1 --2- 172.21.15.38:0/2136548867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3934106a70 0x7f393419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:46.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.385+0000 7f3933400700 1 -- 172.21.15.38:0/2136548867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3934106a70 msgr2=0x7f393419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:46.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.385+0000 7f3933400700 1 --2- 172.21.15.38:0/2136548867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3934106a70 0x7f393419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:46.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.385+0000 7f3933400700 1 -- 172.21.15.38:0/2136548867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39240095f0 con 0x7f3934100a70 2024-03-24T00:54:46.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.386+0000 7f3933400700 1 --2- 172.21.15.38:0/2136548867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3934100a70 0x7f393419ea40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f39240092e0 tx=0x7f392400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:46.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.386+0000 7f3930c00700 1 -- 172.21.15.38:0/2136548867 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f392401d070 con 0x7f3934100a70 2024-03-24T00:54:46.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.386+0000 7f3930c00700 1 -- 172.21.15.38:0/2136548867 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f392400f460 con 0x7f3934100a70 2024-03-24T00:54:46.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.386+0000 7f393a400700 1 -- 172.21.15.38:0/2136548867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3934198e00 con 0x7f3934100a70 2024-03-24T00:54:46.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.386+0000 7f393a400700 1 -- 172.21.15.38:0/2136548867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39341992f0 con 0x7f3934100a70 2024-03-24T00:54:46.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.386+0000 7f3930c00700 1 -- 172.21.15.38:0/2136548867 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3924017630 con 0x7f3934100a70 2024-03-24T00:54:46.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.388+0000 7f393a400700 1 -- 172.21.15.38:0/2136548867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3934104d20 con 0x7f3934100a70 2024-03-24T00:54:46.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.389+0000 7f3930c00700 1 -- 172.21.15.38:0/2136548867 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f392400f5d0 con 0x7f3934100a70 2024-03-24T00:54:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.390+0000 7f3930c00700 1 --2- 172.21.15.38:0/2136548867 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f391806e0d0 0x7f3918070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:54:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.390+0000 7f3930c00700 1 -- 172.21.15.38:0/2136548867 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f392408f4d0 con 0x7f3934100a70 2024-03-24T00:54:46.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.390+0000 7f392aa00700 1 --2- 172.21.15.38:0/2136548867 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f391806e0d0 0x7f3918070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:54:46.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.391+0000 7f392aa00700 1 --2- 172.21.15.38:0/2136548867 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f391806e0d0 0x7f3918070590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f391c006fd0 tx=0x7f391c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:54:46.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.395+0000 7f3930c00700 1 -- 172.21.15.38:0/2136548867 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f392405c2b0 con 0x7f3934100a70 2024-03-24T00:54:46.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:46 smithi038 ceph-mon[112108]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:46.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:46 smithi038 ceph-mon[112108]: pgmap v27: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:46.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:46 smithi038 ceph-mon[112108]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:46.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:46 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1497706347' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:54:46.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:46 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3747130740' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:54:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:46 smithi088 ceph-mon[87351]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:46 smithi088 ceph-mon[87351]: pgmap v27: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:46 smithi088 ceph-mon[87351]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:46 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1497706347' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:54:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:46 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3747130740' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:54:46.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.724+0000 7f393a400700 1 -- 172.21.15.38:0/2136548867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f393404ea80 con 0x7f3934100a70 2024-03-24T00:54:46.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.724+0000 7f3930c00700 1 -- 172.21.15.38:0/2136548867 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f392405be40 con 0x7f3934100a70 2024-03-24T00:54:46.725 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_OK 2024-03-24T00:54:46.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.727+0000 7f393a400700 1 -- 172.21.15.38:0/2136548867 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f391806e0d0 msgr2=0x7f3918070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:46.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.727+0000 7f393a400700 1 --2- 172.21.15.38:0/2136548867 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f391806e0d0 0x7f3918070590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f391c006fd0 tx=0x7f391c008040 comp rx=0 tx=0).stop 2024-03-24T00:54:46.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.727+0000 7f393a400700 1 -- 172.21.15.38:0/2136548867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3934100a70 msgr2=0x7f393419ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:54:46.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.727+0000 7f393a400700 1 --2- 172.21.15.38:0/2136548867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3934100a70 0x7f393419ea40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f39240092e0 tx=0x7f392400be20 comp rx=0 tx=0).stop 2024-03-24T00:54:46.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.727+0000 7f393a400700 1 -- 172.21.15.38:0/2136548867 shutdown_connections 2024-03-24T00:54:46.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.727+0000 7f393a400700 1 --2- 172.21.15.38:0/2136548867 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f391806e0d0 0x7f3918070590 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:46.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.727+0000 7f393a400700 1 --2- 172.21.15.38:0/2136548867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3934100a70 0x7f393419ea40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:46.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.727+0000 7f393a400700 1 --2- 172.21.15.38:0/2136548867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3934106a70 0x7f393419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:54:46.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.727+0000 7f393a400700 1 -- 172.21.15.38:0/2136548867 >> 172.21.15.38:0/2136548867 conn(0x7f39340fc580 msgr2=0x7f39340732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:54:46.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.728+0000 7f393a400700 1 -- 172.21.15.38:0/2136548867 shutdown_connections 2024-03-24T00:54:46.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:54:46.728+0000 7f393a400700 1 -- 172.21.15.38:0/2136548867 wait complete. 2024-03-24T00:54:47.557 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:47 smithi038 ceph-mon[112108]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:47.557 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:47 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:47.557 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:47 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:47.557 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:47 smithi038 ceph-mon[112108]: Reconfiguring osd.0 (monmap changed)... 2024-03-24T00:54:47.557 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:47 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T00:54:47.558 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:47 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:47.558 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:47 smithi038 ceph-mon[112108]: Reconfiguring daemon osd.0 on smithi038 2024-03-24T00:54:47.558 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:47 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2136548867' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:54:47.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:47 smithi088 ceph-mon[87351]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:54:47.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:47 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:47 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:47 smithi088 ceph-mon[87351]: Reconfiguring osd.0 (monmap changed)... 2024-03-24T00:54:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:47 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T00:54:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:47 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:47 smithi088 ceph-mon[87351]: Reconfiguring daemon osd.0 on smithi038 2024-03-24T00:54:47.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:47 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2136548867' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:54:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:48 smithi088 ceph-mon[87351]: pgmap v28: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:48.710 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:48 smithi038 ceph-mon[112108]: pgmap v28: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:50.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:50 smithi038 ceph-mon[112108]: pgmap v29: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:50.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:50 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:50.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:50 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:50.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:50 smithi038 ceph-mon[112108]: Reconfiguring osd.1 (monmap changed)... 2024-03-24T00:54:50.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:50 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T00:54:50.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:50 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:50.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:50 smithi038 ceph-mon[112108]: Reconfiguring daemon osd.1 on smithi038 2024-03-24T00:54:50.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:50 smithi088 ceph-mon[87351]: pgmap v29: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:50.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:50 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:50.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:50 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:50.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:50 smithi088 ceph-mon[87351]: Reconfiguring osd.1 (monmap changed)... 2024-03-24T00:54:50.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:50 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T00:54:50.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:50 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:50.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:50 smithi088 ceph-mon[87351]: Reconfiguring daemon osd.1 on smithi038 2024-03-24T00:54:52.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:52 smithi038 ceph-mon[112108]: pgmap v30: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:52.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:52.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:52.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T00:54:52.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:52.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:52 smithi088 ceph-mon[87351]: pgmap v30: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T00:54:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:53.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:53 smithi038 ceph-mon[112108]: Reconfiguring osd.2 (monmap changed)... 2024-03-24T00:54:53.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:53 smithi038 ceph-mon[112108]: Reconfiguring daemon osd.2 on smithi038 2024-03-24T00:54:53.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:53 smithi088 ceph-mon[87351]: Reconfiguring osd.2 (monmap changed)... 2024-03-24T00:54:53.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:53 smithi088 ceph-mon[87351]: Reconfiguring daemon osd.2 on smithi038 2024-03-24T00:54:54.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:54 smithi038 ceph-mon[112108]: pgmap v31: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:54.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:54 smithi088 ceph-mon[87351]: pgmap v31: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:55.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:55.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:55.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:55 smithi088 ceph-mon[87351]: Reconfiguring mds.cephfs.smithi038.yhkmsb (monmap changed)... 2024-03-24T00:54:55.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.yhkmsb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:54:55.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:55.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:55 smithi088 ceph-mon[87351]: Reconfiguring daemon mds.cephfs.smithi038.yhkmsb on smithi038 2024-03-24T00:54:55.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:55.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:55.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:55 smithi038 ceph-mon[112108]: Reconfiguring mds.cephfs.smithi038.yhkmsb (monmap changed)... 2024-03-24T00:54:55.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.yhkmsb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:54:55.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:55.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:55 smithi038 ceph-mon[112108]: Reconfiguring daemon mds.cephfs.smithi038.yhkmsb on smithi038 2024-03-24T00:54:56.317 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:56 smithi038 ceph-mon[112108]: pgmap v32: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:56 smithi088 ceph-mon[87351]: pgmap v32: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:57.664 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:57.664 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:57.665 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:57 smithi038 ceph-mon[112108]: Reconfiguring mds.cephfs.smithi038.xnpqhp (monmap changed)... 2024-03-24T00:54:57.665 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.xnpqhp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:54:57.665 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:57.665 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:57 smithi038 ceph-mon[112108]: Reconfiguring daemon mds.cephfs.smithi038.xnpqhp on smithi038 2024-03-24T00:54:57.665 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:57 smithi038 ceph-mon[112108]: pgmap v33: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:57.665 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:54:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:57 smithi088 ceph-mon[87351]: Reconfiguring mds.cephfs.smithi038.xnpqhp (monmap changed)... 2024-03-24T00:54:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.xnpqhp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:54:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:57 smithi088 ceph-mon[87351]: Reconfiguring daemon mds.cephfs.smithi038.xnpqhp on smithi038 2024-03-24T00:54:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:57 smithi088 ceph-mon[87351]: pgmap v33: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:54:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:54:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:59 smithi088 ceph-mon[87351]: pgmap v34: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:54:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:59 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:59 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:54:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:59 smithi088 ceph-mon[87351]: Reconfiguring ceph-exporter.smithi088 (monmap changed)... 2024-03-24T00:54:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:59 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:54:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:59 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:54:59.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:59 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-24T00:54:59.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:59 smithi088 ceph-mon[87351]: Unable to update caps for client.ceph-exporter.smithi088 2024-03-24T00:54:59.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:59 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi088"}]: dispatch 2024-03-24T00:54:59.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:59 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:54:59.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:54:59 smithi088 ceph-mon[87351]: Reconfiguring daemon ceph-exporter.smithi088 on smithi088 2024-03-24T00:55:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:59 smithi038 ceph-mon[112108]: pgmap v34: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:59 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:59 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:59 smithi038 ceph-mon[112108]: Reconfiguring ceph-exporter.smithi088 (monmap changed)... 2024-03-24T00:55:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:59 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:55:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:59 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T00:55:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:59 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-24T00:55:00.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:59 smithi038 ceph-mon[112108]: Unable to update caps for client.ceph-exporter.smithi088 2024-03-24T00:55:00.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:59 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi088"}]: dispatch 2024-03-24T00:55:00.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:59 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:00.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:54:59 smithi038 ceph-mon[112108]: Reconfiguring daemon ceph-exporter.smithi088 on smithi088 2024-03-24T00:55:02.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:02 smithi088 ceph-mon[87351]: pgmap v35: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:02.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:02 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:02.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:02 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:02.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:02 smithi088 ceph-mon[87351]: Reconfiguring crash.smithi088 (monmap changed)... 2024-03-24T00:55:02.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:02 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:55:02.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:02 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:02.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:02 smithi088 ceph-mon[87351]: Reconfiguring daemon crash.smithi088 on smithi088 2024-03-24T00:55:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:02 smithi038 ceph-mon[112108]: pgmap v35: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:02 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:02 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:02 smithi038 ceph-mon[112108]: Reconfiguring crash.smithi088 (monmap changed)... 2024-03-24T00:55:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:02 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:55:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:02 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:02 smithi038 ceph-mon[112108]: Reconfiguring daemon crash.smithi088 on smithi088 2024-03-24T00:55:04.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:04 smithi088 ceph-mon[87351]: pgmap v36: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:04.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:04 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:04.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:04 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:04.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:04 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:55:04.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:04 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:55:04.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:04 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:04.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:04 smithi038 ceph-mon[112108]: pgmap v36: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:04.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:04 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:04.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:04 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:04.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:04 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.njctea", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T00:55:04.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:04 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T00:55:04.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:04 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:05.429 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:05 smithi088 ceph-mon[87351]: Reconfiguring mgr.smithi088.njctea (monmap changed)... 2024-03-24T00:55:05.429 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:05 smithi088 ceph-mon[87351]: Reconfiguring daemon mgr.smithi088.njctea on smithi088 2024-03-24T00:55:05.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:05 smithi038 ceph-mon[112108]: Reconfiguring mgr.smithi088.njctea (monmap changed)... 2024-03-24T00:55:05.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:05 smithi038 ceph-mon[112108]: Reconfiguring daemon mgr.smithi088.njctea on smithi088 2024-03-24T00:55:06.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:06 smithi088 ceph-mon[87351]: pgmap v37: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:06.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:06 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:06.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:06 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:06.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:06 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:55:06.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:06 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:55:06.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:06 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:06.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:06 smithi038 ceph-mon[112108]: pgmap v37: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:06.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:06 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:06.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:06 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:06.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:06 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T00:55:06.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:06 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T00:55:06.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:06 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:07.544 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:07 smithi088 ceph-mon[87351]: Reconfiguring mon.smithi088 (monmap changed)... 2024-03-24T00:55:07.544 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:07 smithi088 ceph-mon[87351]: Reconfiguring daemon mon.smithi088 on smithi088 2024-03-24T00:55:07.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:07 smithi038 ceph-mon[112108]: Reconfiguring mon.smithi088 (monmap changed)... 2024-03-24T00:55:07.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:07 smithi038 ceph-mon[112108]: Reconfiguring daemon mon.smithi088 on smithi088 2024-03-24T00:55:08.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:08 smithi038 ceph-mon[112108]: pgmap v38: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:08.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:08 smithi088 ceph-mon[87351]: pgmap v38: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:09.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:09 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:09.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:09 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:09.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:09 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T00:55:09.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:09 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:09.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:09 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:09.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:09 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:09.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:09 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T00:55:09.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:09 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:10.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:10 smithi088 ceph-mon[87351]: Reconfiguring osd.3 (monmap changed)... 2024-03-24T00:55:10.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:10 smithi088 ceph-mon[87351]: Reconfiguring daemon osd.3 on smithi088 2024-03-24T00:55:10.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:10 smithi088 ceph-mon[87351]: pgmap v39: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:10.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:10 smithi038 ceph-mon[112108]: Reconfiguring osd.3 (monmap changed)... 2024-03-24T00:55:10.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:10 smithi038 ceph-mon[112108]: Reconfiguring daemon osd.3 on smithi088 2024-03-24T00:55:10.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:10 smithi038 ceph-mon[112108]: pgmap v39: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:12.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:11 smithi088 ceph-mon[87351]: pgmap v40: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:12.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:12.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:12.093 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:11 smithi088 ceph-mon[87351]: Reconfiguring osd.4 (monmap changed)... 2024-03-24T00:55:12.093 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T00:55:12.093 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:12.093 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:11 smithi088 ceph-mon[87351]: Reconfiguring daemon osd.4 on smithi088 2024-03-24T00:55:12.093 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:55:12.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:11 smithi038 ceph-mon[112108]: pgmap v40: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:12.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:12.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:12.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:11 smithi038 ceph-mon[112108]: Reconfiguring osd.4 (monmap changed)... 2024-03-24T00:55:12.316 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T00:55:12.316 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:12.316 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:11 smithi038 ceph-mon[112108]: Reconfiguring daemon osd.4 on smithi088 2024-03-24T00:55:12.316 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:55:14.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:13 smithi088 ceph-mon[87351]: pgmap v41: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:55:14.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:13 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:14.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:13 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:14.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:13 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T00:55:14.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:13 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:14.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:13 smithi038 ceph-mon[112108]: pgmap v41: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:55:14.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:13 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:13 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:13 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T00:55:14.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:13 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:15.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:14 smithi088 ceph-mon[87351]: Reconfiguring osd.5 (monmap changed)... 2024-03-24T00:55:15.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:14 smithi088 ceph-mon[87351]: Reconfiguring daemon osd.5 on smithi088 2024-03-24T00:55:15.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:14 smithi038 ceph-mon[112108]: Reconfiguring osd.5 (monmap changed)... 2024-03-24T00:55:15.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:14 smithi038 ceph-mon[112108]: Reconfiguring daemon osd.5 on smithi088 2024-03-24T00:55:15.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:15 smithi088 ceph-mon[87351]: pgmap v42: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:16.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:15 smithi038 ceph-mon[112108]: pgmap v42: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.906+0000 7fbf56a00700 1 -- 172.21.15.38:0/506649516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf50107cc0 msgr2=0x7fbf5010bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.906+0000 7fbf56a00700 1 --2- 172.21.15.38:0/506649516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf50107cc0 0x7fbf5010bd10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009910 tx=0x7fbf3c009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.906+0000 7fbf56a00700 1 -- 172.21.15.38:0/506649516 shutdown_connections 2024-03-24T00:55:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.906+0000 7fbf56a00700 1 --2- 172.21.15.38:0/506649516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf50107cc0 0x7fbf5010bd10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.906+0000 7fbf56a00700 1 --2- 172.21.15.38:0/506649516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf50107310 0x7fbf501076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.906+0000 7fbf56a00700 1 -- 172.21.15.38:0/506649516 >> 172.21.15.38:0/506649516 conn(0x7fbf500768e0 msgr2=0x7fbf50078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.906+0000 7fbf56a00700 1 -- 172.21.15.38:0/506649516 shutdown_connections 2024-03-24T00:55:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.906+0000 7fbf56a00700 1 -- 172.21.15.38:0/506649516 wait complete. 2024-03-24T00:55:16.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.907+0000 7fbf56a00700 1 Processor -- start 2024-03-24T00:55:16.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.907+0000 7fbf56a00700 1 -- start start 2024-03-24T00:55:16.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.908+0000 7fbf56a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf50107310 0x7fbf5019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:16.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.908+0000 7fbf56a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf50107cc0 0x7fbf5019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:16.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.908+0000 7fbf56a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5019dbe0 con 0x7fbf50107cc0 2024-03-24T00:55:16.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.908+0000 7fbf56a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf501a18b0 con 0x7fbf50107310 2024-03-24T00:55:16.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.908+0000 7fbf4f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf50107cc0 0x7fbf5019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:16.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.908+0000 7fbf4f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf50107cc0 0x7fbf5019d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42834/0 (socket says 172.21.15.38:42834) 2024-03-24T00:55:16.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.908+0000 7fbf4f400700 1 -- 172.21.15.38:0/426625831 learned_addr learned my addr 172.21.15.38:0/426625831 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:16.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.908+0000 7fbf4fe00700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf50107310 0x7fbf5019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:16.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.909+0000 7fbf4f400700 1 -- 172.21.15.38:0/426625831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf50107310 msgr2=0x7fbf5019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:16.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.909+0000 7fbf4f400700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf50107310 0x7fbf5019cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:16.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.909+0000 7fbf4f400700 1 -- 172.21.15.38:0/426625831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0095f0 con 0x7fbf50107cc0 2024-03-24T00:55:16.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.909+0000 7fbf4fe00700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf50107310 0x7fbf5019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T00:55:16.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.909+0000 7fbf4f400700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf50107cc0 0x7fbf5019d500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c005b40 tx=0x7fbf3c00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:16.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.909+0000 7fbf4cc00700 1 -- 172.21.15.38:0/426625831 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf3c01d070 con 0x7fbf50107cc0 2024-03-24T00:55:16.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.909+0000 7fbf4cc00700 1 -- 172.21.15.38:0/426625831 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf3c0052a0 con 0x7fbf50107cc0 2024-03-24T00:55:16.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.910+0000 7fbf4cc00700 1 -- 172.21.15.38:0/426625831 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf3c003e50 con 0x7fbf50107cc0 2024-03-24T00:55:16.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.910+0000 7fbf56a00700 1 -- 172.21.15.38:0/426625831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf501a1b30 con 0x7fbf50107cc0 2024-03-24T00:55:16.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.910+0000 7fbf56a00700 1 -- 172.21.15.38:0/426625831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf501a2020 con 0x7fbf50107cc0 2024-03-24T00:55:16.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.911+0000 7fbf56a00700 1 -- 172.21.15.38:0/426625831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5010f710 con 0x7fbf50107cc0 2024-03-24T00:55:16.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.912+0000 7fbf4cc00700 1 -- 172.21.15.38:0/426625831 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7fbf3c00f6f0 con 0x7fbf50107cc0 2024-03-24T00:55:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.913+0000 7fbf4cc00700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf3406e0d0 0x7fbf34070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.913+0000 7fbf4cc00700 1 -- 172.21.15.38:0/426625831 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbf3c08f460 con 0x7fbf50107cc0 2024-03-24T00:55:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.913+0000 7fbf4fe00700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf3406e0d0 0x7fbf34070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.914+0000 7fbf4fe00700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf3406e0d0 0x7fbf34070590 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbf40011490 tx=0x7fbf40009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:16.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:16.918+0000 7fbf4cc00700 1 -- 172.21.15.38:0/426625831 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbf3c0586f0 con 0x7fbf50107cc0 2024-03-24T00:55:17.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.177+0000 7fbf56a00700 1 -- 172.21.15.38:0/426625831 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf501a2240 con 0x7fbf3406e0d0 2024-03-24T00:55:17.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.179+0000 7fbf4cc00700 1 -- 172.21.15.38:0/426625831 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fbf501a2240 con 0x7fbf3406e0d0 2024-03-24T00:55:17.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.182+0000 7fbf56a00700 1 -- 172.21.15.38:0/426625831 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf3406e0d0 msgr2=0x7fbf34070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:17.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.182+0000 7fbf56a00700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf3406e0d0 0x7fbf34070590 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbf40011490 tx=0x7fbf40009260 comp rx=0 tx=0).stop 2024-03-24T00:55:17.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.182+0000 7fbf56a00700 1 -- 172.21.15.38:0/426625831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf50107cc0 msgr2=0x7fbf5019d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:17.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.182+0000 7fbf56a00700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf50107cc0 0x7fbf5019d500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c005b40 tx=0x7fbf3c00b5a0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.183+0000 7fbf56a00700 1 -- 172.21.15.38:0/426625831 shutdown_connections 2024-03-24T00:55:17.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.183+0000 7fbf56a00700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf3406e0d0 0x7fbf34070590 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.183+0000 7fbf56a00700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf50107310 0x7fbf5019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.183+0000 7fbf56a00700 1 --2- 172.21.15.38:0/426625831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf50107cc0 0x7fbf5019d500 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.183+0000 7fbf56a00700 1 -- 172.21.15.38:0/426625831 >> 172.21.15.38:0/426625831 conn(0x7fbf500768e0 msgr2=0x7fbf50076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.183+0000 7fbf56a00700 1 -- 172.21.15.38:0/426625831 shutdown_connections 2024-03-24T00:55:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.183+0000 7fbf56a00700 1 -- 172.21.15.38:0/426625831 wait complete. 2024-03-24T00:55:17.200 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:55:17.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.347+0000 7f3f90a00700 1 -- 172.21.15.38:0/2977889531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f8c102530 msgr2=0x7f3f8c10aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:17.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.347+0000 7f3f90a00700 1 --2- 172.21.15.38:0/2977889531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f8c102530 0x7f3f8c10aa20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009910 tx=0x7f3f74009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:17.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.347+0000 7f3f90a00700 1 -- 172.21.15.38:0/2977889531 shutdown_connections 2024-03-24T00:55:17.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.347+0000 7f3f90a00700 1 --2- 172.21.15.38:0/2977889531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f8c102530 0x7f3f8c10aa20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.347+0000 7f3f90a00700 1 --2- 172.21.15.38:0/2977889531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f8c101c10 0x7f3f8c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.347+0000 7f3f90a00700 1 -- 172.21.15.38:0/2977889531 >> 172.21.15.38:0/2977889531 conn(0x7f3f8c076640 msgr2=0x7f3f8c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:17.349 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:17.350 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:17.350 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:17 smithi038 ceph-mon[112108]: Reconfiguring mds.cephfs.smithi088.cbfvvp (monmap changed)... 2024-03-24T00:55:17.350 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.cbfvvp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:55:17.350 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:17.350 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:17 smithi038 ceph-mon[112108]: Reconfiguring daemon mds.cephfs.smithi088.cbfvvp on smithi088 2024-03-24T00:55:17.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.348+0000 7f3f90a00700 1 -- 172.21.15.38:0/2977889531 shutdown_connections 2024-03-24T00:55:17.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.348+0000 7f3f90a00700 1 -- 172.21.15.38:0/2977889531 wait complete. 2024-03-24T00:55:17.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.349+0000 7f3f90a00700 1 Processor -- start 2024-03-24T00:55:17.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.349+0000 7f3f90a00700 1 -- start start 2024-03-24T00:55:17.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.349+0000 7f3f90a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f8c101c10 0x7f3f8c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.349+0000 7f3f90a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f8c102530 0x7f3f8c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.349+0000 7f3f90a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c19f470 con 0x7f3f8c101c10 2024-03-24T00:55:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.349+0000 7f3f90a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c198a50 con 0x7f3f8c102530 2024-03-24T00:55:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.350+0000 7f3f89e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f8c101c10 0x7f3f8c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.350+0000 7f3f89e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f8c101c10 0x7f3f8c19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42852/0 (socket says 172.21.15.38:42852) 2024-03-24T00:55:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.350+0000 7f3f89e00700 1 -- 172.21.15.38:0/2193140759 learned_addr learned my addr 172.21.15.38:0/2193140759 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.350+0000 7f3f81400700 1 --2- 172.21.15.38:0/2193140759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f8c102530 0x7f3f8c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.350+0000 7f3f89e00700 1 -- 172.21.15.38:0/2193140759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f8c102530 msgr2=0x7f3f8c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.350+0000 7f3f89e00700 1 --2- 172.21.15.38:0/2193140759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f8c102530 0x7f3f8c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.350+0000 7f3f89e00700 1 -- 172.21.15.38:0/2193140759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740095f0 con 0x7f3f8c101c10 2024-03-24T00:55:17.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.351+0000 7f3f89e00700 1 --2- 172.21.15.38:0/2193140759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f8c101c10 0x7f3f8c19e910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00d8d0 tx=0x7f3f7c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:17.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.351+0000 7f3f83400700 1 -- 172.21.15.38:0/2193140759 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f7c0098a0 con 0x7f3f8c101c10 2024-03-24T00:55:17.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.351+0000 7f3f83400700 1 -- 172.21.15.38:0/2193140759 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f7c010460 con 0x7f3f8c101c10 2024-03-24T00:55:17.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.351+0000 7f3f90a00700 1 -- 172.21.15.38:0/2193140759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8c198d30 con 0x7f3f8c101c10 2024-03-24T00:55:17.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.351+0000 7f3f83400700 1 -- 172.21.15.38:0/2193140759 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f7c00f5f0 con 0x7f3f8c101c10 2024-03-24T00:55:17.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.351+0000 7f3f90a00700 1 -- 172.21.15.38:0/2193140759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8c199250 con 0x7f3f8c101c10 2024-03-24T00:55:17.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.353+0000 7f3f90a00700 1 -- 172.21.15.38:0/2193140759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8c100f40 con 0x7f3f8c101c10 2024-03-24T00:55:17.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.354+0000 7f3f83400700 1 -- 172.21.15.38:0/2193140759 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f3f7c009a00 con 0x7f3f8c101c10 2024-03-24T00:55:17.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.355+0000 7f3f83400700 1 --2- 172.21.15.38:0/2193140759 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3f7006e0d0 0x7f3f70070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:17.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.355+0000 7f3f83400700 1 -- 172.21.15.38:0/2193140759 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3f7c08dd90 con 0x7f3f8c101c10 2024-03-24T00:55:17.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.355+0000 7f3f81400700 1 --2- 172.21.15.38:0/2193140759 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3f7006e0d0 0x7f3f70070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:17.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.356+0000 7f3f81400700 1 --2- 172.21.15.38:0/2193140759 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3f7006e0d0 0x7f3f70070590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009fd0 tx=0x7f3f7401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:17.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.359+0000 7f3f83400700 1 -- 172.21.15.38:0/2193140759 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3f7c05abf0 con 0x7f3f8c101c10 2024-03-24T00:55:17.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:17.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:17.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:17 smithi088 ceph-mon[87351]: Reconfiguring mds.cephfs.smithi088.cbfvvp (monmap changed)... 2024-03-24T00:55:17.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.cbfvvp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:55:17.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:17.366 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:17 smithi088 ceph-mon[87351]: Reconfiguring daemon mds.cephfs.smithi088.cbfvvp on smithi088 2024-03-24T00:55:17.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.609+0000 7f3f90a00700 1 -- 172.21.15.38:0/2193140759 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f8c199f80 con 0x7f3f7006e0d0 2024-03-24T00:55:17.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.611+0000 7f3f83400700 1 -- 172.21.15.38:0/2193140759 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f3f8c199f80 con 0x7f3f7006e0d0 2024-03-24T00:55:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.614+0000 7f3f90a00700 1 -- 172.21.15.38:0/2193140759 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3f7006e0d0 msgr2=0x7f3f70070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.614+0000 7f3f90a00700 1 --2- 172.21.15.38:0/2193140759 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3f7006e0d0 0x7f3f70070590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009fd0 tx=0x7f3f7401a040 comp rx=0 tx=0).stop 2024-03-24T00:55:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.614+0000 7f3f90a00700 1 -- 172.21.15.38:0/2193140759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f8c101c10 msgr2=0x7f3f8c19e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.614+0000 7f3f90a00700 1 --2- 172.21.15.38:0/2193140759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f8c101c10 0x7f3f8c19e910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00d8d0 tx=0x7f3f7c00dbe0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.614+0000 7f3f90a00700 1 -- 172.21.15.38:0/2193140759 shutdown_connections 2024-03-24T00:55:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.614+0000 7f3f90a00700 1 --2- 172.21.15.38:0/2193140759 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3f7006e0d0 0x7f3f70070590 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.614+0000 7f3f90a00700 1 --2- 172.21.15.38:0/2193140759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f8c101c10 0x7f3f8c19e910 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.614+0000 7f3f90a00700 1 --2- 172.21.15.38:0/2193140759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f8c102530 0x7f3f8c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.614+0000 7f3f90a00700 1 -- 172.21.15.38:0/2193140759 >> 172.21.15.38:0/2193140759 conn(0x7f3f8c076640 msgr2=0x7f3f8c109350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.615+0000 7f3f90a00700 1 -- 172.21.15.38:0/2193140759 shutdown_connections 2024-03-24T00:55:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.615+0000 7f3f90a00700 1 -- 172.21.15.38:0/2193140759 wait complete. 2024-03-24T00:55:17.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.775+0000 7f0259000700 1 -- 172.21.15.38:0/2682282066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02540731e0 msgr2=0x7f02540735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:17.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.775+0000 7f0259000700 1 --2- 172.21.15.38:0/2682282066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02540731e0 0x7f02540735c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0244009910 tx=0x7f0244009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:17.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.775+0000 7f0259000700 1 -- 172.21.15.38:0/2682282066 shutdown_connections 2024-03-24T00:55:17.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.775+0000 7f0259000700 1 --2- 172.21.15.38:0/2682282066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0254073b00 0x7f025410cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.775+0000 7f0259000700 1 --2- 172.21.15.38:0/2682282066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02540731e0 0x7f02540735c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.775+0000 7f0259000700 1 -- 172.21.15.38:0/2682282066 >> 172.21.15.38:0/2682282066 conn(0x7f02540fc570 msgr2=0x7f02540fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:17.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.776+0000 7f0259000700 1 -- 172.21.15.38:0/2682282066 shutdown_connections 2024-03-24T00:55:17.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.776+0000 7f0259000700 1 -- 172.21.15.38:0/2682282066 wait complete. 2024-03-24T00:55:17.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.777+0000 7f0259000700 1 Processor -- start 2024-03-24T00:55:17.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.777+0000 7f0259000700 1 -- start start 2024-03-24T00:55:17.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.777+0000 7f0259000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02540731e0 0x7f025419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:17.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.777+0000 7f0259000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0254073b00 0x7f025419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:17.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.777+0000 7f0259000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025419f440 con 0x7f0254073b00 2024-03-24T00:55:17.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.777+0000 7f0259000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0254198a20 con 0x7f02540731e0 2024-03-24T00:55:17.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.778+0000 7f0251400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0254073b00 0x7f025419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:17.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.778+0000 7f0251400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0254073b00 0x7f025419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42868/0 (socket says 172.21.15.38:42868) 2024-03-24T00:55:17.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.778+0000 7f0251400700 1 -- 172.21.15.38:0/2500223834 learned_addr learned my addr 172.21.15.38:0/2500223834 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:17.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.778+0000 7f0251e00700 1 --2- 172.21.15.38:0/2500223834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02540731e0 0x7f025419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:17.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.778+0000 7f0251400700 1 -- 172.21.15.38:0/2500223834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02540731e0 msgr2=0x7f025419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:17.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.778+0000 7f0251400700 1 --2- 172.21.15.38:0/2500223834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02540731e0 0x7f025419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:17.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.778+0000 7f0251400700 1 -- 172.21.15.38:0/2500223834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02440095f0 con 0x7f0254073b00 2024-03-24T00:55:17.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.778+0000 7f0251400700 1 --2- 172.21.15.38:0/2500223834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0254073b00 0x7f025419ee20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f024000d8d0 tx=0x7f024000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:17.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.779+0000 7f024aa00700 1 -- 172.21.15.38:0/2500223834 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0240009940 con 0x7f0254073b00 2024-03-24T00:55:17.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.779+0000 7f024aa00700 1 -- 172.21.15.38:0/2500223834 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0240010460 con 0x7f0254073b00 2024-03-24T00:55:17.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.779+0000 7f024aa00700 1 -- 172.21.15.38:0/2500223834 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f024000f5f0 con 0x7f0254073b00 2024-03-24T00:55:17.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.779+0000 7f0259000700 1 -- 172.21.15.38:0/2500223834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0254198d00 con 0x7f0254073b00 2024-03-24T00:55:17.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.779+0000 7f0259000700 1 -- 172.21.15.38:0/2500223834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0254199250 con 0x7f0254073b00 2024-03-24T00:55:17.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.780+0000 7f0259000700 1 -- 172.21.15.38:0/2500223834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0254107670 con 0x7f0254073b00 2024-03-24T00:55:17.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.781+0000 7f024aa00700 1 -- 172.21.15.38:0/2500223834 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f02400105d0 con 0x7f0254073b00 2024-03-24T00:55:17.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.782+0000 7f024aa00700 1 --2- 172.21.15.38:0/2500223834 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f023806e0d0 0x7f0238070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:17.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.782+0000 7f024aa00700 1 -- 172.21.15.38:0/2500223834 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f024008df20 con 0x7f0254073b00 2024-03-24T00:55:17.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.782+0000 7f0251e00700 1 --2- 172.21.15.38:0/2500223834 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f023806e0d0 0x7f0238070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:17.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.783+0000 7f0251e00700 1 --2- 172.21.15.38:0/2500223834 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f023806e0d0 0x7f0238070590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0244006010 tx=0x7f024400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:17.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:17.787+0000 7f024aa00700 1 -- 172.21.15.38:0/2500223834 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f024005ace0 con 0x7f0254073b00 2024-03-24T00:55:18.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.017+0000 7f0259000700 1 -- 172.21.15.38:0/2500223834 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f02541994c0 con 0x7f023806e0d0 2024-03-24T00:55:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.037+0000 7f024aa00700 1 -- 172.21.15.38:0/2500223834 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f02541994c0 con 0x7f023806e0d0 2024-03-24T00:55:18.038 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:55:18.038 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (3m) 73s ago 17m 14.4M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:55:18.038 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (17m) 73s ago 17m 8426k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (16m) 49s ago 16m 8270k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (17m) 73s ago 17m 6660k - 18.2.2-952-g738c2fd6 68aff9aff3b2 c542833ea418 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (16m) 49s ago 16m 6669k - 18.2.2-952-g738c2fd6 68aff9aff3b2 ef285af01313 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (2m) 73s ago 17m 75.2M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (10m) 73s ago 10m 97.5M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (10m) 73s ago 10m 151M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (10m) 49s ago 10m 24.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (10m) 49s ago 10m 24.8M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (6m) 73s ago 19m 478M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (5m) 49s ago 16m 448M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (91s) 73s ago 19m 29.0M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (55s) 49s ago 16m 20.4M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (4m) 73s ago 17m 7684k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:55:18.039 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (4m) 49s ago 16m 7868k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:55:18.040 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (15m) 73s ago 15m 258M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:55:18.040 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (14m) 73s ago 14m 272M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:55:18.040 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (13m) 73s ago 13m 228M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:55:18.040 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (13m) 49s ago 13m 303M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:55:18.040 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (12m) 49s ago 12m 283M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:55:18.040 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (12m) 49s ago 12m 252M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:55:18.040 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (4m) 73s ago 16m 66.3M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:55:18.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.039+0000 7f0259000700 1 -- 172.21.15.38:0/2500223834 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f023806e0d0 msgr2=0x7f0238070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:18.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.039+0000 7f0259000700 1 --2- 172.21.15.38:0/2500223834 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f023806e0d0 0x7f0238070590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0244006010 tx=0x7f024400b540 comp rx=0 tx=0).stop 2024-03-24T00:55:18.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.039+0000 7f0259000700 1 -- 172.21.15.38:0/2500223834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0254073b00 msgr2=0x7f025419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:18.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.039+0000 7f0259000700 1 --2- 172.21.15.38:0/2500223834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0254073b00 0x7f025419ee20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f024000d8d0 tx=0x7f024000dc90 comp rx=0 tx=0).stop 2024-03-24T00:55:18.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.039+0000 7f0259000700 1 -- 172.21.15.38:0/2500223834 shutdown_connections 2024-03-24T00:55:18.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.039+0000 7f0259000700 1 --2- 172.21.15.38:0/2500223834 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f023806e0d0 0x7f0238070590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.039+0000 7f0259000700 1 --2- 172.21.15.38:0/2500223834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02540731e0 0x7f025419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.039+0000 7f0259000700 1 --2- 172.21.15.38:0/2500223834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0254073b00 0x7f025419ee20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.039+0000 7f0259000700 1 -- 172.21.15.38:0/2500223834 >> 172.21.15.38:0/2500223834 conn(0x7f02540fc570 msgr2=0x7f0254105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.040+0000 7f0259000700 1 -- 172.21.15.38:0/2500223834 shutdown_connections 2024-03-24T00:55:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.040+0000 7f0259000700 1 -- 172.21.15.38:0/2500223834 wait complete. 2024-03-24T00:55:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:18 smithi088 ceph-mon[87351]: pgmap v43: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:18 smithi088 ceph-mon[87351]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.186+0000 7f3b74800700 1 -- 172.21.15.38:0/2594111278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b6c102ef0 msgr2=0x7f3b6c1032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.186+0000 7f3b74800700 1 --2- 172.21.15.38:0/2594111278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b6c102ef0 0x7f3b6c1032d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009910 tx=0x7f3b60009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.186+0000 7f3b74800700 1 -- 172.21.15.38:0/2594111278 shutdown_connections 2024-03-24T00:55:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.186+0000 7f3b74800700 1 --2- 172.21.15.38:0/2594111278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b6c1038a0 0x7f3b6c1078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.186+0000 7f3b74800700 1 --2- 172.21.15.38:0/2594111278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b6c102ef0 0x7f3b6c1032d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.186+0000 7f3b74800700 1 -- 172.21.15.38:0/2594111278 >> 172.21.15.38:0/2594111278 conn(0x7f3b6c0fe780 msgr2=0x7f3b6c100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.187+0000 7f3b74800700 1 -- 172.21.15.38:0/2594111278 shutdown_connections 2024-03-24T00:55:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.187+0000 7f3b74800700 1 -- 172.21.15.38:0/2594111278 wait complete. 2024-03-24T00:55:18.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.188+0000 7f3b74800700 1 Processor -- start 2024-03-24T00:55:18.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.188+0000 7f3b74800700 1 -- start start 2024-03-24T00:55:18.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.188+0000 7f3b74800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b6c102ef0 0x7f3b6c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:18.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.188+0000 7f3b74800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b6c1038a0 0x7f3b6c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:18.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.188+0000 7f3b74800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c19f570 con 0x7f3b6c1038a0 2024-03-24T00:55:18.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.188+0000 7f3b74800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c198ae0 con 0x7f3b6c102ef0 2024-03-24T00:55:18.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.188+0000 7f3b71400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b6c1038a0 0x7f3b6c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:18.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.188+0000 7f3b71400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b6c1038a0 0x7f3b6c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42886/0 (socket says 172.21.15.38:42886) 2024-03-24T00:55:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.188+0000 7f3b71400700 1 -- 172.21.15.38:0/1527506992 learned_addr learned my addr 172.21.15.38:0/1527506992 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.189+0000 7f3b71e00700 1 --2- 172.21.15.38:0/1527506992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b6c102ef0 0x7f3b6c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.189+0000 7f3b71400700 1 -- 172.21.15.38:0/1527506992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b6c102ef0 msgr2=0x7f3b6c19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.189+0000 7f3b71400700 1 --2- 172.21.15.38:0/1527506992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b6c102ef0 0x7f3b6c19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.189+0000 7f3b71400700 1 -- 172.21.15.38:0/1527506992 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b600095f0 con 0x7f3b6c1038a0 2024-03-24T00:55:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.189+0000 7f3b71400700 1 --2- 172.21.15.38:0/1527506992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b6c1038a0 0x7f3b6c19eee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00d900 tx=0x7f3b5c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.189+0000 7f3b66a00700 1 -- 172.21.15.38:0/1527506992 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b5c009900 con 0x7f3b6c1038a0 2024-03-24T00:55:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.190+0000 7f3b66a00700 1 -- 172.21.15.38:0/1527506992 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b5c010460 con 0x7f3b6c1038a0 2024-03-24T00:55:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.190+0000 7f3b74800700 1 -- 172.21.15.38:0/1527506992 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6c198dc0 con 0x7f3b6c1038a0 2024-03-24T00:55:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.190+0000 7f3b66a00700 1 -- 172.21.15.38:0/1527506992 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b5c00f5f0 con 0x7f3b6c1038a0 2024-03-24T00:55:18.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.190+0000 7f3b74800700 1 -- 172.21.15.38:0/1527506992 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6c199310 con 0x7f3b6c1038a0 2024-03-24T00:55:18.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.191+0000 7f3b74800700 1 -- 172.21.15.38:0/1527506992 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6c10b290 con 0x7f3b6c1038a0 2024-03-24T00:55:18.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.192+0000 7f3b66a00700 1 -- 172.21.15.38:0/1527506992 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f3b5c0105d0 con 0x7f3b6c1038a0 2024-03-24T00:55:18.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.193+0000 7f3b66a00700 1 --2- 172.21.15.38:0/1527506992 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3b5406e0e0 0x7f3b540705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:18.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.193+0000 7f3b66a00700 1 -- 172.21.15.38:0/1527506992 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3b5c08dd90 con 0x7f3b6c1038a0 2024-03-24T00:55:18.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.194+0000 7f3b71e00700 1 --2- 172.21.15.38:0/1527506992 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3b5406e0e0 0x7f3b540705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:18.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.194+0000 7f3b71e00700 1 --2- 172.21.15.38:0/1527506992 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3b5406e0e0 0x7f3b540705a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b60005b40 tx=0x7f3b6001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.198+0000 7f3b66a00700 1 -- 172.21.15.38:0/1527506992 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3b5c05ab50 con 0x7f3b6c1038a0 2024-03-24T00:55:18.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:18 smithi038 ceph-mon[112108]: pgmap v43: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:18.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:18 smithi038 ceph-mon[112108]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:18.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.518+0000 7f3b74800700 1 -- 172.21.15.38:0/1527506992 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3b6c1043d0 con 0x7f3b6c1038a0 2024-03-24T00:55:18.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.519+0000 7f3b66a00700 1 -- 172.21.15.38:0/1527506992 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+730 (secure 0 0 0) 0x7f3b5c05a6e0 con 0x7f3b6c1038a0 2024-03-24T00:55:18.521 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:55:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:55:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:55:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:55:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:55:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:55:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:55:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:55:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:55:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:55:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:55:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:55:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:55:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:55:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 10, 2024-03-24T00:55:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-24T00:55:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:55:18.522 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:55:18.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.523+0000 7f3b74800700 1 -- 172.21.15.38:0/1527506992 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3b5406e0e0 msgr2=0x7f3b540705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:18.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.523+0000 7f3b74800700 1 --2- 172.21.15.38:0/1527506992 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3b5406e0e0 0x7f3b540705a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b60005b40 tx=0x7f3b6001a040 comp rx=0 tx=0).stop 2024-03-24T00:55:18.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.523+0000 7f3b74800700 1 -- 172.21.15.38:0/1527506992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b6c1038a0 msgr2=0x7f3b6c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:18.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.523+0000 7f3b74800700 1 --2- 172.21.15.38:0/1527506992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b6c1038a0 0x7f3b6c19eee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00d900 tx=0x7f3b5c00dcc0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.523+0000 7f3b74800700 1 -- 172.21.15.38:0/1527506992 shutdown_connections 2024-03-24T00:55:18.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.523+0000 7f3b74800700 1 --2- 172.21.15.38:0/1527506992 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3b5406e0e0 0x7f3b540705a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.523+0000 7f3b74800700 1 --2- 172.21.15.38:0/1527506992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b6c102ef0 0x7f3b6c19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.523+0000 7f3b74800700 1 --2- 172.21.15.38:0/1527506992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b6c1038a0 0x7f3b6c19eee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.523+0000 7f3b74800700 1 -- 172.21.15.38:0/1527506992 >> 172.21.15.38:0/1527506992 conn(0x7f3b6c0fe780 msgr2=0x7f3b6c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:18.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.523+0000 7f3b74800700 1 -- 172.21.15.38:0/1527506992 shutdown_connections 2024-03-24T00:55:18.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.523+0000 7f3b74800700 1 -- 172.21.15.38:0/1527506992 wait complete. 2024-03-24T00:55:18.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.681+0000 7f7635000700 1 -- 172.21.15.38:0/4110081714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76300ffa00 msgr2=0x7f76300ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:18.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.681+0000 7f7635000700 1 --2- 172.21.15.38:0/4110081714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76300ffa00 0x7f76300ffde0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f762000b0a0 tx=0x7f762000b3b0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.681+0000 7f7635000700 1 -- 172.21.15.38:0/4110081714 shutdown_connections 2024-03-24T00:55:18.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.681+0000 7f7635000700 1 --2- 172.21.15.38:0/4110081714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7630100320 0x7f76301047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.681+0000 7f7635000700 1 --2- 172.21.15.38:0/4110081714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76300ffa00 0x7f76300ffde0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.681+0000 7f7635000700 1 -- 172.21.15.38:0/4110081714 >> 172.21.15.38:0/4110081714 conn(0x7f7630076640 msgr2=0x7f76300fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:18.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.681+0000 7f7635000700 1 -- 172.21.15.38:0/4110081714 shutdown_connections 2024-03-24T00:55:18.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.681+0000 7f7635000700 1 -- 172.21.15.38:0/4110081714 wait complete. 2024-03-24T00:55:18.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.682+0000 7f7635000700 1 Processor -- start 2024-03-24T00:55:18.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.682+0000 7f7635000700 1 -- start start 2024-03-24T00:55:18.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.682+0000 7f7635000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76300ffa00 0x7f763019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:18.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.683+0000 7f7635000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7630100320 0x7f763019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:18.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.683+0000 7f7635000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763019dc60 con 0x7f7630100320 2024-03-24T00:55:18.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.683+0000 7f7635000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76301a1930 con 0x7f76300ffa00 2024-03-24T00:55:18.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.683+0000 7f762d400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7630100320 0x7f763019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:18.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.683+0000 7f762d400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7630100320 0x7f763019d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42900/0 (socket says 172.21.15.38:42900) 2024-03-24T00:55:18.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.683+0000 7f762d400700 1 -- 172.21.15.38:0/3826338257 learned_addr learned my addr 172.21.15.38:0/3826338257 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:18.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.683+0000 7f762de00700 1 --2- 172.21.15.38:0/3826338257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76300ffa00 0x7f763019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:18.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.683+0000 7f762d400700 1 -- 172.21.15.38:0/3826338257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76300ffa00 msgr2=0x7f763019d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:18.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.683+0000 7f762d400700 1 --2- 172.21.15.38:0/3826338257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76300ffa00 0x7f763019d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.683+0000 7f762d400700 1 -- 172.21.15.38:0/3826338257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7620009d00 con 0x7f7630100320 2024-03-24T00:55:18.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.684+0000 7f762d400700 1 --2- 172.21.15.38:0/3826338257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7630100320 0x7f763019d580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f761c00e940 tx=0x7f761c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:18.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.684+0000 7f7626a00700 1 -- 172.21.15.38:0/3826338257 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f761c0096e0 con 0x7f7630100320 2024-03-24T00:55:18.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.684+0000 7f7635000700 1 -- 172.21.15.38:0/3826338257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76301a1c10 con 0x7f7630100320 2024-03-24T00:55:18.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.684+0000 7f7626a00700 1 -- 172.21.15.38:0/3826338257 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f761c00c7c0 con 0x7f7630100320 2024-03-24T00:55:18.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.684+0000 7f7626a00700 1 -- 172.21.15.38:0/3826338257 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f761c005580 con 0x7f7630100320 2024-03-24T00:55:18.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.684+0000 7f7635000700 1 -- 172.21.15.38:0/3826338257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76301a2160 con 0x7f7630100320 2024-03-24T00:55:18.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.685+0000 7f7635000700 1 -- 172.21.15.38:0/3826338257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7630068d60 con 0x7f7630100320 2024-03-24T00:55:18.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.686+0000 7f7626a00700 1 -- 172.21.15.38:0/3826338257 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f761c00c930 con 0x7f7630100320 2024-03-24T00:55:18.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.687+0000 7f7626a00700 1 --2- 172.21.15.38:0/3826338257 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f761406e0e0 0x7f76140705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:18.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.687+0000 7f7626a00700 1 -- 172.21.15.38:0/3826338257 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f761c014070 con 0x7f7630100320 2024-03-24T00:55:18.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.687+0000 7f762de00700 1 --2- 172.21.15.38:0/3826338257 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f761406e0e0 0x7f76140705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:18.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.688+0000 7f762de00700 1 --2- 172.21.15.38:0/3826338257 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f761406e0e0 0x7f76140705a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7620000f80 tx=0x7f76200060c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:18.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.691+0000 7f7626a00700 1 -- 172.21.15.38:0/3826338257 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f761c057400 con 0x7f7630100320 2024-03-24T00:55:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.960+0000 7f7635000700 1 -- 172.21.15.38:0/3826338257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f76301a2480 con 0x7f7630100320 2024-03-24T00:55:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.961+0000 7f7626a00700 1 -- 172.21.15.38:0/3826338257 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1900 (secure 0 0 0) 0x7f761c05aac0 con 0x7f7630100320 2024-03-24T00:55:18.962 INFO:teuthology.orchestra.run.smithi038.stdout:e20 2024-03-24T00:55:18.962 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:55:18.962 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:55:18.962 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:55:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:55:18.962 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:55:18.962 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:55:18.962 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 20 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:55:18.963 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:55:18.964 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:55:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:55:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:55:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.963+0000 7f7635000700 1 -- 172.21.15.38:0/3826338257 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f761406e0e0 msgr2=0x7f76140705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.963+0000 7f7635000700 1 --2- 172.21.15.38:0/3826338257 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f761406e0e0 0x7f76140705a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7620000f80 tx=0x7f76200060c0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.963+0000 7f7635000700 1 -- 172.21.15.38:0/3826338257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7630100320 msgr2=0x7f763019d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.963+0000 7f7635000700 1 --2- 172.21.15.38:0/3826338257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7630100320 0x7f763019d580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f761c00e940 tx=0x7f761c00ec50 comp rx=0 tx=0).stop 2024-03-24T00:55:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.964+0000 7f7635000700 1 -- 172.21.15.38:0/3826338257 shutdown_connections 2024-03-24T00:55:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.964+0000 7f7635000700 1 --2- 172.21.15.38:0/3826338257 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f761406e0e0 0x7f76140705a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.964+0000 7f7635000700 1 --2- 172.21.15.38:0/3826338257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76300ffa00 0x7f763019d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.964+0000 7f7635000700 1 --2- 172.21.15.38:0/3826338257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7630100320 0x7f763019d580 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.964+0000 7f7635000700 1 -- 172.21.15.38:0/3826338257 >> 172.21.15.38:0/3826338257 conn(0x7f7630076640 msgr2=0x7f763010e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.964+0000 7f7635000700 1 -- 172.21.15.38:0/3826338257 shutdown_connections 2024-03-24T00:55:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:18.964+0000 7f7635000700 1 -- 172.21.15.38:0/3826338257 wait complete. 2024-03-24T00:55:18.967 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:55:19.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.122+0000 7f2650c00700 1 -- 172.21.15.38:0/1183802008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f264c107310 msgr2=0x7f264c1076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:19.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.122+0000 7f2650c00700 1 --2- 172.21.15.38:0/1183802008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f264c107310 0x7f264c1076f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f263c009910 tx=0x7f263c009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:19.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.122+0000 7f2650c00700 1 -- 172.21.15.38:0/1183802008 shutdown_connections 2024-03-24T00:55:19.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.122+0000 7f2650c00700 1 --2- 172.21.15.38:0/1183802008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f264c107cc0 0x7f264c10bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.122+0000 7f2650c00700 1 --2- 172.21.15.38:0/1183802008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f264c107310 0x7f264c1076f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.122+0000 7f2650c00700 1 -- 172.21.15.38:0/1183802008 >> 172.21.15.38:0/1183802008 conn(0x7f264c0768e0 msgr2=0x7f264c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:19.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.122+0000 7f2650c00700 1 -- 172.21.15.38:0/1183802008 shutdown_connections 2024-03-24T00:55:19.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.122+0000 7f2650c00700 1 -- 172.21.15.38:0/1183802008 wait complete. 2024-03-24T00:55:19.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.123+0000 7f2650c00700 1 Processor -- start 2024-03-24T00:55:19.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.123+0000 7f2650c00700 1 -- start start 2024-03-24T00:55:19.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.124+0000 7f2650c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f264c107310 0x7f264c19d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:19.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.124+0000 7f2650c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f264c107cc0 0x7f264c19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:19.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.124+0000 7f2650c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c19dc90 con 0x7f264c107cc0 2024-03-24T00:55:19.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.124+0000 7f2650c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1a18a0 con 0x7f264c107310 2024-03-24T00:55:19.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.124+0000 7f2649400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f264c107cc0 0x7f264c19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:19.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.124+0000 7f2649400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f264c107cc0 0x7f264c19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42920/0 (socket says 172.21.15.38:42920) 2024-03-24T00:55:19.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.124+0000 7f2649400700 1 -- 172.21.15.38:0/4014762360 learned_addr learned my addr 172.21.15.38:0/4014762360 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:19.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.124+0000 7f2649e00700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f264c107310 0x7f264c19d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:19.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.124+0000 7f2649400700 1 -- 172.21.15.38:0/4014762360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f264c107310 msgr2=0x7f264c19d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:19.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.124+0000 7f2649400700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f264c107310 0x7f264c19d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.124+0000 7f2649400700 1 -- 172.21.15.38:0/4014762360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c0095f0 con 0x7f264c107cc0 2024-03-24T00:55:19.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.125+0000 7f2649e00700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f264c107310 0x7f264c19d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:55:19.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.125+0000 7f2649400700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f264c107cc0 0x7f264c19d5b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f263800c8f0 tx=0x7f263800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:19.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.125+0000 7f2642a00700 1 -- 172.21.15.38:0/4014762360 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2638007a50 con 0x7f264c107cc0 2024-03-24T00:55:19.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.125+0000 7f2642a00700 1 -- 172.21.15.38:0/4014762360 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2638007bb0 con 0x7f264c107cc0 2024-03-24T00:55:19.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.125+0000 7f2650c00700 1 -- 172.21.15.38:0/4014762360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264c1a1b80 con 0x7f264c107cc0 2024-03-24T00:55:19.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.125+0000 7f2642a00700 1 -- 172.21.15.38:0/4014762360 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2638018630 con 0x7f264c107cc0 2024-03-24T00:55:19.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.125+0000 7f2650c00700 1 -- 172.21.15.38:0/4014762360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c1a20d0 con 0x7f264c107cc0 2024-03-24T00:55:19.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.127+0000 7f2650c00700 1 -- 172.21.15.38:0/4014762360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264c10f6b0 con 0x7f264c107cc0 2024-03-24T00:55:19.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.128+0000 7f2642a00700 1 -- 172.21.15.38:0/4014762360 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f2638007d20 con 0x7f264c107cc0 2024-03-24T00:55:19.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.129+0000 7f2642a00700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f263006e0d0 0x7f2630070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:19.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.129+0000 7f2642a00700 1 -- 172.21.15.38:0/4014762360 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f263808df30 con 0x7f264c107cc0 2024-03-24T00:55:19.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.129+0000 7f2649e00700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f263006e0d0 0x7f2630070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:19.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.130+0000 7f2649e00700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f263006e0d0 0x7f2630070590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f263c009fd0 tx=0x7f263c00b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.133+0000 7f2642a00700 1 -- 172.21.15.38:0/4014762360 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f26380c2740 con 0x7f264c107cc0 2024-03-24T00:55:19.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:19 smithi038 ceph-mon[112108]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:19.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:19 smithi038 ceph-mon[112108]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:19.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:19 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:19.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:19 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:19.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:19 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.yhksbr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:55:19.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:19 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:19.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:19 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1527506992' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:19.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:19 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3826338257' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:55:19.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:19 smithi088 ceph-mon[87351]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:19.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:19 smithi088 ceph-mon[87351]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:19.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:19 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:19.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:19 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:19.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:19 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.yhksbr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:55:19.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:19 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:19.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:19 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1527506992' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:19.361 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:19 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3826338257' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:55:19.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.379+0000 7f2650c00700 1 -- 172.21.15.38:0/4014762360 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f264c19e330 con 0x7f263006e0d0 2024-03-24T00:55:19.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.381+0000 7f2642a00700 1 -- 172.21.15.38:0/4014762360 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f264c19e330 con 0x7f263006e0d0 2024-03-24T00:55:19.382 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:55:19.382 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:55:19.382 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:55:19.382 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:55:19.383 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:55:19.383 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T00:55:19.383 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-03-24T00:55:19.383 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:55:19.383 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "4/23 daemons upgraded", 2024-03-24T00:55:19.383 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mon daemons", 2024-03-24T00:55:19.383 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:55:19.383 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:55:19.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.383+0000 7f2650c00700 1 -- 172.21.15.38:0/4014762360 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f263006e0d0 msgr2=0x7f2630070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:19.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.383+0000 7f2650c00700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f263006e0d0 0x7f2630070590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f263c009fd0 tx=0x7f263c00b560 comp rx=0 tx=0).stop 2024-03-24T00:55:19.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.383+0000 7f2650c00700 1 -- 172.21.15.38:0/4014762360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f264c107cc0 msgr2=0x7f264c19d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:19.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.383+0000 7f2650c00700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f264c107cc0 0x7f264c19d5b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f263800c8f0 tx=0x7f263800ccb0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.384+0000 7f2650c00700 1 -- 172.21.15.38:0/4014762360 shutdown_connections 2024-03-24T00:55:19.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.384+0000 7f2650c00700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f263006e0d0 0x7f2630070590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.384+0000 7f2650c00700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f264c107310 0x7f264c19d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.384+0000 7f2650c00700 1 --2- 172.21.15.38:0/4014762360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f264c107cc0 0x7f264c19d5b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.384+0000 7f2650c00700 1 -- 172.21.15.38:0/4014762360 >> 172.21.15.38:0/4014762360 conn(0x7f264c0768e0 msgr2=0x7f264c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:19.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.384+0000 7f2650c00700 1 -- 172.21.15.38:0/4014762360 shutdown_connections 2024-03-24T00:55:19.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.384+0000 7f2650c00700 1 -- 172.21.15.38:0/4014762360 wait complete. 2024-03-24T00:55:19.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.535+0000 7f1888000700 1 -- 172.21.15.38:0/4204214210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880101c10 msgr2=0x7f1880101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:19.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.535+0000 7f1888000700 1 --2- 172.21.15.38:0/4204214210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880101c10 0x7f1880101ff0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f187400b0a0 tx=0x7f187400b3b0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.535+0000 7f1888000700 1 -- 172.21.15.38:0/4204214210 shutdown_connections 2024-03-24T00:55:19.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.535+0000 7f1888000700 1 --2- 172.21.15.38:0/4204214210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1880102530 0x7f188010aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.535+0000 7f1888000700 1 --2- 172.21.15.38:0/4204214210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880101c10 0x7f1880101ff0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.535+0000 7f1888000700 1 -- 172.21.15.38:0/4204214210 >> 172.21.15.38:0/4204214210 conn(0x7f1880076640 msgr2=0x7f18800fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:19.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.535+0000 7f1888000700 1 -- 172.21.15.38:0/4204214210 shutdown_connections 2024-03-24T00:55:19.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.535+0000 7f1888000700 1 -- 172.21.15.38:0/4204214210 wait complete. 2024-03-24T00:55:19.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.536+0000 7f1888000700 1 Processor -- start 2024-03-24T00:55:19.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.536+0000 7f1888000700 1 -- start start 2024-03-24T00:55:19.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.536+0000 7f1888000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1880101c10 0x7f188019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:19.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.536+0000 7f1888000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880102530 0x7f188019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:19.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.537+0000 7f1888000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188019d2e0 con 0x7f1880102530 2024-03-24T00:55:19.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.537+0000 7f1888000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1880196850 con 0x7f1880101c10 2024-03-24T00:55:19.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.537+0000 7f1884c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880102530 0x7f188019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:19.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.537+0000 7f1884c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880102530 0x7f188019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42936/0 (socket says 172.21.15.38:42936) 2024-03-24T00:55:19.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.537+0000 7f1884c00700 1 -- 172.21.15.38:0/3081101805 learned_addr learned my addr 172.21.15.38:0/3081101805 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:19.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.537+0000 7f1885600700 1 --2- 172.21.15.38:0/3081101805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1880101c10 0x7f188019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:19.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.537+0000 7f1884c00700 1 -- 172.21.15.38:0/3081101805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1880101c10 msgr2=0x7f188019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:19.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.537+0000 7f1884c00700 1 --2- 172.21.15.38:0/3081101805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1880101c10 0x7f188019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.537+0000 7f1884c00700 1 -- 172.21.15.38:0/3081101805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1874009d00 con 0x7f1880102530 2024-03-24T00:55:19.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.538+0000 7f1884c00700 1 --2- 172.21.15.38:0/3081101805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880102530 0x7f188019cc50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f186c00d6d0 tx=0x7f186c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:19.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.538+0000 7f187e000700 1 -- 172.21.15.38:0/3081101805 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f186c00f860 con 0x7f1880102530 2024-03-24T00:55:19.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.538+0000 7f187e000700 1 -- 172.21.15.38:0/3081101805 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f186c00fe80 con 0x7f1880102530 2024-03-24T00:55:19.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.538+0000 7f187e000700 1 -- 172.21.15.38:0/3081101805 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f186c010ea0 con 0x7f1880102530 2024-03-24T00:55:19.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.538+0000 7f1888000700 1 -- 172.21.15.38:0/3081101805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1880196b30 con 0x7f1880102530 2024-03-24T00:55:19.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.538+0000 7f1888000700 1 -- 172.21.15.38:0/3081101805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1880197080 con 0x7f1880102530 2024-03-24T00:55:19.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.539+0000 7f1888000700 1 -- 172.21.15.38:0/3081101805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1880105330 con 0x7f1880102530 2024-03-24T00:55:19.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.540+0000 7f187e000700 1 -- 172.21.15.38:0/3081101805 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 38) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f186c010460 con 0x7f1880102530 2024-03-24T00:55:19.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.541+0000 7f187e000700 1 --2- 172.21.15.38:0/3081101805 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f186806e0d0 0x7f1868070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:19.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.541+0000 7f187e000700 1 -- 172.21.15.38:0/3081101805 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f186c08da80 con 0x7f1880102530 2024-03-24T00:55:19.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.541+0000 7f1885600700 1 --2- 172.21.15.38:0/3081101805 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f186806e0d0 0x7f1868070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:19.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.542+0000 7f1885600700 1 --2- 172.21.15.38:0/3081101805 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f186806e0d0 0x7f1868070590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1874009480 tx=0x7f1874000f40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:19.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.546+0000 7f187e000700 1 -- 172.21.15.38:0/3081101805 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f186c05a8e0 con 0x7f1880102530 2024-03-24T00:55:19.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.850+0000 7f1888000700 1 -- 172.21.15.38:0/3081101805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1880068d60 con 0x7f1880102530 2024-03-24T00:55:19.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.851+0000 7f187e000700 1 -- 172.21.15.38:0/3081101805 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f186c020090 con 0x7f1880102530 2024-03-24T00:55:19.852 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_OK 2024-03-24T00:55:19.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.854+0000 7f1888000700 1 -- 172.21.15.38:0/3081101805 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f186806e0d0 msgr2=0x7f1868070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:19.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.854+0000 7f1888000700 1 --2- 172.21.15.38:0/3081101805 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f186806e0d0 0x7f1868070590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1874009480 tx=0x7f1874000f40 comp rx=0 tx=0).stop 2024-03-24T00:55:19.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.854+0000 7f1888000700 1 -- 172.21.15.38:0/3081101805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880102530 msgr2=0x7f188019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:19.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.854+0000 7f1888000700 1 --2- 172.21.15.38:0/3081101805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880102530 0x7f188019cc50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f186c00d6d0 tx=0x7f186c00da90 comp rx=0 tx=0).stop 2024-03-24T00:55:19.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.854+0000 7f1888000700 1 -- 172.21.15.38:0/3081101805 shutdown_connections 2024-03-24T00:55:19.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.854+0000 7f1888000700 1 --2- 172.21.15.38:0/3081101805 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f186806e0d0 0x7f1868070590 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.854+0000 7f1888000700 1 --2- 172.21.15.38:0/3081101805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1880101c10 0x7f188019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.854+0000 7f1888000700 1 --2- 172.21.15.38:0/3081101805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1880102530 0x7f188019cc50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:19.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.854+0000 7f1888000700 1 -- 172.21.15.38:0/3081101805 >> 172.21.15.38:0/3081101805 conn(0x7f1880076640 msgr2=0x7f1880102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:19.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.854+0000 7f1888000700 1 -- 172.21.15.38:0/3081101805 shutdown_connections 2024-03-24T00:55:19.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:19.854+0000 7f1888000700 1 -- 172.21.15.38:0/3081101805 wait complete. 2024-03-24T00:55:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:20 smithi038 ceph-mon[112108]: Reconfiguring mds.cephfs.smithi088.yhksbr (monmap changed)... 2024-03-24T00:55:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:20 smithi038 ceph-mon[112108]: Reconfiguring daemon mds.cephfs.smithi088.yhksbr on smithi088 2024-03-24T00:55:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:20 smithi038 ceph-mon[112108]: pgmap v44: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:20 smithi038 ceph-mon[112108]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:20.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:20 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3081101805' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:55:20.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:20 smithi088 ceph-mon[87351]: Reconfiguring mds.cephfs.smithi088.yhksbr (monmap changed)... 2024-03-24T00:55:20.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:20 smithi088 ceph-mon[87351]: Reconfiguring daemon mds.cephfs.smithi088.yhksbr on smithi088 2024-03-24T00:55:20.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:20 smithi088 ceph-mon[87351]: pgmap v44: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:20.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:20 smithi088 ceph-mon[87351]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:20.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:20 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3081101805' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:55:21.908 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: pgmap v45: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:21.909 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:21.909 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:21.909 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:55:21.909 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:21.909 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:21.909 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: Upgrade: Setting container_image for all mon 2024-03-24T00:55:21.909 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:21.909 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi038"}]: dispatch 2024-03-24T00:55:21.909 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi038"}]': finished 2024-03-24T00:55:21.910 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi088"}]: dispatch 2024-03-24T00:55:21.910 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi088"}]': finished 2024-03-24T00:55:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: pgmap v45: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:55:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: Upgrade: Setting container_image for all mon 2024-03-24T00:55:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi038"}]: dispatch 2024-03-24T00:55:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi038"}]': finished 2024-03-24T00:55:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi088"}]: dispatch 2024-03-24T00:55:21.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi088"}]': finished 2024-03-24T00:55:24.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:24 smithi088 ceph-mon[87351]: pgmap v46: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:55:24.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:24 smithi088 ceph-mon[87351]: Upgrade: Updating crash.smithi038 (1/2) 2024-03-24T00:55:24.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:24 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:24.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:24 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:55:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:24 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:24 smithi088 ceph-mon[87351]: Deploying daemon crash.smithi038 on smithi038 2024-03-24T00:55:24.468 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:24 smithi038 ceph-mon[112108]: pgmap v46: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T00:55:24.468 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:24 smithi038 ceph-mon[112108]: Upgrade: Updating crash.smithi038 (1/2) 2024-03-24T00:55:24.468 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:24 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:24.468 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:24 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:55:24.468 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:24 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:24.468 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:24 smithi038 ceph-mon[112108]: Deploying daemon crash.smithi038 on smithi038 2024-03-24T00:55:26.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:26 smithi088 ceph-mon[87351]: pgmap v47: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:55:26.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:26 smithi038 ceph-mon[112108]: pgmap v47: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:55:27.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:55:27.454 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:55:28.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:28 smithi088 ceph-mon[87351]: pgmap v48: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:28.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:28 smithi038 ceph-mon[112108]: pgmap v48: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:29 smithi088 ceph-mon[87351]: pgmap v49: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-24T00:55:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:29 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:29 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:29 smithi088 ceph-mon[87351]: mgrmap e39: smithi038.huhioz(active, since 92s), standbys: smithi088.njctea 2024-03-24T00:55:30.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:29 smithi038 ceph-mon[112108]: pgmap v49: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-24T00:55:30.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:29 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:30.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:29 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:30.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:29 smithi038 ceph-mon[112108]: mgrmap e39: smithi038.huhioz(active, since 92s), standbys: smithi088.njctea 2024-03-24T00:55:32.395 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:32 smithi088 ceph-mon[87351]: pgmap v50: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T00:55:32.395 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:32 smithi088 ceph-mon[87351]: Upgrade: Updating crash.smithi088 (2/2) 2024-03-24T00:55:32.396 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:32 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:32.396 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:32 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:55:32.396 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:32 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:32.396 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:32 smithi088 ceph-mon[87351]: Deploying daemon crash.smithi088 on smithi088 2024-03-24T00:55:32.487 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:32 smithi038 ceph-mon[112108]: pgmap v50: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T00:55:32.487 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:32 smithi038 ceph-mon[112108]: Upgrade: Updating crash.smithi088 (2/2) 2024-03-24T00:55:32.487 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:32 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:32.487 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:32 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T00:55:32.487 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:32 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:32.487 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:32 smithi038 ceph-mon[112108]: Deploying daemon crash.smithi088 on smithi088 2024-03-24T00:55:34.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:34 smithi088 ceph-mon[87351]: pgmap v51: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-24T00:55:34.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:34 smithi038 ceph-mon[112108]: pgmap v51: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-24T00:55:36.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:36 smithi088 ceph-mon[87351]: pgmap v52: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-24T00:55:36.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:36 smithi038 ceph-mon[112108]: pgmap v52: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-24T00:55:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:37 smithi038 ceph-mon[112108]: pgmap v53: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T00:55:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:38.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:55:38.169 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:37 smithi088 ceph-mon[87351]: pgmap v53: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T00:55:38.170 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:38.170 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:38.170 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:55:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:39 smithi038 ceph-mon[112108]: pgmap v54: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-24T00:55:40.174 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:39 smithi088 ceph-mon[87351]: pgmap v54: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-24T00:55:42.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:42 smithi088 ceph-mon[87351]: pgmap v55: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:42.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:42.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:42.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:55:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:42 smithi038 ceph-mon[112108]: pgmap v55: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:55:44.605 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:44 smithi088 ceph-mon[87351]: pgmap v56: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:44.605 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:44 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:44.605 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:44 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:44.606 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:44 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:44.606 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:44 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:44 smithi038 ceph-mon[112108]: pgmap v56: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:44 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:44 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:44 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:44.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:44 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:46.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:46 smithi038 ceph-mon[112108]: pgmap v57: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:55:46.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:46 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:46.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:46 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:46 smithi088 ceph-mon[87351]: pgmap v57: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:55:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:46 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:46.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:46 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:48.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:48 smithi038 ceph-mon[112108]: pgmap v58: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:48 smithi088 ceph-mon[87351]: pgmap v58: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:50.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.038+0000 7f7d6e800700 1 -- 172.21.15.38:0/3261838032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d68102550 msgr2=0x7f7d6810aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:50.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.038+0000 7f7d6e800700 1 --2- 172.21.15.38:0/3261838032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d68102550 0x7f7d6810aa40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009910 tx=0x7f7d54009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.038+0000 7f7d6e800700 1 -- 172.21.15.38:0/3261838032 shutdown_connections 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.038+0000 7f7d6e800700 1 --2- 172.21.15.38:0/3261838032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d68102550 0x7f7d6810aa40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.038+0000 7f7d6e800700 1 --2- 172.21.15.38:0/3261838032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d68101ba0 0x7f7d68101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.038+0000 7f7d6e800700 1 -- 172.21.15.38:0/3261838032 >> 172.21.15.38:0/3261838032 conn(0x7f7d680fb480 msgr2=0x7f7d680fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.038+0000 7f7d6e800700 1 -- 172.21.15.38:0/3261838032 shutdown_connections 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.038+0000 7f7d6e800700 1 -- 172.21.15.38:0/3261838032 wait complete. 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.039+0000 7f7d6e800700 1 Processor -- start 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.039+0000 7f7d6e800700 1 -- start start 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.040+0000 7f7d6e800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d68101ba0 0x7f7d6819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.040+0000 7f7d6e800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d68102550 0x7f7d6819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.040+0000 7f7d6e800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6819f510 con 0x7f7d68101ba0 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.040+0000 7f7d6e800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d68198a80 con 0x7f7d68102550 2024-03-24T00:55:50.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.040+0000 7f7d67400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d68101ba0 0x7f7d6819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:50.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.040+0000 7f7d67400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d68101ba0 0x7f7d6819e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37390/0 (socket says 172.21.15.38:37390) 2024-03-24T00:55:50.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.040+0000 7f7d67400700 1 -- 172.21.15.38:0/1730258843 learned_addr learned my addr 172.21.15.38:0/1730258843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:50.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.041+0000 7f7d66a00700 1 --2- 172.21.15.38:0/1730258843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d68102550 0x7f7d6819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:50.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.041+0000 7f7d67400700 1 -- 172.21.15.38:0/1730258843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d68102550 msgr2=0x7f7d6819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:50.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.041+0000 7f7d67400700 1 --2- 172.21.15.38:0/1730258843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d68102550 0x7f7d6819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.041+0000 7f7d67400700 1 -- 172.21.15.38:0/1730258843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540095f0 con 0x7f7d68101ba0 2024-03-24T00:55:50.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.041+0000 7f7d67400700 1 --2- 172.21.15.38:0/1730258843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d68101ba0 0x7f7d6819e940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800d8d0 tx=0x7f7d5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:50.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.041+0000 7f7d5fe00700 1 -- 172.21.15.38:0/1730258843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d58009940 con 0x7f7d68101ba0 2024-03-24T00:55:50.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.041+0000 7f7d5fe00700 1 -- 172.21.15.38:0/1730258843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d58010460 con 0x7f7d68101ba0 2024-03-24T00:55:50.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.041+0000 7f7d6e800700 1 -- 172.21.15.38:0/1730258843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d68198d60 con 0x7f7d68101ba0 2024-03-24T00:55:50.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.041+0000 7f7d5fe00700 1 -- 172.21.15.38:0/1730258843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d5800f5f0 con 0x7f7d68101ba0 2024-03-24T00:55:50.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.042+0000 7f7d6e800700 1 -- 172.21.15.38:0/1730258843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d681992b0 con 0x7f7d68101ba0 2024-03-24T00:55:50.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.043+0000 7f7d6e800700 1 -- 172.21.15.38:0/1730258843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6810a5a0 con 0x7f7d68101ba0 2024-03-24T00:55:50.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.044+0000 7f7d5fe00700 1 -- 172.21.15.38:0/1730258843 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f7d580105d0 con 0x7f7d68101ba0 2024-03-24T00:55:50.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.045+0000 7f7d5fe00700 1 --2- 172.21.15.38:0/1730258843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d4c06dfb0 0x7f7d4c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:50.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.045+0000 7f7d5fe00700 1 -- 172.21.15.38:0/1730258843 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7d5808ddc0 con 0x7f7d68101ba0 2024-03-24T00:55:50.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.045+0000 7f7d66a00700 1 --2- 172.21.15.38:0/1730258843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d4c06dfb0 0x7f7d4c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:50.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.046+0000 7f7d66a00700 1 --2- 172.21.15.38:0/1730258843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d4c06dfb0 0x7f7d4c070470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006010 tx=0x7f7d5400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.051+0000 7f7d5fe00700 1 -- 172.21.15.38:0/1730258843 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7d58091030 con 0x7f7d68101ba0 2024-03-24T00:55:50.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:49 smithi038 ceph-mon[112108]: pgmap v59: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:50.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:49 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:50.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:49 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:50.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:49 smithi088 ceph-mon[87351]: pgmap v59: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:50.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:49 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:50.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:49 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:50.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.435+0000 7f7d6e800700 1 -- 172.21.15.38:0/1730258843 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d68199f40 con 0x7f7d4c06dfb0 2024-03-24T00:55:50.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.437+0000 7f7d5fe00700 1 -- 172.21.15.38:0/1730258843 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f7d68199f40 con 0x7f7d4c06dfb0 2024-03-24T00:55:50.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.439+0000 7f7d6e800700 1 -- 172.21.15.38:0/1730258843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d4c06dfb0 msgr2=0x7f7d4c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.439+0000 7f7d6e800700 1 --2- 172.21.15.38:0/1730258843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d4c06dfb0 0x7f7d4c070470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006010 tx=0x7f7d5400b540 comp rx=0 tx=0).stop 2024-03-24T00:55:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.439+0000 7f7d6e800700 1 -- 172.21.15.38:0/1730258843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d68101ba0 msgr2=0x7f7d6819e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.439+0000 7f7d6e800700 1 --2- 172.21.15.38:0/1730258843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d68101ba0 0x7f7d6819e940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800d8d0 tx=0x7f7d5800dc90 comp rx=0 tx=0).stop 2024-03-24T00:55:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.440+0000 7f7d6e800700 1 -- 172.21.15.38:0/1730258843 shutdown_connections 2024-03-24T00:55:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.440+0000 7f7d6e800700 1 --2- 172.21.15.38:0/1730258843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d4c06dfb0 0x7f7d4c070470 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.440+0000 7f7d6e800700 1 --2- 172.21.15.38:0/1730258843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d68101ba0 0x7f7d6819e940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.440+0000 7f7d6e800700 1 --2- 172.21.15.38:0/1730258843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d68102550 0x7f7d6819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.440+0000 7f7d6e800700 1 -- 172.21.15.38:0/1730258843 >> 172.21.15.38:0/1730258843 conn(0x7f7d680fb480 msgr2=0x7f7d68100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:50.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.440+0000 7f7d6e800700 1 -- 172.21.15.38:0/1730258843 shutdown_connections 2024-03-24T00:55:50.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.440+0000 7f7d6e800700 1 -- 172.21.15.38:0/1730258843 wait complete. 2024-03-24T00:55:50.456 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:55:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.607+0000 7f508c000700 1 -- 172.21.15.38:0/1720855299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5084100ab0 msgr2=0x7f5084100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.607+0000 7f508c000700 1 --2- 172.21.15.38:0/1720855299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5084100ab0 0x7f5084100f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5070009910 tx=0x7f5070009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.607+0000 7f508c000700 1 -- 172.21.15.38:0/1720855299 shutdown_connections 2024-03-24T00:55:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.607+0000 7f508c000700 1 --2- 172.21.15.38:0/1720855299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5084100ab0 0x7f5084100f30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.607+0000 7f508c000700 1 --2- 172.21.15.38:0/1720855299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5084106ab0 0x7f5084106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.607+0000 7f508c000700 1 -- 172.21.15.38:0/1720855299 >> 172.21.15.38:0/1720855299 conn(0x7f50840fc5a0 msgr2=0x7f50840fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.607+0000 7f508c000700 1 -- 172.21.15.38:0/1720855299 shutdown_connections 2024-03-24T00:55:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.607+0000 7f508c000700 1 -- 172.21.15.38:0/1720855299 wait complete. 2024-03-24T00:55:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.609+0000 7f508c000700 1 Processor -- start 2024-03-24T00:55:50.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.609+0000 7f508c000700 1 -- start start 2024-03-24T00:55:50.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.609+0000 7f508c000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5084100ab0 0x7f508419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:50.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.609+0000 7f508c000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5084106ab0 0x7f508419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:50.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.609+0000 7f508c000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508419f500 con 0x7f5084106ab0 2024-03-24T00:55:50.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.609+0000 7f508c000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5084198a20 con 0x7f5084100ab0 2024-03-24T00:55:50.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.610+0000 7f5088c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5084106ab0 0x7f508419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:50.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.610+0000 7f5088c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5084106ab0 0x7f508419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37422/0 (socket says 172.21.15.38:37422) 2024-03-24T00:55:50.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.610+0000 7f5088c00700 1 -- 172.21.15.38:0/2806009678 learned_addr learned my addr 172.21.15.38:0/2806009678 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:50.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.610+0000 7f5089600700 1 --2- 172.21.15.38:0/2806009678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5084100ab0 0x7f508419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:50.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.610+0000 7f5088c00700 1 -- 172.21.15.38:0/2806009678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5084100ab0 msgr2=0x7f508419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:50.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.610+0000 7f5088c00700 1 --2- 172.21.15.38:0/2806009678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5084100ab0 0x7f508419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.610+0000 7f5088c00700 1 -- 172.21.15.38:0/2806009678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700095f0 con 0x7f5084106ab0 2024-03-24T00:55:50.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.610+0000 7f5088c00700 1 --2- 172.21.15.38:0/2806009678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5084106ab0 0x7f508419eeb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5070006010 tx=0x7f507000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:50.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.611+0000 7f5082000700 1 -- 172.21.15.38:0/2806009678 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f507001d070 con 0x7f5084106ab0 2024-03-24T00:55:50.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.611+0000 7f5082000700 1 -- 172.21.15.38:0/2806009678 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f507000f460 con 0x7f5084106ab0 2024-03-24T00:55:50.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.611+0000 7f5082000700 1 -- 172.21.15.38:0/2806009678 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5070017630 con 0x7f5084106ab0 2024-03-24T00:55:50.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.611+0000 7f508c000700 1 -- 172.21.15.38:0/2806009678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5084198ca0 con 0x7f5084106ab0 2024-03-24T00:55:50.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.611+0000 7f508c000700 1 -- 172.21.15.38:0/2806009678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5084199190 con 0x7f5084106ab0 2024-03-24T00:55:50.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.612+0000 7f508c000700 1 -- 172.21.15.38:0/2806009678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5084104d60 con 0x7f5084106ab0 2024-03-24T00:55:50.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.613+0000 7f5082000700 1 -- 172.21.15.38:0/2806009678 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f507000f5d0 con 0x7f5084106ab0 2024-03-24T00:55:50.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.614+0000 7f5082000700 1 --2- 172.21.15.38:0/2806009678 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f506c06dfb0 0x7f506c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:50.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.614+0000 7f5082000700 1 -- 172.21.15.38:0/2806009678 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f507008f210 con 0x7f5084106ab0 2024-03-24T00:55:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.615+0000 7f5089600700 1 --2- 172.21.15.38:0/2806009678 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f506c06dfb0 0x7f506c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.615+0000 7f5089600700 1 --2- 172.21.15.38:0/2806009678 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f506c06dfb0 0x7f506c070470 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5078005ea0 tx=0x7f5078005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:50.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.619+0000 7f5082000700 1 -- 172.21.15.38:0/2806009678 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5070058780 con 0x7f5084106ab0 2024-03-24T00:55:50.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.879+0000 7f508c000700 1 -- 172.21.15.38:0/2806009678 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5084199f00 con 0x7f506c06dfb0 2024-03-24T00:55:50.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.881+0000 7f5082000700 1 -- 172.21.15.38:0/2806009678 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f5084199f00 con 0x7f506c06dfb0 2024-03-24T00:55:50.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.884+0000 7f508c000700 1 -- 172.21.15.38:0/2806009678 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f506c06dfb0 msgr2=0x7f506c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.884+0000 7f508c000700 1 --2- 172.21.15.38:0/2806009678 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f506c06dfb0 0x7f506c070470 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5078005ea0 tx=0x7f5078005e30 comp rx=0 tx=0).stop 2024-03-24T00:55:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.884+0000 7f508c000700 1 -- 172.21.15.38:0/2806009678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5084106ab0 msgr2=0x7f508419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.884+0000 7f508c000700 1 --2- 172.21.15.38:0/2806009678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5084106ab0 0x7f508419eeb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5070006010 tx=0x7f507000be20 comp rx=0 tx=0).stop 2024-03-24T00:55:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.884+0000 7f508c000700 1 -- 172.21.15.38:0/2806009678 shutdown_connections 2024-03-24T00:55:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.884+0000 7f508c000700 1 --2- 172.21.15.38:0/2806009678 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f506c06dfb0 0x7f506c070470 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.884+0000 7f508c000700 1 --2- 172.21.15.38:0/2806009678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5084100ab0 0x7f508419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.884+0000 7f508c000700 1 --2- 172.21.15.38:0/2806009678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5084106ab0 0x7f508419eeb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.884+0000 7f508c000700 1 -- 172.21.15.38:0/2806009678 >> 172.21.15.38:0/2806009678 conn(0x7f50840fc5a0 msgr2=0x7f50840732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.885+0000 7f508c000700 1 -- 172.21.15.38:0/2806009678 shutdown_connections 2024-03-24T00:55:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:50.885+0000 7f508c000700 1 -- 172.21.15.38:0/2806009678 wait complete. 2024-03-24T00:55:51.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.043+0000 7fed99800700 1 -- 172.21.15.38:0/3246452852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed94073180 msgr2=0x7fed94073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:51.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.043+0000 7fed99800700 1 --2- 172.21.15.38:0/3246452852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed94073180 0x7fed94073560 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fed84009910 tx=0x7fed84009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:51.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.043+0000 7fed99800700 1 -- 172.21.15.38:0/3246452852 shutdown_connections 2024-03-24T00:55:51.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.043+0000 7fed99800700 1 --2- 172.21.15.38:0/3246452852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed94073aa0 0x7fed9410cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.043+0000 7fed99800700 1 --2- 172.21.15.38:0/3246452852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed94073180 0x7fed94073560 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.043+0000 7fed99800700 1 -- 172.21.15.38:0/3246452852 >> 172.21.15.38:0/3246452852 conn(0x7fed940fc5a0 msgr2=0x7fed940fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:51.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.044+0000 7fed99800700 1 -- 172.21.15.38:0/3246452852 shutdown_connections 2024-03-24T00:55:51.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.044+0000 7fed99800700 1 -- 172.21.15.38:0/3246452852 wait complete. 2024-03-24T00:55:51.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.045+0000 7fed99800700 1 Processor -- start 2024-03-24T00:55:51.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.045+0000 7fed99800700 1 -- start start 2024-03-24T00:55:51.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.045+0000 7fed99800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed94073180 0x7fed9419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:51.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.045+0000 7fed99800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed94073aa0 0x7fed9419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:51.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.045+0000 7fed99800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9419f420 con 0x7fed94073180 2024-03-24T00:55:51.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.045+0000 7fed99800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed94198a00 con 0x7fed94073aa0 2024-03-24T00:55:51.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.046+0000 7fed92a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed94073180 0x7fed9419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:51.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.046+0000 7fed92a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed94073180 0x7fed9419e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37436/0 (socket says 172.21.15.38:37436) 2024-03-24T00:55:51.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.046+0000 7fed92a00700 1 -- 172.21.15.38:0/2441172155 learned_addr learned my addr 172.21.15.38:0/2441172155 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:51.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.046+0000 7fed8be00700 1 --2- 172.21.15.38:0/2441172155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed94073aa0 0x7fed9419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:51.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.046+0000 7fed92a00700 1 -- 172.21.15.38:0/2441172155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed94073aa0 msgr2=0x7fed9419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:51.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.046+0000 7fed92a00700 1 --2- 172.21.15.38:0/2441172155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed94073aa0 0x7fed9419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.046+0000 7fed92a00700 1 -- 172.21.15.38:0/2441172155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed840095f0 con 0x7fed94073180 2024-03-24T00:55:51.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.047+0000 7fed92a00700 1 --2- 172.21.15.38:0/2441172155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed94073180 0x7fed9419e8c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fed840092e0 tx=0x7fed8400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:51.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.047+0000 7fed8b400700 1 -- 172.21.15.38:0/2441172155 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed8401d070 con 0x7fed94073180 2024-03-24T00:55:51.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.047+0000 7fed8b400700 1 -- 172.21.15.38:0/2441172155 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed840056e0 con 0x7fed94073180 2024-03-24T00:55:51.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.047+0000 7fed99800700 1 -- 172.21.15.38:0/2441172155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed94198c80 con 0x7fed94073180 2024-03-24T00:55:51.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.047+0000 7fed8b400700 1 -- 172.21.15.38:0/2441172155 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed8400fe80 con 0x7fed94073180 2024-03-24T00:55:51.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.047+0000 7fed99800700 1 -- 172.21.15.38:0/2441172155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed94199170 con 0x7fed94073180 2024-03-24T00:55:51.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.049+0000 7fed99800700 1 -- 172.21.15.38:0/2441172155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed941076e0 con 0x7fed94073180 2024-03-24T00:55:51.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.050+0000 7fed8b400700 1 -- 172.21.15.38:0/2441172155 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fed8400f6f0 con 0x7fed94073180 2024-03-24T00:55:51.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.051+0000 7fed8b400700 1 --2- 172.21.15.38:0/2441172155 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fed7806dfb0 0x7fed78070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:51.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.051+0000 7fed8b400700 1 -- 172.21.15.38:0/2441172155 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fed8408f170 con 0x7fed94073180 2024-03-24T00:55:51.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.051+0000 7fed8be00700 1 --2- 172.21.15.38:0/2441172155 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fed7806dfb0 0x7fed78070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:51.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.052+0000 7fed8be00700 1 --2- 172.21.15.38:0/2441172155 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fed7806dfb0 0x7fed78070470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fed7c005950 tx=0x7fed7c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:51.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.058+0000 7fed8b400700 1 -- 172.21.15.38:0/2441172155 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fed8405c150 con 0x7fed94073180 2024-03-24T00:55:51.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.300+0000 7fed99800700 1 -- 172.21.15.38:0/2441172155 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fed94199f10 con 0x7fed7806dfb0 2024-03-24T00:55:51.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.312+0000 7fed8b400700 1 -- 172.21.15.38:0/2441172155 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fed94199f10 con 0x7fed7806dfb0 2024-03-24T00:55:51.313 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:55:51.313 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (4m) 7s ago 18m 14.5M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:55:51.313 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (18m) 7s ago 18m 8581k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:55:51.313 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (17m) 9s ago 17m 8363k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:55:51.313 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (23s) 7s ago 18m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T00:55:51.313 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (15s) 9s ago 17m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T00:55:51.313 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (3m) 7s ago 17m 73.0M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:55:51.313 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (10m) 7s ago 10m 97.9M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:55:51.313 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (10m) 7s ago 10m 150M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:55:51.313 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (10m) 9s ago 10m 24.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (10m) 9s ago 10m 24.8M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (7m) 7s ago 19m 497M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (5m) 9s ago 16m 449M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (2m) 7s ago 20m 45.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (88s) 9s ago 16m 41.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (5m) 7s ago 18m 7768k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (5m) 9s ago 16m 7755k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (15m) 7s ago 15m 259M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 6a42525b34cb 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (15m) 7s ago 15m 273M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (14m) 7s ago 14m 228M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (13m) 9s ago 13m 304M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (13m) 9s ago 13m 283M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (12m) 9s ago 12m 254M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:55:51.314 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (4m) 7s ago 17m 69.9M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:55:51.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.315+0000 7fed99800700 1 -- 172.21.15.38:0/2441172155 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fed7806dfb0 msgr2=0x7fed78070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:51.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.315+0000 7fed99800700 1 --2- 172.21.15.38:0/2441172155 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fed7806dfb0 0x7fed78070470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fed7c005950 tx=0x7fed7c0058e0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.315+0000 7fed99800700 1 -- 172.21.15.38:0/2441172155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed94073180 msgr2=0x7fed9419e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:51.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.315+0000 7fed99800700 1 --2- 172.21.15.38:0/2441172155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed94073180 0x7fed9419e8c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fed840092e0 tx=0x7fed8400be20 comp rx=0 tx=0).stop 2024-03-24T00:55:51.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.316+0000 7fed99800700 1 -- 172.21.15.38:0/2441172155 shutdown_connections 2024-03-24T00:55:51.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.316+0000 7fed99800700 1 --2- 172.21.15.38:0/2441172155 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fed7806dfb0 0x7fed78070470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.316+0000 7fed99800700 1 --2- 172.21.15.38:0/2441172155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed94073180 0x7fed9419e8c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.316+0000 7fed99800700 1 --2- 172.21.15.38:0/2441172155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed94073aa0 0x7fed9419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.316+0000 7fed99800700 1 -- 172.21.15.38:0/2441172155 >> 172.21.15.38:0/2441172155 conn(0x7fed940fc5a0 msgr2=0x7fed941052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:51.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.316+0000 7fed99800700 1 -- 172.21.15.38:0/2441172155 shutdown_connections 2024-03-24T00:55:51.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.316+0000 7fed99800700 1 -- 172.21.15.38:0/2441172155 wait complete. 2024-03-24T00:55:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.471+0000 7f7d5c400700 1 -- 172.21.15.38:0/2265067884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54102530 msgr2=0x7f7d5410aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.471+0000 7f7d5c400700 1 --2- 172.21.15.38:0/2265067884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54102530 0x7f7d5410aa20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009910 tx=0x7f7d40009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.472+0000 7f7d5c400700 1 -- 172.21.15.38:0/2265067884 shutdown_connections 2024-03-24T00:55:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.472+0000 7f7d5c400700 1 --2- 172.21.15.38:0/2265067884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54102530 0x7f7d5410aa20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.472+0000 7f7d5c400700 1 --2- 172.21.15.38:0/2265067884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d54101c10 0x7f7d54101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.472+0000 7f7d5c400700 1 -- 172.21.15.38:0/2265067884 >> 172.21.15.38:0/2265067884 conn(0x7f7d54076640 msgr2=0x7f7d540fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:51.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.472+0000 7f7d5c400700 1 -- 172.21.15.38:0/2265067884 shutdown_connections 2024-03-24T00:55:51.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.472+0000 7f7d5c400700 1 -- 172.21.15.38:0/2265067884 wait complete. 2024-03-24T00:55:51.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.473+0000 7f7d5c400700 1 Processor -- start 2024-03-24T00:55:51.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.473+0000 7f7d5c400700 1 -- start start 2024-03-24T00:55:51.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.474+0000 7f7d5c400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54101c10 0x7f7d5419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:51.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.474+0000 7f7d5c400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d54102530 0x7f7d5419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:51.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.474+0000 7f7d5c400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5419f470 con 0x7f7d54101c10 2024-03-24T00:55:51.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.474+0000 7f7d5c400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d54198a50 con 0x7f7d54102530 2024-03-24T00:55:51.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.474+0000 7f7d59a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54101c10 0x7f7d5419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:51.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.474+0000 7f7d59a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54101c10 0x7f7d5419e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37454/0 (socket says 172.21.15.38:37454) 2024-03-24T00:55:51.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.474+0000 7f7d59a00700 1 -- 172.21.15.38:0/3757688824 learned_addr learned my addr 172.21.15.38:0/3757688824 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:51.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.474+0000 7f7d59a00700 1 -- 172.21.15.38:0/3757688824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d54102530 msgr2=0x7f7d5419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:51.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.474+0000 7f7d59a00700 1 --2- 172.21.15.38:0/3757688824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d54102530 0x7f7d5419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.474+0000 7f7d59a00700 1 -- 172.21.15.38:0/3757688824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d400095f0 con 0x7f7d54101c10 2024-03-24T00:55:51.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.475+0000 7f7d59a00700 1 --2- 172.21.15.38:0/3757688824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54101c10 0x7f7d5419e910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800d8d0 tx=0x7f7d4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:51.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.475+0000 7f7d52000700 1 -- 172.21.15.38:0/3757688824 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d48009940 con 0x7f7d54101c10 2024-03-24T00:55:51.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.475+0000 7f7d52000700 1 -- 172.21.15.38:0/3757688824 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d48010460 con 0x7f7d54101c10 2024-03-24T00:55:51.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.475+0000 7f7d5c400700 1 -- 172.21.15.38:0/3757688824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d54198d30 con 0x7f7d54101c10 2024-03-24T00:55:51.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.475+0000 7f7d52000700 1 -- 172.21.15.38:0/3757688824 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d4800f5f0 con 0x7f7d54101c10 2024-03-24T00:55:51.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.475+0000 7f7d5c400700 1 -- 172.21.15.38:0/3757688824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d54199280 con 0x7f7d54101c10 2024-03-24T00:55:51.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.476+0000 7f7d5c400700 1 -- 172.21.15.38:0/3757688824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d54100f40 con 0x7f7d54101c10 2024-03-24T00:55:51.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.477+0000 7f7d52000700 1 -- 172.21.15.38:0/3757688824 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f7d480105d0 con 0x7f7d54101c10 2024-03-24T00:55:51.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.479+0000 7f7d52000700 1 --2- 172.21.15.38:0/3757688824 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d3c06dfa0 0x7f7d3c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:51.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.479+0000 7f7d52000700 1 -- 172.21.15.38:0/3757688824 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7d4808dca0 con 0x7f7d54101c10 2024-03-24T00:55:51.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.479+0000 7f7d59000700 1 --2- 172.21.15.38:0/3757688824 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d3c06dfa0 0x7f7d3c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:51.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.480+0000 7f7d59000700 1 --2- 172.21.15.38:0/3757688824 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d3c06dfa0 0x7f7d3c070460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000b5c0 tx=0x7f7d40005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:51.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.484+0000 7f7d52000700 1 -- 172.21.15.38:0/3757688824 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7d4805ac80 con 0x7f7d54101c10 2024-03-24T00:55:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.815+0000 7f7d5c400700 1 -- 172.21.15.38:0/3757688824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7d54199f10 con 0x7f7d54101c10 2024-03-24T00:55:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.816+0000 7f7d52000700 1 -- 172.21.15.38:0/3757688824 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+730 (secure 0 0 0) 0x7f7d4805a810 con 0x7f7d54101c10 2024-03-24T00:55:51.816 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:55:51.816 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:55:51.816 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:55:51.816 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:55:51.816 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:55:51.816 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:55:51.816 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:55:51.816 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:55:51.816 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6 2024-03-24T00:55:51.817 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:55:51.817 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:55:51.817 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:55:51.817 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:55:51.817 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:55:51.817 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 10, 2024-03-24T00:55:51.817 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-24T00:55:51.817 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:55:51.817 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:55:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.818+0000 7f7d5c400700 1 -- 172.21.15.38:0/3757688824 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d3c06dfa0 msgr2=0x7f7d3c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.818+0000 7f7d5c400700 1 --2- 172.21.15.38:0/3757688824 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d3c06dfa0 0x7f7d3c070460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000b5c0 tx=0x7f7d40005960 comp rx=0 tx=0).stop 2024-03-24T00:55:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.818+0000 7f7d5c400700 1 -- 172.21.15.38:0/3757688824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54101c10 msgr2=0x7f7d5419e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.818+0000 7f7d5c400700 1 --2- 172.21.15.38:0/3757688824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54101c10 0x7f7d5419e910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800d8d0 tx=0x7f7d4800dc90 comp rx=0 tx=0).stop 2024-03-24T00:55:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.819+0000 7f7d5c400700 1 -- 172.21.15.38:0/3757688824 shutdown_connections 2024-03-24T00:55:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.819+0000 7f7d5c400700 1 --2- 172.21.15.38:0/3757688824 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7d3c06dfa0 0x7f7d3c070460 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.819+0000 7f7d5c400700 1 --2- 172.21.15.38:0/3757688824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54101c10 0x7f7d5419e910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.819+0000 7f7d5c400700 1 --2- 172.21.15.38:0/3757688824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d54102530 0x7f7d5419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.819+0000 7f7d5c400700 1 -- 172.21.15.38:0/3757688824 >> 172.21.15.38:0/3757688824 conn(0x7f7d54076640 msgr2=0x7f7d54109350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.819+0000 7f7d5c400700 1 -- 172.21.15.38:0/3757688824 shutdown_connections 2024-03-24T00:55:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.819+0000 7f7d5c400700 1 -- 172.21.15.38:0/3757688824 wait complete. 2024-03-24T00:55:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.972+0000 7fa5eba00700 1 -- 172.21.15.38:0/3840121263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5e4101c10 msgr2=0x7fa5e4101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.972+0000 7fa5eba00700 1 --2- 172.21.15.38:0/3840121263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5e4101c10 0x7fa5e4101ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009910 tx=0x7fa5d8009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.972+0000 7fa5eba00700 1 -- 172.21.15.38:0/3840121263 shutdown_connections 2024-03-24T00:55:51.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.972+0000 7fa5eba00700 1 --2- 172.21.15.38:0/3840121263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5e4102530 0x7fa5e410aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.972+0000 7fa5eba00700 1 --2- 172.21.15.38:0/3840121263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5e4101c10 0x7fa5e4101ff0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.972+0000 7fa5eba00700 1 -- 172.21.15.38:0/3840121263 >> 172.21.15.38:0/3840121263 conn(0x7fa5e4076640 msgr2=0x7fa5e40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:51.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.972+0000 7fa5eba00700 1 -- 172.21.15.38:0/3840121263 shutdown_connections 2024-03-24T00:55:51.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.972+0000 7fa5eba00700 1 -- 172.21.15.38:0/3840121263 wait complete. 2024-03-24T00:55:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.973+0000 7fa5eba00700 1 Processor -- start 2024-03-24T00:55:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.973+0000 7fa5eba00700 1 -- start start 2024-03-24T00:55:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5eba00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5e4101c10 0x7fa5e419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5eba00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5e4102530 0x7fa5e419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5eba00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e41989b0 con 0x7fa5e4102530 2024-03-24T00:55:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5eba00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4198b20 con 0x7fa5e4101c10 2024-03-24T00:55:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5e3e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5e4102530 0x7fa5e419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5e3e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5e4102530 0x7fa5e419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37474/0 (socket says 172.21.15.38:37474) 2024-03-24T00:55:51.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5e3e00700 1 -- 172.21.15.38:0/1853349470 learned_addr learned my addr 172.21.15.38:0/1853349470 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:51.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5e9000700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5e4101c10 0x7fa5e419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:51.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5e3e00700 1 -- 172.21.15.38:0/1853349470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5e4101c10 msgr2=0x7fa5e419e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:51.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5e3e00700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5e4101c10 0x7fa5e419e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:51.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5e3e00700 1 -- 172.21.15.38:0/1853349470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d80095f0 con 0x7fa5e4102530 2024-03-24T00:55:51.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.974+0000 7fa5e9000700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5e4101c10 0x7fa5e419e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:55:51.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.975+0000 7fa5e3e00700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5e4102530 0x7fa5e419ee40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e940 tx=0x7fa5d000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:51.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.975+0000 7fa5e1600700 1 -- 172.21.15.38:0/1853349470 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d00096e0 con 0x7fa5e4102530 2024-03-24T00:55:51.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.975+0000 7fa5e1600700 1 -- 172.21.15.38:0/1853349470 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5d0004cf0 con 0x7fa5e4102530 2024-03-24T00:55:51.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.975+0000 7fa5eba00700 1 -- 172.21.15.38:0/1853349470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e4198e00 con 0x7fa5e4102530 2024-03-24T00:55:51.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.975+0000 7fa5e1600700 1 -- 172.21.15.38:0/1853349470 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d000eee0 con 0x7fa5e4102530 2024-03-24T00:55:51.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.975+0000 7fa5eba00700 1 -- 172.21.15.38:0/1853349470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e4199350 con 0x7fa5e4102530 2024-03-24T00:55:51.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.976+0000 7fa5eba00700 1 -- 172.21.15.38:0/1853349470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e4068d60 con 0x7fa5e4102530 2024-03-24T00:55:51.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.977+0000 7fa5e1600700 1 -- 172.21.15.38:0/1853349470 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fa5d000cba0 con 0x7fa5e4102530 2024-03-24T00:55:51.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.979+0000 7fa5e1600700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa5cc06dfa0 0x7fa5cc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:51.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.979+0000 7fa5e1600700 1 -- 172.21.15.38:0/1853349470 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa5d0014070 con 0x7fa5e4102530 2024-03-24T00:55:51.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.979+0000 7fa5e9000700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa5cc06dfa0 0x7fa5cc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:51.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.980+0000 7fa5e9000700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa5cc06dfa0 0x7fa5cc070460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800b760 tx=0x7fa5d8005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:51.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:51.984+0000 7fa5e1600700 1 -- 172.21.15.38:0/1853349470 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa5d00574a0 con 0x7fa5e4102530 2024-03-24T00:55:52.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.283+0000 7fa5eba00700 1 -- 172.21.15.38:0/1853349470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa5e41995d0 con 0x7fa5e4102530 2024-03-24T00:55:52.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.284+0000 7fa5e1600700 1 -- 172.21.15.38:0/1853349470 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1900 (secure 0 0 0) 0x7fa5d005ab60 con 0x7fa5e4102530 2024-03-24T00:55:52.285 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:51 smithi038 ceph-mon[112108]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:52.285 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:51 smithi038 ceph-mon[112108]: pgmap v60: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:52.285 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:51 smithi038 ceph-mon[112108]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:52.285 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:51 smithi038 ceph-mon[112108]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:52.285 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:51 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:52.285 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:51 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:52.285 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:51 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:52.285 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:51 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:55:52.285 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:51 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:52.286 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:51 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3757688824' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:52.286 INFO:teuthology.orchestra.run.smithi038.stdout:e20 2024-03-24T00:55:52.286 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:55:52.286 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:55:52.286 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:55:52.286 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:55:52.286 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:55:52.286 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:55:52.286 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 20 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:55:52.287 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:55:52.288 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:55:52.289 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:55:52.289 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:55:52.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.287+0000 7fa5eba00700 1 -- 172.21.15.38:0/1853349470 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa5cc06dfa0 msgr2=0x7fa5cc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:52.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.287+0000 7fa5eba00700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa5cc06dfa0 0x7fa5cc070460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800b760 tx=0x7fa5d8005940 comp rx=0 tx=0).stop 2024-03-24T00:55:52.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.287+0000 7fa5eba00700 1 -- 172.21.15.38:0/1853349470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5e4102530 msgr2=0x7fa5e419ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:52.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.287+0000 7fa5eba00700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5e4102530 0x7fa5e419ee40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e940 tx=0x7fa5d000ed00 comp rx=0 tx=0).stop 2024-03-24T00:55:52.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.287+0000 7fa5eba00700 1 -- 172.21.15.38:0/1853349470 shutdown_connections 2024-03-24T00:55:52.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.287+0000 7fa5eba00700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa5cc06dfa0 0x7fa5cc070460 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.287+0000 7fa5eba00700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5e4101c10 0x7fa5e419e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.287+0000 7fa5eba00700 1 --2- 172.21.15.38:0/1853349470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5e4102530 0x7fa5e419ee40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.287+0000 7fa5eba00700 1 -- 172.21.15.38:0/1853349470 >> 172.21.15.38:0/1853349470 conn(0x7fa5e4076640 msgr2=0x7fa5e410a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:52.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.288+0000 7fa5eba00700 1 -- 172.21.15.38:0/1853349470 shutdown_connections 2024-03-24T00:55:52.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.288+0000 7fa5eba00700 1 -- 172.21.15.38:0/1853349470 wait complete. 2024-03-24T00:55:52.290 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:55:52.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:51 smithi088 ceph-mon[87351]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:51 smithi088 ceph-mon[87351]: pgmap v60: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:51 smithi088 ceph-mon[87351]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:51 smithi088 ceph-mon[87351]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:51 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:51 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:51 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:51 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:55:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:51 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:51 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3757688824' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:52.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.452+0000 7f93a0c00700 1 -- 172.21.15.38:0/3893800173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c073b00 msgr2=0x7f939c10cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:52.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.452+0000 7f93a0c00700 1 --2- 172.21.15.38:0/3893800173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c073b00 0x7f939c10cbe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9388009910 tx=0x7f9388009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:52.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.452+0000 7f93a0c00700 1 -- 172.21.15.38:0/3893800173 shutdown_connections 2024-03-24T00:55:52.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.452+0000 7f93a0c00700 1 --2- 172.21.15.38:0/3893800173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c073b00 0x7f939c10cbe0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.452+0000 7f93a0c00700 1 --2- 172.21.15.38:0/3893800173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f939c0731e0 0x7f939c0735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.452+0000 7f93a0c00700 1 -- 172.21.15.38:0/3893800173 >> 172.21.15.38:0/3893800173 conn(0x7f939c0fc3f0 msgr2=0x7f939c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:52.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.452+0000 7f93a0c00700 1 -- 172.21.15.38:0/3893800173 shutdown_connections 2024-03-24T00:55:52.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.452+0000 7f93a0c00700 1 -- 172.21.15.38:0/3893800173 wait complete. 2024-03-24T00:55:52.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.454+0000 7f93a0c00700 1 Processor -- start 2024-03-24T00:55:52.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.454+0000 7f93a0c00700 1 -- start start 2024-03-24T00:55:52.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.455+0000 7f93a0c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c0731e0 0x7f939c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.455+0000 7f93a0c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f939c073b00 0x7f939c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.455+0000 7f93a0c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c19f250 con 0x7f939c0731e0 2024-03-24T00:55:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.455+0000 7f93a0c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c1988a0 con 0x7f939c073b00 2024-03-24T00:55:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.455+0000 7f939b400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c0731e0 0x7f939c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.455+0000 7f939b400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c0731e0 0x7f939c19e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37492/0 (socket says 172.21.15.38:37492) 2024-03-24T00:55:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.455+0000 7f939b400700 1 -- 172.21.15.38:0/462733888 learned_addr learned my addr 172.21.15.38:0/462733888 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:52.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.455+0000 7f939b400700 1 -- 172.21.15.38:0/462733888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f939c073b00 msgr2=0x7f939c19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.455+0000 7f939b400700 1 --2- 172.21.15.38:0/462733888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f939c073b00 0x7f939c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.455+0000 7f939b400700 1 -- 172.21.15.38:0/462733888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880095f0 con 0x7f939c0731e0 2024-03-24T00:55:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.456+0000 7f939b400700 1 --2- 172.21.15.38:0/462733888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c0731e0 0x7f939c19e710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f938c00c930 tx=0x7f938c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.456+0000 7f9393e00700 1 -- 172.21.15.38:0/462733888 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f938c007ad0 con 0x7f939c0731e0 2024-03-24T00:55:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.456+0000 7f9393e00700 1 -- 172.21.15.38:0/462733888 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f938c007c30 con 0x7f939c0731e0 2024-03-24T00:55:52.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.456+0000 7f9393e00700 1 -- 172.21.15.38:0/462733888 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f938c018610 con 0x7f939c0731e0 2024-03-24T00:55:52.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.456+0000 7f93a0c00700 1 -- 172.21.15.38:0/462733888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c198b80 con 0x7f939c0731e0 2024-03-24T00:55:52.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.456+0000 7f93a0c00700 1 -- 172.21.15.38:0/462733888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c199080 con 0x7f939c0731e0 2024-03-24T00:55:52.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.457+0000 7f93a0c00700 1 -- 172.21.15.38:0/462733888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939c10a380 con 0x7f939c0731e0 2024-03-24T00:55:52.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.459+0000 7f9393e00700 1 -- 172.21.15.38:0/462733888 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f938c00f450 con 0x7f939c0731e0 2024-03-24T00:55:52.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.459+0000 7f9393e00700 1 --2- 172.21.15.38:0/462733888 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f938006dfb0 0x7f9380070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:52.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.460+0000 7f9393e00700 1 -- 172.21.15.38:0/462733888 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f938c08dff0 con 0x7f939c0731e0 2024-03-24T00:55:52.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.460+0000 7f939aa00700 1 --2- 172.21.15.38:0/462733888 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f938006dfb0 0x7f9380070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:52.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.461+0000 7f939aa00700 1 --2- 172.21.15.38:0/462733888 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f938006dfb0 0x7f9380070470 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9388006010 tx=0x7f938800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:52.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.464+0000 7f9393e00700 1 -- 172.21.15.38:0/462733888 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f938c0c2740 con 0x7f939c0731e0 2024-03-24T00:55:52.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.720+0000 7f93a0c00700 1 -- 172.21.15.38:0/462733888 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f939c199db0 con 0x7f938006dfb0 2024-03-24T00:55:52.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.721+0000 7f9393e00700 1 -- 172.21.15.38:0/462733888 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f939c199db0 con 0x7f938006dfb0 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout: "crash" 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "6/23 daemons upgraded", 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:55:52.723 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:55:52.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.724+0000 7f93a0c00700 1 -- 172.21.15.38:0/462733888 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f938006dfb0 msgr2=0x7f9380070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:52.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.724+0000 7f93a0c00700 1 --2- 172.21.15.38:0/462733888 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f938006dfb0 0x7f9380070470 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9388006010 tx=0x7f938800b540 comp rx=0 tx=0).stop 2024-03-24T00:55:52.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.724+0000 7f93a0c00700 1 -- 172.21.15.38:0/462733888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c0731e0 msgr2=0x7f939c19e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:52.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.724+0000 7f93a0c00700 1 --2- 172.21.15.38:0/462733888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c0731e0 0x7f939c19e710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f938c00c930 tx=0x7f938c00ccf0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.724+0000 7f93a0c00700 1 -- 172.21.15.38:0/462733888 shutdown_connections 2024-03-24T00:55:52.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.724+0000 7f93a0c00700 1 --2- 172.21.15.38:0/462733888 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f938006dfb0 0x7f9380070470 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.724+0000 7f93a0c00700 1 --2- 172.21.15.38:0/462733888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f939c0731e0 0x7f939c19e710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.724+0000 7f93a0c00700 1 --2- 172.21.15.38:0/462733888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f939c073b00 0x7f939c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.724+0000 7f93a0c00700 1 -- 172.21.15.38:0/462733888 >> 172.21.15.38:0/462733888 conn(0x7f939c0fc3f0 msgr2=0x7f939c1074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:52.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.724+0000 7f93a0c00700 1 -- 172.21.15.38:0/462733888 shutdown_connections 2024-03-24T00:55:52.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.725+0000 7f93a0c00700 1 -- 172.21.15.38:0/462733888 wait complete. 2024-03-24T00:55:52.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.880+0000 7f4c39400700 1 -- 172.21.15.38:0/2225879266 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c2c0054b0 con 0x7f4c3c101c10 2024-03-24T00:55:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.881+0000 7f4c41200700 1 -- 172.21.15.38:0/2225879266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c3c101c10 msgr2=0x7f4c3c101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.881+0000 7f4c41200700 1 --2- 172.21.15.38:0/2225879266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c3c101c10 0x7f4c3c101ff0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009910 tx=0x7f4c2c009c20 comp rx=0 tx=0).stop 2024-03-24T00:55:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.881+0000 7f4c41200700 1 -- 172.21.15.38:0/2225879266 shutdown_connections 2024-03-24T00:55:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.881+0000 7f4c41200700 1 --2- 172.21.15.38:0/2225879266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c102530 0x7f4c3c10aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.881+0000 7f4c41200700 1 --2- 172.21.15.38:0/2225879266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c3c101c10 0x7f4c3c101ff0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.881+0000 7f4c41200700 1 -- 172.21.15.38:0/2225879266 >> 172.21.15.38:0/2225879266 conn(0x7f4c3c076640 msgr2=0x7f4c3c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:52.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.881+0000 7f4c41200700 1 -- 172.21.15.38:0/2225879266 shutdown_connections 2024-03-24T00:55:52.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.881+0000 7f4c41200700 1 -- 172.21.15.38:0/2225879266 wait complete. 2024-03-24T00:55:52.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.882+0000 7f4c41200700 1 Processor -- start 2024-03-24T00:55:52.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.882+0000 7f4c41200700 1 -- start start 2024-03-24T00:55:52.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.883+0000 7f4c41200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c3c101c10 0x7f4c3c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:52.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.883+0000 7f4c41200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c102530 0x7f4c3c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:52.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.883+0000 7f4c41200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c19b110 con 0x7f4c3c101c10 2024-03-24T00:55:52.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.883+0000 7f4c41200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c1946f0 con 0x7f4c3c102530 2024-03-24T00:55:52.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.883+0000 7f4c39e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c3c101c10 0x7f4c3c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:52.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.883+0000 7f4c39e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c3c101c10 0x7f4c3c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37506/0 (socket says 172.21.15.38:37506) 2024-03-24T00:55:52.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.883+0000 7f4c39e00700 1 -- 172.21.15.38:0/3671953829 learned_addr learned my addr 172.21.15.38:0/3671953829 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:55:52.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.883+0000 7f4c39e00700 1 -- 172.21.15.38:0/3671953829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c102530 msgr2=0x7f4c3c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:52.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.883+0000 7f4c39e00700 1 --2- 172.21.15.38:0/3671953829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c102530 0x7f4c3c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:52.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.883+0000 7f4c39e00700 1 -- 172.21.15.38:0/3671953829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0095f0 con 0x7f4c3c101c10 2024-03-24T00:55:52.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.884+0000 7f4c39e00700 1 --2- 172.21.15.38:0/3671953829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c3c101c10 0x7f4c3c19a5b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c006010 tx=0x7f4c2c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:52.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.884+0000 7f4c32a00700 1 -- 172.21.15.38:0/3671953829 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c2c01d070 con 0x7f4c3c101c10 2024-03-24T00:55:52.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.884+0000 7f4c32a00700 1 -- 172.21.15.38:0/3671953829 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c2c00f460 con 0x7f4c3c101c10 2024-03-24T00:55:52.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.884+0000 7f4c41200700 1 -- 172.21.15.38:0/3671953829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c194970 con 0x7f4c3c101c10 2024-03-24T00:55:52.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.884+0000 7f4c32a00700 1 -- 172.21.15.38:0/3671953829 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c2c017630 con 0x7f4c3c101c10 2024-03-24T00:55:52.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.884+0000 7f4c41200700 1 -- 172.21.15.38:0/3671953829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c194e60 con 0x7f4c3c101c10 2024-03-24T00:55:52.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.885+0000 7f4c41200700 1 -- 172.21.15.38:0/3671953829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c068d60 con 0x7f4c3c101c10 2024-03-24T00:55:52.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.886+0000 7f4c32a00700 1 -- 172.21.15.38:0/3671953829 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f4c2c00f5d0 con 0x7f4c3c101c10 2024-03-24T00:55:52.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.887+0000 7f4c32a00700 1 --2- 172.21.15.38:0/3671953829 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4c200723c0 0x7f4c20074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:55:52.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.887+0000 7f4c32a00700 1 -- 172.21.15.38:0/3671953829 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4c2c08f180 con 0x7f4c3c101c10 2024-03-24T00:55:52.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.887+0000 7f4c33e00700 1 --2- 172.21.15.38:0/3671953829 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4c200723c0 0x7f4c20074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:55:52.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.888+0000 7f4c33e00700 1 --2- 172.21.15.38:0/3671953829 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4c200723c0 0x7f4c20074880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006fd0 tx=0x7f4c24006dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:55:52.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:52.892+0000 7f4c32a00700 1 -- 172.21.15.38:0/3671953829 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4c2c05c2c0 con 0x7f4c3c101c10 2024-03-24T00:55:53.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.223+0000 7f4c41200700 1 -- 172.21.15.38:0/3671953829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4c3c04ea80 con 0x7f4c3c101c10 2024-03-24T00:55:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.224+0000 7f4c32a00700 1 -- 172.21.15.38:0/3671953829 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f4c3c04ea80 con 0x7f4c3c101c10 2024-03-24T00:55:53.225 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_OK 2024-03-24T00:55:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.226+0000 7f4c41200700 1 -- 172.21.15.38:0/3671953829 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4c200723c0 msgr2=0x7f4c20074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.226+0000 7f4c41200700 1 --2- 172.21.15.38:0/3671953829 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4c200723c0 0x7f4c20074880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006fd0 tx=0x7f4c24006dc0 comp rx=0 tx=0).stop 2024-03-24T00:55:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.226+0000 7f4c41200700 1 -- 172.21.15.38:0/3671953829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c3c101c10 msgr2=0x7f4c3c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:55:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.226+0000 7f4c41200700 1 --2- 172.21.15.38:0/3671953829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c3c101c10 0x7f4c3c19a5b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c006010 tx=0x7f4c2c00be20 comp rx=0 tx=0).stop 2024-03-24T00:55:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.226+0000 7f4c41200700 1 -- 172.21.15.38:0/3671953829 shutdown_connections 2024-03-24T00:55:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.226+0000 7f4c41200700 1 --2- 172.21.15.38:0/3671953829 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4c200723c0 0x7f4c20074880 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.226+0000 7f4c41200700 1 --2- 172.21.15.38:0/3671953829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c3c101c10 0x7f4c3c19a5b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.226+0000 7f4c41200700 1 --2- 172.21.15.38:0/3671953829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c3c102530 0x7f4c3c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:55:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.226+0000 7f4c41200700 1 -- 172.21.15.38:0/3671953829 >> 172.21.15.38:0/3671953829 conn(0x7f4c3c076640 msgr2=0x7f4c3c0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:55:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.227+0000 7f4c41200700 1 -- 172.21.15.38:0/3671953829 shutdown_connections 2024-03-24T00:55:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:55:53.227+0000 7f4c41200700 1 -- 172.21.15.38:0/3671953829 wait complete. 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: Upgrade: Setting container_image for all crash 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi038"}]: dispatch 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi038"}]': finished 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi088"}]: dispatch 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi088"}]': finished 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T00:55:53.304 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T00:55:53.305 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: Upgrade: osd.0 is safe to restart 2024-03-24T00:55:53.305 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:52 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1853349470' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:55:53.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:55:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:55:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: Upgrade: Setting container_image for all crash 2024-03-24T00:55:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi038"}]: dispatch 2024-03-24T00:55:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi038"}]': finished 2024-03-24T00:55:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi088"}]: dispatch 2024-03-24T00:55:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi088"}]': finished 2024-03-24T00:55:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T00:55:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T00:55:53.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: Upgrade: osd.0 is safe to restart 2024-03-24T00:55:53.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:52 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1853349470' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:55:54.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:54 smithi038 ceph-mon[112108]: pgmap v61: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:54.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:54 smithi038 ceph-mon[112108]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:54.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:54 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3671953829' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:55:54.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:54 smithi088 ceph-mon[87351]: pgmap v61: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:55:54.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:54 smithi088 ceph-mon[87351]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:55:54.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:54 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3671953829' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:55:55.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:55 smithi038 ceph-mon[112108]: Upgrade: Updating osd.0 2024-03-24T00:55:55.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:55.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T00:55:55.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:55.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:55 smithi038 ceph-mon[112108]: Deploying daemon osd.0 on smithi038 2024-03-24T00:55:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:55 smithi088 ceph-mon[87351]: Upgrade: Updating osd.0 2024-03-24T00:55:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:55:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T00:55:55.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:55:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:55 smithi088 ceph-mon[87351]: Deploying daemon osd.0 on smithi038 2024-03-24T00:55:56.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:56 smithi038 ceph-mon[112108]: pgmap v62: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:56 smithi088 ceph-mon[87351]: pgmap v62: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:57.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:55:57.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:55:58.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:55:58 smithi038 ceph-mon[112108]: pgmap v63: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:58.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:55:58 smithi088 ceph-mon[87351]: pgmap v63: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:55:59.359 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:55:58 smithi038 systemd[1]: Stopping Ceph osd.0 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:55:59.859 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:55:59 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0[47766]: 2024-03-24T00:55:59.463+0000 7fcb26a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T00:55:59.860 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:55:59 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0[47766]: 2024-03-24T00:55:59.463+0000 7fcb26a00700 -1 osd.0 50 *** Got signal Terminated *** 2024-03-24T00:55:59.860 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:55:59 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0[47766]: 2024-03-24T00:55:59.463+0000 7fcb26a00700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T00:56:00.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:00 smithi038 ceph-mon[112108]: pgmap v64: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:56:00.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:00 smithi038 ceph-mon[112108]: osd.0 marked itself down and dead 2024-03-24T00:56:00.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:00 smithi088 ceph-mon[87351]: pgmap v64: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:56:00.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:00 smithi088 ceph-mon[87351]: osd.0 marked itself down and dead 2024-03-24T00:56:01.090 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:00 smithi038 podman[123506]: 2024-03-24 00:56:00.813741531 +0000 UTC m=+1.492920654 container died 6a42525b34cb885af0148b009715a2240713e8bf8420f1de84f0cc40c159c492 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-738c2fd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:56:01.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:01 smithi038 ceph-mon[112108]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:56:01.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:01 smithi038 ceph-mon[112108]: osdmap e51: 6 total, 5 up, 6 in 2024-03-24T00:56:01.359 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:01 smithi038 podman[123506]: 2024-03-24 00:56:01.29078172 +0000 UTC m=+1.969960840 container cleanup 6a42525b34cb885af0148b009715a2240713e8bf8420f1de84f0cc40c159c492 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=reef-738c2fd, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-03-24T00:56:01.359 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:01 smithi038 bash[123506]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0 2024-03-24T00:56:01.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:01 smithi088 ceph-mon[87351]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:56:01.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:01 smithi088 ceph-mon[87351]: osdmap e51: 6 total, 5 up, 6 in 2024-03-24T00:56:02.110 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:01 smithi038 podman[123526]: 2024-03-24 00:56:01.667696615 +0000 UTC m=+0.839730690 container remove 6a42525b34cb885af0148b009715a2240713e8bf8420f1de84f0cc40c159c492 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0, io.buildah.version=1.33.5, RELEASE=reef-738c2fd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T00:56:02.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:02 smithi038 ceph-mon[112108]: pgmap v66: 65 pgs: 9 stale+active+clean, 56 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:56:02.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:02 smithi038 ceph-mon[112108]: osdmap e52: 6 total, 5 up, 6 in 2024-03-24T00:56:02.610 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:02 smithi038 podman[123615]: 2024-03-24 00:56:02.377287645 +0000 UTC m=+0.031872882 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:56:02.610 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:02 smithi038 podman[123615]: 2024-03-24 00:56:02.493835944 +0000 UTC m=+0.148421208 container create 41b568c4f46fb8c421a209cdf3b6e255980ac91783807748fab07b93dba8f00f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-24T00:56:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:02 smithi088 ceph-mon[87351]: pgmap v66: 65 pgs: 9 stale+active+clean, 56 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:56:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:02 smithi088 ceph-mon[87351]: osdmap e52: 6 total, 5 up, 6 in 2024-03-24T00:56:03.110 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:02 smithi038 podman[123615]: 2024-03-24 00:56:02.715183679 +0000 UTC m=+0.369768984 container init 41b568c4f46fb8c421a209cdf3b6e255980ac91783807748fab07b93dba8f00f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-24T00:56:03.110 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:02 smithi038 podman[123615]: 2024-03-24 00:56:02.729092899 +0000 UTC m=+0.383678166 container start 41b568c4f46fb8c421a209cdf3b6e255980ac91783807748fab07b93dba8f00f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:56:03.110 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:02 smithi038 podman[123615]: 2024-03-24 00:56:02.804825406 +0000 UTC m=+0.459410645 container attach 41b568c4f46fb8c421a209cdf3b6e255980ac91783807748fab07b93dba8f00f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-24T00:56:03.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:03 smithi038 ceph-mon[112108]: Health check failed: Degraded data redundancy: 52/273 objects degraded (19.048%), 19 pgs degraded (PG_DEGRADED) 2024-03-24T00:56:03.550 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:03 smithi038 podman[123615]: 2024-03-24 00:56:03.182617914 +0000 UTC m=+0.837203208 container died 41b568c4f46fb8c421a209cdf3b6e255980ac91783807748fab07b93dba8f00f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-24T00:56:03.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:03 smithi088 ceph-mon[87351]: Health check failed: Degraded data redundancy: 52/273 objects degraded (19.048%), 19 pgs degraded (PG_DEGRADED) 2024-03-24T00:56:04.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:04 smithi038 ceph-mon[112108]: pgmap v68: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:04.547 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:04 smithi038 podman[123641]: 2024-03-24 00:56:04.133887695 +0000 UTC m=+0.930043141 container remove 41b568c4f46fb8c421a209cdf3b6e255980ac91783807748fab07b93dba8f00f (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-24T00:56:04.547 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:04 smithi038 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-24T00:56:04.547 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:04 smithi038 systemd[1]: Stopped Ceph osd.0 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:56:04.547 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:04 smithi038 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.0.service: Consumed 1min 47.894s CPU time. 2024-03-24T00:56:04.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:04 smithi088 ceph-mon[87351]: pgmap v68: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:04.859 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:04 smithi038 systemd[1]: Starting Ceph osd.0 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:56:05.226 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:04 smithi038 podman[123744]: 2024-03-24 00:56:04.814881605 +0000 UTC m=+0.033837526 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:56:05.226 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:05 smithi038 podman[123744]: 2024-03-24 00:56:05.068586356 +0000 UTC m=+0.287542318 container create 84ac656cd77e8f7ebffbf47d53ed15488cbc7c8f65042f19bcbed53ec0f82a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-24T00:56:05.610 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:05 smithi038 podman[123744]: 2024-03-24 00:56:05.31513455 +0000 UTC m=+0.534090564 container init 84ac656cd77e8f7ebffbf47d53ed15488cbc7c8f65042f19bcbed53ec0f82a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-24T00:56:05.610 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:05 smithi038 podman[123744]: 2024-03-24 00:56:05.329166733 +0000 UTC m=+0.548122698 container start 84ac656cd77e8f7ebffbf47d53ed15488cbc7c8f65042f19bcbed53ec0f82a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-24T00:56:05.610 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:05 smithi038 podman[123744]: 2024-03-24 00:56:05.429324613 +0000 UTC m=+0.648280560 container attach 84ac656cd77e8f7ebffbf47d53ed15488cbc7c8f65042f19bcbed53ec0f82a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T00:56:06.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:06 smithi038 ceph-mon[112108]: pgmap v69: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:06.611 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate[123758]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T00:56:06.611 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate[123758]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T00:56:06.611 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 bash[123744]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T00:56:06.611 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 bash[123744]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T00:56:06.611 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 bash[123744]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T00:56:06.611 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 bash[123744]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T00:56:06.612 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 bash[123744]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-24T00:56:06.612 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 bash[123744]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T00:56:06.612 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 bash[123744]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-24T00:56:06.612 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate[123758]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T00:56:06.612 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate[123758]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T00:56:06.612 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate[123758]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-24T00:56:06.612 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate[123758]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T00:56:06.612 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate[123758]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-24T00:56:06.612 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:06 smithi038 podman[123744]: 2024-03-24 00:56:06.312216611 +0000 UTC m=+1.531172612 container died 84ac656cd77e8f7ebffbf47d53ed15488cbc7c8f65042f19bcbed53ec0f82a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:56:06.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:06 smithi088 ceph-mon[87351]: pgmap v69: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:07.532 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:07 smithi038 podman[123890]: 2024-03-24 00:56:07.16644009 +0000 UTC m=+0.833720481 container remove 84ac656cd77e8f7ebffbf47d53ed15488cbc7c8f65042f19bcbed53ec0f82a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-activate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-24T00:56:07.859 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:07 smithi038 podman[123951]: 2024-03-24 00:56:07.439363711 +0000 UTC m=+0.040170780 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:56:07.860 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:07 smithi038 podman[123951]: 2024-03-24 00:56:07.567429533 +0000 UTC m=+0.168236629 container create a74a6e0da73c4dc1861a47f131dcad5ae1dd7e15b87bfadb0b1d7e844828cd14 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-24T00:56:08.303 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:07 smithi038 podman[123951]: 2024-03-24 00:56:07.963111167 +0000 UTC m=+0.563918266 container init a74a6e0da73c4dc1861a47f131dcad5ae1dd7e15b87bfadb0b1d7e844828cd14 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-24T00:56:08.303 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:07 smithi038 podman[123951]: 2024-03-24 00:56:07.975007189 +0000 UTC m=+0.575814255 container start a74a6e0da73c4dc1861a47f131dcad5ae1dd7e15b87bfadb0b1d7e844828cd14 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:56:08.303 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:08 smithi038 ceph-osd[123970]: -- 172.21.15.38:0/917977119 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555ef68ff680 con 0x555ef68c8c00 2024-03-24T00:56:08.303 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:08 smithi038 bash[123951]: a74a6e0da73c4dc1861a47f131dcad5ae1dd7e15b87bfadb0b1d7e844828cd14 2024-03-24T00:56:08.303 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:08 smithi038 systemd[1]: Started Ceph osd.0 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:56:08.609 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:08 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0[123966]: 2024-03-24T00:56:08.547+0000 7fd358baa7c0 -1 Falling back to public interface 2024-03-24T00:56:08.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:08 smithi038 ceph-mon[112108]: pgmap v70: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:08.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:08 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:08.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:08 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:08.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:08 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:56:08.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:08 smithi088 ceph-mon[87351]: pgmap v70: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:08.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:08 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:08.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:08 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:08.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:08 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:56:10.641 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:10 smithi038 ceph-mon[112108]: pgmap v71: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:10.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:10 smithi088 ceph-mon[87351]: pgmap v71: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:12.483 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:12 smithi088 ceph-mon[87351]: pgmap v72: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:12.483 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:12 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:12.483 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:12 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:56:12.568 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:12 smithi038 ceph-mon[112108]: pgmap v72: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:12.569 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:12.569 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:56:14.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:14 smithi088 ceph-mon[87351]: pgmap v73: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 4 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:14.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:14 smithi038 ceph-mon[112108]: pgmap v73: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 4 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:14.859 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:14 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0[123966]: 2024-03-24T00:56:14.495+0000 7fd358baa7c0 -1 osd.0 50 log_to_monitors true 2024-03-24T00:56:15.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:15 smithi088 ceph-mon[87351]: from='osd.0 [v2:172.21.15.38:6802/1079856560,v1:172.21.15.38:6803/1079856560]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T00:56:15.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:15 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:15.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:15 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:15.859 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 00:56:15 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0[123966]: 2024-03-24T00:56:15.438+0000 7fd348000700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T00:56:15.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:15 smithi038 ceph-mon[112108]: from='osd.0 [v2:172.21.15.38:6802/1079856560,v1:172.21.15.38:6803/1079856560]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T00:56:15.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:15 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:15.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:15 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:16 smithi088 ceph-mon[87351]: pgmap v74: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:16 smithi088 ceph-mon[87351]: from='osd.0 [v2:172.21.15.38:6802/1079856560,v1:172.21.15.38:6803/1079856560]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T00:56:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:16 smithi088 ceph-mon[87351]: osdmap e53: 6 total, 5 up, 6 in 2024-03-24T00:56:16.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:16 smithi088 ceph-mon[87351]: from='osd.0 [v2:172.21.15.38:6802/1079856560,v1:172.21.15.38:6803/1079856560]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:56:16.723 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:16 smithi038 ceph-mon[112108]: pgmap v74: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:16.723 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:16 smithi038 ceph-mon[112108]: from='osd.0 [v2:172.21.15.38:6802/1079856560,v1:172.21.15.38:6803/1079856560]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T00:56:16.723 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:16 smithi038 ceph-mon[112108]: osdmap e53: 6 total, 5 up, 6 in 2024-03-24T00:56:16.723 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:16 smithi038 ceph-mon[112108]: from='osd.0 [v2:172.21.15.38:6802/1079856560,v1:172.21.15.38:6803/1079856560]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:56:17.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:17 smithi088 ceph-mon[87351]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:56:17.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:17 smithi088 ceph-mon[87351]: osd.0 [v2:172.21.15.38:6802/1079856560,v1:172.21.15.38:6803/1079856560] boot 2024-03-24T00:56:17.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:17 smithi088 ceph-mon[87351]: osdmap e54: 6 total, 6 up, 6 in 2024-03-24T00:56:17.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:56:17.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:17.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:17 smithi038 ceph-mon[112108]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:56:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:17 smithi038 ceph-mon[112108]: osd.0 [v2:172.21.15.38:6802/1079856560,v1:172.21.15.38:6803/1079856560] boot 2024-03-24T00:56:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:17 smithi038 ceph-mon[112108]: osdmap e54: 6 total, 6 up, 6 in 2024-03-24T00:56:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T00:56:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:17.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:18.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:18 smithi038 ceph-mon[112108]: pgmap v77: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:18.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:18 smithi038 ceph-mon[112108]: osdmap e55: 6 total, 6 up, 6 in 2024-03-24T00:56:18.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:18 smithi088 ceph-mon[87351]: pgmap v77: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 52/273 objects degraded (19.048%) 2024-03-24T00:56:18.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:18 smithi088 ceph-mon[87351]: osdmap e55: 6 total, 6 up, 6 in 2024-03-24T00:56:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:19 smithi038 ceph-mon[112108]: pgmap v79: 65 pgs: 2 active+undersized+degraded, 7 active+undersized, 56 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 7/273 objects degraded (2.564%) 2024-03-24T00:56:19.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:19 smithi088 ceph-mon[87351]: pgmap v79: 65 pgs: 2 active+undersized+degraded, 7 active+undersized, 56 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 7/273 objects degraded (2.564%) 2024-03-24T00:56:20.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:20 smithi038 ceph-mon[112108]: Health check update: Degraded data redundancy: 7/273 objects degraded (2.564%), 2 pgs degraded (PG_DEGRADED) 2024-03-24T00:56:20.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:20 smithi088 ceph-mon[87351]: Health check update: Degraded data redundancy: 7/273 objects degraded (2.564%), 2 pgs degraded (PG_DEGRADED) 2024-03-24T00:56:21.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:21 smithi038 ceph-mon[112108]: pgmap v80: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:56:21.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:21 smithi088 ceph-mon[87351]: pgmap v80: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:56:22.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:22 smithi038 ceph-mon[112108]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/273 objects degraded (2.564%), 2 pgs degraded) 2024-03-24T00:56:22.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:22 smithi038 ceph-mon[112108]: Cluster is now healthy 2024-03-24T00:56:22.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:22 smithi088 ceph-mon[87351]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/273 objects degraded (2.564%), 2 pgs degraded) 2024-03-24T00:56:22.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:22 smithi088 ceph-mon[87351]: Cluster is now healthy 2024-03-24T00:56:23.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.412+0000 7f80ac200700 1 -- 172.21.15.38:0/804033794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80a4101c10 msgr2=0x7f80a4101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:23.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.412+0000 7f80ac200700 1 --2- 172.21.15.38:0/804033794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80a4101c10 0x7f80a4101ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8098009910 tx=0x7f8098009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:23.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.412+0000 7f80ac200700 1 -- 172.21.15.38:0/804033794 shutdown_connections 2024-03-24T00:56:23.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.412+0000 7f80ac200700 1 --2- 172.21.15.38:0/804033794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a4102530 0x7f80a410aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:23.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.412+0000 7f80ac200700 1 --2- 172.21.15.38:0/804033794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80a4101c10 0x7f80a4101ff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:23.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.412+0000 7f80ac200700 1 -- 172.21.15.38:0/804033794 >> 172.21.15.38:0/804033794 conn(0x7f80a4076640 msgr2=0x7f80a40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:23.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.412+0000 7f80ac200700 1 -- 172.21.15.38:0/804033794 shutdown_connections 2024-03-24T00:56:23.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.413+0000 7f80ac200700 1 -- 172.21.15.38:0/804033794 wait complete. 2024-03-24T00:56:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.414+0000 7f80ac200700 1 Processor -- start 2024-03-24T00:56:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.414+0000 7f80ac200700 1 -- start start 2024-03-24T00:56:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.414+0000 7f80ac200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a4101c10 0x7f80a40ff870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.414+0000 7f80ac200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80a4102530 0x7f80a40ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.414+0000 7f80ac200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a41002f0 con 0x7f80a4102530 2024-03-24T00:56:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.414+0000 7f80ac200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a4100430 con 0x7f80a4101c10 2024-03-24T00:56:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.415+0000 7f80a8e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80a4102530 0x7f80a40ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.415+0000 7f80a8e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80a4102530 0x7f80a40ffdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60454/0 (socket says 172.21.15.38:60454) 2024-03-24T00:56:23.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.415+0000 7f80a8e00700 1 -- 172.21.15.38:0/3038714600 learned_addr learned my addr 172.21.15.38:0/3038714600 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:23.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.415+0000 7f80a9800700 1 --2- 172.21.15.38:0/3038714600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a4101c10 0x7f80a40ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:23.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.415+0000 7f80a8e00700 1 -- 172.21.15.38:0/3038714600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a4101c10 msgr2=0x7f80a40ff870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.415+0000 7f80a8e00700 1 --2- 172.21.15.38:0/3038714600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a4101c10 0x7f80a40ff870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.415+0000 7f80a8e00700 1 -- 172.21.15.38:0/3038714600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80980095f0 con 0x7f80a4102530 2024-03-24T00:56:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.416+0000 7f80a8e00700 1 --2- 172.21.15.38:0/3038714600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80a4102530 0x7f80a40ffdb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f809000d6d0 tx=0x7f809000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.416+0000 7f80a2000700 1 -- 172.21.15.38:0/3038714600 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809000f860 con 0x7f80a4102530 2024-03-24T00:56:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.416+0000 7f80a2000700 1 -- 172.21.15.38:0/3038714600 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f809000fe80 con 0x7f80a4102530 2024-03-24T00:56:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.416+0000 7f80ac200700 1 -- 172.21.15.38:0/3038714600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a41a2c10 con 0x7f80a4102530 2024-03-24T00:56:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.416+0000 7f80a2000700 1 -- 172.21.15.38:0/3038714600 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8090010ea0 con 0x7f80a4102530 2024-03-24T00:56:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.416+0000 7f80ac200700 1 -- 172.21.15.38:0/3038714600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a41a2f70 con 0x7f80a4102530 2024-03-24T00:56:23.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.418+0000 7f80ac200700 1 -- 172.21.15.38:0/3038714600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a4068d60 con 0x7f80a4102530 2024-03-24T00:56:23.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.419+0000 7f80a2000700 1 -- 172.21.15.38:0/3038714600 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f8090010460 con 0x7f80a4102530 2024-03-24T00:56:23.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.420+0000 7f80a2000700 1 --2- 172.21.15.38:0/3038714600 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f808c06dfb0 0x7f808c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:23.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.420+0000 7f80a2000700 1 -- 172.21.15.38:0/3038714600 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f809008dab0 con 0x7f80a4102530 2024-03-24T00:56:23.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.420+0000 7f80a9800700 1 --2- 172.21.15.38:0/3038714600 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f808c06dfb0 0x7f808c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:23.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.421+0000 7f80a9800700 1 --2- 172.21.15.38:0/3038714600 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f808c06dfb0 0x7f808c070470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8098006010 tx=0x7f809801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:23.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.425+0000 7f80a2000700 1 -- 172.21.15.38:0/3038714600 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f809008fe60 con 0x7f80a4102530 2024-03-24T00:56:23.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.703+0000 7f80ac200700 1 -- 172.21.15.38:0/3038714600 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80a4107d30 con 0x7f808c06dfb0 2024-03-24T00:56:23.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.705+0000 7f80a2000700 1 -- 172.21.15.38:0/3038714600 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f80a4107d30 con 0x7f808c06dfb0 2024-03-24T00:56:23.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.708+0000 7f80ac200700 1 -- 172.21.15.38:0/3038714600 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f808c06dfb0 msgr2=0x7f808c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:23.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.708+0000 7f80ac200700 1 --2- 172.21.15.38:0/3038714600 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f808c06dfb0 0x7f808c070470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8098006010 tx=0x7f809801a040 comp rx=0 tx=0).stop 2024-03-24T00:56:23.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.708+0000 7f80ac200700 1 -- 172.21.15.38:0/3038714600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80a4102530 msgr2=0x7f80a40ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:23.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.708+0000 7f80ac200700 1 --2- 172.21.15.38:0/3038714600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80a4102530 0x7f80a40ffdb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f809000d6d0 tx=0x7f809000d9e0 comp rx=0 tx=0).stop 2024-03-24T00:56:23.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.708+0000 7f80ac200700 1 -- 172.21.15.38:0/3038714600 shutdown_connections 2024-03-24T00:56:23.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.708+0000 7f80ac200700 1 --2- 172.21.15.38:0/3038714600 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f808c06dfb0 0x7f808c070470 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:23.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.708+0000 7f80ac200700 1 --2- 172.21.15.38:0/3038714600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a4101c10 0x7f80a40ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:23.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.708+0000 7f80ac200700 1 --2- 172.21.15.38:0/3038714600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80a4102530 0x7f80a40ffdb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:23.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.708+0000 7f80ac200700 1 -- 172.21.15.38:0/3038714600 >> 172.21.15.38:0/3038714600 conn(0x7f80a4076640 msgr2=0x7f80a4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:23.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.708+0000 7f80ac200700 1 -- 172.21.15.38:0/3038714600 shutdown_connections 2024-03-24T00:56:23.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.709+0000 7f80ac200700 1 -- 172.21.15.38:0/3038714600 wait complete. 2024-03-24T00:56:23.726 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:56:23.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.865+0000 7f972e400700 1 -- 172.21.15.38:0/531815958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97280ff7a0 msgr2=0x7f97280ffb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:23.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.865+0000 7f972e400700 1 --2- 172.21.15.38:0/531815958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97280ff7a0 0x7f97280ffb80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9718009910 tx=0x7f9718009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:23.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.865+0000 7f972e400700 1 -- 172.21.15.38:0/531815958 shutdown_connections 2024-03-24T00:56:23.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.865+0000 7f972e400700 1 --2- 172.21.15.38:0/531815958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9728100150 0x7f972810ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:23.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.865+0000 7f972e400700 1 --2- 172.21.15.38:0/531815958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97280ff7a0 0x7f97280ffb80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.865+0000 7f972e400700 1 -- 172.21.15.38:0/531815958 >> 172.21.15.38:0/531815958 conn(0x7f9728076640 msgr2=0x7f97280fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.865+0000 7f972e400700 1 -- 172.21.15.38:0/531815958 shutdown_connections 2024-03-24T00:56:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.865+0000 7f972e400700 1 -- 172.21.15.38:0/531815958 wait complete. 2024-03-24T00:56:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.866+0000 7f972e400700 1 Processor -- start 2024-03-24T00:56:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.866+0000 7f972e400700 1 -- start start 2024-03-24T00:56:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.866+0000 7f972e400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97280ff7a0 0x7f972819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.866+0000 7f972e400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9728100150 0x7f972819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.866+0000 7f972e400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972819f360 con 0x7f9728100150 2024-03-24T00:56:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.866+0000 7f972e400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97281989d0 con 0x7f97280ff7a0 2024-03-24T00:56:23.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.867+0000 7f9726a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9728100150 0x7f972819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:23.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.867+0000 7f9726a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9728100150 0x7f972819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60474/0 (socket says 172.21.15.38:60474) 2024-03-24T00:56:23.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.867+0000 7f9726a00700 1 -- 172.21.15.38:0/1009706165 learned_addr learned my addr 172.21.15.38:0/1009706165 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:23.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.867+0000 7f9727400700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97280ff7a0 0x7f972819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:23.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.867+0000 7f9726a00700 1 -- 172.21.15.38:0/1009706165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97280ff7a0 msgr2=0x7f972819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:23.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.867+0000 7f9726a00700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97280ff7a0 0x7f972819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:23.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.867+0000 7f9726a00700 1 -- 172.21.15.38:0/1009706165 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97180095f0 con 0x7f9728100150 2024-03-24T00:56:23.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.867+0000 7f9727400700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97280ff7a0 0x7f972819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:56:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.868+0000 7f9726a00700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9728100150 0x7f972819edd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f971400c930 tx=0x7f971400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.868+0000 7f971fe00700 1 -- 172.21.15.38:0/1009706165 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9714007ad0 con 0x7f9728100150 2024-03-24T00:56:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.868+0000 7f971fe00700 1 -- 172.21.15.38:0/1009706165 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9714007c30 con 0x7f9728100150 2024-03-24T00:56:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.868+0000 7f971fe00700 1 -- 172.21.15.38:0/1009706165 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9714018610 con 0x7f9728100150 2024-03-24T00:56:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.868+0000 7f972e400700 1 -- 172.21.15.38:0/1009706165 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9728198cb0 con 0x7f9728100150 2024-03-24T00:56:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.868+0000 7f972e400700 1 -- 172.21.15.38:0/1009706165 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97281991b0 con 0x7f9728100150 2024-03-24T00:56:23.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.869+0000 7f972e400700 1 -- 172.21.15.38:0/1009706165 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9728068d60 con 0x7f9728100150 2024-03-24T00:56:23.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.870+0000 7f971fe00700 1 -- 172.21.15.38:0/1009706165 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f971400f450 con 0x7f9728100150 2024-03-24T00:56:23.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.871+0000 7f971fe00700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f970c06dfa0 0x7f970c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:23.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.871+0000 7f971fe00700 1 -- 172.21.15.38:0/1009706165 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f971408df20 con 0x7f9728100150 2024-03-24T00:56:23.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.872+0000 7f9727400700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f970c06dfa0 0x7f970c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:23.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.872+0000 7f9727400700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f970c06dfa0 0x7f970c070460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f971800b5c0 tx=0x7f971800b7f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:23.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:23.880+0000 7f971fe00700 1 -- 172.21.15.38:0/1009706165 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f971405af00 con 0x7f9728100150 2024-03-24T00:56:24.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.146+0000 7f972e400700 1 -- 172.21.15.38:0/1009706165 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9728199590 con 0x7f970c06dfa0 2024-03-24T00:56:24.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.148+0000 7f971fe00700 1 -- 172.21.15.38:0/1009706165 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9728199590 con 0x7f970c06dfa0 2024-03-24T00:56:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.150+0000 7f972e400700 1 -- 172.21.15.38:0/1009706165 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f970c06dfa0 msgr2=0x7f970c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.150+0000 7f972e400700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f970c06dfa0 0x7f970c070460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f971800b5c0 tx=0x7f971800b7f0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.150+0000 7f972e400700 1 -- 172.21.15.38:0/1009706165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9728100150 msgr2=0x7f972819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.150+0000 7f972e400700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9728100150 0x7f972819edd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f971400c930 tx=0x7f971400ccf0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.150+0000 7f972e400700 1 -- 172.21.15.38:0/1009706165 shutdown_connections 2024-03-24T00:56:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.150+0000 7f972e400700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f970c06dfa0 0x7f970c070460 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.150+0000 7f972e400700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97280ff7a0 0x7f972819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.150+0000 7f972e400700 1 --2- 172.21.15.38:0/1009706165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9728100150 0x7f972819edd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.150+0000 7f972e400700 1 -- 172.21.15.38:0/1009706165 >> 172.21.15.38:0/1009706165 conn(0x7f9728076640 msgr2=0x7f9728109580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.151+0000 7f972e400700 1 -- 172.21.15.38:0/1009706165 shutdown_connections 2024-03-24T00:56:24.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.151+0000 7f972e400700 1 -- 172.21.15.38:0/1009706165 wait complete. 2024-03-24T00:56:24.252 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: pgmap v81: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-24T00:56:24.252 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:24.252 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:24.252 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:56:24.253 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:56:24.253 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:24.253 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:56:24.253 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:24.253 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:24.253 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:24.253 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T00:56:24.253 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T00:56:24.253 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:23 smithi038 ceph-mon[112108]: Upgrade: osd.1 is safe to restart 2024-03-24T00:56:24.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.310+0000 7fd7fde00700 1 -- 172.21.15.38:0/698940943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f81000e0 msgr2=0x7fd7f810ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:24.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.310+0000 7fd7fde00700 1 --2- 172.21.15.38:0/698940943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f81000e0 0x7fd7f810ce30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4009910 tx=0x7fd7e4009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:24.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.310+0000 7fd7fde00700 1 -- 172.21.15.38:0/698940943 shutdown_connections 2024-03-24T00:56:24.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.310+0000 7fd7fde00700 1 --2- 172.21.15.38:0/698940943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f81000e0 0x7fd7f810ce30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.310+0000 7fd7fde00700 1 --2- 172.21.15.38:0/698940943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7f80ff7c0 0x7fd7f80ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.310+0000 7fd7fde00700 1 -- 172.21.15.38:0/698940943 >> 172.21.15.38:0/698940943 conn(0x7fd7f80fb480 msgr2=0x7fd7f80fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:24.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.310+0000 7fd7fde00700 1 -- 172.21.15.38:0/698940943 shutdown_connections 2024-03-24T00:56:24.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.310+0000 7fd7fde00700 1 -- 172.21.15.38:0/698940943 wait complete. 2024-03-24T00:56:24.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.311+0000 7fd7fde00700 1 Processor -- start 2024-03-24T00:56:24.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.311+0000 7fd7fde00700 1 -- start start 2024-03-24T00:56:24.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.312+0000 7fd7fde00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7f80ff7c0 0x7fd7f819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:24.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.312+0000 7fd7fde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f81000e0 0x7fd7f819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:24.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.312+0000 7fd7fde00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f819f520 con 0x7fd7f81000e0 2024-03-24T00:56:24.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.312+0000 7fd7fde00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8198b50 con 0x7fd7f80ff7c0 2024-03-24T00:56:24.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.312+0000 7fd7f6000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f81000e0 0x7fd7f819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:24.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.312+0000 7fd7f6000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f81000e0 0x7fd7f819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60484/0 (socket says 172.21.15.38:60484) 2024-03-24T00:56:24.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.312+0000 7fd7f6000700 1 -- 172.21.15.38:0/207737959 learned_addr learned my addr 172.21.15.38:0/207737959 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:24.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.313+0000 7fd7f6a00700 1 --2- 172.21.15.38:0/207737959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7f80ff7c0 0x7fd7f819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:24.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.313+0000 7fd7f6000700 1 -- 172.21.15.38:0/207737959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7f80ff7c0 msgr2=0x7fd7f819e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:24.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.313+0000 7fd7f6000700 1 --2- 172.21.15.38:0/207737959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7f80ff7c0 0x7fd7f819e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.313+0000 7fd7f6000700 1 -- 172.21.15.38:0/207737959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e40095f0 con 0x7fd7f81000e0 2024-03-24T00:56:24.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.313+0000 7fd7f6000700 1 --2- 172.21.15.38:0/207737959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f81000e0 0x7fd7f819ef00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4006010 tx=0x7fd7e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:24.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.313+0000 7fd7fca00700 1 -- 172.21.15.38:0/207737959 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7e401d070 con 0x7fd7f81000e0 2024-03-24T00:56:24.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.314+0000 7fd7fca00700 1 -- 172.21.15.38:0/207737959 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7e400f460 con 0x7fd7f81000e0 2024-03-24T00:56:24.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.314+0000 7fd7fde00700 1 -- 172.21.15.38:0/207737959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f8198dd0 con 0x7fd7f81000e0 2024-03-24T00:56:24.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.314+0000 7fd7fca00700 1 -- 172.21.15.38:0/207737959 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7e4017630 con 0x7fd7f81000e0 2024-03-24T00:56:24.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.314+0000 7fd7fde00700 1 -- 172.21.15.38:0/207737959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f81992c0 con 0x7fd7f81000e0 2024-03-24T00:56:24.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.315+0000 7fd7fde00700 1 -- 172.21.15.38:0/207737959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f810acb0 con 0x7fd7f81000e0 2024-03-24T00:56:24.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.316+0000 7fd7fca00700 1 -- 172.21.15.38:0/207737959 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd7e400f5d0 con 0x7fd7f81000e0 2024-03-24T00:56:24.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.317+0000 7fd7fca00700 1 --2- 172.21.15.38:0/207737959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd7dc06dfb0 0x7fd7dc070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:24.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.317+0000 7fd7fca00700 1 -- 172.21.15.38:0/207737959 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd7e408f250 con 0x7fd7f81000e0 2024-03-24T00:56:24.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.317+0000 7fd7f6a00700 1 --2- 172.21.15.38:0/207737959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd7dc06dfb0 0x7fd7dc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:24.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.318+0000 7fd7f6a00700 1 --2- 172.21.15.38:0/207737959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd7dc06dfb0 0x7fd7dc070470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8006fd0 tx=0x7fd7e8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:24.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.322+0000 7fd7fca00700 1 -- 172.21.15.38:0/207737959 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd7e405c270 con 0x7fd7f81000e0 2024-03-24T00:56:24.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: pgmap v81: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-24T00:56:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:56:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:56:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:56:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:24.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:24.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:24.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T00:56:24.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T00:56:24.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:23 smithi088 ceph-mon[87351]: Upgrade: osd.1 is safe to restart 2024-03-24T00:56:24.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.565+0000 7fd7fde00700 1 -- 172.21.15.38:0/207737959 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7f8199fc0 con 0x7fd7dc06dfb0 2024-03-24T00:56:24.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.599+0000 7fd7fca00700 1 -- 172.21.15.38:0/207737959 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd7f8199fc0 con 0x7fd7dc06dfb0 2024-03-24T00:56:24.600 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:56:24.600 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (4m) 9s ago 18m 13.4M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:56:24.600 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (19m) 9s ago 19m 8627k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:56:24.600 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (17m) 43s ago 17m 8363k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:56:24.600 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (56s) 9s ago 19m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T00:56:24.600 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (48s) 43s ago 17m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T00:56:24.601 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (4m) 9s ago 18m 72.9M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:56:24.601 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (11m) 9s ago 11m 97.9M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:56:24.601 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (11m) 9s ago 11m 150M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:56:24.601 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (11m) 43s ago 11m 24.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:56:24.601 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (11m) 43s ago 11m 24.8M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:56:24.601 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (7m) 9s ago 20m 499M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:56:24.601 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (6m) 43s ago 17m 449M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:56:24.602 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (2m) 9s ago 20m 47.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T00:56:24.602 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (2m) 43s ago 17m 41.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T00:56:24.602 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (5m) 9s ago 18m 8539k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:56:24.602 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (5m) 43s ago 17m 7755k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:56:24.602 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (17s) 9s ago 16m 11.4M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T00:56:24.602 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (15m) 9s ago 15m 272M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 925408492cb5 2024-03-24T00:56:24.602 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (15m) 9s ago 15m 228M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:56:24.602 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (14m) 43s ago 14m 304M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:56:24.603 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (13m) 43s ago 13m 283M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:56:24.603 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (13m) 43s ago 13m 254M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:56:24.603 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (5m) 9s ago 18m 70.0M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:56:24.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.602+0000 7fd7fde00700 1 -- 172.21.15.38:0/207737959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd7dc06dfb0 msgr2=0x7fd7dc070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:24.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.602+0000 7fd7fde00700 1 --2- 172.21.15.38:0/207737959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd7dc06dfb0 0x7fd7dc070470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8006fd0 tx=0x7fd7e8008040 comp rx=0 tx=0).stop 2024-03-24T00:56:24.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.602+0000 7fd7fde00700 1 -- 172.21.15.38:0/207737959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f81000e0 msgr2=0x7fd7f819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:24.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.602+0000 7fd7fde00700 1 --2- 172.21.15.38:0/207737959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f81000e0 0x7fd7f819ef00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4006010 tx=0x7fd7e400be20 comp rx=0 tx=0).stop 2024-03-24T00:56:24.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.602+0000 7fd7fde00700 1 -- 172.21.15.38:0/207737959 shutdown_connections 2024-03-24T00:56:24.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.602+0000 7fd7fde00700 1 --2- 172.21.15.38:0/207737959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd7dc06dfb0 0x7fd7dc070470 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.602+0000 7fd7fde00700 1 --2- 172.21.15.38:0/207737959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7f80ff7c0 0x7fd7f819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.602+0000 7fd7fde00700 1 --2- 172.21.15.38:0/207737959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f81000e0 0x7fd7f819ef00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.602+0000 7fd7fde00700 1 -- 172.21.15.38:0/207737959 >> 172.21.15.38:0/207737959 conn(0x7fd7f80fb480 msgr2=0x7fd7f8107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:24.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.602+0000 7fd7fde00700 1 -- 172.21.15.38:0/207737959 shutdown_connections 2024-03-24T00:56:24.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.602+0000 7fd7fde00700 1 -- 172.21.15.38:0/207737959 wait complete. 2024-03-24T00:56:24.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.752+0000 7f22f6a00700 1 -- 172.21.15.38:0/1633585179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f0102570 msgr2=0x7f22f010aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:24.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.752+0000 7f22f6a00700 1 --2- 172.21.15.38:0/1633585179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f0102570 0x7f22f010aa60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009910 tx=0x7f22dc009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:24.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.752+0000 7f22f6a00700 1 -- 172.21.15.38:0/1633585179 shutdown_connections 2024-03-24T00:56:24.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.752+0000 7f22f6a00700 1 --2- 172.21.15.38:0/1633585179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f0102570 0x7f22f010aa60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.752+0000 7f22f6a00700 1 --2- 172.21.15.38:0/1633585179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f0101c50 0x7f22f0102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.752+0000 7f22f6a00700 1 -- 172.21.15.38:0/1633585179 >> 172.21.15.38:0/1633585179 conn(0x7f22f0076640 msgr2=0x7f22f00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:24.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.752+0000 7f22f6a00700 1 -- 172.21.15.38:0/1633585179 shutdown_connections 2024-03-24T00:56:24.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.752+0000 7f22f6a00700 1 -- 172.21.15.38:0/1633585179 wait complete. 2024-03-24T00:56:24.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.753+0000 7f22f6a00700 1 Processor -- start 2024-03-24T00:56:24.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.753+0000 7f22f6a00700 1 -- start start 2024-03-24T00:56:24.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22f6a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f0101c50 0x7f22f019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:24.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22f6a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f0102570 0x7f22f019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:24.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22f6a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f01989e0 con 0x7f22f0101c50 2024-03-24T00:56:24.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22f6a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f0198b50 con 0x7f22f0102570 2024-03-24T00:56:24.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22efe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f0101c50 0x7f22f019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:24.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22efe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f0101c50 0x7f22f019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60504/0 (socket says 172.21.15.38:60504) 2024-03-24T00:56:24.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22efe00700 1 -- 172.21.15.38:0/2528116961 learned_addr learned my addr 172.21.15.38:0/2528116961 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:24.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22ef400700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f0102570 0x7f22f019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:24.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22efe00700 1 -- 172.21.15.38:0/2528116961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f0102570 msgr2=0x7f22f019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:24.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22efe00700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f0102570 0x7f22f019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:24.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22efe00700 1 -- 172.21.15.38:0/2528116961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc0095f0 con 0x7f22f0101c50 2024-03-24T00:56:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.754+0000 7f22ef400700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f0102570 0x7f22f019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:56:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.755+0000 7f22efe00700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f0101c50 0x7f22f019e930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f22e000d8d0 tx=0x7f22e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.755+0000 7f22ecc00700 1 -- 172.21.15.38:0/2528116961 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22e0009940 con 0x7f22f0101c50 2024-03-24T00:56:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.756+0000 7f22f6a00700 1 -- 172.21.15.38:0/2528116961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f0198e30 con 0x7f22f0101c50 2024-03-24T00:56:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.755+0000 7f22ecc00700 1 -- 172.21.15.38:0/2528116961 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22e0010460 con 0x7f22f0101c50 2024-03-24T00:56:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.756+0000 7f22ecc00700 1 -- 172.21.15.38:0/2528116961 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22e000f5f0 con 0x7f22f0101c50 2024-03-24T00:56:24.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.756+0000 7f22f6a00700 1 -- 172.21.15.38:0/2528116961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f0199350 con 0x7f22f0101c50 2024-03-24T00:56:24.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.757+0000 7f22f6a00700 1 -- 172.21.15.38:0/2528116961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f0068d20 con 0x7f22f0101c50 2024-03-24T00:56:24.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.759+0000 7f22ecc00700 1 -- 172.21.15.38:0/2528116961 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f22e0009aa0 con 0x7f22f0101c50 2024-03-24T00:56:24.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.760+0000 7f22ecc00700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f22d406dfb0 0x7f22d4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:24.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.760+0000 7f22ecc00700 1 -- 172.21.15.38:0/2528116961 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f22e008dd70 con 0x7f22f0101c50 2024-03-24T00:56:24.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.760+0000 7f22ef400700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f22d406dfb0 0x7f22d4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:24.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.761+0000 7f22ef400700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f22d406dfb0 0x7f22d4070470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22f019a1e0 tx=0x7f22dc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:24.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:24.769+0000 7f22ecc00700 1 -- 172.21.15.38:0/2528116961 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f22e00571a0 con 0x7f22f0101c50 2024-03-24T00:56:25.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.113+0000 7f22f6a00700 1 -- 172.21.15.38:0/2528116961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f22f0199570 con 0x7f22f0101c50 2024-03-24T00:56:25.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.115+0000 7f22ecc00700 1 -- 172.21.15.38:0/2528116961 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f22e005a860 con 0x7f22f0101c50 2024-03-24T00:56:25.115 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:56:25.115 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:56:25.115 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:56:25.115 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:56:25.116 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:56:25.116 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:56:25.116 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:56:25.116 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:56:25.116 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 5, 2024-03-24T00:56:25.116 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-24T00:56:25.116 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:56:25.116 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:56:25.116 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:56:25.116 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:56:25.116 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:56:25.117 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 9, 2024-03-24T00:56:25.117 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 5 2024-03-24T00:56:25.117 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:56:25.117 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:56:25.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.117+0000 7f22f6a00700 1 -- 172.21.15.38:0/2528116961 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f22d406dfb0 msgr2=0x7f22d4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:25.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.117+0000 7f22f6a00700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f22d406dfb0 0x7f22d4070470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22f019a1e0 tx=0x7f22dc01a040 comp rx=0 tx=0).stop 2024-03-24T00:56:25.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.117+0000 7f22f6a00700 1 -- 172.21.15.38:0/2528116961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f0101c50 msgr2=0x7f22f019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:25.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.117+0000 7f22f6a00700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f0101c50 0x7f22f019e930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f22e000d8d0 tx=0x7f22e000dc90 comp rx=0 tx=0).stop 2024-03-24T00:56:25.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.118+0000 7f22f6a00700 1 -- 172.21.15.38:0/2528116961 shutdown_connections 2024-03-24T00:56:25.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.118+0000 7f22f6a00700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f22d406dfb0 0x7f22d4070470 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.118+0000 7f22f6a00700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22f0101c50 0x7f22f019e930 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.118+0000 7f22f6a00700 1 --2- 172.21.15.38:0/2528116961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f0102570 0x7f22f019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.118+0000 7f22f6a00700 1 -- 172.21.15.38:0/2528116961 >> 172.21.15.38:0/2528116961 conn(0x7f22f0076640 msgr2=0x7f22f010a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:25.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.118+0000 7f22f6a00700 1 -- 172.21.15.38:0/2528116961 shutdown_connections 2024-03-24T00:56:25.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.118+0000 7f22f6a00700 1 -- 172.21.15.38:0/2528116961 wait complete. 2024-03-24T00:56:25.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.269+0000 7f6933400700 1 -- 172.21.15.38:0/387735214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f692c0ff800 msgr2=0x7f692c0ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:25.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.269+0000 7f6933400700 1 --2- 172.21.15.38:0/387735214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f692c0ff800 0x7f692c0ffbe0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6920009910 tx=0x7f6920009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:25.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.269+0000 7f6933400700 1 -- 172.21.15.38:0/387735214 shutdown_connections 2024-03-24T00:56:25.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.269+0000 7f6933400700 1 --2- 172.21.15.38:0/387735214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c1001b0 0x7f692c10ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.269+0000 7f6933400700 1 --2- 172.21.15.38:0/387735214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f692c0ff800 0x7f692c0ffbe0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.269+0000 7f6933400700 1 -- 172.21.15.38:0/387735214 >> 172.21.15.38:0/387735214 conn(0x7f692c076640 msgr2=0x7f692c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:25.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:24 smithi038 ceph-mon[112108]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:25.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:25 smithi038 ceph-mon[112108]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:25.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.269+0000 7f6933400700 1 -- 172.21.15.38:0/387735214 shutdown_connections 2024-03-24T00:56:25.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.269+0000 7f6933400700 1 -- 172.21.15.38:0/387735214 wait complete. 2024-03-24T00:56:25.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.270+0000 7f6933400700 1 Processor -- start 2024-03-24T00:56:25.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.271+0000 7f6933400700 1 -- start start 2024-03-24T00:56:25.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.271+0000 7f6933400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff800 0x7f692c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:25.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.271+0000 7f6933400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f692c1001b0 0x7f692c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.271+0000 7f6933400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c19f3a0 con 0x7f692c1001b0 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.271+0000 7f6933400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c198980 con 0x7f692c0ff800 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.272+0000 7f692be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f692c1001b0 0x7f692c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.272+0000 7f692be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f692c1001b0 0x7f692c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60510/0 (socket says 172.21.15.38:60510) 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.272+0000 7f692be00700 1 -- 172.21.15.38:0/4005111736 learned_addr learned my addr 172.21.15.38:0/4005111736 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.272+0000 7f6930a00700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff800 0x7f692c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.272+0000 7f692be00700 1 -- 172.21.15.38:0/4005111736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff800 msgr2=0x7f692c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.272+0000 7f692be00700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff800 0x7f692c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.272+0000 7f692be00700 1 -- 172.21.15.38:0/4005111736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200095f0 con 0x7f692c1001b0 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.272+0000 7f6930a00700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff800 0x7f692c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.272+0000 7f692be00700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f692c1001b0 0x7f692c19ed80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f691800c8a0 tx=0x7f691800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.273+0000 7f6929600700 1 -- 172.21.15.38:0/4005111736 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691800cea0 con 0x7f692c1001b0 2024-03-24T00:56:25.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.273+0000 7f6929600700 1 -- 172.21.15.38:0/4005111736 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6918004830 con 0x7f692c1001b0 2024-03-24T00:56:25.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.273+0000 7f6933400700 1 -- 172.21.15.38:0/4005111736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c198c60 con 0x7f692c1001b0 2024-03-24T00:56:25.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.273+0000 7f6929600700 1 -- 172.21.15.38:0/4005111736 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69180055b0 con 0x7f692c1001b0 2024-03-24T00:56:25.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.273+0000 7f6933400700 1 -- 172.21.15.38:0/4005111736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c1991b0 con 0x7f692c1001b0 2024-03-24T00:56:25.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.274+0000 7f6933400700 1 -- 172.21.15.38:0/4005111736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692c068d00 con 0x7f692c1001b0 2024-03-24T00:56:25.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.275+0000 7f6929600700 1 -- 172.21.15.38:0/4005111736 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f69180049a0 con 0x7f692c1001b0 2024-03-24T00:56:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.276+0000 7f6929600700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f691406dfb0 0x7f6914070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.276+0000 7f6929600700 1 -- 172.21.15.38:0/4005111736 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f691808da50 con 0x7f692c1001b0 2024-03-24T00:56:25.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.277+0000 7f6930a00700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f691406dfb0 0x7f6914070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:25.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.277+0000 7f6930a00700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f691406dfb0 0x7f6914070470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6920005b40 tx=0x7f6920005ab0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:25.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.284+0000 7f6929600700 1 -- 172.21.15.38:0/4005111736 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f691805a9b0 con 0x7f692c1001b0 2024-03-24T00:56:25.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:25 smithi088 ceph-mon[87351]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:25.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:25 smithi088 ceph-mon[87351]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:25.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.555+0000 7f6933400700 1 -- 172.21.15.38:0/4005111736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f692c198df0 con 0x7f692c1001b0 2024-03-24T00:56:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.556+0000 7f6929600700 1 -- 172.21.15.38:0/4005111736 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1900 (secure 0 0 0) 0x7f691801d780 con 0x7f692c1001b0 2024-03-24T00:56:25.557 INFO:teuthology.orchestra.run.smithi038.stdout:e20 2024-03-24T00:56:25.557 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:56:25.558 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:56:25.558 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:56:25.558 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:56:25.559 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:56:25.559 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:56:25.559 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 20 2024-03-24T00:56:25.559 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:56:25.560 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:56:25.560 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:56:25.560 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:56:25.561 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:56:25.562 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:56:25.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.558+0000 7f6933400700 1 -- 172.21.15.38:0/4005111736 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f691406dfb0 msgr2=0x7f6914070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:25.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.558+0000 7f6933400700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f691406dfb0 0x7f6914070470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6920005b40 tx=0x7f6920005ab0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.558+0000 7f6933400700 1 -- 172.21.15.38:0/4005111736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f692c1001b0 msgr2=0x7f692c19ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:25.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.559+0000 7f6933400700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f692c1001b0 0x7f692c19ed80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f691800c8a0 tx=0x7f691800cc60 comp rx=0 tx=0).stop 2024-03-24T00:56:25.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.559+0000 7f6933400700 1 -- 172.21.15.38:0/4005111736 shutdown_connections 2024-03-24T00:56:25.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.559+0000 7f6933400700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f691406dfb0 0x7f6914070470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.559+0000 7f6933400700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff800 0x7f692c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.559+0000 7f6933400700 1 --2- 172.21.15.38:0/4005111736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f692c1001b0 0x7f692c19ed80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.559+0000 7f6933400700 1 -- 172.21.15.38:0/4005111736 >> 172.21.15.38:0/4005111736 conn(0x7f692c076640 msgr2=0x7f692c1095e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:25.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.559+0000 7f6933400700 1 -- 172.21.15.38:0/4005111736 shutdown_connections 2024-03-24T00:56:25.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.559+0000 7f6933400700 1 -- 172.21.15.38:0/4005111736 wait complete. 2024-03-24T00:56:25.564 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:56:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.724+0000 7f09a3600700 1 -- 172.21.15.38:0/1396302454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f099c0739f0 msgr2=0x7f099c10cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.724+0000 7f09a3600700 1 --2- 172.21.15.38:0/1396302454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f099c0739f0 0x7f099c10cdf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0988009910 tx=0x7f0988009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.724+0000 7f09a3600700 1 -- 172.21.15.38:0/1396302454 shutdown_connections 2024-03-24T00:56:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.724+0000 7f09a3600700 1 --2- 172.21.15.38:0/1396302454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f099c0739f0 0x7f099c10cdf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.724+0000 7f09a3600700 1 --2- 172.21.15.38:0/1396302454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f099c0730d0 0x7f099c0734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.724+0000 7f09a3600700 1 -- 172.21.15.38:0/1396302454 >> 172.21.15.38:0/1396302454 conn(0x7f099c0fc580 msgr2=0x7f099c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.724+0000 7f09a3600700 1 -- 172.21.15.38:0/1396302454 shutdown_connections 2024-03-24T00:56:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.724+0000 7f09a3600700 1 -- 172.21.15.38:0/1396302454 wait complete. 2024-03-24T00:56:25.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.725+0000 7f09a3600700 1 Processor -- start 2024-03-24T00:56:25.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.725+0000 7f09a3600700 1 -- start start 2024-03-24T00:56:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.726+0000 7f09a3600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f099c0730d0 0x7f099c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.726+0000 7f09a3600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f099c0739f0 0x7f099c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.726+0000 7f09a3600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c19f420 con 0x7f099c0739f0 2024-03-24T00:56:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.726+0000 7f09a3600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c198a00 con 0x7f099c0730d0 2024-03-24T00:56:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.726+0000 7f099be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f099c0739f0 0x7f099c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.726+0000 7f099be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f099c0739f0 0x7f099c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60524/0 (socket says 172.21.15.38:60524) 2024-03-24T00:56:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.726+0000 7f099be00700 1 -- 172.21.15.38:0/2796680259 learned_addr learned my addr 172.21.15.38:0/2796680259 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.726+0000 7f09a0c00700 1 --2- 172.21.15.38:0/2796680259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f099c0730d0 0x7f099c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.726+0000 7f099be00700 1 -- 172.21.15.38:0/2796680259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f099c0730d0 msgr2=0x7f099c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.727+0000 7f099be00700 1 --2- 172.21.15.38:0/2796680259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f099c0730d0 0x7f099c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.727+0000 7f099be00700 1 -- 172.21.15.38:0/2796680259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09880095f0 con 0x7f099c0739f0 2024-03-24T00:56:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.727+0000 7f099be00700 1 --2- 172.21.15.38:0/2796680259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f099c0739f0 0x7f099c19ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f098800bd70 tx=0x7f098800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.727+0000 7f0999600700 1 -- 172.21.15.38:0/2796680259 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f098801d070 con 0x7f099c0739f0 2024-03-24T00:56:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.727+0000 7f0999600700 1 -- 172.21.15.38:0/2796680259 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09880056e0 con 0x7f099c0739f0 2024-03-24T00:56:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.727+0000 7f09a3600700 1 -- 172.21.15.38:0/2796680259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c198c80 con 0x7f099c0739f0 2024-03-24T00:56:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.727+0000 7f0999600700 1 -- 172.21.15.38:0/2796680259 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0988017400 con 0x7f099c0739f0 2024-03-24T00:56:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.727+0000 7f09a3600700 1 -- 172.21.15.38:0/2796680259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c199170 con 0x7f099c0739f0 2024-03-24T00:56:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.729+0000 7f09a3600700 1 -- 172.21.15.38:0/2796680259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c107700 con 0x7f099c0739f0 2024-03-24T00:56:25.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.730+0000 7f0999600700 1 -- 172.21.15.38:0/2796680259 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f098800f6f0 con 0x7f099c0739f0 2024-03-24T00:56:25.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.731+0000 7f0999600700 1 --2- 172.21.15.38:0/2796680259 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f098406dfa0 0x7f0984070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:25.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.731+0000 7f0999600700 1 -- 172.21.15.38:0/2796680259 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f098808f130 con 0x7f099c0739f0 2024-03-24T00:56:25.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.731+0000 7f09a0c00700 1 --2- 172.21.15.38:0/2796680259 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f098406dfa0 0x7f0984070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:25.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.732+0000 7f09a0c00700 1 --2- 172.21.15.38:0/2796680259 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f098406dfa0 0x7f0984070460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0990005950 tx=0x7f09900058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:25.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.736+0000 7f0999600700 1 -- 172.21.15.38:0/2796680259 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f098805c110 con 0x7f099c0739f0 2024-03-24T00:56:25.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.992+0000 7f09a3600700 1 -- 172.21.15.38:0/2796680259 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f099c199ec0 con 0x7f098406dfa0 2024-03-24T00:56:25.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.994+0000 7f0999600700 1 -- 172.21.15.38:0/2796680259 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f099c199ec0 con 0x7f098406dfa0 2024-03-24T00:56:25.994 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:56:25.994 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:56:25.995 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:56:25.995 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:56:25.995 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:56:25.995 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T00:56:25.995 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-03-24T00:56:25.995 INFO:teuthology.orchestra.run.smithi038.stdout: "crash" 2024-03-24T00:56:25.995 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:56:25.995 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "7/23 daemons upgraded", 2024-03-24T00:56:25.995 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T00:56:25.995 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:56:25.995 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:56:25.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.997+0000 7f09a3600700 1 -- 172.21.15.38:0/2796680259 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f098406dfa0 msgr2=0x7f0984070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:25.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.997+0000 7f09a3600700 1 --2- 172.21.15.38:0/2796680259 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f098406dfa0 0x7f0984070460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0990005950 tx=0x7f09900058e0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.997+0000 7f09a3600700 1 -- 172.21.15.38:0/2796680259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f099c0739f0 msgr2=0x7f099c19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:25.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.997+0000 7f09a3600700 1 --2- 172.21.15.38:0/2796680259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f099c0739f0 0x7f099c19ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f098800bd70 tx=0x7f098800be50 comp rx=0 tx=0).stop 2024-03-24T00:56:25.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.997+0000 7f09a3600700 1 -- 172.21.15.38:0/2796680259 shutdown_connections 2024-03-24T00:56:25.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.997+0000 7f09a3600700 1 --2- 172.21.15.38:0/2796680259 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f098406dfa0 0x7f0984070460 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.997+0000 7f09a3600700 1 --2- 172.21.15.38:0/2796680259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f099c0730d0 0x7f099c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.997+0000 7f09a3600700 1 --2- 172.21.15.38:0/2796680259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f099c0739f0 0x7f099c19ee00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:25.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.997+0000 7f09a3600700 1 -- 172.21.15.38:0/2796680259 >> 172.21.15.38:0/2796680259 conn(0x7f099c0fc580 msgr2=0x7f099c1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:25.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.997+0000 7f09a3600700 1 -- 172.21.15.38:0/2796680259 shutdown_connections 2024-03-24T00:56:25.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:25.997+0000 7f09a3600700 1 -- 172.21.15.38:0/2796680259 wait complete. 2024-03-24T00:56:26.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.157+0000 7fd4d0800700 1 -- 172.21.15.38:0/2138960287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c8102530 msgr2=0x7fd4c810aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:26.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.157+0000 7fd4d0800700 1 --2- 172.21.15.38:0/2138960287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c8102530 0x7fd4c810aa20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009910 tx=0x7fd4b8009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:26.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.157+0000 7fd4d0800700 1 -- 172.21.15.38:0/2138960287 shutdown_connections 2024-03-24T00:56:26.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.157+0000 7fd4d0800700 1 --2- 172.21.15.38:0/2138960287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c8102530 0x7fd4c810aa20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:26.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.157+0000 7fd4d0800700 1 --2- 172.21.15.38:0/2138960287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8101c10 0x7fd4c8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:26.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.157+0000 7fd4d0800700 1 -- 172.21.15.38:0/2138960287 >> 172.21.15.38:0/2138960287 conn(0x7fd4c8076640 msgr2=0x7fd4c80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:26.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.157+0000 7fd4d0800700 1 -- 172.21.15.38:0/2138960287 shutdown_connections 2024-03-24T00:56:26.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.157+0000 7fd4d0800700 1 -- 172.21.15.38:0/2138960287 wait complete. 2024-03-24T00:56:26.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.158+0000 7fd4d0800700 1 Processor -- start 2024-03-24T00:56:26.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.159+0000 7fd4d0800700 1 -- start start 2024-03-24T00:56:26.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.159+0000 7fd4d0800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c8101c10 0x7fd4c819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:26.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.159+0000 7fd4d0800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8102530 0x7fd4c819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:26.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.159+0000 7fd4d0800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c819d2c0 con 0x7fd4c8101c10 2024-03-24T00:56:26.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.159+0000 7fd4d0800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c8196830 con 0x7fd4c8102530 2024-03-24T00:56:26.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.159+0000 7fd4cde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c8101c10 0x7fd4c819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:26.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.159+0000 7fd4cde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c8101c10 0x7fd4c819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60546/0 (socket says 172.21.15.38:60546) 2024-03-24T00:56:26.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.159+0000 7fd4cde00700 1 -- 172.21.15.38:0/2716486336 learned_addr learned my addr 172.21.15.38:0/2716486336 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:26.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.160+0000 7fd4cd400700 1 --2- 172.21.15.38:0/2716486336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8102530 0x7fd4c819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:26.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.160+0000 7fd4cde00700 1 -- 172.21.15.38:0/2716486336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8102530 msgr2=0x7fd4c819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:26.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.160+0000 7fd4cde00700 1 --2- 172.21.15.38:0/2716486336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8102530 0x7fd4c819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:26.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.160+0000 7fd4cde00700 1 -- 172.21.15.38:0/2716486336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b80095f0 con 0x7fd4c8101c10 2024-03-24T00:56:26.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.160+0000 7fd4cde00700 1 --2- 172.21.15.38:0/2716486336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c8101c10 0x7fd4c819c6f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00d900 tx=0x7fd4bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:26.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.160+0000 7fd4c2a00700 1 -- 172.21.15.38:0/2716486336 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4bc0041f0 con 0x7fd4c8101c10 2024-03-24T00:56:26.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.160+0000 7fd4d0800700 1 -- 172.21.15.38:0/2716486336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c8196b10 con 0x7fd4c8101c10 2024-03-24T00:56:26.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.160+0000 7fd4c2a00700 1 -- 172.21.15.38:0/2716486336 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4bc004cf0 con 0x7fd4c8101c10 2024-03-24T00:56:26.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.160+0000 7fd4c2a00700 1 -- 172.21.15.38:0/2716486336 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4bc00b750 con 0x7fd4c8101c10 2024-03-24T00:56:26.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.161+0000 7fd4d0800700 1 -- 172.21.15.38:0/2716486336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c8197060 con 0x7fd4c8101c10 2024-03-24T00:56:26.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.162+0000 7fd4d0800700 1 -- 172.21.15.38:0/2716486336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c8105330 con 0x7fd4c8101c10 2024-03-24T00:56:26.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.163+0000 7fd4c2a00700 1 -- 172.21.15.38:0/2716486336 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd4bc004350 con 0x7fd4c8101c10 2024-03-24T00:56:26.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.164+0000 7fd4c2a00700 1 --2- 172.21.15.38:0/2716486336 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd4b006dfb0 0x7fd4b0070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:26.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.164+0000 7fd4c2a00700 1 -- 172.21.15.38:0/2716486336 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd4bc08def0 con 0x7fd4c8101c10 2024-03-24T00:56:26.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.164+0000 7fd4cd400700 1 --2- 172.21.15.38:0/2716486336 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd4b006dfb0 0x7fd4b0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:26.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.165+0000 7fd4cd400700 1 --2- 172.21.15.38:0/2716486336 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd4b006dfb0 0x7fd4b0070470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8197c50 tx=0x7fd4b801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:26.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.168+0000 7fd4c2a00700 1 -- 172.21.15.38:0/2716486336 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd4bc05ae50 con 0x7fd4c8101c10 2024-03-24T00:56:26.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:26 smithi038 ceph-mon[112108]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:26.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:26 smithi038 ceph-mon[112108]: pgmap v82: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:56:26.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:26 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2528116961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:26.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:26 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/4005111736' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:56:26.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:26 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:26.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:26 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T00:56:26.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:26 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:56:26.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:26 smithi088 ceph-mon[87351]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:26.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:26 smithi088 ceph-mon[87351]: pgmap v82: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:56:26.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:26 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2528116961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:26.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:26 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/4005111736' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:56:26.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:26 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:26.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:26 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T00:56:26.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:26 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:56:26.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.486+0000 7fd4d0800700 1 -- 172.21.15.38:0/2716486336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd4c8197cf0 con 0x7fd4c8101c10 2024-03-24T00:56:26.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.487+0000 7fd4c2a00700 1 -- 172.21.15.38:0/2716486336 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd4bc016020 con 0x7fd4c8101c10 2024-03-24T00:56:26.487 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_OK 2024-03-24T00:56:26.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.491+0000 7fd4d0800700 1 -- 172.21.15.38:0/2716486336 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd4b006dfb0 msgr2=0x7fd4b0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:26.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.491+0000 7fd4d0800700 1 --2- 172.21.15.38:0/2716486336 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd4b006dfb0 0x7fd4b0070470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8197c50 tx=0x7fd4b801a040 comp rx=0 tx=0).stop 2024-03-24T00:56:26.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.491+0000 7fd4d0800700 1 -- 172.21.15.38:0/2716486336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c8101c10 msgr2=0x7fd4c819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:26.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.491+0000 7fd4d0800700 1 --2- 172.21.15.38:0/2716486336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c8101c10 0x7fd4c819c6f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00d900 tx=0x7fd4bc00dcc0 comp rx=0 tx=0).stop 2024-03-24T00:56:26.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.491+0000 7fd4d0800700 1 -- 172.21.15.38:0/2716486336 shutdown_connections 2024-03-24T00:56:26.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.491+0000 7fd4d0800700 1 --2- 172.21.15.38:0/2716486336 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd4b006dfb0 0x7fd4b0070470 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:26.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.491+0000 7fd4d0800700 1 --2- 172.21.15.38:0/2716486336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c8101c10 0x7fd4c819c6f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:26.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.491+0000 7fd4d0800700 1 --2- 172.21.15.38:0/2716486336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8102530 0x7fd4c819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:26.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.491+0000 7fd4d0800700 1 -- 172.21.15.38:0/2716486336 >> 172.21.15.38:0/2716486336 conn(0x7fd4c8076640 msgr2=0x7fd4c8102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:26.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.491+0000 7fd4d0800700 1 -- 172.21.15.38:0/2716486336 shutdown_connections 2024-03-24T00:56:26.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:26.491+0000 7fd4d0800700 1 -- 172.21.15.38:0/2716486336 wait complete. 2024-03-24T00:56:27.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:27 smithi038 ceph-mon[112108]: Upgrade: Updating osd.1 2024-03-24T00:56:27.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:27 smithi038 ceph-mon[112108]: Deploying daemon osd.1 on smithi038 2024-03-24T00:56:27.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:27 smithi038 ceph-mon[112108]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:27.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:27 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2716486336' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:56:27.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:27.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:27.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:56:27.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:27 smithi088 ceph-mon[87351]: Upgrade: Updating osd.1 2024-03-24T00:56:27.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:27 smithi088 ceph-mon[87351]: Deploying daemon osd.1 on smithi038 2024-03-24T00:56:27.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:27 smithi088 ceph-mon[87351]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:27.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:27 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2716486336' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:56:27.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:27.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:27.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:56:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:28 smithi038 ceph-mon[112108]: pgmap v83: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-24T00:56:28.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:28 smithi088 ceph-mon[87351]: pgmap v83: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-24T00:56:30.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:30 smithi038 ceph-mon[112108]: pgmap v84: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-24T00:56:30.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:30 smithi088 ceph-mon[87351]: pgmap v84: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-24T00:56:31.110 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:30 smithi038 systemd[1]: Stopping Ceph osd.1 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:56:31.609 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:31 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1[55392]: 2024-03-24T00:56:31.199+0000 7f3e3e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T00:56:31.610 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:31 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1[55392]: 2024-03-24T00:56:31.200+0000 7f3e3e000700 -1 osd.1 55 *** Got signal Terminated *** 2024-03-24T00:56:31.610 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:31 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1[55392]: 2024-03-24T00:56:31.200+0000 7f3e3e000700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T00:56:32.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:32 smithi038 ceph-mon[112108]: pgmap v85: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:56:32.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:32 smithi038 ceph-mon[112108]: osd.1 marked itself down and dead 2024-03-24T00:56:32.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:32 smithi088 ceph-mon[87351]: pgmap v85: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:56:32.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:32 smithi088 ceph-mon[87351]: osd.1 marked itself down and dead 2024-03-24T00:56:32.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:32 smithi038 podman[128907]: 2024-03-24 00:56:32.571398709 +0000 UTC m=+1.553280453 container died 925408492cb5f47cf97a64b6b927d579ee05f7f97f0393f46793ebb64f1ce3f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=reef-738c2fd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-24T00:56:33.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:33 smithi088 ceph-mon[87351]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:56:33.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:33 smithi088 ceph-mon[87351]: osdmap e56: 6 total, 5 up, 6 in 2024-03-24T00:56:33.344 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:33 smithi038 ceph-mon[112108]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:56:33.344 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:33 smithi038 ceph-mon[112108]: osdmap e56: 6 total, 5 up, 6 in 2024-03-24T00:56:33.344 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:33 smithi038 podman[128907]: 2024-03-24 00:56:33.124093544 +0000 UTC m=+2.105975242 container cleanup 925408492cb5f47cf97a64b6b927d579ee05f7f97f0393f46793ebb64f1ce3f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-738c2fd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-24T00:56:33.344 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:33 smithi038 bash[128907]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1 2024-03-24T00:56:33.609 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:33 smithi038 podman[128927]: 2024-03-24 00:56:33.342200939 +0000 UTC m=+0.756154551 container remove 925408492cb5f47cf97a64b6b927d579ee05f7f97f0393f46793ebb64f1ce3f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=reef-738c2fd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-24T00:56:34.195 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:34 smithi038 ceph-mon[112108]: pgmap v87: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-03-24T00:56:34.195 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:34 smithi038 ceph-mon[112108]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-03-24T00:56:34.195 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:34 smithi038 ceph-mon[112108]: osdmap e57: 6 total, 5 up, 6 in 2024-03-24T00:56:34.195 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:34 smithi038 podman[129014]: 2024-03-24 00:56:34.001852465 +0000 UTC m=+0.031981199 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:56:34.195 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:34 smithi038 podman[129014]: 2024-03-24 00:56:34.19474643 +0000 UTC m=+0.224875201 container create a33798bc27240c32ad43f52c9f225631c2de81140881836c1afff4fb8ea05d04 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-24T00:56:34.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:34 smithi088 ceph-mon[87351]: pgmap v87: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-03-24T00:56:34.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:34 smithi088 ceph-mon[87351]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-03-24T00:56:34.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:34 smithi088 ceph-mon[87351]: osdmap e57: 6 total, 5 up, 6 in 2024-03-24T00:56:34.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:34 smithi038 podman[129014]: 2024-03-24 00:56:34.463681865 +0000 UTC m=+0.493810643 container init a33798bc27240c32ad43f52c9f225631c2de81140881836c1afff4fb8ea05d04 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:56:34.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:34 smithi038 podman[129014]: 2024-03-24 00:56:34.477187133 +0000 UTC m=+0.507315890 container start a33798bc27240c32ad43f52c9f225631c2de81140881836c1afff4fb8ea05d04 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:56:34.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:34 smithi038 podman[129014]: 2024-03-24 00:56:34.570905115 +0000 UTC m=+0.601033892 container attach a33798bc27240c32ad43f52c9f225631c2de81140881836c1afff4fb8ea05d04 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T00:56:35.355 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:34 smithi038 podman[129014]: 2024-03-24 00:56:34.931668709 +0000 UTC m=+0.961797464 container died a33798bc27240c32ad43f52c9f225631c2de81140881836c1afff4fb8ea05d04 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T00:56:36.044 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:35 smithi038 podman[129040]: 2024-03-24 00:56:35.766254106 +0000 UTC m=+0.815422701 container remove a33798bc27240c32ad43f52c9f225631c2de81140881836c1afff4fb8ea05d04 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-24T00:56:36.044 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:35 smithi038 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-24T00:56:36.044 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:35 smithi038 systemd[1]: Stopped Ceph osd.1 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:56:36.044 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:35 smithi038 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.1.service: Consumed 1min 57.421s CPU time. 2024-03-24T00:56:36.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:36 smithi038 ceph-mon[112108]: pgmap v89: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:56:36.359 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:36 smithi038 systemd[1]: Starting Ceph osd.1 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:56:36.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:36 smithi088 ceph-mon[87351]: pgmap v89: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:56:36.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:36 smithi038 podman[129141]: 2024-03-24 00:56:36.495488633 +0000 UTC m=+0.032397280 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:56:36.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:36 smithi038 podman[129141]: 2024-03-24 00:56:36.592308552 +0000 UTC m=+0.129217236 container create f5b7686e6416ca7ecac4c22934338e64aafeb924a4993cec9b860cb909ed5dd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:56:37.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:37 smithi038 ceph-mon[112108]: Health check failed: Degraded data redundancy: 4/273 objects degraded (1.465%), 2 pgs degraded (PG_DEGRADED) 2024-03-24T00:56:37.359 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:36 smithi038 podman[129141]: 2024-03-24 00:56:36.887776267 +0000 UTC m=+0.424684931 container init f5b7686e6416ca7ecac4c22934338e64aafeb924a4993cec9b860cb909ed5dd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T00:56:37.360 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:36 smithi038 podman[129141]: 2024-03-24 00:56:36.899167553 +0000 UTC m=+0.436076204 container start f5b7686e6416ca7ecac4c22934338e64aafeb924a4993cec9b860cb909ed5dd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-24T00:56:37.360 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:36 smithi038 podman[129141]: 2024-03-24 00:56:36.978676497 +0000 UTC m=+0.515585204 container attach f5b7686e6416ca7ecac4c22934338e64aafeb924a4993cec9b860cb909ed5dd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-24T00:56:37.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:37 smithi088 ceph-mon[87351]: Health check failed: Degraded data redundancy: 4/273 objects degraded (1.465%), 2 pgs degraded (PG_DEGRADED) 2024-03-24T00:56:38.102 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate[129156]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T00:56:38.102 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate[129156]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T00:56:38.102 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 bash[129141]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T00:56:38.102 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 bash[129141]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T00:56:38.102 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 bash[129141]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T00:56:38.102 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 bash[129141]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T00:56:38.102 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 bash[129141]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-24T00:56:38.102 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 bash[129141]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T00:56:38.102 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 bash[129141]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-24T00:56:38.103 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate[129156]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T00:56:38.103 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate[129156]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T00:56:38.103 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate[129156]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-24T00:56:38.103 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate[129156]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T00:56:38.103 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate[129156]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-24T00:56:38.103 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:37 smithi038 podman[129141]: 2024-03-24 00:56:37.868179178 +0000 UTC m=+1.405087861 container died f5b7686e6416ca7ecac4c22934338e64aafeb924a4993cec9b860cb909ed5dd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux ) 2024-03-24T00:56:38.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:38 smithi038 ceph-mon[112108]: pgmap v90: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4/273 objects degraded (1.465%) 2024-03-24T00:56:38.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:38 smithi088 ceph-mon[87351]: pgmap v90: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4/273 objects degraded (1.465%) 2024-03-24T00:56:38.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:38 smithi038 podman[129280]: 2024-03-24 00:56:38.59859108 +0000 UTC m=+0.707796890 container remove f5b7686e6416ca7ecac4c22934338e64aafeb924a4993cec9b860cb909ed5dd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-activate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True) 2024-03-24T00:56:39.228 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:39 smithi038 ceph-mon[112108]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-24T00:56:39.229 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:38 smithi038 podman[129346]: 2024-03-24 00:56:38.861594533 +0000 UTC m=+0.035564577 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:56:39.229 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:38 smithi038 podman[129346]: 2024-03-24 00:56:38.957676348 +0000 UTC m=+0.131646424 container create cb72ac446ba713b767f4e0e8673d9dfcf93beb01a5572fd1d5b1854422d89958 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-24T00:56:39.229 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:39 smithi038 podman[129346]: 2024-03-24 00:56:39.228188572 +0000 UTC m=+0.402158648 container init cb72ac446ba713b767f4e0e8673d9dfcf93beb01a5572fd1d5b1854422d89958 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-24T00:56:39.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:39 smithi088 ceph-mon[87351]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-24T00:56:39.609 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:39 smithi038 podman[129346]: 2024-03-24 00:56:39.239186405 +0000 UTC m=+0.413156481 container start cb72ac446ba713b767f4e0e8673d9dfcf93beb01a5572fd1d5b1854422d89958 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-03-24T00:56:39.609 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:39 smithi038 ceph-osd[129365]: -- 172.21.15.38:0/610251500 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558ff6f6b680 con 0x558ff6f35000 2024-03-24T00:56:39.610 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:39 smithi038 bash[129346]: cb72ac446ba713b767f4e0e8673d9dfcf93beb01a5572fd1d5b1854422d89958 2024-03-24T00:56:39.610 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:39 smithi038 systemd[1]: Started Ceph osd.1 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:56:40.109 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:39 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1[129361]: 2024-03-24T00:56:39.806+0000 7f066929b7c0 -1 Falling back to public interface 2024-03-24T00:56:40.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:40 smithi088 ceph-mon[87351]: pgmap v91: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/273 objects degraded (15.385%) 2024-03-24T00:56:40.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:40 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:40.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:40 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:40.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:40 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:56:40.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:40 smithi038 ceph-mon[112108]: pgmap v91: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/273 objects degraded (15.385%) 2024-03-24T00:56:40.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:40 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:40.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:40 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:40.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:40 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:56:42.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:42 smithi088 ceph-mon[87351]: pgmap v92: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/273 objects degraded (15.385%) 2024-03-24T00:56:42.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:42.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:56:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:42 smithi038 ceph-mon[112108]: pgmap v92: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/273 objects degraded (15.385%) 2024-03-24T00:56:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:56:44.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:44 smithi038 ceph-mon[112108]: pgmap v93: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/273 objects degraded (15.385%) 2024-03-24T00:56:44.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:44 smithi088 ceph-mon[87351]: pgmap v93: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/273 objects degraded (15.385%) 2024-03-24T00:56:46.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:46 smithi088 ceph-mon[87351]: pgmap v94: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/273 objects degraded (15.385%) 2024-03-24T00:56:46.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:46 smithi038 ceph-mon[112108]: pgmap v94: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/273 objects degraded (15.385%) 2024-03-24T00:56:46.609 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:46 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1[129361]: 2024-03-24T00:56:46.414+0000 7f066929b7c0 -1 osd.1 55 log_to_monitors true 2024-03-24T00:56:47.609 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 00:56:47 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1[129361]: 2024-03-24T00:56:47.211+0000 7f0658600700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T00:56:47.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:47 smithi038 ceph-mon[112108]: from='osd.1 [v2:172.21.15.38:6810/1120765274,v1:172.21.15.38:6811/1120765274]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T00:56:47.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:47 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:47.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:47 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:47.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:47 smithi038 ceph-mon[112108]: Health check update: Degraded data redundancy: 42/273 objects degraded (15.385%), 15 pgs degraded (PG_DEGRADED) 2024-03-24T00:56:47.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:47 smithi088 ceph-mon[87351]: from='osd.1 [v2:172.21.15.38:6810/1120765274,v1:172.21.15.38:6811/1120765274]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T00:56:47.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:47 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:47.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:47 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:47.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:47 smithi088 ceph-mon[87351]: Health check update: Degraded data redundancy: 42/273 objects degraded (15.385%), 15 pgs degraded (PG_DEGRADED) 2024-03-24T00:56:48.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:48 smithi038 ceph-mon[112108]: pgmap v95: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/273 objects degraded (15.385%) 2024-03-24T00:56:48.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:48 smithi038 ceph-mon[112108]: from='osd.1 [v2:172.21.15.38:6810/1120765274,v1:172.21.15.38:6811/1120765274]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T00:56:48.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:48 smithi038 ceph-mon[112108]: osdmap e58: 6 total, 5 up, 6 in 2024-03-24T00:56:48.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:48 smithi038 ceph-mon[112108]: from='osd.1 [v2:172.21.15.38:6810/1120765274,v1:172.21.15.38:6811/1120765274]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:56:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:48 smithi088 ceph-mon[87351]: pgmap v95: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/273 objects degraded (15.385%) 2024-03-24T00:56:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:48 smithi088 ceph-mon[87351]: from='osd.1 [v2:172.21.15.38:6810/1120765274,v1:172.21.15.38:6811/1120765274]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T00:56:48.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:48 smithi088 ceph-mon[87351]: osdmap e58: 6 total, 5 up, 6 in 2024-03-24T00:56:48.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:48 smithi088 ceph-mon[87351]: from='osd.1 [v2:172.21.15.38:6810/1120765274,v1:172.21.15.38:6811/1120765274]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:56:49.559 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:49 smithi038 ceph-mon[112108]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:56:49.559 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:49 smithi038 ceph-mon[112108]: osd.1 [v2:172.21.15.38:6810/1120765274,v1:172.21.15.38:6811/1120765274] boot 2024-03-24T00:56:49.559 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:49 smithi038 ceph-mon[112108]: osdmap e59: 6 total, 6 up, 6 in 2024-03-24T00:56:49.559 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:49 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:56:49.559 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:49 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:49.559 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:49 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:49.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:49 smithi088 ceph-mon[87351]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:56:49.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:49 smithi088 ceph-mon[87351]: osd.1 [v2:172.21.15.38:6810/1120765274,v1:172.21.15.38:6811/1120765274] boot 2024-03-24T00:56:49.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:49 smithi088 ceph-mon[87351]: osdmap e59: 6 total, 6 up, 6 in 2024-03-24T00:56:49.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:49 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T00:56:49.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:49 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:49.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:49 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:50.465 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:50 smithi038 ceph-mon[112108]: pgmap v98: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 42/273 objects degraded (15.385%) 2024-03-24T00:56:50.465 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:50 smithi038 ceph-mon[112108]: osdmap e60: 6 total, 6 up, 6 in 2024-03-24T00:56:50.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:50 smithi088 ceph-mon[87351]: pgmap v98: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 42/273 objects degraded (15.385%) 2024-03-24T00:56:50.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:50 smithi088 ceph-mon[87351]: osdmap e60: 6 total, 6 up, 6 in 2024-03-24T00:56:52.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:52 smithi038 ceph-mon[112108]: pgmap v100: 65 pgs: 15 active+undersized+degraded, 17 active+undersized, 33 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 42/273 objects degraded (15.385%) 2024-03-24T00:56:52.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:52 smithi088 ceph-mon[87351]: pgmap v100: 65 pgs: 15 active+undersized+degraded, 17 active+undersized, 33 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 42/273 objects degraded (15.385%) 2024-03-24T00:56:53.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:53 smithi038 ceph-mon[112108]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 42/273 objects degraded (15.385%), 15 pgs degraded) 2024-03-24T00:56:53.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:53 smithi038 ceph-mon[112108]: Cluster is now healthy 2024-03-24T00:56:53.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:53 smithi088 ceph-mon[87351]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 42/273 objects degraded (15.385%), 15 pgs degraded) 2024-03-24T00:56:53.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:53 smithi088 ceph-mon[87351]: Cluster is now healthy 2024-03-24T00:56:54.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:54 smithi038 ceph-mon[112108]: pgmap v101: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-24T00:56:54.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:54 smithi088 ceph-mon[87351]: pgmap v101: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-24T00:56:55.557 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:55.557 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:55.557 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:56:55.557 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:56:55.557 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:55.558 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:56:55.558 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:55.558 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:55.558 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:55.558 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T00:56:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:56:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:56:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:56:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T00:56:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.669+0000 7fd013200700 1 -- 172.21.15.38:0/3727775379 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0141008e0 msgr2=0x7fd014100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.669+0000 7fd013200700 1 --2- 172.21.15.38:0/3727775379 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0141008e0 0x7fd014100d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd000009910 tx=0x7fd000009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:56.671 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:56 smithi038 ceph-mon[112108]: pgmap v102: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-24T00:56:56.671 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:56 smithi038 ceph-mon[112108]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T00:56:56.671 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:56 smithi038 ceph-mon[112108]: Upgrade: osd.2 is safe to restart 2024-03-24T00:56:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.669+0000 7fd013200700 1 -- 172.21.15.38:0/3727775379 shutdown_connections 2024-03-24T00:56:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.670+0000 7fd013200700 1 --2- 172.21.15.38:0/3727775379 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0141008e0 0x7fd014100d60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.670+0000 7fd013200700 1 --2- 172.21.15.38:0/3727775379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0141068e0 0x7fd014106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.670+0000 7fd013200700 1 -- 172.21.15.38:0/3727775379 >> 172.21.15.38:0/3727775379 conn(0x7fd0140fc3f0 msgr2=0x7fd0140fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.670+0000 7fd013200700 1 -- 172.21.15.38:0/3727775379 shutdown_connections 2024-03-24T00:56:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.670+0000 7fd013200700 1 -- 172.21.15.38:0/3727775379 wait complete. 2024-03-24T00:56:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.671+0000 7fd013200700 1 Processor -- start 2024-03-24T00:56:56.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.672+0000 7fd013200700 1 -- start start 2024-03-24T00:56:56.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.672+0000 7fd013200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0141008e0 0x7fd0141944a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:56.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.672+0000 7fd013200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0141068e0 0x7fd0141949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:56.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.672+0000 7fd013200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd014194f20 con 0x7fd0141068e0 2024-03-24T00:56:56.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.672+0000 7fd013200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd014195090 con 0x7fd0141008e0 2024-03-24T00:56:56.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.672+0000 7fd011400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0141068e0 0x7fd0141949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:56.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.673+0000 7fd011400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0141068e0 0x7fd0141949e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60204/0 (socket says 172.21.15.38:60204) 2024-03-24T00:56:56.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.673+0000 7fd011400700 1 -- 172.21.15.38:0/590911322 learned_addr learned my addr 172.21.15.38:0/590911322 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:56.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.673+0000 7fd011e00700 1 --2- 172.21.15.38:0/590911322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0141008e0 0x7fd0141944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:56.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.673+0000 7fd011400700 1 -- 172.21.15.38:0/590911322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0141008e0 msgr2=0x7fd0141944a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:56.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.673+0000 7fd011400700 1 --2- 172.21.15.38:0/590911322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0141008e0 0x7fd0141944a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:56.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.673+0000 7fd011400700 1 -- 172.21.15.38:0/590911322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0000095f0 con 0x7fd0141068e0 2024-03-24T00:56:56.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.674+0000 7fd011400700 1 --2- 172.21.15.38:0/590911322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0141068e0 0x7fd0141949e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd000000c00 tx=0x7fd00000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:56.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.674+0000 7fd00aa00700 1 -- 172.21.15.38:0/590911322 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd00001d070 con 0x7fd0141068e0 2024-03-24T00:56:56.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.674+0000 7fd00aa00700 1 -- 172.21.15.38:0/590911322 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0000056e0 con 0x7fd0141068e0 2024-03-24T00:56:56.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.674+0000 7fd00aa00700 1 -- 172.21.15.38:0/590911322 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd00000fe80 con 0x7fd0141068e0 2024-03-24T00:56:56.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.674+0000 7fd013200700 1 -- 172.21.15.38:0/590911322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0141952c0 con 0x7fd0141068e0 2024-03-24T00:56:56.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.674+0000 7fd013200700 1 -- 172.21.15.38:0/590911322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0141a70a0 con 0x7fd0141068e0 2024-03-24T00:56:56.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.675+0000 7fd013200700 1 -- 172.21.15.38:0/590911322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd014104b90 con 0x7fd0141068e0 2024-03-24T00:56:56.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.677+0000 7fd00aa00700 1 -- 172.21.15.38:0/590911322 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd00000f6f0 con 0x7fd0141068e0 2024-03-24T00:56:56.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.678+0000 7fd00aa00700 1 --2- 172.21.15.38:0/590911322 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcff806dfb0 0x7fcff8070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:56.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.678+0000 7fd00aa00700 1 -- 172.21.15.38:0/590911322 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd00008f1b0 con 0x7fd0141068e0 2024-03-24T00:56:56.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.679+0000 7fd011e00700 1 --2- 172.21.15.38:0/590911322 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcff806dfb0 0x7fcff8070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:56 smithi088 ceph-mon[87351]: pgmap v102: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4 op/s 2024-03-24T00:56:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:56 smithi088 ceph-mon[87351]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T00:56:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:56 smithi088 ceph-mon[87351]: Upgrade: osd.2 is safe to restart 2024-03-24T00:56:56.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.679+0000 7fd011e00700 1 --2- 172.21.15.38:0/590911322 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcff806dfb0 0x7fcff8070470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd004005950 tx=0x7fd0040058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:56.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.682+0000 7fd00aa00700 1 -- 172.21.15.38:0/590911322 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd0000587a0 con 0x7fd0141068e0 2024-03-24T00:56:56.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.943+0000 7fd013200700 1 -- 172.21.15.38:0/590911322 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd014195930 con 0x7fcff806dfb0 2024-03-24T00:56:56.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.947+0000 7fd00aa00700 1 -- 172.21.15.38:0/590911322 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd014195930 con 0x7fcff806dfb0 2024-03-24T00:56:56.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.952+0000 7fd013200700 1 -- 172.21.15.38:0/590911322 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcff806dfb0 msgr2=0x7fcff8070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:56.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.952+0000 7fd013200700 1 --2- 172.21.15.38:0/590911322 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcff806dfb0 0x7fcff8070470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd004005950 tx=0x7fd0040058e0 comp rx=0 tx=0).stop 2024-03-24T00:56:56.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.952+0000 7fd013200700 1 -- 172.21.15.38:0/590911322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0141068e0 msgr2=0x7fd0141949e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:56.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.952+0000 7fd013200700 1 --2- 172.21.15.38:0/590911322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0141068e0 0x7fd0141949e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd000000c00 tx=0x7fd00000be20 comp rx=0 tx=0).stop 2024-03-24T00:56:56.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.952+0000 7fd013200700 1 -- 172.21.15.38:0/590911322 shutdown_connections 2024-03-24T00:56:56.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.952+0000 7fd013200700 1 --2- 172.21.15.38:0/590911322 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcff806dfb0 0x7fcff8070470 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:56.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.952+0000 7fd013200700 1 --2- 172.21.15.38:0/590911322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0141008e0 0x7fd0141944a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:56.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.952+0000 7fd013200700 1 --2- 172.21.15.38:0/590911322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0141068e0 0x7fd0141949e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:56.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.952+0000 7fd013200700 1 -- 172.21.15.38:0/590911322 >> 172.21.15.38:0/590911322 conn(0x7fd0140fc3f0 msgr2=0x7fd0140fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:56.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.953+0000 7fd013200700 1 -- 172.21.15.38:0/590911322 shutdown_connections 2024-03-24T00:56:56.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:56.953+0000 7fd013200700 1 -- 172.21.15.38:0/590911322 wait complete. 2024-03-24T00:56:56.977 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:56:57.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.122+0000 7f5f7f400700 1 -- 172.21.15.38:0/2262085467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f78101c70 msgr2=0x7f5f78102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:57.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.122+0000 7f5f7f400700 1 --2- 172.21.15.38:0/2262085467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f78101c70 0x7f5f78102050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009940 tx=0x7f5f6c009c50 comp rx=0 tx=0).stop 2024-03-24T00:56:57.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.123+0000 7f5f7f400700 1 -- 172.21.15.38:0/2262085467 shutdown_connections 2024-03-24T00:56:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.123+0000 7f5f7f400700 1 --2- 172.21.15.38:0/2262085467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f78102590 0x7f5f7810aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.123+0000 7f5f7f400700 1 --2- 172.21.15.38:0/2262085467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f78101c70 0x7f5f78102050 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.123+0000 7f5f7f400700 1 -- 172.21.15.38:0/2262085467 >> 172.21.15.38:0/2262085467 conn(0x7f5f78076640 msgr2=0x7f5f780fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.123+0000 7f5f7f400700 1 -- 172.21.15.38:0/2262085467 shutdown_connections 2024-03-24T00:56:57.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.123+0000 7f5f7f400700 1 -- 172.21.15.38:0/2262085467 wait complete. 2024-03-24T00:56:57.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.124+0000 7f5f7f400700 1 Processor -- start 2024-03-24T00:56:57.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.124+0000 7f5f7f400700 1 -- start start 2024-03-24T00:56:57.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.124+0000 7f5f7f400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f78101c70 0x7f5f7819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.125+0000 7f5f7f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f78102590 0x7f5f7819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.125+0000 7f5f7f400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7819f400 con 0x7f5f78102590 2024-03-24T00:56:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.125+0000 7f5f7f400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f781989e0 con 0x7f5f78101c70 2024-03-24T00:56:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.125+0000 7f5f77e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f78102590 0x7f5f7819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.125+0000 7f5f77e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f78102590 0x7f5f7819ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60222/0 (socket says 172.21.15.38:60222) 2024-03-24T00:56:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.125+0000 7f5f77e00700 1 -- 172.21.15.38:0/3990929218 learned_addr learned my addr 172.21.15.38:0/3990929218 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.125+0000 7f5f7ca00700 1 --2- 172.21.15.38:0/3990929218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f78101c70 0x7f5f7819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.125+0000 7f5f77e00700 1 -- 172.21.15.38:0/3990929218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f78101c70 msgr2=0x7f5f7819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.125+0000 7f5f77e00700 1 --2- 172.21.15.38:0/3990929218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f78101c70 0x7f5f7819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.125+0000 7f5f77e00700 1 -- 172.21.15.38:0/3990929218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c0095f0 con 0x7f5f78102590 2024-03-24T00:56:57.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.126+0000 7f5f77e00700 1 --2- 172.21.15.38:0/3990929218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f78102590 0x7f5f7819ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400d900 tx=0x7f5f6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:57.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.126+0000 7f5f75600700 1 -- 172.21.15.38:0/3990929218 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f64009900 con 0x7f5f78102590 2024-03-24T00:56:57.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.126+0000 7f5f75600700 1 -- 172.21.15.38:0/3990929218 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f64010460 con 0x7f5f78102590 2024-03-24T00:56:57.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.126+0000 7f5f7f400700 1 -- 172.21.15.38:0/3990929218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f78198cc0 con 0x7f5f78102590 2024-03-24T00:56:57.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.126+0000 7f5f75600700 1 -- 172.21.15.38:0/3990929218 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f6400f5f0 con 0x7f5f78102590 2024-03-24T00:56:57.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.126+0000 7f5f7f400700 1 -- 172.21.15.38:0/3990929218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f78199210 con 0x7f5f78102590 2024-03-24T00:56:57.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.128+0000 7f5f7f400700 1 -- 172.21.15.38:0/3990929218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f78068d00 con 0x7f5f78102590 2024-03-24T00:56:57.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.129+0000 7f5f75600700 1 -- 172.21.15.38:0/3990929218 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f5f64009c70 con 0x7f5f78102590 2024-03-24T00:56:57.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.130+0000 7f5f75600700 1 --2- 172.21.15.38:0/3990929218 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5f6006dfa0 0x7f5f60070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:57.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.130+0000 7f5f75600700 1 -- 172.21.15.38:0/3990929218 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5f6408d470 con 0x7f5f78102590 2024-03-24T00:56:57.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.130+0000 7f5f7ca00700 1 --2- 172.21.15.38:0/3990929218 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5f6006dfa0 0x7f5f60070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:57.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.131+0000 7f5f7ca00700 1 --2- 172.21.15.38:0/3990929218 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5f6006dfa0 0x7f5f60070460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0059d0 tx=0x7f5f6c005920 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:57.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.136+0000 7f5f75600700 1 -- 172.21.15.38:0/3990929218 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5f6405a4d0 con 0x7f5f78102590 2024-03-24T00:56:57.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.395+0000 7f5f7f400700 1 -- 172.21.15.38:0/3990929218 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f78199540 con 0x7f5f6006dfa0 2024-03-24T00:56:57.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.397+0000 7f5f75600700 1 -- 172.21.15.38:0/3990929218 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5f78199540 con 0x7f5f6006dfa0 2024-03-24T00:56:57.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.400+0000 7f5f7f400700 1 -- 172.21.15.38:0/3990929218 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5f6006dfa0 msgr2=0x7f5f60070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:57.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.400+0000 7f5f7f400700 1 --2- 172.21.15.38:0/3990929218 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5f6006dfa0 0x7f5f60070460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0059d0 tx=0x7f5f6c005920 comp rx=0 tx=0).stop 2024-03-24T00:56:57.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.400+0000 7f5f7f400700 1 -- 172.21.15.38:0/3990929218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f78102590 msgr2=0x7f5f7819ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:57.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.400+0000 7f5f7f400700 1 --2- 172.21.15.38:0/3990929218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f78102590 0x7f5f7819ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400d900 tx=0x7f5f6400dcc0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.400+0000 7f5f7f400700 1 -- 172.21.15.38:0/3990929218 shutdown_connections 2024-03-24T00:56:57.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.400+0000 7f5f7f400700 1 --2- 172.21.15.38:0/3990929218 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5f6006dfa0 0x7f5f60070460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.400+0000 7f5f7f400700 1 --2- 172.21.15.38:0/3990929218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f78101c70 0x7f5f7819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.400+0000 7f5f7f400700 1 --2- 172.21.15.38:0/3990929218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f78102590 0x7f5f7819ede0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.400+0000 7f5f7f400700 1 -- 172.21.15.38:0/3990929218 >> 172.21.15.38:0/3990929218 conn(0x7f5f78076640 msgr2=0x7f5f7810a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:57.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.401+0000 7f5f7f400700 1 -- 172.21.15.38:0/3990929218 shutdown_connections 2024-03-24T00:56:57.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.401+0000 7f5f7f400700 1 -- 172.21.15.38:0/3990929218 wait complete. 2024-03-24T00:56:57.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.557+0000 7f4597000700 1 -- 172.21.15.38:0/2281206614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4590101c10 msgr2=0x7f4590101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:57.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.557+0000 7f4597000700 1 --2- 172.21.15.38:0/2281206614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4590101c10 0x7f4590101ff0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4580009910 tx=0x7f4580009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:57.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.557+0000 7f4597000700 1 -- 172.21.15.38:0/2281206614 shutdown_connections 2024-03-24T00:56:57.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.557+0000 7f4597000700 1 --2- 172.21.15.38:0/2281206614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4590102530 0x7f459010aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.557+0000 7f4597000700 1 --2- 172.21.15.38:0/2281206614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4590101c10 0x7f4590101ff0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.557+0000 7f4597000700 1 -- 172.21.15.38:0/2281206614 >> 172.21.15.38:0/2281206614 conn(0x7f4590076640 msgr2=0x7f45900fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:57.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.557+0000 7f4597000700 1 -- 172.21.15.38:0/2281206614 shutdown_connections 2024-03-24T00:56:57.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.557+0000 7f4597000700 1 -- 172.21.15.38:0/2281206614 wait complete. 2024-03-24T00:56:57.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.558+0000 7f4597000700 1 Processor -- start 2024-03-24T00:56:57.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.559+0000 7f4597000700 1 -- start start 2024-03-24T00:56:57.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.559+0000 7f4597000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4590101c10 0x7f459019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:57.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.559+0000 7f4597000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4590102530 0x7f459019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:57.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.559+0000 7f4597000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459019f370 con 0x7f4590101c10 2024-03-24T00:56:57.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.559+0000 7f4597000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45901989e0 con 0x7f4590102530 2024-03-24T00:56:57.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.560+0000 7f458fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4590101c10 0x7f459019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:57.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.560+0000 7f458fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4590101c10 0x7f459019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60240/0 (socket says 172.21.15.38:60240) 2024-03-24T00:56:57.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.560+0000 7f458fe00700 1 -- 172.21.15.38:0/4265839449 learned_addr learned my addr 172.21.15.38:0/4265839449 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.560+0000 7f458f400700 1 --2- 172.21.15.38:0/4265839449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4590102530 0x7f459019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.560+0000 7f458fe00700 1 -- 172.21.15.38:0/4265839449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4590102530 msgr2=0x7f459019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.560+0000 7f458fe00700 1 --2- 172.21.15.38:0/4265839449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4590102530 0x7f459019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.560+0000 7f458fe00700 1 -- 172.21.15.38:0/4265839449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45800095f0 con 0x7f4590101c10 2024-03-24T00:56:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.561+0000 7f458fe00700 1 --2- 172.21.15.38:0/4265839449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4590101c10 0x7f459019e8a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4580006010 tx=0x7f458000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.561+0000 7f458cc00700 1 -- 172.21.15.38:0/4265839449 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458001d070 con 0x7f4590101c10 2024-03-24T00:56:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.561+0000 7f458cc00700 1 -- 172.21.15.38:0/4265839449 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f458000f460 con 0x7f4590101c10 2024-03-24T00:56:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.561+0000 7f4597000700 1 -- 172.21.15.38:0/4265839449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4590198c60 con 0x7f4590101c10 2024-03-24T00:56:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.561+0000 7f458cc00700 1 -- 172.21.15.38:0/4265839449 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4580017630 con 0x7f4590101c10 2024-03-24T00:56:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.561+0000 7f4597000700 1 -- 172.21.15.38:0/4265839449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4590199150 con 0x7f4590101c10 2024-03-24T00:56:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.562+0000 7f4597000700 1 -- 172.21.15.38:0/4265839449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4590109580 con 0x7f4590101c10 2024-03-24T00:56:57.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.564+0000 7f458cc00700 1 -- 172.21.15.38:0/4265839449 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f458000f5d0 con 0x7f4590101c10 2024-03-24T00:56:57.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.564+0000 7f458cc00700 1 --2- 172.21.15.38:0/4265839449 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f457406dfa0 0x7f4574070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:57.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.565+0000 7f458cc00700 1 -- 172.21.15.38:0/4265839449 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f458008f440 con 0x7f4590101c10 2024-03-24T00:56:57.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.565+0000 7f458f400700 1 --2- 172.21.15.38:0/4265839449 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f457406dfa0 0x7f4574070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:57.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.565+0000 7f458f400700 1 --2- 172.21.15.38:0/4265839449 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f457406dfa0 0x7f4574070460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f457c005ea0 tx=0x7f457c005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:57.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.572+0000 7f458cc00700 1 -- 172.21.15.38:0/4265839449 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f458005c4e0 con 0x7f4590101c10 2024-03-24T00:56:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.823+0000 7f4597000700 1 -- 172.21.15.38:0/4265839449 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4590199ea0 con 0x7f457406dfa0 2024-03-24T00:56:57.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.855+0000 7f458cc00700 1 -- 172.21.15.38:0/4265839449 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4590199ea0 con 0x7f457406dfa0 2024-03-24T00:56:57.857 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:56:57.857 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (5m) 11s ago 19m 13.5M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:56:57.857 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (19m) 11s ago 19m 8837k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:56:57.857 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (18m) 76s ago 18m 8363k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:56:57.857 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (89s) 11s ago 19m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T00:56:57.857 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (81s) 76s ago 18m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T00:56:57.857 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (4m) 11s ago 18m 74.6M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (11m) 11s ago 11m 97.6M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (12m) 11s ago 12m 150M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (11m) 76s ago 11m 24.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (11m) 76s ago 11m 24.8M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (8m) 11s ago 21m 500M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (6m) 76s ago 17m 449M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (3m) 11s ago 21m 48.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (2m) 76s ago 17m 41.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (6m) 11s ago 19m 8036k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (6m) 76s ago 18m 7755k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (50s) 11s ago 16m 96.6M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (18s) 11s ago 16m 11.2M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cb72ac446ba7 2024-03-24T00:56:57.858 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (15m) 11s ago 15m 227M 1218M 18.2.2-952-g738c2fd6 68aff9aff3b2 00659d84b259 2024-03-24T00:56:57.859 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (15m) 76s ago 15m 304M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:56:57.859 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (14m) 76s ago 14m 283M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:56:57.859 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (13m) 76s ago 13m 254M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:56:57.859 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (5m) 11s ago 18m 70.4M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:56:57.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.858+0000 7f4597000700 1 -- 172.21.15.38:0/4265839449 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f457406dfa0 msgr2=0x7f4574070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:57.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.858+0000 7f4597000700 1 --2- 172.21.15.38:0/4265839449 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f457406dfa0 0x7f4574070460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f457c005ea0 tx=0x7f457c005e30 comp rx=0 tx=0).stop 2024-03-24T00:56:57.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.858+0000 7f4597000700 1 -- 172.21.15.38:0/4265839449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4590101c10 msgr2=0x7f459019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:57.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.858+0000 7f4597000700 1 --2- 172.21.15.38:0/4265839449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4590101c10 0x7f459019e8a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4580006010 tx=0x7f458000be20 comp rx=0 tx=0).stop 2024-03-24T00:56:57.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.858+0000 7f4597000700 1 -- 172.21.15.38:0/4265839449 shutdown_connections 2024-03-24T00:56:57.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.858+0000 7f4597000700 1 --2- 172.21.15.38:0/4265839449 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f457406dfa0 0x7f4574070460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.858+0000 7f4597000700 1 --2- 172.21.15.38:0/4265839449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4590101c10 0x7f459019e8a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.858+0000 7f4597000700 1 --2- 172.21.15.38:0/4265839449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4590102530 0x7f459019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:57.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.858+0000 7f4597000700 1 -- 172.21.15.38:0/4265839449 >> 172.21.15.38:0/4265839449 conn(0x7f4590076640 msgr2=0x7f459010a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:57.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.858+0000 7f4597000700 1 -- 172.21.15.38:0/4265839449 shutdown_connections 2024-03-24T00:56:57.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:57.859+0000 7f4597000700 1 -- 172.21.15.38:0/4265839449 wait complete. 2024-03-24T00:56:57.929 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:57 smithi038 ceph-mon[112108]: pgmap v103: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T00:56:57.929 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:57.929 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:56:57.929 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:57 smithi038 ceph-mon[112108]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:57.929 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:57 smithi038 ceph-mon[112108]: Upgrade: Updating osd.2 2024-03-24T00:56:57.929 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:57.929 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T00:56:57.929 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:56:57.929 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:57 smithi038 ceph-mon[112108]: Deploying daemon osd.2 on smithi038 2024-03-24T00:56:57.929 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:57 smithi038 ceph-mon[112108]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:57 smithi088 ceph-mon[87351]: pgmap v103: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T00:56:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:56:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:57 smithi088 ceph-mon[87351]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:57 smithi088 ceph-mon[87351]: Upgrade: Updating osd.2 2024-03-24T00:56:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:56:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T00:56:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:56:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:57 smithi088 ceph-mon[87351]: Deploying daemon osd.2 on smithi038 2024-03-24T00:56:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:57 smithi088 ceph-mon[87351]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.009+0000 7f816fc00700 1 -- 172.21.15.38:0/1913211101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8168107cc0 msgr2=0x7f816810bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.009+0000 7f816fc00700 1 --2- 172.21.15.38:0/1913211101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8168107cc0 0x7f816810bd10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8154009910 tx=0x7f8154009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.009+0000 7f816fc00700 1 -- 172.21.15.38:0/1913211101 shutdown_connections 2024-03-24T00:56:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.009+0000 7f816fc00700 1 --2- 172.21.15.38:0/1913211101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8168107cc0 0x7f816810bd10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.009+0000 7f816fc00700 1 --2- 172.21.15.38:0/1913211101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8168107310 0x7f81681076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.009+0000 7f816fc00700 1 -- 172.21.15.38:0/1913211101 >> 172.21.15.38:0/1913211101 conn(0x7f81680768e0 msgr2=0x7f8168078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.009+0000 7f816fc00700 1 -- 172.21.15.38:0/1913211101 shutdown_connections 2024-03-24T00:56:58.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.010+0000 7f816fc00700 1 -- 172.21.15.38:0/1913211101 wait complete. 2024-03-24T00:56:58.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.010+0000 7f816fc00700 1 Processor -- start 2024-03-24T00:56:58.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.011+0000 7f816fc00700 1 -- start start 2024-03-24T00:56:58.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.011+0000 7f816fc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8168107310 0x7f816819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:58.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.011+0000 7f816fc00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8168107cc0 0x7f816819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:58.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.011+0000 7f816fc00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816819db50 con 0x7f8168107310 2024-03-24T00:56:58.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.011+0000 7f816fc00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816819dc90 con 0x7f8168107cc0 2024-03-24T00:56:58.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.011+0000 7f816d200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8168107310 0x7f816819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:58.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.011+0000 7f816d200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8168107310 0x7f816819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60254/0 (socket says 172.21.15.38:60254) 2024-03-24T00:56:58.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.011+0000 7f816d200700 1 -- 172.21.15.38:0/3585140717 learned_addr learned my addr 172.21.15.38:0/3585140717 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:58.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.012+0000 7f8167e00700 1 --2- 172.21.15.38:0/3585140717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8168107cc0 0x7f816819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:58.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.012+0000 7f816d200700 1 -- 172.21.15.38:0/3585140717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8168107cc0 msgr2=0x7f816819d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:58.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.012+0000 7f816d200700 1 --2- 172.21.15.38:0/3585140717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8168107cc0 0x7f816819d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.012+0000 7f816d200700 1 -- 172.21.15.38:0/3585140717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81540095f0 con 0x7f8168107310 2024-03-24T00:56:58.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.012+0000 7f816d200700 1 --2- 172.21.15.38:0/3585140717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8168107310 0x7f816819cfc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f815c00c930 tx=0x7f815c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.012+0000 7f8165600700 1 -- 172.21.15.38:0/3585140717 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f815c007ad0 con 0x7f8168107310 2024-03-24T00:56:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.012+0000 7f8165600700 1 -- 172.21.15.38:0/3585140717 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f815c007c30 con 0x7f8168107310 2024-03-24T00:56:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.012+0000 7f816fc00700 1 -- 172.21.15.38:0/3585140717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81681a1a20 con 0x7f8168107310 2024-03-24T00:56:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.012+0000 7f8165600700 1 -- 172.21.15.38:0/3585140717 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f815c018610 con 0x7f8168107310 2024-03-24T00:56:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.013+0000 7f816fc00700 1 -- 172.21.15.38:0/3585140717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81681a1f70 con 0x7f8168107310 2024-03-24T00:56:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.013+0000 7f816fc00700 1 -- 172.21.15.38:0/3585140717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816810f710 con 0x7f8168107310 2024-03-24T00:56:58.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.015+0000 7f8165600700 1 -- 172.21.15.38:0/3585140717 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f815c00f450 con 0x7f8168107310 2024-03-24T00:56:58.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.016+0000 7f8165600700 1 --2- 172.21.15.38:0/3585140717 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f815006dfa0 0x7f8150070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:58.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.016+0000 7f8165600700 1 -- 172.21.15.38:0/3585140717 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f815c08dff0 con 0x7f8168107310 2024-03-24T00:56:58.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.016+0000 7f8167e00700 1 --2- 172.21.15.38:0/3585140717 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f815006dfa0 0x7f8150070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:58.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.017+0000 7f8167e00700 1 --2- 172.21.15.38:0/3585140717 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f815006dfa0 0x7f8150070460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f815400b5c0 tx=0x7f8154005380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:58.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.020+0000 7f8165600700 1 -- 172.21.15.38:0/3585140717 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f815c0c2740 con 0x7f8168107310 2024-03-24T00:56:58.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.352+0000 7f816fc00700 1 -- 172.21.15.38:0/3585140717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f81681087f0 con 0x7f8168107310 2024-03-24T00:56:58.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.353+0000 7f8165600700 1 -- 172.21.15.38:0/3585140717 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f815c057420 con 0x7f8168107310 2024-03-24T00:56:58.355 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:56:58.355 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:56:58.355 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:56:58.355 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:56:58.355 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:56:58.355 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:56:58.356 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:56:58.356 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:56:58.356 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4, 2024-03-24T00:56:58.356 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:56:58.356 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:56:58.356 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:56:58.356 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:56:58.356 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:56:58.356 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:56:58.357 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 8, 2024-03-24T00:56:58.357 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-24T00:56:58.357 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:56:58.357 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:56:58.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.356+0000 7f816fc00700 1 -- 172.21.15.38:0/3585140717 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f815006dfa0 msgr2=0x7f8150070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:58.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.356+0000 7f816fc00700 1 --2- 172.21.15.38:0/3585140717 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f815006dfa0 0x7f8150070460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f815400b5c0 tx=0x7f8154005380 comp rx=0 tx=0).stop 2024-03-24T00:56:58.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.356+0000 7f816fc00700 1 -- 172.21.15.38:0/3585140717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8168107310 msgr2=0x7f816819cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:58.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.356+0000 7f816fc00700 1 --2- 172.21.15.38:0/3585140717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8168107310 0x7f816819cfc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f815c00c930 tx=0x7f815c00ccf0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.356+0000 7f816fc00700 1 -- 172.21.15.38:0/3585140717 shutdown_connections 2024-03-24T00:56:58.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.356+0000 7f816fc00700 1 --2- 172.21.15.38:0/3585140717 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f815006dfa0 0x7f8150070460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.356+0000 7f816fc00700 1 --2- 172.21.15.38:0/3585140717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8168107310 0x7f816819cfc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.356+0000 7f816fc00700 1 --2- 172.21.15.38:0/3585140717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8168107cc0 0x7f816819d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.356+0000 7f816fc00700 1 -- 172.21.15.38:0/3585140717 >> 172.21.15.38:0/3585140717 conn(0x7f81680768e0 msgr2=0x7f8168076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:58.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.356+0000 7f816fc00700 1 -- 172.21.15.38:0/3585140717 shutdown_connections 2024-03-24T00:56:58.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.356+0000 7f816fc00700 1 -- 172.21.15.38:0/3585140717 wait complete. 2024-03-24T00:56:58.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.503+0000 7f83d0800700 1 -- 172.21.15.38:0/356906676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c81038a0 msgr2=0x7f83c81078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:58.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.503+0000 7f83d0800700 1 --2- 172.21.15.38:0/356906676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c81038a0 0x7f83c81078f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009910 tx=0x7f83b8009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.503+0000 7f83d0800700 1 -- 172.21.15.38:0/356906676 shutdown_connections 2024-03-24T00:56:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.503+0000 7f83d0800700 1 --2- 172.21.15.38:0/356906676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c81038a0 0x7f83c81078f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.503+0000 7f83d0800700 1 --2- 172.21.15.38:0/356906676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c8102ef0 0x7f83c81032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.503+0000 7f83d0800700 1 -- 172.21.15.38:0/356906676 >> 172.21.15.38:0/356906676 conn(0x7f83c80fe780 msgr2=0x7f83c8100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.503+0000 7f83d0800700 1 -- 172.21.15.38:0/356906676 shutdown_connections 2024-03-24T00:56:58.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.503+0000 7f83d0800700 1 -- 172.21.15.38:0/356906676 wait complete. 2024-03-24T00:56:58.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.504+0000 7f83d0800700 1 Processor -- start 2024-03-24T00:56:58.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.504+0000 7f83d0800700 1 -- start start 2024-03-24T00:56:58.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.505+0000 7f83d0800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c8102ef0 0x7f83c819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:58.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.505+0000 7f83d0800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c81038a0 0x7f83c819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:58.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.505+0000 7f83d0800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c819f550 con 0x7f83c8102ef0 2024-03-24T00:56:58.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.505+0000 7f83d0800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c8198b10 con 0x7f83c81038a0 2024-03-24T00:56:58.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.505+0000 7f83cde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c8102ef0 0x7f83c819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:58.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.505+0000 7f83cde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c8102ef0 0x7f83c819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60260/0 (socket says 172.21.15.38:60260) 2024-03-24T00:56:58.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.505+0000 7f83cde00700 1 -- 172.21.15.38:0/1266750654 learned_addr learned my addr 172.21.15.38:0/1266750654 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:58.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.505+0000 7f83cd400700 1 --2- 172.21.15.38:0/1266750654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c81038a0 0x7f83c819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:58.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.506+0000 7f83cde00700 1 -- 172.21.15.38:0/1266750654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c81038a0 msgr2=0x7f83c819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:58.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.506+0000 7f83cde00700 1 --2- 172.21.15.38:0/1266750654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c81038a0 0x7f83c819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.506+0000 7f83cde00700 1 -- 172.21.15.38:0/1266750654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b80095f0 con 0x7f83c8102ef0 2024-03-24T00:56:58.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.506+0000 7f83cde00700 1 --2- 172.21.15.38:0/1266750654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c8102ef0 0x7f83c819e980 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00d8d0 tx=0x7f83bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:58.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.506+0000 7f83c2a00700 1 -- 172.21.15.38:0/1266750654 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83bc009940 con 0x7f83c8102ef0 2024-03-24T00:56:58.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.506+0000 7f83d0800700 1 -- 172.21.15.38:0/1266750654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c8198df0 con 0x7f83c8102ef0 2024-03-24T00:56:58.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.506+0000 7f83c2a00700 1 -- 172.21.15.38:0/1266750654 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83bc010460 con 0x7f83c8102ef0 2024-03-24T00:56:58.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.506+0000 7f83c2a00700 1 -- 172.21.15.38:0/1266750654 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83bc00f5f0 con 0x7f83c8102ef0 2024-03-24T00:56:58.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.506+0000 7f83d0800700 1 -- 172.21.15.38:0/1266750654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c8199340 con 0x7f83c8102ef0 2024-03-24T00:56:58.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.508+0000 7f83d0800700 1 -- 172.21.15.38:0/1266750654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c810b2f0 con 0x7f83c8102ef0 2024-03-24T00:56:58.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.509+0000 7f83c2a00700 1 -- 172.21.15.38:0/1266750654 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f83bc0105d0 con 0x7f83c8102ef0 2024-03-24T00:56:58.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.509+0000 7f83c2a00700 1 --2- 172.21.15.38:0/1266750654 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f83b006dfa0 0x7f83b0070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:58.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.510+0000 7f83c2a00700 1 -- 172.21.15.38:0/1266750654 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f83bc08dca0 con 0x7f83c8102ef0 2024-03-24T00:56:58.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.510+0000 7f83cd400700 1 --2- 172.21.15.38:0/1266750654 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f83b006dfa0 0x7f83b0070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:58.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.510+0000 7f83cd400700 1 --2- 172.21.15.38:0/1266750654 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f83b006dfa0 0x7f83b0070460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83c8100d80 tx=0x7f83b801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:58.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.515+0000 7f83c2a00700 1 -- 172.21.15.38:0/1266750654 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f83bc05ac80 con 0x7f83c8102ef0 2024-03-24T00:56:58.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.784+0000 7f83d0800700 1 -- 172.21.15.38:0/1266750654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f83c8199f80 con 0x7f83c8102ef0 2024-03-24T00:56:58.786 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:58 smithi038 ceph-mon[112108]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:58.786 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:58 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3585140717' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:58.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.785+0000 7f83c2a00700 1 -- 172.21.15.38:0/1266750654 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1900 (secure 0 0 0) 0x7f83bc016780 con 0x7f83c8102ef0 2024-03-24T00:56:58.787 INFO:teuthology.orchestra.run.smithi038.stdout:e20 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 20 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:56:58.788 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:56:58.789 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:56:58.790 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T00:56:58.790 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:56:58.790 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:56:58.790 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:56:58.790 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:56:58.790 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:56:58.790 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:56:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.788+0000 7f83d0800700 1 -- 172.21.15.38:0/1266750654 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f83b006dfa0 msgr2=0x7f83b0070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.788+0000 7f83d0800700 1 --2- 172.21.15.38:0/1266750654 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f83b006dfa0 0x7f83b0070460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83c8100d80 tx=0x7f83b801a040 comp rx=0 tx=0).stop 2024-03-24T00:56:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.788+0000 7f83d0800700 1 -- 172.21.15.38:0/1266750654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c8102ef0 msgr2=0x7f83c819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.788+0000 7f83d0800700 1 --2- 172.21.15.38:0/1266750654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c8102ef0 0x7f83c819e980 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00d8d0 tx=0x7f83bc00dc90 comp rx=0 tx=0).stop 2024-03-24T00:56:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.788+0000 7f83d0800700 1 -- 172.21.15.38:0/1266750654 shutdown_connections 2024-03-24T00:56:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.788+0000 7f83d0800700 1 --2- 172.21.15.38:0/1266750654 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f83b006dfa0 0x7f83b0070460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.788+0000 7f83d0800700 1 --2- 172.21.15.38:0/1266750654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c8102ef0 0x7f83c819e980 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.788+0000 7f83d0800700 1 --2- 172.21.15.38:0/1266750654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c81038a0 0x7f83c819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.788+0000 7f83d0800700 1 -- 172.21.15.38:0/1266750654 >> 172.21.15.38:0/1266750654 conn(0x7f83c80fe780 msgr2=0x7f83c80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.788+0000 7f83d0800700 1 -- 172.21.15.38:0/1266750654 shutdown_connections 2024-03-24T00:56:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.788+0000 7f83d0800700 1 -- 172.21.15.38:0/1266750654 wait complete. 2024-03-24T00:56:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:56:58.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:58 smithi088 ceph-mon[87351]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:56:58.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:58 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3585140717' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:56:58.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.937+0000 7fc257400700 1 -- 172.21.15.38:0/2371068343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2500ff950 msgr2=0x7fc2500ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:58.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.937+0000 7fc257400700 1 --2- 172.21.15.38:0/2371068343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2500ff950 0x7fc2500ffdd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009910 tx=0x7fc23c009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:58.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.938+0000 7fc257400700 1 -- 172.21.15.38:0/2371068343 shutdown_connections 2024-03-24T00:56:58.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.938+0000 7fc257400700 1 --2- 172.21.15.38:0/2371068343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2500ff950 0x7fc2500ffdd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.938+0000 7fc257400700 1 --2- 172.21.15.38:0/2371068343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc250105950 0x7fc250105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.938+0000 7fc257400700 1 -- 172.21.15.38:0/2371068343 >> 172.21.15.38:0/2371068343 conn(0x7fc250076640 msgr2=0x7fc2500fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:58.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.938+0000 7fc257400700 1 -- 172.21.15.38:0/2371068343 shutdown_connections 2024-03-24T00:56:58.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.938+0000 7fc257400700 1 -- 172.21.15.38:0/2371068343 wait complete. 2024-03-24T00:56:58.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.939+0000 7fc257400700 1 Processor -- start 2024-03-24T00:56:58.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.939+0000 7fc257400700 1 -- start start 2024-03-24T00:56:58.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.940+0000 7fc257400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2500ff950 0x7fc25019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:58.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.940+0000 7fc257400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc250105950 0x7fc25019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:58.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.940+0000 7fc257400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25019f330 con 0x7fc2500ff950 2024-03-24T00:56:58.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.940+0000 7fc257400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2501989a0 con 0x7fc250105950 2024-03-24T00:56:58.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.940+0000 7fc254a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2500ff950 0x7fc25019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:58.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.940+0000 7fc254a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2500ff950 0x7fc25019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60270/0 (socket says 172.21.15.38:60270) 2024-03-24T00:56:58.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.940+0000 7fc254a00700 1 -- 172.21.15.38:0/2614041594 learned_addr learned my addr 172.21.15.38:0/2614041594 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:58.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.940+0000 7fc254a00700 1 -- 172.21.15.38:0/2614041594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc250105950 msgr2=0x7fc25019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:58.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.940+0000 7fc254a00700 1 --2- 172.21.15.38:0/2614041594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc250105950 0x7fc25019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:58.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.940+0000 7fc254a00700 1 -- 172.21.15.38:0/2614041594 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c0095f0 con 0x7fc2500ff950 2024-03-24T00:56:58.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.941+0000 7fc254a00700 1 --2- 172.21.15.38:0/2614041594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2500ff950 0x7fc25019e860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc24400c930 tx=0x7fc24400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:58.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.941+0000 7fc24d600700 1 -- 172.21.15.38:0/2614041594 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc244007ad0 con 0x7fc2500ff950 2024-03-24T00:56:58.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.941+0000 7fc24d600700 1 -- 172.21.15.38:0/2614041594 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc244007c30 con 0x7fc2500ff950 2024-03-24T00:56:58.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.941+0000 7fc257400700 1 -- 172.21.15.38:0/2614041594 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc250198c80 con 0x7fc2500ff950 2024-03-24T00:56:58.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.941+0000 7fc24d600700 1 -- 172.21.15.38:0/2614041594 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc244018610 con 0x7fc2500ff950 2024-03-24T00:56:58.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.941+0000 7fc257400700 1 -- 172.21.15.38:0/2614041594 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2501991d0 con 0x7fc2500ff950 2024-03-24T00:56:58.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.942+0000 7fc257400700 1 -- 172.21.15.38:0/2614041594 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2500fe1d0 con 0x7fc2500ff950 2024-03-24T00:56:58.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.943+0000 7fc24d600700 1 -- 172.21.15.38:0/2614041594 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fc24400f450 con 0x7fc2500ff950 2024-03-24T00:56:58.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.944+0000 7fc24d600700 1 --2- 172.21.15.38:0/2614041594 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc23806dfb0 0x7fc238070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:58.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.944+0000 7fc24d600700 1 -- 172.21.15.38:0/2614041594 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc24408dff0 con 0x7fc2500ff950 2024-03-24T00:56:58.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.944+0000 7fc24fe00700 1 --2- 172.21.15.38:0/2614041594 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc23806dfb0 0x7fc238070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:58.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.945+0000 7fc24fe00700 1 --2- 172.21.15.38:0/2614041594 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc23806dfb0 0x7fc238070470 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc23c006010 tx=0x7fc23c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:58.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:58.949+0000 7fc24d600700 1 -- 172.21.15.38:0/2614041594 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc2440c2740 con 0x7fc2500ff950 2024-03-24T00:56:59.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.189+0000 7fc257400700 1 -- 172.21.15.38:0/2614041594 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc250199e60 con 0x7fc23806dfb0 2024-03-24T00:56:59.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.192+0000 7fc24d600700 1 -- 172.21.15.38:0/2614041594 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc250199e60 con 0x7fc23806dfb0 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout: "crash" 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "8/23 daemons upgraded", 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T00:56:59.194 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:56:59.195 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:56:59.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.194+0000 7fc257400700 1 -- 172.21.15.38:0/2614041594 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc23806dfb0 msgr2=0x7fc238070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:59.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.194+0000 7fc257400700 1 --2- 172.21.15.38:0/2614041594 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc23806dfb0 0x7fc238070470 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc23c006010 tx=0x7fc23c00b540 comp rx=0 tx=0).stop 2024-03-24T00:56:59.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.194+0000 7fc257400700 1 -- 172.21.15.38:0/2614041594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2500ff950 msgr2=0x7fc25019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:59.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.194+0000 7fc257400700 1 --2- 172.21.15.38:0/2614041594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2500ff950 0x7fc25019e860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc24400c930 tx=0x7fc24400ccf0 comp rx=0 tx=0).stop 2024-03-24T00:56:59.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.194+0000 7fc257400700 1 -- 172.21.15.38:0/2614041594 shutdown_connections 2024-03-24T00:56:59.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.194+0000 7fc257400700 1 --2- 172.21.15.38:0/2614041594 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc23806dfb0 0x7fc238070470 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:59.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.194+0000 7fc257400700 1 --2- 172.21.15.38:0/2614041594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2500ff950 0x7fc25019e860 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:59.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.194+0000 7fc257400700 1 --2- 172.21.15.38:0/2614041594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc250105950 0x7fc25019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:59.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.194+0000 7fc257400700 1 -- 172.21.15.38:0/2614041594 >> 172.21.15.38:0/2614041594 conn(0x7fc250076640 msgr2=0x7fc2501030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:59.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.195+0000 7fc257400700 1 -- 172.21.15.38:0/2614041594 shutdown_connections 2024-03-24T00:56:59.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.195+0000 7fc257400700 1 -- 172.21.15.38:0/2614041594 wait complete. 2024-03-24T00:56:59.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.350+0000 7fde25c00700 1 -- 172.21.15.38:0/1122703230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde20102550 msgr2=0x7fde2010aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:59.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.350+0000 7fde25c00700 1 --2- 172.21.15.38:0/1122703230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde20102550 0x7fde2010aa40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009910 tx=0x7fde0c009c20 comp rx=0 tx=0).stop 2024-03-24T00:56:59.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.350+0000 7fde25c00700 1 -- 172.21.15.38:0/1122703230 shutdown_connections 2024-03-24T00:56:59.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.350+0000 7fde25c00700 1 --2- 172.21.15.38:0/1122703230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde20102550 0x7fde2010aa40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:59.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.350+0000 7fde25c00700 1 --2- 172.21.15.38:0/1122703230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde20101ba0 0x7fde20101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:59.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.350+0000 7fde25c00700 1 -- 172.21.15.38:0/1122703230 >> 172.21.15.38:0/1122703230 conn(0x7fde200fb480 msgr2=0x7fde200fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:59.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.351+0000 7fde25c00700 1 -- 172.21.15.38:0/1122703230 shutdown_connections 2024-03-24T00:56:59.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.351+0000 7fde25c00700 1 -- 172.21.15.38:0/1122703230 wait complete. 2024-03-24T00:56:59.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.352+0000 7fde25c00700 1 Processor -- start 2024-03-24T00:56:59.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.352+0000 7fde25c00700 1 -- start start 2024-03-24T00:56:59.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.353+0000 7fde25c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde20101ba0 0x7fde200ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:59.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.353+0000 7fde25c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde20102550 0x7fde200ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:59.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.353+0000 7fde25c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde20101860 con 0x7fde20102550 2024-03-24T00:56:59.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.353+0000 7fde25c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde20100310 con 0x7fde20101ba0 2024-03-24T00:56:59.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.353+0000 7fde1e000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde20102550 0x7fde200ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:59.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.354+0000 7fde1e000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde20102550 0x7fde200ffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60284/0 (socket says 172.21.15.38:60284) 2024-03-24T00:56:59.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.354+0000 7fde1e000700 1 -- 172.21.15.38:0/2839711809 learned_addr learned my addr 172.21.15.38:0/2839711809 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:56:59.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.354+0000 7fde1ea00700 1 --2- 172.21.15.38:0/2839711809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde20101ba0 0x7fde200ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:59.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.354+0000 7fde1e000700 1 -- 172.21.15.38:0/2839711809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde20101ba0 msgr2=0x7fde200ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:59.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.354+0000 7fde1e000700 1 --2- 172.21.15.38:0/2839711809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde20101ba0 0x7fde200ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:59.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.354+0000 7fde1e000700 1 -- 172.21.15.38:0/2839711809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c0095f0 con 0x7fde20102550 2024-03-24T00:56:59.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.355+0000 7fde1e000700 1 --2- 172.21.15.38:0/2839711809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde20102550 0x7fde200ffdd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005b40 tx=0x7fde0c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:59.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.355+0000 7fde17400700 1 -- 172.21.15.38:0/2839711809 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde0c01d070 con 0x7fde20102550 2024-03-24T00:56:59.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.355+0000 7fde25c00700 1 -- 172.21.15.38:0/2839711809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde20100560 con 0x7fde20102550 2024-03-24T00:56:59.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.355+0000 7fde17400700 1 -- 172.21.15.38:0/2839711809 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde0c00f460 con 0x7fde20102550 2024-03-24T00:56:59.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.355+0000 7fde17400700 1 -- 172.21.15.38:0/2839711809 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde0c005720 con 0x7fde20102550 2024-03-24T00:56:59.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.355+0000 7fde25c00700 1 -- 172.21.15.38:0/2839711809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde201a2d80 con 0x7fde20102550 2024-03-24T00:56:59.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.356+0000 7fde25c00700 1 -- 172.21.15.38:0/2839711809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde200695f0 con 0x7fde20102550 2024-03-24T00:56:59.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.358+0000 7fde17400700 1 -- 172.21.15.38:0/2839711809 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fde0c003a60 con 0x7fde20102550 2024-03-24T00:56:59.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.359+0000 7fde17400700 1 --2- 172.21.15.38:0/2839711809 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fde0406dfb0 0x7fde04070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:56:59.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.359+0000 7fde17400700 1 -- 172.21.15.38:0/2839711809 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fde0c08f090 con 0x7fde20102550 2024-03-24T00:56:59.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.359+0000 7fde1ea00700 1 --2- 172.21.15.38:0/2839711809 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fde0406dfb0 0x7fde04070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:56:59.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.360+0000 7fde1ea00700 1 --2- 172.21.15.38:0/2839711809 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fde0406dfb0 0x7fde04070470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fde1000f440 tx=0x7fde100079e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:56:59.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.364+0000 7fde17400700 1 -- 172.21.15.38:0/2839711809 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fde0c058540 con 0x7fde20102550 2024-03-24T00:56:59.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.708+0000 7fde25c00700 1 -- 172.21.15.38:0/2839711809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fde2004ea80 con 0x7fde20102550 2024-03-24T00:56:59.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.709+0000 7fde17400700 1 -- 172.21.15.38:0/2839711809 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fde0c05bc00 con 0x7fde20102550 2024-03-24T00:56:59.710 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_OK 2024-03-24T00:56:59.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.711+0000 7fde25c00700 1 -- 172.21.15.38:0/2839711809 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fde0406dfb0 msgr2=0x7fde04070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:59.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.711+0000 7fde25c00700 1 --2- 172.21.15.38:0/2839711809 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fde0406dfb0 0x7fde04070470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fde1000f440 tx=0x7fde100079e0 comp rx=0 tx=0).stop 2024-03-24T00:56:59.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.711+0000 7fde25c00700 1 -- 172.21.15.38:0/2839711809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde20102550 msgr2=0x7fde200ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:56:59.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.711+0000 7fde25c00700 1 --2- 172.21.15.38:0/2839711809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde20102550 0x7fde200ffdd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005b40 tx=0x7fde0c00bd70 comp rx=0 tx=0).stop 2024-03-24T00:56:59.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.712+0000 7fde25c00700 1 -- 172.21.15.38:0/2839711809 shutdown_connections 2024-03-24T00:56:59.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.712+0000 7fde25c00700 1 --2- 172.21.15.38:0/2839711809 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fde0406dfb0 0x7fde04070470 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:59.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.712+0000 7fde25c00700 1 --2- 172.21.15.38:0/2839711809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde20101ba0 0x7fde200ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:59.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.712+0000 7fde25c00700 1 --2- 172.21.15.38:0/2839711809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde20102550 0x7fde200ffdd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:56:59.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.712+0000 7fde25c00700 1 -- 172.21.15.38:0/2839711809 >> 172.21.15.38:0/2839711809 conn(0x7fde200fb480 msgr2=0x7fde20102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:56:59.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.712+0000 7fde25c00700 1 -- 172.21.15.38:0/2839711809 shutdown_connections 2024-03-24T00:56:59.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:56:59.712+0000 7fde25c00700 1 -- 172.21.15.38:0/2839711809 wait complete. 2024-03-24T00:56:59.938 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:59 smithi038 ceph-mon[112108]: pgmap v104: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:56:59.939 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:59 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1266750654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:56:59.939 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:56:59 smithi038 ceph-mon[112108]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:57:00.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:59 smithi088 ceph-mon[87351]: pgmap v104: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:57:00.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:59 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1266750654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:57:00.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:56:59 smithi088 ceph-mon[87351]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:57:00.959 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:00 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2839711809' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:57:01.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:00 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2839711809' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:57:02.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:01 smithi038 ceph-mon[112108]: pgmap v105: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:57:02.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:01 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:02.110 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:01 smithi038 systemd[1]: Stopping Ceph osd.2 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:57:02.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:01 smithi088 ceph-mon[87351]: pgmap v105: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:57:02.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:01 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:02.610 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:02 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2[62960]: 2024-03-24T00:57:02.303+0000 7f652d200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T00:57:02.610 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:02 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2[62960]: 2024-03-24T00:57:02.303+0000 7f652d200700 -1 osd.2 60 *** Got signal Terminated *** 2024-03-24T00:57:02.610 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:02 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2[62960]: 2024-03-24T00:57:02.303+0000 7f652d200700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T00:57:03.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:02 smithi038 ceph-mon[112108]: osd.2 marked itself down and dead 2024-03-24T00:57:03.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:02 smithi088 ceph-mon[87351]: osd.2 marked itself down and dead 2024-03-24T00:57:03.609 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:03 smithi038 podman[134301]: 2024-03-24 00:57:03.190141461 +0000 UTC m=+1.035028411 container died 00659d84b2598a9e298c86676a6ae7ee400931b861d636f53809dd3133c77f77 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=reef-738c2fd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-24T00:57:04.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:03 smithi038 ceph-mon[112108]: pgmap v106: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:57:04.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:03 smithi038 ceph-mon[112108]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:57:04.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:03 smithi038 ceph-mon[112108]: osdmap e61: 6 total, 5 up, 6 in 2024-03-24T00:57:04.023 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:03 smithi038 podman[134301]: 2024-03-24 00:57:03.779864585 +0000 UTC m=+1.624751551 container cleanup 00659d84b2598a9e298c86676a6ae7ee400931b861d636f53809dd3133c77f77 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-738c2fd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-24T00:57:04.023 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:03 smithi038 bash[134301]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2 2024-03-24T00:57:04.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:03 smithi088 ceph-mon[87351]: pgmap v106: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:57:04.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:03 smithi088 ceph-mon[87351]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:57:04.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:03 smithi088 ceph-mon[87351]: osdmap e61: 6 total, 5 up, 6 in 2024-03-24T00:57:04.359 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:04 smithi038 podman[134321]: 2024-03-24 00:57:04.022629846 +0000 UTC m=+0.819256022 container remove 00659d84b2598a9e298c86676a6ae7ee400931b861d636f53809dd3133c77f77 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=reef-738c2fd, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-24T00:57:04.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:04 smithi038 ceph-mon[112108]: osdmap e62: 6 total, 5 up, 6 in 2024-03-24T00:57:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:04 smithi088 ceph-mon[87351]: osdmap e62: 6 total, 5 up, 6 in 2024-03-24T00:57:05.194 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:04 smithi038 podman[134412]: 2024-03-24 00:57:04.805233855 +0000 UTC m=+0.037802206 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:57:05.195 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:04 smithi038 podman[134412]: 2024-03-24 00:57:04.956884233 +0000 UTC m=+0.189452613 container create 73baa53313292e078b364a7cf19a3844e2ffa04f7d951dfd75a67c1ac5c7ba68 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-24T00:57:05.195 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:05 smithi038 podman[134412]: 2024-03-24 00:57:05.193766868 +0000 UTC m=+0.426335319 container init 73baa53313292e078b364a7cf19a3844e2ffa04f7d951dfd75a67c1ac5c7ba68 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-24T00:57:05.609 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:05 smithi038 podman[134412]: 2024-03-24 00:57:05.205651863 +0000 UTC m=+0.438220266 container start 73baa53313292e078b364a7cf19a3844e2ffa04f7d951dfd75a67c1ac5c7ba68 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-24T00:57:05.610 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:05 smithi038 podman[134412]: 2024-03-24 00:57:05.275852056 +0000 UTC m=+0.508420469 container attach 73baa53313292e078b364a7cf19a3844e2ffa04f7d951dfd75a67c1ac5c7ba68 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-24T00:57:06.077 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:05 smithi038 ceph-mon[112108]: pgmap v109: 65 pgs: 9 stale+active+clean, 56 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:57:06.077 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:05 smithi038 podman[134412]: 2024-03-24 00:57:05.667717607 +0000 UTC m=+0.900286028 container died 73baa53313292e078b364a7cf19a3844e2ffa04f7d951dfd75a67c1ac5c7ba68 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-24T00:57:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:05 smithi088 ceph-mon[87351]: pgmap v109: 65 pgs: 9 stale+active+clean, 56 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T00:57:06.726 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:06 smithi038 podman[134437]: 2024-03-24 00:57:06.387619237 +0000 UTC m=+0.704518668 container remove 73baa53313292e078b364a7cf19a3844e2ffa04f7d951dfd75a67c1ac5c7ba68 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T00:57:06.726 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:06 smithi038 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-24T00:57:06.726 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:06 smithi038 systemd[1]: Stopped Ceph osd.2 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:57:06.726 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:06 smithi038 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.2.service: Consumed 1min 37.518s CPU time. 2024-03-24T00:57:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:06 smithi038 ceph-mon[112108]: Health check failed: Degraded data redundancy: 10/273 objects degraded (3.663%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:07.110 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:06 smithi038 systemd[1]: Starting Ceph osd.2 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:57:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:06 smithi088 ceph-mon[87351]: Health check failed: Degraded data redundancy: 10/273 objects degraded (3.663%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:07.462 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:07 smithi038 podman[134536]: 2024-03-24 00:57:07.089701309 +0000 UTC m=+0.037760150 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:57:07.462 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:07 smithi038 podman[134536]: 2024-03-24 00:57:07.205566261 +0000 UTC m=+0.153625144 container create 34998669e4fdb51307f7e4f648da9dbe5c4612fb74fed227936ec6db9905d947 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2) 2024-03-24T00:57:07.462 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:07 smithi038 podman[134536]: 2024-03-24 00:57:07.461808463 +0000 UTC m=+0.409867398 container init 34998669e4fdb51307f7e4f648da9dbe5c4612fb74fed227936ec6db9905d947 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:57:07.795 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:07 smithi038 podman[134536]: 2024-03-24 00:57:07.475256113 +0000 UTC m=+0.423315012 container start 34998669e4fdb51307f7e4f648da9dbe5c4612fb74fed227936ec6db9905d947 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True) 2024-03-24T00:57:07.795 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:07 smithi038 podman[134536]: 2024-03-24 00:57:07.599734913 +0000 UTC m=+0.547793832 container attach 34998669e4fdb51307f7e4f648da9dbe5c4612fb74fed227936ec6db9905d947 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-24T00:57:08.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:07 smithi038 ceph-mon[112108]: pgmap v110: 65 pgs: 3 active+undersized+degraded, 6 stale+active+clean, 5 active+undersized, 51 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 10/273 objects degraded (3.663%) 2024-03-24T00:57:08.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:07 smithi088 ceph-mon[87351]: pgmap v110: 65 pgs: 3 active+undersized+degraded, 6 stale+active+clean, 5 active+undersized, 51 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 10/273 objects degraded (3.663%) 2024-03-24T00:57:08.408 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate[134551]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T00:57:08.408 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate[134551]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T00:57:08.408 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 bash[134536]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T00:57:08.408 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 bash[134536]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T00:57:08.408 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 bash[134536]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T00:57:08.408 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 bash[134536]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T00:57:08.409 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 bash[134536]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-24T00:57:08.409 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 bash[134536]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T00:57:08.409 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 bash[134536]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-24T00:57:08.409 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate[134551]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T00:57:08.409 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate[134551]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T00:57:08.409 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate[134551]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-24T00:57:08.409 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate[134551]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T00:57:08.409 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate[134551]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-24T00:57:08.409 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:08 smithi038 podman[134536]: 2024-03-24 00:57:08.407306568 +0000 UTC m=+1.355365487 container died 34998669e4fdb51307f7e4f648da9dbe5c4612fb74fed227936ec6db9905d947 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:57:09.541 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:09 smithi038 podman[134676]: 2024-03-24 00:57:09.178366928 +0000 UTC m=+0.746412868 container remove 34998669e4fdb51307f7e4f648da9dbe5c4612fb74fed227936ec6db9905d947 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-24T00:57:09.801 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:09 smithi038 podman[134737]: 2024-03-24 00:57:09.447561149 +0000 UTC m=+0.036382946 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:57:09.801 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:09 smithi038 podman[134737]: 2024-03-24 00:57:09.554144542 +0000 UTC m=+0.142966370 container create c73f3e8276b63995c4ddb70752dcd636dc841609ba1adc15392f26e8a6a8831b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-24T00:57:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:09 smithi038 ceph-mon[112108]: pgmap v111: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:10.110 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:09 smithi038 podman[134737]: 2024-03-24 00:57:09.841099575 +0000 UTC m=+0.429921404 container init c73f3e8276b63995c4ddb70752dcd636dc841609ba1adc15392f26e8a6a8831b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-24T00:57:10.110 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:09 smithi038 podman[134737]: 2024-03-24 00:57:09.852975691 +0000 UTC m=+0.441797486 container start c73f3e8276b63995c4ddb70752dcd636dc841609ba1adc15392f26e8a6a8831b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-24T00:57:10.110 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:09 smithi038 ceph-osd[134756]: -- 172.21.15.38:0/4168602110 <== mon.1 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5555e508b680 con 0x5555e5055000 2024-03-24T00:57:10.110 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:09 smithi038 bash[134737]: c73f3e8276b63995c4ddb70752dcd636dc841609ba1adc15392f26e8a6a8831b 2024-03-24T00:57:10.110 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:09 smithi038 systemd[1]: Started Ceph osd.2 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:57:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:09 smithi088 ceph-mon[87351]: pgmap v111: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:10.859 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:10 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2[134752]: 2024-03-24T00:57:10.423+0000 7fba42f837c0 -1 Falling back to public interface 2024-03-24T00:57:11.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:11.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:11.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:57:11.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:11.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:11.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:11.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:11.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:57:11.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:11.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:12.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:12 smithi088 ceph-mon[87351]: pgmap v112: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 1 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:12.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:12 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:12.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:12 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:57:12.482 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:12 smithi038 ceph-mon[112108]: pgmap v112: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 1 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:12.482 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:12.482 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:57:14.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:14 smithi088 ceph-mon[87351]: pgmap v113: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:14.455 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:14 smithi038 ceph-mon[112108]: pgmap v113: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:16.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:16 smithi088 ceph-mon[87351]: pgmap v114: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:16.440 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:16 smithi038 ceph-mon[112108]: pgmap v114: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:16.440 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:16 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2[134752]: 2024-03-24T00:57:16.204+0000 7fba42f837c0 -1 osd.2 60 log_to_monitors true 2024-03-24T00:57:17.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:17 smithi088 ceph-mon[87351]: from='osd.2 [v2:172.21.15.38:6818/874077504,v1:172.21.15.38:6819/874077504]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T00:57:17.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:17 smithi088 ceph-mon[87351]: Health check update: Degraded data redundancy: 39/273 objects degraded (14.286%), 12 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:17.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:17 smithi038 ceph-mon[112108]: from='osd.2 [v2:172.21.15.38:6818/874077504,v1:172.21.15.38:6819/874077504]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T00:57:17.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:17 smithi038 ceph-mon[112108]: Health check update: Degraded data redundancy: 39/273 objects degraded (14.286%), 12 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:17.609 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 00:57:17 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2[134752]: 2024-03-24T00:57:17.172+0000 7fba32400700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T00:57:18.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:18 smithi088 ceph-mon[87351]: pgmap v115: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:18.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:18 smithi088 ceph-mon[87351]: from='osd.2 [v2:172.21.15.38:6818/874077504,v1:172.21.15.38:6819/874077504]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T00:57:18.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:18 smithi088 ceph-mon[87351]: osdmap e63: 6 total, 5 up, 6 in 2024-03-24T00:57:18.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:18 smithi088 ceph-mon[87351]: from='osd.2 [v2:172.21.15.38:6818/874077504,v1:172.21.15.38:6819/874077504]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:57:18.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:18 smithi038 ceph-mon[112108]: pgmap v115: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:18.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:18 smithi038 ceph-mon[112108]: from='osd.2 [v2:172.21.15.38:6818/874077504,v1:172.21.15.38:6819/874077504]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T00:57:18.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:18 smithi038 ceph-mon[112108]: osdmap e63: 6 total, 5 up, 6 in 2024-03-24T00:57:18.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:18 smithi038 ceph-mon[112108]: from='osd.2 [v2:172.21.15.38:6818/874077504,v1:172.21.15.38:6819/874077504]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-03-24T00:57:19.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:19 smithi088 ceph-mon[87351]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:57:19.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:19 smithi088 ceph-mon[87351]: osd.2 [v2:172.21.15.38:6818/874077504,v1:172.21.15.38:6819/874077504] boot 2024-03-24T00:57:19.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:19 smithi088 ceph-mon[87351]: osdmap e64: 6 total, 6 up, 6 in 2024-03-24T00:57:19.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:19 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:57:19.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:19 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:19.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:19 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:19 smithi038 ceph-mon[112108]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:57:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:19 smithi038 ceph-mon[112108]: osd.2 [v2:172.21.15.38:6818/874077504,v1:172.21.15.38:6819/874077504] boot 2024-03-24T00:57:19.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:19 smithi038 ceph-mon[112108]: osdmap e64: 6 total, 6 up, 6 in 2024-03-24T00:57:19.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:19 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T00:57:19.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:19 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:19.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:19 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:20.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:20 smithi038 ceph-mon[112108]: pgmap v118: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:20.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:20 smithi038 ceph-mon[112108]: osdmap e65: 6 total, 6 up, 6 in 2024-03-24T00:57:20.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:20 smithi088 ceph-mon[87351]: pgmap v118: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 39/273 objects degraded (14.286%) 2024-03-24T00:57:20.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:20 smithi088 ceph-mon[87351]: osdmap e65: 6 total, 6 up, 6 in 2024-03-24T00:57:21.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:21.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:21.798 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:21.798 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:22.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:22 smithi088 ceph-mon[87351]: pgmap v120: 65 pgs: 11 active+undersized+degraded, 11 active+undersized, 43 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 35/273 objects degraded (12.821%) 2024-03-24T00:57:22.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:22 smithi088 ceph-mon[87351]: Health check update: Degraded data redundancy: 35/273 objects degraded (12.821%), 11 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:22.734 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:22 smithi038 ceph-mon[112108]: pgmap v120: 65 pgs: 11 active+undersized+degraded, 11 active+undersized, 43 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 35/273 objects degraded (12.821%) 2024-03-24T00:57:22.735 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:22 smithi038 ceph-mon[112108]: Health check update: Degraded data redundancy: 35/273 objects degraded (12.821%), 11 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:23.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:23 smithi088 ceph-mon[87351]: pgmap v121: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-24T00:57:23.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:23 smithi088 ceph-mon[87351]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 35/273 objects degraded (12.821%), 11 pgs degraded) 2024-03-24T00:57:23.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:23 smithi088 ceph-mon[87351]: Cluster is now healthy 2024-03-24T00:57:23.819 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:23 smithi038 ceph-mon[112108]: pgmap v121: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-24T00:57:23.820 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:23 smithi038 ceph-mon[112108]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 35/273 objects degraded (12.821%), 11 pgs degraded) 2024-03-24T00:57:23.820 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:23 smithi038 ceph-mon[112108]: Cluster is now healthy 2024-03-24T00:57:25.980 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:25 smithi038 ceph-mon[112108]: pgmap v122: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-24T00:57:26.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:25 smithi088 ceph-mon[87351]: pgmap v122: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-24T00:57:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:57:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:57:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:27.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:57:27.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:57:27.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:27.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:27.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:27.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T00:57:27.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:27.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:27.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:57:27.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:57:27.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:27.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:27.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:57:27.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:57:27.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:27.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:27.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:27.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T00:57:28.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:28 smithi088 ceph-mon[87351]: pgmap v123: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T00:57:28.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:28 smithi088 ceph-mon[87351]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T00:57:28.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:28 smithi088 ceph-mon[87351]: Upgrade: osd.3 is safe to restart 2024-03-24T00:57:28.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:28 smithi038 ceph-mon[112108]: pgmap v123: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T00:57:28.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:28 smithi038 ceph-mon[112108]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T00:57:28.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:28 smithi038 ceph-mon[112108]: Upgrade: osd.3 is safe to restart 2024-03-24T00:57:29.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.889+0000 7f07bf200700 1 -- 172.21.15.38:0/2281816999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07c0107b90 msgr2=0x7f07c010bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:29.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.889+0000 7f07bf200700 1 --2- 172.21.15.38:0/2281816999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07c0107b90 0x7f07c010bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009910 tx=0x7f07ac009c20 comp rx=0 tx=0).stop 2024-03-24T00:57:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.890+0000 7f07bf200700 1 -- 172.21.15.38:0/2281816999 shutdown_connections 2024-03-24T00:57:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.890+0000 7f07bf200700 1 --2- 172.21.15.38:0/2281816999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07c0107b90 0x7f07c010bb30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.890+0000 7f07bf200700 1 --2- 172.21.15.38:0/2281816999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07c01071e0 0x7f07c01075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.890+0000 7f07bf200700 1 -- 172.21.15.38:0/2281816999 >> 172.21.15.38:0/2281816999 conn(0x7f07c0076700 msgr2=0x7f07c0078b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.890+0000 7f07bf200700 1 -- 172.21.15.38:0/2281816999 shutdown_connections 2024-03-24T00:57:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.890+0000 7f07bf200700 1 -- 172.21.15.38:0/2281816999 wait complete. 2024-03-24T00:57:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.891+0000 7f07bf200700 1 Processor -- start 2024-03-24T00:57:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.891+0000 7f07bf200700 1 -- start start 2024-03-24T00:57:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.892+0000 7f07bf200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07c01071e0 0x7f07c019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.892+0000 7f07bf200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07c0107b90 0x7f07c019d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.892+0000 7f07bf200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c019da40 con 0x7f07c01071e0 2024-03-24T00:57:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.892+0000 7f07bf200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c01a1710 con 0x7f07c0107b90 2024-03-24T00:57:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.892+0000 7f07bde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07c01071e0 0x7f07c019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.892+0000 7f07bde00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07c01071e0 0x7f07c019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56872/0 (socket says 172.21.15.38:56872) 2024-03-24T00:57:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.892+0000 7f07bde00700 1 -- 172.21.15.38:0/827680954 learned_addr learned my addr 172.21.15.38:0/827680954 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:57:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.892+0000 7f07bd400700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07c0107b90 0x7f07c019d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:29.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.893+0000 7f07bde00700 1 -- 172.21.15.38:0/827680954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07c0107b90 msgr2=0x7f07c019d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:29.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.893+0000 7f07bde00700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07c0107b90 0x7f07c019d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:29.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.893+0000 7f07bde00700 1 -- 172.21.15.38:0/827680954 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac0095f0 con 0x7f07c01071e0 2024-03-24T00:57:29.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.893+0000 7f07bd400700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07c0107b90 0x7f07c019d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T00:57:29.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.893+0000 7f07bde00700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07c01071e0 0x7f07c019ce20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f07b000e920 tx=0x7f07b000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:29.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.893+0000 7f07b6a00700 1 -- 172.21.15.38:0/827680954 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07b000c9d0 con 0x7f07c01071e0 2024-03-24T00:57:29.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.893+0000 7f07b6a00700 1 -- 172.21.15.38:0/827680954 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07b0010430 con 0x7f07c01071e0 2024-03-24T00:57:29.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.893+0000 7f07b6a00700 1 -- 172.21.15.38:0/827680954 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07b0018630 con 0x7f07c01071e0 2024-03-24T00:57:29.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.893+0000 7f07bf200700 1 -- 172.21.15.38:0/827680954 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c01a19f0 con 0x7f07c01071e0 2024-03-24T00:57:29.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.893+0000 7f07bf200700 1 -- 172.21.15.38:0/827680954 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c01a1f10 con 0x7f07c01071e0 2024-03-24T00:57:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.895+0000 7f07bf200700 1 -- 172.21.15.38:0/827680954 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c010f530 con 0x7f07c01071e0 2024-03-24T00:57:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.896+0000 7f07b6a00700 1 -- 172.21.15.38:0/827680954 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f07b00105a0 con 0x7f07c01071e0 2024-03-24T00:57:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.896+0000 7f07b6a00700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07a406dfb0 0x7f07a4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.897+0000 7f07b6a00700 1 -- 172.21.15.38:0/827680954 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f07b0014070 con 0x7f07c01071e0 2024-03-24T00:57:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.897+0000 7f07bd400700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07a406dfb0 0x7f07a4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.898+0000 7f07bd400700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07a406dfb0 0x7f07a4070470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009fd0 tx=0x7f07ac005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:29.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:29.902+0000 7f07b6a00700 1 -- 172.21.15.38:0/827680954 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f07b0053740 con 0x7f07c01071e0 2024-03-24T00:57:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.148+0000 7f07bf200700 1 -- 172.21.15.38:0/827680954 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f07c019e1d0 con 0x7f07a406dfb0 2024-03-24T00:57:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.152+0000 7f07b6a00700 1 -- 172.21.15.38:0/827680954 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f07c019e1d0 con 0x7f07a406dfb0 2024-03-24T00:57:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.155+0000 7f07bf200700 1 -- 172.21.15.38:0/827680954 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07a406dfb0 msgr2=0x7f07a4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.155+0000 7f07bf200700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07a406dfb0 0x7f07a4070470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009fd0 tx=0x7f07ac005a90 comp rx=0 tx=0).stop 2024-03-24T00:57:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.155+0000 7f07bf200700 1 -- 172.21.15.38:0/827680954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07c01071e0 msgr2=0x7f07c019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.155+0000 7f07bf200700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07c01071e0 0x7f07c019ce20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f07b000e920 tx=0x7f07b000ec30 comp rx=0 tx=0).stop 2024-03-24T00:57:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.155+0000 7f07bf200700 1 -- 172.21.15.38:0/827680954 shutdown_connections 2024-03-24T00:57:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.155+0000 7f07bf200700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07a406dfb0 0x7f07a4070470 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.155+0000 7f07bf200700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07c01071e0 0x7f07c019ce20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.155+0000 7f07bf200700 1 --2- 172.21.15.38:0/827680954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07c0107b90 0x7f07c019d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.155+0000 7f07bf200700 1 -- 172.21.15.38:0/827680954 >> 172.21.15.38:0/827680954 conn(0x7f07c0076700 msgr2=0x7f07c0077ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.156+0000 7f07bf200700 1 -- 172.21.15.38:0/827680954 shutdown_connections 2024-03-24T00:57:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.156+0000 7f07bf200700 1 -- 172.21.15.38:0/827680954 wait complete. 2024-03-24T00:57:30.172 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:57:30.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.337+0000 7fd282000700 1 -- 172.21.15.38:0/3376617960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c0ffac0 msgr2=0x7fd27c0ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:30.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.337+0000 7fd282000700 1 --2- 172.21.15.38:0/3376617960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c0ffac0 0x7fd27c0ffea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009910 tx=0x7fd26c009c20 comp rx=0 tx=0).stop 2024-03-24T00:57:30.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.337+0000 7fd282000700 1 -- 172.21.15.38:0/3376617960 shutdown_connections 2024-03-24T00:57:30.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.337+0000 7fd282000700 1 --2- 172.21.15.38:0/3376617960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27c1003e0 0x7fd27c1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.337+0000 7fd282000700 1 --2- 172.21.15.38:0/3376617960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c0ffac0 0x7fd27c0ffea0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.337+0000 7fd282000700 1 -- 172.21.15.38:0/3376617960 >> 172.21.15.38:0/3376617960 conn(0x7fd27c0fb540 msgr2=0x7fd27c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:30.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.337+0000 7fd282000700 1 -- 172.21.15.38:0/3376617960 shutdown_connections 2024-03-24T00:57:30.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.337+0000 7fd282000700 1 -- 172.21.15.38:0/3376617960 wait complete. 2024-03-24T00:57:30.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.338+0000 7fd282000700 1 Processor -- start 2024-03-24T00:57:30.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.338+0000 7fd282000700 1 -- start start 2024-03-24T00:57:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.339+0000 7fd282000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27c0ffac0 0x7fd27c19d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.339+0000 7fd282000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c1003e0 0x7fd27c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.339+0000 7fd282000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c19dc80 con 0x7fd27c1003e0 2024-03-24T00:57:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.339+0000 7fd282000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c1a1950 con 0x7fd27c0ffac0 2024-03-24T00:57:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.339+0000 7fd27aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c1003e0 0x7fd27c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.339+0000 7fd27aa00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c1003e0 0x7fd27c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56902/0 (socket says 172.21.15.38:56902) 2024-03-24T00:57:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.339+0000 7fd27aa00700 1 -- 172.21.15.38:0/2852974663 learned_addr learned my addr 172.21.15.38:0/2852974663 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:57:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.339+0000 7fd27b400700 1 --2- 172.21.15.38:0/2852974663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27c0ffac0 0x7fd27c19d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.340+0000 7fd27aa00700 1 -- 172.21.15.38:0/2852974663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27c0ffac0 msgr2=0x7fd27c19d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.340+0000 7fd27aa00700 1 --2- 172.21.15.38:0/2852974663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27c0ffac0 0x7fd27c19d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.340+0000 7fd27aa00700 1 -- 172.21.15.38:0/2852974663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c0095f0 con 0x7fd27c1003e0 2024-03-24T00:57:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.340+0000 7fd27aa00700 1 --2- 172.21.15.38:0/2852974663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c1003e0 0x7fd27c19d5a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd26800d6d0 tx=0x7fd26800d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.341+0000 7fd273e00700 1 -- 172.21.15.38:0/2852974663 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26800f860 con 0x7fd27c1003e0 2024-03-24T00:57:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.341+0000 7fd273e00700 1 -- 172.21.15.38:0/2852974663 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd26800fe80 con 0x7fd27c1003e0 2024-03-24T00:57:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.341+0000 7fd282000700 1 -- 172.21.15.38:0/2852974663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c1a1c30 con 0x7fd27c1003e0 2024-03-24T00:57:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.341+0000 7fd273e00700 1 -- 172.21.15.38:0/2852974663 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd268010ea0 con 0x7fd27c1003e0 2024-03-24T00:57:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.341+0000 7fd282000700 1 -- 172.21.15.38:0/2852974663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c1a2180 con 0x7fd27c1003e0 2024-03-24T00:57:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.342+0000 7fd282000700 1 -- 172.21.15.38:0/2852974663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27c069850 con 0x7fd27c1003e0 2024-03-24T00:57:30.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.343+0000 7fd273e00700 1 -- 172.21.15.38:0/2852974663 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd268010460 con 0x7fd27c1003e0 2024-03-24T00:57:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.344+0000 7fd273e00700 1 --2- 172.21.15.38:0/2852974663 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd26006dfa0 0x7fd260070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.344+0000 7fd273e00700 1 -- 172.21.15.38:0/2852974663 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd26808d970 con 0x7fd27c1003e0 2024-03-24T00:57:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.344+0000 7fd27b400700 1 --2- 172.21.15.38:0/2852974663 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd26006dfa0 0x7fd260070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.345+0000 7fd27b400700 1 --2- 172.21.15.38:0/2852974663 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd26006dfa0 0x7fd260070460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006010 tx=0x7fd26c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:30.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.349+0000 7fd273e00700 1 -- 172.21.15.38:0/2852974663 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd268056ec0 con 0x7fd27c1003e0 2024-03-24T00:57:30.384 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:30 smithi088 ceph-mon[87351]: pgmap v124: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:57:30.384 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:30 smithi088 ceph-mon[87351]: Upgrade: Updating osd.3 2024-03-24T00:57:30.384 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:30 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:30.384 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:30 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T00:57:30.384 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:30 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:57:30.384 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:30 smithi088 ceph-mon[87351]: Deploying daemon osd.3 on smithi088 2024-03-24T00:57:30.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.603+0000 7fd282000700 1 -- 172.21.15.38:0/2852974663 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd27c1a2450 con 0x7fd26006dfa0 2024-03-24T00:57:30.603 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:30 smithi038 ceph-mon[112108]: pgmap v124: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:57:30.603 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:30 smithi038 ceph-mon[112108]: Upgrade: Updating osd.3 2024-03-24T00:57:30.603 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:30 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:30.603 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:30 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T00:57:30.603 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:30 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:57:30.603 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:30 smithi038 ceph-mon[112108]: Deploying daemon osd.3 on smithi088 2024-03-24T00:57:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.605+0000 7fd273e00700 1 -- 172.21.15.38:0/2852974663 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd27c1a2450 con 0x7fd26006dfa0 2024-03-24T00:57:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.608+0000 7fd282000700 1 -- 172.21.15.38:0/2852974663 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd26006dfa0 msgr2=0x7fd260070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.608+0000 7fd282000700 1 --2- 172.21.15.38:0/2852974663 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd26006dfa0 0x7fd260070460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006010 tx=0x7fd26c01a040 comp rx=0 tx=0).stop 2024-03-24T00:57:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.608+0000 7fd282000700 1 -- 172.21.15.38:0/2852974663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c1003e0 msgr2=0x7fd27c19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.608+0000 7fd282000700 1 --2- 172.21.15.38:0/2852974663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c1003e0 0x7fd27c19d5a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd26800d6d0 tx=0x7fd26800d9e0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.608+0000 7fd282000700 1 -- 172.21.15.38:0/2852974663 shutdown_connections 2024-03-24T00:57:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.608+0000 7fd282000700 1 --2- 172.21.15.38:0/2852974663 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd26006dfa0 0x7fd260070460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.608+0000 7fd282000700 1 --2- 172.21.15.38:0/2852974663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27c0ffac0 0x7fd27c19d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.608+0000 7fd282000700 1 --2- 172.21.15.38:0/2852974663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd27c1003e0 0x7fd27c19d5a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.608+0000 7fd282000700 1 -- 172.21.15.38:0/2852974663 >> 172.21.15.38:0/2852974663 conn(0x7fd27c0fb540 msgr2=0x7fd27c10e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.608+0000 7fd282000700 1 -- 172.21.15.38:0/2852974663 shutdown_connections 2024-03-24T00:57:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.609+0000 7fd282000700 1 -- 172.21.15.38:0/2852974663 wait complete. 2024-03-24T00:57:30.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.772+0000 7fd433000700 1 -- 172.21.15.38:0/609537195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd42c1038a0 msgr2=0x7fd42c1078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:30.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.772+0000 7fd433000700 1 --2- 172.21.15.38:0/609537195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd42c1038a0 0x7fd42c1078f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd418009910 tx=0x7fd418009c20 comp rx=0 tx=0).stop 2024-03-24T00:57:30.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.772+0000 7fd433000700 1 -- 172.21.15.38:0/609537195 shutdown_connections 2024-03-24T00:57:30.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.772+0000 7fd433000700 1 --2- 172.21.15.38:0/609537195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd42c1038a0 0x7fd42c1078f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.772+0000 7fd433000700 1 --2- 172.21.15.38:0/609537195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd42c102ef0 0x7fd42c1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.772+0000 7fd433000700 1 -- 172.21.15.38:0/609537195 >> 172.21.15.38:0/609537195 conn(0x7fd42c0fe780 msgr2=0x7fd42c100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:30.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.773+0000 7fd433000700 1 -- 172.21.15.38:0/609537195 shutdown_connections 2024-03-24T00:57:30.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.773+0000 7fd433000700 1 -- 172.21.15.38:0/609537195 wait complete. 2024-03-24T00:57:30.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.774+0000 7fd433000700 1 Processor -- start 2024-03-24T00:57:30.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.774+0000 7fd433000700 1 -- start start 2024-03-24T00:57:30.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.774+0000 7fd433000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd42c102ef0 0x7fd42c19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:30.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.774+0000 7fd433000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd42c1038a0 0x7fd42c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:30.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.774+0000 7fd433000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c198ab0 con 0x7fd42c1038a0 2024-03-24T00:57:30.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.774+0000 7fd433000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c198c20 con 0x7fd42c102ef0 2024-03-24T00:57:30.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.775+0000 7fd42b400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd42c1038a0 0x7fd42c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:30.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.775+0000 7fd42b400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd42c1038a0 0x7fd42c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56930/0 (socket says 172.21.15.38:56930) 2024-03-24T00:57:30.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.775+0000 7fd42b400700 1 -- 172.21.15.38:0/3755100153 learned_addr learned my addr 172.21.15.38:0/3755100153 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:57:30.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.775+0000 7fd42be00700 1 --2- 172.21.15.38:0/3755100153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd42c102ef0 0x7fd42c19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:30.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.775+0000 7fd42b400700 1 -- 172.21.15.38:0/3755100153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd42c102ef0 msgr2=0x7fd42c19ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:30.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.775+0000 7fd42b400700 1 --2- 172.21.15.38:0/3755100153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd42c102ef0 0x7fd42c19ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.775+0000 7fd42b400700 1 -- 172.21.15.38:0/3755100153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4180095f0 con 0x7fd42c1038a0 2024-03-24T00:57:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.776+0000 7fd42b400700 1 --2- 172.21.15.38:0/3755100153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd42c1038a0 0x7fd42c19ef40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd418005b40 tx=0x7fd41800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.776+0000 7fd428c00700 1 -- 172.21.15.38:0/3755100153 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd41801d070 con 0x7fd42c1038a0 2024-03-24T00:57:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.776+0000 7fd428c00700 1 -- 172.21.15.38:0/3755100153 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd41800f460 con 0x7fd42c1038a0 2024-03-24T00:57:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.776+0000 7fd428c00700 1 -- 172.21.15.38:0/3755100153 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd418017630 con 0x7fd42c1038a0 2024-03-24T00:57:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.776+0000 7fd433000700 1 -- 172.21.15.38:0/3755100153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42c198ea0 con 0x7fd42c1038a0 2024-03-24T00:57:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.776+0000 7fd433000700 1 -- 172.21.15.38:0/3755100153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42c199340 con 0x7fd42c1038a0 2024-03-24T00:57:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.777+0000 7fd433000700 1 -- 172.21.15.38:0/3755100153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42c10b2f0 con 0x7fd42c1038a0 2024-03-24T00:57:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.778+0000 7fd428c00700 1 -- 172.21.15.38:0/3755100153 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd418003a60 con 0x7fd42c1038a0 2024-03-24T00:57:30.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.779+0000 7fd428c00700 1 --2- 172.21.15.38:0/3755100153 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd41006dfb0 0x7fd410070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:30.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.779+0000 7fd428c00700 1 -- 172.21.15.38:0/3755100153 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd41808f190 con 0x7fd42c1038a0 2024-03-24T00:57:30.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.780+0000 7fd42be00700 1 --2- 172.21.15.38:0/3755100153 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd41006dfb0 0x7fd410070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.780+0000 7fd42be00700 1 --2- 172.21.15.38:0/3755100153 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd41006dfb0 0x7fd410070470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd41c005fd0 tx=0x7fd41c00a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:30.784+0000 7fd428c00700 1 -- 172.21.15.38:0/3755100153 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd418058660 con 0x7fd42c1038a0 2024-03-24T00:57:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.025+0000 7fd433000700 1 -- 172.21.15.38:0/3755100153 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd42c199f90 con 0x7fd41006dfb0 2024-03-24T00:57:31.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.057+0000 7fd428c00700 1 -- 172.21.15.38:0/3755100153 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd42c199f90 con 0x7fd41006dfb0 2024-03-24T00:57:31.058 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:57:31.058 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (5m) 12s ago 19m 13.5M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:57:31.058 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (20m) 12s ago 20m 9068k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:57:31.058 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (18m) 109s ago 18m 8363k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:57:31.058 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (2m) 12s ago 20m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T00:57:31.058 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (115s) 109s ago 18m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T00:57:31.058 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (5m) 12s ago 19m 75.3M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:57:31.058 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (12m) 12s ago 12m 97.9M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (12m) 12s ago 12m 150M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (12m) 109s ago 12m 24.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (12m) 109s ago 12m 24.8M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (8m) 12s ago 21m 502M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (7m) 109s ago 18m 449M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (3m) 12s ago 21m 48.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (3m) 109s ago 18m 41.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (7m) 12s ago 20m 7755k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (6m) 109s ago 18m 7755k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (83s) 12s ago 17m 97.1M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (52s) 12s ago 16m 104M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cb72ac446ba7 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (21s) 12s ago 16m 11.2M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c73f3e8276b6 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (15m) 109s ago 15m 304M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 e940a2e3af63 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (14m) 109s ago 14m 283M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:57:31.059 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (14m) 109s ago 14m 254M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:57:31.060 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (6m) 12s ago 19m 70.6M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:57:31.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.060+0000 7fd433000700 1 -- 172.21.15.38:0/3755100153 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd41006dfb0 msgr2=0x7fd410070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:31.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.060+0000 7fd433000700 1 --2- 172.21.15.38:0/3755100153 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd41006dfb0 0x7fd410070470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd41c005fd0 tx=0x7fd41c00a560 comp rx=0 tx=0).stop 2024-03-24T00:57:31.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.060+0000 7fd433000700 1 -- 172.21.15.38:0/3755100153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd42c1038a0 msgr2=0x7fd42c19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:31.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.060+0000 7fd433000700 1 --2- 172.21.15.38:0/3755100153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd42c1038a0 0x7fd42c19ef40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd418005b40 tx=0x7fd41800be20 comp rx=0 tx=0).stop 2024-03-24T00:57:31.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.060+0000 7fd433000700 1 -- 172.21.15.38:0/3755100153 shutdown_connections 2024-03-24T00:57:31.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.060+0000 7fd433000700 1 --2- 172.21.15.38:0/3755100153 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd41006dfb0 0x7fd410070470 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.060+0000 7fd433000700 1 --2- 172.21.15.38:0/3755100153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd42c102ef0 0x7fd42c19ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.060+0000 7fd433000700 1 --2- 172.21.15.38:0/3755100153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd42c1038a0 0x7fd42c19ef40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.060+0000 7fd433000700 1 -- 172.21.15.38:0/3755100153 >> 172.21.15.38:0/3755100153 conn(0x7fd42c0fe780 msgr2=0x7fd42c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:31.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.061+0000 7fd433000700 1 -- 172.21.15.38:0/3755100153 shutdown_connections 2024-03-24T00:57:31.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.061+0000 7fd433000700 1 -- 172.21.15.38:0/3755100153 wait complete. 2024-03-24T00:57:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.222+0000 7f5746800700 1 -- 172.21.15.38:0/3207068630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5740102530 msgr2=0x7f574010aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.222+0000 7f5746800700 1 --2- 172.21.15.38:0/3207068630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5740102530 0x7f574010aa20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5728009910 tx=0x7f5728009c20 comp rx=0 tx=0).stop 2024-03-24T00:57:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.222+0000 7f5746800700 1 -- 172.21.15.38:0/3207068630 shutdown_connections 2024-03-24T00:57:31.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.222+0000 7f5746800700 1 --2- 172.21.15.38:0/3207068630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5740102530 0x7f574010aa20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.222+0000 7f5746800700 1 --2- 172.21.15.38:0/3207068630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5740101c10 0x7f5740101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.222+0000 7f5746800700 1 -- 172.21.15.38:0/3207068630 >> 172.21.15.38:0/3207068630 conn(0x7f5740076640 msgr2=0x7f57400fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:31.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.223+0000 7f5746800700 1 -- 172.21.15.38:0/3207068630 shutdown_connections 2024-03-24T00:57:31.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.223+0000 7f5746800700 1 -- 172.21.15.38:0/3207068630 wait complete. 2024-03-24T00:57:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.224+0000 7f5746800700 1 Processor -- start 2024-03-24T00:57:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.224+0000 7f5746800700 1 -- start start 2024-03-24T00:57:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.224+0000 7f5746800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5740101c10 0x7f574019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.224+0000 7f5746800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5740102530 0x7f574019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.224+0000 7f5746800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574019d2c0 con 0x7f5740101c10 2024-03-24T00:57:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.224+0000 7f5746800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5740196830 con 0x7f5740102530 2024-03-24T00:57:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.224+0000 7f573f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5740101c10 0x7f574019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.225+0000 7f573f400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5740101c10 0x7f574019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56948/0 (socket says 172.21.15.38:56948) 2024-03-24T00:57:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.225+0000 7f573f400700 1 -- 172.21.15.38:0/2589688848 learned_addr learned my addr 172.21.15.38:0/2589688848 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:57:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.225+0000 7f5737e00700 1 --2- 172.21.15.38:0/2589688848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5740102530 0x7f574019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.225+0000 7f573f400700 1 -- 172.21.15.38:0/2589688848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5740102530 msgr2=0x7f574019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.225+0000 7f573f400700 1 --2- 172.21.15.38:0/2589688848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5740102530 0x7f574019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.225+0000 7f573f400700 1 -- 172.21.15.38:0/2589688848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57280095f0 con 0x7f5740101c10 2024-03-24T00:57:31.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.225+0000 7f573f400700 1 --2- 172.21.15.38:0/2589688848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5740101c10 0x7f574019c6f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f573000c8a0 tx=0x7f573000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:31.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.226+0000 7f573cc00700 1 -- 172.21.15.38:0/2589688848 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573000cea0 con 0x7f5740101c10 2024-03-24T00:57:31.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.226+0000 7f573cc00700 1 -- 172.21.15.38:0/2589688848 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5730004830 con 0x7f5740101c10 2024-03-24T00:57:31.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.226+0000 7f5746800700 1 -- 172.21.15.38:0/2589688848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5740196b10 con 0x7f5740101c10 2024-03-24T00:57:31.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.226+0000 7f573cc00700 1 -- 172.21.15.38:0/2589688848 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57300055b0 con 0x7f5740101c10 2024-03-24T00:57:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.226+0000 7f5746800700 1 -- 172.21.15.38:0/2589688848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5740197060 con 0x7f5740101c10 2024-03-24T00:57:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.227+0000 7f5746800700 1 -- 172.21.15.38:0/2589688848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5740105330 con 0x7f5740101c10 2024-03-24T00:57:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.228+0000 7f573cc00700 1 -- 172.21.15.38:0/2589688848 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f57300049a0 con 0x7f5740101c10 2024-03-24T00:57:31.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.229+0000 7f573cc00700 1 --2- 172.21.15.38:0/2589688848 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f572406dfa0 0x7f5724070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:31.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.229+0000 7f573cc00700 1 -- 172.21.15.38:0/2589688848 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f573008d9b0 con 0x7f5740101c10 2024-03-24T00:57:31.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.229+0000 7f5737e00700 1 --2- 172.21.15.38:0/2589688848 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f572406dfa0 0x7f5724070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:31.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.230+0000 7f5737e00700 1 --2- 172.21.15.38:0/2589688848 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f572406dfa0 0x7f5724070460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5740197c50 tx=0x7f572801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:31.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.234+0000 7f573cc00700 1 -- 172.21.15.38:0/2589688848 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f573005a9b0 con 0x7f5740101c10 2024-03-24T00:57:31.270 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:31 smithi088 ceph-mon[87351]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:57:31.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:31 smithi038 ceph-mon[112108]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:57:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.547+0000 7f5746800700 1 -- 172.21.15.38:0/2589688848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5740197cf0 con 0x7f5740101c10 2024-03-24T00:57:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.548+0000 7f573cc00700 1 -- 172.21.15.38:0/2589688848 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f5730004c50 con 0x7f5740101c10 2024-03-24T00:57:31.548 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:57:31.548 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:57:31.548 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:57:31.548 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:57:31.548 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:57:31.548 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:57:31.548 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:57:31.548 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:57:31.548 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 3, 2024-03-24T00:57:31.548 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 3 2024-03-24T00:57:31.548 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:57:31.549 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:57:31.549 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:57:31.549 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:57:31.549 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:57:31.549 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 7, 2024-03-24T00:57:31.549 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 7 2024-03-24T00:57:31.549 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:57:31.549 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:57:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.550+0000 7f5746800700 1 -- 172.21.15.38:0/2589688848 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f572406dfa0 msgr2=0x7f5724070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.550+0000 7f5746800700 1 --2- 172.21.15.38:0/2589688848 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f572406dfa0 0x7f5724070460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5740197c50 tx=0x7f572801a040 comp rx=0 tx=0).stop 2024-03-24T00:57:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.550+0000 7f5746800700 1 -- 172.21.15.38:0/2589688848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5740101c10 msgr2=0x7f574019c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.550+0000 7f5746800700 1 --2- 172.21.15.38:0/2589688848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5740101c10 0x7f574019c6f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f573000c8a0 tx=0x7f573000cc60 comp rx=0 tx=0).stop 2024-03-24T00:57:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.551+0000 7f5746800700 1 -- 172.21.15.38:0/2589688848 shutdown_connections 2024-03-24T00:57:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.551+0000 7f5746800700 1 --2- 172.21.15.38:0/2589688848 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f572406dfa0 0x7f5724070460 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.551+0000 7f5746800700 1 --2- 172.21.15.38:0/2589688848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5740101c10 0x7f574019c6f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.551+0000 7f5746800700 1 --2- 172.21.15.38:0/2589688848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5740102530 0x7f574019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.551+0000 7f5746800700 1 -- 172.21.15.38:0/2589688848 >> 172.21.15.38:0/2589688848 conn(0x7f5740076640 msgr2=0x7f5740102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.551+0000 7f5746800700 1 -- 172.21.15.38:0/2589688848 shutdown_connections 2024-03-24T00:57:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.551+0000 7f5746800700 1 -- 172.21.15.38:0/2589688848 wait complete. 2024-03-24T00:57:31.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.705+0000 7f973be00700 1 -- 172.21.15.38:0/407384896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9734107310 msgr2=0x7f97341076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:31.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.705+0000 7f973be00700 1 --2- 172.21.15.38:0/407384896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9734107310 0x7f97341076f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f972800b0a0 tx=0x7f972800b3b0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.705+0000 7f973be00700 1 -- 172.21.15.38:0/407384896 shutdown_connections 2024-03-24T00:57:31.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.705+0000 7f973be00700 1 --2- 172.21.15.38:0/407384896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9734107cc0 0x7f973410bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.705+0000 7f973be00700 1 --2- 172.21.15.38:0/407384896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9734107310 0x7f97341076f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.705+0000 7f973be00700 1 -- 172.21.15.38:0/407384896 >> 172.21.15.38:0/407384896 conn(0x7f97340768e0 msgr2=0x7f9734078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:31.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.705+0000 7f973be00700 1 -- 172.21.15.38:0/407384896 shutdown_connections 2024-03-24T00:57:31.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.705+0000 7f973be00700 1 -- 172.21.15.38:0/407384896 wait complete. 2024-03-24T00:57:31.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.707+0000 7f973be00700 1 Processor -- start 2024-03-24T00:57:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.707+0000 7f973be00700 1 -- start start 2024-03-24T00:57:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.707+0000 7f973be00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9734107310 0x7f973419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.707+0000 7f973be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9734107cc0 0x7f973419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.708+0000 7f973be00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973419dc50 con 0x7f9734107cc0 2024-03-24T00:57:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.708+0000 7f973be00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97341a1920 con 0x7f9734107310 2024-03-24T00:57:31.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.708+0000 7f9738a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9734107cc0 0x7f973419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:31.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.708+0000 7f9738a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9734107cc0 0x7f973419d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56966/0 (socket says 172.21.15.38:56966) 2024-03-24T00:57:31.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.708+0000 7f9738a00700 1 -- 172.21.15.38:0/45087700 learned_addr learned my addr 172.21.15.38:0/45087700 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:57:31.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.708+0000 7f9739400700 1 --2- 172.21.15.38:0/45087700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9734107310 0x7f973419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:31.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.708+0000 7f9738a00700 1 -- 172.21.15.38:0/45087700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9734107310 msgr2=0x7f973419d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:31.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.708+0000 7f9738a00700 1 --2- 172.21.15.38:0/45087700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9734107310 0x7f973419d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.708+0000 7f9738a00700 1 -- 172.21.15.38:0/45087700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9728009d00 con 0x7f9734107cc0 2024-03-24T00:57:31.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.709+0000 7f9738a00700 1 --2- 172.21.15.38:0/45087700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9734107cc0 0x7f973419d570 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f972000d8d0 tx=0x7f972000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:31.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.709+0000 7f9732000700 1 -- 172.21.15.38:0/45087700 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97200041f0 con 0x7f9734107cc0 2024-03-24T00:57:31.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.709+0000 7f9732000700 1 -- 172.21.15.38:0/45087700 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9720007620 con 0x7f9734107cc0 2024-03-24T00:57:31.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.709+0000 7f9732000700 1 -- 172.21.15.38:0/45087700 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9720003c50 con 0x7f9734107cc0 2024-03-24T00:57:31.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.709+0000 7f973be00700 1 -- 172.21.15.38:0/45087700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97341a1c00 con 0x7f9734107cc0 2024-03-24T00:57:31.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.709+0000 7f973be00700 1 -- 172.21.15.38:0/45087700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97341a2150 con 0x7f9734107cc0 2024-03-24T00:57:31.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.711+0000 7f973be00700 1 -- 172.21.15.38:0/45087700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973410f6b0 con 0x7f9734107cc0 2024-03-24T00:57:31.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.712+0000 7f9732000700 1 -- 172.21.15.38:0/45087700 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f9720004350 con 0x7f9734107cc0 2024-03-24T00:57:31.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.713+0000 7f9732000700 1 --2- 172.21.15.38:0/45087700 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f971c06dfb0 0x7f971c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:31.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.713+0000 7f9732000700 1 -- 172.21.15.38:0/45087700 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f972008d890 con 0x7f9734107cc0 2024-03-24T00:57:31.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.714+0000 7f9739400700 1 --2- 172.21.15.38:0/45087700 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f971c06dfb0 0x7f971c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:31.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.714+0000 7f9739400700 1 --2- 172.21.15.38:0/45087700 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f971c06dfb0 0x7f971c070470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97280060b0 tx=0x7f9728006000 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:31.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.718+0000 7f9732000700 1 -- 172.21.15.38:0/45087700 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9720056d40 con 0x7f9734107cc0 2024-03-24T00:57:31.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.991+0000 7f973be00700 1 -- 172.21.15.38:0/45087700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f973419e340 con 0x7f9734107cc0 2024-03-24T00:57:31.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.992+0000 7f9732000700 1 -- 172.21.15.38:0/45087700 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1900 (secure 0 0 0) 0x7f972005a400 con 0x7f9734107cc0 2024-03-24T00:57:31.994 INFO:teuthology.orchestra.run.smithi038.stdout:e20 2024-03-24T00:57:31.994 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:57:31.994 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:57:31.994 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:57:31.994 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:57:31.994 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:57:31.994 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 20 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:57:31.995 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:57:31.996 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:57:31.997 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:57:31.997 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:57:31.997 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:57:31.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.995+0000 7f973be00700 1 -- 172.21.15.38:0/45087700 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f971c06dfb0 msgr2=0x7f971c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:31.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.995+0000 7f973be00700 1 --2- 172.21.15.38:0/45087700 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f971c06dfb0 0x7f971c070470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97280060b0 tx=0x7f9728006000 comp rx=0 tx=0).stop 2024-03-24T00:57:31.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.995+0000 7f973be00700 1 -- 172.21.15.38:0/45087700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9734107cc0 msgr2=0x7f973419d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:31.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.995+0000 7f973be00700 1 --2- 172.21.15.38:0/45087700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9734107cc0 0x7f973419d570 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f972000d8d0 tx=0x7f972000dc90 comp rx=0 tx=0).stop 2024-03-24T00:57:31.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.995+0000 7f973be00700 1 -- 172.21.15.38:0/45087700 shutdown_connections 2024-03-24T00:57:31.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.995+0000 7f973be00700 1 --2- 172.21.15.38:0/45087700 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f971c06dfb0 0x7f971c070470 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.995+0000 7f973be00700 1 --2- 172.21.15.38:0/45087700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9734107310 0x7f973419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.995+0000 7f973be00700 1 --2- 172.21.15.38:0/45087700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9734107cc0 0x7f973419d570 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:31.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.995+0000 7f973be00700 1 -- 172.21.15.38:0/45087700 >> 172.21.15.38:0/45087700 conn(0x7f97340768e0 msgr2=0x7f9734076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:31.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.995+0000 7f973be00700 1 -- 172.21.15.38:0/45087700 shutdown_connections 2024-03-24T00:57:31.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:31.995+0000 7f973be00700 1 -- 172.21.15.38:0/45087700 wait complete. 2024-03-24T00:57:31.998 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:57:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.152+0000 7f4205c00700 1 -- 172.21.15.38:0/629927437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4200107310 msgr2=0x7f42001076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.152+0000 7f4205c00700 1 --2- 172.21.15.38:0/629927437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4200107310 0x7f42001076f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009910 tx=0x7f41f0009c20 comp rx=0 tx=0).stop 2024-03-24T00:57:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.152+0000 7f4205c00700 1 -- 172.21.15.38:0/629927437 shutdown_connections 2024-03-24T00:57:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.152+0000 7f4205c00700 1 --2- 172.21.15.38:0/629927437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4200107cc0 0x7f420010bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.153+0000 7f4205c00700 1 --2- 172.21.15.38:0/629927437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4200107310 0x7f42001076f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.153+0000 7f4205c00700 1 -- 172.21.15.38:0/629927437 >> 172.21.15.38:0/629927437 conn(0x7f42000768e0 msgr2=0x7f4200078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.153+0000 7f4205c00700 1 -- 172.21.15.38:0/629927437 shutdown_connections 2024-03-24T00:57:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.153+0000 7f4205c00700 1 -- 172.21.15.38:0/629927437 wait complete. 2024-03-24T00:57:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.154+0000 7f4205c00700 1 Processor -- start 2024-03-24T00:57:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.154+0000 7f4205c00700 1 -- start start 2024-03-24T00:57:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.155+0000 7f4205c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4200107310 0x7f420019d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.155+0000 7f4205c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4200107cc0 0x7f420019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.155+0000 7f4205c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420019dc90 con 0x7f4200107310 2024-03-24T00:57:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.155+0000 7f4205c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42001a18a0 con 0x7f4200107cc0 2024-03-24T00:57:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.155+0000 7f41fea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4200107310 0x7f420019d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.155+0000 7f41fea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4200107310 0x7f420019d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56988/0 (socket says 172.21.15.38:56988) 2024-03-24T00:57:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.155+0000 7f41fea00700 1 -- 172.21.15.38:0/988367676 learned_addr learned my addr 172.21.15.38:0/988367676 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:57:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.155+0000 7f41fe000700 1 --2- 172.21.15.38:0/988367676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4200107cc0 0x7f420019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.156+0000 7f41fea00700 1 -- 172.21.15.38:0/988367676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4200107cc0 msgr2=0x7f420019d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.156+0000 7f41fea00700 1 --2- 172.21.15.38:0/988367676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4200107cc0 0x7f420019d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.156+0000 7f41fea00700 1 -- 172.21.15.38:0/988367676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f00095f0 con 0x7f4200107310 2024-03-24T00:57:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.156+0000 7f41fea00700 1 --2- 172.21.15.38:0/988367676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4200107310 0x7f420019d070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f41f00092e0 tx=0x7f41f000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.157+0000 7f41f7400700 1 -- 172.21.15.38:0/988367676 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41f001d070 con 0x7f4200107310 2024-03-24T00:57:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.157+0000 7f41f7400700 1 -- 172.21.15.38:0/988367676 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41f00056e0 con 0x7f4200107310 2024-03-24T00:57:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.157+0000 7f4205c00700 1 -- 172.21.15.38:0/988367676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42001a1b20 con 0x7f4200107310 2024-03-24T00:57:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.157+0000 7f41f7400700 1 -- 172.21.15.38:0/988367676 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41f000fe10 con 0x7f4200107310 2024-03-24T00:57:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.157+0000 7f4205c00700 1 -- 172.21.15.38:0/988367676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42001a2010 con 0x7f4200107310 2024-03-24T00:57:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.158+0000 7f4205c00700 1 -- 172.21.15.38:0/988367676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420010f6b0 con 0x7f4200107310 2024-03-24T00:57:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.159+0000 7f41f7400700 1 -- 172.21.15.38:0/988367676 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f41f000f680 con 0x7f4200107310 2024-03-24T00:57:32.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.160+0000 7f41f7400700 1 --2- 172.21.15.38:0/988367676 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f41e406dfb0 0x7f41e4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:32.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.160+0000 7f41f7400700 1 -- 172.21.15.38:0/988367676 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f41f008f110 con 0x7f4200107310 2024-03-24T00:57:32.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.161+0000 7f41fe000700 1 --2- 172.21.15.38:0/988367676 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f41e406dfb0 0x7f41e4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.161+0000 7f41fe000700 1 --2- 172.21.15.38:0/988367676 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f41e406dfb0 0x7f41e4070470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4200108e00 tx=0x7f41ec0073d0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.165+0000 7f41f7400700 1 -- 172.21.15.38:0/988367676 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f41f0058540 con 0x7f4200107310 2024-03-24T00:57:32.316 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:32 smithi038 ceph-mon[112108]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:57:32.316 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:32 smithi038 ceph-mon[112108]: pgmap v125: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:57:32.316 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:32 smithi038 ceph-mon[112108]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:57:32.316 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:32 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2589688848' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:32.316 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:32 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:32.316 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:32 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/45087700' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:57:32.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.404+0000 7f4205c00700 1 -- 172.21.15.38:0/988367676 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f420019e2c0 con 0x7f41e406dfb0 2024-03-24T00:57:32.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.406+0000 7f41f7400700 1 -- 172.21.15.38:0/988367676 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f420019e2c0 con 0x7f41e406dfb0 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: "crash" 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "9/23 daemons upgraded", 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:57:32.407 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:57:32.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.409+0000 7f4205c00700 1 -- 172.21.15.38:0/988367676 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f41e406dfb0 msgr2=0x7f41e4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:32.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.409+0000 7f4205c00700 1 --2- 172.21.15.38:0/988367676 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f41e406dfb0 0x7f41e4070470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4200108e00 tx=0x7f41ec0073d0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.409+0000 7f4205c00700 1 -- 172.21.15.38:0/988367676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4200107310 msgr2=0x7f420019d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:32.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.409+0000 7f4205c00700 1 --2- 172.21.15.38:0/988367676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4200107310 0x7f420019d070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f41f00092e0 tx=0x7f41f000bd70 comp rx=0 tx=0).stop 2024-03-24T00:57:32.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.409+0000 7f4205c00700 1 -- 172.21.15.38:0/988367676 shutdown_connections 2024-03-24T00:57:32.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.409+0000 7f4205c00700 1 --2- 172.21.15.38:0/988367676 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f41e406dfb0 0x7f41e4070470 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.409+0000 7f4205c00700 1 --2- 172.21.15.38:0/988367676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4200107310 0x7f420019d070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.409+0000 7f4205c00700 1 --2- 172.21.15.38:0/988367676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4200107cc0 0x7f420019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.409+0000 7f4205c00700 1 -- 172.21.15.38:0/988367676 >> 172.21.15.38:0/988367676 conn(0x7f42000768e0 msgr2=0x7f4200077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:32.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.409+0000 7f4205c00700 1 -- 172.21.15.38:0/988367676 shutdown_connections 2024-03-24T00:57:32.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.410+0000 7f4205c00700 1 -- 172.21.15.38:0/988367676 wait complete. 2024-03-24T00:57:32.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:32 smithi088 ceph-mon[87351]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:57:32.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:32 smithi088 ceph-mon[87351]: pgmap v125: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:57:32.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:32 smithi088 ceph-mon[87351]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:57:32.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:32 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2589688848' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:32.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:32 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:32.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:32 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/45087700' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:57:32.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.571+0000 7f7798000700 1 -- 172.21.15.38:0/3060092879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7790102ed0 msgr2=0x7f77901032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:32.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.571+0000 7f7798000700 1 --2- 172.21.15.38:0/3060092879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7790102ed0 0x7f77901032b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7784009910 tx=0x7f7784009c20 comp rx=0 tx=0).stop 2024-03-24T00:57:32.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.571+0000 7f7798000700 1 -- 172.21.15.38:0/3060092879 shutdown_connections 2024-03-24T00:57:32.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.571+0000 7f7798000700 1 --2- 172.21.15.38:0/3060092879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7790103880 0x7f77901078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.571+0000 7f7798000700 1 --2- 172.21.15.38:0/3060092879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7790102ed0 0x7f77901032b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.571+0000 7f7798000700 1 -- 172.21.15.38:0/3060092879 >> 172.21.15.38:0/3060092879 conn(0x7f77900fe780 msgr2=0x7f7790100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.571+0000 7f7798000700 1 -- 172.21.15.38:0/3060092879 shutdown_connections 2024-03-24T00:57:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.571+0000 7f7798000700 1 -- 172.21.15.38:0/3060092879 wait complete. 2024-03-24T00:57:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.572+0000 7f7798000700 1 Processor -- start 2024-03-24T00:57:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.572+0000 7f7798000700 1 -- start start 2024-03-24T00:57:32.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.573+0000 7f7798000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7790102ed0 0x7f779019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:32.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.573+0000 7f7798000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7790103880 0x7f779019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:32.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.573+0000 7f7798000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779019f550 con 0x7f7790103880 2024-03-24T00:57:32.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.573+0000 7f7798000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7790198b30 con 0x7f7790102ed0 2024-03-24T00:57:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.573+0000 7f7794c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7790103880 0x7f779019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.573+0000 7f7794c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7790103880 0x7f779019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57014/0 (socket says 172.21.15.38:57014) 2024-03-24T00:57:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.573+0000 7f7794c00700 1 -- 172.21.15.38:0/965169335 learned_addr learned my addr 172.21.15.38:0/965169335 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:57:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.574+0000 7f7795600700 1 --2- 172.21.15.38:0/965169335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7790102ed0 0x7f779019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.574+0000 7f7794c00700 1 -- 172.21.15.38:0/965169335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7790102ed0 msgr2=0x7f779019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.574+0000 7f7794c00700 1 --2- 172.21.15.38:0/965169335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7790102ed0 0x7f779019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.574+0000 7f7794c00700 1 -- 172.21.15.38:0/965169335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77840095f0 con 0x7f7790103880 2024-03-24T00:57:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.574+0000 7f7794c00700 1 --2- 172.21.15.38:0/965169335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7790103880 0x7f779019ef30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f777c00b480 tx=0x7f777c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.575+0000 7f778e000700 1 -- 172.21.15.38:0/965169335 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f777c0118c0 con 0x7f7790103880 2024-03-24T00:57:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.575+0000 7f778e000700 1 -- 172.21.15.38:0/965169335 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f777c010380 con 0x7f7790103880 2024-03-24T00:57:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.575+0000 7f7798000700 1 -- 172.21.15.38:0/965169335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7790198e10 con 0x7f7790103880 2024-03-24T00:57:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.575+0000 7f7798000700 1 -- 172.21.15.38:0/965169335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7790199360 con 0x7f7790103880 2024-03-24T00:57:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.575+0000 7f778e000700 1 -- 172.21.15.38:0/965169335 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f777c00f4f0 con 0x7f7790103880 2024-03-24T00:57:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.576+0000 7f7798000700 1 -- 172.21.15.38:0/965169335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77700052f0 con 0x7f7790103880 2024-03-24T00:57:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.578+0000 7f778e000700 1 -- 172.21.15.38:0/965169335 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f777c0104f0 con 0x7f7790103880 2024-03-24T00:57:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.579+0000 7f778e000700 1 --2- 172.21.15.38:0/965169335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f777806dfb0 0x7f7778070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:57:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.579+0000 7f778e000700 1 -- 172.21.15.38:0/965169335 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f777c08d830 con 0x7f7790103880 2024-03-24T00:57:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.579+0000 7f7795600700 1 --2- 172.21.15.38:0/965169335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f777806dfb0 0x7f7778070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:57:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.580+0000 7f7795600700 1 --2- 172.21.15.38:0/965169335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f777806dfb0 0x7f7778070470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f778400b5c0 tx=0x7f778401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:57:32.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.584+0000 7f778e000700 1 -- 172.21.15.38:0/965169335 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f777c05a830 con 0x7f7790103880 2024-03-24T00:57:32.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.900+0000 7f7798000700 1 -- 172.21.15.38:0/965169335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7770005160 con 0x7f7790103880 2024-03-24T00:57:32.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.900+0000 7f778e000700 1 -- 172.21.15.38:0/965169335 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f777c05a3c0 con 0x7f7790103880 2024-03-24T00:57:32.900 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_OK 2024-03-24T00:57:32.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.903+0000 7f7798000700 1 -- 172.21.15.38:0/965169335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f777806dfb0 msgr2=0x7f7778070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:32.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.903+0000 7f7798000700 1 --2- 172.21.15.38:0/965169335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f777806dfb0 0x7f7778070470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f778400b5c0 tx=0x7f778401a040 comp rx=0 tx=0).stop 2024-03-24T00:57:32.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.903+0000 7f7798000700 1 -- 172.21.15.38:0/965169335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7790103880 msgr2=0x7f779019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:57:32.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.903+0000 7f7798000700 1 --2- 172.21.15.38:0/965169335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7790103880 0x7f779019ef30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f777c00b480 tx=0x7f777c00b840 comp rx=0 tx=0).stop 2024-03-24T00:57:32.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.903+0000 7f7798000700 1 -- 172.21.15.38:0/965169335 shutdown_connections 2024-03-24T00:57:32.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.904+0000 7f7798000700 1 --2- 172.21.15.38:0/965169335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f777806dfb0 0x7f7778070470 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.904+0000 7f7798000700 1 --2- 172.21.15.38:0/965169335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7790102ed0 0x7f779019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.904+0000 7f7798000700 1 --2- 172.21.15.38:0/965169335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7790103880 0x7f779019ef30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:57:32.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.904+0000 7f7798000700 1 -- 172.21.15.38:0/965169335 >> 172.21.15.38:0/965169335 conn(0x7f77900fe780 msgr2=0x7f77901068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:57:32.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.904+0000 7f7798000700 1 -- 172.21.15.38:0/965169335 shutdown_connections 2024-03-24T00:57:32.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:57:32.904+0000 7f7798000700 1 -- 172.21.15.38:0/965169335 wait complete. 2024-03-24T00:57:33.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:33 smithi038 ceph-mon[112108]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:57:33.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:33 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/965169335' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:57:33.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:33 smithi088 ceph-mon[87351]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:57:33.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:33 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/965169335' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:57:34.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:34 smithi088 ceph-mon[87351]: pgmap v126: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:57:34.431 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:34 smithi088 systemd[1]: Stopping Ceph osd.3 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:57:34.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:34 smithi038 ceph-mon[112108]: pgmap v126: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T00:57:34.930 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:34 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[42559]: 2024-03-24T00:57:34.479+0000 7fa257000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T00:57:34.930 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:34 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[42559]: 2024-03-24T00:57:34.479+0000 7fa257000700 -1 osd.3 65 *** Got signal Terminated *** 2024-03-24T00:57:34.931 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:34 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[42559]: 2024-03-24T00:57:34.479+0000 7fa257000700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T00:57:35.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:35 smithi038 ceph-mon[112108]: osd.3 marked itself down and dead 2024-03-24T00:57:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:35 smithi088 ceph-mon[87351]: osd.3 marked itself down and dead 2024-03-24T00:57:36.180 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:35 smithi088 podman[96467]: 2024-03-24 00:57:35.901991103 +0000 UTC m=+1.566354351 container died e940a2e3af63e59336444534c5a59f173b33fef99f430514fc8ee027fb8dc5a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=reef-738c2fd, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T00:57:36.507 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:36 smithi088 ceph-mon[87351]: pgmap v127: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:57:36.507 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:36 smithi088 ceph-mon[87351]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:57:36.507 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:36 smithi088 ceph-mon[87351]: osdmap e66: 6 total, 5 up, 6 in 2024-03-24T00:57:36.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:36 smithi038 ceph-mon[112108]: pgmap v127: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:57:36.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:36 smithi038 ceph-mon[112108]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:57:36.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:36 smithi038 ceph-mon[112108]: osdmap e66: 6 total, 5 up, 6 in 2024-03-24T00:57:36.842 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:36 smithi088 podman[96467]: 2024-03-24 00:57:36.506920352 +0000 UTC m=+2.171283644 container cleanup e940a2e3af63e59336444534c5a59f173b33fef99f430514fc8ee027fb8dc5a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, RELEASE=reef-738c2fd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-24T00:57:36.842 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:36 smithi088 bash[96467]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3 2024-03-24T00:57:37.180 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:36 smithi088 podman[96487]: 2024-03-24 00:57:36.841364066 +0000 UTC m=+0.925955889 container remove e940a2e3af63e59336444534c5a59f173b33fef99f430514fc8ee027fb8dc5a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3, RELEASE=reef-738c2fd, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:57:37.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:37 smithi038 ceph-mon[112108]: osdmap e67: 6 total, 5 up, 6 in 2024-03-24T00:57:37.646 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:37 smithi088 ceph-mon[87351]: osdmap e67: 6 total, 5 up, 6 in 2024-03-24T00:57:37.646 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:37 smithi088 podman[96573]: 2024-03-24 00:57:37.364939388 +0000 UTC m=+0.034149990 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:57:37.647 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:37 smithi088 podman[96573]: 2024-03-24 00:57:37.475499738 +0000 UTC m=+0.144710372 container create 1698f4497a33578104052cde42dda24051ab961d47803405ed3e4187f78d0742 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T00:57:37.930 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:37 smithi088 podman[96573]: 2024-03-24 00:57:37.746054484 +0000 UTC m=+0.415265124 container init 1698f4497a33578104052cde42dda24051ab961d47803405ed3e4187f78d0742 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:57:37.930 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:37 smithi088 podman[96573]: 2024-03-24 00:57:37.756841805 +0000 UTC m=+0.426052420 container start 1698f4497a33578104052cde42dda24051ab961d47803405ed3e4187f78d0742 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:57:37.930 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:37 smithi088 podman[96573]: 2024-03-24 00:57:37.828358928 +0000 UTC m=+0.497569541 container attach 1698f4497a33578104052cde42dda24051ab961d47803405ed3e4187f78d0742 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-24T00:57:38.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:38 smithi088 ceph-mon[87351]: pgmap v130: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-24T00:57:38.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:38 smithi088 ceph-mon[87351]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-24T00:57:38.575 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:38 smithi088 podman[96599]: 2024-03-24 00:57:38.245796339 +0000 UTC m=+0.036325403 container died 1698f4497a33578104052cde42dda24051ab961d47803405ed3e4187f78d0742 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-24T00:57:38.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:38 smithi038 ceph-mon[112108]: pgmap v130: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-24T00:57:38.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:38 smithi038 ceph-mon[112108]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-24T00:57:39.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:38 smithi088 podman[96599]: 2024-03-24 00:57:38.906562039 +0000 UTC m=+0.697091132 container remove 1698f4497a33578104052cde42dda24051ab961d47803405ed3e4187f78d0742 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:57:39.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:38 smithi088 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-24T00:57:39.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:38 smithi088 systemd[1]: Stopped Ceph osd.3 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:57:39.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:38 smithi088 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.3.service: Consumed 2min 25.226s CPU time. 2024-03-24T00:57:39.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:39 smithi038 ceph-mon[112108]: Health check failed: Degraded data redundancy: 47/273 objects degraded (17.216%), 16 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:39.658 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:39 smithi088 ceph-mon[87351]: Health check failed: Degraded data redundancy: 47/273 objects degraded (17.216%), 16 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:39.658 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:39 smithi088 systemd[1]: Starting Ceph osd.3 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:57:39.930 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:39 smithi088 podman[96696]: 2024-03-24 00:57:39.562069846 +0000 UTC m=+0.033574603 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:57:39.931 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:39 smithi088 podman[96696]: 2024-03-24 00:57:39.679254276 +0000 UTC m=+0.150759068 container create 29cfd9c1904eb64494abd2da724dd723cc6750bf05fdc96225d57ef8efd0947e (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-24T00:57:40.327 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:39 smithi088 podman[96696]: 2024-03-24 00:57:39.995780367 +0000 UTC m=+0.467285210 container init 29cfd9c1904eb64494abd2da724dd723cc6750bf05fdc96225d57ef8efd0947e (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-24T00:57:40.327 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 podman[96696]: 2024-03-24 00:57:40.007611232 +0000 UTC m=+0.479115999 container start 29cfd9c1904eb64494abd2da724dd723cc6750bf05fdc96225d57ef8efd0947e (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-24T00:57:40.327 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 podman[96696]: 2024-03-24 00:57:40.085840271 +0000 UTC m=+0.557345057 container attach 29cfd9c1904eb64494abd2da724dd723cc6750bf05fdc96225d57ef8efd0947e (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-24T00:57:40.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:40 smithi038 ceph-mon[112108]: pgmap v131: 65 pgs: 16 active+undersized+degraded, 7 peering, 15 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 47/273 objects degraded (17.216%) 2024-03-24T00:57:40.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:40 smithi088 ceph-mon[87351]: pgmap v131: 65 pgs: 16 active+undersized+degraded, 7 peering, 15 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 47/273 objects degraded (17.216%) 2024-03-24T00:57:41.180 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate[96710]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate[96710]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 bash[96696]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 bash[96696]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 bash[96696]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 bash[96696]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 bash[96696]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 bash[96696]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 bash[96696]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate[96710]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate[96710]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate[96710]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate[96710]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T00:57:41.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate[96710]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-24T00:57:41.182 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:40 smithi088 podman[96696]: 2024-03-24 00:57:40.935679358 +0000 UTC m=+1.407184148 container died 29cfd9c1904eb64494abd2da724dd723cc6750bf05fdc96225d57ef8efd0947e (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-24T00:57:41.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:41 smithi088 ceph-mon[87351]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-24T00:57:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:41 smithi038 ceph-mon[112108]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-24T00:57:42.075 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:41 smithi088 podman[96846]: 2024-03-24 00:57:41.705802555 +0000 UTC m=+0.749576285 container remove 29cfd9c1904eb64494abd2da724dd723cc6750bf05fdc96225d57ef8efd0947e (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-24T00:57:42.368 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:42 smithi088 podman[96903]: 2024-03-24 00:57:41.979881595 +0000 UTC m=+0.038452418 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:57:42.368 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:42 smithi088 podman[96903]: 2024-03-24 00:57:42.081737483 +0000 UTC m=+0.140308330 container create b2afaff761d884fdffa4d28a130599ae9b373a2a32709ade53235c11a58648c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-24T00:57:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:42 smithi038 ceph-mon[112108]: pgmap v132: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 56/273 objects degraded (20.513%) 2024-03-24T00:57:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:57:42.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:42 smithi088 ceph-mon[87351]: pgmap v132: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 56/273 objects degraded (20.513%) 2024-03-24T00:57:42.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:42.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:57:42.680 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:42 smithi088 podman[96903]: 2024-03-24 00:57:42.368515047 +0000 UTC m=+0.427085941 container init b2afaff761d884fdffa4d28a130599ae9b373a2a32709ade53235c11a58648c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-24T00:57:42.681 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:42 smithi088 podman[96903]: 2024-03-24 00:57:42.379238637 +0000 UTC m=+0.437809464 container start b2afaff761d884fdffa4d28a130599ae9b373a2a32709ade53235c11a58648c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:57:42.681 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:42 smithi088 ceph-osd[96922]: -- 172.21.15.88:0/2039464861 <== mon.1 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55737be0f680 con 0x55737bdd8c00 2024-03-24T00:57:42.681 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:42 smithi088 bash[96903]: b2afaff761d884fdffa4d28a130599ae9b373a2a32709ade53235c11a58648c3 2024-03-24T00:57:42.681 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:42 smithi088 systemd[1]: Started Ceph osd.3 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:57:43.180 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:42 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T00:57:42.946+0000 7ff9c6d717c0 -1 Falling back to public interface 2024-03-24T00:57:43.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:43.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:43.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:43 smithi038 ceph-mon[112108]: pgmap v133: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 56/273 objects degraded (20.513%) 2024-03-24T00:57:43.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:57:43.896 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:43.897 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:43.897 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:43 smithi088 ceph-mon[87351]: pgmap v133: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 56/273 objects degraded (20.513%) 2024-03-24T00:57:43.897 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:57:45.914 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:45 smithi088 ceph-mon[87351]: pgmap v134: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 56/273 objects degraded (20.513%) 2024-03-24T00:57:46.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:45 smithi038 ceph-mon[112108]: pgmap v134: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 56/273 objects degraded (20.513%) 2024-03-24T00:57:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:47 smithi038 ceph-mon[112108]: pgmap v135: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 56/273 objects degraded (20.513%) 2024-03-24T00:57:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:47 smithi038 ceph-mon[112108]: Health check update: Degraded data redundancy: 56/273 objects degraded (20.513%), 19 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:47 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:48.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:47 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:48.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:47 smithi088 ceph-mon[87351]: pgmap v135: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 56/273 objects degraded (20.513%) 2024-03-24T00:57:48.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:47 smithi088 ceph-mon[87351]: Health check update: Degraded data redundancy: 56/273 objects degraded (20.513%), 19 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:48.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:47 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:48.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:47 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:49.881 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:49 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T00:57:49.465+0000 7ff9c6d717c0 -1 osd.3 65 log_to_monitors true 2024-03-24T00:57:50.136 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 00:57:49 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T00:57:49.975+0000 7ff9b6000700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T00:57:50.137 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:49 smithi088 ceph-mon[87351]: pgmap v136: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 56/273 objects degraded (20.513%) 2024-03-24T00:57:50.137 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:49 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:50.137 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:49 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:50.137 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:49 smithi088 ceph-mon[87351]: from='osd.3 [v2:172.21.15.88:6800/1863418220,v1:172.21.15.88:6801/1863418220]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T00:57:50.137 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:49 smithi088 ceph-mon[87351]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T00:57:50.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:49 smithi038 ceph-mon[112108]: pgmap v136: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 56/273 objects degraded (20.513%) 2024-03-24T00:57:50.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:49 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:50.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:49 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:50.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:49 smithi038 ceph-mon[112108]: from='osd.3 [v2:172.21.15.88:6800/1863418220,v1:172.21.15.88:6801/1863418220]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T00:57:50.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:49 smithi038 ceph-mon[112108]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T00:57:51.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:50 smithi088 ceph-mon[87351]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T00:57:51.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:50 smithi088 ceph-mon[87351]: from='osd.3 [v2:172.21.15.88:6800/1863418220,v1:172.21.15.88:6801/1863418220]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:57:51.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:50 smithi088 ceph-mon[87351]: osdmap e68: 6 total, 5 up, 6 in 2024-03-24T00:57:51.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:50 smithi088 ceph-mon[87351]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:57:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:50 smithi038 ceph-mon[112108]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T00:57:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:50 smithi038 ceph-mon[112108]: from='osd.3 [v2:172.21.15.88:6800/1863418220,v1:172.21.15.88:6801/1863418220]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:57:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:50 smithi038 ceph-mon[112108]: osdmap e68: 6 total, 5 up, 6 in 2024-03-24T00:57:51.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:50 smithi038 ceph-mon[112108]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:57:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:51 smithi088 ceph-mon[87351]: pgmap v138: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 56/273 objects degraded (20.513%) 2024-03-24T00:57:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:51 smithi088 ceph-mon[87351]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:57:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:51 smithi088 ceph-mon[87351]: osd.3 [v2:172.21.15.88:6800/1863418220,v1:172.21.15.88:6801/1863418220] boot 2024-03-24T00:57:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:51 smithi088 ceph-mon[87351]: osdmap e69: 6 total, 6 up, 6 in 2024-03-24T00:57:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:51 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:57:52.262 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:51 smithi038 ceph-mon[112108]: pgmap v138: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 56/273 objects degraded (20.513%) 2024-03-24T00:57:52.262 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:51 smithi038 ceph-mon[112108]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:57:52.262 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:51 smithi038 ceph-mon[112108]: osd.3 [v2:172.21.15.88:6800/1863418220,v1:172.21.15.88:6801/1863418220] boot 2024-03-24T00:57:52.262 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:51 smithi038 ceph-mon[112108]: osdmap e69: 6 total, 6 up, 6 in 2024-03-24T00:57:52.262 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:51 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T00:57:53.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:52 smithi038 ceph-mon[112108]: osdmap e70: 6 total, 6 up, 6 in 2024-03-24T00:57:53.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:52 smithi088 ceph-mon[87351]: osdmap e70: 6 total, 6 up, 6 in 2024-03-24T00:57:54.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:53 smithi088 ceph-mon[87351]: pgmap v141: 65 pgs: 11 active+undersized+degraded, 9 active+undersized, 45 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 31/273 objects degraded (11.355%) 2024-03-24T00:57:54.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:53 smithi088 ceph-mon[87351]: Health check update: Degraded data redundancy: 31/273 objects degraded (11.355%), 11 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:54.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:53 smithi038 ceph-mon[112108]: pgmap v141: 65 pgs: 11 active+undersized+degraded, 9 active+undersized, 45 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 31/273 objects degraded (11.355%) 2024-03-24T00:57:54.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:53 smithi038 ceph-mon[112108]: Health check update: Degraded data redundancy: 31/273 objects degraded (11.355%), 11 pgs degraded (PG_DEGRADED) 2024-03-24T00:57:55.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:55.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:55.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:57:55.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:57:55.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:55.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:57:55.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:55.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:55.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:55.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:55 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T00:57:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:57:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:57:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:57:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:57:55.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:55 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T00:57:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:56 smithi088 ceph-mon[87351]: pgmap v142: 65 pgs: 11 active+undersized+degraded, 9 active+undersized, 45 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 31/273 objects degraded (11.355%) 2024-03-24T00:57:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:56 smithi088 ceph-mon[87351]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T00:57:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:56 smithi088 ceph-mon[87351]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-24T00:57:56.741 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:56 smithi038 ceph-mon[112108]: pgmap v142: 65 pgs: 11 active+undersized+degraded, 9 active+undersized, 45 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 31/273 objects degraded (11.355%) 2024-03-24T00:57:56.742 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:56 smithi038 ceph-mon[112108]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T00:57:56.742 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:56 smithi038 ceph-mon[112108]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-24T00:57:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:57 smithi088 ceph-mon[87351]: pgmap v143: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-24T00:57:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:57:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:57 smithi088 ceph-mon[87351]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 31/273 objects degraded (11.355%), 11 pgs degraded) 2024-03-24T00:57:57.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:57 smithi088 ceph-mon[87351]: Cluster is now healthy 2024-03-24T00:57:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:57 smithi038 ceph-mon[112108]: pgmap v143: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-24T00:57:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:57:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:57:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:57 smithi038 ceph-mon[112108]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 31/273 objects degraded (11.355%), 11 pgs degraded) 2024-03-24T00:57:58.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:57 smithi038 ceph-mon[112108]: Cluster is now healthy 2024-03-24T00:58:00.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:57:59 smithi038 ceph-mon[112108]: pgmap v144: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:58:00.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:57:59 smithi088 ceph-mon[87351]: pgmap v144: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:58:02.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:01 smithi038 ceph-mon[112108]: pgmap v145: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:02.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:01 smithi088 ceph-mon[87351]: pgmap v145: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:03.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.081+0000 7f0815600700 1 -- 172.21.15.38:0/2613531213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08100ffa00 msgr2=0x7f08100ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:03.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.081+0000 7f0815600700 1 --2- 172.21.15.38:0/2613531213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08100ffa00 0x7f08100ffde0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f080000b0a0 tx=0x7f080000b3b0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.082 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:02 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:03.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.081+0000 7f0815600700 1 -- 172.21.15.38:0/2613531213 shutdown_connections 2024-03-24T00:58:03.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.081+0000 7f0815600700 1 --2- 172.21.15.38:0/2613531213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810100320 0x7f08101047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.081+0000 7f0815600700 1 --2- 172.21.15.38:0/2613531213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08100ffa00 0x7f08100ffde0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.081+0000 7f0815600700 1 -- 172.21.15.38:0/2613531213 >> 172.21.15.38:0/2613531213 conn(0x7f0810076640 msgr2=0x7f08100fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:03.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.081+0000 7f0815600700 1 -- 172.21.15.38:0/2613531213 shutdown_connections 2024-03-24T00:58:03.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.081+0000 7f0815600700 1 -- 172.21.15.38:0/2613531213 wait complete. 2024-03-24T00:58:03.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.082+0000 7f0815600700 1 Processor -- start 2024-03-24T00:58:03.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.082+0000 7f0815600700 1 -- start start 2024-03-24T00:58:03.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.083+0000 7f0815600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08100ffa00 0x7f081019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.083+0000 7f0815600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810100320 0x7f081019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.083+0000 7f0815600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081019dc80 con 0x7f08100ffa00 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.083+0000 7f0815600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08101a18e0 con 0x7f0810100320 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.083+0000 7f080ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08100ffa00 0x7f081019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.083+0000 7f080ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08100ffa00 0x7f081019d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47642/0 (socket says 172.21.15.38:47642) 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.083+0000 7f080ea00700 1 -- 172.21.15.38:0/2000512407 learned_addr learned my addr 172.21.15.38:0/2000512407 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.083+0000 7f0807e00700 1 --2- 172.21.15.38:0/2000512407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810100320 0x7f081019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.084+0000 7f080ea00700 1 -- 172.21.15.38:0/2000512407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810100320 msgr2=0x7f081019d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.084+0000 7f080ea00700 1 --2- 172.21.15.38:0/2000512407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810100320 0x7f081019d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.084+0000 7f080ea00700 1 -- 172.21.15.38:0/2000512407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800009d00 con 0x7f08100ffa00 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.084+0000 7f080ea00700 1 --2- 172.21.15.38:0/2000512407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08100ffa00 0x7f081019d060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0800015040 tx=0x7f0800003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:03.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.084+0000 7f0807400700 1 -- 172.21.15.38:0/2000512407 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f080000e040 con 0x7f08100ffa00 2024-03-24T00:58:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.084+0000 7f0807400700 1 -- 172.21.15.38:0/2000512407 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0800004040 con 0x7f08100ffa00 2024-03-24T00:58:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.084+0000 7f0807400700 1 -- 172.21.15.38:0/2000512407 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f080001d6b0 con 0x7f08100ffa00 2024-03-24T00:58:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.084+0000 7f0815600700 1 -- 172.21.15.38:0/2000512407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08101a1b60 con 0x7f08100ffa00 2024-03-24T00:58:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.085+0000 7f0815600700 1 -- 172.21.15.38:0/2000512407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08101a2050 con 0x7f08100ffa00 2024-03-24T00:58:03.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.086+0000 7f0815600700 1 -- 172.21.15.38:0/2000512407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0810068d60 con 0x7f08100ffa00 2024-03-24T00:58:03.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.087+0000 7f0807400700 1 -- 172.21.15.38:0/2000512407 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f08000041b0 con 0x7f08100ffa00 2024-03-24T00:58:03.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.088+0000 7f0807400700 1 --2- 172.21.15.38:0/2000512407 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07f406dfa0 0x7f07f4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:03.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.088+0000 7f0807400700 1 -- 172.21.15.38:0/2000512407 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f080008f390 con 0x7f08100ffa00 2024-03-24T00:58:03.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.088+0000 7f0807e00700 1 --2- 172.21.15.38:0/2000512407 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07f406dfa0 0x7f07f4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:03.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.089+0000 7f0807e00700 1 --2- 172.21.15.38:0/2000512407 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07f406dfa0 0x7f07f4070460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07f8005950 tx=0x7f07f80058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:03.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.093+0000 7f0807400700 1 -- 172.21.15.38:0/2000512407 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f08000588e0 con 0x7f08100ffa00 2024-03-24T00:58:03.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:02 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:03.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.339+0000 7f0815600700 1 -- 172.21.15.38:0/2000512407 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f08101a2380 con 0x7f07f406dfa0 2024-03-24T00:58:03.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.341+0000 7f0807400700 1 -- 172.21.15.38:0/2000512407 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f08101a2380 con 0x7f07f406dfa0 2024-03-24T00:58:03.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.343+0000 7f0815600700 1 -- 172.21.15.38:0/2000512407 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07f406dfa0 msgr2=0x7f07f4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:03.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.343+0000 7f0815600700 1 --2- 172.21.15.38:0/2000512407 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07f406dfa0 0x7f07f4070460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07f8005950 tx=0x7f07f80058e0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.343+0000 7f0815600700 1 -- 172.21.15.38:0/2000512407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08100ffa00 msgr2=0x7f081019d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:03.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.343+0000 7f0815600700 1 --2- 172.21.15.38:0/2000512407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08100ffa00 0x7f081019d060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0800015040 tx=0x7f0800003ce0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.344+0000 7f0815600700 1 -- 172.21.15.38:0/2000512407 shutdown_connections 2024-03-24T00:58:03.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.344+0000 7f0815600700 1 --2- 172.21.15.38:0/2000512407 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f07f406dfa0 0x7f07f4070460 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.344+0000 7f0815600700 1 --2- 172.21.15.38:0/2000512407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08100ffa00 0x7f081019d060 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.344+0000 7f0815600700 1 --2- 172.21.15.38:0/2000512407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810100320 0x7f081019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.344+0000 7f0815600700 1 -- 172.21.15.38:0/2000512407 >> 172.21.15.38:0/2000512407 conn(0x7f0810076640 msgr2=0x7f081010e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:03.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.344+0000 7f0815600700 1 -- 172.21.15.38:0/2000512407 shutdown_connections 2024-03-24T00:58:03.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.344+0000 7f0815600700 1 -- 172.21.15.38:0/2000512407 wait complete. 2024-03-24T00:58:03.361 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:58:03.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.503+0000 7f7bf5600700 1 -- 172.21.15.38:0/1316248432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf00ffc00 msgr2=0x7f7bf0105740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:03.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.503+0000 7f7bf5600700 1 --2- 172.21.15.38:0/1316248432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf00ffc00 0x7f7bf0105740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc009910 tx=0x7f7bdc009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:03.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.503+0000 7f7bf5600700 1 -- 172.21.15.38:0/1316248432 shutdown_connections 2024-03-24T00:58:03.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.503+0000 7f7bf5600700 1 --2- 172.21.15.38:0/1316248432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf00ffc00 0x7f7bf0105740 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.503+0000 7f7bf5600700 1 --2- 172.21.15.38:0/1316248432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bf00ff250 0x7f7bf00ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.503+0000 7f7bf5600700 1 -- 172.21.15.38:0/1316248432 >> 172.21.15.38:0/1316248432 conn(0x7f7bf0075e10 msgr2=0x7f7bf00fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:03.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.504+0000 7f7bf5600700 1 -- 172.21.15.38:0/1316248432 shutdown_connections 2024-03-24T00:58:03.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.504+0000 7f7bf5600700 1 -- 172.21.15.38:0/1316248432 wait complete. 2024-03-24T00:58:03.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.505+0000 7f7bf5600700 1 Processor -- start 2024-03-24T00:58:03.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.505+0000 7f7bf5600700 1 -- start start 2024-03-24T00:58:03.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.505+0000 7f7bf5600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf00ff250 0x7f7bf019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:03.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.505+0000 7f7bf5600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bf00ffc00 0x7f7bf019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:03.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.505+0000 7f7bf5600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf019d210 con 0x7f7bf00ff250 2024-03-24T00:58:03.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.505+0000 7f7bf5600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf0196780 con 0x7f7bf00ffc00 2024-03-24T00:58:03.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.506+0000 7f7beea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf00ff250 0x7f7bf019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:03.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.506+0000 7f7beea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf00ff250 0x7f7bf019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47654/0 (socket says 172.21.15.38:47654) 2024-03-24T00:58:03.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.506+0000 7f7beea00700 1 -- 172.21.15.38:0/2182778304 learned_addr learned my addr 172.21.15.38:0/2182778304 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:03.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.506+0000 7f7bee000700 1 --2- 172.21.15.38:0/2182778304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bf00ffc00 0x7f7bf019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:03.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.506+0000 7f7beea00700 1 -- 172.21.15.38:0/2182778304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bf00ffc00 msgr2=0x7f7bf019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:03.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.506+0000 7f7beea00700 1 --2- 172.21.15.38:0/2182778304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bf00ffc00 0x7f7bf019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.506+0000 7f7beea00700 1 -- 172.21.15.38:0/2182778304 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc0095f0 con 0x7f7bf00ff250 2024-03-24T00:58:03.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.507+0000 7f7beea00700 1 --2- 172.21.15.38:0/2182778304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf00ff250 0x7f7bf019c640 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7be000d8d0 tx=0x7f7be000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:03.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.507+0000 7f7be7400700 1 -- 172.21.15.38:0/2182778304 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7be0009940 con 0x7f7bf00ff250 2024-03-24T00:58:03.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.507+0000 7f7be7400700 1 -- 172.21.15.38:0/2182778304 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7be0010460 con 0x7f7bf00ff250 2024-03-24T00:58:03.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.507+0000 7f7be7400700 1 -- 172.21.15.38:0/2182778304 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7be000f5f0 con 0x7f7bf00ff250 2024-03-24T00:58:03.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.507+0000 7f7bf5600700 1 -- 172.21.15.38:0/2182778304 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf0196a60 con 0x7f7bf00ff250 2024-03-24T00:58:03.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.507+0000 7f7bf5600700 1 -- 172.21.15.38:0/2182778304 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf0197030 con 0x7f7bf00ff250 2024-03-24T00:58:03.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.508+0000 7f7bf5600700 1 -- 172.21.15.38:0/2182778304 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf0109090 con 0x7f7bf00ff250 2024-03-24T00:58:03.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.509+0000 7f7be7400700 1 -- 172.21.15.38:0/2182778304 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f7be00105d0 con 0x7f7bf00ff250 2024-03-24T00:58:03.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.510+0000 7f7be7400700 1 --2- 172.21.15.38:0/2182778304 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7bd406dfa0 0x7f7bd4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:03.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.510+0000 7f7be7400700 1 -- 172.21.15.38:0/2182778304 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7be008dc00 con 0x7f7bf00ff250 2024-03-24T00:58:03.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.511+0000 7f7bee000700 1 --2- 172.21.15.38:0/2182778304 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7bd406dfa0 0x7f7bd4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:03.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.511+0000 7f7bee000700 1 --2- 172.21.15.38:0/2182778304 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7bd406dfa0 0x7f7bd4070460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0059d0 tx=0x7f7bdc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:03.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.515+0000 7f7be7400700 1 -- 172.21.15.38:0/2182778304 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7be0093020 con 0x7f7bf00ff250 2024-03-24T00:58:03.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.761+0000 7f7bf5600700 1 -- 172.21.15.38:0/2182778304 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bf0197cb0 con 0x7f7bd406dfa0 2024-03-24T00:58:03.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.763+0000 7f7be7400700 1 -- 172.21.15.38:0/2182778304 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7bf0197cb0 con 0x7f7bd406dfa0 2024-03-24T00:58:03.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.766+0000 7f7bf5600700 1 -- 172.21.15.38:0/2182778304 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7bd406dfa0 msgr2=0x7f7bd4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:03.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.766+0000 7f7bf5600700 1 --2- 172.21.15.38:0/2182778304 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7bd406dfa0 0x7f7bd4070460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0059d0 tx=0x7f7bdc01a040 comp rx=0 tx=0).stop 2024-03-24T00:58:03.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.766+0000 7f7bf5600700 1 -- 172.21.15.38:0/2182778304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf00ff250 msgr2=0x7f7bf019c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:03.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.766+0000 7f7bf5600700 1 --2- 172.21.15.38:0/2182778304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf00ff250 0x7f7bf019c640 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7be000d8d0 tx=0x7f7be000dc90 comp rx=0 tx=0).stop 2024-03-24T00:58:03.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.766+0000 7f7bf5600700 1 -- 172.21.15.38:0/2182778304 shutdown_connections 2024-03-24T00:58:03.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.766+0000 7f7bf5600700 1 --2- 172.21.15.38:0/2182778304 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7bd406dfa0 0x7f7bd4070460 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.766+0000 7f7bf5600700 1 --2- 172.21.15.38:0/2182778304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bf00ff250 0x7f7bf019c640 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.766+0000 7f7bf5600700 1 --2- 172.21.15.38:0/2182778304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bf00ffc00 0x7f7bf019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.766+0000 7f7bf5600700 1 -- 172.21.15.38:0/2182778304 >> 172.21.15.38:0/2182778304 conn(0x7f7bf0075e10 msgr2=0x7f7bf0104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:03.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.767+0000 7f7bf5600700 1 -- 172.21.15.38:0/2182778304 shutdown_connections 2024-03-24T00:58:03.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.767+0000 7f7bf5600700 1 -- 172.21.15.38:0/2182778304 wait complete. 2024-03-24T00:58:03.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.923+0000 7f7e6f200700 1 -- 172.21.15.38:0/1891052445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e70108ac0 msgr2=0x7f7e70108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:03.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.923+0000 7f7e6f200700 1 --2- 172.21.15.38:0/1891052445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e70108ac0 0x7f7e70108ea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009910 tx=0x7f7e60009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:03.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.923+0000 7f7e6f200700 1 -- 172.21.15.38:0/1891052445 shutdown_connections 2024-03-24T00:58:03.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.923+0000 7f7e6f200700 1 --2- 172.21.15.38:0/1891052445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e70102ac0 0x7f7e70102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.923+0000 7f7e6f200700 1 --2- 172.21.15.38:0/1891052445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e70108ac0 0x7f7e70108ea0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.923+0000 7f7e6f200700 1 -- 172.21.15.38:0/1891052445 >> 172.21.15.38:0/1891052445 conn(0x7f7e700fe570 msgr2=0x7f7e70100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:03.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.923+0000 7f7e6f200700 1 -- 172.21.15.38:0/1891052445 shutdown_connections 2024-03-24T00:58:03.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.923+0000 7f7e6f200700 1 -- 172.21.15.38:0/1891052445 wait complete. 2024-03-24T00:58:03.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.924+0000 7f7e6f200700 1 Processor -- start 2024-03-24T00:58:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.924+0000 7f7e6f200700 1 -- start start 2024-03-24T00:58:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.925+0000 7f7e6f200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e70102ac0 0x7f7e7019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.925+0000 7f7e6f200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e70108ac0 0x7f7e7019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.925+0000 7f7e6f200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7019f320 con 0x7f7e70108ac0 2024-03-24T00:58:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.925+0000 7f7e6f200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e70198900 con 0x7f7e70102ac0 2024-03-24T00:58:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.925+0000 7f7e6d400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e70108ac0 0x7f7e7019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.925+0000 7f7e6d400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e70108ac0 0x7f7e7019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47684/0 (socket says 172.21.15.38:47684) 2024-03-24T00:58:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.925+0000 7f7e6d400700 1 -- 172.21.15.38:0/3131672013 learned_addr learned my addr 172.21.15.38:0/3131672013 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.926+0000 7f7e6de00700 1 --2- 172.21.15.38:0/3131672013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e70102ac0 0x7f7e7019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.926+0000 7f7e6d400700 1 -- 172.21.15.38:0/3131672013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e70102ac0 msgr2=0x7f7e7019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.926+0000 7f7e6d400700 1 --2- 172.21.15.38:0/3131672013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e70102ac0 0x7f7e7019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.926+0000 7f7e6d400700 1 -- 172.21.15.38:0/3131672013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e600095f0 con 0x7f7e70108ac0 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.926+0000 7f7e6d400700 1 --2- 172.21.15.38:0/3131672013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e70108ac0 0x7f7e7019ed00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00b4b0 tx=0x7f7e5c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.926+0000 7f7e66a00700 1 -- 172.21.15.38:0/3131672013 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e5c011800 con 0x7f7e70108ac0 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.926+0000 7f7e66a00700 1 -- 172.21.15.38:0/3131672013 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e5c011e20 con 0x7f7e70108ac0 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.926+0000 7f7e66a00700 1 -- 172.21.15.38:0/3131672013 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e5c00f2e0 con 0x7f7e70108ac0 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.926+0000 7f7e6f200700 1 -- 172.21.15.38:0/3131672013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e70198be0 con 0x7f7e70108ac0 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.927+0000 7f7e6f200700 1 -- 172.21.15.38:0/3131672013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e70199130 con 0x7f7e70108ac0 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.928+0000 7f7e6f200700 1 -- 172.21.15.38:0/3131672013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7010b0c0 con 0x7f7e70108ac0 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.929+0000 7f7e66a00700 1 -- 172.21.15.38:0/3131672013 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f7e5c010380 con 0x7f7e70108ac0 2024-03-24T00:58:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.930+0000 7f7e66a00700 1 --2- 172.21.15.38:0/3131672013 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7e5406dfb0 0x7f7e54070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:03.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.930+0000 7f7e66a00700 1 -- 172.21.15.38:0/3131672013 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7e5c08d820 con 0x7f7e70108ac0 2024-03-24T00:58:03.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.930+0000 7f7e6de00700 1 --2- 172.21.15.38:0/3131672013 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7e5406dfb0 0x7f7e54070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:03.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.931+0000 7f7e6de00700 1 --2- 172.21.15.38:0/3131672013 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7e5406dfb0 0x7f7e54070470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000b5c0 tx=0x7f7e6001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:03.935+0000 7f7e66a00700 1 -- 172.21.15.38:0/3131672013 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7e5c05a950 con 0x7f7e70108ac0 2024-03-24T00:58:04.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:03 smithi038 ceph-mon[112108]: pgmap v146: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-24T00:58:04.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:03 smithi038 ceph-mon[112108]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:04.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.160+0000 7f7e6f200700 1 -- 172.21.15.38:0/3131672013 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e70199e30 con 0x7f7e5406dfb0 2024-03-24T00:58:04.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.177+0000 7f7e66a00700 1 -- 172.21.15.38:0/3131672013 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f7e70199e30 con 0x7f7e5406dfb0 2024-03-24T00:58:04.179 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:58:04.180 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (6m) 45s ago 20m 13.5M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:58:04.180 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (20m) 45s ago 20m 9068k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:58:04.180 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (19m) 17s ago 19m 8548k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:58:04.180 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (2m) 45s ago 20m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T00:58:04.180 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (2m) 17s ago 19m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T00:58:04.180 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (5m) 45s ago 20m 75.3M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (12m) 45s ago 12m 97.9M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (13m) 45s ago 13m 150M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (13m) 17s ago 13m 24.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (12m) 17s ago 12m 24.9M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (9m) 45s ago 22m 502M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (8m) 17s ago 19m 450M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (4m) 45s ago 22m 48.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (3m) 17s ago 18m 42.9M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (7m) 45s ago 20m 7755k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (7m) 17s ago 19m 8158k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (116s) 45s ago 17m 97.1M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (85s) 45s ago 17m 104M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cb72ac446ba7 2024-03-24T00:58:04.181 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (54s) 45s ago 16m 11.2M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c73f3e8276b6 2024-03-24T00:58:04.182 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (22s) 17s ago 16m 11.1M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b2afaff761d8 2024-03-24T00:58:04.182 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (15m) 17s ago 15m 285M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 d5aec32ff459 2024-03-24T00:58:04.182 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (14m) 17s ago 14m 250M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:58:04.182 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (6m) 45s ago 19m 70.6M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:58:04.182 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:03 smithi088 ceph-mon[87351]: pgmap v146: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-24T00:58:04.183 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:03 smithi088 ceph-mon[87351]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:04.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.180+0000 7f7e6f200700 1 -- 172.21.15.38:0/3131672013 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7e5406dfb0 msgr2=0x7f7e54070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:04.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.180+0000 7f7e6f200700 1 --2- 172.21.15.38:0/3131672013 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7e5406dfb0 0x7f7e54070470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000b5c0 tx=0x7f7e6001a040 comp rx=0 tx=0).stop 2024-03-24T00:58:04.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.180+0000 7f7e6f200700 1 -- 172.21.15.38:0/3131672013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e70108ac0 msgr2=0x7f7e7019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:04.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.180+0000 7f7e6f200700 1 --2- 172.21.15.38:0/3131672013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e70108ac0 0x7f7e7019ed00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00b4b0 tx=0x7f7e5c00b870 comp rx=0 tx=0).stop 2024-03-24T00:58:04.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.180+0000 7f7e6f200700 1 -- 172.21.15.38:0/3131672013 shutdown_connections 2024-03-24T00:58:04.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.180+0000 7f7e6f200700 1 --2- 172.21.15.38:0/3131672013 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7e5406dfb0 0x7f7e54070470 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.180+0000 7f7e6f200700 1 --2- 172.21.15.38:0/3131672013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e70102ac0 0x7f7e7019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.180+0000 7f7e6f200700 1 --2- 172.21.15.38:0/3131672013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e70108ac0 0x7f7e7019ed00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.180+0000 7f7e6f200700 1 -- 172.21.15.38:0/3131672013 >> 172.21.15.38:0/3131672013 conn(0x7f7e700fe570 msgr2=0x7f7e70100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:04.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.180+0000 7f7e6f200700 1 -- 172.21.15.38:0/3131672013 shutdown_connections 2024-03-24T00:58:04.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.180+0000 7f7e6f200700 1 -- 172.21.15.38:0/3131672013 wait complete. 2024-03-24T00:58:04.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.340+0000 7fbba1e00700 1 -- 172.21.15.38:0/1858158545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb9c102530 msgr2=0x7fbb9c10aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:04.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.340+0000 7fbba1e00700 1 --2- 172.21.15.38:0/1858158545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb9c102530 0x7fbb9c10aa20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009910 tx=0x7fbb84009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:04.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.340+0000 7fbba1e00700 1 -- 172.21.15.38:0/1858158545 shutdown_connections 2024-03-24T00:58:04.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.340+0000 7fbba1e00700 1 --2- 172.21.15.38:0/1858158545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb9c102530 0x7fbb9c10aa20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.340+0000 7fbba1e00700 1 --2- 172.21.15.38:0/1858158545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb9c101c10 0x7fbb9c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.340+0000 7fbba1e00700 1 -- 172.21.15.38:0/1858158545 >> 172.21.15.38:0/1858158545 conn(0x7fbb9c076640 msgr2=0x7fbb9c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:04.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.340+0000 7fbba1e00700 1 -- 172.21.15.38:0/1858158545 shutdown_connections 2024-03-24T00:58:04.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.340+0000 7fbba1e00700 1 -- 172.21.15.38:0/1858158545 wait complete. 2024-03-24T00:58:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.341+0000 7fbba1e00700 1 Processor -- start 2024-03-24T00:58:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.342+0000 7fbba1e00700 1 -- start start 2024-03-24T00:58:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.342+0000 7fbba1e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb9c101c10 0x7fbb9c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.342+0000 7fbba1e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb9c102530 0x7fbb9c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.342+0000 7fbba1e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c19f470 con 0x7fbb9c102530 2024-03-24T00:58:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.342+0000 7fbba1e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c198a50 con 0x7fbb9c101c10 2024-03-24T00:58:04.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.342+0000 7fbb92000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb9c102530 0x7fbb9c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:04.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.342+0000 7fbb92000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb9c102530 0x7fbb9c19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47702/0 (socket says 172.21.15.38:47702) 2024-03-24T00:58:04.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.343+0000 7fbb92000700 1 -- 172.21.15.38:0/1134141794 learned_addr learned my addr 172.21.15.38:0/1134141794 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:04.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.343+0000 7fbb9aa00700 1 --2- 172.21.15.38:0/1134141794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb9c101c10 0x7fbb9c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:04.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.343+0000 7fbb92000700 1 -- 172.21.15.38:0/1134141794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb9c101c10 msgr2=0x7fbb9c19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:04.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.343+0000 7fbb92000700 1 --2- 172.21.15.38:0/1134141794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb9c101c10 0x7fbb9c19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.343+0000 7fbb92000700 1 -- 172.21.15.38:0/1134141794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840095f0 con 0x7fbb9c102530 2024-03-24T00:58:04.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.343+0000 7fbb92000700 1 --2- 172.21.15.38:0/1134141794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb9c102530 0x7fbb9c19ee50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbb84005b40 tx=0x7fbb8400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:04.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.344+0000 7fbba0a00700 1 -- 172.21.15.38:0/1134141794 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb8401d070 con 0x7fbb9c102530 2024-03-24T00:58:04.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.344+0000 7fbba0a00700 1 -- 172.21.15.38:0/1134141794 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb84003d50 con 0x7fbb9c102530 2024-03-24T00:58:04.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.344+0000 7fbba0a00700 1 -- 172.21.15.38:0/1134141794 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb8400fe90 con 0x7fbb9c102530 2024-03-24T00:58:04.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.344+0000 7fbba1e00700 1 -- 172.21.15.38:0/1134141794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9c198cd0 con 0x7fbb9c102530 2024-03-24T00:58:04.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.344+0000 7fbba1e00700 1 -- 172.21.15.38:0/1134141794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9c1991c0 con 0x7fbb9c102530 2024-03-24T00:58:04.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.345+0000 7fbba1e00700 1 -- 172.21.15.38:0/1134141794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9c100f40 con 0x7fbb9c102530 2024-03-24T00:58:04.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.346+0000 7fbba0a00700 1 -- 172.21.15.38:0/1134141794 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fbb84003890 con 0x7fbb9c102530 2024-03-24T00:58:04.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.347+0000 7fbba0a00700 1 --2- 172.21.15.38:0/1134141794 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbb8006dfb0 0x7fbb80070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:04.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.347+0000 7fbba0a00700 1 -- 172.21.15.38:0/1134141794 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbb8408f130 con 0x7fbb9c102530 2024-03-24T00:58:04.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.347+0000 7fbb9aa00700 1 --2- 172.21.15.38:0/1134141794 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbb8006dfb0 0x7fbb80070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:04.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.348+0000 7fbb9aa00700 1 --2- 172.21.15.38:0/1134141794 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbb8006dfb0 0x7fbb80070470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0060b0 tx=0x7fbb8c007b80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:04.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.352+0000 7fbba0a00700 1 -- 172.21.15.38:0/1134141794 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbb8405c110 con 0x7fbb9c102530 2024-03-24T00:58:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.671+0000 7fbba1e00700 1 -- 172.21.15.38:0/1134141794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbb9c068d60 con 0x7fbb9c102530 2024-03-24T00:58:04.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.672+0000 7fbba0a00700 1 -- 172.21.15.38:0/1134141794 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7fbb8405bca0 con 0x7fbb9c102530 2024-03-24T00:58:04.673 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:58:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:58:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:58:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:58:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:58:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:58:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:58:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:58:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2, 2024-03-24T00:58:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-24T00:58:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:58:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:58:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:58:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:58:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:58:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 6, 2024-03-24T00:58:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 8 2024-03-24T00:58:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:58:04.674 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:58:04.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.675+0000 7fbba1e00700 1 -- 172.21.15.38:0/1134141794 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbb8006dfb0 msgr2=0x7fbb80070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:04.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.675+0000 7fbba1e00700 1 --2- 172.21.15.38:0/1134141794 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbb8006dfb0 0x7fbb80070470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0060b0 tx=0x7fbb8c007b80 comp rx=0 tx=0).stop 2024-03-24T00:58:04.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.675+0000 7fbba1e00700 1 -- 172.21.15.38:0/1134141794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb9c102530 msgr2=0x7fbb9c19ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:04.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.675+0000 7fbba1e00700 1 --2- 172.21.15.38:0/1134141794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb9c102530 0x7fbb9c19ee50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbb84005b40 tx=0x7fbb8400bc10 comp rx=0 tx=0).stop 2024-03-24T00:58:04.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.675+0000 7fbba1e00700 1 -- 172.21.15.38:0/1134141794 shutdown_connections 2024-03-24T00:58:04.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.675+0000 7fbba1e00700 1 --2- 172.21.15.38:0/1134141794 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbb8006dfb0 0x7fbb80070470 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.675+0000 7fbba1e00700 1 --2- 172.21.15.38:0/1134141794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb9c101c10 0x7fbb9c19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.675+0000 7fbba1e00700 1 --2- 172.21.15.38:0/1134141794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb9c102530 0x7fbb9c19ee50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.675+0000 7fbba1e00700 1 -- 172.21.15.38:0/1134141794 >> 172.21.15.38:0/1134141794 conn(0x7fbb9c076640 msgr2=0x7fbb9c109350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.675+0000 7fbba1e00700 1 -- 172.21.15.38:0/1134141794 shutdown_connections 2024-03-24T00:58:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.675+0000 7fbba1e00700 1 -- 172.21.15.38:0/1134141794 wait complete. 2024-03-24T00:58:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.830+0000 7f7133400700 1 -- 172.21.15.38:0/3924643479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f712c0ff7a0 msgr2=0x7f712c0ffb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.830+0000 7f7133400700 1 --2- 172.21.15.38:0/3924643479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f712c0ff7a0 0x7f712c0ffb80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f711c009910 tx=0x7f711c009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.831+0000 7f7133400700 1 -- 172.21.15.38:0/3924643479 shutdown_connections 2024-03-24T00:58:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.831+0000 7f7133400700 1 --2- 172.21.15.38:0/3924643479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f712c100150 0x7f712c10ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.831+0000 7f7133400700 1 --2- 172.21.15.38:0/3924643479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f712c0ff7a0 0x7f712c0ffb80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.831+0000 7f7133400700 1 -- 172.21.15.38:0/3924643479 >> 172.21.15.38:0/3924643479 conn(0x7f712c076640 msgr2=0x7f712c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.831+0000 7f7133400700 1 -- 172.21.15.38:0/3924643479 shutdown_connections 2024-03-24T00:58:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.831+0000 7f7133400700 1 -- 172.21.15.38:0/3924643479 wait complete. 2024-03-24T00:58:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.832+0000 7f7133400700 1 Processor -- start 2024-03-24T00:58:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.832+0000 7f7133400700 1 -- start start 2024-03-24T00:58:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.833+0000 7f7133400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f712c0ff7a0 0x7f712c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.833+0000 7f7133400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f712c100150 0x7f712c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.833+0000 7f7133400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c19f3f0 con 0x7f712c0ff7a0 2024-03-24T00:58:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.833+0000 7f7133400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c1989d0 con 0x7f712c100150 2024-03-24T00:58:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.833+0000 7f7130a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f712c0ff7a0 0x7f712c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.833+0000 7f7130a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f712c0ff7a0 0x7f712c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47730/0 (socket says 172.21.15.38:47730) 2024-03-24T00:58:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.833+0000 7f7130a00700 1 -- 172.21.15.38:0/2397346144 learned_addr learned my addr 172.21.15.38:0/2397346144 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.833+0000 7f7123e00700 1 --2- 172.21.15.38:0/2397346144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f712c100150 0x7f712c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.833+0000 7f7130a00700 1 -- 172.21.15.38:0/2397346144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f712c100150 msgr2=0x7f712c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.833+0000 7f7130a00700 1 --2- 172.21.15.38:0/2397346144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f712c100150 0x7f712c19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.833+0000 7f7130a00700 1 -- 172.21.15.38:0/2397346144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c0095f0 con 0x7f712c0ff7a0 2024-03-24T00:58:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.834+0000 7f7130a00700 1 --2- 172.21.15.38:0/2397346144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f712c0ff7a0 0x7f712c19e890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f711c006010 tx=0x7f711c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.834+0000 7f712a000700 1 -- 172.21.15.38:0/2397346144 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f711c01d070 con 0x7f712c0ff7a0 2024-03-24T00:58:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.834+0000 7f712a000700 1 -- 172.21.15.38:0/2397346144 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f711c003d50 con 0x7f712c0ff7a0 2024-03-24T00:58:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.834+0000 7f712a000700 1 -- 172.21.15.38:0/2397346144 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f711c021410 con 0x7f712c0ff7a0 2024-03-24T00:58:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.834+0000 7f7133400700 1 -- 172.21.15.38:0/2397346144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c198c50 con 0x7f712c0ff7a0 2024-03-24T00:58:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.834+0000 7f7133400700 1 -- 172.21.15.38:0/2397346144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c1991a0 con 0x7f712c0ff7a0 2024-03-24T00:58:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.835+0000 7f7133400700 1 -- 172.21.15.38:0/2397346144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c068d60 con 0x7f712c0ff7a0 2024-03-24T00:58:04.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.836+0000 7f712a000700 1 -- 172.21.15.38:0/2397346144 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f711c00f6f0 con 0x7f712c0ff7a0 2024-03-24T00:58:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.838+0000 7f712a000700 1 --2- 172.21.15.38:0/2397346144 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f711006dfa0 0x7f7110070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.838+0000 7f712a000700 1 -- 172.21.15.38:0/2397346144 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f711c08ee90 con 0x7f712c0ff7a0 2024-03-24T00:58:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.838+0000 7f7123e00700 1 --2- 172.21.15.38:0/2397346144 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f711006dfa0 0x7f7110070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.839+0000 7f7123e00700 1 --2- 172.21.15.38:0/2397346144 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f711006dfa0 0x7f7110070460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7114005950 tx=0x7f71140058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:04.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:04.842+0000 7f712a000700 1 -- 172.21.15.38:0/2397346144 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f711c05bdf0 con 0x7f712c0ff7a0 2024-03-24T00:58:05.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:04 smithi038 ceph-mon[112108]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:05.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:04 smithi038 ceph-mon[112108]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:05.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:04 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1134141794' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:58:05.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.109+0000 7f7133400700 1 -- 172.21.15.38:0/2397346144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f712c1994d0 con 0x7f712c0ff7a0 2024-03-24T00:58:05.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.110+0000 7f712a000700 1 -- 172.21.15.38:0/2397346144 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1900 (secure 0 0 0) 0x7f711c05b980 con 0x7f712c0ff7a0 2024-03-24T00:58:05.111 INFO:teuthology.orchestra.run.smithi038.stdout:e20 2024-03-24T00:58:05.111 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:58:05.111 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 20 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:58:05.112 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:58:05.113 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:58:05.114 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:58:05.114 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T00:58:05.114 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:58:05.114 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:58:05.114 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:58:05.114 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:58:05.114 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:58:05.114 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:58:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.112+0000 7f7133400700 1 -- 172.21.15.38:0/2397346144 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f711006dfa0 msgr2=0x7f7110070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.112+0000 7f7133400700 1 --2- 172.21.15.38:0/2397346144 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f711006dfa0 0x7f7110070460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7114005950 tx=0x7f71140058e0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.112+0000 7f7133400700 1 -- 172.21.15.38:0/2397346144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f712c0ff7a0 msgr2=0x7f712c19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.112+0000 7f7133400700 1 --2- 172.21.15.38:0/2397346144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f712c0ff7a0 0x7f712c19e890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f711c006010 tx=0x7f711c0056a0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.112+0000 7f7133400700 1 -- 172.21.15.38:0/2397346144 shutdown_connections 2024-03-24T00:58:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.112+0000 7f7133400700 1 --2- 172.21.15.38:0/2397346144 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f711006dfa0 0x7f7110070460 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.112+0000 7f7133400700 1 --2- 172.21.15.38:0/2397346144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f712c0ff7a0 0x7f712c19e890 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.112+0000 7f7133400700 1 --2- 172.21.15.38:0/2397346144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f712c100150 0x7f712c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.112+0000 7f7133400700 1 -- 172.21.15.38:0/2397346144 >> 172.21.15.38:0/2397346144 conn(0x7f712c076640 msgr2=0x7f712c109580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.113+0000 7f7133400700 1 -- 172.21.15.38:0/2397346144 shutdown_connections 2024-03-24T00:58:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.113+0000 7f7133400700 1 -- 172.21.15.38:0/2397346144 wait complete. 2024-03-24T00:58:05.116 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:58:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:04 smithi088 ceph-mon[87351]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:04 smithi088 ceph-mon[87351]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:05.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:04 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1134141794' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:58:05.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.266+0000 7fd9bb000700 1 -- 172.21.15.38:0/1426780549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b4107cc0 msgr2=0x7fd9b410bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:05.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.266+0000 7fd9bb000700 1 --2- 172.21.15.38:0/1426780549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b4107cc0 0x7fd9b410bd10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009910 tx=0x7fd9a0009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:05.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.266+0000 7fd9bb000700 1 -- 172.21.15.38:0/1426780549 shutdown_connections 2024-03-24T00:58:05.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.266+0000 7fd9bb000700 1 --2- 172.21.15.38:0/1426780549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b4107cc0 0x7fd9b410bd10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.266+0000 7fd9bb000700 1 --2- 172.21.15.38:0/1426780549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4107310 0x7fd9b41076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.266+0000 7fd9bb000700 1 -- 172.21.15.38:0/1426780549 >> 172.21.15.38:0/1426780549 conn(0x7fd9b40768e0 msgr2=0x7fd9b4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:05.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.266+0000 7fd9bb000700 1 -- 172.21.15.38:0/1426780549 shutdown_connections 2024-03-24T00:58:05.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.266+0000 7fd9bb000700 1 -- 172.21.15.38:0/1426780549 wait complete. 2024-03-24T00:58:05.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.267+0000 7fd9bb000700 1 Processor -- start 2024-03-24T00:58:05.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.267+0000 7fd9bb000700 1 -- start start 2024-03-24T00:58:05.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.267+0000 7fd9bb000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4107310 0x7fd9b419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:05.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.268+0000 7fd9bb000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b4107cc0 0x7fd9b419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:05.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.268+0000 7fd9bb000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b419dbe0 con 0x7fd9b4107cc0 2024-03-24T00:58:05.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.268+0000 7fd9bb000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b41a18b0 con 0x7fd9b4107310 2024-03-24T00:58:05.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.268+0000 7fd9b3400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b4107cc0 0x7fd9b419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:05.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.268+0000 7fd9b3400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b4107cc0 0x7fd9b419d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47752/0 (socket says 172.21.15.38:47752) 2024-03-24T00:58:05.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.268+0000 7fd9b3400700 1 -- 172.21.15.38:0/1874105054 learned_addr learned my addr 172.21.15.38:0/1874105054 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:05.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.268+0000 7fd9b3e00700 1 --2- 172.21.15.38:0/1874105054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4107310 0x7fd9b419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:05.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.268+0000 7fd9b3400700 1 -- 172.21.15.38:0/1874105054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4107310 msgr2=0x7fd9b419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:05.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.268+0000 7fd9b3400700 1 --2- 172.21.15.38:0/1874105054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4107310 0x7fd9b419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.269+0000 7fd9b3400700 1 -- 172.21.15.38:0/1874105054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a00095f0 con 0x7fd9b4107cc0 2024-03-24T00:58:05.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.269+0000 7fd9b3400700 1 --2- 172.21.15.38:0/1874105054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b4107cc0 0x7fd9b419d500 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0000c00 tx=0x7fd9a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:05.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.269+0000 7fd9b0c00700 1 -- 172.21.15.38:0/1874105054 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9a001d070 con 0x7fd9b4107cc0 2024-03-24T00:58:05.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.269+0000 7fd9b0c00700 1 -- 172.21.15.38:0/1874105054 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9a0005700 con 0x7fd9b4107cc0 2024-03-24T00:58:05.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.269+0000 7fd9bb000700 1 -- 172.21.15.38:0/1874105054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b41a1b30 con 0x7fd9b4107cc0 2024-03-24T00:58:05.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.269+0000 7fd9b0c00700 1 -- 172.21.15.38:0/1874105054 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9a000fe50 con 0x7fd9b4107cc0 2024-03-24T00:58:05.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.269+0000 7fd9bb000700 1 -- 172.21.15.38:0/1874105054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b41a2020 con 0x7fd9b4107cc0 2024-03-24T00:58:05.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.271+0000 7fd9bb000700 1 -- 172.21.15.38:0/1874105054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b410f710 con 0x7fd9b4107cc0 2024-03-24T00:58:05.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.272+0000 7fd9b0c00700 1 -- 172.21.15.38:0/1874105054 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd9a000f640 con 0x7fd9b4107cc0 2024-03-24T00:58:05.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.273+0000 7fd9b0c00700 1 --2- 172.21.15.38:0/1874105054 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd99806dfb0 0x7fd998070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:05.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.273+0000 7fd9b0c00700 1 -- 172.21.15.38:0/1874105054 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd9a008f1f0 con 0x7fd9b4107cc0 2024-03-24T00:58:05.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.273+0000 7fd9b3e00700 1 --2- 172.21.15.38:0/1874105054 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd99806dfb0 0x7fd998070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:05.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.274+0000 7fd9b3e00700 1 --2- 172.21.15.38:0/1874105054 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd99806dfb0 0x7fd998070470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4005950 tx=0x7fd9a40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:05.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.277+0000 7fd9b0c00700 1 -- 172.21.15.38:0/1874105054 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd9a0058620 con 0x7fd9b4107cc0 2024-03-24T00:58:05.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.522+0000 7fd9bb000700 1 -- 172.21.15.38:0/1874105054 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9b419e340 con 0x7fd99806dfb0 2024-03-24T00:58:05.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.524+0000 7fd9b0c00700 1 -- 172.21.15.38:0/1874105054 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd9b419e340 con 0x7fd99806dfb0 2024-03-24T00:58:05.525 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:58:05.525 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:58:05.525 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:58:05.525 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:58:05.525 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:58:05.526 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T00:58:05.526 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-03-24T00:58:05.526 INFO:teuthology.orchestra.run.smithi038.stdout: "crash" 2024-03-24T00:58:05.526 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:58:05.526 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "10/23 daemons upgraded", 2024-03-24T00:58:05.526 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T00:58:05.526 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:58:05.526 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:58:05.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.527+0000 7fd9bb000700 1 -- 172.21.15.38:0/1874105054 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd99806dfb0 msgr2=0x7fd998070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:05.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.527+0000 7fd9bb000700 1 --2- 172.21.15.38:0/1874105054 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd99806dfb0 0x7fd998070470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4005950 tx=0x7fd9a40058e0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.527+0000 7fd9bb000700 1 -- 172.21.15.38:0/1874105054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b4107cc0 msgr2=0x7fd9b419d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:05.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.527+0000 7fd9bb000700 1 --2- 172.21.15.38:0/1874105054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b4107cc0 0x7fd9b419d500 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0000c00 tx=0x7fd9a000be20 comp rx=0 tx=0).stop 2024-03-24T00:58:05.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.527+0000 7fd9bb000700 1 -- 172.21.15.38:0/1874105054 shutdown_connections 2024-03-24T00:58:05.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.527+0000 7fd9bb000700 1 --2- 172.21.15.38:0/1874105054 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd99806dfb0 0x7fd998070470 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.527+0000 7fd9bb000700 1 --2- 172.21.15.38:0/1874105054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4107310 0x7fd9b419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.527+0000 7fd9bb000700 1 --2- 172.21.15.38:0/1874105054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b4107cc0 0x7fd9b419d500 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.527+0000 7fd9bb000700 1 -- 172.21.15.38:0/1874105054 >> 172.21.15.38:0/1874105054 conn(0x7fd9b40768e0 msgr2=0x7fd9b4076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:05.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.527+0000 7fd9bb000700 1 -- 172.21.15.38:0/1874105054 shutdown_connections 2024-03-24T00:58:05.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.527+0000 7fd9bb000700 1 -- 172.21.15.38:0/1874105054 wait complete. 2024-03-24T00:58:05.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.674+0000 7f9cea600700 1 -- 172.21.15.38:0/2607759864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ce4107b90 msgr2=0x7f9ce410bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:05.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.674+0000 7f9cea600700 1 --2- 172.21.15.38:0/2607759864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ce4107b90 0x7f9ce410bb30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009910 tx=0x7f9cd0009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:05.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.674+0000 7f9cea600700 1 -- 172.21.15.38:0/2607759864 shutdown_connections 2024-03-24T00:58:05.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.674+0000 7f9cea600700 1 --2- 172.21.15.38:0/2607759864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ce4107b90 0x7f9ce410bb30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.674+0000 7f9cea600700 1 --2- 172.21.15.38:0/2607759864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce41071e0 0x7f9ce41075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.674+0000 7f9cea600700 1 -- 172.21.15.38:0/2607759864 >> 172.21.15.38:0/2607759864 conn(0x7f9ce4076700 msgr2=0x7f9ce4078b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:05.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.674+0000 7f9cea600700 1 -- 172.21.15.38:0/2607759864 shutdown_connections 2024-03-24T00:58:05.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.674+0000 7f9cea600700 1 -- 172.21.15.38:0/2607759864 wait complete. 2024-03-24T00:58:05.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.675+0000 7f9cea600700 1 Processor -- start 2024-03-24T00:58:05.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.675+0000 7f9cea600700 1 -- start start 2024-03-24T00:58:05.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.676+0000 7f9cea600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ce41071e0 0x7f9ce419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:05.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.676+0000 7f9cea600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce4107b90 0x7f9ce419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:05.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.676+0000 7f9cea600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce419db40 con 0x7f9ce41071e0 2024-03-24T00:58:05.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.676+0000 7f9cea600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce41a1810 con 0x7f9ce4107b90 2024-03-24T00:58:05.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.676+0000 7f9ce9200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ce41071e0 0x7f9ce419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:05.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.676+0000 7f9ce9200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ce41071e0 0x7f9ce419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47770/0 (socket says 172.21.15.38:47770) 2024-03-24T00:58:05.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.676+0000 7f9ce9200700 1 -- 172.21.15.38:0/3723275771 learned_addr learned my addr 172.21.15.38:0/3723275771 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:05.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.676+0000 7f9ce3e00700 1 --2- 172.21.15.38:0/3723275771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce4107b90 0x7f9ce419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:05.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.676+0000 7f9ce9200700 1 -- 172.21.15.38:0/3723275771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce4107b90 msgr2=0x7f9ce419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:05.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.676+0000 7f9ce9200700 1 --2- 172.21.15.38:0/3723275771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce4107b90 0x7f9ce419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.676+0000 7f9ce9200700 1 -- 172.21.15.38:0/3723275771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd00095f0 con 0x7f9ce41071e0 2024-03-24T00:58:05.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.677+0000 7f9ce9200700 1 --2- 172.21.15.38:0/3723275771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ce41071e0 0x7f9ce419cf20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800b4b0 tx=0x7f9cd800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.677+0000 7f9ce1600700 1 -- 172.21.15.38:0/3723275771 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cd8011800 con 0x7f9ce41071e0 2024-03-24T00:58:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.677+0000 7f9ce1600700 1 -- 172.21.15.38:0/3723275771 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cd8011e20 con 0x7f9ce41071e0 2024-03-24T00:58:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.677+0000 7f9ce1600700 1 -- 172.21.15.38:0/3723275771 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cd800f2e0 con 0x7f9ce41071e0 2024-03-24T00:58:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.677+0000 7f9cea600700 1 -- 172.21.15.38:0/3723275771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce41a1af0 con 0x7f9ce41071e0 2024-03-24T00:58:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.677+0000 7f9cea600700 1 -- 172.21.15.38:0/3723275771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce41a1ff0 con 0x7f9ce41071e0 2024-03-24T00:58:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.678+0000 7f9cea600700 1 -- 172.21.15.38:0/3723275771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce410f460 con 0x7f9ce41071e0 2024-03-24T00:58:05.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.679+0000 7f9ce1600700 1 -- 172.21.15.38:0/3723275771 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f9cd8010380 con 0x7f9ce41071e0 2024-03-24T00:58:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.680+0000 7f9ce1600700 1 --2- 172.21.15.38:0/3723275771 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ccc06dfb0 0x7f9ccc070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.680+0000 7f9ce1600700 1 -- 172.21.15.38:0/3723275771 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9cd808d900 con 0x7f9ce41071e0 2024-03-24T00:58:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.680+0000 7f9ce3e00700 1 --2- 172.21.15.38:0/3723275771 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ccc06dfb0 0x7f9ccc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.681+0000 7f9ce3e00700 1 --2- 172.21.15.38:0/3723275771 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ccc06dfb0 0x7f9ccc070470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0006010 tx=0x7f9cd000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.685+0000 7f9ce1600700 1 -- 172.21.15.38:0/3723275771 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9cd8056dd0 con 0x7f9ce41071e0 2024-03-24T00:58:05.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.992+0000 7f9cea600700 1 -- 172.21.15.38:0/3723275771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9ce4108610 con 0x7f9ce41071e0 2024-03-24T00:58:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.993+0000 7f9ce1600700 1 -- 172.21.15.38:0/3723275771 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f9cd805a490 con 0x7f9ce41071e0 2024-03-24T00:58:05.994 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_OK 2024-03-24T00:58:05.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.996+0000 7f9cea600700 1 -- 172.21.15.38:0/3723275771 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ccc06dfb0 msgr2=0x7f9ccc070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:05.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.996+0000 7f9cea600700 1 --2- 172.21.15.38:0/3723275771 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ccc06dfb0 0x7f9ccc070470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0006010 tx=0x7f9cd000b540 comp rx=0 tx=0).stop 2024-03-24T00:58:05.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.996+0000 7f9cea600700 1 -- 172.21.15.38:0/3723275771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ce41071e0 msgr2=0x7f9ce419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:05.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.996+0000 7f9cea600700 1 --2- 172.21.15.38:0/3723275771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ce41071e0 0x7f9ce419cf20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800b4b0 tx=0x7f9cd800b870 comp rx=0 tx=0).stop 2024-03-24T00:58:05.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.996+0000 7f9cea600700 1 -- 172.21.15.38:0/3723275771 shutdown_connections 2024-03-24T00:58:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.996+0000 7f9cea600700 1 --2- 172.21.15.38:0/3723275771 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ccc06dfb0 0x7f9ccc070470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.996+0000 7f9cea600700 1 --2- 172.21.15.38:0/3723275771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ce41071e0 0x7f9ce419cf20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.996+0000 7f9cea600700 1 --2- 172.21.15.38:0/3723275771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce4107b90 0x7f9ce419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.996+0000 7f9cea600700 1 -- 172.21.15.38:0/3723275771 >> 172.21.15.38:0/3723275771 conn(0x7f9ce4076700 msgr2=0x7f9ce4077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.996+0000 7f9cea600700 1 -- 172.21.15.38:0/3723275771 shutdown_connections 2024-03-24T00:58:05.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:05.996+0000 7f9cea600700 1 -- 172.21.15.38:0/3723275771 wait complete. 2024-03-24T00:58:06.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:05 smithi038 ceph-mon[112108]: pgmap v147: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:58:06.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:05 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2397346144' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:58:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:05 smithi088 ceph-mon[87351]: pgmap v147: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:58:06.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:05 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2397346144' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:58:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:06 smithi038 ceph-mon[112108]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:07.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:06 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3723275771' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:58:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:06 smithi088 ceph-mon[87351]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:07.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:06 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3723275771' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:58:08.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:07 smithi038 ceph-mon[112108]: pgmap v148: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:08.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:07 smithi088 ceph-mon[87351]: pgmap v148: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:09 smithi038 ceph-mon[112108]: pgmap v149: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:10.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:09 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T00:58:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:09 smithi088 ceph-mon[87351]: pgmap v149: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:09 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T00:58:11.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:10 smithi038 ceph-mon[112108]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T00:58:11.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:10 smithi038 ceph-mon[112108]: Upgrade: osd.4 is safe to restart 2024-03-24T00:58:11.136 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:10 smithi088 ceph-mon[87351]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T00:58:11.136 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:10 smithi088 ceph-mon[87351]: Upgrade: osd.4 is safe to restart 2024-03-24T00:58:11.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:11 smithi088 ceph-mon[87351]: pgmap v150: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:11.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:11.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:58:12.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:11 smithi038 ceph-mon[112108]: pgmap v150: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:12.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:12.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:58:13.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:13 smithi088 ceph-mon[87351]: Upgrade: Updating osd.4 2024-03-24T00:58:13.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:13 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:13.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:13 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T00:58:13.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:13 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:58:13.354 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:13 smithi088 ceph-mon[87351]: Deploying daemon osd.4 on smithi088 2024-03-24T00:58:13.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:13 smithi038 ceph-mon[112108]: Upgrade: Updating osd.4 2024-03-24T00:58:13.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:13 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:13.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:13 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T00:58:13.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:13 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:58:13.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:13 smithi038 ceph-mon[112108]: Deploying daemon osd.4 on smithi088 2024-03-24T00:58:14.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:14 smithi088 ceph-mon[87351]: pgmap v151: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:14.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:14 smithi038 ceph-mon[112108]: pgmap v151: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:16.313 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:16 smithi088 ceph-mon[87351]: pgmap v152: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:58:16.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:16 smithi038 ceph-mon[112108]: pgmap v152: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:58:17.430 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:17 smithi088 systemd[1]: Stopping Ceph osd.4 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:58:17.931 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:17 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[49480]: 2024-03-24T00:58:17.548+0000 7f439c000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T00:58:17.931 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:17 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[49480]: 2024-03-24T00:58:17.548+0000 7f439c000700 -1 osd.4 70 *** Got signal Terminated *** 2024-03-24T00:58:17.931 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:17 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[49480]: 2024-03-24T00:58:17.548+0000 7f439c000700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T00:58:18.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:18 smithi088 ceph-mon[87351]: pgmap v153: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:18.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:18 smithi088 ceph-mon[87351]: osd.4 marked itself down and dead 2024-03-24T00:58:18.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:18 smithi038 ceph-mon[112108]: pgmap v153: 65 pgs: 65 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:18.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:18 smithi038 ceph-mon[112108]: osd.4 marked itself down and dead 2024-03-24T00:58:19.115 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:18 smithi088 podman[101369]: 2024-03-24 00:58:18.825317813 +0000 UTC m=+1.435208726 container died d5aec32ff4592252b8e78bab5ba2d01eacd7f95bb9cd060d2bdee0ad22f3c735 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=reef-738c2fd, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-24T00:58:19.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:19 smithi088 ceph-mon[87351]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:58:19.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:19 smithi088 ceph-mon[87351]: osdmap e71: 6 total, 5 up, 6 in 2024-03-24T00:58:19.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:19 smithi088 ceph-mon[87351]: osdmap e72: 6 total, 5 up, 6 in 2024-03-24T00:58:19.431 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:19 smithi088 podman[101369]: 2024-03-24 00:58:19.258968362 +0000 UTC m=+1.868859329 container cleanup d5aec32ff4592252b8e78bab5ba2d01eacd7f95bb9cd060d2bdee0ad22f3c735 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=reef-738c2fd, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-24T00:58:19.433 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:19 smithi088 bash[101369]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4 2024-03-24T00:58:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:19 smithi038 ceph-mon[112108]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:58:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:19 smithi038 ceph-mon[112108]: osdmap e71: 6 total, 5 up, 6 in 2024-03-24T00:58:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:19 smithi038 ceph-mon[112108]: osdmap e72: 6 total, 5 up, 6 in 2024-03-24T00:58:20.180 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:19 smithi088 podman[101387]: 2024-03-24 00:58:19.810553546 +0000 UTC m=+0.976788811 container remove d5aec32ff4592252b8e78bab5ba2d01eacd7f95bb9cd060d2bdee0ad22f3c735 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-738c2fd, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:58:20.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:20 smithi038 ceph-mon[112108]: pgmap v155: 65 pgs: 6 stale+active+clean, 59 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-24T00:58:20.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:20 smithi088 ceph-mon[87351]: pgmap v155: 65 pgs: 6 stale+active+clean, 59 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-24T00:58:21.066 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:20 smithi088 podman[101473]: 2024-03-24 00:58:20.602931372 +0000 UTC m=+0.034810125 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:58:21.066 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:20 smithi088 podman[101473]: 2024-03-24 00:58:20.745231121 +0000 UTC m=+0.177109922 container create 7687300ca3c716b3eeab2fe87d63959ff86dc2de77f21fc8111d9455ced63d6a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-24T00:58:21.066 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:21 smithi088 podman[101473]: 2024-03-24 00:58:21.065420122 +0000 UTC m=+0.497298906 container init 7687300ca3c716b3eeab2fe87d63959ff86dc2de77f21fc8111d9455ced63d6a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux ) 2024-03-24T00:58:21.430 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:21 smithi088 podman[101473]: 2024-03-24 00:58:21.077891129 +0000 UTC m=+0.509769940 container start 7687300ca3c716b3eeab2fe87d63959ff86dc2de77f21fc8111d9455ced63d6a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-24T00:58:21.430 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:21 smithi088 podman[101473]: 2024-03-24 00:58:21.189541064 +0000 UTC m=+0.621419861 container attach 7687300ca3c716b3eeab2fe87d63959ff86dc2de77f21fc8111d9455ced63d6a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-24T00:58:21.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:21 smithi088 ceph-mon[87351]: Health check failed: Degraded data redundancy: 30/273 objects degraded (10.989%), 8 pgs degraded (PG_DEGRADED) 2024-03-24T00:58:21.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:21 smithi038 ceph-mon[112108]: Health check failed: Degraded data redundancy: 30/273 objects degraded (10.989%), 8 pgs degraded (PG_DEGRADED) 2024-03-24T00:58:21.847 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:21 smithi088 podman[101473]: 2024-03-24 00:58:21.508863811 +0000 UTC m=+0.940742615 container died 7687300ca3c716b3eeab2fe87d63959ff86dc2de77f21fc8111d9455ced63d6a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-24T00:58:22.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:22 smithi038 ceph-mon[112108]: pgmap v157: 65 pgs: 8 active+undersized+degraded, 3 stale+active+clean, 11 active+undersized, 43 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 30/273 objects degraded (10.989%) 2024-03-24T00:58:22.623 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:22 smithi088 ceph-mon[87351]: pgmap v157: 65 pgs: 8 active+undersized+degraded, 3 stale+active+clean, 11 active+undersized, 43 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 30/273 objects degraded (10.989%) 2024-03-24T00:58:22.624 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:22 smithi088 podman[101499]: 2024-03-24 00:58:22.184399139 +0000 UTC m=+0.662938624 container remove 7687300ca3c716b3eeab2fe87d63959ff86dc2de77f21fc8111d9455ced63d6a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-24T00:58:22.624 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:22 smithi088 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-24T00:58:22.624 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:22 smithi088 systemd[1]: Stopped Ceph osd.4 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:58:22.624 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:22 smithi088 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.4.service: Consumed 1min 58.689s CPU time. 2024-03-24T00:58:22.930 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:22 smithi088 systemd[1]: Starting Ceph osd.4 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:58:23.350 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:22 smithi088 podman[101601]: 2024-03-24 00:58:22.872286595 +0000 UTC m=+0.031266306 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:58:23.350 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:23 smithi088 podman[101601]: 2024-03-24 00:58:23.00266719 +0000 UTC m=+0.161646922 container create cac3277202508ee89e38b3dc6f750cb25878e66ba47e7f865443a6eb151b5cf3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-24T00:58:23.350 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:23 smithi088 podman[101601]: 2024-03-24 00:58:23.350036901 +0000 UTC m=+0.509016644 container init cac3277202508ee89e38b3dc6f750cb25878e66ba47e7f865443a6eb151b5cf3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-24T00:58:23.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:23 smithi088 podman[101601]: 2024-03-24 00:58:23.361546352 +0000 UTC m=+0.520526122 container start cac3277202508ee89e38b3dc6f750cb25878e66ba47e7f865443a6eb151b5cf3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-24T00:58:23.681 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:23 smithi088 podman[101601]: 2024-03-24 00:58:23.463596495 +0000 UTC m=+0.622576288 container attach cac3277202508ee89e38b3dc6f750cb25878e66ba47e7f865443a6eb151b5cf3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:58:24.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:24 smithi038 ceph-mon[112108]: pgmap v158: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:24.673 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:24 smithi088 ceph-mon[87351]: pgmap v158: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate[101616]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate[101616]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 bash[101601]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 bash[101601]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 bash[101601]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 bash[101601]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 bash[101601]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 bash[101601]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 bash[101601]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate[101616]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate[101616]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate[101616]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-24T00:58:24.676 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate[101616]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T00:58:24.677 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate[101616]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-24T00:58:24.677 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:24 smithi088 podman[101601]: 2024-03-24 00:58:24.268041211 +0000 UTC m=+1.427020968 container died cac3277202508ee89e38b3dc6f750cb25878e66ba47e7f865443a6eb151b5cf3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:58:25.426 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:25 smithi088 podman[101753]: 2024-03-24 00:58:25.069365722 +0000 UTC m=+0.782941972 container remove cac3277202508ee89e38b3dc6f750cb25878e66ba47e7f865443a6eb151b5cf3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T00:58:25.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:25 smithi088 podman[101814]: 2024-03-24 00:58:25.333452884 +0000 UTC m=+0.033770194 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:58:25.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:25 smithi088 podman[101814]: 2024-03-24 00:58:25.451535744 +0000 UTC m=+0.151853074 container create 9a6f202043cd0328c1ae77b5f33271cda1e20b3e04f04a22555b0f59d9e99c3a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-24T00:58:26.181 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:25 smithi088 podman[101814]: 2024-03-24 00:58:25.788511712 +0000 UTC m=+0.488829054 container init 9a6f202043cd0328c1ae77b5f33271cda1e20b3e04f04a22555b0f59d9e99c3a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-24T00:58:26.182 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:25 smithi088 podman[101814]: 2024-03-24 00:58:25.800452866 +0000 UTC m=+0.500770172 container start 9a6f202043cd0328c1ae77b5f33271cda1e20b3e04f04a22555b0f59d9e99c3a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-24T00:58:26.182 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:25 smithi088 ceph-osd[101833]: -- 172.21.15.88:0/3090078848 <== mon.1 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f92911d680 con 0x55f9290e6c00 2024-03-24T00:58:26.182 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:25 smithi088 bash[101814]: 9a6f202043cd0328c1ae77b5f33271cda1e20b3e04f04a22555b0f59d9e99c3a 2024-03-24T00:58:26.182 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:25 smithi088 systemd[1]: Started Ceph osd.4 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:58:26.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:26 smithi038 ceph-mon[112108]: pgmap v159: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:26.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:26 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:26.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:26 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:26.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:26 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:58:26.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:26 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T00:58:26.376+0000 7f137f3dd7c0 -1 Falling back to public interface 2024-03-24T00:58:26.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:26 smithi088 ceph-mon[87351]: pgmap v159: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:26.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:26 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:26.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:26 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:26.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:26 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:58:27.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:27 smithi088 ceph-mon[87351]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:27.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:27.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:58:27.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:27 smithi088 ceph-mon[87351]: Health check update: Degraded data redundancy: 43/273 objects degraded (15.751%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T00:58:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:27 smithi038 ceph-mon[112108]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:58:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:27 smithi038 ceph-mon[112108]: Health check update: Degraded data redundancy: 43/273 objects degraded (15.751%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T00:58:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:29 smithi088 ceph-mon[87351]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:30.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:29 smithi038 ceph-mon[112108]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:31.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:31 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:31.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:31 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:31.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:31 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:31.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:31 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:32.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:32 smithi038 ceph-mon[112108]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:32.614 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:32 smithi088 ceph-mon[87351]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:32.930 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:32 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T00:58:32.613+0000 7f137f3dd7c0 -1 osd.4 70 log_to_monitors true 2024-03-24T00:58:33.513 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:33 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:33.513 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:33 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:33.513 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:33 smithi088 ceph-mon[87351]: from='osd.4 [v2:172.21.15.88:6808/2757972748,v1:172.21.15.88:6809/2757972748]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T00:58:33.513 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:33 smithi088 ceph-mon[87351]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T00:58:33.513 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 00:58:33 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T00:58:33.379+0000 7f136e800700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T00:58:33.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:33 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:33.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:33 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:33.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:33 smithi038 ceph-mon[112108]: from='osd.4 [v2:172.21.15.88:6808/2757972748,v1:172.21.15.88:6809/2757972748]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T00:58:33.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:33 smithi038 ceph-mon[112108]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T00:58:34.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:34 smithi088 ceph-mon[87351]: pgmap v163: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:34.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:34 smithi088 ceph-mon[87351]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T00:58:34.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:34 smithi088 ceph-mon[87351]: osdmap e73: 6 total, 5 up, 6 in 2024-03-24T00:58:34.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:34 smithi088 ceph-mon[87351]: from='osd.4 [v2:172.21.15.88:6808/2757972748,v1:172.21.15.88:6809/2757972748]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:58:34.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:34 smithi088 ceph-mon[87351]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:58:34.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:34 smithi038 ceph-mon[112108]: pgmap v163: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 958 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:34.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:34 smithi038 ceph-mon[112108]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T00:58:34.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:34 smithi038 ceph-mon[112108]: osdmap e73: 6 total, 5 up, 6 in 2024-03-24T00:58:34.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:34 smithi038 ceph-mon[112108]: from='osd.4 [v2:172.21.15.88:6808/2757972748,v1:172.21.15.88:6809/2757972748]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:58:34.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:34 smithi038 ceph-mon[112108]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:58:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:35 smithi088 ceph-mon[87351]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:58:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:35 smithi088 ceph-mon[87351]: osd.4 [v2:172.21.15.88:6808/2757972748,v1:172.21.15.88:6809/2757972748] boot 2024-03-24T00:58:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:35 smithi088 ceph-mon[87351]: osdmap e74: 6 total, 6 up, 6 in 2024-03-24T00:58:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:35 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:58:35.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:35 smithi088 ceph-mon[87351]: osdmap e75: 6 total, 6 up, 6 in 2024-03-24T00:58:35.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:35 smithi038 ceph-mon[112108]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:58:35.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:35 smithi038 ceph-mon[112108]: osd.4 [v2:172.21.15.88:6808/2757972748,v1:172.21.15.88:6809/2757972748] boot 2024-03-24T00:58:35.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:35 smithi038 ceph-mon[112108]: osdmap e74: 6 total, 6 up, 6 in 2024-03-24T00:58:35.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:35 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T00:58:35.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:35 smithi038 ceph-mon[112108]: osdmap e75: 6 total, 6 up, 6 in 2024-03-24T00:58:36.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.172+0000 7f2516000700 1 -- 172.21.15.38:0/3576270191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2510108b60 msgr2=0x7f2510108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:36.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.172+0000 7f2516000700 1 --2- 172.21.15.38:0/3576270191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2510108b60 0x7f2510108f40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2504009910 tx=0x7f2504009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:36.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.172+0000 7f2516000700 1 -- 172.21.15.38:0/3576270191 shutdown_connections 2024-03-24T00:58:36.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.172+0000 7f2516000700 1 --2- 172.21.15.38:0/3576270191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2510102ad0 0x7f2510102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.172+0000 7f2516000700 1 --2- 172.21.15.38:0/3576270191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2510108b60 0x7f2510108f40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.173+0000 7f2516000700 1 -- 172.21.15.38:0/3576270191 >> 172.21.15.38:0/3576270191 conn(0x7f25100fe5e0 msgr2=0x7f2510100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:36.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.173+0000 7f2516000700 1 -- 172.21.15.38:0/3576270191 shutdown_connections 2024-03-24T00:58:36.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.173+0000 7f2516000700 1 -- 172.21.15.38:0/3576270191 wait complete. 2024-03-24T00:58:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.174+0000 7f2516000700 1 Processor -- start 2024-03-24T00:58:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.174+0000 7f2516000700 1 -- start start 2024-03-24T00:58:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.174+0000 7f2516000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2510102ad0 0x7f251019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.174+0000 7f2516000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2510108b60 0x7f251019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.174+0000 7f2516000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251019f2a0 con 0x7f2510108b60 2024-03-24T00:58:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.174+0000 7f2516000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2510198810 con 0x7f2510102ad0 2024-03-24T00:58:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.175+0000 7f250fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2510108b60 0x7f251019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.175+0000 7f250fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2510108b60 0x7f251019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45268/0 (socket says 172.21.15.38:45268) 2024-03-24T00:58:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.175+0000 7f250fe00700 1 -- 172.21.15.38:0/3517171843 learned_addr learned my addr 172.21.15.38:0/3517171843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:36.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.175+0000 7f2514c00700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2510102ad0 0x7f251019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.175+0000 7f250fe00700 1 -- 172.21.15.38:0/3517171843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2510102ad0 msgr2=0x7f251019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.175+0000 7f250fe00700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2510102ad0 0x7f251019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.175+0000 7f250fe00700 1 -- 172.21.15.38:0/3517171843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25040095f0 con 0x7f2510108b60 2024-03-24T00:58:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.175+0000 7f2514c00700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2510102ad0 0x7f251019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:58:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.176+0000 7f250fe00700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2510108b60 0x7f251019ec10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f250000d8d0 tx=0x7f250000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.176+0000 7f250d600700 1 -- 172.21.15.38:0/3517171843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2500009940 con 0x7f2510108b60 2024-03-24T00:58:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.176+0000 7f250d600700 1 -- 172.21.15.38:0/3517171843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2500010460 con 0x7f2510108b60 2024-03-24T00:58:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.176+0000 7f2516000700 1 -- 172.21.15.38:0/3517171843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2510198af0 con 0x7f2510108b60 2024-03-24T00:58:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.176+0000 7f250d600700 1 -- 172.21.15.38:0/3517171843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f250000f5f0 con 0x7f2510108b60 2024-03-24T00:58:36.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.176+0000 7f2516000700 1 -- 172.21.15.38:0/3517171843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2510199040 con 0x7f2510108b60 2024-03-24T00:58:36.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.177+0000 7f2516000700 1 -- 172.21.15.38:0/3517171843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251010b0d0 con 0x7f2510108b60 2024-03-24T00:58:36.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.178+0000 7f250d600700 1 -- 172.21.15.38:0/3517171843 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f25000105d0 con 0x7f2510108b60 2024-03-24T00:58:36.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.179+0000 7f250d600700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f24f806dfa0 0x7f24f8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:36.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.179+0000 7f250d600700 1 -- 172.21.15.38:0/3517171843 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f250008dc00 con 0x7f2510108b60 2024-03-24T00:58:36.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.180+0000 7f2514c00700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f24f806dfa0 0x7f24f8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:36.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.180+0000 7f2514c00700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f24f806dfa0 0x7f24f8070460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2504005f50 tx=0x7f2504005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:36.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.185+0000 7f250d600700 1 -- 172.21.15.38:0/3517171843 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2500093020 con 0x7f2510108b60 2024-03-24T00:58:36.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.431+0000 7f2516000700 1 -- 172.21.15.38:0/3517171843 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2510199d20 con 0x7f24f806dfa0 2024-03-24T00:58:36.433 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:36 smithi038 ceph-mon[112108]: pgmap v166: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:36.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.434+0000 7f250d600700 1 -- 172.21.15.38:0/3517171843 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f2510199d20 con 0x7f24f806dfa0 2024-03-24T00:58:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.437+0000 7f2516000700 1 -- 172.21.15.38:0/3517171843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f24f806dfa0 msgr2=0x7f24f8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.437+0000 7f2516000700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f24f806dfa0 0x7f24f8070460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2504005f50 tx=0x7f2504005dc0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.437+0000 7f2516000700 1 -- 172.21.15.38:0/3517171843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2510108b60 msgr2=0x7f251019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.437+0000 7f2516000700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2510108b60 0x7f251019ec10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f250000d8d0 tx=0x7f250000dc90 comp rx=0 tx=0).stop 2024-03-24T00:58:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.437+0000 7f2516000700 1 -- 172.21.15.38:0/3517171843 shutdown_connections 2024-03-24T00:58:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.437+0000 7f2516000700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f24f806dfa0 0x7f24f8070460 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.437+0000 7f2516000700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2510102ad0 0x7f251019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.437+0000 7f2516000700 1 --2- 172.21.15.38:0/3517171843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2510108b60 0x7f251019ec10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.438+0000 7f2516000700 1 -- 172.21.15.38:0/3517171843 >> 172.21.15.38:0/3517171843 conn(0x7f25100fe5e0 msgr2=0x7f25100ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.438+0000 7f2516000700 1 -- 172.21.15.38:0/3517171843 shutdown_connections 2024-03-24T00:58:36.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.438+0000 7f2516000700 1 -- 172.21.15.38:0/3517171843 wait complete. 2024-03-24T00:58:36.456 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:58:36.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.604+0000 7f1187800700 1 -- 172.21.15.38:0/3871458772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1180102ef0 msgr2=0x7f11801032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:36.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.604+0000 7f1187800700 1 --2- 172.21.15.38:0/3871458772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1180102ef0 0x7f11801032d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1174009910 tx=0x7f1174009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:36.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.604+0000 7f1187800700 1 -- 172.21.15.38:0/3871458772 shutdown_connections 2024-03-24T00:58:36.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.604+0000 7f1187800700 1 --2- 172.21.15.38:0/3871458772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11801038a0 0x7f11801078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.604+0000 7f1187800700 1 --2- 172.21.15.38:0/3871458772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1180102ef0 0x7f11801032d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.604+0000 7f1187800700 1 -- 172.21.15.38:0/3871458772 >> 172.21.15.38:0/3871458772 conn(0x7f11800fe780 msgr2=0x7f1180100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:36.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.604+0000 7f1187800700 1 -- 172.21.15.38:0/3871458772 shutdown_connections 2024-03-24T00:58:36.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.604+0000 7f1187800700 1 -- 172.21.15.38:0/3871458772 wait complete. 2024-03-24T00:58:36.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.605+0000 7f1187800700 1 Processor -- start 2024-03-24T00:58:36.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.605+0000 7f1187800700 1 -- start start 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.606+0000 7f1187800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1180102ef0 0x7f118019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.606+0000 7f1187800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11801038a0 0x7f118019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.606+0000 7f1187800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118019f570 con 0x7f11801038a0 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.606+0000 7f1187800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1180198ae0 con 0x7f1180102ef0 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.606+0000 7f117fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11801038a0 0x7f118019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.606+0000 7f117fe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11801038a0 0x7f118019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45274/0 (socket says 172.21.15.38:45274) 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.606+0000 7f117fe00700 1 -- 172.21.15.38:0/1486407711 learned_addr learned my addr 172.21.15.38:0/1486407711 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.606+0000 7f1184e00700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1180102ef0 0x7f118019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.607+0000 7f117fe00700 1 -- 172.21.15.38:0/1486407711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1180102ef0 msgr2=0x7f118019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.607+0000 7f117fe00700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1180102ef0 0x7f118019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.607+0000 7f117fe00700 1 -- 172.21.15.38:0/1486407711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11740095f0 con 0x7f11801038a0 2024-03-24T00:58:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.607+0000 7f1184e00700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1180102ef0 0x7f118019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:58:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.607+0000 7f117fe00700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11801038a0 0x7f118019eee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f116c00d900 tx=0x7f116c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.607+0000 7f117d600700 1 -- 172.21.15.38:0/1486407711 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f116c009900 con 0x7f11801038a0 2024-03-24T00:58:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.607+0000 7f117d600700 1 -- 172.21.15.38:0/1486407711 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f116c010460 con 0x7f11801038a0 2024-03-24T00:58:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.607+0000 7f1187800700 1 -- 172.21.15.38:0/1486407711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1180198dc0 con 0x7f11801038a0 2024-03-24T00:58:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.607+0000 7f117d600700 1 -- 172.21.15.38:0/1486407711 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f116c00f5f0 con 0x7f11801038a0 2024-03-24T00:58:36.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.608+0000 7f1187800700 1 -- 172.21.15.38:0/1486407711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1180199310 con 0x7f11801038a0 2024-03-24T00:58:36.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.609+0000 7f1187800700 1 -- 172.21.15.38:0/1486407711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118010b290 con 0x7f11801038a0 2024-03-24T00:58:36.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.610+0000 7f117d600700 1 -- 172.21.15.38:0/1486407711 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f116c0105d0 con 0x7f11801038a0 2024-03-24T00:58:36.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.611+0000 7f117d600700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f116806dfb0 0x7f1168070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:36.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.611+0000 7f117d600700 1 -- 172.21.15.38:0/1486407711 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f116c08db10 con 0x7f11801038a0 2024-03-24T00:58:36.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.611+0000 7f1184e00700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f116806dfb0 0x7f1168070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:36.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.612+0000 7f1184e00700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f116806dfb0 0x7f1168070470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1174005b40 tx=0x7f117401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:36.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.616+0000 7f117d600700 1 -- 172.21.15.38:0/1486407711 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f116c05aaf0 con 0x7f11801038a0 2024-03-24T00:58:36.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:36 smithi088 ceph-mon[87351]: pgmap v166: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 43/273 objects degraded (15.751%) 2024-03-24T00:58:36.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.858+0000 7f1187800700 1 -- 172.21.15.38:0/1486407711 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1180199580 con 0x7f116806dfb0 2024-03-24T00:58:36.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.861+0000 7f117d600700 1 -- 172.21.15.38:0/1486407711 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f1180199580 con 0x7f116806dfb0 2024-03-24T00:58:36.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.864+0000 7f1187800700 1 -- 172.21.15.38:0/1486407711 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f116806dfb0 msgr2=0x7f1168070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:36.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.864+0000 7f1187800700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f116806dfb0 0x7f1168070470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1174005b40 tx=0x7f117401a040 comp rx=0 tx=0).stop 2024-03-24T00:58:36.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.864+0000 7f1187800700 1 -- 172.21.15.38:0/1486407711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11801038a0 msgr2=0x7f118019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:36.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.864+0000 7f1187800700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11801038a0 0x7f118019eee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f116c00d900 tx=0x7f116c00dcc0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.864+0000 7f1187800700 1 -- 172.21.15.38:0/1486407711 shutdown_connections 2024-03-24T00:58:36.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.864+0000 7f1187800700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f116806dfb0 0x7f1168070470 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.864+0000 7f1187800700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1180102ef0 0x7f118019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.864+0000 7f1187800700 1 --2- 172.21.15.38:0/1486407711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11801038a0 0x7f118019eee0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:36.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.864+0000 7f1187800700 1 -- 172.21.15.38:0/1486407711 >> 172.21.15.38:0/1486407711 conn(0x7f11800fe780 msgr2=0x7f11800ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:36.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.864+0000 7f1187800700 1 -- 172.21.15.38:0/1486407711 shutdown_connections 2024-03-24T00:58:36.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:36.864+0000 7f1187800700 1 -- 172.21.15.38:0/1486407711 wait complete. 2024-03-24T00:58:37.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.015+0000 7efd9f800700 1 -- 172.21.15.38:0/3738537229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd98102530 msgr2=0x7efd9810aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:37.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.015+0000 7efd9f800700 1 --2- 172.21.15.38:0/3738537229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd98102530 0x7efd9810aa20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efd84009910 tx=0x7efd84009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:37.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.016+0000 7efd9f800700 1 -- 172.21.15.38:0/3738537229 shutdown_connections 2024-03-24T00:58:37.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.016+0000 7efd9f800700 1 --2- 172.21.15.38:0/3738537229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd98102530 0x7efd9810aa20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.016+0000 7efd9f800700 1 --2- 172.21.15.38:0/3738537229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd98101c10 0x7efd98101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.016+0000 7efd9f800700 1 -- 172.21.15.38:0/3738537229 >> 172.21.15.38:0/3738537229 conn(0x7efd98076640 msgr2=0x7efd980fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:37.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.016+0000 7efd9f800700 1 -- 172.21.15.38:0/3738537229 shutdown_connections 2024-03-24T00:58:37.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.016+0000 7efd9f800700 1 -- 172.21.15.38:0/3738537229 wait complete. 2024-03-24T00:58:37.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.017+0000 7efd9f800700 1 Processor -- start 2024-03-24T00:58:37.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.017+0000 7efd9f800700 1 -- start start 2024-03-24T00:58:37.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.018+0000 7efd9f800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd98101c10 0x7efd980ff870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:37.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.018+0000 7efd9f800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd98102530 0x7efd980ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:37.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.018+0000 7efd9f800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd98101840 con 0x7efd98101c10 2024-03-24T00:58:37.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.018+0000 7efd9f800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd981002f0 con 0x7efd98102530 2024-03-24T00:58:37.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.018+0000 7efd9ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd98101c10 0x7efd980ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:37.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.018+0000 7efd9ce00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd98101c10 0x7efd980ff870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45298/0 (socket says 172.21.15.38:45298) 2024-03-24T00:58:37.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.018+0000 7efd9ce00700 1 -- 172.21.15.38:0/2410747732 learned_addr learned my addr 172.21.15.38:0/2410747732 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:37.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.018+0000 7efd97e00700 1 --2- 172.21.15.38:0/2410747732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd98102530 0x7efd980ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:37.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.019+0000 7efd9ce00700 1 -- 172.21.15.38:0/2410747732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd98102530 msgr2=0x7efd980ffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:37.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.019+0000 7efd9ce00700 1 --2- 172.21.15.38:0/2410747732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd98102530 0x7efd980ffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.019+0000 7efd9ce00700 1 -- 172.21.15.38:0/2410747732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd840095f0 con 0x7efd98101c10 2024-03-24T00:58:37.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.019+0000 7efd9ce00700 1 --2- 172.21.15.38:0/2410747732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd98101c10 0x7efd980ff870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c930 tx=0x7efd8c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:37.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.019+0000 7efd95600700 1 -- 172.21.15.38:0/2410747732 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd8c007ad0 con 0x7efd98101c10 2024-03-24T00:58:37.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.019+0000 7efd95600700 1 -- 172.21.15.38:0/2410747732 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd8c007c30 con 0x7efd98101c10 2024-03-24T00:58:37.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.019+0000 7efd9f800700 1 -- 172.21.15.38:0/2410747732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd981005a0 con 0x7efd98101c10 2024-03-24T00:58:37.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.019+0000 7efd95600700 1 -- 172.21.15.38:0/2410747732 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd8c018610 con 0x7efd98101c10 2024-03-24T00:58:37.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.019+0000 7efd9f800700 1 -- 172.21.15.38:0/2410747732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd981a2e30 con 0x7efd98101c10 2024-03-24T00:58:37.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.021+0000 7efd9f800700 1 -- 172.21.15.38:0/2410747732 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd98068d60 con 0x7efd98101c10 2024-03-24T00:58:37.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.022+0000 7efd95600700 1 -- 172.21.15.38:0/2410747732 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7efd8c00f450 con 0x7efd98101c10 2024-03-24T00:58:37.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.023+0000 7efd95600700 1 --2- 172.21.15.38:0/2410747732 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efd8006dfa0 0x7efd80070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:37.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.023+0000 7efd95600700 1 -- 172.21.15.38:0/2410747732 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efd8c08df20 con 0x7efd98101c10 2024-03-24T00:58:37.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.023+0000 7efd97e00700 1 --2- 172.21.15.38:0/2410747732 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efd8006dfa0 0x7efd80070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:37.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.024+0000 7efd97e00700 1 --2- 172.21.15.38:0/2410747732 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efd8006dfa0 0x7efd80070460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd98100fa0 tx=0x7efd8401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:37.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.029+0000 7efd95600700 1 -- 172.21.15.38:0/2410747732 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efd8c0573d0 con 0x7efd98101c10 2024-03-24T00:58:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.261+0000 7efd9f800700 1 -- 172.21.15.38:0/2410747732 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efd980611f0 con 0x7efd8006dfa0 2024-03-24T00:58:37.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.292+0000 7efd95600700 1 -- 172.21.15.38:0/2410747732 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7efd980611f0 con 0x7efd8006dfa0 2024-03-24T00:58:37.294 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:58:37.294 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (6m) 78s ago 20m 13.5M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:58:37.294 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (21m) 78s ago 21m 9068k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (19m) 7s ago 19m 8816k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (3m) 78s ago 21m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (3m) 7s ago 19m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (6m) 78s ago 20m 75.3M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (13m) 78s ago 13m 97.9M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (13m) 78s ago 13m 150M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (13m) 7s ago 13m 24.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (13m) 7s ago 13m 24.9M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (9m) 78s ago 22m 502M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (8m) 7s ago 19m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (4m) 78s ago 23m 48.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (4m) 7s ago 19m 43.3M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (8m) 78s ago 21m 7755k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (8m) 7s ago 19m 7675k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:58:37.295 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (2m) 78s ago 18m 97.1M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T00:58:37.296 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (118s) 78s ago 17m 104M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cb72ac446ba7 2024-03-24T00:58:37.296 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (87s) 78s ago 17m 11.2M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c73f3e8276b6 2024-03-24T00:58:37.296 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (55s) 7s ago 16m 108M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b2afaff761d8 2024-03-24T00:58:37.296 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (11s) 7s ago 16m 11.5M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9a6f202043cd 2024-03-24T00:58:37.296 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (15m) 7s ago 15m 249M 2241M 18.2.2-952-g738c2fd6 68aff9aff3b2 99e27e238750 2024-03-24T00:58:37.296 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (7m) 78s ago 20m 70.6M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:58:37.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.296+0000 7efd9f800700 1 -- 172.21.15.38:0/2410747732 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efd8006dfa0 msgr2=0x7efd80070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:37.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.296+0000 7efd9f800700 1 --2- 172.21.15.38:0/2410747732 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efd8006dfa0 0x7efd80070460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd98100fa0 tx=0x7efd8401a040 comp rx=0 tx=0).stop 2024-03-24T00:58:37.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.296+0000 7efd9f800700 1 -- 172.21.15.38:0/2410747732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd98101c10 msgr2=0x7efd980ff870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:37.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.296+0000 7efd9f800700 1 --2- 172.21.15.38:0/2410747732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd98101c10 0x7efd980ff870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c930 tx=0x7efd8c00ccf0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.296+0000 7efd9f800700 1 -- 172.21.15.38:0/2410747732 shutdown_connections 2024-03-24T00:58:37.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.296+0000 7efd9f800700 1 --2- 172.21.15.38:0/2410747732 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efd8006dfa0 0x7efd80070460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.296+0000 7efd9f800700 1 --2- 172.21.15.38:0/2410747732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd98101c10 0x7efd980ff870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.296+0000 7efd9f800700 1 --2- 172.21.15.38:0/2410747732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd98102530 0x7efd980ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.296+0000 7efd9f800700 1 -- 172.21.15.38:0/2410747732 >> 172.21.15.38:0/2410747732 conn(0x7efd98076640 msgr2=0x7efd98102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:37.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.297+0000 7efd9f800700 1 -- 172.21.15.38:0/2410747732 shutdown_connections 2024-03-24T00:58:37.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.297+0000 7efd9f800700 1 -- 172.21.15.38:0/2410747732 wait complete. 2024-03-24T00:58:37.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.457+0000 7f257d400700 1 -- 172.21.15.38:0/4292098018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2578073150 msgr2=0x7f2578073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:37.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.457+0000 7f257d400700 1 --2- 172.21.15.38:0/4292098018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2578073150 0x7f2578073530 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2568009910 tx=0x7f2568009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:37.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.457+0000 7f257d400700 1 -- 172.21.15.38:0/4292098018 shutdown_connections 2024-03-24T00:58:37.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.457+0000 7f257d400700 1 --2- 172.21.15.38:0/4292098018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2578073a70 0x7f2578111560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.457+0000 7f257d400700 1 --2- 172.21.15.38:0/4292098018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2578073150 0x7f2578073530 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.457+0000 7f257d400700 1 -- 172.21.15.38:0/4292098018 >> 172.21.15.38:0/4292098018 conn(0x7f25780fc5a0 msgr2=0x7f25780fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:37.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.457+0000 7f257d400700 1 -- 172.21.15.38:0/4292098018 shutdown_connections 2024-03-24T00:58:37.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.457+0000 7f257d400700 1 -- 172.21.15.38:0/4292098018 wait complete. 2024-03-24T00:58:37.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.458+0000 7f257d400700 1 Processor -- start 2024-03-24T00:58:37.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.459+0000 7f257d400700 1 -- start start 2024-03-24T00:58:37.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.459+0000 7f257d400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2578073150 0x7f257819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:37.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.459+0000 7f257d400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2578073a70 0x7f257819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:37.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.459+0000 7f257d400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257819db60 con 0x7f2578073150 2024-03-24T00:58:37.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.459+0000 7f257d400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25781a1830 con 0x7f2578073a70 2024-03-24T00:58:37.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.459+0000 7f2576800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2578073150 0x7f257819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.459+0000 7f2576800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2578073150 0x7f257819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45330/0 (socket says 172.21.15.38:45330) 2024-03-24T00:58:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.459+0000 7f2576800700 1 -- 172.21.15.38:0/1538220335 learned_addr learned my addr 172.21.15.38:0/1538220335 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.460+0000 7f2575e00700 1 --2- 172.21.15.38:0/1538220335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2578073a70 0x7f257819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.460+0000 7f2576800700 1 -- 172.21.15.38:0/1538220335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2578073a70 msgr2=0x7f257819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.460+0000 7f2576800700 1 --2- 172.21.15.38:0/1538220335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2578073a70 0x7f257819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.460+0000 7f2576800700 1 -- 172.21.15.38:0/1538220335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25680095f0 con 0x7f2578073150 2024-03-24T00:58:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.460+0000 7f2576800700 1 --2- 172.21.15.38:0/1538220335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2578073150 0x7f257819cf40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2568006010 tx=0x7f256800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.460+0000 7f256f400700 1 -- 172.21.15.38:0/1538220335 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256801d070 con 0x7f2578073150 2024-03-24T00:58:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.460+0000 7f256f400700 1 -- 172.21.15.38:0/1538220335 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25680056e0 con 0x7f2578073150 2024-03-24T00:58:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.460+0000 7f256f400700 1 -- 172.21.15.38:0/1538220335 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2568017400 con 0x7f2578073150 2024-03-24T00:58:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.460+0000 7f257d400700 1 -- 172.21.15.38:0/1538220335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25781a1b10 con 0x7f2578073150 2024-03-24T00:58:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.461+0000 7f257d400700 1 -- 172.21.15.38:0/1538220335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25781a2060 con 0x7f2578073150 2024-03-24T00:58:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.462+0000 7f257d400700 1 -- 172.21.15.38:0/1538220335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257810ece0 con 0x7f2578073150 2024-03-24T00:58:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.463+0000 7f256f400700 1 -- 172.21.15.38:0/1538220335 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f256800f6f0 con 0x7f2578073150 2024-03-24T00:58:37.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.464+0000 7f256f400700 1 --2- 172.21.15.38:0/1538220335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f255c06dfa0 0x7f255c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:37.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.464+0000 7f256f400700 1 -- 172.21.15.38:0/1538220335 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f256808efe0 con 0x7f2578073150 2024-03-24T00:58:37.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.464+0000 7f2575e00700 1 --2- 172.21.15.38:0/1538220335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f255c06dfa0 0x7f255c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:37.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.465+0000 7f2575e00700 1 --2- 172.21.15.38:0/1538220335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f255c06dfa0 0x7f255c070460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2564005950 tx=0x7f25640058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:37.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.469+0000 7f256f400700 1 -- 172.21.15.38:0/1538220335 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f25680584d0 con 0x7f2578073150 2024-03-24T00:58:37.556 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:37 smithi038 ceph-mon[112108]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:37.622 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:37 smithi088 ceph-mon[87351]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.789+0000 7f257d400700 1 -- 172.21.15.38:0/1538220335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f25781a22d0 con 0x7f2578073150 2024-03-24T00:58:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.791+0000 7f256f400700 1 -- 172.21.15.38:0/1538220335 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f256805bb90 con 0x7f2578073150 2024-03-24T00:58:37.792 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:58:37.792 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:58:37.792 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:58:37.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 1, 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 5 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 5, 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 9 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:58:37.793 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:58:37.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.794+0000 7f257d400700 1 -- 172.21.15.38:0/1538220335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f255c06dfa0 msgr2=0x7f255c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:37.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.794+0000 7f257d400700 1 --2- 172.21.15.38:0/1538220335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f255c06dfa0 0x7f255c070460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2564005950 tx=0x7f25640058e0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.794+0000 7f257d400700 1 -- 172.21.15.38:0/1538220335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2578073150 msgr2=0x7f257819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:37.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.794+0000 7f257d400700 1 --2- 172.21.15.38:0/1538220335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2578073150 0x7f257819cf40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2568006010 tx=0x7f256800be20 comp rx=0 tx=0).stop 2024-03-24T00:58:37.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.794+0000 7f257d400700 1 -- 172.21.15.38:0/1538220335 shutdown_connections 2024-03-24T00:58:37.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.794+0000 7f257d400700 1 --2- 172.21.15.38:0/1538220335 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f255c06dfa0 0x7f255c070460 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.794+0000 7f257d400700 1 --2- 172.21.15.38:0/1538220335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2578073150 0x7f257819cf40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.794+0000 7f257d400700 1 --2- 172.21.15.38:0/1538220335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2578073a70 0x7f257819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.794+0000 7f257d400700 1 -- 172.21.15.38:0/1538220335 >> 172.21.15.38:0/1538220335 conn(0x7f25780fc5a0 msgr2=0x7f2578104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:37.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.794+0000 7f257d400700 1 -- 172.21.15.38:0/1538220335 shutdown_connections 2024-03-24T00:58:37.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.795+0000 7f257d400700 1 -- 172.21.15.38:0/1538220335 wait complete. 2024-03-24T00:58:37.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.941+0000 7ff33d400700 1 -- 172.21.15.38:0/2864647926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381038a0 msgr2=0x7ff3381078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:37.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.941+0000 7ff33d400700 1 --2- 172.21.15.38:0/2864647926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381038a0 0x7ff3381078f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff320009910 tx=0x7ff320009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:37.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.941+0000 7ff33d400700 1 -- 172.21.15.38:0/2864647926 shutdown_connections 2024-03-24T00:58:37.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.941+0000 7ff33d400700 1 --2- 172.21.15.38:0/2864647926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381038a0 0x7ff3381078f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.941+0000 7ff33d400700 1 --2- 172.21.15.38:0/2864647926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff338102ef0 0x7ff3381032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.941+0000 7ff33d400700 1 -- 172.21.15.38:0/2864647926 >> 172.21.15.38:0/2864647926 conn(0x7ff3380fe780 msgr2=0x7ff338100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:37.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.941+0000 7ff33d400700 1 -- 172.21.15.38:0/2864647926 shutdown_connections 2024-03-24T00:58:37.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.941+0000 7ff33d400700 1 -- 172.21.15.38:0/2864647926 wait complete. 2024-03-24T00:58:37.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.942+0000 7ff33d400700 1 Processor -- start 2024-03-24T00:58:37.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.942+0000 7ff33d400700 1 -- start start 2024-03-24T00:58:37.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.943+0000 7ff33d400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff338102ef0 0x7ff33819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:37.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.943+0000 7ff33d400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3381038a0 0x7ff33819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:37.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.943+0000 7ff33d400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff338198ab0 con 0x7ff338102ef0 2024-03-24T00:58:37.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.943+0000 7ff33d400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff338198c20 con 0x7ff3381038a0 2024-03-24T00:58:37.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.943+0000 7ff336800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff338102ef0 0x7ff33819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.943+0000 7ff336800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff338102ef0 0x7ff33819ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45346/0 (socket says 172.21.15.38:45346) 2024-03-24T00:58:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.943+0000 7ff336800700 1 -- 172.21.15.38:0/3841644587 learned_addr learned my addr 172.21.15.38:0/3841644587 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.943+0000 7ff32de00700 1 --2- 172.21.15.38:0/3841644587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3381038a0 0x7ff33819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:37.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.944+0000 7ff336800700 1 -- 172.21.15.38:0/3841644587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3381038a0 msgr2=0x7ff33819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:37.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.944+0000 7ff336800700 1 --2- 172.21.15.38:0/3841644587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3381038a0 0x7ff33819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:37.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.944+0000 7ff336800700 1 -- 172.21.15.38:0/3841644587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3200095f0 con 0x7ff338102ef0 2024-03-24T00:58:37.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.944+0000 7ff336800700 1 --2- 172.21.15.38:0/3841644587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff338102ef0 0x7ff33819ea00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff32800ef10 tx=0x7ff32800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:37.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.944+0000 7ff32fe00700 1 -- 172.21.15.38:0/3841644587 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32800c9f0 con 0x7ff338102ef0 2024-03-24T00:58:37.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.944+0000 7ff32fe00700 1 -- 172.21.15.38:0/3841644587 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff32800f040 con 0x7ff338102ef0 2024-03-24T00:58:37.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.944+0000 7ff32fe00700 1 -- 172.21.15.38:0/3841644587 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3280146b0 con 0x7ff338102ef0 2024-03-24T00:58:37.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.944+0000 7ff33d400700 1 -- 172.21.15.38:0/3841644587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff338198ea0 con 0x7ff338102ef0 2024-03-24T00:58:37.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.944+0000 7ff33d400700 1 -- 172.21.15.38:0/3841644587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3381993c0 con 0x7ff338102ef0 2024-03-24T00:58:37.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.946+0000 7ff33d400700 1 -- 172.21.15.38:0/3841644587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33810b2f0 con 0x7ff338102ef0 2024-03-24T00:58:37.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.947+0000 7ff32fe00700 1 -- 172.21.15.38:0/3841644587 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7ff32800cb50 con 0x7ff338102ef0 2024-03-24T00:58:37.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.947+0000 7ff32fe00700 1 --2- 172.21.15.38:0/3841644587 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff31c06dfb0 0x7ff31c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:37.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.948+0000 7ff32fe00700 1 -- 172.21.15.38:0/3841644587 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff32808dff0 con 0x7ff338102ef0 2024-03-24T00:58:37.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.948+0000 7ff32de00700 1 --2- 172.21.15.38:0/3841644587 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff31c06dfb0 0x7ff31c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:37.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.949+0000 7ff32de00700 1 --2- 172.21.15.38:0/3841644587 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff31c06dfb0 0x7ff31c070470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff320005b40 tx=0x7ff320005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:37.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:37.953+0000 7ff32fe00700 1 -- 172.21.15.38:0/3841644587 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff328057500 con 0x7ff338102ef0 2024-03-24T00:58:38.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.240+0000 7ff33d400700 1 -- 172.21.15.38:0/3841644587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff338199f90 con 0x7ff338102ef0 2024-03-24T00:58:38.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.240+0000 7ff32fe00700 1 -- 172.21.15.38:0/3841644587 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1900 (secure 0 0 0) 0x7ff328007e60 con 0x7ff338102ef0 2024-03-24T00:58:38.242 INFO:teuthology.orchestra.run.smithi038.stdout:e20 2024-03-24T00:58:38.242 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:58:38.242 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 20 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:58:38.243 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:58:38.244 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:58:38.245 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:58:38.245 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T00:58:38.245 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:58:38.245 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:58:38.245 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:58:38.245 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:58:38.245 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:58:38.245 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:58:38.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.244+0000 7ff33d400700 1 -- 172.21.15.38:0/3841644587 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff31c06dfb0 msgr2=0x7ff31c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:38.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.244+0000 7ff33d400700 1 --2- 172.21.15.38:0/3841644587 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff31c06dfb0 0x7ff31c070470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff320005b40 tx=0x7ff320005a90 comp rx=0 tx=0).stop 2024-03-24T00:58:38.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.244+0000 7ff33d400700 1 -- 172.21.15.38:0/3841644587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff338102ef0 msgr2=0x7ff33819ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:38.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.244+0000 7ff33d400700 1 --2- 172.21.15.38:0/3841644587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff338102ef0 0x7ff33819ea00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff32800ef10 tx=0x7ff32800c3a0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.244+0000 7ff33d400700 1 -- 172.21.15.38:0/3841644587 shutdown_connections 2024-03-24T00:58:38.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.244+0000 7ff33d400700 1 --2- 172.21.15.38:0/3841644587 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff31c06dfb0 0x7ff31c070470 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.244+0000 7ff33d400700 1 --2- 172.21.15.38:0/3841644587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff338102ef0 0x7ff33819ea00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.244+0000 7ff33d400700 1 --2- 172.21.15.38:0/3841644587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3381038a0 0x7ff33819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.244+0000 7ff33d400700 1 -- 172.21.15.38:0/3841644587 >> 172.21.15.38:0/3841644587 conn(0x7ff3380fe780 msgr2=0x7ff3380ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:38.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.244+0000 7ff33d400700 1 -- 172.21.15.38:0/3841644587 shutdown_connections 2024-03-24T00:58:38.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.244+0000 7ff33d400700 1 -- 172.21.15.38:0/3841644587 wait complete. 2024-03-24T00:58:38.247 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:58:38.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.396+0000 7f9d63000700 1 -- 172.21.15.38:0/3432387950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d5c107310 msgr2=0x7f9d5c1076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:38.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.396+0000 7f9d63000700 1 --2- 172.21.15.38:0/3432387950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d5c107310 0x7f9d5c1076f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009910 tx=0x7f9d4c009c20 comp rx=0 tx=0).stop 2024-03-24T00:58:38.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.396+0000 7f9d63000700 1 -- 172.21.15.38:0/3432387950 shutdown_connections 2024-03-24T00:58:38.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.396+0000 7f9d63000700 1 --2- 172.21.15.38:0/3432387950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d5c107cc0 0x7f9d5c10bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.396+0000 7f9d63000700 1 --2- 172.21.15.38:0/3432387950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d5c107310 0x7f9d5c1076f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.396+0000 7f9d63000700 1 -- 172.21.15.38:0/3432387950 >> 172.21.15.38:0/3432387950 conn(0x7f9d5c0768e0 msgr2=0x7f9d5c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:38.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.396+0000 7f9d63000700 1 -- 172.21.15.38:0/3432387950 shutdown_connections 2024-03-24T00:58:38.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.396+0000 7f9d63000700 1 -- 172.21.15.38:0/3432387950 wait complete. 2024-03-24T00:58:38.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.397+0000 7f9d63000700 1 Processor -- start 2024-03-24T00:58:38.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.397+0000 7f9d63000700 1 -- start start 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.397+0000 7f9d63000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d5c107310 0x7f9d5c19d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.398+0000 7f9d63000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d5c107cc0 0x7f9d5c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.398+0000 7f9d63000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c19dd20 con 0x7f9d5c107cc0 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.398+0000 7f9d63000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c1a19a0 con 0x7f9d5c107310 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.398+0000 7f9d53e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d5c107cc0 0x7f9d5c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.398+0000 7f9d53e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d5c107cc0 0x7f9d5c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45374/0 (socket says 172.21.15.38:45374) 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.398+0000 7f9d53e00700 1 -- 172.21.15.38:0/1125323884 learned_addr learned my addr 172.21.15.38:0/1125323884 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.398+0000 7f9d5be00700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d5c107310 0x7f9d5c19d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.398+0000 7f9d53e00700 1 -- 172.21.15.38:0/1125323884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d5c107310 msgr2=0x7f9d5c19d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.398+0000 7f9d53e00700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d5c107310 0x7f9d5c19d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.398+0000 7f9d53e00700 1 -- 172.21.15.38:0/1125323884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c0095f0 con 0x7f9d5c107cc0 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.398+0000 7f9d5be00700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d5c107310 0x7f9d5c19d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:58:38.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.399+0000 7f9d53e00700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d5c107cc0 0x7f9d5c19d640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400b480 tx=0x7f9d4400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:38.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.399+0000 7f9d59600700 1 -- 172.21.15.38:0/1125323884 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d440118c0 con 0x7f9d5c107cc0 2024-03-24T00:58:38.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.399+0000 7f9d59600700 1 -- 172.21.15.38:0/1125323884 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d44010380 con 0x7f9d5c107cc0 2024-03-24T00:58:38.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.399+0000 7f9d59600700 1 -- 172.21.15.38:0/1125323884 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4400f4f0 con 0x7f9d5c107cc0 2024-03-24T00:58:38.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.399+0000 7f9d63000700 1 -- 172.21.15.38:0/1125323884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5c1a1c80 con 0x7f9d5c107cc0 2024-03-24T00:58:38.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.399+0000 7f9d63000700 1 -- 172.21.15.38:0/1125323884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c1a2180 con 0x7f9d5c107cc0 2024-03-24T00:58:38.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.400+0000 7f9d63000700 1 -- 172.21.15.38:0/1125323884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5c10f6b0 con 0x7f9d5c107cc0 2024-03-24T00:58:38.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.401+0000 7f9d59600700 1 -- 172.21.15.38:0/1125323884 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f9d440104f0 con 0x7f9d5c107cc0 2024-03-24T00:58:38.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.402+0000 7f9d59600700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9d4006dfa0 0x7f9d40070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:38.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.402+0000 7f9d59600700 1 -- 172.21.15.38:0/1125323884 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9d4408d9b0 con 0x7f9d5c107cc0 2024-03-24T00:58:38.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.402+0000 7f9d5be00700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9d4006dfa0 0x7f9d40070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:38.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.403+0000 7f9d5be00700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9d4006dfa0 0x7f9d40070460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c005f50 tx=0x7f9d4c005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:38.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.407+0000 7f9d59600700 1 -- 172.21.15.38:0/1125323884 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9d44056de0 con 0x7f9d5c107cc0 2024-03-24T00:58:38.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: pgmap v168: 65 pgs: 10 peering, 7 active+undersized+degraded, 11 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 15/273 objects degraded (5.495%) 2024-03-24T00:58:38.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:38.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:38.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: Health check update: Degraded data redundancy: 15/273 objects degraded (5.495%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T00:58:38.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:38.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:38.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:58:38.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:58:38.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:38.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1538220335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:58:38.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:58:38.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:58:38.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:58:38.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:58:38.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T00:58:38.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:38 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3841644587' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:58:38.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.657+0000 7f9d63000700 1 -- 172.21.15.38:0/1125323884 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d5c1a2450 con 0x7f9d4006dfa0 2024-03-24T00:58:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.663+0000 7f9d59600700 1 -- 172.21.15.38:0/1125323884 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9d5c1a2450 con 0x7f9d4006dfa0 2024-03-24T00:58:38.667 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:58:38.667 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:58:38.667 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:58:38.667 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:58:38.667 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:58:38.667 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T00:58:38.667 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-03-24T00:58:38.667 INFO:teuthology.orchestra.run.smithi038.stdout: "crash" 2024-03-24T00:58:38.667 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:58:38.667 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "11/23 daemons upgraded", 2024-03-24T00:58:38.668 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T00:58:38.668 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:58:38.668 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:58:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.668+0000 7f9d63000700 1 -- 172.21.15.38:0/1125323884 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9d4006dfa0 msgr2=0x7f9d40070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.668+0000 7f9d63000700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9d4006dfa0 0x7f9d40070460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c005f50 tx=0x7f9d4c005dc0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.668+0000 7f9d63000700 1 -- 172.21.15.38:0/1125323884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d5c107cc0 msgr2=0x7f9d5c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.668+0000 7f9d63000700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d5c107cc0 0x7f9d5c19d640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400b480 tx=0x7f9d4400b840 comp rx=0 tx=0).stop 2024-03-24T00:58:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.668+0000 7f9d63000700 1 -- 172.21.15.38:0/1125323884 shutdown_connections 2024-03-24T00:58:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.668+0000 7f9d63000700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9d4006dfa0 0x7f9d40070460 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.668+0000 7f9d63000700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d5c107310 0x7f9d5c19d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.668+0000 7f9d63000700 1 --2- 172.21.15.38:0/1125323884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d5c107cc0 0x7f9d5c19d640 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.668+0000 7f9d63000700 1 -- 172.21.15.38:0/1125323884 >> 172.21.15.38:0/1125323884 conn(0x7f9d5c0768e0 msgr2=0x7f9d5c077c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.669+0000 7f9d63000700 1 -- 172.21.15.38:0/1125323884 shutdown_connections 2024-03-24T00:58:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.669+0000 7f9d63000700 1 -- 172.21.15.38:0/1125323884 wait complete. 2024-03-24T00:58:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: pgmap v168: 65 pgs: 10 peering, 7 active+undersized+degraded, 11 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 15/273 objects degraded (5.495%) 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: Health check update: Degraded data redundancy: 15/273 objects degraded (5.495%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1538220335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:58:38.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T00:58:38.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:38 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3841644587' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:58:38.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.826+0000 7fa272000700 1 -- 172.21.15.38:0/208101625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa26c1068e0 msgr2=0x7fa26c106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:38.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.826+0000 7fa272000700 1 --2- 172.21.15.38:0/208101625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa26c1068e0 0x7fa26c106cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa26000b0a0 tx=0x7fa26000b3b0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.827+0000 7fa272000700 1 -- 172.21.15.38:0/208101625 shutdown_connections 2024-03-24T00:58:38.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.827+0000 7fa272000700 1 --2- 172.21.15.38:0/208101625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa26c1008e0 0x7fa26c100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.827+0000 7fa272000700 1 --2- 172.21.15.38:0/208101625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa26c1068e0 0x7fa26c106cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.827+0000 7fa272000700 1 -- 172.21.15.38:0/208101625 >> 172.21.15.38:0/208101625 conn(0x7fa26c0fc3f0 msgr2=0x7fa26c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:38.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.827+0000 7fa272000700 1 -- 172.21.15.38:0/208101625 shutdown_connections 2024-03-24T00:58:38.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.827+0000 7fa272000700 1 -- 172.21.15.38:0/208101625 wait complete. 2024-03-24T00:58:38.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.828+0000 7fa272000700 1 Processor -- start 2024-03-24T00:58:38.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.828+0000 7fa272000700 1 -- start start 2024-03-24T00:58:38.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.829+0000 7fa272000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa26c1008e0 0x7fa26c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:38.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.829+0000 7fa272000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa26c1068e0 0x7fa26c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:38.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.829+0000 7fa272000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c073b30 con 0x7fa26c1068e0 2024-03-24T00:58:38.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.829+0000 7fa272000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c073c70 con 0x7fa26c1008e0 2024-03-24T00:58:38.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.829+0000 7fa26be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa26c1068e0 0x7fa26c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:38.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.829+0000 7fa26be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa26c1068e0 0x7fa26c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45380/0 (socket says 172.21.15.38:45380) 2024-03-24T00:58:38.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.829+0000 7fa26be00700 1 -- 172.21.15.38:0/1399793544 learned_addr learned my addr 172.21.15.38:0/1399793544 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:58:38.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.829+0000 7fa270c00700 1 --2- 172.21.15.38:0/1399793544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa26c1008e0 0x7fa26c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:38.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.829+0000 7fa26be00700 1 -- 172.21.15.38:0/1399793544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa26c1008e0 msgr2=0x7fa26c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:38.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.829+0000 7fa26be00700 1 --2- 172.21.15.38:0/1399793544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa26c1008e0 0x7fa26c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:38.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.829+0000 7fa26be00700 1 -- 172.21.15.38:0/1399793544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa260009d00 con 0x7fa26c1068e0 2024-03-24T00:58:38.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.830+0000 7fa26be00700 1 --2- 172.21.15.38:0/1399793544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa26c1068e0 0x7fa26c0735f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa25800c8a0 tx=0x7fa25800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:38.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.830+0000 7fa269600700 1 -- 172.21.15.38:0/1399793544 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa25800cea0 con 0x7fa26c1068e0 2024-03-24T00:58:38.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.830+0000 7fa269600700 1 -- 172.21.15.38:0/1399793544 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa258004830 con 0x7fa26c1068e0 2024-03-24T00:58:38.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.830+0000 7fa272000700 1 -- 172.21.15.38:0/1399793544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26c1a2a80 con 0x7fa26c1068e0 2024-03-24T00:58:38.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.830+0000 7fa269600700 1 -- 172.21.15.38:0/1399793544 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2580055b0 con 0x7fa26c1068e0 2024-03-24T00:58:38.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.830+0000 7fa272000700 1 -- 172.21.15.38:0/1399793544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c1a2de0 con 0x7fa26c1068e0 2024-03-24T00:58:38.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.831+0000 7fa272000700 1 -- 172.21.15.38:0/1399793544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26c108ee0 con 0x7fa26c1068e0 2024-03-24T00:58:38.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.832+0000 7fa269600700 1 -- 172.21.15.38:0/1399793544 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fa2580049a0 con 0x7fa26c1068e0 2024-03-24T00:58:38.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.833+0000 7fa269600700 1 --2- 172.21.15.38:0/1399793544 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa25406dfb0 0x7fa254070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:58:38.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.834+0000 7fa269600700 1 -- 172.21.15.38:0/1399793544 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa25808da50 con 0x7fa26c1068e0 2024-03-24T00:58:38.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.834+0000 7fa270c00700 1 --2- 172.21.15.38:0/1399793544 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa25406dfb0 0x7fa254070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:58:38.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.834+0000 7fa270c00700 1 --2- 172.21.15.38:0/1399793544 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa25406dfb0 0x7fa254070470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa260009850 tx=0x7fa2600097c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:58:38.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:38.838+0000 7fa269600700 1 -- 172.21.15.38:0/1399793544 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa258056e80 con 0x7fa26c1068e0 2024-03-24T00:58:39.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.153+0000 7fa272000700 1 -- 172.21.15.38:0/1399793544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa26c102020 con 0x7fa26c1068e0 2024-03-24T00:58:39.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.154+0000 7fa269600700 1 -- 172.21.15.38:0/1399793544 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+568 (secure 0 0 0) 0x7fa25801d250 con 0x7fa26c1068e0 2024-03-24T00:58:39.155 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_WARN Degraded data redundancy: 15/273 objects degraded (5.495%), 7 pgs degraded 2024-03-24T00:58:39.155 INFO:teuthology.orchestra.run.smithi038.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 15/273 objects degraded (5.495%), 7 pgs degraded 2024-03-24T00:58:39.155 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-03-24T00:58:39.155 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-03-24T00:58:39.155 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-03-24T00:58:39.155 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-03-24T00:58:39.155 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-03-24T00:58:39.155 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-03-24T00:58:39.155 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-03-24T00:58:39.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.157+0000 7fa272000700 1 -- 172.21.15.38:0/1399793544 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa25406dfb0 msgr2=0x7fa254070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:39.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.157+0000 7fa272000700 1 --2- 172.21.15.38:0/1399793544 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa25406dfb0 0x7fa254070470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa260009850 tx=0x7fa2600097c0 comp rx=0 tx=0).stop 2024-03-24T00:58:39.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.157+0000 7fa272000700 1 -- 172.21.15.38:0/1399793544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa26c1068e0 msgr2=0x7fa26c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:58:39.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.157+0000 7fa272000700 1 --2- 172.21.15.38:0/1399793544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa26c1068e0 0x7fa26c0735f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa25800c8a0 tx=0x7fa25800cc60 comp rx=0 tx=0).stop 2024-03-24T00:58:39.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.157+0000 7fa272000700 1 -- 172.21.15.38:0/1399793544 shutdown_connections 2024-03-24T00:58:39.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.157+0000 7fa272000700 1 --2- 172.21.15.38:0/1399793544 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa25406dfb0 0x7fa254070470 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:39.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.157+0000 7fa272000700 1 --2- 172.21.15.38:0/1399793544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa26c1008e0 0x7fa26c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:39.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.157+0000 7fa272000700 1 --2- 172.21.15.38:0/1399793544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa26c1068e0 0x7fa26c0735f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:58:39.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.157+0000 7fa272000700 1 -- 172.21.15.38:0/1399793544 >> 172.21.15.38:0/1399793544 conn(0x7fa26c0fc3f0 msgr2=0x7fa26c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:58:39.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.157+0000 7fa272000700 1 -- 172.21.15.38:0/1399793544 shutdown_connections 2024-03-24T00:58:39.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:58:39.157+0000 7fa272000700 1 -- 172.21.15.38:0/1399793544 wait complete. 2024-03-24T00:58:39.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:39 smithi088 ceph-mon[87351]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T00:58:39.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:39 smithi088 ceph-mon[87351]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-24T00:58:39.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:39 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1399793544' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:58:39.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:39 smithi038 ceph-mon[112108]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T00:58:39.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:39 smithi038 ceph-mon[112108]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-24T00:58:39.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:39 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1399793544' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:58:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:40 smithi038 ceph-mon[112108]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:40 smithi038 ceph-mon[112108]: pgmap v169: 65 pgs: 10 peering, 55 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-24T00:58:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:40 smithi038 ceph-mon[112108]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/273 objects degraded (5.495%), 7 pgs degraded) 2024-03-24T00:58:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:40 smithi038 ceph-mon[112108]: Cluster is now healthy 2024-03-24T00:58:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:40 smithi088 ceph-mon[87351]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:58:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:40 smithi088 ceph-mon[87351]: pgmap v169: 65 pgs: 10 peering, 55 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-24T00:58:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:40 smithi088 ceph-mon[87351]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/273 objects degraded (5.495%), 7 pgs degraded) 2024-03-24T00:58:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:40 smithi088 ceph-mon[87351]: Cluster is now healthy 2024-03-24T00:58:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:42 smithi038 ceph-mon[112108]: pgmap v170: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:58:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:58:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:42 smithi088 ceph-mon[87351]: pgmap v170: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:58:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:42.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:58:44.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:44 smithi088 ceph-mon[87351]: pgmap v171: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-24T00:58:44.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:44 smithi038 ceph-mon[112108]: pgmap v171: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-24T00:58:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:46 smithi088 ceph-mon[87351]: pgmap v172: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:58:46.723 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:46 smithi038 ceph-mon[112108]: pgmap v172: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-24T00:58:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:47 smithi038 ceph-mon[112108]: pgmap v173: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-24T00:58:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:47 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:48.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:47 smithi088 ceph-mon[87351]: pgmap v173: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-24T00:58:48.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:47 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:50.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:49 smithi038 ceph-mon[112108]: pgmap v174: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:49 smithi088 ceph-mon[87351]: pgmap v174: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:51.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:51 smithi088 ceph-mon[87351]: pgmap v175: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:52.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:51 smithi038 ceph-mon[112108]: pgmap v175: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:54.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:53 smithi038 ceph-mon[112108]: pgmap v176: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:54.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:53 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T00:58:54.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:53 smithi038 ceph-mon[112108]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T00:58:54.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:53 smithi038 ceph-mon[112108]: Upgrade: osd.5 is safe to restart 2024-03-24T00:58:54.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:53 smithi088 ceph-mon[87351]: pgmap v176: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:54.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:53 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T00:58:54.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:53 smithi088 ceph-mon[87351]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T00:58:54.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:53 smithi088 ceph-mon[87351]: Upgrade: osd.5 is safe to restart 2024-03-24T00:58:56.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:56 smithi088 ceph-mon[87351]: pgmap v177: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:58:56.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:56 smithi088 ceph-mon[87351]: Upgrade: Updating osd.5 2024-03-24T00:58:56.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:56 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:56.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:56 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T00:58:56.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:56 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:58:56.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:56 smithi088 ceph-mon[87351]: Deploying daemon osd.5 on smithi088 2024-03-24T00:58:56.724 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:56 smithi038 ceph-mon[112108]: pgmap v177: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T00:58:56.724 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:56 smithi038 ceph-mon[112108]: Upgrade: Updating osd.5 2024-03-24T00:58:56.724 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:56 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:58:56.724 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:56 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T00:58:56.724 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:56 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:58:56.724 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:56 smithi038 ceph-mon[112108]: Deploying daemon osd.5 on smithi088 2024-03-24T00:58:57.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:58:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:58:58.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:58 smithi088 ceph-mon[87351]: pgmap v178: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:58.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:58 smithi038 ceph-mon[112108]: pgmap v178: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:59.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:58:59 smithi088 ceph-mon[87351]: pgmap v179: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:58:59.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:58:59 smithi038 ceph-mon[112108]: pgmap v179: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:59:00.931 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:00 smithi088 systemd[1]: Stopping Ceph osd.5 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:59:01.431 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:00 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[56557]: 2024-03-24T00:59:00.951+0000 7fc4f5a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T00:59:01.431 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:00 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[56557]: 2024-03-24T00:59:00.951+0000 7fc4f5a00700 -1 osd.5 75 *** Got signal Terminated *** 2024-03-24T00:59:01.431 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:00 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[56557]: 2024-03-24T00:59:00.951+0000 7fc4f5a00700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T00:59:02.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:01 smithi038 ceph-mon[112108]: pgmap v180: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:59:02.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:01 smithi038 ceph-mon[112108]: osd.5 marked itself down and dead 2024-03-24T00:59:02.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:01 smithi088 ceph-mon[87351]: pgmap v180: 65 pgs: 65 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T00:59:02.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:01 smithi088 ceph-mon[87351]: osd.5 marked itself down and dead 2024-03-24T00:59:02.600 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:02 smithi088 podman[106553]: 2024-03-24 00:59:02.277808852 +0000 UTC m=+1.483351907 container died 99e27e23875007b9ef8abde63e09e8ef8c2774f7cdef53840a35195aa6ebbe07 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=reef-738c2fd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-24T00:59:02.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:02 smithi088 ceph-mon[87351]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:59:02.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:02 smithi088 ceph-mon[87351]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-24T00:59:02.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:02 smithi088 ceph-mon[87351]: osdmap e76: 6 total, 5 up, 6 in 2024-03-24T00:59:02.931 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:02 smithi088 podman[106553]: 2024-03-24 00:59:02.736778605 +0000 UTC m=+1.942321700 container cleanup 99e27e23875007b9ef8abde63e09e8ef8c2774f7cdef53840a35195aa6ebbe07 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=reef-738c2fd, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:59:02.931 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:02 smithi088 bash[106553]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5 2024-03-24T00:59:03.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:02 smithi038 ceph-mon[112108]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T00:59:03.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:02 smithi038 ceph-mon[112108]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-24T00:59:03.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:02 smithi038 ceph-mon[112108]: osdmap e76: 6 total, 5 up, 6 in 2024-03-24T00:59:03.430 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:03 smithi088 podman[106572]: 2024-03-24 00:59:03.015331715 +0000 UTC m=+0.723625230 container remove 99e27e23875007b9ef8abde63e09e8ef8c2774f7cdef53840a35195aa6ebbe07 (image=quay.ceph.io/ceph-ci/ceph@sha256:98326f2c709ec05ace1337a0445b7e81810fa2d7810e897bde71d5d14691565d, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=reef-738c2fd, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-24T00:59:04.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:03 smithi038 ceph-mon[112108]: pgmap v182: 65 pgs: 8 peering, 9 stale+active+clean, 48 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:04.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:03 smithi038 ceph-mon[112108]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-24T00:59:04.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:03 smithi038 ceph-mon[112108]: osdmap e77: 6 total, 5 up, 6 in 2024-03-24T00:59:04.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:03 smithi088 ceph-mon[87351]: pgmap v182: 65 pgs: 8 peering, 9 stale+active+clean, 48 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:04.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:03 smithi088 ceph-mon[87351]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-24T00:59:04.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:03 smithi088 ceph-mon[87351]: osdmap e77: 6 total, 5 up, 6 in 2024-03-24T00:59:04.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:03 smithi088 podman[106659]: 2024-03-24 00:59:03.724887947 +0000 UTC m=+0.032637276 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:59:04.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:03 smithi088 podman[106659]: 2024-03-24 00:59:03.856066689 +0000 UTC m=+0.163816047 container create a0d45899ea35467d6ad246032801fc5cfa8c0598e3d6f920b0117f5d85a7e395 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:59:04.458 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:04 smithi088 podman[106659]: 2024-03-24 00:59:04.326328555 +0000 UTC m=+0.634077997 container init a0d45899ea35467d6ad246032801fc5cfa8c0598e3d6f920b0117f5d85a7e395 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T00:59:04.459 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:04 smithi088 podman[106659]: 2024-03-24 00:59:04.338455149 +0000 UTC m=+0.646204537 container start a0d45899ea35467d6ad246032801fc5cfa8c0598e3d6f920b0117f5d85a7e395 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-24T00:59:04.768 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:04 smithi088 podman[106659]: 2024-03-24 00:59:04.458545179 +0000 UTC m=+0.766294537 container attach a0d45899ea35467d6ad246032801fc5cfa8c0598e3d6f920b0117f5d85a7e395 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-24T00:59:05.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:04 smithi038 ceph-mon[112108]: Health check failed: Degraded data redundancy: 6/273 objects degraded (2.198%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T00:59:05.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:04 smithi088 ceph-mon[87351]: Health check failed: Degraded data redundancy: 6/273 objects degraded (2.198%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T00:59:05.117 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:04 smithi088 podman[106685]: 2024-03-24 00:59:04.825859845 +0000 UTC m=+0.034773282 container died a0d45899ea35467d6ad246032801fc5cfa8c0598e3d6f920b0117f5d85a7e395 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-24T00:59:05.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:05 smithi088 ceph-mon[87351]: pgmap v184: 65 pgs: 3 active+undersized+degraded, 8 peering, 5 stale+active+clean, 4 active+undersized, 45 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 6/273 objects degraded (2.198%) 2024-03-24T00:59:05.931 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:05 smithi088 podman[106685]: 2024-03-24 00:59:05.595339289 +0000 UTC m=+0.804252742 container remove a0d45899ea35467d6ad246032801fc5cfa8c0598e3d6f920b0117f5d85a7e395 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:59:05.931 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:05 smithi088 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-24T00:59:05.931 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:05 smithi088 systemd[1]: Stopped Ceph osd.5 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:59:05.931 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:05 smithi088 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.5.service: Consumed 2min 7.980s CPU time. 2024-03-24T00:59:06.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:05 smithi038 ceph-mon[112108]: pgmap v184: 65 pgs: 3 active+undersized+degraded, 8 peering, 5 stale+active+clean, 4 active+undersized, 45 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 6/273 objects degraded (2.198%) 2024-03-24T00:59:06.392 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:06 smithi088 systemd[1]: Starting Ceph osd.5 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T00:59:06.681 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:06 smithi088 podman[106782]: 2024-03-24 00:59:06.296808744 +0000 UTC m=+0.035181472 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:59:06.681 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:06 smithi088 podman[106782]: 2024-03-24 00:59:06.455084007 +0000 UTC m=+0.193456765 container create d5538e7f0c0786de4d027b93fcda6e1dbe916672d4fc221bd4bd3d96da787fc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-03-24T00:59:07.087 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:06 smithi088 podman[106782]: 2024-03-24 00:59:06.725843491 +0000 UTC m=+0.464216246 container init d5538e7f0c0786de4d027b93fcda6e1dbe916672d4fc221bd4bd3d96da787fc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:59:07.088 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:06 smithi088 podman[106782]: 2024-03-24 00:59:06.73746776 +0000 UTC m=+0.475840504 container start d5538e7f0c0786de4d027b93fcda6e1dbe916672d4fc221bd4bd3d96da787fc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-24T00:59:07.088 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:06 smithi088 podman[106782]: 2024-03-24 00:59:06.816059021 +0000 UTC m=+0.554431802 container attach d5538e7f0c0786de4d027b93fcda6e1dbe916672d4fc221bd4bd3d96da787fc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-24T00:59:07.819 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate[106796]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate[106796]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 bash[106782]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 bash[106782]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 bash[106782]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 bash[106782]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 bash[106782]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 bash[106782]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 bash[106782]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate[106796]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate[106796]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate[106796]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-24T00:59:07.820 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate[106796]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T00:59:07.821 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate[106796]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-24T00:59:07.821 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:07 smithi088 podman[106782]: 2024-03-24 00:59:07.62142773 +0000 UTC m=+1.359800468 container died d5538e7f0c0786de4d027b93fcda6e1dbe916672d4fc221bd4bd3d96da787fc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-24T00:59:08.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:07 smithi038 ceph-mon[112108]: pgmap v185: 65 pgs: 9 active+undersized+degraded, 8 peering, 11 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 28/273 objects degraded (10.256%) 2024-03-24T00:59:08.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:07 smithi088 ceph-mon[87351]: pgmap v185: 65 pgs: 9 active+undersized+degraded, 8 peering, 11 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 28/273 objects degraded (10.256%) 2024-03-24T00:59:08.871 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:08 smithi088 podman[106924]: 2024-03-24 00:59:08.515482858 +0000 UTC m=+0.874823933 container remove d5538e7f0c0786de4d027b93fcda6e1dbe916672d4fc221bd4bd3d96da787fc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-activate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-24T00:59:09.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:08 smithi038 ceph-mon[112108]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-24T00:59:09.140 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:08 smithi088 podman[106984]: 2024-03-24 00:59:08.784981129 +0000 UTC m=+0.037462194 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T00:59:09.140 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:08 smithi088 podman[106984]: 2024-03-24 00:59:08.945849457 +0000 UTC m=+0.198330540 container create 866388fc07f6c498c8065a24c3c89c5144475d811ca6b80c6cf79415d9279cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T00:59:09.141 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:08 smithi088 ceph-mon[87351]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-24T00:59:09.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.337+0000 7f0024800700 1 -- 172.21.15.38:0/1412585891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f001c107310 msgr2=0x7f001c1076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:09.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.337+0000 7f0024800700 1 --2- 172.21.15.38:0/1412585891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f001c107310 0x7f001c1076f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0010009910 tx=0x7f0010009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:09.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.337+0000 7f0024800700 1 -- 172.21.15.38:0/1412585891 shutdown_connections 2024-03-24T00:59:09.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.337+0000 7f0024800700 1 --2- 172.21.15.38:0/1412585891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f001c107cc0 0x7f001c10bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:09.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.337+0000 7f0024800700 1 --2- 172.21.15.38:0/1412585891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f001c107310 0x7f001c1076f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:09.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.337+0000 7f0024800700 1 -- 172.21.15.38:0/1412585891 >> 172.21.15.38:0/1412585891 conn(0x7f001c0768e0 msgr2=0x7f001c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:09.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.337+0000 7f0024800700 1 -- 172.21.15.38:0/1412585891 shutdown_connections 2024-03-24T00:59:09.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.337+0000 7f0024800700 1 -- 172.21.15.38:0/1412585891 wait complete. 2024-03-24T00:59:09.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.338+0000 7f0024800700 1 Processor -- start 2024-03-24T00:59:09.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.338+0000 7f0024800700 1 -- start start 2024-03-24T00:59:09.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0024800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f001c107310 0x7f001c19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:09.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0024800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f001c107cc0 0x7f001c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:09.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0024800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c19dc50 con 0x7f001c107310 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0024800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c1a18d0 con 0x7f001c107cc0 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0021e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f001c107310 0x7f001c19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0021e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f001c107310 0x7f001c19d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54194/0 (socket says 172.21.15.38:54194) 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0021e00700 1 -- 172.21.15.38:0/3690963000 learned_addr learned my addr 172.21.15.38:0/3690963000 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0021400700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f001c107cc0 0x7f001c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0021e00700 1 -- 172.21.15.38:0/3690963000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f001c107cc0 msgr2=0x7f001c19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0021e00700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f001c107cc0 0x7f001c19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0021e00700 1 -- 172.21.15.38:0/3690963000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00100095f0 con 0x7f001c107310 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.339+0000 7f0021400700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f001c107cc0 0x7f001c19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.340+0000 7f0021e00700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f001c107310 0x7f001c19d030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f00100092e0 tx=0x7f001000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.340+0000 7f0016a00700 1 -- 172.21.15.38:0/3690963000 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f001001d070 con 0x7f001c107310 2024-03-24T00:59:09.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.340+0000 7f0016a00700 1 -- 172.21.15.38:0/3690963000 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00100052a0 con 0x7f001c107310 2024-03-24T00:59:09.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.340+0000 7f0024800700 1 -- 172.21.15.38:0/3690963000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c1a1b50 con 0x7f001c107310 2024-03-24T00:59:09.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.340+0000 7f0016a00700 1 -- 172.21.15.38:0/3690963000 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0010017400 con 0x7f001c107310 2024-03-24T00:59:09.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.340+0000 7f0024800700 1 -- 172.21.15.38:0/3690963000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c1a2040 con 0x7f001c107310 2024-03-24T00:59:09.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.342+0000 7f0024800700 1 -- 172.21.15.38:0/3690963000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c10f6b0 con 0x7f001c107310 2024-03-24T00:59:09.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.343+0000 7f0016a00700 1 -- 172.21.15.38:0/3690963000 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f001000f820 con 0x7f001c107310 2024-03-24T00:59:09.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.344+0000 7f0016a00700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f000406dfb0 0x7f0004070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:09.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.344+0000 7f0016a00700 1 -- 172.21.15.38:0/3690963000 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f001008efd0 con 0x7f001c107310 2024-03-24T00:59:09.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.344+0000 7f0021400700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f000406dfb0 0x7f0004070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:09.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.345+0000 7f0021400700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f000406dfb0 0x7f0004070470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f001c19e540 tx=0x7f000c009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:09.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.349+0000 7f0016a00700 1 -- 172.21.15.38:0/3690963000 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f00100c3740 con 0x7f001c107310 2024-03-24T00:59:09.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.603+0000 7f0024800700 1 -- 172.21.15.38:0/3690963000 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f001c1a22b0 con 0x7f000406dfb0 2024-03-24T00:59:09.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.606+0000 7f0016a00700 1 -- 172.21.15.38:0/3690963000 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f001c1a22b0 con 0x7f000406dfb0 2024-03-24T00:59:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.608+0000 7f0024800700 1 -- 172.21.15.38:0/3690963000 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f000406dfb0 msgr2=0x7f0004070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.608+0000 7f0024800700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f000406dfb0 0x7f0004070470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f001c19e540 tx=0x7f000c009260 comp rx=0 tx=0).stop 2024-03-24T00:59:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.608+0000 7f0024800700 1 -- 172.21.15.38:0/3690963000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f001c107310 msgr2=0x7f001c19d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.608+0000 7f0024800700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f001c107310 0x7f001c19d030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f00100092e0 tx=0x7f001000b6c0 comp rx=0 tx=0).stop 2024-03-24T00:59:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.609+0000 7f0024800700 1 -- 172.21.15.38:0/3690963000 shutdown_connections 2024-03-24T00:59:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.609+0000 7f0024800700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f000406dfb0 0x7f0004070470 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.609+0000 7f0024800700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f001c107310 0x7f001c19d030 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.609+0000 7f0024800700 1 --2- 172.21.15.38:0/3690963000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f001c107cc0 0x7f001c19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.609+0000 7f0024800700 1 -- 172.21.15.38:0/3690963000 >> 172.21.15.38:0/3690963000 conn(0x7f001c0768e0 msgr2=0x7f001c076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.609+0000 7f0024800700 1 -- 172.21.15.38:0/3690963000 shutdown_connections 2024-03-24T00:59:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.609+0000 7f0024800700 1 -- 172.21.15.38:0/3690963000 wait complete. 2024-03-24T00:59:09.625 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:59:09.680 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:09 smithi088 podman[106984]: 2024-03-24 00:59:09.232808695 +0000 UTC m=+0.485289784 container init 866388fc07f6c498c8065a24c3c89c5144475d811ca6b80c6cf79415d9279cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T00:59:09.680 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:09 smithi088 podman[106984]: 2024-03-24 00:59:09.246051199 +0000 UTC m=+0.498532254 container start 866388fc07f6c498c8065a24c3c89c5144475d811ca6b80c6cf79415d9279cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-03-24T00:59:09.680 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:09 smithi088 ceph-osd[107002]: -- 172.21.15.88:0/1351501902 <== mon.1 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55596c9f7680 con 0x55596c9c0c00 2024-03-24T00:59:09.680 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:09 smithi088 bash[106984]: 866388fc07f6c498c8065a24c3c89c5144475d811ca6b80c6cf79415d9279cb3 2024-03-24T00:59:09.680 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:09 smithi088 systemd[1]: Started Ceph osd.5 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T00:59:09.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.770+0000 7f642a400700 1 -- 172.21.15.38:0/2678380407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64241068e0 msgr2=0x7f6424106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:09.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.770+0000 7f642a400700 1 --2- 172.21.15.38:0/2678380407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64241068e0 0x7f6424106cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f640c009910 tx=0x7f640c009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:09.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.770+0000 7f642a400700 1 -- 172.21.15.38:0/2678380407 shutdown_connections 2024-03-24T00:59:09.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.770+0000 7f642a400700 1 --2- 172.21.15.38:0/2678380407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6424068480 0x7f64240688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:09.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.770+0000 7f642a400700 1 --2- 172.21.15.38:0/2678380407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64241068e0 0x7f6424106cc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:09.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.770+0000 7f642a400700 1 -- 172.21.15.38:0/2678380407 >> 172.21.15.38:0/2678380407 conn(0x7f6424075370 msgr2=0x7f6424075780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:09.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.770+0000 7f642a400700 1 -- 172.21.15.38:0/2678380407 shutdown_connections 2024-03-24T00:59:09.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.770+0000 7f642a400700 1 -- 172.21.15.38:0/2678380407 wait complete. 2024-03-24T00:59:09.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.771+0000 7f642a400700 1 Processor -- start 2024-03-24T00:59:09.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.771+0000 7f642a400700 1 -- start start 2024-03-24T00:59:09.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.772+0000 7f642a400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6424068480 0x7f64240ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:09.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.772+0000 7f642a400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64241068e0 0x7f64240fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:09.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.772+0000 7f642a400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6424100440 con 0x7f6424068480 2024-03-24T00:59:09.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.772+0000 7f642a400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6424100580 con 0x7f64241068e0 2024-03-24T00:59:09.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.772+0000 7f6429000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6424068480 0x7f64240ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:09.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.772+0000 7f6429000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6424068480 0x7f64240ff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54206/0 (socket says 172.21.15.38:54206) 2024-03-24T00:59:09.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.772+0000 7f6429000700 1 -- 172.21.15.38:0/220894415 learned_addr learned my addr 172.21.15.38:0/220894415 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:09.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.772+0000 7f641be00700 1 --2- 172.21.15.38:0/220894415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64241068e0 0x7f64240fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:09.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.773+0000 7f6429000700 1 -- 172.21.15.38:0/220894415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64241068e0 msgr2=0x7f64240fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:09.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.773+0000 7f6429000700 1 --2- 172.21.15.38:0/220894415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64241068e0 0x7f64240fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:09.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.773+0000 7f6429000700 1 -- 172.21.15.38:0/220894415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0095f0 con 0x7f6424068480 2024-03-24T00:59:09.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.773+0000 7f6429000700 1 --2- 172.21.15.38:0/220894415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6424068480 0x7f64240ff9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f640c000c00 tx=0x7f640c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:09.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.773+0000 7f6422000700 1 -- 172.21.15.38:0/220894415 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f640c01d070 con 0x7f6424068480 2024-03-24T00:59:09.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.773+0000 7f642a400700 1 -- 172.21.15.38:0/220894415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6424100800 con 0x7f6424068480 2024-03-24T00:59:09.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.773+0000 7f6422000700 1 -- 172.21.15.38:0/220894415 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f640c0056e0 con 0x7f6424068480 2024-03-24T00:59:09.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.773+0000 7f6422000700 1 -- 172.21.15.38:0/220894415 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f640c00fe40 con 0x7f6424068480 2024-03-24T00:59:09.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.774+0000 7f642a400700 1 -- 172.21.15.38:0/220894415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64241a2c20 con 0x7f6424068480 2024-03-24T00:59:09.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.775+0000 7f642a400700 1 -- 172.21.15.38:0/220894415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6424108e90 con 0x7f6424068480 2024-03-24T00:59:09.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.776+0000 7f6422000700 1 -- 172.21.15.38:0/220894415 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f640c00f680 con 0x7f6424068480 2024-03-24T00:59:09.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.777+0000 7f6422000700 1 --2- 172.21.15.38:0/220894415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f640406dfb0 0x7f6404070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:09.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.777+0000 7f6422000700 1 -- 172.21.15.38:0/220894415 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f640c08eff0 con 0x7f6424068480 2024-03-24T00:59:09.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.777+0000 7f641be00700 1 --2- 172.21.15.38:0/220894415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f640406dfb0 0x7f6404070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:09.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.778+0000 7f641be00700 1 --2- 172.21.15.38:0/220894415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f640406dfb0 0x7f6404070470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f64241010f0 tx=0x7f641400b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:09.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:09.782+0000 7f6422000700 1 -- 172.21.15.38:0/220894415 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f640c0584c0 con 0x7f6424068480 2024-03-24T00:59:10.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.034+0000 7f642a400700 1 -- 172.21.15.38:0/220894415 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64240611f0 con 0x7f640406dfb0 2024-03-24T00:59:10.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.037+0000 7f6422000700 1 -- 172.21.15.38:0/220894415 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f64240611f0 con 0x7f640406dfb0 2024-03-24T00:59:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.040+0000 7f642a400700 1 -- 172.21.15.38:0/220894415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f640406dfb0 msgr2=0x7f6404070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.040+0000 7f642a400700 1 --2- 172.21.15.38:0/220894415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f640406dfb0 0x7f6404070470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f64241010f0 tx=0x7f641400b410 comp rx=0 tx=0).stop 2024-03-24T00:59:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.040+0000 7f642a400700 1 -- 172.21.15.38:0/220894415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6424068480 msgr2=0x7f64240ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.040+0000 7f642a400700 1 --2- 172.21.15.38:0/220894415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6424068480 0x7f64240ff9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f640c000c00 tx=0x7f640c00bd70 comp rx=0 tx=0).stop 2024-03-24T00:59:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.040+0000 7f642a400700 1 -- 172.21.15.38:0/220894415 shutdown_connections 2024-03-24T00:59:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.040+0000 7f642a400700 1 --2- 172.21.15.38:0/220894415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f640406dfb0 0x7f6404070470 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.040+0000 7f642a400700 1 --2- 172.21.15.38:0/220894415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6424068480 0x7f64240ff9c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.040+0000 7f642a400700 1 --2- 172.21.15.38:0/220894415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64241068e0 0x7f64240fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.040+0000 7f642a400700 1 -- 172.21.15.38:0/220894415 >> 172.21.15.38:0/220894415 conn(0x7f6424075370 msgr2=0x7f6424105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:10.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.040+0000 7f642a400700 1 -- 172.21.15.38:0/220894415 shutdown_connections 2024-03-24T00:59:10.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.040+0000 7f642a400700 1 -- 172.21.15.38:0/220894415 wait complete. 2024-03-24T00:59:10.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:09 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T00:59:09.816+0000 7f708dfa37c0 -1 Falling back to public interface 2024-03-24T00:59:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:09 smithi088 ceph-mon[87351]: pgmap v186: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 41/273 objects degraded (15.018%) 2024-03-24T00:59:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:09 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:09 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:10.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:09 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:10.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.205+0000 7fa478800700 1 -- 172.21.15.38:0/2085448703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470073a70 msgr2=0x7fa470111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:10.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.205+0000 7fa478800700 1 --2- 172.21.15.38:0/2085448703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470073a70 0x7fa470111540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa460009910 tx=0x7fa460009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:10.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.206+0000 7fa478800700 1 -- 172.21.15.38:0/2085448703 shutdown_connections 2024-03-24T00:59:10.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.206+0000 7fa478800700 1 --2- 172.21.15.38:0/2085448703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470073a70 0x7fa470111540 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.206+0000 7fa478800700 1 --2- 172.21.15.38:0/2085448703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa470073150 0x7fa470073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.206+0000 7fa478800700 1 -- 172.21.15.38:0/2085448703 >> 172.21.15.38:0/2085448703 conn(0x7fa4700fc580 msgr2=0x7fa4700fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:10.206 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:09 smithi038 ceph-mon[112108]: pgmap v186: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 41/273 objects degraded (15.018%) 2024-03-24T00:59:10.207 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:09 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:10.207 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:09 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:10.207 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:09 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:10.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.206+0000 7fa478800700 1 -- 172.21.15.38:0/2085448703 shutdown_connections 2024-03-24T00:59:10.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.206+0000 7fa478800700 1 -- 172.21.15.38:0/2085448703 wait complete. 2024-03-24T00:59:10.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.207+0000 7fa478800700 1 Processor -- start 2024-03-24T00:59:10.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.207+0000 7fa478800700 1 -- start start 2024-03-24T00:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.208+0000 7fa478800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa470073150 0x7fa47019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.208+0000 7fa478800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470073a70 0x7fa47019d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.208+0000 7fa478800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47019dc00 con 0x7fa470073a70 2024-03-24T00:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.208+0000 7fa478800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4701a18d0 con 0x7fa470073150 2024-03-24T00:59:10.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.208+0000 7fa475400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470073a70 0x7fa47019d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.208+0000 7fa475400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470073a70 0x7fa47019d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54210/0 (socket says 172.21.15.38:54210) 2024-03-24T00:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.208+0000 7fa475400700 1 -- 172.21.15.38:0/140655789 learned_addr learned my addr 172.21.15.38:0/140655789 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.208+0000 7fa475e00700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa470073150 0x7fa47019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.209+0000 7fa475400700 1 -- 172.21.15.38:0/140655789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa470073150 msgr2=0x7fa47019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.209+0000 7fa475400700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa470073150 0x7fa47019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.209+0000 7fa475400700 1 -- 172.21.15.38:0/140655789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600095f0 con 0x7fa470073a70 2024-03-24T00:59:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.209+0000 7fa475e00700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa470073150 0x7fa47019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T00:59:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.209+0000 7fa475400700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470073a70 0x7fa47019d520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa460000c00 tx=0x7fa46000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.209+0000 7fa46aa00700 1 -- 172.21.15.38:0/140655789 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa46001d070 con 0x7fa470073a70 2024-03-24T00:59:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.209+0000 7fa46aa00700 1 -- 172.21.15.38:0/140655789 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4600056e0 con 0x7fa470073a70 2024-03-24T00:59:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.209+0000 7fa46aa00700 1 -- 172.21.15.38:0/140655789 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa46000fe80 con 0x7fa470073a70 2024-03-24T00:59:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.209+0000 7fa478800700 1 -- 172.21.15.38:0/140655789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4701a1bb0 con 0x7fa470073a70 2024-03-24T00:59:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.210+0000 7fa478800700 1 -- 172.21.15.38:0/140655789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4701a2100 con 0x7fa470073a70 2024-03-24T00:59:10.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.211+0000 7fa478800700 1 -- 172.21.15.38:0/140655789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa470074890 con 0x7fa470073a70 2024-03-24T00:59:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.212+0000 7fa46aa00700 1 -- 172.21.15.38:0/140655789 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fa46000f6f0 con 0x7fa470073a70 2024-03-24T00:59:10.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.213+0000 7fa46aa00700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa45806dfb0 0x7fa458070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:10.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.213+0000 7fa46aa00700 1 -- 172.21.15.38:0/140655789 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa46008f010 con 0x7fa470073a70 2024-03-24T00:59:10.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.213+0000 7fa475e00700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa45806dfb0 0x7fa458070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:10.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.214+0000 7fa475e00700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa45806dfb0 0x7fa458070470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa464005950 tx=0x7fa4640058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:10.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.218+0000 7fa46aa00700 1 -- 172.21.15.38:0/140655789 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa460058560 con 0x7fa470073a70 2024-03-24T00:59:10.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.459+0000 7fa478800700 1 -- 172.21.15.38:0/140655789 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4701a23e0 con 0x7fa45806dfb0 2024-03-24T00:59:10.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.484+0000 7fa46aa00700 1 -- 172.21.15.38:0/140655789 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa4701a23e0 con 0x7fa45806dfb0 2024-03-24T00:59:10.484 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:59:10.484 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (7m) 112s ago 21m 13.5M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:59:10.484 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (21m) 112s ago 21m 9068k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:59:10.484 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (20m) 40s ago 20m 8816k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (3m) 112s ago 21m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (3m) 40s ago 20m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (6m) 112s ago 21m 75.3M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (14m) 112s ago 14m 97.9M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (14m) 112s ago 14m 150M - 18.2.2-952-g738c2fd6 68aff9aff3b2 4d724e47bf82 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (14m) 40s ago 14m 24.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (14m) 40s ago 14m 24.9M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (10m) 112s ago 23m 502M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (9m) 40s ago 20m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (5m) 112s ago 23m 48.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (4m) 40s ago 20m 43.3M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (8m) 112s ago 21m 7755k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (8m) 40s ago 20m 7675k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (3m) 112s ago 19m 97.1M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T00:59:10.485 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (2m) 112s ago 18m 104M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cb72ac446ba7 2024-03-24T00:59:10.486 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (2m) 112s ago 17m 11.2M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c73f3e8276b6 2024-03-24T00:59:10.486 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (88s) 40s ago 17m 108M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b2afaff761d8 2024-03-24T00:59:10.486 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (45s) 40s ago 16m 11.5M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9a6f202043cd 2024-03-24T00:59:10.486 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 starting - - - 2241M 2024-03-24T00:59:10.486 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (8m) 112s ago 20m 70.6M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:59:10.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.488+0000 7fa478800700 1 -- 172.21.15.38:0/140655789 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa45806dfb0 msgr2=0x7fa458070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:10.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.488+0000 7fa478800700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa45806dfb0 0x7fa458070470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa464005950 tx=0x7fa4640058e0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.488+0000 7fa478800700 1 -- 172.21.15.38:0/140655789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470073a70 msgr2=0x7fa47019d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:10.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.488+0000 7fa478800700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470073a70 0x7fa47019d520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa460000c00 tx=0x7fa46000be20 comp rx=0 tx=0).stop 2024-03-24T00:59:10.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.488+0000 7fa478800700 1 -- 172.21.15.38:0/140655789 shutdown_connections 2024-03-24T00:59:10.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.488+0000 7fa478800700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa45806dfb0 0x7fa458070470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.488+0000 7fa478800700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa470073150 0x7fa47019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.488+0000 7fa478800700 1 --2- 172.21.15.38:0/140655789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa470073a70 0x7fa47019d520 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.488+0000 7fa478800700 1 -- 172.21.15.38:0/140655789 >> 172.21.15.38:0/140655789 conn(0x7fa4700fc580 msgr2=0x7fa4701007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:10.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.489+0000 7fa478800700 1 -- 172.21.15.38:0/140655789 shutdown_connections 2024-03-24T00:59:10.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.489+0000 7fa478800700 1 -- 172.21.15.38:0/140655789 wait complete. 2024-03-24T00:59:10.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.652+0000 7fb8d1000700 1 -- 172.21.15.38:0/4275970202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8cc073aa0 msgr2=0x7fb8cc10cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:10.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.652+0000 7fb8d1000700 1 --2- 172.21.15.38:0/4275970202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8cc073aa0 0x7fb8cc10cdd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80098a0 tx=0x7fb8b8009bb0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.652+0000 7fb8d1000700 1 -- 172.21.15.38:0/4275970202 shutdown_connections 2024-03-24T00:59:10.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.652+0000 7fb8d1000700 1 --2- 172.21.15.38:0/4275970202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8cc073aa0 0x7fb8cc10cdd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.652+0000 7fb8d1000700 1 --2- 172.21.15.38:0/4275970202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8cc073180 0x7fb8cc073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.652+0000 7fb8d1000700 1 -- 172.21.15.38:0/4275970202 >> 172.21.15.38:0/4275970202 conn(0x7fb8cc0fc580 msgr2=0x7fb8cc0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:10.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.652+0000 7fb8d1000700 1 -- 172.21.15.38:0/4275970202 shutdown_connections 2024-03-24T00:59:10.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.652+0000 7fb8d1000700 1 -- 172.21.15.38:0/4275970202 wait complete. 2024-03-24T00:59:10.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.653+0000 7fb8d1000700 1 Processor -- start 2024-03-24T00:59:10.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.654+0000 7fb8d1000700 1 -- start start 2024-03-24T00:59:10.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.654+0000 7fb8d1000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8cc073180 0x7fb8cc19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:10.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.654+0000 7fb8d1000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8cc073aa0 0x7fb8cc19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:10.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.654+0000 7fb8d1000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc19f430 con 0x7fb8cc073aa0 2024-03-24T00:59:10.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.654+0000 7fb8d1000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc198a10 con 0x7fb8cc073180 2024-03-24T00:59:10.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.654+0000 7fb8c9400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8cc073aa0 0x7fb8cc19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:10.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.655+0000 7fb8c9400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8cc073aa0 0x7fb8cc19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54230/0 (socket says 172.21.15.38:54230) 2024-03-24T00:59:10.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.655+0000 7fb8c9400700 1 -- 172.21.15.38:0/4114268513 learned_addr learned my addr 172.21.15.38:0/4114268513 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:10.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.655+0000 7fb8c9e00700 1 --2- 172.21.15.38:0/4114268513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8cc073180 0x7fb8cc19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:10.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.655+0000 7fb8c9400700 1 -- 172.21.15.38:0/4114268513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8cc073180 msgr2=0x7fb8cc19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:10.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.655+0000 7fb8c9400700 1 --2- 172.21.15.38:0/4114268513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8cc073180 0x7fb8cc19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.655+0000 7fb8c9400700 1 -- 172.21.15.38:0/4114268513 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b8009520 con 0x7fb8cc073aa0 2024-03-24T00:59:10.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.656+0000 7fb8c9400700 1 --2- 172.21.15.38:0/4114268513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8cc073aa0 0x7fb8cc19ee10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8000c00 tx=0x7fb8b80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:10.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.656+0000 7fb8c2a00700 1 -- 172.21.15.38:0/4114268513 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b801c070 con 0x7fb8cc073aa0 2024-03-24T00:59:10.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.656+0000 7fb8d1000700 1 -- 172.21.15.38:0/4114268513 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8cc198c90 con 0x7fb8cc073aa0 2024-03-24T00:59:10.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.656+0000 7fb8c2a00700 1 -- 172.21.15.38:0/4114268513 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8b800fbf0 con 0x7fb8cc073aa0 2024-03-24T00:59:10.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.656+0000 7fb8c2a00700 1 -- 172.21.15.38:0/4114268513 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b8022d50 con 0x7fb8cc073aa0 2024-03-24T00:59:10.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.656+0000 7fb8d1000700 1 -- 172.21.15.38:0/4114268513 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8cc199180 con 0x7fb8cc073aa0 2024-03-24T00:59:10.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.657+0000 7fb8d1000700 1 -- 172.21.15.38:0/4114268513 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8cc1076e0 con 0x7fb8cc073aa0 2024-03-24T00:59:10.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.659+0000 7fb8c2a00700 1 -- 172.21.15.38:0/4114268513 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fb8b8020070 con 0x7fb8cc073aa0 2024-03-24T00:59:10.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.659+0000 7fb8c2a00700 1 --2- 172.21.15.38:0/4114268513 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8b006dfb0 0x7fb8b0070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:10.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.660+0000 7fb8c2a00700 1 -- 172.21.15.38:0/4114268513 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb8b808ef60 con 0x7fb8cc073aa0 2024-03-24T00:59:10.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.660+0000 7fb8c9e00700 1 --2- 172.21.15.38:0/4114268513 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8b006dfb0 0x7fb8b0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:10.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.661+0000 7fb8c9e00700 1 --2- 172.21.15.38:0/4114268513 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8b006dfb0 0x7fb8b0070470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc005fd0 tx=0x7fb8bc009310 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:10.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.665+0000 7fb8c2a00700 1 -- 172.21.15.38:0/4114268513 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb8b805bec0 con 0x7fb8cc073aa0 2024-03-24T00:59:10.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.985+0000 7fb8d1000700 1 -- 172.21.15.38:0/4114268513 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb8cc199ed0 con 0x7fb8cc073aa0 2024-03-24T00:59:10.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.986+0000 7fb8c2a00700 1 -- 172.21.15.38:0/4114268513 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+728 (secure 0 0 0) 0x7fb8b805ba50 con 0x7fb8cc073aa0 2024-03-24T00:59:10.986 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:59:10.986 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 5 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 4, 2024-03-24T00:59:10.987 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 9 2024-03-24T00:59:10.988 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:59:10.988 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:59:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.989+0000 7fb8d1000700 1 -- 172.21.15.38:0/4114268513 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8b006dfb0 msgr2=0x7fb8b0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.989+0000 7fb8d1000700 1 --2- 172.21.15.38:0/4114268513 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8b006dfb0 0x7fb8b0070470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc005fd0 tx=0x7fb8bc009310 comp rx=0 tx=0).stop 2024-03-24T00:59:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.989+0000 7fb8d1000700 1 -- 172.21.15.38:0/4114268513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8cc073aa0 msgr2=0x7fb8cc19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.989+0000 7fb8d1000700 1 --2- 172.21.15.38:0/4114268513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8cc073aa0 0x7fb8cc19ee10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8000c00 tx=0x7fb8b80047b0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.989+0000 7fb8d1000700 1 -- 172.21.15.38:0/4114268513 shutdown_connections 2024-03-24T00:59:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.989+0000 7fb8d1000700 1 --2- 172.21.15.38:0/4114268513 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8b006dfb0 0x7fb8b0070470 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.989+0000 7fb8d1000700 1 --2- 172.21.15.38:0/4114268513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8cc073180 0x7fb8cc19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.989+0000 7fb8d1000700 1 --2- 172.21.15.38:0/4114268513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8cc073aa0 0x7fb8cc19ee10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.989+0000 7fb8d1000700 1 -- 172.21.15.38:0/4114268513 >> 172.21.15.38:0/4114268513 conn(0x7fb8cc0fc580 msgr2=0x7fb8cc1052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.989+0000 7fb8d1000700 1 -- 172.21.15.38:0/4114268513 shutdown_connections 2024-03-24T00:59:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:10.989+0000 7fb8d1000700 1 -- 172.21.15.38:0/4114268513 wait complete. 2024-03-24T00:59:11.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.155+0000 7fea0e200700 1 -- 172.21.15.38:0/925936482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08107cc0 msgr2=0x7fea0810bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:11.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.155+0000 7fea0e200700 1 --2- 172.21.15.38:0/925936482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08107cc0 0x7fea0810bd10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009910 tx=0x7fe9f4009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:11.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.155+0000 7fea0e200700 1 -- 172.21.15.38:0/925936482 shutdown_connections 2024-03-24T00:59:11.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.155+0000 7fea0e200700 1 --2- 172.21.15.38:0/925936482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08107cc0 0x7fea0810bd10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.155+0000 7fea0e200700 1 --2- 172.21.15.38:0/925936482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08107310 0x7fea081076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.155+0000 7fea0e200700 1 -- 172.21.15.38:0/925936482 >> 172.21.15.38:0/925936482 conn(0x7fea080768e0 msgr2=0x7fea08078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:11.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.156+0000 7fea0e200700 1 -- 172.21.15.38:0/925936482 shutdown_connections 2024-03-24T00:59:11.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.156+0000 7fea0e200700 1 -- 172.21.15.38:0/925936482 wait complete. 2024-03-24T00:59:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.157+0000 7fea0e200700 1 Processor -- start 2024-03-24T00:59:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.157+0000 7fea0e200700 1 -- start start 2024-03-24T00:59:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.157+0000 7fea0e200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08107310 0x7fea0819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.157+0000 7fea0e200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08107cc0 0x7fea0819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.157+0000 7fea0e200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0819dbe0 con 0x7fea08107cc0 2024-03-24T00:59:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.157+0000 7fea0e200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea081a18b0 con 0x7fea08107310 2024-03-24T00:59:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.158+0000 7fea06a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08107cc0 0x7fea0819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.158+0000 7fea07400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08107310 0x7fea0819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.158+0000 7fea06a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08107cc0 0x7fea0819d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54246/0 (socket says 172.21.15.38:54246) 2024-03-24T00:59:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.158+0000 7fea06a00700 1 -- 172.21.15.38:0/1174522413 learned_addr learned my addr 172.21.15.38:0/1174522413 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.158+0000 7fea06a00700 1 -- 172.21.15.38:0/1174522413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08107310 msgr2=0x7fea0819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.158+0000 7fea06a00700 1 --2- 172.21.15.38:0/1174522413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08107310 0x7fea0819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.158+0000 7fea06a00700 1 -- 172.21.15.38:0/1174522413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f8009520 con 0x7fea08107cc0 2024-03-24T00:59:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.159+0000 7fea06a00700 1 --2- 172.21.15.38:0/1174522413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08107cc0 0x7fea0819d500 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009fd0 tx=0x7fe9f400b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:11.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.159+0000 7fe9ffe00700 1 -- 172.21.15.38:0/1174522413 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9f401d070 con 0x7fea08107cc0 2024-03-24T00:59:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.159+0000 7fe9ffe00700 1 -- 172.21.15.38:0/1174522413 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9f400fd80 con 0x7fea08107cc0 2024-03-24T00:59:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.159+0000 7fea0e200700 1 -- 172.21.15.38:0/1174522413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f40095f0 con 0x7fea08107cc0 2024-03-24T00:59:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.159+0000 7fe9ffe00700 1 -- 172.21.15.38:0/1174522413 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9f4017630 con 0x7fea08107cc0 2024-03-24T00:59:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.159+0000 7fea0e200700 1 -- 172.21.15.38:0/1174522413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea081a1e30 con 0x7fea08107cc0 2024-03-24T00:59:11.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.160+0000 7fea0e200700 1 -- 172.21.15.38:0/1174522413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0810f710 con 0x7fea08107cc0 2024-03-24T00:59:11.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.162+0000 7fe9ffe00700 1 -- 172.21.15.38:0/1174522413 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fe9f4022470 con 0x7fea08107cc0 2024-03-24T00:59:11.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.162+0000 7fe9ffe00700 1 --2- 172.21.15.38:0/1174522413 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe9ec06dfb0 0x7fe9ec070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:11.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.163+0000 7fe9ffe00700 1 -- 172.21.15.38:0/1174522413 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe9f408f130 con 0x7fea08107cc0 2024-03-24T00:59:11.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.163+0000 7fea07400700 1 --2- 172.21.15.38:0/1174522413 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe9ec06dfb0 0x7fe9ec070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:11.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.164+0000 7fea07400700 1 --2- 172.21.15.38:0/1174522413 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe9ec06dfb0 0x7fe9ec070470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009df0 tx=0x7fe9f8009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.168+0000 7fe9ffe00700 1 -- 172.21.15.38:0/1174522413 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe9f40585e0 con 0x7fea08107cc0 2024-03-24T00:59:11.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.452+0000 7fea0e200700 1 -- 172.21.15.38:0/1174522413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fea081087f0 con 0x7fea08107cc0 2024-03-24T00:59:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.453+0000 7fe9ffe00700 1 -- 172.21.15.38:0/1174522413 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1900 (secure 0 0 0) 0x7fe9f405bca0 con 0x7fea08107cc0 2024-03-24T00:59:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:e20 2024-03-24T00:59:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:59:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:59:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:59:11.454 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:59:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:59:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:59:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 20 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:45:17.720123+0000 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 47 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:59:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=14462,1=24271} 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:14462} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.38:6826/2539263555,v1:172.21.15.38:6827/2539263555] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{0:14478} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.38:6828/3798421518,v1:172.21.15.38:6829/3798421518] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:24271} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6826/1446532250,v1:172.21.15.88:6827/1446532250] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:59:11.456 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{1:24279} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2705783970,v1:172.21.15.88:6825/2705783970] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:59:11.457 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:59:11.457 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:59:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.457+0000 7fea0e200700 1 -- 172.21.15.38:0/1174522413 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe9ec06dfb0 msgr2=0x7fe9ec070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.457+0000 7fea0e200700 1 --2- 172.21.15.38:0/1174522413 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe9ec06dfb0 0x7fe9ec070470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009df0 tx=0x7fe9f8009260 comp rx=0 tx=0).stop 2024-03-24T00:59:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.457+0000 7fea0e200700 1 -- 172.21.15.38:0/1174522413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08107cc0 msgr2=0x7fea0819d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.457+0000 7fea0e200700 1 --2- 172.21.15.38:0/1174522413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08107cc0 0x7fea0819d500 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009fd0 tx=0x7fe9f400b5a0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.457+0000 7fea0e200700 1 -- 172.21.15.38:0/1174522413 shutdown_connections 2024-03-24T00:59:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.457+0000 7fea0e200700 1 --2- 172.21.15.38:0/1174522413 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe9ec06dfb0 0x7fe9ec070470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.457+0000 7fea0e200700 1 --2- 172.21.15.38:0/1174522413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08107310 0x7fea0819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.458+0000 7fea0e200700 1 --2- 172.21.15.38:0/1174522413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08107cc0 0x7fea0819d500 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.458+0000 7fea0e200700 1 -- 172.21.15.38:0/1174522413 >> 172.21.15.38:0/1174522413 conn(0x7fea080768e0 msgr2=0x7fea08076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.458+0000 7fea0e200700 1 -- 172.21.15.38:0/1174522413 shutdown_connections 2024-03-24T00:59:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.458+0000 7fea0e200700 1 -- 172.21.15.38:0/1174522413 wait complete. 2024-03-24T00:59:11.459 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-03-24T00:59:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.618+0000 7f13fd200700 1 -- 172.21.15.38:0/4002002253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f8102530 msgr2=0x7f13f810aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.618+0000 7f13fd200700 1 --2- 172.21.15.38:0/4002002253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f8102530 0x7f13f810aa20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009910 tx=0x7f13e0009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.618+0000 7f13fd200700 1 -- 172.21.15.38:0/4002002253 shutdown_connections 2024-03-24T00:59:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.618+0000 7f13fd200700 1 --2- 172.21.15.38:0/4002002253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f8102530 0x7f13f810aa20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.618+0000 7f13fd200700 1 --2- 172.21.15.38:0/4002002253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13f8101c10 0x7f13f8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.618+0000 7f13fd200700 1 -- 172.21.15.38:0/4002002253 >> 172.21.15.38:0/4002002253 conn(0x7f13f8076640 msgr2=0x7f13f80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.619+0000 7f13fd200700 1 -- 172.21.15.38:0/4002002253 shutdown_connections 2024-03-24T00:59:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.619+0000 7f13fd200700 1 -- 172.21.15.38:0/4002002253 wait complete. 2024-03-24T00:59:11.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.620+0000 7f13fd200700 1 Processor -- start 2024-03-24T00:59:11.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.620+0000 7f13fd200700 1 -- start start 2024-03-24T00:59:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.620+0000 7f13fd200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f8101c10 0x7f13f819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.620+0000 7f13fd200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13f8102530 0x7f13f819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.620+0000 7f13fd200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f819d2c0 con 0x7f13f8101c10 2024-03-24T00:59:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.620+0000 7f13fd200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f8196830 con 0x7f13f8102530 2024-03-24T00:59:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.620+0000 7f13f5e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f8101c10 0x7f13f819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.621+0000 7f13f5e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f8101c10 0x7f13f819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54270/0 (socket says 172.21.15.38:54270) 2024-03-24T00:59:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.621+0000 7f13f5e00700 1 -- 172.21.15.38:0/1103971156 learned_addr learned my addr 172.21.15.38:0/1103971156 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.621+0000 7f13ed400700 1 --2- 172.21.15.38:0/1103971156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13f8102530 0x7f13f819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.621+0000 7f13f5e00700 1 -- 172.21.15.38:0/1103971156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13f8102530 msgr2=0x7f13f819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.621+0000 7f13f5e00700 1 --2- 172.21.15.38:0/1103971156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13f8102530 0x7f13f819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.621+0000 7f13f5e00700 1 -- 172.21.15.38:0/1103971156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e00095f0 con 0x7f13f8101c10 2024-03-24T00:59:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.621+0000 7f13f5e00700 1 --2- 172.21.15.38:0/1103971156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f8101c10 0x7f13f819c6f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13e800d8d0 tx=0x7f13e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.622+0000 7f13ef400700 1 -- 172.21.15.38:0/1103971156 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13e8009940 con 0x7f13f8101c10 2024-03-24T00:59:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.622+0000 7f13ef400700 1 -- 172.21.15.38:0/1103971156 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13e8010460 con 0x7f13f8101c10 2024-03-24T00:59:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.622+0000 7f13fd200700 1 -- 172.21.15.38:0/1103971156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f8196b10 con 0x7f13f8101c10 2024-03-24T00:59:11.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.622+0000 7f13ef400700 1 -- 172.21.15.38:0/1103971156 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13e800f5f0 con 0x7f13f8101c10 2024-03-24T00:59:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.622+0000 7f13fd200700 1 -- 172.21.15.38:0/1103971156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f8197060 con 0x7f13f8101c10 2024-03-24T00:59:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.623+0000 7f13fd200700 1 -- 172.21.15.38:0/1103971156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f8105330 con 0x7f13f8101c10 2024-03-24T00:59:11.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.624+0000 7f13ef400700 1 -- 172.21.15.38:0/1103971156 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f13e80105d0 con 0x7f13f8101c10 2024-03-24T00:59:11.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.625+0000 7f13ef400700 1 --2- 172.21.15.38:0/1103971156 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f13dc06dfb0 0x7f13dc070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:11.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.625+0000 7f13ef400700 1 -- 172.21.15.38:0/1103971156 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f13e808ddc0 con 0x7f13f8101c10 2024-03-24T00:59:11.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.625+0000 7f13ed400700 1 --2- 172.21.15.38:0/1103971156 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f13dc06dfb0 0x7f13dc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:11.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.626+0000 7f13ed400700 1 --2- 172.21.15.38:0/1103971156 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f13dc06dfb0 0x7f13dc070470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f13f8197c50 tx=0x7f13e001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:11.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.629+0000 7f13ef400700 1 -- 172.21.15.38:0/1103971156 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f13e8091030 con 0x7f13f8101c10 2024-03-24T00:59:11.744 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:11 smithi088 ceph-mon[87351]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:11.744 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:11 smithi088 ceph-mon[87351]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:11.744 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:11.744 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:11 smithi088 ceph-mon[87351]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:11.744 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:11.745 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:11 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/4114268513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:11.745 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:11 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1174522413' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:59:11.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:11 smithi038 ceph-mon[112108]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:11.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:11 smithi038 ceph-mon[112108]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:11.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:11.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:11 smithi038 ceph-mon[112108]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:11.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:11.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:11 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/4114268513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:11.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:11 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1174522413' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:59:11.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.870+0000 7f13fd200700 1 -- 172.21.15.38:0/1103971156 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13f8197d60 con 0x7f13dc06dfb0 2024-03-24T00:59:11.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.872+0000 7f13ef400700 1 -- 172.21.15.38:0/1103971156 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f13f8197d60 con 0x7f13dc06dfb0 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout: "crash" 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "11/23 daemons upgraded", 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:59:11.873 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:59:11.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.875+0000 7f13fd200700 1 -- 172.21.15.38:0/1103971156 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f13dc06dfb0 msgr2=0x7f13dc070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:11.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.875+0000 7f13fd200700 1 --2- 172.21.15.38:0/1103971156 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f13dc06dfb0 0x7f13dc070470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f13f8197c50 tx=0x7f13e001a040 comp rx=0 tx=0).stop 2024-03-24T00:59:11.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.875+0000 7f13fd200700 1 -- 172.21.15.38:0/1103971156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f8101c10 msgr2=0x7f13f819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:11.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.875+0000 7f13fd200700 1 --2- 172.21.15.38:0/1103971156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f8101c10 0x7f13f819c6f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13e800d8d0 tx=0x7f13e800dc90 comp rx=0 tx=0).stop 2024-03-24T00:59:11.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.875+0000 7f13fd200700 1 -- 172.21.15.38:0/1103971156 shutdown_connections 2024-03-24T00:59:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.875+0000 7f13fd200700 1 --2- 172.21.15.38:0/1103971156 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f13dc06dfb0 0x7f13dc070470 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.875+0000 7f13fd200700 1 --2- 172.21.15.38:0/1103971156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13f8101c10 0x7f13f819c6f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.875+0000 7f13fd200700 1 --2- 172.21.15.38:0/1103971156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13f8102530 0x7f13f819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.875+0000 7f13fd200700 1 -- 172.21.15.38:0/1103971156 >> 172.21.15.38:0/1103971156 conn(0x7f13f8076640 msgr2=0x7f13f8102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.875+0000 7f13fd200700 1 -- 172.21.15.38:0/1103971156 shutdown_connections 2024-03-24T00:59:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:11.875+0000 7f13fd200700 1 -- 172.21.15.38:0/1103971156 wait complete. 2024-03-24T00:59:12.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.031+0000 7fe468200700 1 -- 172.21.15.38:0/370703753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4601038a0 msgr2=0x7fe4601078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:12.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.031+0000 7fe468200700 1 --2- 172.21.15.38:0/370703753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4601038a0 0x7fe4601078f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009910 tx=0x7fe44c009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.031+0000 7fe468200700 1 -- 172.21.15.38:0/370703753 shutdown_connections 2024-03-24T00:59:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.031+0000 7fe468200700 1 --2- 172.21.15.38:0/370703753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4601038a0 0x7fe4601078f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.031+0000 7fe468200700 1 --2- 172.21.15.38:0/370703753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe460102ef0 0x7fe4601032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.031+0000 7fe468200700 1 -- 172.21.15.38:0/370703753 >> 172.21.15.38:0/370703753 conn(0x7fe4600fe780 msgr2=0x7fe460100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.031+0000 7fe468200700 1 -- 172.21.15.38:0/370703753 shutdown_connections 2024-03-24T00:59:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.031+0000 7fe468200700 1 -- 172.21.15.38:0/370703753 wait complete. 2024-03-24T00:59:12.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.032+0000 7fe468200700 1 Processor -- start 2024-03-24T00:59:12.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.033+0000 7fe468200700 1 -- start start 2024-03-24T00:59:12.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.033+0000 7fe468200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe460102ef0 0x7fe46019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:12.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.033+0000 7fe468200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4601038a0 0x7fe46019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:12.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.033+0000 7fe468200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe460198ab0 con 0x7fe460102ef0 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.033+0000 7fe468200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe460198c20 con 0x7fe4601038a0 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.033+0000 7fe465800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe460102ef0 0x7fe46019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.033+0000 7fe465800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe460102ef0 0x7fe46019ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54288/0 (socket says 172.21.15.38:54288) 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.033+0000 7fe465800700 1 -- 172.21.15.38:0/457977469 learned_addr learned my addr 172.21.15.38:0/457977469 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.034+0000 7fe464e00700 1 --2- 172.21.15.38:0/457977469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4601038a0 0x7fe46019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.034+0000 7fe465800700 1 -- 172.21.15.38:0/457977469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4601038a0 msgr2=0x7fe46019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.034+0000 7fe465800700 1 --2- 172.21.15.38:0/457977469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4601038a0 0x7fe46019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.034+0000 7fe465800700 1 -- 172.21.15.38:0/457977469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c0095f0 con 0x7fe460102ef0 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.034+0000 7fe465800700 1 --2- 172.21.15.38:0/457977469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe460102ef0 0x7fe46019ea00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe45400d8d0 tx=0x7fe45400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.035+0000 7fe45e000700 1 -- 172.21.15.38:0/457977469 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4540098a0 con 0x7fe460102ef0 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.035+0000 7fe45e000700 1 -- 172.21.15.38:0/457977469 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe454010460 con 0x7fe460102ef0 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.035+0000 7fe468200700 1 -- 172.21.15.38:0/457977469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe460198f00 con 0x7fe460102ef0 2024-03-24T00:59:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.035+0000 7fe45e000700 1 -- 172.21.15.38:0/457977469 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe45400f5f0 con 0x7fe460102ef0 2024-03-24T00:59:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.035+0000 7fe468200700 1 -- 172.21.15.38:0/457977469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe460199420 con 0x7fe460102ef0 2024-03-24T00:59:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.036+0000 7fe468200700 1 -- 172.21.15.38:0/457977469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46010b2f0 con 0x7fe460102ef0 2024-03-24T00:59:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.037+0000 7fe45e000700 1 -- 172.21.15.38:0/457977469 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fe454009a00 con 0x7fe460102ef0 2024-03-24T00:59:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.038+0000 7fe45e000700 1 --2- 172.21.15.38:0/457977469 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe44806dfb0 0x7fe448070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.038+0000 7fe45e000700 1 -- 172.21.15.38:0/457977469 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe45408dbb0 con 0x7fe460102ef0 2024-03-24T00:59:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.038+0000 7fe464e00700 1 --2- 172.21.15.38:0/457977469 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe44806dfb0 0x7fe448070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.039+0000 7fe464e00700 1 --2- 172.21.15.38:0/457977469 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe44806dfb0 0x7fe448070470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009fd0 tx=0x7fe44c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.044+0000 7fe45e000700 1 -- 172.21.15.38:0/457977469 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe454057060 con 0x7fe460102ef0 2024-03-24T00:59:12.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.360+0000 7fe468200700 1 -- 172.21.15.38:0/457977469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe460199f90 con 0x7fe460102ef0 2024-03-24T00:59:12.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.361+0000 7fe45e000700 1 -- 172.21.15.38:0/457977469 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1290 (secure 0 0 0) 0x7fe454016520 con 0x7fe460102ef0 2024-03-24T00:59:12.361 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_WARN 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 41/273 objects degraded (15.018%), 14 pgs degraded 2024-03-24T00:59:12.362 INFO:teuthology.orchestra.run.smithi038.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-24T00:59:12.362 INFO:teuthology.orchestra.run.smithi038.stdout: osd.5 (root=default,host=smithi088) is down 2024-03-24T00:59:12.362 INFO:teuthology.orchestra.run.smithi038.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-03-24T00:59:12.362 INFO:teuthology.orchestra.run.smithi038.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-03-24T00:59:12.362 INFO:teuthology.orchestra.run.smithi038.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 41/273 objects degraded (15.018%), 14 pgs degraded 2024-03-24T00:59:12.362 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-03-24T00:59:12.362 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-03-24T00:59:12.363 INFO:teuthology.orchestra.run.smithi038.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-03-24T00:59:12.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.364+0000 7fe468200700 1 -- 172.21.15.38:0/457977469 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe44806dfb0 msgr2=0x7fe448070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:12.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.364+0000 7fe468200700 1 --2- 172.21.15.38:0/457977469 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe44806dfb0 0x7fe448070470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009fd0 tx=0x7fe44c01a040 comp rx=0 tx=0).stop 2024-03-24T00:59:12.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.364+0000 7fe468200700 1 -- 172.21.15.38:0/457977469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe460102ef0 msgr2=0x7fe46019ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:12.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.364+0000 7fe468200700 1 --2- 172.21.15.38:0/457977469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe460102ef0 0x7fe46019ea00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe45400d8d0 tx=0x7fe45400dbe0 comp rx=0 tx=0).stop 2024-03-24T00:59:12.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.364+0000 7fe468200700 1 -- 172.21.15.38:0/457977469 shutdown_connections 2024-03-24T00:59:12.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.364+0000 7fe468200700 1 --2- 172.21.15.38:0/457977469 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe44806dfb0 0x7fe448070470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:12.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.364+0000 7fe468200700 1 --2- 172.21.15.38:0/457977469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe460102ef0 0x7fe46019ea00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:12.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.364+0000 7fe468200700 1 --2- 172.21.15.38:0/457977469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4601038a0 0x7fe46019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:12.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.364+0000 7fe468200700 1 -- 172.21.15.38:0/457977469 >> 172.21.15.38:0/457977469 conn(0x7fe4600fe780 msgr2=0x7fe4600ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:12.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.364+0000 7fe468200700 1 -- 172.21.15.38:0/457977469 shutdown_connections 2024-03-24T00:59:12.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:12.364+0000 7fe468200700 1 -- 172.21.15.38:0/457977469 wait complete. 2024-03-24T00:59:12.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:12 smithi038 ceph-mon[112108]: pgmap v187: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 41/273 objects degraded (15.018%) 2024-03-24T00:59:12.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:12.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:59:12.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:12 smithi038 ceph-mon[112108]: Health check update: Degraded data redundancy: 41/273 objects degraded (15.018%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T00:59:12.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:12 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/457977469' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:59:12.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:12 smithi088 ceph-mon[87351]: pgmap v187: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 41/273 objects degraded (15.018%) 2024-03-24T00:59:12.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:12 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:12.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:12 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:59:12.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:12 smithi088 ceph-mon[87351]: Health check update: Degraded data redundancy: 41/273 objects degraded (15.018%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T00:59:12.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:12 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/457977469' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:59:13.806 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:13 smithi088 ceph-mon[87351]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:13.823 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:13 smithi038 ceph-mon[112108]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:14.857 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:14 smithi088 ceph-mon[87351]: pgmap v188: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 41/273 objects degraded (15.018%) 2024-03-24T00:59:14.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:14 smithi038 ceph-mon[112108]: pgmap v188: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 41/273 objects degraded (15.018%) 2024-03-24T00:59:15.930 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:15 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T00:59:15.498+0000 7f708dfa37c0 -1 osd.5 75 log_to_monitors true 2024-03-24T00:59:16.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:15 smithi038 ceph-mon[112108]: pgmap v189: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s; 41/273 objects degraded (15.018%) 2024-03-24T00:59:16.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:15 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:16.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:15 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:16.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:15 smithi038 ceph-mon[112108]: from='osd.5 [v2:172.21.15.88:6816/3428886242,v1:172.21.15.88:6817/3428886242]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T00:59:16.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:15 smithi038 ceph-mon[112108]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T00:59:16.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:15 smithi088 ceph-mon[87351]: pgmap v189: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 879 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s; 41/273 objects degraded (15.018%) 2024-03-24T00:59:16.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:15 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:16.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:15 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:16.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:15 smithi088 ceph-mon[87351]: from='osd.5 [v2:172.21.15.88:6816/3428886242,v1:172.21.15.88:6817/3428886242]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T00:59:16.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:15 smithi088 ceph-mon[87351]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T00:59:17.680 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 00:59:17 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T00:59:17.543+0000 7f7078400700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T00:59:17.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:17 smithi088 ceph-mon[87351]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T00:59:17.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:17 smithi088 ceph-mon[87351]: osdmap e78: 6 total, 5 up, 6 in 2024-03-24T00:59:17.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:17 smithi088 ceph-mon[87351]: from='osd.5 [v2:172.21.15.88:6816/3428886242,v1:172.21.15.88:6817/3428886242]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:59:17.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:17 smithi088 ceph-mon[87351]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:59:17.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:17 smithi088 ceph-mon[87351]: pgmap v191: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 41/273 objects degraded (15.018%) 2024-03-24T00:59:17.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:17.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:17.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:17 smithi038 ceph-mon[112108]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T00:59:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:17 smithi038 ceph-mon[112108]: osdmap e78: 6 total, 5 up, 6 in 2024-03-24T00:59:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:17 smithi038 ceph-mon[112108]: from='osd.5 [v2:172.21.15.88:6816/3428886242,v1:172.21.15.88:6817/3428886242]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:59:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:17 smithi038 ceph-mon[112108]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-03-24T00:59:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:17 smithi038 ceph-mon[112108]: pgmap v191: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 41/273 objects degraded (15.018%) 2024-03-24T00:59:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:17.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:18.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:18 smithi038 ceph-mon[112108]: from='osd.5 ' entity='osd.5' 2024-03-24T00:59:18.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:18 smithi088 ceph-mon[87351]: from='osd.5 ' entity='osd.5' 2024-03-24T00:59:19.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:19 smithi088 ceph-mon[87351]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:59:19.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:19 smithi088 ceph-mon[87351]: osd.5 [v2:172.21.15.88:6816/3428886242,v1:172.21.15.88:6817/3428886242] boot 2024-03-24T00:59:19.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:19 smithi088 ceph-mon[87351]: osdmap e79: 6 total, 6 up, 6 in 2024-03-24T00:59:19.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:19 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:59:19.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:19 smithi088 ceph-mon[87351]: pgmap v193: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 41/273 objects degraded (15.018%) 2024-03-24T00:59:19.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:19 smithi088 ceph-mon[87351]: osdmap e80: 6 total, 6 up, 6 in 2024-03-24T00:59:19.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:19 smithi038 ceph-mon[112108]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T00:59:19.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:19 smithi038 ceph-mon[112108]: osd.5 [v2:172.21.15.88:6816/3428886242,v1:172.21.15.88:6817/3428886242] boot 2024-03-24T00:59:19.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:19 smithi038 ceph-mon[112108]: osdmap e79: 6 total, 6 up, 6 in 2024-03-24T00:59:19.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:19 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T00:59:19.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:19 smithi038 ceph-mon[112108]: pgmap v193: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 195 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 41/273 objects degraded (15.018%) 2024-03-24T00:59:19.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:19 smithi038 ceph-mon[112108]: osdmap e80: 6 total, 6 up, 6 in 2024-03-24T00:59:21.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:20 smithi038 ceph-mon[112108]: Health check update: Degraded data redundancy: 24/273 objects degraded (8.791%), 9 pgs degraded (PG_DEGRADED) 2024-03-24T00:59:21.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:20 smithi088 ceph-mon[87351]: Health check update: Degraded data redundancy: 24/273 objects degraded (8.791%), 9 pgs degraded (PG_DEGRADED) 2024-03-24T00:59:22.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:21 smithi038 ceph-mon[112108]: pgmap v195: 65 pgs: 9 active+undersized+degraded, 11 active+undersized, 45 active+clean; 195 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 24/273 objects degraded (8.791%) 2024-03-24T00:59:22.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:21 smithi088 ceph-mon[87351]: pgmap v195: 65 pgs: 9 active+undersized+degraded, 11 active+undersized, 45 active+clean; 195 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 24/273 objects degraded (8.791%) 2024-03-24T00:59:23.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:23.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:23.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:59:23.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: pgmap v196: 65 pgs: 2 active+undersized+degraded, 1 active+undersized, 62 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 3/273 objects degraded (1.099%) 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: Upgrade: Setting container_image for all osd 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-24T00:59:23.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-24T00:59:23.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-24T00:59:23.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-24T00:59:23.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-24T00:59:23.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-24T00:59:23.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-24T00:59:23.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-24T00:59:23.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-24T00:59:23.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: Upgrade: Setting require_osd_release to 19 squid 2024-03-24T00:59:23.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:23 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-24T00:59:23.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:23.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:23.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:59:23.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:59:23.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:23.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: pgmap v196: 65 pgs: 2 active+undersized+degraded, 1 active+undersized, 62 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 3/273 objects degraded (1.099%) 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: Upgrade: Setting container_image for all osd 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-24T00:59:23.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-24T00:59:23.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-24T00:59:23.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-24T00:59:23.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-24T00:59:23.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-24T00:59:23.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-24T00:59:23.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: Upgrade: Setting require_osd_release to 19 squid 2024-03-24T00:59:23.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:23 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-24T00:59:24.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:24 smithi038 ceph-mon[112108]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-24T00:59:24.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:24 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-24T00:59:24.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:24 smithi038 ceph-mon[112108]: osdmap e81: 6 total, 6 up, 6 in 2024-03-24T00:59:24.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:24 smithi038 ceph-mon[112108]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-24T00:59:24.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:24 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:24.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:24 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-24T00:59:24.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:24 smithi088 ceph-mon[87351]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-24T00:59:24.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:24 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-24T00:59:24.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:24 smithi088 ceph-mon[87351]: osdmap e81: 6 total, 6 up, 6 in 2024-03-24T00:59:24.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:24 smithi088 ceph-mon[87351]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-24T00:59:24.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:24 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:24.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:24 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-24T00:59:25.494 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[112104]: 2024-03-24T00:59:25.362+0000 7f3254a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T00:59:25.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: osdmap e82: 6 total, 6 up, 6 in 2024-03-24T00:59:25.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: pgmap v199: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:25.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-24T00:59:25.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-24T00:59:25.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:active} 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T00:59:25.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-24T00:59:25.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-24T00:59:25.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: osdmap e83: 6 total, 6 up, 6 in 2024-03-24T00:59:25.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.88:6824/2738193611,v1:172.21.15.88:6825/2738193611] up:boot 2024-03-24T00:59:25.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6828/2851856830,v1:172.21.15.38:6829/2851856830] up:boot 2024-03-24T00:59:25.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-24T00:59:25.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T00:59:25.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:25 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:59:25.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: osdmap e82: 6 total, 6 up, 6 in 2024-03-24T00:59:25.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: pgmap v199: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:25.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-24T00:59:25.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-24T00:59:25.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi088.yhksbr=up:active} 2024-03-24T00:59:25.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:25.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:59:25.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:59:25.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:25.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: osdmap e83: 6 total, 6 up, 6 in 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.88:6824/2738193611,v1:172.21.15.88:6825/2738193611] up:boot 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6828/2851856830,v1:172.21.15.38:6829/2851856830] up:boot 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-24T00:59:25.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T00:59:25.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:25 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T00:59:26.816 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:26 smithi038 ceph-mon[112108]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/273 objects degraded (1.099%), 2 pgs degraded) 2024-03-24T00:59:26.817 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:26 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6826/3718992353,v1:172.21.15.38:6827/3718992353] up:boot 2024-03-24T00:59:26.817 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:26 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.88:6826/983744224,v1:172.21.15.88:6827/983744224] up:boot 2024-03-24T00:59:26.817 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:26 smithi038 ceph-mon[112108]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T00:59:26.817 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:26 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:59:26.817 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:26 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T00:59:26.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:26 smithi088 ceph-mon[87351]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/273 objects degraded (1.099%), 2 pgs degraded) 2024-03-24T00:59:26.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:26 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6826/3718992353,v1:172.21.15.38:6827/3718992353] up:boot 2024-03-24T00:59:26.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:26 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.88:6826/983744224,v1:172.21.15.88:6827/983744224] up:boot 2024-03-24T00:59:26.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:26 smithi088 ceph-mon[87351]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T00:59:26.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:26 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:59:26.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:26 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T00:59:27.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:27.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:59:27.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:27 smithi088 ceph-mon[87351]: pgmap v201: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-24T00:59:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:59:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:27 smithi038 ceph-mon[112108]: pgmap v201: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-24T00:59:29.099 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:28 smithi038 ceph-mon[112108]: Upgrade: Updating mds.cephfs.smithi038.yhkmsb 2024-03-24T00:59:29.100 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:28 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:29.100 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:28 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.yhkmsb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:59:29.100 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:28 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:59:29.100 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:28 smithi038 ceph-mon[112108]: Deploying daemon mds.cephfs.smithi038.yhkmsb on smithi038 2024-03-24T00:59:29.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:28 smithi088 ceph-mon[87351]: Upgrade: Updating mds.cephfs.smithi038.yhkmsb 2024-03-24T00:59:29.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:28 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:29.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:28 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.yhkmsb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:59:29.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:28 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:59:29.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:28 smithi088 ceph-mon[87351]: Deploying daemon mds.cephfs.smithi038.yhkmsb on smithi038 2024-03-24T00:59:29.967 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:29 smithi038 ceph-mon[112108]: pgmap v202: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-24T00:59:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:29 smithi088 ceph-mon[87351]: pgmap v202: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-24T00:59:32.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:31 smithi038 ceph-mon[112108]: pgmap v203: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-24T00:59:32.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:31 smithi038 ceph-mon[112108]: osdmap e84: 6 total, 6 up, 6 in 2024-03-24T00:59:32.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:31 smithi038 ceph-mon[112108]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-24T00:59:32.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:31 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:59:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:31 smithi088 ceph-mon[87351]: pgmap v203: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-24T00:59:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:31 smithi088 ceph-mon[87351]: osdmap e84: 6 total, 6 up, 6 in 2024-03-24T00:59:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:31 smithi088 ceph-mon[87351]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-24T00:59:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:31 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:59:33.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:32 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:33.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:32 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:34.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:33 smithi038 ceph-mon[112108]: pgmap v205: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T00:59:34.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:33 smithi088 ceph-mon[87351]: pgmap v205: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T00:59:36.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:35 smithi038 ceph-mon[112108]: pgmap v206: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 439 B/s rd, 0 op/s 2024-03-24T00:59:36.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:35 smithi088 ceph-mon[87351]: pgmap v206: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 439 B/s rd, 0 op/s 2024-03-24T00:59:37.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:37.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:37.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:37.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:37 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] up:boot 2024-03-24T00:59:37.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:37 smithi088 ceph-mon[87351]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T00:59:37.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:37 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:59:37.434 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:37.434 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:37.434 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:37.435 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:37 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] up:boot 2024-03-24T00:59:37.435 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:37 smithi038 ceph-mon[112108]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T00:59:37.435 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:37 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T00:59:38.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:38 smithi088 ceph-mon[87351]: pgmap v207: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-24T00:59:38.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:38 smithi038 ceph-mon[112108]: pgmap v207: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-24T00:59:40.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:40 smithi038 ceph-mon[112108]: pgmap v208: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:40.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:40 smithi088 ceph-mon[87351]: pgmap v208: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:42.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:42 smithi088 ceph-mon[87351]: pgmap v209: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:42.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:59:42.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.540+0000 7f79bba00700 1 -- 172.21.15.38:0/418198712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b40ffa50 msgr2=0x7f79b40ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:42.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.540+0000 7f79bba00700 1 --2- 172.21.15.38:0/418198712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b40ffa50 0x7f79b40ffe30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009910 tx=0x7f79a8009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:42.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:42 smithi038 ceph-mon[112108]: pgmap v209: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:42.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:59:42.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.540+0000 7f79bba00700 1 -- 172.21.15.38:0/418198712 shutdown_connections 2024-03-24T00:59:42.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.540+0000 7f79bba00700 1 --2- 172.21.15.38:0/418198712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b4100370 0x7f79b4104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:42.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.540+0000 7f79bba00700 1 --2- 172.21.15.38:0/418198712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b40ffa50 0x7f79b40ffe30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:42.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.540+0000 7f79bba00700 1 -- 172.21.15.38:0/418198712 >> 172.21.15.38:0/418198712 conn(0x7f79b40fb4b0 msgr2=0x7f79b40fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:42.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.540+0000 7f79bba00700 1 -- 172.21.15.38:0/418198712 shutdown_connections 2024-03-24T00:59:42.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.540+0000 7f79bba00700 1 -- 172.21.15.38:0/418198712 wait complete. 2024-03-24T00:59:42.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.541+0000 7f79bba00700 1 Processor -- start 2024-03-24T00:59:42.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.542+0000 7f79bba00700 1 -- start start 2024-03-24T00:59:42.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.542+0000 7f79bba00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b40ffa50 0x7f79b419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:42.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.542+0000 7f79bba00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b4100370 0x7f79b419d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:42.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.542+0000 7f79bba00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b419dc10 con 0x7f79b40ffa50 2024-03-24T00:59:42.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.542+0000 7f79bba00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b41a18e0 con 0x7f79b4100370 2024-03-24T00:59:42.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.542+0000 7f79b9000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b40ffa50 0x7f79b419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:42.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.543+0000 7f79b9000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b40ffa50 0x7f79b419cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39770/0 (socket says 172.21.15.38:39770) 2024-03-24T00:59:42.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.543+0000 7f79b9000700 1 -- 172.21.15.38:0/23831194 learned_addr learned my addr 172.21.15.38:0/23831194 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:42.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.543+0000 7f79b3e00700 1 --2- 172.21.15.38:0/23831194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b4100370 0x7f79b419d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:42.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.543+0000 7f79b9000700 1 -- 172.21.15.38:0/23831194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b4100370 msgr2=0x7f79b419d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:42.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.543+0000 7f79b9000700 1 --2- 172.21.15.38:0/23831194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b4100370 0x7f79b419d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:42.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.543+0000 7f79b9000700 1 -- 172.21.15.38:0/23831194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a80095f0 con 0x7f79b40ffa50 2024-03-24T00:59:42.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.544+0000 7f79b9000700 1 --2- 172.21.15.38:0/23831194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b40ffa50 0x7f79b419cff0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005f10 tx=0x7f79a800b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:42.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.544+0000 7f79b1600700 1 -- 172.21.15.38:0/23831194 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79a801d070 con 0x7f79b40ffa50 2024-03-24T00:59:42.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.544+0000 7f79b1600700 1 -- 172.21.15.38:0/23831194 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79a80052a0 con 0x7f79b40ffa50 2024-03-24T00:59:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.544+0000 7f79bba00700 1 -- 172.21.15.38:0/23831194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b41a1b60 con 0x7f79b40ffa50 2024-03-24T00:59:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.544+0000 7f79b1600700 1 -- 172.21.15.38:0/23831194 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79a8003e50 con 0x7f79b40ffa50 2024-03-24T00:59:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.544+0000 7f79bba00700 1 -- 172.21.15.38:0/23831194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b41a2050 con 0x7f79b40ffa50 2024-03-24T00:59:42.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.545+0000 7f79bba00700 1 -- 172.21.15.38:0/23831194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b40698b0 con 0x7f79b40ffa50 2024-03-24T00:59:42.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.547+0000 7f79b1600700 1 -- 172.21.15.38:0/23831194 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f79a800f6f0 con 0x7f79b40ffa50 2024-03-24T00:59:42.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.547+0000 7f79b1600700 1 --2- 172.21.15.38:0/23831194 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f799c06dfa0 0x7f799c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:42.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.548+0000 7f79b1600700 1 -- 172.21.15.38:0/23831194 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f79a808f130 con 0x7f79b40ffa50 2024-03-24T00:59:42.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.548+0000 7f79b3e00700 1 --2- 172.21.15.38:0/23831194 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f799c06dfa0 0x7f799c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:42.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.548+0000 7f79b3e00700 1 --2- 172.21.15.38:0/23831194 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f799c06dfa0 0x7f799c070460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f79b419e550 tx=0x7f79a0009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:42.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.555+0000 7f79b1600700 1 -- 172.21.15.38:0/23831194 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f79a80584b0 con 0x7f79b40ffa50 2024-03-24T00:59:42.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.845+0000 7f79bba00700 1 -- 172.21.15.38:0/23831194 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79b41a2330 con 0x7f799c06dfa0 2024-03-24T00:59:42.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.847+0000 7f79b1600700 1 -- 172.21.15.38:0/23831194 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f79b41a2330 con 0x7f799c06dfa0 2024-03-24T00:59:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.851+0000 7f79bba00700 1 -- 172.21.15.38:0/23831194 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f799c06dfa0 msgr2=0x7f799c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:42.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.851+0000 7f79bba00700 1 --2- 172.21.15.38:0/23831194 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f799c06dfa0 0x7f799c070460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f79b419e550 tx=0x7f79a0009260 comp rx=0 tx=0).stop 2024-03-24T00:59:42.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.851+0000 7f79bba00700 1 -- 172.21.15.38:0/23831194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b40ffa50 msgr2=0x7f79b419cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:42.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.851+0000 7f79bba00700 1 --2- 172.21.15.38:0/23831194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b40ffa50 0x7f79b419cff0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005f10 tx=0x7f79a800b5a0 comp rx=0 tx=0).stop 2024-03-24T00:59:42.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.851+0000 7f79bba00700 1 -- 172.21.15.38:0/23831194 shutdown_connections 2024-03-24T00:59:42.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.851+0000 7f79bba00700 1 --2- 172.21.15.38:0/23831194 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f799c06dfa0 0x7f799c070460 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:42.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.851+0000 7f79bba00700 1 --2- 172.21.15.38:0/23831194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79b40ffa50 0x7f79b419cff0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:42.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.851+0000 7f79bba00700 1 --2- 172.21.15.38:0/23831194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b4100370 0x7f79b419d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:42.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.851+0000 7f79bba00700 1 -- 172.21.15.38:0/23831194 >> 172.21.15.38:0/23831194 conn(0x7f79b40fb4b0 msgr2=0x7f79b410e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:42.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.852+0000 7f79bba00700 1 -- 172.21.15.38:0/23831194 shutdown_connections 2024-03-24T00:59:42.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:42.852+0000 7f79bba00700 1 -- 172.21.15.38:0/23831194 wait complete. 2024-03-24T00:59:42.868 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T00:59:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.007+0000 7f5294800700 1 -- 172.21.15.38:0/3595820020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f528c1038a0 msgr2=0x7f528c1078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.007+0000 7f5294800700 1 --2- 172.21.15.38:0/3595820020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f528c1038a0 0x7f528c1078f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f527c009910 tx=0x7f527c009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.007+0000 7f5294800700 1 -- 172.21.15.38:0/3595820020 shutdown_connections 2024-03-24T00:59:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.007+0000 7f5294800700 1 --2- 172.21.15.38:0/3595820020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f528c1038a0 0x7f528c1078f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.007+0000 7f5294800700 1 --2- 172.21.15.38:0/3595820020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c102ef0 0x7f528c1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.007+0000 7f5294800700 1 -- 172.21.15.38:0/3595820020 >> 172.21.15.38:0/3595820020 conn(0x7f528c0fe780 msgr2=0x7f528c100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.008+0000 7f5294800700 1 -- 172.21.15.38:0/3595820020 shutdown_connections 2024-03-24T00:59:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.008+0000 7f5294800700 1 -- 172.21.15.38:0/3595820020 wait complete. 2024-03-24T00:59:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.009+0000 7f5294800700 1 Processor -- start 2024-03-24T00:59:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.009+0000 7f5294800700 1 -- start start 2024-03-24T00:59:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.009+0000 7f5294800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f528c102ef0 0x7f528c19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.009+0000 7f5294800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c1038a0 0x7f528c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.009+0000 7f5294800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c19f590 con 0x7f528c102ef0 2024-03-24T00:59:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.009+0000 7f5294800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c198ab0 con 0x7f528c1038a0 2024-03-24T00:59:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.009+0000 7f5291e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f528c102ef0 0x7f528c19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.009+0000 7f5291e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f528c102ef0 0x7f528c19ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55306/0 (socket says 172.21.15.38:55306) 2024-03-24T00:59:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.009+0000 7f5291e00700 1 -- 172.21.15.38:0/978806475 learned_addr learned my addr 172.21.15.38:0/978806475 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.010+0000 7f5291400700 1 --2- 172.21.15.38:0/978806475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c1038a0 0x7f528c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.010+0000 7f5291e00700 1 -- 172.21.15.38:0/978806475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c1038a0 msgr2=0x7f528c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.010+0000 7f5291e00700 1 --2- 172.21.15.38:0/978806475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c1038a0 0x7f528c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.010+0000 7f5291e00700 1 -- 172.21.15.38:0/978806475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c0095f0 con 0x7f528c102ef0 2024-03-24T00:59:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.011+0000 7f5291e00700 1 --2- 172.21.15.38:0/978806475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f528c102ef0 0x7f528c19ea00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f528000c930 tx=0x7f528000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.011+0000 7f5286a00700 1 -- 172.21.15.38:0/978806475 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5280007ad0 con 0x7f528c102ef0 2024-03-24T00:59:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.011+0000 7f5294800700 1 -- 172.21.15.38:0/978806475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c198d90 con 0x7f528c102ef0 2024-03-24T00:59:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.011+0000 7f5294800700 1 -- 172.21.15.38:0/978806475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c1992b0 con 0x7f528c102ef0 2024-03-24T00:59:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.011+0000 7f5286a00700 1 -- 172.21.15.38:0/978806475 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f528000ce80 con 0x7f528c102ef0 2024-03-24T00:59:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.011+0000 7f5286a00700 1 -- 172.21.15.38:0/978806475 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5280018630 con 0x7f528c102ef0 2024-03-24T00:59:43.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.012+0000 7f5294800700 1 -- 172.21.15.38:0/978806475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52700052f0 con 0x7f528c102ef0 2024-03-24T00:59:43.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.014+0000 7f5286a00700 1 -- 172.21.15.38:0/978806475 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f5280018790 con 0x7f528c102ef0 2024-03-24T00:59:43.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.016+0000 7f5286a00700 1 --2- 172.21.15.38:0/978806475 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f527406dfb0 0x7f5274070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:43.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.016+0000 7f5286a00700 1 -- 172.21.15.38:0/978806475 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f528008dc30 con 0x7f528c102ef0 2024-03-24T00:59:43.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.016+0000 7f5291400700 1 --2- 172.21.15.38:0/978806475 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f527406dfb0 0x7f5274070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:43.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.017+0000 7f5291400700 1 --2- 172.21.15.38:0/978806475 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f527406dfb0 0x7f5274070470 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f527c009fd0 tx=0x7f527c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:43.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.021+0000 7f5286a00700 1 -- 172.21.15.38:0/978806475 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5280057140 con 0x7f528c102ef0 2024-03-24T00:59:43.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.280+0000 7f5294800700 1 -- 172.21.15.38:0/978806475 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5270000bc0 con 0x7f527406dfb0 2024-03-24T00:59:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.282+0000 7f5286a00700 1 -- 172.21.15.38:0/978806475 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5270000bc0 con 0x7f527406dfb0 2024-03-24T00:59:43.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.285+0000 7f5294800700 1 -- 172.21.15.38:0/978806475 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f527406dfb0 msgr2=0x7f5274070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:43.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.285+0000 7f5294800700 1 --2- 172.21.15.38:0/978806475 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f527406dfb0 0x7f5274070470 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f527c009fd0 tx=0x7f527c01a040 comp rx=0 tx=0).stop 2024-03-24T00:59:43.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.285+0000 7f5294800700 1 -- 172.21.15.38:0/978806475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f528c102ef0 msgr2=0x7f528c19ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:43.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.285+0000 7f5294800700 1 --2- 172.21.15.38:0/978806475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f528c102ef0 0x7f528c19ea00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f528000c930 tx=0x7f528000cc40 comp rx=0 tx=0).stop 2024-03-24T00:59:43.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.285+0000 7f5294800700 1 -- 172.21.15.38:0/978806475 shutdown_connections 2024-03-24T00:59:43.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.285+0000 7f5294800700 1 --2- 172.21.15.38:0/978806475 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f527406dfb0 0x7f5274070470 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.285+0000 7f5294800700 1 --2- 172.21.15.38:0/978806475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f528c102ef0 0x7f528c19ea00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.285+0000 7f5294800700 1 --2- 172.21.15.38:0/978806475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c1038a0 0x7f528c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.285+0000 7f5294800700 1 -- 172.21.15.38:0/978806475 >> 172.21.15.38:0/978806475 conn(0x7f528c0fe780 msgr2=0x7f528c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:43.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.285+0000 7f5294800700 1 -- 172.21.15.38:0/978806475 shutdown_connections 2024-03-24T00:59:43.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.285+0000 7f5294800700 1 -- 172.21.15.38:0/978806475 wait complete. 2024-03-24T00:59:43.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.446+0000 7f0904600700 1 -- 172.21.15.38:0/2490241593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08fc101cd0 msgr2=0x7f08fc1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:43.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.446+0000 7f0904600700 1 --2- 172.21.15.38:0/2490241593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08fc101cd0 0x7f08fc1020b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009910 tx=0x7f08f0009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:43.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.447+0000 7f0904600700 1 -- 172.21.15.38:0/2490241593 shutdown_connections 2024-03-24T00:59:43.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.447+0000 7f0904600700 1 --2- 172.21.15.38:0/2490241593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc1025f0 0x7f08fc10aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.447+0000 7f0904600700 1 --2- 172.21.15.38:0/2490241593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08fc101cd0 0x7f08fc1020b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.447+0000 7f0904600700 1 -- 172.21.15.38:0/2490241593 >> 172.21.15.38:0/2490241593 conn(0x7f08fc0fb480 msgr2=0x7f08fc0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:43.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.447+0000 7f0904600700 1 -- 172.21.15.38:0/2490241593 shutdown_connections 2024-03-24T00:59:43.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.447+0000 7f0904600700 1 -- 172.21.15.38:0/2490241593 wait complete. 2024-03-24T00:59:43.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.448+0000 7f0904600700 1 Processor -- start 2024-03-24T00:59:43.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.448+0000 7f0904600700 1 -- start start 2024-03-24T00:59:43.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.449+0000 7f0904600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc101cd0 0x7f08fc19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:43.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.449+0000 7f0904600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08fc1025f0 0x7f08fc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:43.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.449+0000 7f0904600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc19d2a0 con 0x7f08fc1025f0 2024-03-24T00:59:43.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.449+0000 7f0904600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc196810 con 0x7f08fc101cd0 2024-03-24T00:59:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.449+0000 7f0901200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08fc1025f0 0x7f08fc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.449+0000 7f0901200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08fc1025f0 0x7f08fc19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55330/0 (socket says 172.21.15.38:55330) 2024-03-24T00:59:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.449+0000 7f0901200700 1 -- 172.21.15.38:0/1955454068 learned_addr learned my addr 172.21.15.38:0/1955454068 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.449+0000 7f0901c00700 1 --2- 172.21.15.38:0/1955454068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc101cd0 0x7f08fc19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.449+0000 7f0901200700 1 -- 172.21.15.38:0/1955454068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc101cd0 msgr2=0x7f08fc19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.449+0000 7f0901200700 1 --2- 172.21.15.38:0/1955454068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc101cd0 0x7f08fc19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.449+0000 7f0901200700 1 -- 172.21.15.38:0/1955454068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f00095f0 con 0x7f08fc1025f0 2024-03-24T00:59:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.450+0000 7f0901200700 1 --2- 172.21.15.38:0/1955454068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08fc1025f0 0x7f08fc19cc10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f08e800d900 tx=0x7f08e800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.450+0000 7f08fa000700 1 -- 172.21.15.38:0/1955454068 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08e8009900 con 0x7f08fc1025f0 2024-03-24T00:59:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.450+0000 7f08fa000700 1 -- 172.21.15.38:0/1955454068 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08e8010460 con 0x7f08fc1025f0 2024-03-24T00:59:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.450+0000 7f08fa000700 1 -- 172.21.15.38:0/1955454068 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08e800f5f0 con 0x7f08fc1025f0 2024-03-24T00:59:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.450+0000 7f0904600700 1 -- 172.21.15.38:0/1955454068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc196af0 con 0x7f08fc1025f0 2024-03-24T00:59:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.450+0000 7f0904600700 1 -- 172.21.15.38:0/1955454068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc197040 con 0x7f08fc1025f0 2024-03-24T00:59:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.451+0000 7f0904600700 1 -- 172.21.15.38:0/1955454068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08fc1053f0 con 0x7f08fc1025f0 2024-03-24T00:59:43.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.454+0000 7f08fa000700 1 -- 172.21.15.38:0/1955454068 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f08e80105d0 con 0x7f08fc1025f0 2024-03-24T00:59:43.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.455+0000 7f08fa000700 1 --2- 172.21.15.38:0/1955454068 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f08e406dfb0 0x7f08e4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:43.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.455+0000 7f08fa000700 1 -- 172.21.15.38:0/1955454068 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f08e808d8e0 con 0x7f08fc1025f0 2024-03-24T00:59:43.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.455+0000 7f0901c00700 1 --2- 172.21.15.38:0/1955454068 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f08e406dfb0 0x7f08e4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:43.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.456+0000 7f0901c00700 1 --2- 172.21.15.38:0/1955454068 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f08e406dfb0 0x7f08e4070470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f08f0006010 tx=0x7f08f001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.460+0000 7f08fa000700 1 -- 172.21.15.38:0/1955454068 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f08e805a810 con 0x7f08fc1025f0 2024-03-24T00:59:43.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.708+0000 7f0904600700 1 -- 172.21.15.38:0/1955454068 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f08fc1972b0 con 0x7f08e406dfb0 2024-03-24T00:59:43.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.721+0000 7f08fa000700 1 -- 172.21.15.38:0/1955454068 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f08fc1972b0 con 0x7f08e406dfb0 2024-03-24T00:59:43.722 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T00:59:43.722 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (8m) 0s ago 22m 14.0M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T00:59:43.722 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (22m) 0s ago 22m 9504k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T00:59:43.722 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (20m) 28s ago 20m 9009k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T00:59:43.722 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (4m) 0s ago 22m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T00:59:43.722 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (4m) 28s ago 20m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T00:59:43.722 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (7m) 0s ago 21m 75.4M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (14m) 0s ago 14m 11.6M - 18.2.2-952-g738c2fd6 68aff9aff3b2 a2cb4a3bb1b6 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (8s) 0s ago 14m 10.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e597065368f0 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (14m) 28s ago 14m 24.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (14m) 28s ago 14m 24.6M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (10m) 0s ago 23m 509M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (9m) 28s ago 20m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (5m) 0s ago 24m 54.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (5m) 28s ago 20m 47.4M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (9m) 0s ago 22m 7772k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (9m) 28s ago 20m 7910k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (3m) 0s ago 19m 101M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (3m) 0s ago 18m 108M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cb72ac446ba7 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (2m) 0s ago 18m 91.2M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c73f3e8276b6 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (2m) 28s ago 17m 108M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b2afaff761d8 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (78s) 28s ago 17m 104M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9a6f202043cd 2024-03-24T00:59:43.723 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (34s) 28s ago 16m 11.2M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 866388fc07f6 2024-03-24T00:59:43.724 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (8m) 0s ago 21m 73.7M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T00:59:43.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.724+0000 7f0904600700 1 -- 172.21.15.38:0/1955454068 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f08e406dfb0 msgr2=0x7f08e4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:43.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.724+0000 7f0904600700 1 --2- 172.21.15.38:0/1955454068 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f08e406dfb0 0x7f08e4070470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f08f0006010 tx=0x7f08f001a040 comp rx=0 tx=0).stop 2024-03-24T00:59:43.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.724+0000 7f0904600700 1 -- 172.21.15.38:0/1955454068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08fc1025f0 msgr2=0x7f08fc19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:43.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.724+0000 7f0904600700 1 --2- 172.21.15.38:0/1955454068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08fc1025f0 0x7f08fc19cc10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f08e800d900 tx=0x7f08e800dc10 comp rx=0 tx=0).stop 2024-03-24T00:59:43.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.724+0000 7f0904600700 1 -- 172.21.15.38:0/1955454068 shutdown_connections 2024-03-24T00:59:43.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.724+0000 7f0904600700 1 --2- 172.21.15.38:0/1955454068 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f08e406dfb0 0x7f08e4070470 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.724+0000 7f0904600700 1 --2- 172.21.15.38:0/1955454068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08fc101cd0 0x7f08fc19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.724+0000 7f0904600700 1 --2- 172.21.15.38:0/1955454068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08fc1025f0 0x7f08fc19cc10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.724+0000 7f0904600700 1 -- 172.21.15.38:0/1955454068 >> 172.21.15.38:0/1955454068 conn(0x7f08fc0fb480 msgr2=0x7f08fc102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:43.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.724+0000 7f0904600700 1 -- 172.21.15.38:0/1955454068 shutdown_connections 2024-03-24T00:59:43.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.724+0000 7f0904600700 1 -- 172.21.15.38:0/1955454068 wait complete. 2024-03-24T00:59:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.880+0000 7f3e5c400700 1 -- 172.21.15.38:0/4097654538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e54100a70 msgr2=0x7f3e54100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.880+0000 7f3e5c400700 1 --2- 172.21.15.38:0/4097654538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e54100a70 0x7f3e54100ef0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009910 tx=0x7f3e40009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.880+0000 7f3e5c400700 1 -- 172.21.15.38:0/4097654538 shutdown_connections 2024-03-24T00:59:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.880+0000 7f3e5c400700 1 --2- 172.21.15.38:0/4097654538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e54100a70 0x7f3e54100ef0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.880+0000 7f3e5c400700 1 --2- 172.21.15.38:0/4097654538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e54106a70 0x7f3e54106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.880+0000 7f3e5c400700 1 -- 172.21.15.38:0/4097654538 >> 172.21.15.38:0/4097654538 conn(0x7f3e540fc580 msgr2=0x7f3e540fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.880+0000 7f3e5c400700 1 -- 172.21.15.38:0/4097654538 shutdown_connections 2024-03-24T00:59:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.880+0000 7f3e5c400700 1 -- 172.21.15.38:0/4097654538 wait complete. 2024-03-24T00:59:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.881+0000 7f3e5c400700 1 Processor -- start 2024-03-24T00:59:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.882+0000 7f3e5c400700 1 -- start start 2024-03-24T00:59:43.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.882+0000 7f3e5c400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e54100a70 0x7f3e540730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:43.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.882+0000 7f3e5c400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e54106a70 0x7f3e540735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:43.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.882+0000 7f3e5c400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54073b30 con 0x7f3e54106a70 2024-03-24T00:59:43.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.882+0000 7f3e5c400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54073c70 con 0x7f3e54100a70 2024-03-24T00:59:43.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.882+0000 7f3e59000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e54106a70 0x7f3e540735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:43.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.883+0000 7f3e59000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e54106a70 0x7f3e540735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55358/0 (socket says 172.21.15.38:55358) 2024-03-24T00:59:43.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.883+0000 7f3e59000700 1 -- 172.21.15.38:0/2973378903 learned_addr learned my addr 172.21.15.38:0/2973378903 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:43.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.883+0000 7f3e59a00700 1 --2- 172.21.15.38:0/2973378903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e54100a70 0x7f3e540730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.883+0000 7f3e59000700 1 -- 172.21.15.38:0/2973378903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e54100a70 msgr2=0x7f3e540730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.883+0000 7f3e59000700 1 --2- 172.21.15.38:0/2973378903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e54100a70 0x7f3e540730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.883+0000 7f3e59000700 1 -- 172.21.15.38:0/2973378903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e400095f0 con 0x7f3e54106a70 2024-03-24T00:59:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.883+0000 7f3e59000700 1 --2- 172.21.15.38:0/2973378903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e54106a70 0x7f3e540735f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3e40006010 tx=0x7f3e4000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.884+0000 7f3e52000700 1 -- 172.21.15.38:0/2973378903 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4001d070 con 0x7f3e54106a70 2024-03-24T00:59:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.884+0000 7f3e52000700 1 -- 172.21.15.38:0/2973378903 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e4000f460 con 0x7f3e54106a70 2024-03-24T00:59:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.884+0000 7f3e52000700 1 -- 172.21.15.38:0/2973378903 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e40017630 con 0x7f3e54106a70 2024-03-24T00:59:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.884+0000 7f3e5c400700 1 -- 172.21.15.38:0/2973378903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e54073ef0 con 0x7f3e54106a70 2024-03-24T00:59:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.884+0000 7f3e5c400700 1 -- 172.21.15.38:0/2973378903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e541a2ed0 con 0x7f3e54106a70 2024-03-24T00:59:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.885+0000 7f3e5c400700 1 -- 172.21.15.38:0/2973378903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e54109090 con 0x7f3e54106a70 2024-03-24T00:59:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.886+0000 7f3e52000700 1 -- 172.21.15.38:0/2973378903 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f3e4000f5d0 con 0x7f3e54106a70 2024-03-24T00:59:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.887+0000 7f3e52000700 1 --2- 172.21.15.38:0/2973378903 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3e3c06dfa0 0x7f3e3c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.887+0000 7f3e52000700 1 -- 172.21.15.38:0/2973378903 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3e4008f210 con 0x7f3e54106a70 2024-03-24T00:59:43.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.888+0000 7f3e59a00700 1 --2- 172.21.15.38:0/2973378903 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3e3c06dfa0 0x7f3e3c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.889+0000 7f3e59a00700 1 --2- 172.21.15.38:0/2973378903 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3e3c06dfa0 0x7f3e3c070460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3e48006fd0 tx=0x7f3e48008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:43.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:43.892+0000 7f3e52000700 1 -- 172.21.15.38:0/2973378903 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3e40058770 con 0x7f3e54106a70 2024-03-24T00:59:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.224+0000 7f3e5c400700 1 -- 172.21.15.38:0/2973378903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3e54108a70 con 0x7f3e54106a70 2024-03-24T00:59:44.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.225+0000 7f3e52000700 1 -- 172.21.15.38:0/2973378903 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f3e4005be30 con 0x7f3e54106a70 2024-03-24T00:59:44.226 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:59:44.226 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T00:59:44.226 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:59:44.226 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:59:44.226 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T00:59:44.226 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T00:59:44.226 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:59:44.226 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T00:59:44.227 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-24T00:59:44.227 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:59:44.227 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T00:59:44.227 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 3, 2024-03-24T00:59:44.227 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 1 2024-03-24T00:59:44.227 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T00:59:44.227 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T00:59:44.227 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 3, 2024-03-24T00:59:44.227 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 11 2024-03-24T00:59:44.227 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T00:59:44.227 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:59:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.227+0000 7f3e5c400700 1 -- 172.21.15.38:0/2973378903 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3e3c06dfa0 msgr2=0x7f3e3c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.228+0000 7f3e5c400700 1 --2- 172.21.15.38:0/2973378903 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3e3c06dfa0 0x7f3e3c070460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3e48006fd0 tx=0x7f3e48008040 comp rx=0 tx=0).stop 2024-03-24T00:59:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.228+0000 7f3e5c400700 1 -- 172.21.15.38:0/2973378903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e54106a70 msgr2=0x7f3e540735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.228+0000 7f3e5c400700 1 --2- 172.21.15.38:0/2973378903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e54106a70 0x7f3e540735f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3e40006010 tx=0x7f3e4000be20 comp rx=0 tx=0).stop 2024-03-24T00:59:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.228+0000 7f3e5c400700 1 -- 172.21.15.38:0/2973378903 shutdown_connections 2024-03-24T00:59:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.228+0000 7f3e5c400700 1 --2- 172.21.15.38:0/2973378903 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f3e3c06dfa0 0x7f3e3c070460 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.228+0000 7f3e5c400700 1 --2- 172.21.15.38:0/2973378903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e54100a70 0x7f3e540730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.228+0000 7f3e5c400700 1 --2- 172.21.15.38:0/2973378903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e54106a70 0x7f3e540735f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.228+0000 7f3e5c400700 1 -- 172.21.15.38:0/2973378903 >> 172.21.15.38:0/2973378903 conn(0x7f3e540fc580 msgr2=0x7f3e54105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.228+0000 7f3e5c400700 1 -- 172.21.15.38:0/2973378903 shutdown_connections 2024-03-24T00:59:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.228+0000 7f3e5c400700 1 -- 172.21.15.38:0/2973378903 wait complete. 2024-03-24T00:59:44.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.381+0000 7f083ec00700 1 -- 172.21.15.38:0/1506146269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0838102530 msgr2=0x7f083810aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:44.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.381+0000 7f083ec00700 1 --2- 172.21.15.38:0/1506146269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0838102530 0x7f083810aa20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0824009910 tx=0x7f0824009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:44.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.381+0000 7f083ec00700 1 -- 172.21.15.38:0/1506146269 shutdown_connections 2024-03-24T00:59:44.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.381+0000 7f083ec00700 1 --2- 172.21.15.38:0/1506146269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0838102530 0x7f083810aa20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.381+0000 7f083ec00700 1 --2- 172.21.15.38:0/1506146269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0838101c10 0x7f0838101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.381+0000 7f083ec00700 1 -- 172.21.15.38:0/1506146269 >> 172.21.15.38:0/1506146269 conn(0x7f0838076640 msgr2=0x7f08380fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:44.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.381+0000 7f083ec00700 1 -- 172.21.15.38:0/1506146269 shutdown_connections 2024-03-24T00:59:44.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.381+0000 7f083ec00700 1 -- 172.21.15.38:0/1506146269 wait complete. 2024-03-24T00:59:44.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.382+0000 7f083ec00700 1 Processor -- start 2024-03-24T00:59:44.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.382+0000 7f083ec00700 1 -- start start 2024-03-24T00:59:44.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.383+0000 7f083ec00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0838101c10 0x7f08380ff870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:44.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.383+0000 7f083ec00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0838102530 0x7f08380ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:44.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.383+0000 7f083ec00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08381002f0 con 0x7f0838101c10 2024-03-24T00:59:44.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.383+0000 7f083ec00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0838100430 con 0x7f0838102530 2024-03-24T00:59:44.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.383+0000 7f0837e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0838101c10 0x7f08380ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:44.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.383+0000 7f0837e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0838101c10 0x7f08380ff870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55380/0 (socket says 172.21.15.38:55380) 2024-03-24T00:59:44.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.383+0000 7f0837e00700 1 -- 172.21.15.38:0/3374459031 learned_addr learned my addr 172.21.15.38:0/3374459031 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:44.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.384+0000 7f0837400700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0838102530 0x7f08380ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:44.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.384+0000 7f0837e00700 1 -- 172.21.15.38:0/3374459031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0838102530 msgr2=0x7f08380ffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:44.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.384+0000 7f0837e00700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0838102530 0x7f08380ffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.384+0000 7f0837e00700 1 -- 172.21.15.38:0/3374459031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08240095f0 con 0x7f0838101c10 2024-03-24T00:59:44.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.384+0000 7f0837400700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0838102530 0x7f08380ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:59:44.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.384+0000 7f0837e00700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0838101c10 0x7f08380ff870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f082800d8d0 tx=0x7f082800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:44.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.384+0000 7f0834c00700 1 -- 172.21.15.38:0/3374459031 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0828009940 con 0x7f0838101c10 2024-03-24T00:59:44.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.384+0000 7f0834c00700 1 -- 172.21.15.38:0/3374459031 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0828010460 con 0x7f0838101c10 2024-03-24T00:59:44.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.384+0000 7f083ec00700 1 -- 172.21.15.38:0/3374459031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08381a2bb0 con 0x7f0838101c10 2024-03-24T00:59:44.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.384+0000 7f0834c00700 1 -- 172.21.15.38:0/3374459031 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f082800f5f0 con 0x7f0838101c10 2024-03-24T00:59:44.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.384+0000 7f083ec00700 1 -- 172.21.15.38:0/3374459031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08381a2f10 con 0x7f0838101c10 2024-03-24T00:59:44.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.386+0000 7f083ec00700 1 -- 172.21.15.38:0/3374459031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0838068d60 con 0x7f0838101c10 2024-03-24T00:59:44.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.387+0000 7f0834c00700 1 -- 172.21.15.38:0/3374459031 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f08280105d0 con 0x7f0838101c10 2024-03-24T00:59:44.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.387+0000 7f0834c00700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f081c06dfb0 0x7f081c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:44.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.388+0000 7f0834c00700 1 -- 172.21.15.38:0/3374459031 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f082808de70 con 0x7f0838101c10 2024-03-24T00:59:44.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.388+0000 7f0837400700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f081c06dfb0 0x7f081c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:44.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.388+0000 7f0837400700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f081c06dfb0 0x7f081c070470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0824006010 tx=0x7f082400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:44.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.393+0000 7f0834c00700 1 -- 172.21.15.38:0/3374459031 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f08280571f0 con 0x7f0838101c10 2024-03-24T00:59:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:44 smithi038 ceph-mon[112108]: pgmap v210: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:44 smithi038 ceph-mon[112108]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:44 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:44 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:44 smithi038 ceph-mon[112108]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.676+0000 7f083ec00700 1 -- 172.21.15.38:0/3374459031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0838105280 con 0x7f0838101c10 2024-03-24T00:59:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.677+0000 7f0834c00700 1 -- 172.21.15.38:0/3374459031 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1867 (secure 0 0 0) 0x7f082805a8b0 con 0x7f0838101c10 2024-03-24T00:59:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:e25 2024-03-24T00:59:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T00:59:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:59:44.678 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 22 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:59:25.362766+0000 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T00:59:44.679 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 83 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:up {} 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:failed 0,1 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T00:59:44.680 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:59:44.681 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:59:44.681 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-03-24T00:59:44.681 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T00:59:44.681 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{-1:34434} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.38:6828/2851856830,v1:172.21.15.38:6829/2851856830] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:59:44.681 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{-1:34446} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T00:59:44.681 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{-1:44293} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2738193611,v1:172.21.15.88:6825/2738193611] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:59:44.681 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{-1:44303} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/983744224,v1:172.21.15.88:6827/983744224] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T00:59:44.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:44 smithi088 ceph-mon[87351]: pgmap v210: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:44.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:44 smithi088 ceph-mon[87351]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:44.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:44 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:44.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:44 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:44.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:44 smithi088 ceph-mon[87351]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:44.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.679+0000 7f083ec00700 1 -- 172.21.15.38:0/3374459031 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f081c06dfb0 msgr2=0x7f081c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:44.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.679+0000 7f083ec00700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f081c06dfb0 0x7f081c070470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0824006010 tx=0x7f082400b540 comp rx=0 tx=0).stop 2024-03-24T00:59:44.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.679+0000 7f083ec00700 1 -- 172.21.15.38:0/3374459031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0838101c10 msgr2=0x7f08380ff870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:44.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.679+0000 7f083ec00700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0838101c10 0x7f08380ff870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f082800d8d0 tx=0x7f082800dc90 comp rx=0 tx=0).stop 2024-03-24T00:59:44.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.680+0000 7f083ec00700 1 -- 172.21.15.38:0/3374459031 shutdown_connections 2024-03-24T00:59:44.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.680+0000 7f083ec00700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f081c06dfb0 0x7f081c070470 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.680+0000 7f083ec00700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0838101c10 0x7f08380ff870 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.680+0000 7f083ec00700 1 --2- 172.21.15.38:0/3374459031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0838102530 0x7f08380ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.680+0000 7f083ec00700 1 -- 172.21.15.38:0/3374459031 >> 172.21.15.38:0/3374459031 conn(0x7f0838076640 msgr2=0x7f0838102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:44.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.680+0000 7f083ec00700 1 -- 172.21.15.38:0/3374459031 shutdown_connections 2024-03-24T00:59:44.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.680+0000 7f083ec00700 1 -- 172.21.15.38:0/3374459031 wait complete. 2024-03-24T00:59:44.683 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-03-24T00:59:44.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.830+0000 7fcb95600700 1 -- 172.21.15.38:0/2860120249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb90105aa0 msgr2=0x7fcb90105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:44.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.830+0000 7fcb95600700 1 --2- 172.21.15.38:0/2860120249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb90105aa0 0x7fcb90105e80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009910 tx=0x7fcb80009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:44.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.830+0000 7fcb95600700 1 -- 172.21.15.38:0/2860120249 shutdown_connections 2024-03-24T00:59:44.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.830+0000 7fcb95600700 1 --2- 172.21.15.38:0/2860120249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb900ff2a0 0x7fcb900ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.830+0000 7fcb95600700 1 --2- 172.21.15.38:0/2860120249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb90105aa0 0x7fcb90105e80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.830+0000 7fcb95600700 1 -- 172.21.15.38:0/2860120249 >> 172.21.15.38:0/2860120249 conn(0x7fcb90074c80 msgr2=0x7fcb90075090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:44.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.830+0000 7fcb95600700 1 -- 172.21.15.38:0/2860120249 shutdown_connections 2024-03-24T00:59:44.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.830+0000 7fcb95600700 1 -- 172.21.15.38:0/2860120249 wait complete. 2024-03-24T00:59:44.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.832+0000 7fcb95600700 1 Processor -- start 2024-03-24T00:59:44.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.832+0000 7fcb95600700 1 -- start start 2024-03-24T00:59:44.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.832+0000 7fcb95600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb900ff2a0 0x7fcb90100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:44.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.832+0000 7fcb95600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb90105aa0 0x7fcb901009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:44.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.832+0000 7fcb95600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb90100f00 con 0x7fcb90105aa0 2024-03-24T00:59:44.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.832+0000 7fcb95600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb90101040 con 0x7fcb900ff2a0 2024-03-24T00:59:44.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.833+0000 7fcb8e000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb90105aa0 0x7fcb901009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:44.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.833+0000 7fcb8e000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb90105aa0 0x7fcb901009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55392/0 (socket says 172.21.15.38:55392) 2024-03-24T00:59:44.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.833+0000 7fcb8e000700 1 -- 172.21.15.38:0/433920014 learned_addr learned my addr 172.21.15.38:0/433920014 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:44.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.833+0000 7fcb8ea00700 1 --2- 172.21.15.38:0/433920014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb900ff2a0 0x7fcb90100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:44.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.833+0000 7fcb8e000700 1 -- 172.21.15.38:0/433920014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb900ff2a0 msgr2=0x7fcb90100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:44.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.833+0000 7fcb8e000700 1 --2- 172.21.15.38:0/433920014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb900ff2a0 0x7fcb90100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:44.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.833+0000 7fcb8e000700 1 -- 172.21.15.38:0/433920014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb800095f0 con 0x7fcb90105aa0 2024-03-24T00:59:44.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.834+0000 7fcb8e000700 1 --2- 172.21.15.38:0/433920014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb90105aa0 0x7fcb901009c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00d8d0 tx=0x7fcb7c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:44.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.834+0000 7fcb87400700 1 -- 172.21.15.38:0/433920014 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb7c009940 con 0x7fcb90105aa0 2024-03-24T00:59:44.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.834+0000 7fcb87400700 1 -- 172.21.15.38:0/433920014 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb7c010460 con 0x7fcb90105aa0 2024-03-24T00:59:44.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.834+0000 7fcb95600700 1 -- 172.21.15.38:0/433920014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb901a2b90 con 0x7fcb90105aa0 2024-03-24T00:59:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.834+0000 7fcb95600700 1 -- 172.21.15.38:0/433920014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb901a2ef0 con 0x7fcb90105aa0 2024-03-24T00:59:44.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.835+0000 7fcb87400700 1 -- 172.21.15.38:0/433920014 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb7c00f5f0 con 0x7fcb90105aa0 2024-03-24T00:59:44.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.836+0000 7fcb95600700 1 -- 172.21.15.38:0/433920014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb700052f0 con 0x7fcb90105aa0 2024-03-24T00:59:44.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.838+0000 7fcb87400700 1 -- 172.21.15.38:0/433920014 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fcb7c00f820 con 0x7fcb90105aa0 2024-03-24T00:59:44.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.839+0000 7fcb87400700 1 --2- 172.21.15.38:0/433920014 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcb7406dfa0 0x7fcb74070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:44.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.839+0000 7fcb87400700 1 -- 172.21.15.38:0/433920014 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcb7c08dca0 con 0x7fcb90105aa0 2024-03-24T00:59:44.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.839+0000 7fcb8ea00700 1 --2- 172.21.15.38:0/433920014 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcb7406dfa0 0x7fcb74070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:44.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.840+0000 7fcb8ea00700 1 --2- 172.21.15.38:0/433920014 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcb7406dfa0 0x7fcb74070460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb800059d0 tx=0x7fcb8001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:44.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:44.848+0000 7fcb87400700 1 -- 172.21.15.38:0/433920014 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb7c057110 con 0x7fcb90105aa0 2024-03-24T00:59:45.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.101+0000 7fcb95600700 1 -- 172.21.15.38:0/433920014 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb70000bc0 con 0x7fcb7406dfa0 2024-03-24T00:59:45.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.104+0000 7fcb87400700 1 -- 172.21.15.38:0/433920014 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcb70000bc0 con 0x7fcb7406dfa0 2024-03-24T00:59:45.105 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T00:59:45.105 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T00:59:45.105 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T00:59:45.105 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T00:59:45.105 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T00:59:45.106 INFO:teuthology.orchestra.run.smithi038.stdout: "osd", 2024-03-24T00:59:45.106 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T00:59:45.106 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-03-24T00:59:45.106 INFO:teuthology.orchestra.run.smithi038.stdout: "crash" 2024-03-24T00:59:45.106 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T00:59:45.106 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "13/23 daemons upgraded", 2024-03-24T00:59:45.106 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mds daemons", 2024-03-24T00:59:45.106 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T00:59:45.106 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T00:59:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.106+0000 7fcb95600700 1 -- 172.21.15.38:0/433920014 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcb7406dfa0 msgr2=0x7fcb74070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.106+0000 7fcb95600700 1 --2- 172.21.15.38:0/433920014 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcb7406dfa0 0x7fcb74070460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb800059d0 tx=0x7fcb8001a040 comp rx=0 tx=0).stop 2024-03-24T00:59:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.106+0000 7fcb95600700 1 -- 172.21.15.38:0/433920014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb90105aa0 msgr2=0x7fcb901009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.106+0000 7fcb95600700 1 --2- 172.21.15.38:0/433920014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb90105aa0 0x7fcb901009c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00d8d0 tx=0x7fcb7c00dc90 comp rx=0 tx=0).stop 2024-03-24T00:59:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.107+0000 7fcb95600700 1 -- 172.21.15.38:0/433920014 shutdown_connections 2024-03-24T00:59:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.107+0000 7fcb95600700 1 --2- 172.21.15.38:0/433920014 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcb7406dfa0 0x7fcb74070460 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.107+0000 7fcb95600700 1 --2- 172.21.15.38:0/433920014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb900ff2a0 0x7fcb90100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.107+0000 7fcb95600700 1 --2- 172.21.15.38:0/433920014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb90105aa0 0x7fcb901009c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.107+0000 7fcb95600700 1 -- 172.21.15.38:0/433920014 >> 172.21.15.38:0/433920014 conn(0x7fcb90074c80 msgr2=0x7fcb900fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.107+0000 7fcb95600700 1 -- 172.21.15.38:0/433920014 shutdown_connections 2024-03-24T00:59:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.107+0000 7fcb95600700 1 -- 172.21.15.38:0/433920014 wait complete. 2024-03-24T00:59:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.266+0000 7f60dfe00700 1 -- 172.21.15.38:0/3499824797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e0068480 msgr2=0x7f60e0068860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.266+0000 7f60dfe00700 1 --2- 172.21.15.38:0/3499824797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e0068480 0x7f60e0068860 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009910 tx=0x7f60d0009c20 comp rx=0 tx=0).stop 2024-03-24T00:59:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.266+0000 7f60dfe00700 1 -- 172.21.15.38:0/3499824797 shutdown_connections 2024-03-24T00:59:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.266+0000 7f60dfe00700 1 --2- 172.21.15.38:0/3499824797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e0068da0 0x7f60e010cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.266+0000 7f60dfe00700 1 --2- 172.21.15.38:0/3499824797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e0068480 0x7f60e0068860 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.266+0000 7f60dfe00700 1 -- 172.21.15.38:0/3499824797 >> 172.21.15.38:0/3499824797 conn(0x7f60e0075370 msgr2=0x7f60e0075780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.266+0000 7f60dfe00700 1 -- 172.21.15.38:0/3499824797 shutdown_connections 2024-03-24T00:59:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.266+0000 7f60dfe00700 1 -- 172.21.15.38:0/3499824797 wait complete. 2024-03-24T00:59:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.267+0000 7f60dfe00700 1 Processor -- start 2024-03-24T00:59:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.267+0000 7f60dfe00700 1 -- start start 2024-03-24T00:59:45.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60dfe00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e0068480 0x7f60e019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:45.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60dfe00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e0068da0 0x7f60e019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:45.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60dfe00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e019f2a0 con 0x7f60e0068da0 2024-03-24T00:59:45.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60dfe00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e0198810 con 0x7f60e0068480 2024-03-24T00:59:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60d7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e0068da0 0x7f60e019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60d7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e0068da0 0x7f60e019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55404/0 (socket says 172.21.15.38:55404) 2024-03-24T00:59:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60d7e00700 1 -- 172.21.15.38:0/159159694 learned_addr learned my addr 172.21.15.38:0/159159694 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T00:59:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60dea00700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e0068480 0x7f60e019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60d7e00700 1 -- 172.21.15.38:0/159159694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e0068480 msgr2=0x7f60e019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60d7e00700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e0068480 0x7f60e019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60d7e00700 1 -- 172.21.15.38:0/159159694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d00095f0 con 0x7f60e0068da0 2024-03-24T00:59:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.268+0000 7f60dea00700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e0068480 0x7f60e019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T00:59:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.269+0000 7f60d7e00700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e0068da0 0x7f60e019ec10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f60c800c8a0 tx=0x7f60c800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:45.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.269+0000 7f60d7400700 1 -- 172.21.15.38:0/159159694 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60c800cea0 con 0x7f60e0068da0 2024-03-24T00:59:45.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.269+0000 7f60dfe00700 1 -- 172.21.15.38:0/159159694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e0198af0 con 0x7f60e0068da0 2024-03-24T00:59:45.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.269+0000 7f60dfe00700 1 -- 172.21.15.38:0/159159694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e0199040 con 0x7f60e0068da0 2024-03-24T00:59:45.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.269+0000 7f60d7400700 1 -- 172.21.15.38:0/159159694 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60c8004830 con 0x7f60e0068da0 2024-03-24T00:59:45.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.269+0000 7f60d7400700 1 -- 172.21.15.38:0/159159694 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60c80055b0 con 0x7f60e0068da0 2024-03-24T00:59:45.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.270+0000 7f60dfe00700 1 -- 172.21.15.38:0/159159694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60bc0052f0 con 0x7f60e0068da0 2024-03-24T00:59:45.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.271+0000 7f60d7400700 1 -- 172.21.15.38:0/159159694 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f60c8007ad0 con 0x7f60e0068da0 2024-03-24T00:59:45.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.272+0000 7f60d7400700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f60c406dfb0 0x7f60c4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T00:59:45.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.273+0000 7f60d7400700 1 -- 172.21.15.38:0/159159694 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f60c808da00 con 0x7f60e0068da0 2024-03-24T00:59:45.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.273+0000 7f60dea00700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f60c406dfb0 0x7f60c4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T00:59:45.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.274+0000 7f60dea00700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f60c406dfb0 0x7f60c4070470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f60d0006010 tx=0x7f60d000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T00:59:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.278+0000 7f60d7400700 1 -- 172.21.15.38:0/159159694 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f60c805a930 con 0x7f60e0068da0 2024-03-24T00:59:45.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:45 smithi038 ceph-mon[112108]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:45.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:45 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2973378903' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:45.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:45 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3374459031' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:59:45.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.585+0000 7f60dfe00700 1 -- 172.21.15.38:0/159159694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f60bc005160 con 0x7f60e0068da0 2024-03-24T00:59:45.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.586+0000 7f60d7400700 1 -- 172.21.15.38:0/159159694 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f60c805a4c0 con 0x7f60e0068da0 2024-03-24T00:59:45.587 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-24T00:59:45.587 INFO:teuthology.orchestra.run.smithi038.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-24T00:59:45.587 INFO:teuthology.orchestra.run.smithi038.stdout: fs cephfs is degraded 2024-03-24T00:59:45.587 INFO:teuthology.orchestra.run.smithi038.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-24T00:59:45.588 INFO:teuthology.orchestra.run.smithi038.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-24T00:59:45.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.588+0000 7f60dfe00700 1 -- 172.21.15.38:0/159159694 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f60c406dfb0 msgr2=0x7f60c4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:45.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.588+0000 7f60dfe00700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f60c406dfb0 0x7f60c4070470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f60d0006010 tx=0x7f60d000b540 comp rx=0 tx=0).stop 2024-03-24T00:59:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.588+0000 7f60dfe00700 1 -- 172.21.15.38:0/159159694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e0068da0 msgr2=0x7f60e019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T00:59:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.588+0000 7f60dfe00700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e0068da0 0x7f60e019ec10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f60c800c8a0 tx=0x7f60c800cc60 comp rx=0 tx=0).stop 2024-03-24T00:59:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.589+0000 7f60dfe00700 1 -- 172.21.15.38:0/159159694 shutdown_connections 2024-03-24T00:59:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.589+0000 7f60dfe00700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f60c406dfb0 0x7f60c4070470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.589+0000 7f60dfe00700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e0068480 0x7f60e019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.589+0000 7f60dfe00700 1 --2- 172.21.15.38:0/159159694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e0068da0 0x7f60e019ec10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T00:59:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.589+0000 7f60dfe00700 1 -- 172.21.15.38:0/159159694 >> 172.21.15.38:0/159159694 conn(0x7f60e0075370 msgr2=0x7f60e00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T00:59:45.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.589+0000 7f60dfe00700 1 -- 172.21.15.38:0/159159694 shutdown_connections 2024-03-24T00:59:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T00:59:45.589+0000 7f60dfe00700 1 -- 172.21.15.38:0/159159694 wait complete. 2024-03-24T00:59:45.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:45 smithi088 ceph-mon[87351]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:45.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:45 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2973378903' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:45.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:45 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3374459031' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T00:59:46.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:46 smithi038 ceph-mon[112108]: pgmap v211: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:46.428 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:46 smithi038 ceph-mon[112108]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:46.428 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:46 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:46.428 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:46 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:46.428 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:46 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/159159694' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:59:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:46 smithi088 ceph-mon[87351]: pgmap v211: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:46 smithi088 ceph-mon[87351]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T00:59:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:46 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:46 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:46.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:46 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/159159694' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T00:59:48.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:48 smithi038 ceph-mon[112108]: pgmap v212: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:48 smithi088 ceph-mon[87351]: pgmap v212: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:50.557 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:50 smithi038 ceph-mon[112108]: pgmap v213: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:50.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:50 smithi088 ceph-mon[87351]: pgmap v213: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:52.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:52 smithi038 ceph-mon[112108]: pgmap v214: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:52.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:52.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:59:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:59:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:52.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:52 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:52.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:52 smithi088 ceph-mon[87351]: pgmap v214: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:59:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T00:59:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T00:59:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:52.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:52 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T00:59:54.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:54 smithi088 ceph-mon[87351]: pgmap v215: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:54.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:54 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:54.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:54 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.xnpqhp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:59:54.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:54 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:59:54.354 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:54 smithi038 ceph-mon[112108]: pgmap v215: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:54.354 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:54 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T00:59:54.354 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:54 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.xnpqhp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T00:59:54.354 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:54 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T00:59:55.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:55 smithi038 ceph-mon[112108]: Upgrade: Updating mds.cephfs.smithi038.xnpqhp 2024-03-24T00:59:55.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:55 smithi038 ceph-mon[112108]: Deploying daemon mds.cephfs.smithi038.xnpqhp on smithi038 2024-03-24T00:59:55.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:55 smithi088 ceph-mon[87351]: Upgrade: Updating mds.cephfs.smithi038.xnpqhp 2024-03-24T00:59:55.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:55 smithi088 ceph-mon[87351]: Deploying daemon mds.cephfs.smithi038.xnpqhp on smithi038 2024-03-24T00:59:56.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:56 smithi088 ceph-mon[87351]: pgmap v216: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:56.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:56 smithi038 ceph-mon[112108]: pgmap v216: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:57.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:59:57.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T00:59:58.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:58 smithi088 ceph-mon[87351]: pgmap v217: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:58.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:58 smithi088 ceph-mon[87351]: osdmap e85: 6 total, 6 up, 6 in 2024-03-24T00:59:58.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 00:59:58 smithi088 ceph-mon[87351]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-24T00:59:58.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:58 smithi038 ceph-mon[112108]: pgmap v217: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T00:59:58.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:58 smithi038 ceph-mon[112108]: osdmap e85: 6 total, 6 up, 6 in 2024-03-24T00:59:58.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 00:59:58 smithi038 ceph-mon[112108]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-24T01:00:00.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:00 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[112104]: 2024-03-24T00:59:59.999+0000 7f3257c00700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-24T01:00:00.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:00 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[112104]: 2024-03-24T00:59:59.999+0000 7f3257c00700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-24T01:00:00.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:00 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[112104]: 2024-03-24T00:59:59.999+0000 7f3257c00700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-03-24T01:00:00.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:00 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[112104]: 2024-03-24T00:59:59.999+0000 7f3257c00700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-24T01:00:00.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:00 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[112104]: 2024-03-24T01:00:00.000+0000 7f3257c00700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-03-24T01:00:00.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:00 smithi038 ceph-mon[112108]: pgmap v219: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:00.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:00 smithi038 ceph-mon[112108]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-24T01:00:00.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:00 smithi038 ceph-mon[112108]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-24T01:00:00.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:00 smithi038 ceph-mon[112108]: fs cephfs is degraded 2024-03-24T01:00:00.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:00 smithi038 ceph-mon[112108]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-24T01:00:00.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:00 smithi038 ceph-mon[112108]: fs cephfs is offline because no MDS is active for it. 2024-03-24T01:00:00.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:00 smithi088 ceph-mon[87351]: pgmap v219: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:00.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:00 smithi088 ceph-mon[87351]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-24T01:00:00.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:00 smithi088 ceph-mon[87351]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-24T01:00:00.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:00 smithi088 ceph-mon[87351]: fs cephfs is degraded 2024-03-24T01:00:00.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:00 smithi088 ceph-mon[87351]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-24T01:00:00.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:00 smithi088 ceph-mon[87351]: fs cephfs is offline because no MDS is active for it. 2024-03-24T01:00:01.330 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:01 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T01:00:01.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:01 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T01:00:02.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:02 smithi088 ceph-mon[87351]: pgmap v220: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:02.585 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:02 smithi038 ceph-mon[112108]: pgmap v220: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:04.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:04 smithi088 ceph-mon[87351]: pgmap v221: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:04.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:04 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:04.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:04 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:04.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:04 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] up:boot 2024-03-24T01:00:04.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:04 smithi088 ceph-mon[87351]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T01:00:04.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:04 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T01:00:04.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:04 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:00:04.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:04 smithi038 ceph-mon[112108]: pgmap v221: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:04.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:04 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:04.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:04 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:04.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:04 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] up:boot 2024-03-24T01:00:04.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:04 smithi038 ceph-mon[112108]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T01:00:04.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:04 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T01:00:04.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:04 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:00:06.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:06 smithi088 ceph-mon[87351]: pgmap v222: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:06.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:06 smithi038 ceph-mon[112108]: pgmap v222: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:08.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:08 smithi088 ceph-mon[87351]: pgmap v223: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:08.559 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:08 smithi038 ceph-mon[112108]: pgmap v223: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:10.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:10 smithi088 ceph-mon[87351]: pgmap v224: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:10.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:10 smithi038 ceph-mon[112108]: pgmap v224: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:11.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:11.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:11 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:11.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:11.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:11 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:12.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:12 smithi038 ceph-mon[112108]: pgmap v225: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:12.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:00:12.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:12 smithi088 ceph-mon[87351]: pgmap v225: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:12.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:12 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:00:13.525 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:13 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:13.525 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:13 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:13.525 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:13 smithi088 ceph-mon[87351]: pgmap v226: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:13.832 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:13 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:13.833 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:13 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:13.833 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:13 smithi038 ceph-mon[112108]: pgmap v226: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:15.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.760+0000 7f8d48400700 1 -- 172.21.15.38:0/2861110505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40106a70 msgr2=0x7f8d40106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:15.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.760+0000 7f8d48400700 1 --2- 172.21.15.38:0/2861110505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40106a70 0x7f8d40106e50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009910 tx=0x7f8d34009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:15.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.760+0000 7f8d48400700 1 -- 172.21.15.38:0/2861110505 shutdown_connections 2024-03-24T01:00:15.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.760+0000 7f8d48400700 1 --2- 172.21.15.38:0/2861110505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40100a70 0x7f8d40100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:15.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.760+0000 7f8d48400700 1 --2- 172.21.15.38:0/2861110505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40106a70 0x7f8d40106e50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:15.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.760+0000 7f8d48400700 1 -- 172.21.15.38:0/2861110505 >> 172.21.15.38:0/2861110505 conn(0x7f8d400fc5a0 msgr2=0x7f8d400fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:15.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.761+0000 7f8d48400700 1 -- 172.21.15.38:0/2861110505 shutdown_connections 2024-03-24T01:00:15.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.761+0000 7f8d48400700 1 -- 172.21.15.38:0/2861110505 wait complete. 2024-03-24T01:00:15.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.762+0000 7f8d48400700 1 Processor -- start 2024-03-24T01:00:15.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.762+0000 7f8d48400700 1 -- start start 2024-03-24T01:00:15.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.763+0000 7f8d48400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40100a70 0x7f8d4019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:15.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.763+0000 7f8d48400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40106a70 0x7f8d4019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:15.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.763+0000 7f8d48400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4019f3d0 con 0x7f8d40100a70 2024-03-24T01:00:15.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.763+0000 7f8d48400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40198a00 con 0x7f8d40106a70 2024-03-24T01:00:15.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.763+0000 7f8d45a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40100a70 0x7f8d4019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:15.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.763+0000 7f8d45a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40100a70 0x7f8d4019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54206/0 (socket says 172.21.15.38:54206) 2024-03-24T01:00:15.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.763+0000 7f8d45a00700 1 -- 172.21.15.38:0/880610037 learned_addr learned my addr 172.21.15.38:0/880610037 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:15.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.763+0000 7f8d45000700 1 --2- 172.21.15.38:0/880610037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40106a70 0x7f8d4019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:15.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.763+0000 7f8d45a00700 1 -- 172.21.15.38:0/880610037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40106a70 msgr2=0x7f8d4019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:15.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.763+0000 7f8d45a00700 1 --2- 172.21.15.38:0/880610037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40106a70 0x7f8d4019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:15.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.763+0000 7f8d45a00700 1 -- 172.21.15.38:0/880610037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340095f0 con 0x7f8d40100a70 2024-03-24T01:00:15.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.764+0000 7f8d45a00700 1 --2- 172.21.15.38:0/880610037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40100a70 0x7f8d4019e870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8d340092e0 tx=0x7f8d3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:15.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.764+0000 7f8d3e000700 1 -- 172.21.15.38:0/880610037 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d3401d070 con 0x7f8d40100a70 2024-03-24T01:00:15.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.764+0000 7f8d3e000700 1 -- 172.21.15.38:0/880610037 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d340056e0 con 0x7f8d40100a70 2024-03-24T01:00:15.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.764+0000 7f8d3e000700 1 -- 172.21.15.38:0/880610037 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d3400fe80 con 0x7f8d40100a70 2024-03-24T01:00:15.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.764+0000 7f8d48400700 1 -- 172.21.15.38:0/880610037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d40198c80 con 0x7f8d40100a70 2024-03-24T01:00:15.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.764+0000 7f8d48400700 1 -- 172.21.15.38:0/880610037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d40199170 con 0x7f8d40100a70 2024-03-24T01:00:15.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.765+0000 7f8d48400700 1 -- 172.21.15.38:0/880610037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d40104d20 con 0x7f8d40100a70 2024-03-24T01:00:15.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.767+0000 7f8d3e000700 1 -- 172.21.15.38:0/880610037 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f8d3400f6f0 con 0x7f8d40100a70 2024-03-24T01:00:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.768+0000 7f8d3e000700 1 --2- 172.21.15.38:0/880610037 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8d2806dfb0 0x7f8d28070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.768+0000 7f8d3e000700 1 -- 172.21.15.38:0/880610037 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8d3408f290 con 0x7f8d40100a70 2024-03-24T01:00:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.768+0000 7f8d45000700 1 --2- 172.21.15.38:0/880610037 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8d2806dfb0 0x7f8d28070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.769+0000 7f8d45000700 1 --2- 172.21.15.38:0/880610037 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8d2806dfb0 0x7f8d28070470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c005950 tx=0x7f8d2c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:15.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:15.773+0000 7f8d3e000700 1 -- 172.21.15.38:0/880610037 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8d3405c140 con 0x7f8d40100a70 2024-03-24T01:00:16.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.029+0000 7f8d48400700 1 -- 172.21.15.38:0/880610037 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d40199e70 con 0x7f8d2806dfb0 2024-03-24T01:00:16.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:15 smithi038 ceph-mon[112108]: pgmap v227: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.031+0000 7f8d3e000700 1 -- 172.21.15.38:0/880610037 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8d40199e70 con 0x7f8d2806dfb0 2024-03-24T01:00:16.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.034+0000 7f8d48400700 1 -- 172.21.15.38:0/880610037 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8d2806dfb0 msgr2=0x7f8d28070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:16.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.034+0000 7f8d48400700 1 --2- 172.21.15.38:0/880610037 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8d2806dfb0 0x7f8d28070470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c005950 tx=0x7f8d2c0058e0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.034+0000 7f8d48400700 1 -- 172.21.15.38:0/880610037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40100a70 msgr2=0x7f8d4019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:16.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.034+0000 7f8d48400700 1 --2- 172.21.15.38:0/880610037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40100a70 0x7f8d4019e870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8d340092e0 tx=0x7f8d3400be20 comp rx=0 tx=0).stop 2024-03-24T01:00:16.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.034+0000 7f8d48400700 1 -- 172.21.15.38:0/880610037 shutdown_connections 2024-03-24T01:00:16.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.034+0000 7f8d48400700 1 --2- 172.21.15.38:0/880610037 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8d2806dfb0 0x7f8d28070470 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.034+0000 7f8d48400700 1 --2- 172.21.15.38:0/880610037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40100a70 0x7f8d4019e870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.034+0000 7f8d48400700 1 --2- 172.21.15.38:0/880610037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40106a70 0x7f8d4019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.034+0000 7f8d48400700 1 -- 172.21.15.38:0/880610037 >> 172.21.15.38:0/880610037 conn(0x7f8d400fc5a0 msgr2=0x7f8d40073260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:16.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.034+0000 7f8d48400700 1 -- 172.21.15.38:0/880610037 shutdown_connections 2024-03-24T01:00:16.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.035+0000 7f8d48400700 1 -- 172.21.15.38:0/880610037 wait complete. 2024-03-24T01:00:16.052 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T01:00:16.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:15 smithi088 ceph-mon[87351]: pgmap v227: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:16.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.194+0000 7f9af8400700 1 -- 172.21.15.38:0/2963727920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af0102ee0 msgr2=0x7f9af01032c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:16.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.194+0000 7f9af8400700 1 --2- 172.21.15.38:0/2963727920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af0102ee0 0x7f9af01032c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009910 tx=0x7f9ae4009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:16.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.195+0000 7f9af8400700 1 -- 172.21.15.38:0/2963727920 shutdown_connections 2024-03-24T01:00:16.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.195+0000 7f9af8400700 1 --2- 172.21.15.38:0/2963727920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9af0103890 0x7f9af01078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.195+0000 7f9af8400700 1 --2- 172.21.15.38:0/2963727920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af0102ee0 0x7f9af01032c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.195+0000 7f9af8400700 1 -- 172.21.15.38:0/2963727920 >> 172.21.15.38:0/2963727920 conn(0x7f9af00fe750 msgr2=0x7f9af0100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:16.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.195+0000 7f9af8400700 1 -- 172.21.15.38:0/2963727920 shutdown_connections 2024-03-24T01:00:16.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.195+0000 7f9af8400700 1 -- 172.21.15.38:0/2963727920 wait complete. 2024-03-24T01:00:16.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.196+0000 7f9af8400700 1 Processor -- start 2024-03-24T01:00:16.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.196+0000 7f9af8400700 1 -- start start 2024-03-24T01:00:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.197+0000 7f9af8400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9af0102ee0 0x7f9af019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.197+0000 7f9af8400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af0103890 0x7f9af019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.197+0000 7f9af8400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af019f490 con 0x7f9af0103890 2024-03-24T01:00:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.197+0000 7f9af8400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af0198a50 con 0x7f9af0102ee0 2024-03-24T01:00:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.197+0000 7f9af5000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af0103890 0x7f9af019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.197+0000 7f9af5000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af0103890 0x7f9af019ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54228/0 (socket says 172.21.15.38:54228) 2024-03-24T01:00:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.197+0000 7f9af5000700 1 -- 172.21.15.38:0/3962479033 learned_addr learned my addr 172.21.15.38:0/3962479033 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.198+0000 7f9af5a00700 1 --2- 172.21.15.38:0/3962479033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9af0102ee0 0x7f9af019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.198+0000 7f9af5000700 1 -- 172.21.15.38:0/3962479033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9af0102ee0 msgr2=0x7f9af019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.198+0000 7f9af5000700 1 --2- 172.21.15.38:0/3962479033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9af0102ee0 0x7f9af019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.198+0000 7f9af5000700 1 -- 172.21.15.38:0/3962479033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae40095f0 con 0x7f9af0103890 2024-03-24T01:00:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.198+0000 7f9af5000700 1 --2- 172.21.15.38:0/3962479033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af0103890 0x7f9af019ee00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00d900 tx=0x7f9adc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.198+0000 7f9aee000700 1 -- 172.21.15.38:0/3962479033 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9adc009900 con 0x7f9af0103890 2024-03-24T01:00:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.199+0000 7f9aee000700 1 -- 172.21.15.38:0/3962479033 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9adc010460 con 0x7f9af0103890 2024-03-24T01:00:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.199+0000 7f9af8400700 1 -- 172.21.15.38:0/3962479033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af0198d30 con 0x7f9af0103890 2024-03-24T01:00:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.199+0000 7f9aee000700 1 -- 172.21.15.38:0/3962479033 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9adc00f5f0 con 0x7f9af0103890 2024-03-24T01:00:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.199+0000 7f9af8400700 1 -- 172.21.15.38:0/3962479033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af0199280 con 0x7f9af0103890 2024-03-24T01:00:16.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.200+0000 7f9af8400700 1 -- 172.21.15.38:0/3962479033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af010b280 con 0x7f9af0103890 2024-03-24T01:00:16.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.201+0000 7f9aee000700 1 -- 172.21.15.38:0/3962479033 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f9adc0105d0 con 0x7f9af0103890 2024-03-24T01:00:16.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.202+0000 7f9aee000700 1 --2- 172.21.15.38:0/3962479033 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ad806dfb0 0x7f9ad8070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:16.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.202+0000 7f9aee000700 1 -- 172.21.15.38:0/3962479033 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9adc08dcc0 con 0x7f9af0103890 2024-03-24T01:00:16.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.203+0000 7f9af5a00700 1 --2- 172.21.15.38:0/3962479033 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ad806dfb0 0x7f9ad8070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:16.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.203+0000 7f9af5a00700 1 --2- 172.21.15.38:0/3962479033 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ad806dfb0 0x7f9ad8070470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400b5c0 tx=0x7f9ae401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:16.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.208+0000 7f9aee000700 1 -- 172.21.15.38:0/3962479033 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9adc093020 con 0x7f9af0103890 2024-03-24T01:00:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.522+0000 7f9af8400700 1 -- 172.21.15.38:0/3962479033 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9af0199f10 con 0x7f9ad806dfb0 2024-03-24T01:00:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.524+0000 7f9aee000700 1 -- 172.21.15.38:0/3962479033 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9af0199f10 con 0x7f9ad806dfb0 2024-03-24T01:00:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.527+0000 7f9af8400700 1 -- 172.21.15.38:0/3962479033 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ad806dfb0 msgr2=0x7f9ad8070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.527+0000 7f9af8400700 1 --2- 172.21.15.38:0/3962479033 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ad806dfb0 0x7f9ad8070470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400b5c0 tx=0x7f9ae401a040 comp rx=0 tx=0).stop 2024-03-24T01:00:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.527+0000 7f9af8400700 1 -- 172.21.15.38:0/3962479033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af0103890 msgr2=0x7f9af019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.527+0000 7f9af8400700 1 --2- 172.21.15.38:0/3962479033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af0103890 0x7f9af019ee00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00d900 tx=0x7f9adc00dcc0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.527+0000 7f9af8400700 1 -- 172.21.15.38:0/3962479033 shutdown_connections 2024-03-24T01:00:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.527+0000 7f9af8400700 1 --2- 172.21.15.38:0/3962479033 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f9ad806dfb0 0x7f9ad8070470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.527+0000 7f9af8400700 1 --2- 172.21.15.38:0/3962479033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9af0102ee0 0x7f9af019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.527+0000 7f9af8400700 1 --2- 172.21.15.38:0/3962479033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af0103890 0x7f9af019ee00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.527+0000 7f9af8400700 1 -- 172.21.15.38:0/3962479033 >> 172.21.15.38:0/3962479033 conn(0x7f9af00fe750 msgr2=0x7f9af00ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.527+0000 7f9af8400700 1 -- 172.21.15.38:0/3962479033 shutdown_connections 2024-03-24T01:00:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.527+0000 7f9af8400700 1 -- 172.21.15.38:0/3962479033 wait complete. 2024-03-24T01:00:16.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.672+0000 7f8694e00700 1 -- 172.21.15.38:0/2665733718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86900731e0 msgr2=0x7f86900735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:16.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.672+0000 7f8694e00700 1 --2- 172.21.15.38:0/2665733718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86900731e0 0x7f86900735c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8680009910 tx=0x7f8680009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:16.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.672+0000 7f8694e00700 1 -- 172.21.15.38:0/2665733718 shutdown_connections 2024-03-24T01:00:16.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.672+0000 7f8694e00700 1 --2- 172.21.15.38:0/2665733718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8690073b00 0x7f869010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.672+0000 7f8694e00700 1 --2- 172.21.15.38:0/2665733718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86900731e0 0x7f86900735c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.672+0000 7f8694e00700 1 -- 172.21.15.38:0/2665733718 >> 172.21.15.38:0/2665733718 conn(0x7f86900fc3f0 msgr2=0x7f86900fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:16.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.672+0000 7f8694e00700 1 -- 172.21.15.38:0/2665733718 shutdown_connections 2024-03-24T01:00:16.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.672+0000 7f8694e00700 1 -- 172.21.15.38:0/2665733718 wait complete. 2024-03-24T01:00:16.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.673+0000 7f8694e00700 1 Processor -- start 2024-03-24T01:00:16.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.673+0000 7f8694e00700 1 -- start start 2024-03-24T01:00:16.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f8694e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86900731e0 0x7f8690198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:16.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f8694e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8690073b00 0x7f8690198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:16.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f8694e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8690199600 con 0x7f8690073b00 2024-03-24T01:00:16.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f8694e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869019d350 con 0x7f86900731e0 2024-03-24T01:00:16.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f868ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8690073b00 0x7f8690198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:16.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f868ea00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8690073b00 0x7f8690198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54244/0 (socket says 172.21.15.38:54244) 2024-03-24T01:00:16.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f868ea00700 1 -- 172.21.15.38:0/2483756471 learned_addr learned my addr 172.21.15.38:0/2483756471 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f868f400700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86900731e0 0x7f8690198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f868ea00700 1 -- 172.21.15.38:0/2483756471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86900731e0 msgr2=0x7f8690198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f868ea00700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86900731e0 0x7f8690198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f868ea00700 1 -- 172.21.15.38:0/2483756471 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800095f0 con 0x7f8690073b00 2024-03-24T01:00:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.674+0000 7f868f400700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86900731e0 0x7f8690198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T01:00:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.675+0000 7f868ea00700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8690073b00 0x7f8690198f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f867c00c8a0 tx=0x7f867c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.675+0000 7f8687e00700 1 -- 172.21.15.38:0/2483756471 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f867c00cea0 con 0x7f8690073b00 2024-03-24T01:00:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.675+0000 7f8687e00700 1 -- 172.21.15.38:0/2483756471 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f867c004830 con 0x7f8690073b00 2024-03-24T01:00:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.675+0000 7f8694e00700 1 -- 172.21.15.38:0/2483756471 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869019d630 con 0x7f8690073b00 2024-03-24T01:00:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.675+0000 7f8687e00700 1 -- 172.21.15.38:0/2483756471 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f867c0055b0 con 0x7f8690073b00 2024-03-24T01:00:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.675+0000 7f8694e00700 1 -- 172.21.15.38:0/2483756471 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869019db80 con 0x7f8690073b00 2024-03-24T01:00:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.676+0000 7f8694e00700 1 -- 172.21.15.38:0/2483756471 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8690074890 con 0x7f8690073b00 2024-03-24T01:00:16.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.678+0000 7f8687e00700 1 -- 172.21.15.38:0/2483756471 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f867c0049a0 con 0x7f8690073b00 2024-03-24T01:00:16.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.679+0000 7f8687e00700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f867406dfb0 0x7f8674070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:16.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.679+0000 7f8687e00700 1 -- 172.21.15.38:0/2483756471 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f867c08da20 con 0x7f8690073b00 2024-03-24T01:00:16.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.679+0000 7f868f400700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f867406dfb0 0x7f8674070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:16.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.680+0000 7f868f400700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f867406dfb0 0x7f8674070470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8680006010 tx=0x7f868000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:16.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.686+0000 7f8687e00700 1 -- 172.21.15.38:0/2483756471 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f867c05a950 con 0x7f8690073b00 2024-03-24T01:00:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.920+0000 7f8694e00700 1 -- 172.21.15.38:0/2483756471 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8690107520 con 0x7f867406dfb0 2024-03-24T01:00:16.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.934+0000 7f8687e00700 1 -- 172.21.15.38:0/2483756471 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f8680005480 con 0x7f867406dfb0 2024-03-24T01:00:16.934 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T01:00:16.934 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (8m) 6s ago 22m 14.0M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T01:00:16.934 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (22m) 6s ago 22m 9537k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T01:00:16.935 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (21m) 62s ago 21m 9009k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T01:00:16.935 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (4m) 6s ago 22m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T01:00:16.935 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (4m) 62s ago 21m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T01:00:16.935 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (7m) 6s ago 22m 75.4M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T01:00:16.935 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (14s) 6s ago 15m 10.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b78588cbee24 2024-03-24T01:00:16.935 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (41s) 6s ago 15m 11.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e597065368f0 2024-03-24T01:00:16.935 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (15m) 62s ago 15m 24.7M - 18.2.2-952-g738c2fd6 68aff9aff3b2 406242b4e562 2024-03-24T01:00:16.935 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (15m) 62s ago 15m 24.6M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T01:00:16.935 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (11m) 6s ago 24m 510M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T01:00:16.935 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (10m) 62s ago 21m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T01:00:16.935 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (6m) 6s ago 24m 55.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T01:00:16.936 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (5m) 62s ago 21m 47.4M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T01:00:16.936 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (9m) 6s ago 22m 7960k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T01:00:16.936 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (9m) 62s ago 21m 7910k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T01:00:16.936 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (4m) 6s ago 20m 102M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T01:00:16.936 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (3m) 6s ago 19m 108M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cb72ac446ba7 2024-03-24T01:00:16.936 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (3m) 6s ago 18m 91.4M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c73f3e8276b6 2024-03-24T01:00:16.936 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (2m) 62s ago 18m 108M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b2afaff761d8 2024-03-24T01:00:16.936 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (111s) 62s ago 17m 104M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9a6f202043cd 2024-03-24T01:00:16.936 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (68s) 62s ago 17m 11.2M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 866388fc07f6 2024-03-24T01:00:16.936 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (9m) 6s ago 21m 74.1M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T01:00:16.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.936+0000 7f8694e00700 1 -- 172.21.15.38:0/2483756471 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f867406dfb0 msgr2=0x7f8674070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:16.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.936+0000 7f8694e00700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f867406dfb0 0x7f8674070470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8680006010 tx=0x7f868000b540 comp rx=0 tx=0).stop 2024-03-24T01:00:16.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.936+0000 7f8694e00700 1 -- 172.21.15.38:0/2483756471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8690073b00 msgr2=0x7f8690198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:16.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.937+0000 7f8694e00700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8690073b00 0x7f8690198f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f867c00c8a0 tx=0x7f867c00cc60 comp rx=0 tx=0).stop 2024-03-24T01:00:16.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.937+0000 7f8694e00700 1 -- 172.21.15.38:0/2483756471 shutdown_connections 2024-03-24T01:00:16.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.937+0000 7f8694e00700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f867406dfb0 0x7f8674070470 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.937+0000 7f8694e00700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86900731e0 0x7f8690198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.937+0000 7f8694e00700 1 --2- 172.21.15.38:0/2483756471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8690073b00 0x7f8690198f70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:16.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.937+0000 7f8694e00700 1 -- 172.21.15.38:0/2483756471 >> 172.21.15.38:0/2483756471 conn(0x7f86900fc3f0 msgr2=0x7f86901050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:16.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.937+0000 7f8694e00700 1 -- 172.21.15.38:0/2483756471 shutdown_connections 2024-03-24T01:00:16.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:16.937+0000 7f8694e00700 1 -- 172.21.15.38:0/2483756471 wait complete. 2024-03-24T01:00:17.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.089+0000 7f4fc5400700 1 -- 172.21.15.38:0/352659199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc01001b0 msgr2=0x7f4fc010ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:17.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.089+0000 7f4fc5400700 1 --2- 172.21.15.38:0/352659199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc01001b0 0x7f4fc010ce70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009910 tx=0x7f4fa8009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:17.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.089+0000 7f4fc5400700 1 -- 172.21.15.38:0/352659199 shutdown_connections 2024-03-24T01:00:17.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.089+0000 7f4fc5400700 1 --2- 172.21.15.38:0/352659199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc01001b0 0x7f4fc010ce70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.089+0000 7f4fc5400700 1 --2- 172.21.15.38:0/352659199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc00ff800 0x7f4fc00ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.089+0000 7f4fc5400700 1 -- 172.21.15.38:0/352659199 >> 172.21.15.38:0/352659199 conn(0x7f4fc0076640 msgr2=0x7f4fc00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:17.090 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:16 smithi038 ceph-mon[112108]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.089+0000 7f4fc5400700 1 -- 172.21.15.38:0/352659199 shutdown_connections 2024-03-24T01:00:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.089+0000 7f4fc5400700 1 -- 172.21.15.38:0/352659199 wait complete. 2024-03-24T01:00:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.090+0000 7f4fc5400700 1 Processor -- start 2024-03-24T01:00:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.091+0000 7f4fc5400700 1 -- start start 2024-03-24T01:00:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.091+0000 7f4fc5400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc00ff800 0x7f4fc019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.091+0000 7f4fc5400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc01001b0 0x7f4fc019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.091+0000 7f4fc5400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0198ab0 con 0x7f4fc01001b0 2024-03-24T01:00:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.091+0000 7f4fc5400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0198c20 con 0x7f4fc00ff800 2024-03-24T01:00:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.091+0000 7f4fb7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc01001b0 0x7f4fc019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:17.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.091+0000 7f4fb7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc01001b0 0x7f4fc019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54256/0 (socket says 172.21.15.38:54256) 2024-03-24T01:00:17.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.091+0000 7f4fb7e00700 1 -- 172.21.15.38:0/1590162480 learned_addr learned my addr 172.21.15.38:0/1590162480 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:17.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.092+0000 7f4fbe800700 1 --2- 172.21.15.38:0/1590162480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc00ff800 0x7f4fc019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:17.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.092+0000 7f4fb7e00700 1 -- 172.21.15.38:0/1590162480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc00ff800 msgr2=0x7f4fc019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:17.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.092+0000 7f4fb7e00700 1 --2- 172.21.15.38:0/1590162480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc00ff800 0x7f4fc019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.092+0000 7f4fb7e00700 1 -- 172.21.15.38:0/1590162480 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa80095f0 con 0x7f4fc01001b0 2024-03-24T01:00:17.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.092+0000 7f4fb7e00700 1 --2- 172.21.15.38:0/1590162480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc01001b0 0x7f4fc019ef40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80055f0 tx=0x7f4fa80056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.092+0000 7f4fb7400700 1 -- 172.21.15.38:0/1590162480 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa801d070 con 0x7f4fc01001b0 2024-03-24T01:00:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.093+0000 7f4fb7400700 1 -- 172.21.15.38:0/1590162480 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fa800f460 con 0x7f4fc01001b0 2024-03-24T01:00:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.093+0000 7f4fc5400700 1 -- 172.21.15.38:0/1590162480 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc0198f00 con 0x7f4fc01001b0 2024-03-24T01:00:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.093+0000 7f4fb7400700 1 -- 172.21.15.38:0/1590162480 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa8021620 con 0x7f4fc01001b0 2024-03-24T01:00:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.093+0000 7f4fc5400700 1 -- 172.21.15.38:0/1590162480 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc0199450 con 0x7f4fc01001b0 2024-03-24T01:00:17.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.094+0000 7f4fc5400700 1 -- 172.21.15.38:0/1590162480 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc0068d00 con 0x7f4fc01001b0 2024-03-24T01:00:17.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.095+0000 7f4fb7400700 1 -- 172.21.15.38:0/1590162480 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f4fa800f5d0 con 0x7f4fc01001b0 2024-03-24T01:00:17.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.096+0000 7f4fb7400700 1 --2- 172.21.15.38:0/1590162480 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4fa406dfb0 0x7f4fa4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:17.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.096+0000 7f4fb7400700 1 -- 172.21.15.38:0/1590162480 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4fa808f080 con 0x7f4fc01001b0 2024-03-24T01:00:17.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.096+0000 7f4fbe800700 1 --2- 172.21.15.38:0/1590162480 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4fa406dfb0 0x7f4fa4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:17.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.097+0000 7f4fbe800700 1 --2- 172.21.15.38:0/1590162480 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4fa406dfb0 0x7f4fa4070470 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0006fd0 tx=0x7f4fb0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:17.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.101+0000 7f4fb7400700 1 -- 172.21.15.38:0/1590162480 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4fa8058400 con 0x7f4fc01001b0 2024-03-24T01:00:17.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:16 smithi088 ceph-mon[87351]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:17.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.421+0000 7f4fc5400700 1 -- 172.21.15.38:0/1590162480 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4fc01076a0 con 0x7f4fc01001b0 2024-03-24T01:00:17.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.422+0000 7f4fb7400700 1 -- 172.21.15.38:0/1590162480 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f4fa805bac0 con 0x7f4fc01001b0 2024-03-24T01:00:17.422 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T01:00:17.422 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2, 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T01:00:17.423 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 18.2.2-952-g738c2fd6 (738c2fd67dbc0b17e2e99568a03974712d998229) reef (stable)": 2, 2024-03-24T01:00:17.424 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 12 2024-03-24T01:00:17.424 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T01:00:17.424 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T01:00:17.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.425+0000 7f4fc5400700 1 -- 172.21.15.38:0/1590162480 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4fa406dfb0 msgr2=0x7f4fa4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:17.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.425+0000 7f4fc5400700 1 --2- 172.21.15.38:0/1590162480 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4fa406dfb0 0x7f4fa4070470 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0006fd0 tx=0x7f4fb0008040 comp rx=0 tx=0).stop 2024-03-24T01:00:17.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.425+0000 7f4fc5400700 1 -- 172.21.15.38:0/1590162480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc01001b0 msgr2=0x7f4fc019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:17.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.425+0000 7f4fc5400700 1 --2- 172.21.15.38:0/1590162480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc01001b0 0x7f4fc019ef40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80055f0 tx=0x7f4fa80056d0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.425+0000 7f4fc5400700 1 -- 172.21.15.38:0/1590162480 shutdown_connections 2024-03-24T01:00:17.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.425+0000 7f4fc5400700 1 --2- 172.21.15.38:0/1590162480 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4fa406dfb0 0x7f4fa4070470 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.425+0000 7f4fc5400700 1 --2- 172.21.15.38:0/1590162480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc00ff800 0x7f4fc019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.425+0000 7f4fc5400700 1 --2- 172.21.15.38:0/1590162480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc01001b0 0x7f4fc019ef40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.425+0000 7f4fc5400700 1 -- 172.21.15.38:0/1590162480 >> 172.21.15.38:0/1590162480 conn(0x7f4fc0076640 msgr2=0x7f4fc00fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:17.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.425+0000 7f4fc5400700 1 -- 172.21.15.38:0/1590162480 shutdown_connections 2024-03-24T01:00:17.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.425+0000 7f4fc5400700 1 -- 172.21.15.38:0/1590162480 wait complete. 2024-03-24T01:00:17.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.583+0000 7fd059a00700 1 -- 172.21.15.38:0/1747823283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0540731e0 msgr2=0x7fd0540735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:17.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.583+0000 7fd059a00700 1 --2- 172.21.15.38:0/1747823283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0540731e0 0x7fd0540735c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd044009910 tx=0x7fd044009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:17.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.583+0000 7fd059a00700 1 -- 172.21.15.38:0/1747823283 shutdown_connections 2024-03-24T01:00:17.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.583+0000 7fd059a00700 1 --2- 172.21.15.38:0/1747823283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd054073b00 0x7fd05410cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.583+0000 7fd059a00700 1 --2- 172.21.15.38:0/1747823283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0540731e0 0x7fd0540735c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.583+0000 7fd059a00700 1 -- 172.21.15.38:0/1747823283 >> 172.21.15.38:0/1747823283 conn(0x7fd0540fc580 msgr2=0x7fd0540fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:17.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.584+0000 7fd059a00700 1 -- 172.21.15.38:0/1747823283 shutdown_connections 2024-03-24T01:00:17.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.584+0000 7fd059a00700 1 -- 172.21.15.38:0/1747823283 wait complete. 2024-03-24T01:00:17.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.585+0000 7fd059a00700 1 Processor -- start 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.585+0000 7fd059a00700 1 -- start start 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.585+0000 7fd059a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0540731e0 0x7fd05419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.585+0000 7fd059a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd054073b00 0x7fd05419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.585+0000 7fd059a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05419f470 con 0x7fd054073b00 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.585+0000 7fd059a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd054198a50 con 0x7fd0540731e0 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.585+0000 7fd04be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd054073b00 0x7fd05419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.585+0000 7fd04be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd054073b00 0x7fd05419ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54282/0 (socket says 172.21.15.38:54282) 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.585+0000 7fd04be00700 1 -- 172.21.15.38:0/3343880957 learned_addr learned my addr 172.21.15.38:0/3343880957 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.586+0000 7fd052a00700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0540731e0 0x7fd05419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.586+0000 7fd04be00700 1 -- 172.21.15.38:0/3343880957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0540731e0 msgr2=0x7fd05419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.586+0000 7fd04be00700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0540731e0 0x7fd05419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.586+0000 7fd04be00700 1 -- 172.21.15.38:0/3343880957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0440095f0 con 0x7fd054073b00 2024-03-24T01:00:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.586+0000 7fd052a00700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0540731e0 0x7fd05419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T01:00:17.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.586+0000 7fd04be00700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd054073b00 0x7fd05419ee50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00d8d0 tx=0x7fd03c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:17.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.586+0000 7fd04b400700 1 -- 172.21.15.38:0/3343880957 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd03c009940 con 0x7fd054073b00 2024-03-24T01:00:17.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.586+0000 7fd04b400700 1 -- 172.21.15.38:0/3343880957 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd03c010460 con 0x7fd054073b00 2024-03-24T01:00:17.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.586+0000 7fd04b400700 1 -- 172.21.15.38:0/3343880957 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd03c00f5f0 con 0x7fd054073b00 2024-03-24T01:00:17.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.587+0000 7fd059a00700 1 -- 172.21.15.38:0/3343880957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd054198d30 con 0x7fd054073b00 2024-03-24T01:00:17.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.587+0000 7fd059a00700 1 -- 172.21.15.38:0/3343880957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd054199280 con 0x7fd054073b00 2024-03-24T01:00:17.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.588+0000 7fd059a00700 1 -- 172.21.15.38:0/3343880957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05410a510 con 0x7fd054073b00 2024-03-24T01:00:17.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.589+0000 7fd04b400700 1 -- 172.21.15.38:0/3343880957 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd03c0105d0 con 0x7fd054073b00 2024-03-24T01:00:17.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.590+0000 7fd04b400700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd03806dfb0 0x7fd038070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:17.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.590+0000 7fd04b400700 1 -- 172.21.15.38:0/3343880957 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd03c08dd50 con 0x7fd054073b00 2024-03-24T01:00:17.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.590+0000 7fd052a00700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd03806dfb0 0x7fd038070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.591+0000 7fd052a00700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd03806dfb0 0x7fd038070470 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd044006010 tx=0x7fd04400b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:17.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.596+0000 7fd04b400700 1 -- 172.21.15.38:0/3343880957 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd03c05ac80 con 0x7fd054073b00 2024-03-24T01:00:17.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.881+0000 7fd059a00700 1 -- 172.21.15.38:0/3343880957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd054074890 con 0x7fd054073b00 2024-03-24T01:00:17.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.882+0000 7fd04b400700 1 -- 172.21.15.38:0/3343880957 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+1867 (secure 0 0 0) 0x7fd03c016760 con 0x7fd054073b00 2024-03-24T01:00:17.882 INFO:teuthology.orchestra.run.smithi038.stdout:e27 2024-03-24T01:00:17.882 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T01:00:17.882 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T01:00:17.882 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 22 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:59:25.362766+0000 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T01:00:17.883 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 83 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:up {} 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:failed 0,1 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T01:00:17.884 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T01:00:17.885 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T01:00:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:00:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:00:17.885 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-03-24T01:00:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:00:17.885 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{-1:34446} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:00:17.885 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{-1:34478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:00:17.885 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{-1:44293} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2738193611,v1:172.21.15.88:6825/2738193611] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T01:00:17.885 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{-1:44303} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/983744224,v1:172.21.15.88:6827/983744224] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T01:00:17.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.886+0000 7fd059a00700 1 -- 172.21.15.38:0/3343880957 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd03806dfb0 msgr2=0x7fd038070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:17.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.886+0000 7fd059a00700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd03806dfb0 0x7fd038070470 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd044006010 tx=0x7fd04400b540 comp rx=0 tx=0).stop 2024-03-24T01:00:17.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.886+0000 7fd059a00700 1 -- 172.21.15.38:0/3343880957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd054073b00 msgr2=0x7fd05419ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:17.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.886+0000 7fd059a00700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd054073b00 0x7fd05419ee50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00d8d0 tx=0x7fd03c00dc90 comp rx=0 tx=0).stop 2024-03-24T01:00:17.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.886+0000 7fd059a00700 1 -- 172.21.15.38:0/3343880957 shutdown_connections 2024-03-24T01:00:17.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.886+0000 7fd059a00700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd03806dfb0 0x7fd038070470 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.886+0000 7fd059a00700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0540731e0 0x7fd05419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.886+0000 7fd059a00700 1 --2- 172.21.15.38:0/3343880957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd054073b00 0x7fd05419ee50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:17.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.886+0000 7fd059a00700 1 -- 172.21.15.38:0/3343880957 >> 172.21.15.38:0/3343880957 conn(0x7fd0540fc580 msgr2=0x7fd054107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:17.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.886+0000 7fd059a00700 1 -- 172.21.15.38:0/3343880957 shutdown_connections 2024-03-24T01:00:17.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:17.886+0000 7fd059a00700 1 -- 172.21.15.38:0/3343880957 wait complete. 2024-03-24T01:00:17.888 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-03-24T01:00:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.050+0000 7fe59a600700 1 -- 172.21.15.38:0/3736148737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5941071e0 msgr2=0x7fe5941075c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.050+0000 7fe59a600700 1 --2- 172.21.15.38:0/3736148737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5941071e0 0x7fe5941075c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe588009910 tx=0x7fe588009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.051+0000 7fe59a600700 1 -- 172.21.15.38:0/3736148737 shutdown_connections 2024-03-24T01:00:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.051+0000 7fe59a600700 1 --2- 172.21.15.38:0/3736148737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe594107b90 0x7fe59410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.051+0000 7fe59a600700 1 --2- 172.21.15.38:0/3736148737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5941071e0 0x7fe5941075c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.051+0000 7fe59a600700 1 -- 172.21.15.38:0/3736148737 >> 172.21.15.38:0/3736148737 conn(0x7fe594076700 msgr2=0x7fe594078b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.051+0000 7fe59a600700 1 -- 172.21.15.38:0/3736148737 shutdown_connections 2024-03-24T01:00:18.052 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:17 smithi038 ceph-mon[112108]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:18.052 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:17 smithi038 ceph-mon[112108]: pgmap v228: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:18.052 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:17 smithi038 ceph-mon[112108]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:18.052 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:17 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1590162480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:18.052 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:18.052 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:18.052 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:00:18.052 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:00:18.052 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:17 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:18.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.051+0000 7fe59a600700 1 -- 172.21.15.38:0/3736148737 wait complete. 2024-03-24T01:00:18.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.052+0000 7fe59a600700 1 Processor -- start 2024-03-24T01:00:18.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.052+0000 7fe59a600700 1 -- start start 2024-03-24T01:00:18.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.053+0000 7fe59a600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5941071e0 0x7fe59419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:18.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.053+0000 7fe59a600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe594107b90 0x7fe59419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:18.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.053+0000 7fe59a600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59419daf0 con 0x7fe594107b90 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.053+0000 7fe59a600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5941a17c0 con 0x7fe5941071e0 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.053+0000 7fe593e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe594107b90 0x7fe59419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.053+0000 7fe593e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe594107b90 0x7fe59419d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54300/0 (socket says 172.21.15.38:54300) 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.053+0000 7fe593e00700 1 -- 172.21.15.38:0/2817531997 learned_addr learned my addr 172.21.15.38:0/2817531997 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.053+0000 7fe599200700 1 --2- 172.21.15.38:0/2817531997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5941071e0 0x7fe59419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.053+0000 7fe593e00700 1 -- 172.21.15.38:0/2817531997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5941071e0 msgr2=0x7fe59419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.053+0000 7fe593e00700 1 --2- 172.21.15.38:0/2817531997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5941071e0 0x7fe59419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.053+0000 7fe593e00700 1 -- 172.21.15.38:0/2817531997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5880095f0 con 0x7fe594107b90 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.054+0000 7fe593e00700 1 --2- 172.21.15.38:0/2817531997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe594107b90 0x7fe59419d410 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe58000e8f0 tx=0x7fe58000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.054+0000 7fe591600700 1 -- 172.21.15.38:0/2817531997 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe580009790 con 0x7fe594107b90 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.054+0000 7fe59a600700 1 -- 172.21.15.38:0/2817531997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5941a1aa0 con 0x7fe594107b90 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.054+0000 7fe591600700 1 -- 172.21.15.38:0/2817531997 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe580004cf0 con 0x7fe594107b90 2024-03-24T01:00:18.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.054+0000 7fe591600700 1 -- 172.21.15.38:0/2817531997 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe580005690 con 0x7fe594107b90 2024-03-24T01:00:18.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.054+0000 7fe59a600700 1 -- 172.21.15.38:0/2817531997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5941a1ff0 con 0x7fe594107b90 2024-03-24T01:00:18.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.055+0000 7fe59a600700 1 -- 172.21.15.38:0/2817531997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59410f4d0 con 0x7fe594107b90 2024-03-24T01:00:18.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.056+0000 7fe591600700 1 -- 172.21.15.38:0/2817531997 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fe580004750 con 0x7fe594107b90 2024-03-24T01:00:18.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.057+0000 7fe591600700 1 --2- 172.21.15.38:0/2817531997 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe57c06dfa0 0x7fe57c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:18.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.058+0000 7fe591600700 1 -- 172.21.15.38:0/2817531997 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe580014070 con 0x7fe594107b90 2024-03-24T01:00:18.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.058+0000 7fe599200700 1 --2- 172.21.15.38:0/2817531997 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe57c06dfa0 0x7fe57c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:18.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.059+0000 7fe599200700 1 --2- 172.21.15.38:0/2817531997 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe57c06dfa0 0x7fe57c070460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b760 tx=0x7fe588005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:18.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.062+0000 7fe591600700 1 -- 172.21.15.38:0/2817531997 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe5800575c0 con 0x7fe594107b90 2024-03-24T01:00:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:17 smithi088 ceph-mon[87351]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:17 smithi088 ceph-mon[87351]: pgmap v228: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:17 smithi088 ceph-mon[87351]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:17 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1590162480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:18.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:18.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:00:18.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:00:18.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:17 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:18.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.316+0000 7fe59a600700 1 -- 172.21.15.38:0/2817531997 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5941a2270 con 0x7fe57c06dfa0 2024-03-24T01:00:18.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.318+0000 7fe591600700 1 -- 172.21.15.38:0/2817531997 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe5941a2270 con 0x7fe57c06dfa0 2024-03-24T01:00:18.319 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T01:00:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T01:00:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T01:00:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T01:00:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T01:00:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "osd", 2024-03-24T01:00:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T01:00:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-03-24T01:00:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "crash" 2024-03-24T01:00:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T01:00:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "14/23 daemons upgraded", 2024-03-24T01:00:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mds daemons", 2024-03-24T01:00:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T01:00:18.320 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T01:00:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.321+0000 7fe59a600700 1 -- 172.21.15.38:0/2817531997 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe57c06dfa0 msgr2=0x7fe57c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.321+0000 7fe59a600700 1 --2- 172.21.15.38:0/2817531997 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe57c06dfa0 0x7fe57c070460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b760 tx=0x7fe588005a90 comp rx=0 tx=0).stop 2024-03-24T01:00:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.321+0000 7fe59a600700 1 -- 172.21.15.38:0/2817531997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe594107b90 msgr2=0x7fe59419d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.321+0000 7fe59a600700 1 --2- 172.21.15.38:0/2817531997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe594107b90 0x7fe59419d410 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe58000e8f0 tx=0x7fe58000ecb0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.321+0000 7fe59a600700 1 -- 172.21.15.38:0/2817531997 shutdown_connections 2024-03-24T01:00:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.322+0000 7fe59a600700 1 --2- 172.21.15.38:0/2817531997 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe57c06dfa0 0x7fe57c070460 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.322+0000 7fe59a600700 1 --2- 172.21.15.38:0/2817531997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5941071e0 0x7fe59419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.322+0000 7fe59a600700 1 --2- 172.21.15.38:0/2817531997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe594107b90 0x7fe59419d410 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.322+0000 7fe59a600700 1 -- 172.21.15.38:0/2817531997 >> 172.21.15.38:0/2817531997 conn(0x7fe594076700 msgr2=0x7fe594076ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.322+0000 7fe59a600700 1 -- 172.21.15.38:0/2817531997 shutdown_connections 2024-03-24T01:00:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.322+0000 7fe59a600700 1 -- 172.21.15.38:0/2817531997 wait complete. 2024-03-24T01:00:18.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.485+0000 7f28af200700 1 -- 172.21.15.38:0/741422353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28b0107b90 msgr2=0x7f28b010bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:18.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.485+0000 7f28af200700 1 --2- 172.21.15.38:0/741422353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28b0107b90 0x7f28b010bb30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2898009910 tx=0x7f2898009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:18.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.485+0000 7f28af200700 1 -- 172.21.15.38:0/741422353 shutdown_connections 2024-03-24T01:00:18.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.485+0000 7f28af200700 1 --2- 172.21.15.38:0/741422353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28b0107b90 0x7f28b010bb30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.485+0000 7f28af200700 1 --2- 172.21.15.38:0/741422353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b01071e0 0x7f28b01075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.485+0000 7f28af200700 1 -- 172.21.15.38:0/741422353 >> 172.21.15.38:0/741422353 conn(0x7f28b0076700 msgr2=0x7f28b0078b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:18.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.486+0000 7f28af200700 1 -- 172.21.15.38:0/741422353 shutdown_connections 2024-03-24T01:00:18.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.486+0000 7f28af200700 1 -- 172.21.15.38:0/741422353 wait complete. 2024-03-24T01:00:18.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.487+0000 7f28af200700 1 Processor -- start 2024-03-24T01:00:18.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.487+0000 7f28af200700 1 -- start start 2024-03-24T01:00:18.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.487+0000 7f28af200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28b01071e0 0x7f28b019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:18.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.487+0000 7f28af200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0107b90 0x7f28b019d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:18.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.487+0000 7f28af200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b019da20 con 0x7f28b01071e0 2024-03-24T01:00:18.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.487+0000 7f28af200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b01a16f0 con 0x7f28b0107b90 2024-03-24T01:00:18.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.487+0000 7f28ade00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28b01071e0 0x7f28b019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:18.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.488+0000 7f28ade00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28b01071e0 0x7f28b019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54322/0 (socket says 172.21.15.38:54322) 2024-03-24T01:00:18.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.488+0000 7f28ade00700 1 -- 172.21.15.38:0/2931554505 learned_addr learned my addr 172.21.15.38:0/2931554505 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.488+0000 7f28a5400700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0107b90 0x7f28b019d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.488+0000 7f28ade00700 1 -- 172.21.15.38:0/2931554505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0107b90 msgr2=0x7f28b019d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.488+0000 7f28ade00700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0107b90 0x7f28b019d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.488+0000 7f28ade00700 1 -- 172.21.15.38:0/2931554505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28980095f0 con 0x7f28b01071e0 2024-03-24T01:00:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.488+0000 7f28a5400700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0107b90 0x7f28b019d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T01:00:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.488+0000 7f28ade00700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28b01071e0 0x7f28b019ce00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f28a000e920 tx=0x7f28a000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.489+0000 7f28a7400700 1 -- 172.21.15.38:0/2931554505 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a0009790 con 0x7f28b01071e0 2024-03-24T01:00:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.489+0000 7f28a7400700 1 -- 172.21.15.38:0/2931554505 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28a000cba0 con 0x7f28b01071e0 2024-03-24T01:00:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.489+0000 7f28a7400700 1 -- 172.21.15.38:0/2931554505 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a0018650 con 0x7f28b01071e0 2024-03-24T01:00:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.489+0000 7f28af200700 1 -- 172.21.15.38:0/2931554505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b01a19d0 con 0x7f28b01071e0 2024-03-24T01:00:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.489+0000 7f28af200700 1 -- 172.21.15.38:0/2931554505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b01a1f20 con 0x7f28b01071e0 2024-03-24T01:00:18.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.490+0000 7f28af200700 1 -- 172.21.15.38:0/2931554505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b010f530 con 0x7f28b01071e0 2024-03-24T01:00:18.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.491+0000 7f28a7400700 1 -- 172.21.15.38:0/2931554505 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f28a000cd10 con 0x7f28b01071e0 2024-03-24T01:00:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.492+0000 7f28a7400700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f289406dfb0 0x7f2894070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.492+0000 7f28a7400700 1 -- 172.21.15.38:0/2931554505 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f28a0014070 con 0x7f28b01071e0 2024-03-24T01:00:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.492+0000 7f28a5400700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f289406dfb0 0x7f2894070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:18.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.493+0000 7f28a5400700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f289406dfb0 0x7f2894070470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f289800b760 tx=0x7f2898009f70 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:18.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.496+0000 7f28a7400700 1 -- 172.21.15.38:0/2931554505 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f28a0053740 con 0x7f28b01071e0 2024-03-24T01:00:18.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.803+0000 7f28af200700 1 -- 172.21.15.38:0/2931554505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f28b019e110 con 0x7f28b01071e0 2024-03-24T01:00:18.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.804+0000 7f28a7400700 1 -- 172.21.15.38:0/2931554505 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f28a005b0d0 con 0x7f28b01071e0 2024-03-24T01:00:18.804 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-24T01:00:18.804 INFO:teuthology.orchestra.run.smithi038.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-24T01:00:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: fs cephfs is degraded 2024-03-24T01:00:18.804 INFO:teuthology.orchestra.run.smithi038.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-24T01:00:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-24T01:00:18.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.806+0000 7f28af200700 1 -- 172.21.15.38:0/2931554505 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f289406dfb0 msgr2=0x7f2894070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:18.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.806+0000 7f28af200700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f289406dfb0 0x7f2894070470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f289800b760 tx=0x7f2898009f70 comp rx=0 tx=0).stop 2024-03-24T01:00:18.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.806+0000 7f28af200700 1 -- 172.21.15.38:0/2931554505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28b01071e0 msgr2=0x7f28b019ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:18.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.806+0000 7f28af200700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28b01071e0 0x7f28b019ce00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f28a000e920 tx=0x7f28a000ece0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.807+0000 7f28af200700 1 -- 172.21.15.38:0/2931554505 shutdown_connections 2024-03-24T01:00:18.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.807+0000 7f28af200700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f289406dfb0 0x7f2894070470 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.807+0000 7f28af200700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28b01071e0 0x7f28b019ce00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.807+0000 7f28af200700 1 --2- 172.21.15.38:0/2931554505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b0107b90 0x7f28b019d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:18.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.807+0000 7f28af200700 1 -- 172.21.15.38:0/2931554505 >> 172.21.15.38:0/2931554505 conn(0x7f28b0076700 msgr2=0x7f28b0077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:18.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.807+0000 7f28af200700 1 -- 172.21.15.38:0/2931554505 shutdown_connections 2024-03-24T01:00:18.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:18.807+0000 7f28af200700 1 -- 172.21.15.38:0/2931554505 wait complete. 2024-03-24T01:00:19.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:18 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3343880957' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T01:00:19.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:18 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:00:19.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:18 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:19.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:18 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:19.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:18 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:19.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:18 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:19.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:18 smithi088 ceph-mon[87351]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:19.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:18 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3343880957' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T01:00:19.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:18 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:00:19.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:18 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:19.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:18 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:19.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:18 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:19.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:18 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:19.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:18 smithi038 ceph-mon[112108]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:19 smithi038 ceph-mon[112108]: pgmap v229: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:20.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:19 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2931554505' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T01:00:20.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:19 smithi088 ceph-mon[87351]: pgmap v229: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:20.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:19 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2931554505' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T01:00:21.576 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:21 smithi088 ceph-mon[87351]: Upgrade: Updating mds.cephfs.smithi088.cbfvvp 2024-03-24T01:00:21.576 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:21.576 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.cbfvvp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T01:00:21.577 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:21 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:00:21.577 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:21 smithi088 ceph-mon[87351]: Deploying daemon mds.cephfs.smithi088.cbfvvp on smithi088 2024-03-24T01:00:21.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:21 smithi038 ceph-mon[112108]: Upgrade: Updating mds.cephfs.smithi088.cbfvvp 2024-03-24T01:00:21.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:21.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.cbfvvp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T01:00:21.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:21 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:00:21.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:21 smithi038 ceph-mon[112108]: Deploying daemon mds.cephfs.smithi088.cbfvvp on smithi088 2024-03-24T01:00:22.571 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:22 smithi088 ceph-mon[87351]: pgmap v230: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:22.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:22 smithi038 ceph-mon[112108]: pgmap v230: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:24.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:24 smithi038 ceph-mon[112108]: pgmap v231: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:24 smithi088 ceph-mon[87351]: pgmap v231: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:25.589 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:25 smithi088 ceph-mon[87351]: osdmap e86: 6 total, 6 up, 6 in 2024-03-24T01:00:25.589 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:25 smithi088 ceph-mon[87351]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-24T01:00:25.740 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:25 smithi038 ceph-mon[112108]: osdmap e86: 6 total, 6 up, 6 in 2024-03-24T01:00:25.740 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:25 smithi038 ceph-mon[112108]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-24T01:00:26.723 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:26 smithi038 ceph-mon[112108]: pgmap v233: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:26.854 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:26 smithi088 ceph-mon[87351]: pgmap v233: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:27.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:00:27.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:00:28.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:28 smithi038 ceph-mon[112108]: pgmap v234: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:28.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:28 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:28.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:28 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:28.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:28 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:00:28.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:28 smithi088 ceph-mon[87351]: pgmap v234: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:28.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:28 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:28.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:28 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:28.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:28 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:00:29.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:29 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.88:6824/2293661188,v1:172.21.15.88:6825/2293661188] up:boot 2024-03-24T01:00:29.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:29 smithi038 ceph-mon[112108]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T01:00:29.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:29 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T01:00:29.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:29 smithi038 ceph-mon[112108]: pgmap v235: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:29 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.88:6824/2293661188,v1:172.21.15.88:6825/2293661188] up:boot 2024-03-24T01:00:29.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:29 smithi088 ceph-mon[87351]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T01:00:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:29 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T01:00:29.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:29 smithi088 ceph-mon[87351]: pgmap v235: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:32.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:31 smithi038 ceph-mon[112108]: pgmap v236: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:32.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:31 smithi088 ceph-mon[87351]: pgmap v236: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:33.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:33 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:33.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:33 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:33.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:33 smithi038 ceph-mon[112108]: pgmap v237: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:33.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:33 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:33.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:33 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:33.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:33 smithi088 ceph-mon[87351]: pgmap v237: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:35.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:35 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:35.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:35 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:35.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:35 smithi088 ceph-mon[87351]: pgmap v238: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:36.068 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:35 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:36.068 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:35 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:36.068 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:35 smithi038 ceph-mon[112108]: pgmap v238: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:37 smithi088 ceph-mon[87351]: pgmap v239: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:38.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:37 smithi038 ceph-mon[112108]: pgmap v239: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:40.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:39 smithi038 ceph-mon[112108]: pgmap v240: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:40.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:39 smithi088 ceph-mon[87351]: pgmap v240: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:41.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:41 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:41.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:41 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:41.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:41 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:00:41.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:41 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:00:41.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:41 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:41.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:41 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:00:41.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:41 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:41.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:41 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:41.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:41 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:41.612 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:41 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:41.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:41 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:41.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:41 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:41.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:41 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:00:41.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:41 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:00:41.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:41 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:41.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:41 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:00:41.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:41 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:41.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:41 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:41.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:41 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:41.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:41 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:42 smithi038 ceph-mon[112108]: pgmap v241: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:00:42.803 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:42 smithi088 ceph-mon[87351]: pgmap v241: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:42.803 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:00:44.216 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:44 smithi088 ceph-mon[87351]: pgmap v242: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:44.216 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:44 smithi088 ceph-mon[87351]: Upgrade: Updating mds.cephfs.smithi088.yhksbr 2024-03-24T01:00:44.216 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:44 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:44.216 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:44 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.yhksbr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T01:00:44.216 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:44 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:00:44.216 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:44 smithi088 ceph-mon[87351]: Deploying daemon mds.cephfs.smithi088.yhksbr on smithi088 2024-03-24T01:00:44.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:44 smithi038 ceph-mon[112108]: pgmap v242: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:44.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:44 smithi038 ceph-mon[112108]: Upgrade: Updating mds.cephfs.smithi088.yhksbr 2024-03-24T01:00:44.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:44 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:44.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:44 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.yhksbr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T01:00:44.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:44 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:00:44.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:44 smithi038 ceph-mon[112108]: Deploying daemon mds.cephfs.smithi088.yhksbr on smithi088 2024-03-24T01:00:46.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:46 smithi088 ceph-mon[87351]: pgmap v243: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:46.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:46 smithi038 ceph-mon[112108]: pgmap v243: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:48.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:48 smithi038 ceph-mon[112108]: pgmap v244: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:48.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:48 smithi038 ceph-mon[112108]: osdmap e87: 6 total, 6 up, 6 in 2024-03-24T01:00:48.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:48 smithi038 ceph-mon[112108]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-24T01:00:48.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:48 smithi088 ceph-mon[87351]: pgmap v244: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:48.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:48 smithi088 ceph-mon[87351]: osdmap e87: 6 total, 6 up, 6 in 2024-03-24T01:00:48.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:48 smithi088 ceph-mon[87351]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-24T01:00:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.982+0000 7fb567e00700 1 -- 172.21.15.38:0/1390407854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb560075c00 msgr2=0x7fb5601113f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.983+0000 7fb567e00700 1 --2- 172.21.15.38:0/1390407854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb560075c00 0x7fb5601113f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009910 tx=0x7fb54c009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.983+0000 7fb567e00700 1 -- 172.21.15.38:0/1390407854 shutdown_connections 2024-03-24T01:00:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.983+0000 7fb567e00700 1 --2- 172.21.15.38:0/1390407854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb560075c00 0x7fb5601113f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.983+0000 7fb567e00700 1 --2- 172.21.15.38:0/1390407854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5600752e0 0x7fb5600756c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.983+0000 7fb567e00700 1 -- 172.21.15.38:0/1390407854 >> 172.21.15.38:0/1390407854 conn(0x7fb5600fe780 msgr2=0x7fb560100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.983+0000 7fb567e00700 1 -- 172.21.15.38:0/1390407854 shutdown_connections 2024-03-24T01:00:48.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.983+0000 7fb567e00700 1 -- 172.21.15.38:0/1390407854 wait complete. 2024-03-24T01:00:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.984+0000 7fb567e00700 1 Processor -- start 2024-03-24T01:00:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.984+0000 7fb567e00700 1 -- start start 2024-03-24T01:00:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.985+0000 7fb567e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5600752e0 0x7fb56019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.985+0000 7fb567e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb560075c00 0x7fb56019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.985+0000 7fb567e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56019db40 con 0x7fb560075c00 2024-03-24T01:00:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.985+0000 7fb567e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5601a1810 con 0x7fb5600752e0 2024-03-24T01:00:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.985+0000 7fb564a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb560075c00 0x7fb56019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.985+0000 7fb564a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb560075c00 0x7fb56019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42784/0 (socket says 172.21.15.38:42784) 2024-03-24T01:00:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.985+0000 7fb564a00700 1 -- 172.21.15.38:0/2084224775 learned_addr learned my addr 172.21.15.38:0/2084224775 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.985+0000 7fb565400700 1 --2- 172.21.15.38:0/2084224775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5600752e0 0x7fb56019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:48.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.985+0000 7fb564a00700 1 -- 172.21.15.38:0/2084224775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5600752e0 msgr2=0x7fb56019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:48.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.985+0000 7fb564a00700 1 --2- 172.21.15.38:0/2084224775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5600752e0 0x7fb56019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:48.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.986+0000 7fb564a00700 1 -- 172.21.15.38:0/2084224775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c0095f0 con 0x7fb560075c00 2024-03-24T01:00:48.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.986+0000 7fb564a00700 1 --2- 172.21.15.38:0/2084224775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb560075c00 0x7fb56019d460 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009fd0 tx=0x7fb54c00b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:48.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.986+0000 7fb55e000700 1 -- 172.21.15.38:0/2084224775 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54c01d070 con 0x7fb560075c00 2024-03-24T01:00:48.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.986+0000 7fb55e000700 1 -- 172.21.15.38:0/2084224775 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb54c0052a0 con 0x7fb560075c00 2024-03-24T01:00:48.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.986+0000 7fb567e00700 1 -- 172.21.15.38:0/2084224775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5601a1af0 con 0x7fb560075c00 2024-03-24T01:00:48.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.986+0000 7fb55e000700 1 -- 172.21.15.38:0/2084224775 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54c017400 con 0x7fb560075c00 2024-03-24T01:00:48.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.986+0000 7fb567e00700 1 -- 172.21.15.38:0/2084224775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5601a2040 con 0x7fb560075c00 2024-03-24T01:00:48.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.988+0000 7fb567e00700 1 -- 172.21.15.38:0/2084224775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56010eb60 con 0x7fb560075c00 2024-03-24T01:00:48.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.989+0000 7fb55e000700 1 -- 172.21.15.38:0/2084224775 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fb54c00f720 con 0x7fb560075c00 2024-03-24T01:00:48.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.989+0000 7fb55e000700 1 --2- 172.21.15.38:0/2084224775 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb54806dfa0 0x7fb548070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:48.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.990+0000 7fb55e000700 1 -- 172.21.15.38:0/2084224775 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb54c08f0d0 con 0x7fb560075c00 2024-03-24T01:00:48.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.990+0000 7fb565400700 1 --2- 172.21.15.38:0/2084224775 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb54806dfa0 0x7fb548070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:48.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.990+0000 7fb565400700 1 --2- 172.21.15.38:0/2084224775 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb54806dfa0 0x7fb548070460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb554009bf0 tx=0x7fb554009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:48.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:48.995+0000 7fb55e000700 1 -- 172.21.15.38:0/2084224775 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb54c058450 con 0x7fb560075c00 2024-03-24T01:00:49.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.238+0000 7fb567e00700 1 -- 172.21.15.38:0/2084224775 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb56019e230 con 0x7fb54806dfa0 2024-03-24T01:00:49.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.240+0000 7fb55e000700 1 -- 172.21.15.38:0/2084224775 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb56019e230 con 0x7fb54806dfa0 2024-03-24T01:00:49.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.242+0000 7fb567e00700 1 -- 172.21.15.38:0/2084224775 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb54806dfa0 msgr2=0x7fb548070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:49.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.242+0000 7fb567e00700 1 --2- 172.21.15.38:0/2084224775 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb54806dfa0 0x7fb548070460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb554009bf0 tx=0x7fb554009260 comp rx=0 tx=0).stop 2024-03-24T01:00:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.242+0000 7fb567e00700 1 -- 172.21.15.38:0/2084224775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb560075c00 msgr2=0x7fb56019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.242+0000 7fb567e00700 1 --2- 172.21.15.38:0/2084224775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb560075c00 0x7fb56019d460 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009fd0 tx=0x7fb54c00b5f0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.242+0000 7fb567e00700 1 -- 172.21.15.38:0/2084224775 shutdown_connections 2024-03-24T01:00:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.242+0000 7fb567e00700 1 --2- 172.21.15.38:0/2084224775 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb54806dfa0 0x7fb548070460 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.242+0000 7fb567e00700 1 --2- 172.21.15.38:0/2084224775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5600752e0 0x7fb56019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.242+0000 7fb567e00700 1 --2- 172.21.15.38:0/2084224775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb560075c00 0x7fb56019d460 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.242+0000 7fb567e00700 1 -- 172.21.15.38:0/2084224775 >> 172.21.15.38:0/2084224775 conn(0x7fb5600fe780 msgr2=0x7fb5600ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.243+0000 7fb567e00700 1 -- 172.21.15.38:0/2084224775 shutdown_connections 2024-03-24T01:00:49.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.243+0000 7fb567e00700 1 -- 172.21.15.38:0/2084224775 wait complete. 2024-03-24T01:00:49.259 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T01:00:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.406+0000 7fa529000700 1 -- 172.21.15.38:0/4140302933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524073a70 msgr2=0x7fa524111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.406+0000 7fa529000700 1 --2- 172.21.15.38:0/4140302933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524073a70 0x7fa524111540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa510009910 tx=0x7fa510009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.406+0000 7fa529000700 1 -- 172.21.15.38:0/4140302933 shutdown_connections 2024-03-24T01:00:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.406+0000 7fa529000700 1 --2- 172.21.15.38:0/4140302933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524073a70 0x7fa524111540 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.406+0000 7fa529000700 1 --2- 172.21.15.38:0/4140302933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa524073150 0x7fa524073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.406+0000 7fa529000700 1 -- 172.21.15.38:0/4140302933 >> 172.21.15.38:0/4140302933 conn(0x7fa5240fc580 msgr2=0x7fa5240fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.407+0000 7fa529000700 1 -- 172.21.15.38:0/4140302933 shutdown_connections 2024-03-24T01:00:49.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.407+0000 7fa529000700 1 -- 172.21.15.38:0/4140302933 wait complete. 2024-03-24T01:00:49.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.408+0000 7fa529000700 1 Processor -- start 2024-03-24T01:00:49.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.408+0000 7fa529000700 1 -- start start 2024-03-24T01:00:49.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.409+0000 7fa529000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa524073150 0x7fa52419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:49.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.409+0000 7fa529000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524073a70 0x7fa52419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.409+0000 7fa529000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52419dc00 con 0x7fa524073a70 2024-03-24T01:00:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.409+0000 7fa529000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5241a18d0 con 0x7fa524073150 2024-03-24T01:00:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.409+0000 7fa521400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524073a70 0x7fa52419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.409+0000 7fa521400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524073a70 0x7fa52419d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42812/0 (socket says 172.21.15.38:42812) 2024-03-24T01:00:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.409+0000 7fa521400700 1 -- 172.21.15.38:0/779209127 learned_addr learned my addr 172.21.15.38:0/779209127 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.409+0000 7fa521e00700 1 --2- 172.21.15.38:0/779209127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa524073150 0x7fa52419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.410+0000 7fa521400700 1 -- 172.21.15.38:0/779209127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa524073150 msgr2=0x7fa52419cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.410+0000 7fa521400700 1 --2- 172.21.15.38:0/779209127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa524073150 0x7fa52419cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.410+0000 7fa521400700 1 -- 172.21.15.38:0/779209127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100095f0 con 0x7fa524073a70 2024-03-24T01:00:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.410+0000 7fa521400700 1 --2- 172.21.15.38:0/779209127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524073a70 0x7fa52419d520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa5100092e0 tx=0x7fa51000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.410+0000 7fa51aa00700 1 -- 172.21.15.38:0/779209127 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa51001d070 con 0x7fa524073a70 2024-03-24T01:00:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.410+0000 7fa51aa00700 1 -- 172.21.15.38:0/779209127 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5100056e0 con 0x7fa524073a70 2024-03-24T01:00:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.410+0000 7fa51aa00700 1 -- 172.21.15.38:0/779209127 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa51000fe80 con 0x7fa524073a70 2024-03-24T01:00:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.410+0000 7fa529000700 1 -- 172.21.15.38:0/779209127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5241a1bb0 con 0x7fa524073a70 2024-03-24T01:00:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.410+0000 7fa529000700 1 -- 172.21.15.38:0/779209127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5241a2100 con 0x7fa524073a70 2024-03-24T01:00:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.411+0000 7fa529000700 1 -- 172.21.15.38:0/779209127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa524074890 con 0x7fa524073a70 2024-03-24T01:00:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.412+0000 7fa51aa00700 1 -- 172.21.15.38:0/779209127 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fa51000f6f0 con 0x7fa524073a70 2024-03-24T01:00:49.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.414+0000 7fa51aa00700 1 --2- 172.21.15.38:0/779209127 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa50806dfb0 0x7fa508070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:49.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.414+0000 7fa51aa00700 1 -- 172.21.15.38:0/779209127 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa51008f1b0 con 0x7fa524073a70 2024-03-24T01:00:49.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.414+0000 7fa521e00700 1 --2- 172.21.15.38:0/779209127 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa50806dfb0 0x7fa508070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:49.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.415+0000 7fa521e00700 1 --2- 172.21.15.38:0/779209127 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa50806dfb0 0x7fa508070470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa514005950 tx=0x7fa5140058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:49.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.419+0000 7fa51aa00700 1 -- 172.21.15.38:0/779209127 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa5100585b0 con 0x7fa524073a70 2024-03-24T01:00:49.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.667+0000 7fa529000700 1 -- 172.21.15.38:0/779209127 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5241a2430 con 0x7fa50806dfb0 2024-03-24T01:00:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.670+0000 7fa51aa00700 1 -- 172.21.15.38:0/779209127 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa5241a2430 con 0x7fa50806dfb0 2024-03-24T01:00:49.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.673+0000 7fa529000700 1 -- 172.21.15.38:0/779209127 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa50806dfb0 msgr2=0x7fa508070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:49.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.673+0000 7fa529000700 1 --2- 172.21.15.38:0/779209127 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa50806dfb0 0x7fa508070470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa514005950 tx=0x7fa5140058e0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.673+0000 7fa529000700 1 -- 172.21.15.38:0/779209127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524073a70 msgr2=0x7fa52419d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:49.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.673+0000 7fa529000700 1 --2- 172.21.15.38:0/779209127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524073a70 0x7fa52419d520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa5100092e0 tx=0x7fa51000be20 comp rx=0 tx=0).stop 2024-03-24T01:00:49.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.673+0000 7fa529000700 1 -- 172.21.15.38:0/779209127 shutdown_connections 2024-03-24T01:00:49.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.673+0000 7fa529000700 1 --2- 172.21.15.38:0/779209127 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fa50806dfb0 0x7fa508070470 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.673+0000 7fa529000700 1 --2- 172.21.15.38:0/779209127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa524073150 0x7fa52419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.673+0000 7fa529000700 1 --2- 172.21.15.38:0/779209127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa524073a70 0x7fa52419d520 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.673+0000 7fa529000700 1 -- 172.21.15.38:0/779209127 >> 172.21.15.38:0/779209127 conn(0x7fa5240fc580 msgr2=0x7fa5241007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:49.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.673+0000 7fa529000700 1 -- 172.21.15.38:0/779209127 shutdown_connections 2024-03-24T01:00:49.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.673+0000 7fa529000700 1 -- 172.21.15.38:0/779209127 wait complete. 2024-03-24T01:00:49.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.840+0000 7f8ae9400700 1 -- 172.21.15.38:0/3519978900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ae4073aa0 msgr2=0x7f8ae410cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:49.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.840+0000 7f8ae9400700 1 --2- 172.21.15.38:0/3519978900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ae4073aa0 0x7f8ae410cc30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009910 tx=0x7f8ad0009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:49.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.840+0000 7f8ae9400700 1 -- 172.21.15.38:0/3519978900 shutdown_connections 2024-03-24T01:00:49.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.840+0000 7f8ae9400700 1 --2- 172.21.15.38:0/3519978900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ae4073aa0 0x7f8ae410cc30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.840+0000 7f8ae9400700 1 --2- 172.21.15.38:0/3519978900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ae4073180 0x7f8ae4073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.840+0000 7f8ae9400700 1 -- 172.21.15.38:0/3519978900 >> 172.21.15.38:0/3519978900 conn(0x7f8ae40fc400 msgr2=0x7f8ae40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:49.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.840+0000 7f8ae9400700 1 -- 172.21.15.38:0/3519978900 shutdown_connections 2024-03-24T01:00:49.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.840+0000 7f8ae9400700 1 -- 172.21.15.38:0/3519978900 wait complete. 2024-03-24T01:00:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.841+0000 7f8ae9400700 1 Processor -- start 2024-03-24T01:00:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.841+0000 7f8ae9400700 1 -- start start 2024-03-24T01:00:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.842+0000 7f8ae9400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ae4073180 0x7f8ae419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.842+0000 7f8ae9400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ae4073aa0 0x7f8ae419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.842+0000 7f8ae9400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae419f2c0 con 0x7f8ae4073180 2024-03-24T01:00:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.842+0000 7f8ae9400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae4198880 con 0x7f8ae4073aa0 2024-03-24T01:00:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.842+0000 7f8ae3e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ae4073180 0x7f8ae419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.842+0000 7f8ae3e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ae4073180 0x7f8ae419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42828/0 (socket says 172.21.15.38:42828) 2024-03-24T01:00:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.842+0000 7f8ae3e00700 1 -- 172.21.15.38:0/513581109 learned_addr learned my addr 172.21.15.38:0/513581109 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:49.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.842+0000 7f8ae3400700 1 --2- 172.21.15.38:0/513581109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ae4073aa0 0x7f8ae419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:49.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.842+0000 7f8ae3e00700 1 -- 172.21.15.38:0/513581109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ae4073aa0 msgr2=0x7f8ae419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:49.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.842+0000 7f8ae3e00700 1 --2- 172.21.15.38:0/513581109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ae4073aa0 0x7f8ae419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:49.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.843+0000 7f8ae3e00700 1 -- 172.21.15.38:0/513581109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad00095f0 con 0x7f8ae4073180 2024-03-24T01:00:49.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.843+0000 7f8ae3e00700 1 --2- 172.21.15.38:0/513581109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ae4073180 0x7f8ae419e6f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c8a0 tx=0x7f8ad400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:49.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.843+0000 7f8ae0c00700 1 -- 172.21.15.38:0/513581109 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad400cea0 con 0x7f8ae4073180 2024-03-24T01:00:49.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.843+0000 7f8ae0c00700 1 -- 172.21.15.38:0/513581109 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ad4004830 con 0x7f8ae4073180 2024-03-24T01:00:49.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.843+0000 7f8ae0c00700 1 -- 172.21.15.38:0/513581109 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad40055b0 con 0x7f8ae4073180 2024-03-24T01:00:49.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.843+0000 7f8ae9400700 1 -- 172.21.15.38:0/513581109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae4198b60 con 0x7f8ae4073180 2024-03-24T01:00:49.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.843+0000 7f8ae9400700 1 -- 172.21.15.38:0/513581109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae41990b0 con 0x7f8ae4073180 2024-03-24T01:00:49.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.844+0000 7f8ae9400700 1 -- 172.21.15.38:0/513581109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae410a3d0 con 0x7f8ae4073180 2024-03-24T01:00:49.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.845+0000 7f8ae0c00700 1 -- 172.21.15.38:0/513581109 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f8ad40049a0 con 0x7f8ae4073180 2024-03-24T01:00:49.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.846+0000 7f8ae0c00700 1 --2- 172.21.15.38:0/513581109 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8ac806dfb0 0x7f8ac8070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:49.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.847+0000 7f8ae0c00700 1 -- 172.21.15.38:0/513581109 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8ad408da20 con 0x7f8ae4073180 2024-03-24T01:00:49.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.847+0000 7f8ae3400700 1 --2- 172.21.15.38:0/513581109 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8ac806dfb0 0x7f8ac8070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:49.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.847+0000 7f8ae3400700 1 --2- 172.21.15.38:0/513581109 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8ac806dfb0 0x7f8ac8070470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0005100 tx=0x7f8ad001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:49.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:49.851+0000 7f8ae0c00700 1 -- 172.21.15.38:0/513581109 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8ad405a950 con 0x7f8ae4073180 2024-03-24T01:00:50.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.088+0000 7f8ae9400700 1 -- 172.21.15.38:0/513581109 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ae4199d60 con 0x7f8ac806dfb0 2024-03-24T01:00:50.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.101+0000 7f8ae0c00700 1 -- 172.21.15.38:0/513581109 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f8ae4199d60 con 0x7f8ac806dfb0 2024-03-24T01:00:50.102 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T01:00:50.102 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (9m) 39s ago 23m 14.0M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (23m) 39s ago 23m 9537k - 18.2.2-952-g738c2fd6 68aff9aff3b2 f14ef3aa504e 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (22m) 17s ago 22m 9277k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (5m) 39s ago 23m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (5m) 17s ago 21m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (8m) 39s ago 22m 75.4M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (47s) 39s ago 15m 10.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b78588cbee24 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (74s) 39s ago 15m 11.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e597065368f0 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (22s) 17s ago 15m 10.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 49be5a5d2c69 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (15m) 17s ago 15m 12.0M - 18.2.2-952-g738c2fd6 68aff9aff3b2 8eb90e582535 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (12m) 39s ago 24m 510M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (10m) 17s ago 21m 452M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (7m) 39s ago 25m 55.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (6m) 17s ago 21m 48.4M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T01:00:50.103 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (10m) 39s ago 23m 7960k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T01:00:50.104 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (10m) 17s ago 21m 8078k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T01:00:50.104 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (4m) 39s ago 20m 102M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T01:00:50.104 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (4m) 39s ago 20m 108M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cb72ac446ba7 2024-03-24T01:00:50.104 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (3m) 39s ago 19m 91.4M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c73f3e8276b6 2024-03-24T01:00:50.104 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (3m) 17s ago 18m 110M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b2afaff761d8 2024-03-24T01:00:50.104 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (2m) 17s ago 18m 104M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9a6f202043cd 2024-03-24T01:00:50.104 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (101s) 17s ago 17m 100M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 866388fc07f6 2024-03-24T01:00:50.104 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (9m) 39s ago 22m 74.1M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T01:00:50.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.105+0000 7f8ae9400700 1 -- 172.21.15.38:0/513581109 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8ac806dfb0 msgr2=0x7f8ac8070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:50.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.105+0000 7f8ae9400700 1 --2- 172.21.15.38:0/513581109 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8ac806dfb0 0x7f8ac8070470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0005100 tx=0x7f8ad001a040 comp rx=0 tx=0).stop 2024-03-24T01:00:50.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.105+0000 7f8ae9400700 1 -- 172.21.15.38:0/513581109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ae4073180 msgr2=0x7f8ae419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:50.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.105+0000 7f8ae9400700 1 --2- 172.21.15.38:0/513581109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ae4073180 0x7f8ae419e6f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c8a0 tx=0x7f8ad400cc60 comp rx=0 tx=0).stop 2024-03-24T01:00:50.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.105+0000 7f8ae9400700 1 -- 172.21.15.38:0/513581109 shutdown_connections 2024-03-24T01:00:50.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.105+0000 7f8ae9400700 1 --2- 172.21.15.38:0/513581109 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f8ac806dfb0 0x7f8ac8070470 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.105+0000 7f8ae9400700 1 --2- 172.21.15.38:0/513581109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ae4073180 0x7f8ae419e6f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.105+0000 7f8ae9400700 1 --2- 172.21.15.38:0/513581109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ae4073aa0 0x7f8ae419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.105+0000 7f8ae9400700 1 -- 172.21.15.38:0/513581109 >> 172.21.15.38:0/513581109 conn(0x7f8ae40fc400 msgr2=0x7f8ae4107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:50.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.105+0000 7f8ae9400700 1 -- 172.21.15.38:0/513581109 shutdown_connections 2024-03-24T01:00:50.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.105+0000 7f8ae9400700 1 -- 172.21.15.38:0/513581109 wait complete. 2024-03-24T01:00:50.136 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:50 smithi088 ceph-mon[87351]: pgmap v246: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:50.136 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:50 smithi088 ceph-mon[87351]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:50.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.260+0000 7f0f13e00700 1 -- 172.21.15.38:0/2104383404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f0c106a70 msgr2=0x7f0f0c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:50.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.260+0000 7f0f13e00700 1 --2- 172.21.15.38:0/2104383404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f0c106a70 0x7f0f0c106e50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000b0a0 tx=0x7f0f0000b3b0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.260+0000 7f0f13e00700 1 -- 172.21.15.38:0/2104383404 shutdown_connections 2024-03-24T01:00:50.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.260+0000 7f0f13e00700 1 --2- 172.21.15.38:0/2104383404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f0c100a70 0x7f0f0c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.260+0000 7f0f13e00700 1 --2- 172.21.15.38:0/2104383404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f0c106a70 0x7f0f0c106e50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.260+0000 7f0f13e00700 1 -- 172.21.15.38:0/2104383404 >> 172.21.15.38:0/2104383404 conn(0x7f0f0c0fc580 msgr2=0x7f0f0c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:50.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.260+0000 7f0f13e00700 1 -- 172.21.15.38:0/2104383404 shutdown_connections 2024-03-24T01:00:50.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.260+0000 7f0f13e00700 1 -- 172.21.15.38:0/2104383404 wait complete. 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.261+0000 7f0f13e00700 1 Processor -- start 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.262+0000 7f0f13e00700 1 -- start start 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.262+0000 7f0f13e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f0c100a70 0x7f0f0c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.262+0000 7f0f13e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f0c106a70 0x7f0f0c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.262+0000 7f0f13e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c073b30 con 0x7f0f0c106a70 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.262+0000 7f0f13e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c073c70 con 0x7f0f0c100a70 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.262+0000 7f0f10a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f0c106a70 0x7f0f0c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.262+0000 7f0f10a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f0c106a70 0x7f0f0c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42840/0 (socket says 172.21.15.38:42840) 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.262+0000 7f0f10a00700 1 -- 172.21.15.38:0/1052460549 learned_addr learned my addr 172.21.15.38:0/1052460549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.262+0000 7f0f11400700 1 --2- 172.21.15.38:0/1052460549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f0c100a70 0x7f0f0c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.263+0000 7f0f10a00700 1 -- 172.21.15.38:0/1052460549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f0c100a70 msgr2=0x7f0f0c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.263+0000 7f0f10a00700 1 --2- 172.21.15.38:0/1052460549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f0c100a70 0x7f0f0c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.263+0000 7f0f10a00700 1 -- 172.21.15.38:0/1052460549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f00009d00 con 0x7f0f0c106a70 2024-03-24T01:00:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.263+0000 7f0f10a00700 1 --2- 172.21.15.38:0/1052460549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f0c106a70 0x7f0f0c0735f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800b4b0 tx=0x7f0ef800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.263+0000 7f0f0a000700 1 -- 172.21.15.38:0/1052460549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ef8011800 con 0x7f0f0c106a70 2024-03-24T01:00:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.263+0000 7f0f0a000700 1 -- 172.21.15.38:0/1052460549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ef8011e20 con 0x7f0f0c106a70 2024-03-24T01:00:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.263+0000 7f0f13e00700 1 -- 172.21.15.38:0/1052460549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c1a2cf0 con 0x7f0f0c106a70 2024-03-24T01:00:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.263+0000 7f0f0a000700 1 -- 172.21.15.38:0/1052460549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ef800f2e0 con 0x7f0f0c106a70 2024-03-24T01:00:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.263+0000 7f0f13e00700 1 -- 172.21.15.38:0/1052460549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c1a3050 con 0x7f0f0c106a70 2024-03-24T01:00:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.265+0000 7f0f13e00700 1 -- 172.21.15.38:0/1052460549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c109070 con 0x7f0f0c106a70 2024-03-24T01:00:50.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.266+0000 7f0f0a000700 1 -- 172.21.15.38:0/1052460549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f0ef8010380 con 0x7f0f0c106a70 2024-03-24T01:00:50.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.266+0000 7f0f0a000700 1 --2- 172.21.15.38:0/1052460549 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ef406dfb0 0x7f0ef4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.267+0000 7f0f0a000700 1 -- 172.21.15.38:0/1052460549 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0ef808da50 con 0x7f0f0c106a70 2024-03-24T01:00:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.267+0000 7f0f11400700 1 --2- 172.21.15.38:0/1052460549 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ef406dfb0 0x7f0ef4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.268+0000 7f0f11400700 1 --2- 172.21.15.38:0/1052460549 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ef406dfb0 0x7f0ef4070470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009850 tx=0x7f0f000097c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.271+0000 7f0f0a000700 1 -- 172.21.15.38:0/1052460549 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ef8056dd0 con 0x7f0f0c106a70 2024-03-24T01:00:50.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:50 smithi038 ceph-mon[112108]: pgmap v246: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:50.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:50 smithi038 ceph-mon[112108]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:50.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.618+0000 7f0f13e00700 1 -- 172.21.15.38:0/1052460549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0f0c1021b0 con 0x7f0f0c106a70 2024-03-24T01:00:50.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.619+0000 7f0f0a000700 1 -- 172.21.15.38:0/1052460549 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0ef805a490 con 0x7f0f0c106a70 2024-03-24T01:00:50.620 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T01:00:50.620 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 3 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 13 2024-03-24T01:00:50.621 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T01:00:50.622 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T01:00:50.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.622+0000 7f0f13e00700 1 -- 172.21.15.38:0/1052460549 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ef406dfb0 msgr2=0x7f0ef4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:50.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.622+0000 7f0f13e00700 1 --2- 172.21.15.38:0/1052460549 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ef406dfb0 0x7f0ef4070470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009850 tx=0x7f0f000097c0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.622+0000 7f0f13e00700 1 -- 172.21.15.38:0/1052460549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f0c106a70 msgr2=0x7f0f0c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:50.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.622+0000 7f0f13e00700 1 --2- 172.21.15.38:0/1052460549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f0c106a70 0x7f0f0c0735f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800b4b0 tx=0x7f0ef800b870 comp rx=0 tx=0).stop 2024-03-24T01:00:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.622+0000 7f0f13e00700 1 -- 172.21.15.38:0/1052460549 shutdown_connections 2024-03-24T01:00:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.622+0000 7f0f13e00700 1 --2- 172.21.15.38:0/1052460549 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ef406dfb0 0x7f0ef4070470 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.622+0000 7f0f13e00700 1 --2- 172.21.15.38:0/1052460549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f0c100a70 0x7f0f0c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.622+0000 7f0f13e00700 1 --2- 172.21.15.38:0/1052460549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f0c106a70 0x7f0f0c0735f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.622+0000 7f0f13e00700 1 -- 172.21.15.38:0/1052460549 >> 172.21.15.38:0/1052460549 conn(0x7f0f0c0fc580 msgr2=0x7f0f0c0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.622+0000 7f0f13e00700 1 -- 172.21.15.38:0/1052460549 shutdown_connections 2024-03-24T01:00:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.622+0000 7f0f13e00700 1 -- 172.21.15.38:0/1052460549 wait complete. 2024-03-24T01:00:50.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.788+0000 7fe29d600700 1 -- 172.21.15.38:0/3956819411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe298101c10 msgr2=0x7fe298101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:50.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.788+0000 7fe29d600700 1 --2- 172.21.15.38:0/3956819411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe298101c10 0x7fe298101ff0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe288009910 tx=0x7fe288009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:50.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.788+0000 7fe29d600700 1 -- 172.21.15.38:0/3956819411 shutdown_connections 2024-03-24T01:00:50.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.788+0000 7fe29d600700 1 --2- 172.21.15.38:0/3956819411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe298102530 0x7fe29810aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.788+0000 7fe29d600700 1 --2- 172.21.15.38:0/3956819411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe298101c10 0x7fe298101ff0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.788+0000 7fe29d600700 1 -- 172.21.15.38:0/3956819411 >> 172.21.15.38:0/3956819411 conn(0x7fe298076640 msgr2=0x7fe2980fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:50.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.788+0000 7fe29d600700 1 -- 172.21.15.38:0/3956819411 shutdown_connections 2024-03-24T01:00:50.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.788+0000 7fe29d600700 1 -- 172.21.15.38:0/3956819411 wait complete. 2024-03-24T01:00:50.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.789+0000 7fe29d600700 1 Processor -- start 2024-03-24T01:00:50.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.789+0000 7fe29d600700 1 -- start start 2024-03-24T01:00:50.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.790+0000 7fe29d600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe298101c10 0x7fe29819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:50.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.790+0000 7fe29d600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe298102530 0x7fe29819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:50.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.790+0000 7fe29d600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2981989b0 con 0x7fe298101c10 2024-03-24T01:00:50.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.790+0000 7fe29d600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298198b20 con 0x7fe298102530 2024-03-24T01:00:50.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.790+0000 7fe296a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe298101c10 0x7fe29819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:50.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.790+0000 7fe296a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe298101c10 0x7fe29819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42860/0 (socket says 172.21.15.38:42860) 2024-03-24T01:00:50.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.790+0000 7fe296a00700 1 -- 172.21.15.38:0/3601065879 learned_addr learned my addr 172.21.15.38:0/3601065879 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:50.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.790+0000 7fe28e000700 1 --2- 172.21.15.38:0/3601065879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe298102530 0x7fe29819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:50.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.791+0000 7fe296a00700 1 -- 172.21.15.38:0/3601065879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe298102530 msgr2=0x7fe29819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:50.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.791+0000 7fe296a00700 1 --2- 172.21.15.38:0/3601065879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe298102530 0x7fe29819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:50.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.791+0000 7fe296a00700 1 -- 172.21.15.38:0/3601065879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880095f0 con 0x7fe298101c10 2024-03-24T01:00:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.791+0000 7fe296a00700 1 --2- 172.21.15.38:0/3601065879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe298101c10 0x7fe29819e900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe28800bd70 tx=0x7fe28800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.791+0000 7fe28fe00700 1 -- 172.21.15.38:0/3601065879 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe28801d070 con 0x7fe298101c10 2024-03-24T01:00:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.791+0000 7fe28fe00700 1 -- 172.21.15.38:0/3601065879 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2880056e0 con 0x7fe298101c10 2024-03-24T01:00:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.791+0000 7fe29d600700 1 -- 172.21.15.38:0/3601065879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe298198da0 con 0x7fe298101c10 2024-03-24T01:00:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.791+0000 7fe28fe00700 1 -- 172.21.15.38:0/3601065879 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe288017400 con 0x7fe298101c10 2024-03-24T01:00:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.791+0000 7fe29d600700 1 -- 172.21.15.38:0/3601065879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe298199290 con 0x7fe298101c10 2024-03-24T01:00:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.793+0000 7fe29d600700 1 -- 172.21.15.38:0/3601065879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe298068d60 con 0x7fe298101c10 2024-03-24T01:00:50.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.793+0000 7fe28fe00700 1 -- 172.21.15.38:0/3601065879 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fe288003a60 con 0x7fe298101c10 2024-03-24T01:00:50.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.794+0000 7fe28fe00700 1 --2- 172.21.15.38:0/3601065879 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe27c06dfa0 0x7fe27c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:50.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.794+0000 7fe28fe00700 1 -- 172.21.15.38:0/3601065879 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe28808f1f0 con 0x7fe298101c10 2024-03-24T01:00:50.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.795+0000 7fe28e000700 1 --2- 172.21.15.38:0/3601065879 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe27c06dfa0 0x7fe27c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.795+0000 7fe28e000700 1 --2- 172.21.15.38:0/3601065879 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe27c06dfa0 0x7fe27c070460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe280005950 tx=0x7fe2800058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:50.799+0000 7fe28fe00700 1 -- 172.21.15.38:0/3601065879 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe2880585f0 con 0x7fe298101c10 2024-03-24T01:00:51.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.073+0000 7fe29d600700 1 -- 172.21.15.38:0/3601065879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe298199ef0 con 0x7fe298101c10 2024-03-24T01:00:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.074+0000 7fe28fe00700 1 -- 172.21.15.38:0/3601065879 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1693 (secure 0 0 0) 0x7fe28805bcb0 con 0x7fe298101c10 2024-03-24T01:00:51.074 INFO:teuthology.orchestra.run.smithi038.stdout:e30 2024-03-24T01:00:51.074 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T01:00:51.074 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 22 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T00:59:25.362766+0000 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T01:00:51.075 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 83 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:up {} 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:failed 0,1 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T01:00:51.076 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T01:00:51.077 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T01:00:51.077 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T01:00:51.077 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T01:00:51.077 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 0 members: 2024-03-24T01:00:51.077 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:00:51.077 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:00:51.077 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-03-24T01:00:51.077 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:00:51.077 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{-1:34446} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:00:51.077 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{-1:34478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:00:51.077 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{-1:44345} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2293661188,v1:172.21.15.88:6825/2293661188] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:00:51.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.077+0000 7fe29d600700 1 -- 172.21.15.38:0/3601065879 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe27c06dfa0 msgr2=0x7fe27c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:51.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.077+0000 7fe29d600700 1 --2- 172.21.15.38:0/3601065879 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe27c06dfa0 0x7fe27c070460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe280005950 tx=0x7fe2800058e0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.077+0000 7fe29d600700 1 -- 172.21.15.38:0/3601065879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe298101c10 msgr2=0x7fe29819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.077+0000 7fe29d600700 1 --2- 172.21.15.38:0/3601065879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe298101c10 0x7fe29819e900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe28800bd70 tx=0x7fe28800be50 comp rx=0 tx=0).stop 2024-03-24T01:00:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.077+0000 7fe29d600700 1 -- 172.21.15.38:0/3601065879 shutdown_connections 2024-03-24T01:00:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.077+0000 7fe29d600700 1 --2- 172.21.15.38:0/3601065879 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe27c06dfa0 0x7fe27c070460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.077+0000 7fe29d600700 1 --2- 172.21.15.38:0/3601065879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe298101c10 0x7fe29819e900 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.077+0000 7fe29d600700 1 --2- 172.21.15.38:0/3601065879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe298102530 0x7fe29819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.077+0000 7fe29d600700 1 -- 172.21.15.38:0/3601065879 >> 172.21.15.38:0/3601065879 conn(0x7fe298076640 msgr2=0x7fe29810a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.077+0000 7fe29d600700 1 -- 172.21.15.38:0/3601065879 shutdown_connections 2024-03-24T01:00:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.077+0000 7fe29d600700 1 -- 172.21.15.38:0/3601065879 wait complete. 2024-03-24T01:00:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 30 2024-03-24T01:00:51.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.232+0000 7fdb23a00700 1 -- 172.21.15.38:0/3765803873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb1c101c10 msgr2=0x7fdb1c101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:51.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.232+0000 7fdb23a00700 1 --2- 172.21.15.38:0/3765803873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb1c101c10 0x7fdb1c101ff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009910 tx=0x7fdb0c009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:51.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.232+0000 7fdb23a00700 1 -- 172.21.15.38:0/3765803873 shutdown_connections 2024-03-24T01:00:51.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.232+0000 7fdb23a00700 1 --2- 172.21.15.38:0/3765803873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb1c102530 0x7fdb1c10aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.232+0000 7fdb23a00700 1 --2- 172.21.15.38:0/3765803873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb1c101c10 0x7fdb1c101ff0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.232+0000 7fdb23a00700 1 -- 172.21.15.38:0/3765803873 >> 172.21.15.38:0/3765803873 conn(0x7fdb1c076640 msgr2=0x7fdb1c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:51.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.232+0000 7fdb23a00700 1 -- 172.21.15.38:0/3765803873 shutdown_connections 2024-03-24T01:00:51.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.232+0000 7fdb23a00700 1 -- 172.21.15.38:0/3765803873 wait complete. 2024-03-24T01:00:51.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.233+0000 7fdb23a00700 1 Processor -- start 2024-03-24T01:00:51.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.233+0000 7fdb23a00700 1 -- start start 2024-03-24T01:00:51.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.234+0000 7fdb23a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb1c101c10 0x7fdb1c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:51.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.234+0000 7fdb23a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb1c102530 0x7fdb1c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:51.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.234+0000 7fdb23a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c19f400 con 0x7fdb1c102530 2024-03-24T01:00:51.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.234+0000 7fdb23a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c1989e0 con 0x7fdb1c101c10 2024-03-24T01:00:51.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.234+0000 7fdb13e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb1c102530 0x7fdb1c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:51.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.234+0000 7fdb13e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb1c102530 0x7fdb1c19ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42874/0 (socket says 172.21.15.38:42874) 2024-03-24T01:00:51.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.234+0000 7fdb13e00700 1 -- 172.21.15.38:0/1057059611 learned_addr learned my addr 172.21.15.38:0/1057059611 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:51.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.234+0000 7fdb21000700 1 --2- 172.21.15.38:0/1057059611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb1c101c10 0x7fdb1c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:51.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.234+0000 7fdb13e00700 1 -- 172.21.15.38:0/1057059611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb1c101c10 msgr2=0x7fdb1c19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:51.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.234+0000 7fdb13e00700 1 --2- 172.21.15.38:0/1057059611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb1c101c10 0x7fdb1c19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.234+0000 7fdb13e00700 1 -- 172.21.15.38:0/1057059611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c0095f0 con 0x7fdb1c102530 2024-03-24T01:00:51.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.235+0000 7fdb13e00700 1 --2- 172.21.15.38:0/1057059611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb1c102530 0x7fdb1c19ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400e940 tx=0x7fdb0400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:51.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.235+0000 7fdb1a000700 1 -- 172.21.15.38:0/1057059611 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb040096e0 con 0x7fdb1c102530 2024-03-24T01:00:51.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.235+0000 7fdb1a000700 1 -- 172.21.15.38:0/1057059611 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb04004cf0 con 0x7fdb1c102530 2024-03-24T01:00:51.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.235+0000 7fdb23a00700 1 -- 172.21.15.38:0/1057059611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1c198cc0 con 0x7fdb1c102530 2024-03-24T01:00:51.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.235+0000 7fdb1a000700 1 -- 172.21.15.38:0/1057059611 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb0400eee0 con 0x7fdb1c102530 2024-03-24T01:00:51.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.235+0000 7fdb23a00700 1 -- 172.21.15.38:0/1057059611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1c199210 con 0x7fdb1c102530 2024-03-24T01:00:51.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.236+0000 7fdb23a00700 1 -- 172.21.15.38:0/1057059611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1c109580 con 0x7fdb1c102530 2024-03-24T01:00:51.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.237+0000 7fdb1a000700 1 -- 172.21.15.38:0/1057059611 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fdb0400cba0 con 0x7fdb1c102530 2024-03-24T01:00:51.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.238+0000 7fdb1a000700 1 --2- 172.21.15.38:0/1057059611 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb0006dfb0 0x7fdb00070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:51.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.238+0000 7fdb1a000700 1 -- 172.21.15.38:0/1057059611 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdb04014070 con 0x7fdb1c102530 2024-03-24T01:00:51.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.238+0000 7fdb21000700 1 --2- 172.21.15.38:0/1057059611 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb0006dfb0 0x7fdb00070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:51.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.239+0000 7fdb21000700 1 --2- 172.21.15.38:0/1057059611 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb0006dfb0 0x7fdb00070470 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00b760 tx=0x7fdb0c009f90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:51.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.243+0000 7fdb1a000700 1 -- 172.21.15.38:0/1057059611 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb0405afd0 con 0x7fdb1c102530 2024-03-24T01:00:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:51 smithi038 ceph-mon[112108]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:51 smithi038 ceph-mon[112108]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:51 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1052460549' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:51.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:51 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3601065879' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T01:00:51.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:51 smithi088 ceph-mon[87351]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:51.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:51 smithi088 ceph-mon[87351]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:51.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:51 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1052460549' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:00:51.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:51 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3601065879' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T01:00:51.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.482+0000 7fdb23a00700 1 -- 172.21.15.38:0/1057059611 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb1c199f10 con 0x7fdb0006dfb0 2024-03-24T01:00:51.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.484+0000 7fdb1a000700 1 -- 172.21.15.38:0/1057059611 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdb1c199f10 con 0x7fdb0006dfb0 2024-03-24T01:00:51.484 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T01:00:51.484 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout: "osd", 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout: "crash" 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "15/23 daemons upgraded", 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mds daemons", 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T01:00:51.485 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T01:00:51.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.486+0000 7fdb23a00700 1 -- 172.21.15.38:0/1057059611 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb0006dfb0 msgr2=0x7fdb00070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:51.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.486+0000 7fdb23a00700 1 --2- 172.21.15.38:0/1057059611 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb0006dfb0 0x7fdb00070470 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00b760 tx=0x7fdb0c009f90 comp rx=0 tx=0).stop 2024-03-24T01:00:51.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.487+0000 7fdb23a00700 1 -- 172.21.15.38:0/1057059611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb1c102530 msgr2=0x7fdb1c19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:51.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.487+0000 7fdb23a00700 1 --2- 172.21.15.38:0/1057059611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb1c102530 0x7fdb1c19ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400e940 tx=0x7fdb0400ed00 comp rx=0 tx=0).stop 2024-03-24T01:00:51.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.487+0000 7fdb23a00700 1 -- 172.21.15.38:0/1057059611 shutdown_connections 2024-03-24T01:00:51.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.487+0000 7fdb23a00700 1 --2- 172.21.15.38:0/1057059611 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fdb0006dfb0 0x7fdb00070470 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.487+0000 7fdb23a00700 1 --2- 172.21.15.38:0/1057059611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb1c101c10 0x7fdb1c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.487+0000 7fdb23a00700 1 --2- 172.21.15.38:0/1057059611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb1c102530 0x7fdb1c19ede0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.487+0000 7fdb23a00700 1 -- 172.21.15.38:0/1057059611 >> 172.21.15.38:0/1057059611 conn(0x7fdb1c076640 msgr2=0x7fdb1c10a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:51.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.487+0000 7fdb23a00700 1 -- 172.21.15.38:0/1057059611 shutdown_connections 2024-03-24T01:00:51.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.487+0000 7fdb23a00700 1 -- 172.21.15.38:0/1057059611 wait complete. 2024-03-24T01:00:51.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.651+0000 7f73c7a00700 1 -- 172.21.15.38:0/2618560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c0102530 msgr2=0x7f73c010aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:51.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.651+0000 7f73c7a00700 1 --2- 172.21.15.38:0/2618560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c0102530 0x7f73c010aa20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009910 tx=0x7f73a8009c20 comp rx=0 tx=0).stop 2024-03-24T01:00:51.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.651+0000 7f73c7a00700 1 -- 172.21.15.38:0/2618560 shutdown_connections 2024-03-24T01:00:51.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.651+0000 7f73c7a00700 1 --2- 172.21.15.38:0/2618560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c0102530 0x7f73c010aa20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.651+0000 7f73c7a00700 1 --2- 172.21.15.38:0/2618560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73c0101c10 0x7f73c0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.651+0000 7f73c7a00700 1 -- 172.21.15.38:0/2618560 >> 172.21.15.38:0/2618560 conn(0x7f73c0076640 msgr2=0x7f73c00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:51.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.652+0000 7f73c7a00700 1 -- 172.21.15.38:0/2618560 shutdown_connections 2024-03-24T01:00:51.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.652+0000 7f73c7a00700 1 -- 172.21.15.38:0/2618560 wait complete. 2024-03-24T01:00:51.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.653+0000 7f73c7a00700 1 Processor -- start 2024-03-24T01:00:51.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.653+0000 7f73c7a00700 1 -- start start 2024-03-24T01:00:51.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.653+0000 7f73c7a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73c0101c10 0x7f73c019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:51.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.653+0000 7f73c7a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c0102530 0x7f73c019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:51.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.653+0000 7f73c7a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c019f470 con 0x7f73c0102530 2024-03-24T01:00:51.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.653+0000 7f73c7a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c0198a50 con 0x7f73c0101c10 2024-03-24T01:00:51.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.654+0000 7f73b7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c0102530 0x7f73c019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:51.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.654+0000 7f73c5000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73c0101c10 0x7f73c019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:51.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.654+0000 7f73b7e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c0102530 0x7f73c019ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42894/0 (socket says 172.21.15.38:42894) 2024-03-24T01:00:51.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.654+0000 7f73b7e00700 1 -- 172.21.15.38:0/775835132 learned_addr learned my addr 172.21.15.38:0/775835132 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:00:51.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.655+0000 7f73c5000700 1 -- 172.21.15.38:0/775835132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c0102530 msgr2=0x7f73c019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:51.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.655+0000 7f73c5000700 1 --2- 172.21.15.38:0/775835132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c0102530 0x7f73c019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.655+0000 7f73c5000700 1 -- 172.21.15.38:0/775835132 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a80095f0 con 0x7f73c0101c10 2024-03-24T01:00:51.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.655+0000 7f73c5000700 1 --2- 172.21.15.38:0/775835132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73c0101c10 0x7f73c019e910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f73b000d8d0 tx=0x7f73b000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:51.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.656+0000 7f73be000700 1 -- 172.21.15.38:0/775835132 <== mon.1 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73b000f860 con 0x7f73c0101c10 2024-03-24T01:00:51.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.656+0000 7f73c7a00700 1 -- 172.21.15.38:0/775835132 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c0198d30 con 0x7f73c0101c10 2024-03-24T01:00:51.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.656+0000 7f73c7a00700 1 -- 172.21.15.38:0/775835132 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c0199280 con 0x7f73c0101c10 2024-03-24T01:00:51.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.656+0000 7f73be000700 1 -- 172.21.15.38:0/775835132 <== mon.1 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73b000fe80 con 0x7f73c0101c10 2024-03-24T01:00:51.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.656+0000 7f73be000700 1 -- 172.21.15.38:0/775835132 <== mon.1 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73b000e3b0 con 0x7f73c0101c10 2024-03-24T01:00:51.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.657+0000 7f73c7a00700 1 -- 172.21.15.38:0/775835132 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739c0052f0 con 0x7f73c0101c10 2024-03-24T01:00:51.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.659+0000 7f73be000700 1 -- 172.21.15.38:0/775835132 <== mon.1 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f73b0010890 con 0x7f73c0101c10 2024-03-24T01:00:51.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.661+0000 7f73be000700 1 --2- 172.21.15.38:0/775835132 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f73a406dfb0 0x7f73a4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:00:51.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.661+0000 7f73be000700 1 -- 172.21.15.38:0/775835132 <== mon.1 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f73b008d8f0 con 0x7f73c0101c10 2024-03-24T01:00:51.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.661+0000 7f73b7e00700 1 --2- 172.21.15.38:0/775835132 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f73a406dfb0 0x7f73a4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:00:51.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.662+0000 7f73b7e00700 1 --2- 172.21.15.38:0/775835132 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f73a406dfb0 0x7f73a4070470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f73c0199c10 tx=0x7f73a801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:00:51.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.676+0000 7f73be000700 1 -- 172.21.15.38:0/775835132 <== mon.1 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f73b005a8a0 con 0x7f73c0101c10 2024-03-24T01:00:51.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.988+0000 7f73c7a00700 1 -- 172.21.15.38:0/775835132 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f739c005160 con 0x7f73c0101c10 2024-03-24T01:00:51.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.989+0000 7f73be000700 1 -- 172.21.15.38:0/775835132 <== mon.1 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f73b005a430 con 0x7f73c0101c10 2024-03-24T01:00:51.990 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-24T01:00:51.990 INFO:teuthology.orchestra.run.smithi038.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-24T01:00:51.990 INFO:teuthology.orchestra.run.smithi038.stdout: fs cephfs is degraded 2024-03-24T01:00:51.990 INFO:teuthology.orchestra.run.smithi038.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-24T01:00:51.990 INFO:teuthology.orchestra.run.smithi038.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-24T01:00:51.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.992+0000 7f73c7a00700 1 -- 172.21.15.38:0/775835132 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f73a406dfb0 msgr2=0x7f73a4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:51.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.992+0000 7f73c7a00700 1 --2- 172.21.15.38:0/775835132 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f73a406dfb0 0x7f73a4070470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f73c0199c10 tx=0x7f73a801a040 comp rx=0 tx=0).stop 2024-03-24T01:00:51.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.992+0000 7f73c7a00700 1 -- 172.21.15.38:0/775835132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73c0101c10 msgr2=0x7f73c019e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:00:51.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.992+0000 7f73c7a00700 1 --2- 172.21.15.38:0/775835132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73c0101c10 0x7f73c019e910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f73b000d8d0 tx=0x7f73b000dbe0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.992+0000 7f73c7a00700 1 -- 172.21.15.38:0/775835132 shutdown_connections 2024-03-24T01:00:51.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.992+0000 7f73c7a00700 1 --2- 172.21.15.38:0/775835132 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f73a406dfb0 0x7f73a4070470 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.992+0000 7f73c7a00700 1 --2- 172.21.15.38:0/775835132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73c0101c10 0x7f73c019e910 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.992+0000 7f73c7a00700 1 --2- 172.21.15.38:0/775835132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c0102530 0x7f73c019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:00:51.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.992+0000 7f73c7a00700 1 -- 172.21.15.38:0/775835132 >> 172.21.15.38:0/775835132 conn(0x7f73c0076640 msgr2=0x7f73c0109350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:00:51.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.992+0000 7f73c7a00700 1 -- 172.21.15.38:0/775835132 shutdown_connections 2024-03-24T01:00:51.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:00:51.992+0000 7f73c7a00700 1 -- 172.21.15.38:0/775835132 wait complete. 2024-03-24T01:00:52.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:52 smithi038 ceph-mon[112108]: pgmap v247: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:52.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:52 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/775835132' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T01:00:52.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:52 smithi088 ceph-mon[87351]: pgmap v247: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:52.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:52 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/775835132' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T01:00:53.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:53 smithi038 ceph-mon[112108]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:53.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:53 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:53.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:53 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:53.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:53 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:00:53.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:53 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.88:6826/1135457323,v1:172.21.15.88:6827/1135457323] up:boot 2024-03-24T01:00:53.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:53 smithi038 ceph-mon[112108]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T01:00:53.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:53 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T01:00:53.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:53 smithi088 ceph-mon[87351]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:00:53.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:53 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:53.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:53 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:53.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:53 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:00:53.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:53 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.88:6826/1135457323,v1:172.21.15.88:6827/1135457323] up:boot 2024-03-24T01:00:53.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:53 smithi088 ceph-mon[87351]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T01:00:53.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:53 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T01:00:54.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:54 smithi088 ceph-mon[87351]: pgmap v248: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:54.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:54 smithi038 ceph-mon[112108]: pgmap v248: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:56 smithi088 ceph-mon[87351]: pgmap v249: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:56.693 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:56 smithi038 ceph-mon[112108]: pgmap v249: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:57.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:00:57.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:57.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:57.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:00:57.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:57.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:00:58.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:00:58 smithi088 ceph-mon[87351]: pgmap v250: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:00:58.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:00:58 smithi038 ceph-mon[112108]: pgmap v250: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:01:00.250 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:00 smithi088 ceph-mon[87351]: pgmap v251: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:01:00.250 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:00 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:00.250 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:00 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:00.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:00 smithi038 ceph-mon[112108]: pgmap v251: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:01:00.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:00 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:00.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:00 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:02.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:02 smithi038 ceph-mon[112108]: pgmap v252: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:01:02.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:02 smithi088 ceph-mon[87351]: pgmap v252: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:01:04.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:04 smithi038 ceph-mon[112108]: pgmap v253: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:01:04.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:04 smithi088 ceph-mon[87351]: pgmap v253: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:01:05.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:05.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:05.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:05.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: pgmap v254: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: Upgrade: Setting container_image for all mds 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.xnpqhp"}]': finished 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T01:01:05.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.yhkmsb"}]': finished 2024-03-24T01:01:05.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T01:01:05.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.cbfvvp"}]': finished 2024-03-24T01:01:05.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T01:01:05.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.yhksbr"}]': finished 2024-03-24T01:01:05.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: Upgrade: Setting filesystem cephfs Joinable 2024-03-24T01:01:05.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:05 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-24T01:01:06.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:06.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:06.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:06.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: pgmap v254: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: Upgrade: Setting container_image for all mds 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.xnpqhp"}]: dispatch 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.xnpqhp"}]': finished 2024-03-24T01:01:06.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.yhkmsb"}]: dispatch 2024-03-24T01:01:06.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.yhkmsb"}]': finished 2024-03-24T01:01:06.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.cbfvvp"}]: dispatch 2024-03-24T01:01:06.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.cbfvvp"}]': finished 2024-03-24T01:01:06.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.yhksbr"}]: dispatch 2024-03-24T01:01:06.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.yhksbr"}]': finished 2024-03-24T01:01:06.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: Upgrade: Setting filesystem cephfs Joinable 2024-03-24T01:01:06.112 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:05 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-24T01:01:07.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:07 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-24T01:01:07.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:07 smithi038 ceph-mon[112108]: Standby daemon mds.cephfs.smithi038.yhkmsb assigned to filesystem cephfs as rank 0 2024-03-24T01:01:07.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:07 smithi038 ceph-mon[112108]: Standby daemon mds.cephfs.smithi038.xnpqhp assigned to filesystem cephfs as rank 1 2024-03-24T01:01:07.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:07 smithi038 ceph-mon[112108]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T01:01:07.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:07 smithi038 ceph-mon[112108]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T01:01:07.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:07 smithi038 ceph-mon[112108]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-24T01:01:07.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:07 smithi038 ceph-mon[112108]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:replay,1=cephfs.smithi038.xnpqhp=up:replay} 2 up:standby 2024-03-24T01:01:07.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:07 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-24T01:01:07.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:07 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-24T01:01:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:07 smithi088 ceph-mon[87351]: Standby daemon mds.cephfs.smithi038.yhkmsb assigned to filesystem cephfs as rank 0 2024-03-24T01:01:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:07 smithi088 ceph-mon[87351]: Standby daemon mds.cephfs.smithi038.xnpqhp assigned to filesystem cephfs as rank 1 2024-03-24T01:01:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:07 smithi088 ceph-mon[87351]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T01:01:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:07 smithi088 ceph-mon[87351]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-24T01:01:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:07 smithi088 ceph-mon[87351]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-24T01:01:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:07 smithi088 ceph-mon[87351]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:replay,1=cephfs.smithi038.xnpqhp=up:replay} 2 up:standby 2024-03-24T01:01:07.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:07 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-24T01:01:08.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: pgmap v255: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 0 op/s 2024-03-24T01:01:08.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-24T01:01:08.360 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] up:resolve 2024-03-24T01:01:08.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:replay,1=cephfs.smithi038.xnpqhp=up:resolve} 2 up:standby 2024-03-24T01:01:08.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:08.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:08.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: Upgrade: Setting container_image for all rgw 2024-03-24T01:01:08.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:08.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:08.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: Upgrade: Setting container_image for all rbd-mirror 2024-03-24T01:01:08.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:08.361 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:08 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:08.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: pgmap v255: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 0 op/s 2024-03-24T01:01:08.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-24T01:01:08.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] up:resolve 2024-03-24T01:01:08.431 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:replay,1=cephfs.smithi038.xnpqhp=up:resolve} 2 up:standby 2024-03-24T01:01:08.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:08.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:08.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: Upgrade: Setting container_image for all rgw 2024-03-24T01:01:08.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:08.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:08.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: Upgrade: Setting container_image for all rbd-mirror 2024-03-24T01:01:08.432 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:08.433 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:08 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:10.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:10 smithi088 ceph-mon[87351]: pgmap v256: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s rd, 2 op/s 2024-03-24T01:01:10.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:10 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:10.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:10 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T01:01:10.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:10 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:10.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:10 smithi038 ceph-mon[112108]: pgmap v256: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s rd, 2 op/s 2024-03-24T01:01:10.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:10 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:10.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:10 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi038", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T01:01:10.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:10 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:11.244 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:11 smithi038 ceph-mon[112108]: Upgrade: Updating ceph-exporter.smithi038 (1/2) 2024-03-24T01:01:11.244 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:11 smithi038 ceph-mon[112108]: Deploying daemon ceph-exporter.smithi038 on smithi038 2024-03-24T01:01:11.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:11 smithi088 ceph-mon[87351]: Upgrade: Updating ceph-exporter.smithi038 (1/2) 2024-03-24T01:01:11.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:11 smithi088 ceph-mon[87351]: Deploying daemon ceph-exporter.smithi038 on smithi038 2024-03-24T01:01:12.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:12 smithi038 ceph-mon[112108]: pgmap v257: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 6.7 MiB/s rd, 3 op/s 2024-03-24T01:01:12.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:12.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:01:12.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:12 smithi088 ceph-mon[87351]: pgmap v257: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 6.7 MiB/s rd, 3 op/s 2024-03-24T01:01:12.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:12 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:12.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:12 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:01:14.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:14 smithi038 ceph-mon[112108]: pgmap v258: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 5 op/s 2024-03-24T01:01:14.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:14 smithi088 ceph-mon[87351]: pgmap v258: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 5 op/s 2024-03-24T01:01:16.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:16 smithi038 ceph-mon[112108]: pgmap v259: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 5 op/s 2024-03-24T01:01:16.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:16 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] up:resolve 2024-03-24T01:01:16.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:16 smithi038 ceph-mon[112108]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:resolve,1=cephfs.smithi038.xnpqhp=up:resolve} 2 up:standby 2024-03-24T01:01:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:16 smithi088 ceph-mon[87351]: pgmap v259: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 5 op/s 2024-03-24T01:01:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:16 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] up:resolve 2024-03-24T01:01:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:16 smithi088 ceph-mon[87351]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:resolve,1=cephfs.smithi038.xnpqhp=up:resolve} 2 up:standby 2024-03-24T01:01:17.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:17 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] up:reconnect 2024-03-24T01:01:17.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:17 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] up:reconnect 2024-03-24T01:01:17.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:17 smithi038 ceph-mon[112108]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:reconnect,1=cephfs.smithi038.xnpqhp=up:reconnect} 2 up:standby 2024-03-24T01:01:17.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:17 smithi038 ceph-mon[112108]: reconnect by client.14532 v1:192.168.0.1:0/3187733578 after 0.00999981 2024-03-24T01:01:17.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:17 smithi038 ceph-mon[112108]: reconnect by client.14538 v1:172.21.15.88:0/43681896 after 0.00999981 2024-03-24T01:01:17.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:17 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] up:reconnect 2024-03-24T01:01:17.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:17 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] up:reconnect 2024-03-24T01:01:17.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:17 smithi088 ceph-mon[87351]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:reconnect,1=cephfs.smithi038.xnpqhp=up:reconnect} 2 up:standby 2024-03-24T01:01:17.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:17 smithi088 ceph-mon[87351]: reconnect by client.14532 v1:192.168.0.1:0/3187733578 after 0.00999981 2024-03-24T01:01:17.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:17 smithi088 ceph-mon[87351]: reconnect by client.14538 v1:172.21.15.88:0/43681896 after 0.00999981 2024-03-24T01:01:18.334 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:18 smithi038 ceph-mon[112108]: pgmap v260: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-24T01:01:18.334 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:18 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] up:rejoin 2024-03-24T01:01:18.334 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:18 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] up:rejoin 2024-03-24T01:01:18.334 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:18 smithi038 ceph-mon[112108]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:rejoin,1=cephfs.smithi038.xnpqhp=up:rejoin} 2 up:standby 2024-03-24T01:01:18.334 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:18 smithi038 ceph-mon[112108]: daemon mds.cephfs.smithi038.xnpqhp is now active in filesystem cephfs as rank 1 2024-03-24T01:01:18.334 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:18 smithi038 ceph-mon[112108]: daemon mds.cephfs.smithi038.yhkmsb is now active in filesystem cephfs as rank 0 2024-03-24T01:01:18.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:18 smithi088 ceph-mon[87351]: pgmap v260: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-24T01:01:18.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:18 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] up:rejoin 2024-03-24T01:01:18.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:18 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] up:rejoin 2024-03-24T01:01:18.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:18 smithi088 ceph-mon[87351]: fsmap cephfs:2/2 {0=cephfs.smithi038.yhkmsb=up:rejoin,1=cephfs.smithi038.xnpqhp=up:rejoin} 2 up:standby 2024-03-24T01:01:18.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:18 smithi088 ceph-mon[87351]: daemon mds.cephfs.smithi038.xnpqhp is now active in filesystem cephfs as rank 1 2024-03-24T01:01:18.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:18 smithi088 ceph-mon[87351]: daemon mds.cephfs.smithi038.yhkmsb is now active in filesystem cephfs as rank 0 2024-03-24T01:01:19.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:19 smithi088 ceph-mon[87351]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T01:01:19.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:19 smithi088 ceph-mon[87351]: Cluster is now healthy 2024-03-24T01:01:19.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:19 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] up:active 2024-03-24T01:01:19.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:19 smithi088 ceph-mon[87351]: mds.? [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] up:active 2024-03-24T01:01:19.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:19 smithi088 ceph-mon[87351]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi038.xnpqhp=up:active} 2 up:standby 2024-03-24T01:01:19.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:19 smithi088 ceph-mon[87351]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi038.xnpqhp=up:active} 2 up:standby-replay 2024-03-24T01:01:19.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:19 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:19.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:19 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:19 smithi038 ceph-mon[112108]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T01:01:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:19 smithi038 ceph-mon[112108]: Cluster is now healthy 2024-03-24T01:01:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:19 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] up:active 2024-03-24T01:01:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:19 smithi038 ceph-mon[112108]: mds.? [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] up:active 2024-03-24T01:01:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:19 smithi038 ceph-mon[112108]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi038.xnpqhp=up:active} 2 up:standby 2024-03-24T01:01:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:19 smithi038 ceph-mon[112108]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi038.xnpqhp=up:active} 2 up:standby-replay 2024-03-24T01:01:19.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:19 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:19.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:19 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:20.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:20 smithi088 ceph-mon[87351]: pgmap v261: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-03-24T01:01:20.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:20 smithi038 ceph-mon[112108]: pgmap v261: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-03-24T01:01:22.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.229+0000 7f5443200700 1 -- 172.21.15.38:0/1800876802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5444068480 msgr2=0x7f54440688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:22.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.229+0000 7f5443200700 1 --2- 172.21.15.38:0/1800876802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5444068480 0x7f54440688e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5430009910 tx=0x7f5430009c20 comp rx=0 tx=0).stop 2024-03-24T01:01:22.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.229+0000 7f5443200700 1 -- 172.21.15.38:0/1800876802 shutdown_connections 2024-03-24T01:01:22.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.229+0000 7f5443200700 1 --2- 172.21.15.38:0/1800876802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5444068480 0x7f54440688e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.229+0000 7f5443200700 1 --2- 172.21.15.38:0/1800876802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54441068e0 0x7f5444106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.229+0000 7f5443200700 1 -- 172.21.15.38:0/1800876802 >> 172.21.15.38:0/1800876802 conn(0x7f5444075370 msgr2=0x7f5444075780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:22.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.229+0000 7f5443200700 1 -- 172.21.15.38:0/1800876802 shutdown_connections 2024-03-24T01:01:22.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.229+0000 7f5443200700 1 -- 172.21.15.38:0/1800876802 wait complete. 2024-03-24T01:01:22.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.230+0000 7f5443200700 1 Processor -- start 2024-03-24T01:01:22.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.231+0000 7f5443200700 1 -- start start 2024-03-24T01:01:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.231+0000 7f5443200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5444068480 0x7f544419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.231+0000 7f5443200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54441068e0 0x7f544419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.231+0000 7f5443200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544419f2b0 con 0x7f5444068480 2024-03-24T01:01:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.231+0000 7f5443200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5444198870 con 0x7f54441068e0 2024-03-24T01:01:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.231+0000 7f5441e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5444068480 0x7f544419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.231+0000 7f5441e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5444068480 0x7f544419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59232/0 (socket says 172.21.15.38:59232) 2024-03-24T01:01:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.231+0000 7f5441e00700 1 -- 172.21.15.38:0/1353551539 learned_addr learned my addr 172.21.15.38:0/1353551539 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:01:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.232+0000 7f5441400700 1 --2- 172.21.15.38:0/1353551539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54441068e0 0x7f544419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.232+0000 7f5441e00700 1 -- 172.21.15.38:0/1353551539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54441068e0 msgr2=0x7f544419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.232+0000 7f5441e00700 1 --2- 172.21.15.38:0/1353551539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54441068e0 0x7f544419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.232+0000 7f5441e00700 1 -- 172.21.15.38:0/1353551539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54300095f0 con 0x7f5444068480 2024-03-24T01:01:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.232+0000 7f5441e00700 1 --2- 172.21.15.38:0/1353551539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5444068480 0x7f544419e6e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f543400e920 tx=0x7f543400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.232+0000 7f543aa00700 1 -- 172.21.15.38:0/1353551539 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f543400c9d0 con 0x7f5444068480 2024-03-24T01:01:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.233+0000 7f543aa00700 1 -- 172.21.15.38:0/1353551539 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5434010430 con 0x7f5444068480 2024-03-24T01:01:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.233+0000 7f5443200700 1 -- 172.21.15.38:0/1353551539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5444198b50 con 0x7f5444068480 2024-03-24T01:01:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.233+0000 7f543aa00700 1 -- 172.21.15.38:0/1353551539 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5434018630 con 0x7f5444068480 2024-03-24T01:01:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.233+0000 7f5443200700 1 -- 172.21.15.38:0/1353551539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54441990a0 con 0x7f5444068480 2024-03-24T01:01:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.234+0000 7f5443200700 1 -- 172.21.15.38:0/1353551539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5444104b90 con 0x7f5444068480 2024-03-24T01:01:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.236+0000 7f543aa00700 1 -- 172.21.15.38:0/1353551539 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f54340105a0 con 0x7f5444068480 2024-03-24T01:01:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.237+0000 7f543aa00700 1 --2- 172.21.15.38:0/1353551539 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f542806dfb0 0x7f5428070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.237+0000 7f543aa00700 1 -- 172.21.15.38:0/1353551539 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5434014070 con 0x7f5444068480 2024-03-24T01:01:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.238+0000 7f5441400700 1 --2- 172.21.15.38:0/1353551539 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f542806dfb0 0x7f5428070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.238+0000 7f5441400700 1 --2- 172.21.15.38:0/1353551539 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f542806dfb0 0x7f5428070470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5444199c40 tx=0x7f5430005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:22.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.243+0000 7f543aa00700 1 -- 172.21.15.38:0/1353551539 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5434053740 con 0x7f5444068480 2024-03-24T01:01:22.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:22 smithi088 ceph-mon[87351]: pgmap v262: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 9 op/s 2024-03-24T01:01:22.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:22 smithi088 ceph-mon[87351]: Upgrade: Updating ceph-exporter.smithi088 (2/2) 2024-03-24T01:01:22.442 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:22 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:22.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:22 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T01:01:22.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:22 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:22.443 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:22 smithi088 ceph-mon[87351]: Deploying daemon ceph-exporter.smithi088 on smithi088 2024-03-24T01:01:22.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.501+0000 7f5443200700 1 -- 172.21.15.38:0/1353551539 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5444199ce0 con 0x7f542806dfb0 2024-03-24T01:01:22.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:22 smithi038 ceph-mon[112108]: pgmap v262: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 9 op/s 2024-03-24T01:01:22.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:22 smithi038 ceph-mon[112108]: Upgrade: Updating ceph-exporter.smithi088 (2/2) 2024-03-24T01:01:22.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:22 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:22.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:22 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-24T01:01:22.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:22 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:22.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:22 smithi038 ceph-mon[112108]: Deploying daemon ceph-exporter.smithi088 on smithi088 2024-03-24T01:01:22.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.503+0000 7f543aa00700 1 -- 172.21.15.38:0/1353551539 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f5444199ce0 con 0x7f542806dfb0 2024-03-24T01:01:22.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.507+0000 7f5443200700 1 -- 172.21.15.38:0/1353551539 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f542806dfb0 msgr2=0x7f5428070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:22.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.507+0000 7f5443200700 1 --2- 172.21.15.38:0/1353551539 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f542806dfb0 0x7f5428070470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5444199c40 tx=0x7f5430005a90 comp rx=0 tx=0).stop 2024-03-24T01:01:22.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.507+0000 7f5443200700 1 -- 172.21.15.38:0/1353551539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5444068480 msgr2=0x7f544419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:22.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.507+0000 7f5443200700 1 --2- 172.21.15.38:0/1353551539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5444068480 0x7f544419e6e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f543400e920 tx=0x7f543400ec30 comp rx=0 tx=0).stop 2024-03-24T01:01:22.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.508+0000 7f5443200700 1 -- 172.21.15.38:0/1353551539 shutdown_connections 2024-03-24T01:01:22.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.508+0000 7f5443200700 1 --2- 172.21.15.38:0/1353551539 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f542806dfb0 0x7f5428070470 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.508+0000 7f5443200700 1 --2- 172.21.15.38:0/1353551539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5444068480 0x7f544419e6e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.508+0000 7f5443200700 1 --2- 172.21.15.38:0/1353551539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54441068e0 0x7f544419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.508+0000 7f5443200700 1 -- 172.21.15.38:0/1353551539 >> 172.21.15.38:0/1353551539 conn(0x7f5444075370 msgr2=0x7f54441010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:22.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.508+0000 7f5443200700 1 -- 172.21.15.38:0/1353551539 shutdown_connections 2024-03-24T01:01:22.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.508+0000 7f5443200700 1 -- 172.21.15.38:0/1353551539 wait complete. 2024-03-24T01:01:22.527 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T01:01:22.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.684+0000 7f744f000700 1 -- 172.21.15.38:0/3616309442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7448102530 msgr2=0x7f744810aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:22.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.684+0000 7f744f000700 1 --2- 172.21.15.38:0/3616309442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7448102530 0x7f744810aa20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7434009910 tx=0x7f7434009c20 comp rx=0 tx=0).stop 2024-03-24T01:01:22.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.684+0000 7f744f000700 1 -- 172.21.15.38:0/3616309442 shutdown_connections 2024-03-24T01:01:22.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.684+0000 7f744f000700 1 --2- 172.21.15.38:0/3616309442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7448102530 0x7f744810aa20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.684+0000 7f744f000700 1 --2- 172.21.15.38:0/3616309442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7448101c10 0x7f7448101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.684+0000 7f744f000700 1 -- 172.21.15.38:0/3616309442 >> 172.21.15.38:0/3616309442 conn(0x7f7448076640 msgr2=0x7f74480fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:22.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.684+0000 7f744f000700 1 -- 172.21.15.38:0/3616309442 shutdown_connections 2024-03-24T01:01:22.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.684+0000 7f744f000700 1 -- 172.21.15.38:0/3616309442 wait complete. 2024-03-24T01:01:22.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.685+0000 7f744f000700 1 Processor -- start 2024-03-24T01:01:22.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.686+0000 7f744f000700 1 -- start start 2024-03-24T01:01:22.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.686+0000 7f744f000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7448101c10 0x7f74480ff870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:22.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.686+0000 7f744f000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7448102530 0x7f74480ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:22.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.686+0000 7f744f000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74481002f0 con 0x7f7448102530 2024-03-24T01:01:22.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.686+0000 7f744f000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448100430 con 0x7f7448101c10 2024-03-24T01:01:22.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.687+0000 7f7447400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7448102530 0x7f74480ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:22.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.687+0000 7f7447400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7448102530 0x7f74480ffdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59256/0 (socket says 172.21.15.38:59256) 2024-03-24T01:01:22.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.687+0000 7f7447400700 1 -- 172.21.15.38:0/3796090273 learned_addr learned my addr 172.21.15.38:0/3796090273 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:01:22.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.687+0000 7f7447e00700 1 --2- 172.21.15.38:0/3796090273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7448101c10 0x7f74480ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:22.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.687+0000 7f7447400700 1 -- 172.21.15.38:0/3796090273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7448101c10 msgr2=0x7f74480ff870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:22.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.687+0000 7f7447400700 1 --2- 172.21.15.38:0/3796090273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7448101c10 0x7f74480ff870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.687+0000 7f7447400700 1 -- 172.21.15.38:0/3796090273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74340095f0 con 0x7f7448102530 2024-03-24T01:01:22.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.688+0000 7f7447400700 1 --2- 172.21.15.38:0/3796090273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7448102530 0x7f74480ffdb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f74340092e0 tx=0x7f743400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:22.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.688+0000 7f7444c00700 1 -- 172.21.15.38:0/3796090273 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f743401d070 con 0x7f7448102530 2024-03-24T01:01:22.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.688+0000 7f744f000700 1 -- 172.21.15.38:0/3796090273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74481006b0 con 0x7f7448102530 2024-03-24T01:01:22.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.688+0000 7f7444c00700 1 -- 172.21.15.38:0/3796090273 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7434005dc0 con 0x7f7448102530 2024-03-24T01:01:22.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.688+0000 7f7444c00700 1 -- 172.21.15.38:0/3796090273 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f743400fdb0 con 0x7f7448102530 2024-03-24T01:01:22.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.688+0000 7f744f000700 1 -- 172.21.15.38:0/3796090273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74481a2e30 con 0x7f7448102530 2024-03-24T01:01:22.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.690+0000 7f744f000700 1 -- 172.21.15.38:0/3796090273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7448068d60 con 0x7f7448102530 2024-03-24T01:01:22.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.690+0000 7f7444c00700 1 -- 172.21.15.38:0/3796090273 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f743400f460 con 0x7f7448102530 2024-03-24T01:01:22.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.691+0000 7f7444c00700 1 --2- 172.21.15.38:0/3796090273 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f742c06dfa0 0x7f742c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:22.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.692+0000 7f7444c00700 1 -- 172.21.15.38:0/3796090273 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f743408efa0 con 0x7f7448102530 2024-03-24T01:01:22.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.692+0000 7f7447e00700 1 --2- 172.21.15.38:0/3796090273 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f742c06dfa0 0x7f742c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:22.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.693+0000 7f7447e00700 1 --2- 172.21.15.38:0/3796090273 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f742c06dfa0 0x7f742c070460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7438005950 tx=0x7f743800b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:22.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.698+0000 7f7444c00700 1 -- 172.21.15.38:0/3796090273 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7434058440 con 0x7f7448102530 2024-03-24T01:01:22.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.972+0000 7f744f000700 1 -- 172.21.15.38:0/3796090273 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7448105310 con 0x7f742c06dfa0 2024-03-24T01:01:22.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.973+0000 7f7444c00700 1 -- 172.21.15.38:0/3796090273 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f7448105310 con 0x7f742c06dfa0 2024-03-24T01:01:22.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.977+0000 7f744f000700 1 -- 172.21.15.38:0/3796090273 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f742c06dfa0 msgr2=0x7f742c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:22.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.977+0000 7f744f000700 1 --2- 172.21.15.38:0/3796090273 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f742c06dfa0 0x7f742c070460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7438005950 tx=0x7f743800b410 comp rx=0 tx=0).stop 2024-03-24T01:01:22.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.977+0000 7f744f000700 1 -- 172.21.15.38:0/3796090273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7448102530 msgr2=0x7f74480ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:22.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.977+0000 7f744f000700 1 --2- 172.21.15.38:0/3796090273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7448102530 0x7f74480ffdb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f74340092e0 tx=0x7f743400bb40 comp rx=0 tx=0).stop 2024-03-24T01:01:22.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.977+0000 7f744f000700 1 -- 172.21.15.38:0/3796090273 shutdown_connections 2024-03-24T01:01:22.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.977+0000 7f744f000700 1 --2- 172.21.15.38:0/3796090273 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f742c06dfa0 0x7f742c070460 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.977+0000 7f744f000700 1 --2- 172.21.15.38:0/3796090273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7448101c10 0x7f74480ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.977+0000 7f744f000700 1 --2- 172.21.15.38:0/3796090273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7448102530 0x7f74480ffdb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:22.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.978+0000 7f744f000700 1 -- 172.21.15.38:0/3796090273 >> 172.21.15.38:0/3796090273 conn(0x7f7448076640 msgr2=0x7f7448102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:22.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.978+0000 7f744f000700 1 -- 172.21.15.38:0/3796090273 shutdown_connections 2024-03-24T01:01:22.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:22.978+0000 7f744f000700 1 -- 172.21.15.38:0/3796090273 wait complete. 2024-03-24T01:01:23.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.146+0000 7f630de00700 1 -- 172.21.15.38:0/3647409895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6308105950 msgr2=0x7f6308105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:23.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.146+0000 7f630de00700 1 --2- 172.21.15.38:0/3647409895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6308105950 0x7f6308105d30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009910 tx=0x7f62f8009c20 comp rx=0 tx=0).stop 2024-03-24T01:01:23.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.146+0000 7f630de00700 1 -- 172.21.15.38:0/3647409895 shutdown_connections 2024-03-24T01:01:23.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.146+0000 7f630de00700 1 --2- 172.21.15.38:0/3647409895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63080ff950 0x7f63080ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.146+0000 7f630de00700 1 --2- 172.21.15.38:0/3647409895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6308105950 0x7f6308105d30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.146+0000 7f630de00700 1 -- 172.21.15.38:0/3647409895 >> 172.21.15.38:0/3647409895 conn(0x7f6308076640 msgr2=0x7f63080fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:23.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.146+0000 7f630de00700 1 -- 172.21.15.38:0/3647409895 shutdown_connections 2024-03-24T01:01:23.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.146+0000 7f630de00700 1 -- 172.21.15.38:0/3647409895 wait complete. 2024-03-24T01:01:23.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.147+0000 7f630de00700 1 Processor -- start 2024-03-24T01:01:23.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.147+0000 7f630de00700 1 -- start start 2024-03-24T01:01:23.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.148+0000 7f630de00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63080ff950 0x7f630819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:23.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.148+0000 7f630de00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6308105950 0x7f630819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:23.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.148+0000 7f630de00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6308198a10 con 0x7f63080ff950 2024-03-24T01:01:23.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.148+0000 7f630de00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6308198b80 con 0x7f6308105950 2024-03-24T01:01:23.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.148+0000 7f6306a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63080ff950 0x7f630819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:23.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.149+0000 7f6306a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63080ff950 0x7f630819e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34982/0 (socket says 172.21.15.38:34982) 2024-03-24T01:01:23.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.149+0000 7f6306a00700 1 -- 172.21.15.38:0/3338794803 learned_addr learned my addr 172.21.15.38:0/3338794803 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:01:23.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.149+0000 7f6306000700 1 --2- 172.21.15.38:0/3338794803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6308105950 0x7f630819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:23.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.149+0000 7f6306a00700 1 -- 172.21.15.38:0/3338794803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6308105950 msgr2=0x7f630819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:23.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.149+0000 7f6306a00700 1 --2- 172.21.15.38:0/3338794803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6308105950 0x7f630819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.149+0000 7f6306a00700 1 -- 172.21.15.38:0/3338794803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f80095f0 con 0x7f63080ff950 2024-03-24T01:01:23.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.150+0000 7f6306a00700 1 --2- 172.21.15.38:0/3338794803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63080ff950 0x7f630819e960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f62f80092e0 tx=0x7f62f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:23.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.150+0000 7f630ca00700 1 -- 172.21.15.38:0/3338794803 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62f801d070 con 0x7f63080ff950 2024-03-24T01:01:23.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.150+0000 7f630ca00700 1 -- 172.21.15.38:0/3338794803 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62f80056e0 con 0x7f63080ff950 2024-03-24T01:01:23.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.150+0000 7f630ca00700 1 -- 172.21.15.38:0/3338794803 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62f800fe10 con 0x7f63080ff950 2024-03-24T01:01:23.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.150+0000 7f630de00700 1 -- 172.21.15.38:0/3338794803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6308198e00 con 0x7f63080ff950 2024-03-24T01:01:23.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.150+0000 7f630de00700 1 -- 172.21.15.38:0/3338794803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63081992a0 con 0x7f63080ff950 2024-03-24T01:01:23.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.152+0000 7f630de00700 1 -- 172.21.15.38:0/3338794803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6308101090 con 0x7f63080ff950 2024-03-24T01:01:23.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.153+0000 7f630ca00700 1 -- 172.21.15.38:0/3338794803 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f62f800f680 con 0x7f63080ff950 2024-03-24T01:01:23.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.154+0000 7f630ca00700 1 --2- 172.21.15.38:0/3338794803 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f62ec06dfb0 0x7f62ec070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:23.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.154+0000 7f630ca00700 1 -- 172.21.15.38:0/3338794803 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f62f808f200 con 0x7f63080ff950 2024-03-24T01:01:23.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.155+0000 7f6306000700 1 --2- 172.21.15.38:0/3338794803 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f62ec06dfb0 0x7f62ec070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:23.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.156+0000 7f6306000700 1 --2- 172.21.15.38:0/3338794803 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f62ec06dfb0 0x7f62ec070470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f630819a180 tx=0x7f62f400b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:23.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.163+0000 7f630ca00700 1 -- 172.21.15.38:0/3338794803 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f62f8058600 con 0x7f63080ff950 2024-03-24T01:01:23.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.420+0000 7f630de00700 1 -- 172.21.15.38:0/3338794803 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6308199f60 con 0x7f62ec06dfb0 2024-03-24T01:01:23.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.441+0000 7f630ca00700 1 -- 172.21.15.38:0/3338794803 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6308199f60 con 0x7f62ec06dfb0 2024-03-24T01:01:23.442 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (9m) 73s ago 23m 14.0M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 starting - - - - 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (22m) 26s ago 22m 9307k - 18.2.2-952-g738c2fd6 68aff9aff3b2 3254c089657b 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (5m) 73s ago 24m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (5m) 26s ago 22m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (9m) 73s ago 23m 75.4M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (80s) 73s ago 16m 10.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b78588cbee24 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (107s) 73s ago 16m 11.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e597065368f0 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (56s) 26s ago 16m 11.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 49be5a5d2c69 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (31s) 26s ago 16m 10.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b40c5fbe832e 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (12m) 73s ago 25m 510M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (11m) 26s ago 22m 452M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T01:01:23.443 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (7m) 73s ago 25m 55.2M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T01:01:23.444 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (7m) 26s ago 22m 48.3M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T01:01:23.444 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (10m) 73s ago 23m 7960k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T01:01:23.444 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (10m) 26s ago 22m 7642k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T01:01:23.444 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (5m) 73s ago 21m 102M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T01:01:23.444 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (4m) 73s ago 20m 108M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cb72ac446ba7 2024-03-24T01:01:23.444 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (4m) 73s ago 20m 91.4M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c73f3e8276b6 2024-03-24T01:01:23.444 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (3m) 26s ago 19m 110M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b2afaff761d8 2024-03-24T01:01:23.444 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (2m) 26s ago 18m 104M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9a6f202043cd 2024-03-24T01:01:23.444 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (2m) 26s ago 18m 100M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 866388fc07f6 2024-03-24T01:01:23.444 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (10m) 73s ago 23m 74.1M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T01:01:23.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.445+0000 7f630de00700 1 -- 172.21.15.38:0/3338794803 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f62ec06dfb0 msgr2=0x7f62ec070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:23.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.445+0000 7f630de00700 1 --2- 172.21.15.38:0/3338794803 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f62ec06dfb0 0x7f62ec070470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f630819a180 tx=0x7f62f400b410 comp rx=0 tx=0).stop 2024-03-24T01:01:23.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.445+0000 7f630de00700 1 -- 172.21.15.38:0/3338794803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63080ff950 msgr2=0x7f630819e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:23.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.445+0000 7f630de00700 1 --2- 172.21.15.38:0/3338794803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63080ff950 0x7f630819e960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f62f80092e0 tx=0x7f62f800bd70 comp rx=0 tx=0).stop 2024-03-24T01:01:23.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.445+0000 7f630de00700 1 -- 172.21.15.38:0/3338794803 shutdown_connections 2024-03-24T01:01:23.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.445+0000 7f630de00700 1 --2- 172.21.15.38:0/3338794803 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f62ec06dfb0 0x7f62ec070470 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.445+0000 7f630de00700 1 --2- 172.21.15.38:0/3338794803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63080ff950 0x7f630819e960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.445+0000 7f630de00700 1 --2- 172.21.15.38:0/3338794803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6308105950 0x7f630819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.445+0000 7f630de00700 1 -- 172.21.15.38:0/3338794803 >> 172.21.15.38:0/3338794803 conn(0x7f6308076640 msgr2=0x7f6308102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:23.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.445+0000 7f630de00700 1 -- 172.21.15.38:0/3338794803 shutdown_connections 2024-03-24T01:01:23.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.445+0000 7f630de00700 1 -- 172.21.15.38:0/3338794803 wait complete. 2024-03-24T01:01:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.618+0000 7f652a600700 1 -- 172.21.15.38:0/501312898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6524068480 msgr2=0x7f65240688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.618+0000 7f652a600700 1 --2- 172.21.15.38:0/501312898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6524068480 0x7f65240688e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6510009910 tx=0x7f6510009c20 comp rx=0 tx=0).stop 2024-03-24T01:01:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.619+0000 7f652a600700 1 -- 172.21.15.38:0/501312898 shutdown_connections 2024-03-24T01:01:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.619+0000 7f652a600700 1 --2- 172.21.15.38:0/501312898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6524068480 0x7f65240688e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.619+0000 7f652a600700 1 --2- 172.21.15.38:0/501312898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65241068e0 0x7f6524106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.619+0000 7f652a600700 1 -- 172.21.15.38:0/501312898 >> 172.21.15.38:0/501312898 conn(0x7f6524075370 msgr2=0x7f6524075780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.619+0000 7f652a600700 1 -- 172.21.15.38:0/501312898 shutdown_connections 2024-03-24T01:01:23.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.619+0000 7f652a600700 1 -- 172.21.15.38:0/501312898 wait complete. 2024-03-24T01:01:23.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.620+0000 7f652a600700 1 Processor -- start 2024-03-24T01:01:23.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.620+0000 7f652a600700 1 -- start start 2024-03-24T01:01:23.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.620+0000 7f652a600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6524068480 0x7f6524198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:23.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.620+0000 7f652a600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65241068e0 0x7f6524198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:23.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.620+0000 7f652a600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6524199630 con 0x7f6524068480 2024-03-24T01:01:23.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.620+0000 7f652a600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652419d380 con 0x7f65241068e0 2024-03-24T01:01:23.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.621+0000 7f6529200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6524068480 0x7f6524198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:23.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.621+0000 7f6529200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6524068480 0x7f6524198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35004/0 (socket says 172.21.15.38:35004) 2024-03-24T01:01:23.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.621+0000 7f6529200700 1 -- 172.21.15.38:0/76856765 learned_addr learned my addr 172.21.15.38:0/76856765 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:01:23.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.621+0000 7f6523e00700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65241068e0 0x7f6524198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:23.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.621+0000 7f6529200700 1 -- 172.21.15.38:0/76856765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65241068e0 msgr2=0x7f6524198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:23.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.621+0000 7f6529200700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65241068e0 0x7f6524198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.621+0000 7f6529200700 1 -- 172.21.15.38:0/76856765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65100095f0 con 0x7f6524068480 2024-03-24T01:01:23.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.621+0000 7f6523e00700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65241068e0 0x7f6524198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T01:01:23.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.622+0000 7f6529200700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6524068480 0x7f6524198a60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f651800d8d0 tx=0x7f651800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:23.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.622+0000 7f6521600700 1 -- 172.21.15.38:0/76856765 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6518009940 con 0x7f6524068480 2024-03-24T01:01:23.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.622+0000 7f6521600700 1 -- 172.21.15.38:0/76856765 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6518010460 con 0x7f6524068480 2024-03-24T01:01:23.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.622+0000 7f6521600700 1 -- 172.21.15.38:0/76856765 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f651800f5f0 con 0x7f6524068480 2024-03-24T01:01:23.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.622+0000 7f652a600700 1 -- 172.21.15.38:0/76856765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652419d660 con 0x7f6524068480 2024-03-24T01:01:23.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.622+0000 7f652a600700 1 -- 172.21.15.38:0/76856765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652419dbb0 con 0x7f6524068480 2024-03-24T01:01:23.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.624+0000 7f652a600700 1 -- 172.21.15.38:0/76856765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652410ae80 con 0x7f6524068480 2024-03-24T01:01:23.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.625+0000 7f6521600700 1 -- 172.21.15.38:0/76856765 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f65180105d0 con 0x7f6524068480 2024-03-24T01:01:23.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.626+0000 7f6521600700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f650c06dff0 0x7f650c0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:23.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.626+0000 7f6521600700 1 -- 172.21.15.38:0/76856765 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f651808dd60 con 0x7f6524068480 2024-03-24T01:01:23.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.627+0000 7f6523e00700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f650c06dff0 0x7f650c0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:23.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.627+0000 7f6523e00700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f650c06dff0 0x7f650c0704b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6524199fc0 tx=0x7f651001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:23.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.633+0000 7f6521600700 1 -- 172.21.15.38:0/76856765 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f651805adc0 con 0x7f6524068480 2024-03-24T01:01:23.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.990+0000 7f652a600700 1 -- 172.21.15.38:0/76856765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f652419d7f0 con 0x7f6524068480 2024-03-24T01:01:23.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.991+0000 7f6521600700 1 -- 172.21.15.38:0/76856765 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f652419d7f0 con 0x7f6524068480 2024-03-24T01:01:23.992 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T01:01:23.992 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T01:01:23.992 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T01:01:23.992 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:01:23.992 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T01:01:23.992 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T01:01:23.992 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:01:23.992 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T01:01:23.993 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-24T01:01:23.993 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:01:23.993 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T01:01:23.993 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-24T01:01:23.993 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:01:23.993 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T01:01:23.993 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 14 2024-03-24T01:01:23.993 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T01:01:23.993 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T01:01:23.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.995+0000 7f652a600700 1 -- 172.21.15.38:0/76856765 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f650c06dff0 msgr2=0x7f650c0704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:23.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.995+0000 7f652a600700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f650c06dff0 0x7f650c0704b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6524199fc0 tx=0x7f651001a040 comp rx=0 tx=0).stop 2024-03-24T01:01:23.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.995+0000 7f652a600700 1 -- 172.21.15.38:0/76856765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6524068480 msgr2=0x7f6524198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:23.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.995+0000 7f652a600700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6524068480 0x7f6524198a60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f651800d8d0 tx=0x7f651800dc90 comp rx=0 tx=0).stop 2024-03-24T01:01:23.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.995+0000 7f652a600700 1 -- 172.21.15.38:0/76856765 shutdown_connections 2024-03-24T01:01:23.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.995+0000 7f652a600700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f650c06dff0 0x7f650c0704b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.995+0000 7f652a600700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6524068480 0x7f6524198a60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.995+0000 7f652a600700 1 --2- 172.21.15.38:0/76856765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65241068e0 0x7f6524198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:23.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.996+0000 7f652a600700 1 -- 172.21.15.38:0/76856765 >> 172.21.15.38:0/76856765 conn(0x7f6524075370 msgr2=0x7f6524104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:23.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.996+0000 7f652a600700 1 -- 172.21.15.38:0/76856765 shutdown_connections 2024-03-24T01:01:23.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:23.996+0000 7f652a600700 1 -- 172.21.15.38:0/76856765 wait complete. 2024-03-24T01:01:24.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.142+0000 7f9462600700 1 -- 172.21.15.38:0/1983282799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945c1057d0 msgr2=0x7f945c105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:24.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.142+0000 7f9462600700 1 --2- 172.21.15.38:0/1983282799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945c1057d0 0x7f945c105bb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9450009910 tx=0x7f9450009c20 comp rx=0 tx=0).stop 2024-03-24T01:01:24.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.143+0000 7f9462600700 1 -- 172.21.15.38:0/1983282799 shutdown_connections 2024-03-24T01:01:24.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.143+0000 7f9462600700 1 --2- 172.21.15.38:0/1983282799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f945c0ff7d0 0x7f945c0ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.143+0000 7f9462600700 1 --2- 172.21.15.38:0/1983282799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945c1057d0 0x7f945c105bb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.143+0000 7f9462600700 1 -- 172.21.15.38:0/1983282799 >> 172.21.15.38:0/1983282799 conn(0x7f945c0fb2e0 msgr2=0x7f945c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:24.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.143+0000 7f9462600700 1 -- 172.21.15.38:0/1983282799 shutdown_connections 2024-03-24T01:01:24.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.143+0000 7f9462600700 1 -- 172.21.15.38:0/1983282799 wait complete. 2024-03-24T01:01:24.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.145+0000 7f9462600700 1 Processor -- start 2024-03-24T01:01:24.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.145+0000 7f9462600700 1 -- start start 2024-03-24T01:01:24.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.145+0000 7f9462600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945c0ff7d0 0x7f945c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:24.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.145+0000 7f9462600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f945c1057d0 0x7f945c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:24.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.145+0000 7f9462600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c19f2c0 con 0x7f945c0ff7d0 2024-03-24T01:01:24.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.145+0000 7f9462600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c198830 con 0x7f945c1057d0 2024-03-24T01:01:24.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.146+0000 7f9461200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945c0ff7d0 0x7f945c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:24.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.146+0000 7f9461200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945c0ff7d0 0x7f945c19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35022/0 (socket says 172.21.15.38:35022) 2024-03-24T01:01:24.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.146+0000 7f9461200700 1 -- 172.21.15.38:0/1688371189 learned_addr learned my addr 172.21.15.38:0/1688371189 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:01:24.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.146+0000 7f9461200700 1 -- 172.21.15.38:0/1688371189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f945c1057d0 msgr2=0x7f945c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:24.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.146+0000 7f9461200700 1 --2- 172.21.15.38:0/1688371189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f945c1057d0 0x7f945c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.146+0000 7f9461200700 1 -- 172.21.15.38:0/1688371189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94500095f0 con 0x7f945c0ff7d0 2024-03-24T01:01:24.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.147+0000 7f9461200700 1 --2- 172.21.15.38:0/1688371189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945c0ff7d0 0x7f945c19e6f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f94500092e0 tx=0x7f945000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:24.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.147+0000 7f9459600700 1 -- 172.21.15.38:0/1688371189 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f945001d070 con 0x7f945c0ff7d0 2024-03-24T01:01:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.147+0000 7f9459600700 1 -- 172.21.15.38:0/1688371189 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94500056e0 con 0x7f945c0ff7d0 2024-03-24T01:01:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.147+0000 7f9459600700 1 -- 172.21.15.38:0/1688371189 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f945000fe80 con 0x7f945c0ff7d0 2024-03-24T01:01:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.147+0000 7f9462600700 1 -- 172.21.15.38:0/1688371189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945c198ab0 con 0x7f945c0ff7d0 2024-03-24T01:01:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.147+0000 7f9462600700 1 -- 172.21.15.38:0/1688371189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945c198fa0 con 0x7f945c0ff7d0 2024-03-24T01:01:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.149+0000 7f9462600700 1 -- 172.21.15.38:0/1688371189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945c100f10 con 0x7f945c0ff7d0 2024-03-24T01:01:24.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.151+0000 7f9459600700 1 -- 172.21.15.38:0/1688371189 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f945000f6f0 con 0x7f945c0ff7d0 2024-03-24T01:01:24.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.152+0000 7f9459600700 1 --2- 172.21.15.38:0/1688371189 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f944406dfb0 0x7f9444070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:24.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.153+0000 7f9459600700 1 -- 172.21.15.38:0/1688371189 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f945008f3d0 con 0x7f945c0ff7d0 2024-03-24T01:01:24.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.153+0000 7f945be00700 1 --2- 172.21.15.38:0/1688371189 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f944406dfb0 0x7f9444070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:24.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.154+0000 7f945be00700 1 --2- 172.21.15.38:0/1688371189 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f944406dfb0 0x7f9444070470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9448005950 tx=0x7f94480058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.156+0000 7f9459600700 1 -- 172.21.15.38:0/1688371189 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f945005c380 con 0x7f945c0ff7d0 2024-03-24T01:01:24.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.448+0000 7f9462600700 1 -- 172.21.15.38:0/1688371189 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f945c199280 con 0x7f945c0ff7d0 2024-03-24T01:01:24.450 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:24 smithi038 ceph-mon[112108]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:01:24.450 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:24 smithi038 ceph-mon[112108]: pgmap v263: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 170 B/s wr, 15 op/s 2024-03-24T01:01:24.450 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:24 smithi038 ceph-mon[112108]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:01:24.450 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:24 smithi038 ceph-mon[112108]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:01:24.450 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:24 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/76856765' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:24.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.449+0000 7f9459600700 1 -- 172.21.15.38:0/1688371189 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 39 v39) v1 ==== 76+0+1939 (secure 0 0 0) 0x7f945005bf10 con 0x7f945c0ff7d0 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:e39 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 39 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T01:01:18.259747+0000 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T01:01:24.451 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 83 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=34446,1=34478} 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T01:01:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 34446 members: 34478,34446 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:34446} state up:active seq 29 join_fscid=1 addr [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{0:44345} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2293661188,v1:172.21.15.88:6825/2293661188] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{1:34478} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:44373} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/1135457323,v1:172.21.15.88:6827/1135457323] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:01:24.453 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:01:24.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.452+0000 7f9462600700 1 -- 172.21.15.38:0/1688371189 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f944406dfb0 msgr2=0x7f9444070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:24.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.452+0000 7f9462600700 1 --2- 172.21.15.38:0/1688371189 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f944406dfb0 0x7f9444070470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9448005950 tx=0x7f94480058e0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.452+0000 7f9462600700 1 -- 172.21.15.38:0/1688371189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945c0ff7d0 msgr2=0x7f945c19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:24.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.452+0000 7f9462600700 1 --2- 172.21.15.38:0/1688371189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945c0ff7d0 0x7f945c19e6f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f94500092e0 tx=0x7f945000be20 comp rx=0 tx=0).stop 2024-03-24T01:01:24.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.452+0000 7f9462600700 1 -- 172.21.15.38:0/1688371189 shutdown_connections 2024-03-24T01:01:24.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.452+0000 7f9462600700 1 --2- 172.21.15.38:0/1688371189 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f944406dfb0 0x7f9444070470 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.452+0000 7f9462600700 1 --2- 172.21.15.38:0/1688371189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f945c0ff7d0 0x7f945c19e6f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.452+0000 7f9462600700 1 --2- 172.21.15.38:0/1688371189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f945c1057d0 0x7f945c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.452+0000 7f9462600700 1 -- 172.21.15.38:0/1688371189 >> 172.21.15.38:0/1688371189 conn(0x7f945c0fb2e0 msgr2=0x7f945c0fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:24.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.453+0000 7f9462600700 1 -- 172.21.15.38:0/1688371189 shutdown_connections 2024-03-24T01:01:24.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.453+0000 7f9462600700 1 -- 172.21.15.38:0/1688371189 wait complete. 2024-03-24T01:01:24.456 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 39 2024-03-24T01:01:24.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.621+0000 7f4e63400700 1 -- 172.21.15.38:0/1612190601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e5c1038b0 msgr2=0x7f4e5c107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:24.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.621+0000 7f4e63400700 1 --2- 172.21.15.38:0/1612190601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e5c1038b0 0x7f4e5c107900 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009910 tx=0x7f4e48009c20 comp rx=0 tx=0).stop 2024-03-24T01:01:24.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.622+0000 7f4e63400700 1 -- 172.21.15.38:0/1612190601 shutdown_connections 2024-03-24T01:01:24.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.622+0000 7f4e63400700 1 --2- 172.21.15.38:0/1612190601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e5c1038b0 0x7f4e5c107900 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.622+0000 7f4e63400700 1 --2- 172.21.15.38:0/1612190601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e5c102f00 0x7f4e5c1032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.622+0000 7f4e63400700 1 -- 172.21.15.38:0/1612190601 >> 172.21.15.38:0/1612190601 conn(0x7f4e5c0fe770 msgr2=0x7f4e5c100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:24.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.622+0000 7f4e63400700 1 -- 172.21.15.38:0/1612190601 shutdown_connections 2024-03-24T01:01:24.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.622+0000 7f4e63400700 1 -- 172.21.15.38:0/1612190601 wait complete. 2024-03-24T01:01:24.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.623+0000 7f4e63400700 1 Processor -- start 2024-03-24T01:01:24.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.623+0000 7f4e63400700 1 -- start start 2024-03-24T01:01:24.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.624+0000 7f4e63400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e5c102f00 0x7f4e5c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:24.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.624+0000 7f4e63400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e5c1038b0 0x7f4e5c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:24.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.624+0000 7f4e63400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c19f440 con 0x7f4e5c1038b0 2024-03-24T01:01:24.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.624+0000 7f4e63400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c198a20 con 0x7f4e5c102f00 2024-03-24T01:01:24.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.624+0000 7f4e5be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e5c1038b0 0x7f4e5c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:24.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.624+0000 7f4e5be00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e5c1038b0 0x7f4e5c19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35044/0 (socket says 172.21.15.38:35044) 2024-03-24T01:01:24.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.624+0000 7f4e5be00700 1 -- 172.21.15.38:0/3619669747 learned_addr learned my addr 172.21.15.38:0/3619669747 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:01:24.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.624+0000 7f4e60a00700 1 --2- 172.21.15.38:0/3619669747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e5c102f00 0x7f4e5c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:24.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.625+0000 7f4e5be00700 1 -- 172.21.15.38:0/3619669747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e5c102f00 msgr2=0x7f4e5c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:24.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.625+0000 7f4e5be00700 1 --2- 172.21.15.38:0/3619669747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e5c102f00 0x7f4e5c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.625+0000 7f4e5be00700 1 -- 172.21.15.38:0/3619669747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480095f0 con 0x7f4e5c1038b0 2024-03-24T01:01:24.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.625+0000 7f4e5be00700 1 --2- 172.21.15.38:0/3619669747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e5c1038b0 0x7f4e5c19ee20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e48005b40 tx=0x7f4e4800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:24.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.625+0000 7f4e59600700 1 -- 172.21.15.38:0/3619669747 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e4801d070 con 0x7f4e5c1038b0 2024-03-24T01:01:24.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.625+0000 7f4e59600700 1 -- 172.21.15.38:0/3619669747 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e48005700 con 0x7f4e5c1038b0 2024-03-24T01:01:24.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.625+0000 7f4e59600700 1 -- 172.21.15.38:0/3619669747 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e4800fd60 con 0x7f4e5c1038b0 2024-03-24T01:01:24.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.625+0000 7f4e63400700 1 -- 172.21.15.38:0/3619669747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c198ca0 con 0x7f4e5c1038b0 2024-03-24T01:01:24.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.626+0000 7f4e63400700 1 -- 172.21.15.38:0/3619669747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c199190 con 0x7f4e5c1038b0 2024-03-24T01:01:24.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.627+0000 7f4e63400700 1 -- 172.21.15.38:0/3619669747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5c10b300 con 0x7f4e5c1038b0 2024-03-24T01:01:24.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.629+0000 7f4e59600700 1 -- 172.21.15.38:0/3619669747 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f4e4800f560 con 0x7f4e5c1038b0 2024-03-24T01:01:24.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.630+0000 7f4e59600700 1 --2- 172.21.15.38:0/3619669747 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4e4406dfb0 0x7f4e44070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:24.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.630+0000 7f4e59600700 1 -- 172.21.15.38:0/3619669747 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4e4808f180 con 0x7f4e5c1038b0 2024-03-24T01:01:24.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.630+0000 7f4e60a00700 1 --2- 172.21.15.38:0/3619669747 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4e4406dfb0 0x7f4e44070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:24.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.631+0000 7f4e60a00700 1 --2- 172.21.15.38:0/3619669747 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4e4406dfb0 0x7f4e44070470 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4e50005950 tx=0x7f4e5000b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:24.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.635+0000 7f4e59600700 1 -- 172.21.15.38:0/3619669747 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4e4805c0b0 con 0x7f4e5c1038b0 2024-03-24T01:01:24.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:24 smithi088 ceph-mon[87351]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:01:24.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:24 smithi088 ceph-mon[87351]: pgmap v263: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 170 B/s wr, 15 op/s 2024-03-24T01:01:24.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:24 smithi088 ceph-mon[87351]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:01:24.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:24 smithi088 ceph-mon[87351]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:01:24.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:24 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/76856765' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:24.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.982+0000 7f4e63400700 1 -- 172.21.15.38:0/3619669747 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e5c199ee0 con 0x7f4e4406dfb0 2024-03-24T01:01:24.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.984+0000 7f4e59600700 1 -- 172.21.15.38:0/3619669747 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f4e5c199ee0 con 0x7f4e4406dfb0 2024-03-24T01:01:24.985 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T01:01:24.985 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc", 2024-03-24T01:01:24.985 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-03-24T01:01:24.985 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T01:01:24.985 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-03-24T01:01:24.985 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-03-24T01:01:24.986 INFO:teuthology.orchestra.run.smithi038.stdout: "osd", 2024-03-24T01:01:24.986 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-03-24T01:01:24.986 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-03-24T01:01:24.986 INFO:teuthology.orchestra.run.smithi038.stdout: "mds" 2024-03-24T01:01:24.986 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-03-24T01:01:24.986 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "16/23 daemons upgraded", 2024-03-24T01:01:24.986 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-03-24T01:01:24.986 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T01:01:24.986 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T01:01:24.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.986+0000 7f4e63400700 1 -- 172.21.15.38:0/3619669747 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4e4406dfb0 msgr2=0x7f4e44070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:24.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.986+0000 7f4e63400700 1 --2- 172.21.15.38:0/3619669747 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4e4406dfb0 0x7f4e44070470 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4e50005950 tx=0x7f4e5000b410 comp rx=0 tx=0).stop 2024-03-24T01:01:24.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.986+0000 7f4e63400700 1 -- 172.21.15.38:0/3619669747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e5c1038b0 msgr2=0x7f4e5c19ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:24.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.986+0000 7f4e63400700 1 --2- 172.21.15.38:0/3619669747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e5c1038b0 0x7f4e5c19ee20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e48005b40 tx=0x7f4e4800bd70 comp rx=0 tx=0).stop 2024-03-24T01:01:24.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.987+0000 7f4e63400700 1 -- 172.21.15.38:0/3619669747 shutdown_connections 2024-03-24T01:01:24.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.987+0000 7f4e63400700 1 --2- 172.21.15.38:0/3619669747 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f4e4406dfb0 0x7f4e44070470 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.987+0000 7f4e63400700 1 --2- 172.21.15.38:0/3619669747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e5c102f00 0x7f4e5c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.987+0000 7f4e63400700 1 --2- 172.21.15.38:0/3619669747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e5c1038b0 0x7f4e5c19ee20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:24.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.987+0000 7f4e63400700 1 -- 172.21.15.38:0/3619669747 >> 172.21.15.38:0/3619669747 conn(0x7f4e5c0fe770 msgr2=0x7f4e5c0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:24.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.987+0000 7f4e63400700 1 -- 172.21.15.38:0/3619669747 shutdown_connections 2024-03-24T01:01:24.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:24.987+0000 7f4e63400700 1 -- 172.21.15.38:0/3619669747 wait complete. 2024-03-24T01:01:25.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.152+0000 7f39aa800700 1 -- 172.21.15.38:0/253624272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39a4106a70 msgr2=0x7f39a4106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:25.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.152+0000 7f39aa800700 1 --2- 172.21.15.38:0/253624272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39a4106a70 0x7f39a4106e50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3994009910 tx=0x7f3994009c20 comp rx=0 tx=0).stop 2024-03-24T01:01:25.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.152+0000 7f39aa800700 1 -- 172.21.15.38:0/253624272 shutdown_connections 2024-03-24T01:01:25.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.152+0000 7f39aa800700 1 --2- 172.21.15.38:0/253624272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a4100a70 0x7f39a4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:25.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.152+0000 7f39aa800700 1 --2- 172.21.15.38:0/253624272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39a4106a70 0x7f39a4106e50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:25.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.152+0000 7f39aa800700 1 -- 172.21.15.38:0/253624272 >> 172.21.15.38:0/253624272 conn(0x7f39a40fc580 msgr2=0x7f39a40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:25.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.152+0000 7f39aa800700 1 -- 172.21.15.38:0/253624272 shutdown_connections 2024-03-24T01:01:25.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.152+0000 7f39aa800700 1 -- 172.21.15.38:0/253624272 wait complete. 2024-03-24T01:01:25.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.154+0000 7f39aa800700 1 Processor -- start 2024-03-24T01:01:25.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.154+0000 7f39aa800700 1 -- start start 2024-03-24T01:01:25.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.155+0000 7f39aa800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a4100a70 0x7f39a419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:25.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.155+0000 7f39aa800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39a4106a70 0x7f39a419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:25.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.155+0000 7f39aa800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a419f410 con 0x7f39a4106a70 2024-03-24T01:01:25.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.155+0000 7f39aa800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a41989f0 con 0x7f39a4100a70 2024-03-24T01:01:25.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.155+0000 7f39a2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39a4106a70 0x7f39a419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:25.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.155+0000 7f39a2a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39a4106a70 0x7f39a419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35054/0 (socket says 172.21.15.38:35054) 2024-03-24T01:01:25.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.155+0000 7f39a2a00700 1 -- 172.21.15.38:0/1435760554 learned_addr learned my addr 172.21.15.38:0/1435760554 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:01:25.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.155+0000 7f39a3400700 1 --2- 172.21.15.38:0/1435760554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a4100a70 0x7f39a419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:25.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.156+0000 7f39a2a00700 1 -- 172.21.15.38:0/1435760554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a4100a70 msgr2=0x7f39a419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:25.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.156+0000 7f39a2a00700 1 --2- 172.21.15.38:0/1435760554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a4100a70 0x7f39a419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:25.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.156+0000 7f39a2a00700 1 -- 172.21.15.38:0/1435760554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39940095f0 con 0x7f39a4106a70 2024-03-24T01:01:25.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.156+0000 7f39a2a00700 1 --2- 172.21.15.38:0/1435760554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39a4106a70 0x7f39a419edf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f399000e920 tx=0x7f399000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:25.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.156+0000 7f399be00700 1 -- 172.21.15.38:0/1435760554 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f399000c9d0 con 0x7f39a4106a70 2024-03-24T01:01:25.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.156+0000 7f399be00700 1 -- 172.21.15.38:0/1435760554 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3990010430 con 0x7f39a4106a70 2024-03-24T01:01:25.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.156+0000 7f399be00700 1 -- 172.21.15.38:0/1435760554 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3990018630 con 0x7f39a4106a70 2024-03-24T01:01:25.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.157+0000 7f39aa800700 1 -- 172.21.15.38:0/1435760554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a4198cd0 con 0x7f39a4106a70 2024-03-24T01:01:25.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.157+0000 7f39aa800700 1 -- 172.21.15.38:0/1435760554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a41991f0 con 0x7f39a4106a70 2024-03-24T01:01:25.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.158+0000 7f39aa800700 1 -- 172.21.15.38:0/1435760554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a41021b0 con 0x7f39a4106a70 2024-03-24T01:01:25.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.160+0000 7f399be00700 1 -- 172.21.15.38:0/1435760554 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f39900105a0 con 0x7f39a4106a70 2024-03-24T01:01:25.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.160+0000 7f399be00700 1 --2- 172.21.15.38:0/1435760554 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f398806dfa0 0x7f3988070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:25.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.161+0000 7f399be00700 1 -- 172.21.15.38:0/1435760554 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3990014070 con 0x7f39a4106a70 2024-03-24T01:01:25.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.161+0000 7f39a3400700 1 --2- 172.21.15.38:0/1435760554 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f398806dfa0 0x7f3988070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:25.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.162+0000 7f39a3400700 1 --2- 172.21.15.38:0/1435760554 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f398806dfa0 0x7f3988070460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f39940059d0 tx=0x7f3994005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:25.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.166+0000 7f399be00700 1 -- 172.21.15.38:0/1435760554 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f399005b180 con 0x7f39a4106a70 2024-03-24T01:01:25.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:25 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1688371189' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T01:01:25.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.476+0000 7f39aa800700 1 -- 172.21.15.38:0/1435760554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f39a4199460 con 0x7f39a4106a70 2024-03-24T01:01:25.478 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:25 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1688371189' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T01:01:25.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.477+0000 7f399be00700 1 -- 172.21.15.38:0/1435760554 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f399005ad10 con 0x7f39a4106a70 2024-03-24T01:01:25.478 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_OK 2024-03-24T01:01:25.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.480+0000 7f39aa800700 1 -- 172.21.15.38:0/1435760554 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f398806dfa0 msgr2=0x7f3988070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.480+0000 7f39aa800700 1 --2- 172.21.15.38:0/1435760554 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f398806dfa0 0x7f3988070460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f39940059d0 tx=0x7f3994005940 comp rx=0 tx=0).stop 2024-03-24T01:01:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.480+0000 7f39aa800700 1 -- 172.21.15.38:0/1435760554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39a4106a70 msgr2=0x7f39a419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.480+0000 7f39aa800700 1 --2- 172.21.15.38:0/1435760554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39a4106a70 0x7f39a419edf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f399000e920 tx=0x7f399000ec30 comp rx=0 tx=0).stop 2024-03-24T01:01:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.480+0000 7f39aa800700 1 -- 172.21.15.38:0/1435760554 shutdown_connections 2024-03-24T01:01:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.480+0000 7f39aa800700 1 --2- 172.21.15.38:0/1435760554 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f398806dfa0 0x7f3988070460 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.480+0000 7f39aa800700 1 --2- 172.21.15.38:0/1435760554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a4100a70 0x7f39a419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.480+0000 7f39aa800700 1 --2- 172.21.15.38:0/1435760554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39a4106a70 0x7f39a419edf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.480+0000 7f39aa800700 1 -- 172.21.15.38:0/1435760554 >> 172.21.15.38:0/1435760554 conn(0x7f39a40fc580 msgr2=0x7f39a4073f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.481+0000 7f39aa800700 1 -- 172.21.15.38:0/1435760554 shutdown_connections 2024-03-24T01:01:25.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:25.481+0000 7f39aa800700 1 -- 172.21.15.38:0/1435760554 wait complete. 2024-03-24T01:01:26.646 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:26 smithi088 ceph-mon[87351]: pgmap v264: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 13 op/s 2024-03-24T01:01:26.647 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:26 smithi088 ceph-mon[87351]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:01:26.647 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:26 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1435760554' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T01:01:26.732 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:26 smithi038 ceph-mon[112108]: pgmap v264: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 13 op/s 2024-03-24T01:01:26.732 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:26 smithi038 ceph-mon[112108]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:01:26.733 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:26 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1435760554' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T01:01:28.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:28.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:01:28.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:27 smithi038 ceph-mon[112108]: pgmap v265: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-03-24T01:01:28.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:28.110 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:28.111 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:01:28.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:28.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:01:28.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:27 smithi088 ceph-mon[87351]: pgmap v265: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-03-24T01:01:28.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:28.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:28.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:01:29.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:29 smithi038 ceph-mon[112108]: pgmap v266: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.4 KiB/s wr, 17 op/s 2024-03-24T01:01:30.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:29 smithi088 ceph-mon[87351]: pgmap v266: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.4 KiB/s wr, 17 op/s 2024-03-24T01:01:31.677 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:31 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:31.678 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:31 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:31.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:31 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:31.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:31 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:32.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:32 smithi038 ceph-mon[112108]: pgmap v267: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.4 KiB/s wr, 17 op/s 2024-03-24T01:01:32.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:32 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:32.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:32 smithi088 ceph-mon[87351]: pgmap v267: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.4 KiB/s wr, 17 op/s 2024-03-24T01:01:32.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:32 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:33.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:33 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:33.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:33 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:34.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:34 smithi038 ceph-mon[112108]: pgmap v268: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-03-24T01:01:34.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:34 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:34.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:34 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:34.824 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:34 smithi088 ceph-mon[87351]: pgmap v268: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-03-24T01:01:34.824 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:34 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:34.824 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:34 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:36.527 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:36 smithi088 ceph-mon[87351]: pgmap v269: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 6.3 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-24T01:01:36.527 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:36 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:36.527 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:36 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:36.724 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:36 smithi038 ceph-mon[112108]: pgmap v269: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 6.3 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-24T01:01:36.724 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:36 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:36.724 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:36 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:38.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:38 smithi038 ceph-mon[112108]: pgmap v270: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 5.9 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-24T01:01:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:38 smithi088 ceph-mon[87351]: pgmap v270: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 5.9 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-24T01:01:40.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:40 smithi088 ceph-mon[87351]: pgmap v271: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 3 op/s 2024-03-24T01:01:40.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:40 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:40.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:40 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:40.852 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:40 smithi038 ceph-mon[112108]: pgmap v271: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 3 op/s 2024-03-24T01:01:40.852 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:40 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:40.852 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:40 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: pgmap v272: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 4.9 KiB/s rd, 3 op/s 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:42.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:42.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-24T01:01:42.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: pgmap v272: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 4.9 KiB/s rd, 3 op/s 2024-03-24T01:01:42.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:42.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:42.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:42.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:01:42.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:42.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:01:42.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:01:42.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:42.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:42.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:42.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:42.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:42.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-24T01:01:43.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: Upgrade: Setting filesystem cephfs Joinable 2024-03-24T01:01:43.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-24T01:01:43.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi038.xnpqhp=up:active} 2 up:standby-replay 2024-03-24T01:01:43.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:43.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi038"}]: dispatch 2024-03-24T01:01:43.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi038"}]': finished 2024-03-24T01:01:43.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi088"}]: dispatch 2024-03-24T01:01:43.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi088"}]': finished 2024-03-24T01:01:43.861 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.862 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-24T01:01:43.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-24T01:01:43.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-24T01:01:43.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-24T01:01:43.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-24T01:01:43.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-24T01:01:43.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-24T01:01:43.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-24T01:01:43.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-24T01:01:43.864 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:01:43.865 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:43.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:01:43.866 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:43.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: Upgrade: Setting filesystem cephfs Joinable 2024-03-24T01:01:43.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-24T01:01:43.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: fsmap cephfs:2 {0=cephfs.smithi038.yhkmsb=up:active,1=cephfs.smithi038.xnpqhp=up:active} 2 up:standby-replay 2024-03-24T01:01:43.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:43.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi038"}]: dispatch 2024-03-24T01:01:43.932 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi038"}]': finished 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi088"}]: dispatch 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi088"}]': finished 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.933 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:01:43.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-24T01:01:43.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-24T01:01:43.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-24T01:01:43.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-24T01:01:43.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-24T01:01:43.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-24T01:01:43.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-24T01:01:43.934 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-24T01:01:43.935 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-24T01:01:43.936 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-24T01:01:43.936 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-24T01:01:43.936 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.936 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.936 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.936 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.936 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.936 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T01:01:43.936 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-24T01:01:43.936 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-24T01:01:43.937 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:01:43.937 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:43.937 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:01:43.937 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:43.937 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:01:43.937 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:01:43.937 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:01:43.937 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:44.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:44 smithi038 ceph-mon[112108]: Upgrade: Setting container_image for all ceph-exporter 2024-03-24T01:01:44.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:44 smithi038 ceph-mon[112108]: Upgrade: Setting container_image for all iscsi 2024-03-24T01:01:44.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:44 smithi038 ceph-mon[112108]: Upgrade: Setting container_image for all nfs 2024-03-24T01:01:44.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:44 smithi038 ceph-mon[112108]: Upgrade: Setting container_image for all nvmeof 2024-03-24T01:01:44.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:44 smithi038 ceph-mon[112108]: Upgrade: Finalizing container_image settings 2024-03-24T01:01:44.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:44 smithi038 ceph-mon[112108]: Upgrade: Complete! 2024-03-24T01:01:44.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:44 smithi038 ceph-mon[112108]: pgmap v273: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:01:44.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:44 smithi088 ceph-mon[87351]: Upgrade: Setting container_image for all ceph-exporter 2024-03-24T01:01:44.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:44 smithi088 ceph-mon[87351]: Upgrade: Setting container_image for all iscsi 2024-03-24T01:01:44.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:44 smithi088 ceph-mon[87351]: Upgrade: Setting container_image for all nfs 2024-03-24T01:01:44.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:44 smithi088 ceph-mon[87351]: Upgrade: Setting container_image for all nvmeof 2024-03-24T01:01:44.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:44 smithi088 ceph-mon[87351]: Upgrade: Finalizing container_image settings 2024-03-24T01:01:44.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:44 smithi088 ceph-mon[87351]: Upgrade: Complete! 2024-03-24T01:01:44.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:44 smithi088 ceph-mon[87351]: pgmap v273: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:01:45.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:45 smithi038 ceph-mon[112108]: pgmap v274: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:01:45.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:45 smithi088 ceph-mon[87351]: pgmap v274: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:01:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:47 smithi038 ceph-mon[112108]: pgmap v275: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T01:01:48.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:47 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:48.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:47 smithi088 ceph-mon[87351]: pgmap v275: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-24T01:01:48.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:47 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:01:50.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:49 smithi038 ceph-mon[112108]: pgmap v276: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:01:50.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:49 smithi088 ceph-mon[87351]: pgmap v276: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:01:52.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:51 smithi038 ceph-mon[112108]: pgmap v277: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:01:52.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:51 smithi088 ceph-mon[87351]: pgmap v277: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:01:54.109 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:53 smithi038 ceph-mon[112108]: pgmap v278: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:01:54.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:53 smithi088 ceph-mon[87351]: pgmap v278: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:01:55.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.666+0000 7ff6b5200700 1 -- 172.21.15.38:0/3434977210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6b0107cc0 msgr2=0x7ff6b010bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:55.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.666+0000 7ff6b5200700 1 --2- 172.21.15.38:0/3434977210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6b0107cc0 0x7ff6b010bd10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff698009910 tx=0x7ff698009c20 comp rx=0 tx=0).stop 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.666+0000 7ff6b5200700 1 -- 172.21.15.38:0/3434977210 shutdown_connections 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.666+0000 7ff6b5200700 1 --2- 172.21.15.38:0/3434977210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6b0107cc0 0x7ff6b010bd10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.666+0000 7ff6b5200700 1 --2- 172.21.15.38:0/3434977210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b0107310 0x7ff6b01076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.666+0000 7ff6b5200700 1 -- 172.21.15.38:0/3434977210 >> 172.21.15.38:0/3434977210 conn(0x7ff6b00768e0 msgr2=0x7ff6b0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.666+0000 7ff6b5200700 1 -- 172.21.15.38:0/3434977210 shutdown_connections 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.666+0000 7ff6b5200700 1 -- 172.21.15.38:0/3434977210 wait complete. 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.667+0000 7ff6b5200700 1 Processor -- start 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.667+0000 7ff6b5200700 1 -- start start 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.668+0000 7ff6b5200700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6b0107310 0x7ff6b019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.668+0000 7ff6b5200700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b0107cc0 0x7ff6b019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.668+0000 7ff6b5200700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b019dc20 con 0x7ff6b0107310 2024-03-24T01:01:55.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.668+0000 7ff6b5200700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b01a18a0 con 0x7ff6b0107cc0 2024-03-24T01:01:55.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.668+0000 7ff6ade00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6b0107310 0x7ff6b019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:55.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.668+0000 7ff6ade00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6b0107310 0x7ff6b019d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48780/0 (socket says 172.21.15.38:48780) 2024-03-24T01:01:55.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.668+0000 7ff6ade00700 1 -- 172.21.15.38:0/2540446885 learned_addr learned my addr 172.21.15.38:0/2540446885 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:01:55.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.668+0000 7ff6a5400700 1 --2- 172.21.15.38:0/2540446885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b0107cc0 0x7ff6b019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:55.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.669+0000 7ff6ade00700 1 -- 172.21.15.38:0/2540446885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b0107cc0 msgr2=0x7ff6b019d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:55.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.669+0000 7ff6ade00700 1 --2- 172.21.15.38:0/2540446885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b0107cc0 0x7ff6b019d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:55.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.669+0000 7ff6ade00700 1 -- 172.21.15.38:0/2540446885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6980095f0 con 0x7ff6b0107310 2024-03-24T01:01:55.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.669+0000 7ff6ade00700 1 --2- 172.21.15.38:0/2540446885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6b0107310 0x7ff6b019d000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000d8d0 tx=0x7ff6a000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:55.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.670+0000 7ff6a7400700 1 -- 172.21.15.38:0/2540446885 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6a00098a0 con 0x7ff6b0107310 2024-03-24T01:01:55.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.670+0000 7ff6a7400700 1 -- 172.21.15.38:0/2540446885 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6a0010460 con 0x7ff6b0107310 2024-03-24T01:01:55.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.670+0000 7ff6b5200700 1 -- 172.21.15.38:0/2540446885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b01a1b80 con 0x7ff6b0107310 2024-03-24T01:01:55.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.670+0000 7ff6a7400700 1 -- 172.21.15.38:0/2540446885 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6a000f5f0 con 0x7ff6b0107310 2024-03-24T01:01:55.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.670+0000 7ff6b5200700 1 -- 172.21.15.38:0/2540446885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b01a20a0 con 0x7ff6b0107310 2024-03-24T01:01:55.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.671+0000 7ff6b5200700 1 -- 172.21.15.38:0/2540446885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b010f710 con 0x7ff6b0107310 2024-03-24T01:01:55.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.672+0000 7ff6a7400700 1 -- 172.21.15.38:0/2540446885 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7ff6a0009a00 con 0x7ff6b0107310 2024-03-24T01:01:55.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.673+0000 7ff6a7400700 1 --2- 172.21.15.38:0/2540446885 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff69006dfb0 0x7ff690070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:55.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.673+0000 7ff6a7400700 1 -- 172.21.15.38:0/2540446885 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff6a008dbc0 con 0x7ff6b0107310 2024-03-24T01:01:55.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.674+0000 7ff6a5400700 1 --2- 172.21.15.38:0/2540446885 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff69006dfb0 0x7ff690070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:55.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.674+0000 7ff6a5400700 1 --2- 172.21.15.38:0/2540446885 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff69006dfb0 0x7ff690070470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff698009fd0 tx=0x7ff69801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:55.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.680+0000 7ff6a7400700 1 -- 172.21.15.38:0/2540446885 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff6a0057060 con 0x7ff6b0107310 2024-03-24T01:01:55.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.976+0000 7ff6b5200700 1 -- 172.21.15.38:0/2540446885 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6b019e3d0 con 0x7ff69006dfb0 2024-03-24T01:01:55.976 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:55 smithi038 ceph-mon[112108]: pgmap v279: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:01:55.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.978+0000 7ff6a7400700 1 -- 172.21.15.38:0/2540446885 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff6b019e3d0 con 0x7ff69006dfb0 2024-03-24T01:01:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.981+0000 7ff6b5200700 1 -- 172.21.15.38:0/2540446885 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff69006dfb0 msgr2=0x7ff690070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.981+0000 7ff6b5200700 1 --2- 172.21.15.38:0/2540446885 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff69006dfb0 0x7ff690070470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff698009fd0 tx=0x7ff69801a040 comp rx=0 tx=0).stop 2024-03-24T01:01:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.981+0000 7ff6b5200700 1 -- 172.21.15.38:0/2540446885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6b0107310 msgr2=0x7ff6b019d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.981+0000 7ff6b5200700 1 --2- 172.21.15.38:0/2540446885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6b0107310 0x7ff6b019d000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000d8d0 tx=0x7ff6a000dbe0 comp rx=0 tx=0).stop 2024-03-24T01:01:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.982+0000 7ff6b5200700 1 -- 172.21.15.38:0/2540446885 shutdown_connections 2024-03-24T01:01:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.982+0000 7ff6b5200700 1 --2- 172.21.15.38:0/2540446885 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff69006dfb0 0x7ff690070470 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.982+0000 7ff6b5200700 1 --2- 172.21.15.38:0/2540446885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6b0107310 0x7ff6b019d000 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.982+0000 7ff6b5200700 1 --2- 172.21.15.38:0/2540446885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6b0107cc0 0x7ff6b019d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.982+0000 7ff6b5200700 1 -- 172.21.15.38:0/2540446885 >> 172.21.15.38:0/2540446885 conn(0x7ff6b00768e0 msgr2=0x7ff6b0076cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.982+0000 7ff6b5200700 1 -- 172.21.15.38:0/2540446885 shutdown_connections 2024-03-24T01:01:55.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:55.982+0000 7ff6b5200700 1 -- 172.21.15.38:0/2540446885 wait complete. 2024-03-24T01:01:56.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:55 smithi088 ceph-mon[87351]: pgmap v279: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:01:56.950 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph orch ps' 2024-03-24T01:01:57.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:56 smithi088 ceph-mon[87351]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:01:57.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:56 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:01:57.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:56 smithi038 ceph-mon[112108]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:01:57.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:56 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:01:57.440 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T01:01:58.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:01:57 smithi088 ceph-mon[87351]: pgmap v280: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T01:01:58.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:01:57 smithi038 ceph-mon[112108]: pgmap v280: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T01:01:59.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.682+0000 7f11f0e00700 1 -- 172.21.15.38:0/225767063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec108b60 msgr2=0x7f11ec108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:59.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.682+0000 7f11f0e00700 1 --2- 172.21.15.38:0/225767063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec108b60 0x7f11ec108f40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009910 tx=0x7f11dc009c20 comp rx=0 tx=0).stop 2024-03-24T01:01:59.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.682+0000 7f11f0e00700 1 -- 172.21.15.38:0/225767063 shutdown_connections 2024-03-24T01:01:59.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.682+0000 7f11f0e00700 1 --2- 172.21.15.38:0/225767063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11ec102ad0 0x7f11ec102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:59.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.682+0000 7f11f0e00700 1 --2- 172.21.15.38:0/225767063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec108b60 0x7f11ec108f40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:59.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.682+0000 7f11f0e00700 1 -- 172.21.15.38:0/225767063 >> 172.21.15.38:0/225767063 conn(0x7f11ec0fe5a0 msgr2=0x7f11ec1009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:59.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.682+0000 7f11f0e00700 1 -- 172.21.15.38:0/225767063 shutdown_connections 2024-03-24T01:01:59.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.682+0000 7f11f0e00700 1 -- 172.21.15.38:0/225767063 wait complete. 2024-03-24T01:01:59.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.684+0000 7f11f0e00700 1 Processor -- start 2024-03-24T01:01:59.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.684+0000 7f11f0e00700 1 -- start start 2024-03-24T01:01:59.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.684+0000 7f11f0e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec102ad0 0x7f11ec19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:59.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.684+0000 7f11f0e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11ec108b60 0x7f11ec19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:59.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.684+0000 7f11f0e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec19f2c0 con 0x7f11ec102ad0 2024-03-24T01:01:59.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.684+0000 7f11f0e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec198830 con 0x7f11ec108b60 2024-03-24T01:01:59.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.685+0000 7f11eb400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec102ad0 0x7f11ec19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:59.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.685+0000 7f11eb400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec102ad0 0x7f11ec19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48798/0 (socket says 172.21.15.38:48798) 2024-03-24T01:01:59.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.685+0000 7f11eb400700 1 -- 172.21.15.38:0/3915051986 learned_addr learned my addr 172.21.15.38:0/3915051986 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:01:59.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.685+0000 7f11e3e00700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11ec108b60 0x7f11ec19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:59.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.685+0000 7f11eb400700 1 -- 172.21.15.38:0/3915051986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11ec108b60 msgr2=0x7f11ec19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:59.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.685+0000 7f11eb400700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11ec108b60 0x7f11ec19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:59.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.685+0000 7f11eb400700 1 -- 172.21.15.38:0/3915051986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc0095f0 con 0x7f11ec102ad0 2024-03-24T01:01:59.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.685+0000 7f11e3e00700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11ec108b60 0x7f11ec19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T01:01:59.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.686+0000 7f11eb400700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec102ad0 0x7f11ec19e6f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0092e0 tx=0x7f11dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:59.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.686+0000 7f11e8c00700 1 -- 172.21.15.38:0/3915051986 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11dc01d070 con 0x7f11ec102ad0 2024-03-24T01:01:59.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.686+0000 7f11e8c00700 1 -- 172.21.15.38:0/3915051986 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11dc0056e0 con 0x7f11ec102ad0 2024-03-24T01:01:59.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.686+0000 7f11f0e00700 1 -- 172.21.15.38:0/3915051986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ec198ab0 con 0x7f11ec102ad0 2024-03-24T01:01:59.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.686+0000 7f11e8c00700 1 -- 172.21.15.38:0/3915051986 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11dc00fe80 con 0x7f11ec102ad0 2024-03-24T01:01:59.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.687+0000 7f11f0e00700 1 -- 172.21.15.38:0/3915051986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ec198fa0 con 0x7f11ec102ad0 2024-03-24T01:01:59.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.688+0000 7f11f0e00700 1 -- 172.21.15.38:0/3915051986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ec10b0d0 con 0x7f11ec102ad0 2024-03-24T01:01:59.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.690+0000 7f11e8c00700 1 -- 172.21.15.38:0/3915051986 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f11dc00f6f0 con 0x7f11ec102ad0 2024-03-24T01:01:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.691+0000 7f11e8c00700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f11d006dfb0 0x7f11d0070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:01:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.691+0000 7f11e8c00700 1 -- 172.21.15.38:0/3915051986 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f11dc08f220 con 0x7f11ec102ad0 2024-03-24T01:01:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.691+0000 7f11e3e00700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f11d006dfb0 0x7f11d0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:01:59.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.692+0000 7f11e3e00700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f11d006dfb0 0x7f11d0070470 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005950 tx=0x7f11d40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:01:59.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.699+0000 7f11e8c00700 1 -- 172.21.15.38:0/3915051986 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f11dc05c150 con 0x7f11ec102ad0 2024-03-24T01:01:59.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.950+0000 7f11f0e00700 1 -- 172.21.15.38:0/3915051986 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f11ec199cf0 con 0x7f11d006dfb0 2024-03-24T01:01:59.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.977+0000 7f11e8c00700 1 -- 172.21.15.38:0/3915051986 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f11ec199cf0 con 0x7f11d006dfb0 2024-03-24T01:01:59.978 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T01:01:59.978 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (10m) 26s ago 24m 14.5M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T01:01:59.978 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (45s) 26s ago 24m 7818k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 eac5e34d263f 2024-03-24T01:01:59.978 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (33s) 29s ago 23m 7096k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 10a178ef9957 2024-03-24T01:01:59.978 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (6m) 26s ago 24m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T01:01:59.978 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (6m) 29s ago 23m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T01:01:59.978 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (9m) 26s ago 24m 74.5M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T01:01:59.978 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (117s) 26s ago 16m 15.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b78588cbee24 2024-03-24T01:01:59.978 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (2m) 26s ago 17m 82.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e597065368f0 2024-03-24T01:01:59.978 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (92s) 29s ago 16m 82.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 49be5a5d2c69 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (68s) 29s ago 16m 13.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b40c5fbe832e 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (13m) 26s ago 26m 511M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (11m) 29s ago 22m 452M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (8m) 26s ago 26m 57.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (7m) 29s ago 22m 49.4M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (11m) 26s ago 24m 8066k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (11m) 29s ago 23m 7700k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (5m) 26s ago 21m 106M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (5m) 26s ago 21m 119M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cb72ac446ba7 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (4m) 26s ago 20m 100M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c73f3e8276b6 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (4m) 29s ago 20m 124M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b2afaff761d8 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (3m) 29s ago 19m 108M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9a6f202043cd 2024-03-24T01:01:59.979 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (2m) 29s ago 18m 109M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 866388fc07f6 2024-03-24T01:01:59.980 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (10m) 26s ago 23m 75.2M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T01:01:59.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.981+0000 7f11f0e00700 1 -- 172.21.15.38:0/3915051986 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f11d006dfb0 msgr2=0x7f11d0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:59.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.981+0000 7f11f0e00700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f11d006dfb0 0x7f11d0070470 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005950 tx=0x7f11d40058e0 comp rx=0 tx=0).stop 2024-03-24T01:01:59.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.981+0000 7f11f0e00700 1 -- 172.21.15.38:0/3915051986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec102ad0 msgr2=0x7f11ec19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:01:59.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.981+0000 7f11f0e00700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec102ad0 0x7f11ec19e6f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0092e0 tx=0x7f11dc00be20 comp rx=0 tx=0).stop 2024-03-24T01:01:59.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.982+0000 7f11f0e00700 1 -- 172.21.15.38:0/3915051986 shutdown_connections 2024-03-24T01:01:59.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.982+0000 7f11f0e00700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f11d006dfb0 0x7f11d0070470 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:59.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.982+0000 7f11f0e00700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11ec102ad0 0x7f11ec19e6f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:59.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.982+0000 7f11f0e00700 1 --2- 172.21.15.38:0/3915051986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11ec108b60 0x7f11ec19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:01:59.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.982+0000 7f11f0e00700 1 -- 172.21.15.38:0/3915051986 >> 172.21.15.38:0/3915051986 conn(0x7f11ec0fe5a0 msgr2=0x7f11ec0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:01:59.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.982+0000 7f11f0e00700 1 -- 172.21.15.38:0/3915051986 shutdown_connections 2024-03-24T01:01:59.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:01:59.982+0000 7f11f0e00700 1 -- 172.21.15.38:0/3915051986 wait complete. 2024-03-24T01:02:00.342 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:00 smithi038 ceph-mon[112108]: pgmap v281: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-24T01:02:00.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:00 smithi088 ceph-mon[87351]: pgmap v281: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-24T01:02:00.796 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph orch upgrade status' 2024-03-24T01:02:01.237 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T01:02:01.399 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:01 smithi038 ceph-mon[112108]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:02:01.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:01 smithi088 ceph-mon[87351]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:02:02.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:02 smithi088 ceph-mon[87351]: pgmap v282: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:02:02.435 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:02 smithi038 ceph-mon[112108]: pgmap v282: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:02:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.488+0000 7f89d6c00700 1 -- 172.21.15.38:0/892560170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d0102ac0 msgr2=0x7f89d0102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.488+0000 7f89d6c00700 1 --2- 172.21.15.38:0/892560170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d0102ac0 0x7f89d0102f40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009910 tx=0x7f89bc009c20 comp rx=0 tx=0).stop 2024-03-24T01:02:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.488+0000 7f89d6c00700 1 -- 172.21.15.38:0/892560170 shutdown_connections 2024-03-24T01:02:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.488+0000 7f89d6c00700 1 --2- 172.21.15.38:0/892560170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d0102ac0 0x7f89d0102f40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.488+0000 7f89d6c00700 1 --2- 172.21.15.38:0/892560170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d0108ac0 0x7f89d0108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:03.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.488+0000 7f89d6c00700 1 -- 172.21.15.38:0/892560170 >> 172.21.15.38:0/892560170 conn(0x7f89d00fe570 msgr2=0x7f89d0100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:02:03.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.488+0000 7f89d6c00700 1 -- 172.21.15.38:0/892560170 shutdown_connections 2024-03-24T01:02:03.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.488+0000 7f89d6c00700 1 -- 172.21.15.38:0/892560170 wait complete. 2024-03-24T01:02:03.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.490+0000 7f89d6c00700 1 Processor -- start 2024-03-24T01:02:03.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.490+0000 7f89d6c00700 1 -- start start 2024-03-24T01:02:03.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.491+0000 7f89d6c00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d0102ac0 0x7f89d019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:02:03.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.491+0000 7f89d6c00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d0108ac0 0x7f89d019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:02:03.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.491+0000 7f89d6c00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d019f2b0 con 0x7f89d0102ac0 2024-03-24T01:02:03.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.491+0000 7f89d6c00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d0198820 con 0x7f89d0108ac0 2024-03-24T01:02:03.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.491+0000 7f89d5800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d0102ac0 0x7f89d019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:02:03.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.491+0000 7f89d5800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d0102ac0 0x7f89d019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48144/0 (socket says 172.21.15.38:48144) 2024-03-24T01:02:03.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.491+0000 7f89d5800700 1 -- 172.21.15.38:0/639245112 learned_addr learned my addr 172.21.15.38:0/639245112 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:02:03.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.491+0000 7f89d4e00700 1 --2- 172.21.15.38:0/639245112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d0108ac0 0x7f89d019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:02:03.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.492+0000 7f89d5800700 1 -- 172.21.15.38:0/639245112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d0108ac0 msgr2=0x7f89d019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:03.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.492+0000 7f89d5800700 1 --2- 172.21.15.38:0/639245112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d0108ac0 0x7f89d019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:03.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.492+0000 7f89d5800700 1 -- 172.21.15.38:0/639245112 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc0095f0 con 0x7f89d0102ac0 2024-03-24T01:02:03.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.492+0000 7f89d5800700 1 --2- 172.21.15.38:0/639245112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d0102ac0 0x7f89d019e6e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f89c400d8d0 tx=0x7f89c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:02:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.492+0000 7f89ce000700 1 -- 172.21.15.38:0/639245112 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89c4009940 con 0x7f89d0102ac0 2024-03-24T01:02:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.492+0000 7f89ce000700 1 -- 172.21.15.38:0/639245112 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89c4010460 con 0x7f89d0102ac0 2024-03-24T01:02:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.492+0000 7f89ce000700 1 -- 172.21.15.38:0/639245112 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89c400f5f0 con 0x7f89d0102ac0 2024-03-24T01:02:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.492+0000 7f89d6c00700 1 -- 172.21.15.38:0/639245112 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d0198b00 con 0x7f89d0102ac0 2024-03-24T01:02:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.493+0000 7f89d6c00700 1 -- 172.21.15.38:0/639245112 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d0199050 con 0x7f89d0102ac0 2024-03-24T01:02:03.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.494+0000 7f89d6c00700 1 -- 172.21.15.38:0/639245112 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d010b0e0 con 0x7f89d0102ac0 2024-03-24T01:02:03.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.495+0000 7f89ce000700 1 -- 172.21.15.38:0/639245112 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f89c40105d0 con 0x7f89d0102ac0 2024-03-24T01:02:03.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.496+0000 7f89ce000700 1 --2- 172.21.15.38:0/639245112 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f89b806dfb0 0x7f89b8070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:02:03.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.496+0000 7f89ce000700 1 -- 172.21.15.38:0/639245112 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f89c408dce0 con 0x7f89d0102ac0 2024-03-24T01:02:03.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.497+0000 7f89d4e00700 1 --2- 172.21.15.38:0/639245112 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f89b806dfb0 0x7f89b8070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:02:03.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.497+0000 7f89d4e00700 1 --2- 172.21.15.38:0/639245112 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f89b806dfb0 0x7f89b8070470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f89bc005b40 tx=0x7f89bc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:02:03.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.504+0000 7f89ce000700 1 -- 172.21.15.38:0/639245112 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f89c405acb0 con 0x7f89d0102ac0 2024-03-24T01:02:03.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.941+0000 7f89d6c00700 1 -- 172.21.15.38:0/639245112 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f89d0199d50 con 0x7f89b806dfb0 2024-03-24T01:02:03.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.943+0000 7f89ce000700 1 -- 172.21.15.38:0/639245112 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f89d0199d50 con 0x7f89b806dfb0 2024-03-24T01:02:03.943 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T01:02:03.943 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": null, 2024-03-24T01:02:03.943 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": false, 2024-03-24T01:02:03.943 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "", 2024-03-24T01:02:03.943 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [], 2024-03-24T01:02:03.943 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": null, 2024-03-24T01:02:03.943 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "", 2024-03-24T01:02:03.943 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-03-24T01:02:03.943 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T01:02:03.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.946+0000 7f89d6c00700 1 -- 172.21.15.38:0/639245112 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f89b806dfb0 msgr2=0x7f89b8070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:03.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.946+0000 7f89d6c00700 1 --2- 172.21.15.38:0/639245112 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f89b806dfb0 0x7f89b8070470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f89bc005b40 tx=0x7f89bc01a040 comp rx=0 tx=0).stop 2024-03-24T01:02:03.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.946+0000 7f89d6c00700 1 -- 172.21.15.38:0/639245112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d0102ac0 msgr2=0x7f89d019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:03.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.946+0000 7f89d6c00700 1 --2- 172.21.15.38:0/639245112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d0102ac0 0x7f89d019e6e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f89c400d8d0 tx=0x7f89c400dc90 comp rx=0 tx=0).stop 2024-03-24T01:02:03.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.946+0000 7f89d6c00700 1 -- 172.21.15.38:0/639245112 shutdown_connections 2024-03-24T01:02:03.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.946+0000 7f89d6c00700 1 --2- 172.21.15.38:0/639245112 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f89b806dfb0 0x7f89b8070470 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:03.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.946+0000 7f89d6c00700 1 --2- 172.21.15.38:0/639245112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89d0102ac0 0x7f89d019e6e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:03.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.946+0000 7f89d6c00700 1 --2- 172.21.15.38:0/639245112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d0108ac0 0x7f89d019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:03.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.946+0000 7f89d6c00700 1 -- 172.21.15.38:0/639245112 >> 172.21.15.38:0/639245112 conn(0x7f89d00fe570 msgr2=0x7f89d00ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:02:03.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.947+0000 7f89d6c00700 1 -- 172.21.15.38:0/639245112 shutdown_connections 2024-03-24T01:02:03.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:03.947+0000 7f89d6c00700 1 -- 172.21.15.38:0/639245112 wait complete. 2024-03-24T01:02:04.262 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:04 smithi038 ceph-mon[112108]: pgmap v283: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:02:04.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:04 smithi088 ceph-mon[87351]: pgmap v283: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:02:04.735 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph health detail' 2024-03-24T01:02:05.209 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T01:02:05.409 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:05 smithi038 ceph-mon[112108]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:02:05.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:05 smithi088 ceph-mon[87351]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:02:06.410 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:06 smithi038 ceph-mon[112108]: pgmap v284: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:02:06.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:06 smithi088 ceph-mon[87351]: pgmap v284: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:02:07.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.745+0000 7f5a5a000700 1 -- 172.21.15.38:0/1674966276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a54108ac0 msgr2=0x7f5a54108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:07.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.745+0000 7f5a5a000700 1 --2- 172.21.15.38:0/1674966276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a54108ac0 0x7f5a54108ea0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009910 tx=0x7f5a48009c20 comp rx=0 tx=0).stop 2024-03-24T01:02:07.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.745+0000 7f5a5a000700 1 -- 172.21.15.38:0/1674966276 shutdown_connections 2024-03-24T01:02:07.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.745+0000 7f5a5a000700 1 --2- 172.21.15.38:0/1674966276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a54102ac0 0x7f5a54102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:07.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.745+0000 7f5a5a000700 1 --2- 172.21.15.38:0/1674966276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a54108ac0 0x7f5a54108ea0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:07.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.745+0000 7f5a5a000700 1 -- 172.21.15.38:0/1674966276 >> 172.21.15.38:0/1674966276 conn(0x7f5a540fe570 msgr2=0x7f5a54100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:02:07.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.745+0000 7f5a5a000700 1 -- 172.21.15.38:0/1674966276 shutdown_connections 2024-03-24T01:02:07.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.745+0000 7f5a5a000700 1 -- 172.21.15.38:0/1674966276 wait complete. 2024-03-24T01:02:07.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.747+0000 7f5a5a000700 1 Processor -- start 2024-03-24T01:02:07.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.747+0000 7f5a5a000700 1 -- start start 2024-03-24T01:02:07.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.748+0000 7f5a5a000700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a54102ac0 0x7f5a5419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:02:07.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.748+0000 7f5a5a000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a54108ac0 0x7f5a5419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:02:07.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.748+0000 7f5a5a000700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5419f2a0 con 0x7f5a54108ac0 2024-03-24T01:02:07.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.748+0000 7f5a5a000700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a54198860 con 0x7f5a54102ac0 2024-03-24T01:02:07.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.748+0000 7f5a53e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a54108ac0 0x7f5a5419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:02:07.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.748+0000 7f5a53e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a54108ac0 0x7f5a5419ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48172/0 (socket says 172.21.15.38:48172) 2024-03-24T01:02:07.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.748+0000 7f5a53e00700 1 -- 172.21.15.38:0/711110019 learned_addr learned my addr 172.21.15.38:0/711110019 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:02:07.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.748+0000 7f5a58c00700 1 --2- 172.21.15.38:0/711110019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a54102ac0 0x7f5a5419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:02:07.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.749+0000 7f5a53e00700 1 -- 172.21.15.38:0/711110019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a54102ac0 msgr2=0x7f5a5419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:07.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.749+0000 7f5a53e00700 1 --2- 172.21.15.38:0/711110019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a54102ac0 0x7f5a5419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:07.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.749+0000 7f5a53e00700 1 -- 172.21.15.38:0/711110019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a480095f0 con 0x7f5a54108ac0 2024-03-24T01:02:07.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.749+0000 7f5a53e00700 1 --2- 172.21.15.38:0/711110019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a54108ac0 0x7f5a5419ec10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000d900 tx=0x7f5a4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:02:07.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.749+0000 7f5a51600700 1 -- 172.21.15.38:0/711110019 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a40004a00 con 0x7f5a54108ac0 2024-03-24T01:02:07.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.750+0000 7f5a51600700 1 -- 172.21.15.38:0/711110019 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a400054e0 con 0x7f5a54108ac0 2024-03-24T01:02:07.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.750+0000 7f5a5a000700 1 -- 172.21.15.38:0/711110019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a54198b40 con 0x7f5a54108ac0 2024-03-24T01:02:07.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.750+0000 7f5a51600700 1 -- 172.21.15.38:0/711110019 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a40009d60 con 0x7f5a54108ac0 2024-03-24T01:02:07.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.750+0000 7f5a5a000700 1 -- 172.21.15.38:0/711110019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a54199090 con 0x7f5a54108ac0 2024-03-24T01:02:07.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.751+0000 7f5a5a000700 1 -- 172.21.15.38:0/711110019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5410b0c0 con 0x7f5a54108ac0 2024-03-24T01:02:07.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.752+0000 7f5a51600700 1 -- 172.21.15.38:0/711110019 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f5a40005020 con 0x7f5a54108ac0 2024-03-24T01:02:07.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.753+0000 7f5a51600700 1 --2- 172.21.15.38:0/711110019 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5a3c06dfb0 0x7f5a3c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:02:07.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.753+0000 7f5a51600700 1 -- 172.21.15.38:0/711110019 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5a4008df90 con 0x7f5a54108ac0 2024-03-24T01:02:07.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.753+0000 7f5a58c00700 1 --2- 172.21.15.38:0/711110019 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5a3c06dfb0 0x7f5a3c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:02:07.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.754+0000 7f5a58c00700 1 --2- 172.21.15.38:0/711110019 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5a3c06dfb0 0x7f5a3c070470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800b5c0 tx=0x7f5a4801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:02:07.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:07.761+0000 7f5a51600700 1 -- 172.21.15.38:0/711110019 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a4005aec0 con 0x7f5a54108ac0 2024-03-24T01:02:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.212+0000 7f5a5a000700 1 -- 172.21.15.38:0/711110019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5a54104200 con 0x7f5a54108ac0 2024-03-24T01:02:08.213 INFO:teuthology.orchestra.run.smithi038.stdout:HEALTH_OK 2024-03-24T01:02:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.213+0000 7f5a51600700 1 -- 172.21.15.38:0/711110019 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5a40017020 con 0x7f5a54108ac0 2024-03-24T01:02:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.215+0000 7f5a5a000700 1 -- 172.21.15.38:0/711110019 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5a3c06dfb0 msgr2=0x7f5a3c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.215+0000 7f5a5a000700 1 --2- 172.21.15.38:0/711110019 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5a3c06dfb0 0x7f5a3c070470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800b5c0 tx=0x7f5a4801a040 comp rx=0 tx=0).stop 2024-03-24T01:02:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.215+0000 7f5a5a000700 1 -- 172.21.15.38:0/711110019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a54108ac0 msgr2=0x7f5a5419ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.215+0000 7f5a5a000700 1 --2- 172.21.15.38:0/711110019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a54108ac0 0x7f5a5419ec10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000d900 tx=0x7f5a4000dcc0 comp rx=0 tx=0).stop 2024-03-24T01:02:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.216+0000 7f5a5a000700 1 -- 172.21.15.38:0/711110019 shutdown_connections 2024-03-24T01:02:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.216+0000 7f5a5a000700 1 --2- 172.21.15.38:0/711110019 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5a3c06dfb0 0x7f5a3c070470 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.216+0000 7f5a5a000700 1 --2- 172.21.15.38:0/711110019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a54102ac0 0x7f5a5419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.216+0000 7f5a5a000700 1 --2- 172.21.15.38:0/711110019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a54108ac0 0x7f5a5419ec10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.216+0000 7f5a5a000700 1 -- 172.21.15.38:0/711110019 >> 172.21.15.38:0/711110019 conn(0x7f5a540fe570 msgr2=0x7f5a540ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:02:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.216+0000 7f5a5a000700 1 -- 172.21.15.38:0/711110019 shutdown_connections 2024-03-24T01:02:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:08.216+0000 7f5a5a000700 1 -- 172.21.15.38:0/711110019 wait complete. 2024-03-24T01:02:08.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:08 smithi088 ceph-mon[87351]: pgmap v285: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T01:02:08.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:08 smithi038 ceph-mon[112108]: pgmap v285: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-24T01:02:09.144 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph versions' 2024-03-24T01:02:09.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:09 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/711110019' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T01:02:09.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:09 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/711110019' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T01:02:09.604 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T01:02:10.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:10 smithi088 ceph-mon[87351]: pgmap v286: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:02:10.482 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:10 smithi038 ceph-mon[112108]: pgmap v286: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:02:11.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.610+0000 7fc08d600700 1 -- 172.21.15.38:0/2373143304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0880ff950 msgr2=0x7fc0880ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:11.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.610+0000 7fc08d600700 1 --2- 172.21.15.38:0/2373143304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0880ff950 0x7fc0880ffdd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc070009910 tx=0x7fc070009c20 comp rx=0 tx=0).stop 2024-03-24T01:02:11.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.610+0000 7fc08d600700 1 -- 172.21.15.38:0/2373143304 shutdown_connections 2024-03-24T01:02:11.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.610+0000 7fc08d600700 1 --2- 172.21.15.38:0/2373143304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0880ff950 0x7fc0880ffdd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:11.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.610+0000 7fc08d600700 1 --2- 172.21.15.38:0/2373143304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc088105950 0x7fc088105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:11.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.611+0000 7fc08d600700 1 -- 172.21.15.38:0/2373143304 >> 172.21.15.38:0/2373143304 conn(0x7fc088076640 msgr2=0x7fc0880fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:02:11.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.611+0000 7fc08d600700 1 -- 172.21.15.38:0/2373143304 shutdown_connections 2024-03-24T01:02:11.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.611+0000 7fc08d600700 1 -- 172.21.15.38:0/2373143304 wait complete. 2024-03-24T01:02:11.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.612+0000 7fc08d600700 1 Processor -- start 2024-03-24T01:02:11.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.612+0000 7fc08d600700 1 -- start start 2024-03-24T01:02:11.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.612+0000 7fc08d600700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0880ff950 0x7fc08819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:02:11.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.612+0000 7fc08d600700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc088105950 0x7fc08819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:02:11.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.612+0000 7fc08d600700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08819f3e0 con 0x7fc0880ff950 2024-03-24T01:02:11.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.612+0000 7fc08d600700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc088198a10 con 0x7fc088105950 2024-03-24T01:02:11.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.613+0000 7fc086a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0880ff950 0x7fc08819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:02:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.613+0000 7fc086a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0880ff950 0x7fc08819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48180/0 (socket says 172.21.15.38:48180) 2024-03-24T01:02:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.613+0000 7fc086a00700 1 -- 172.21.15.38:0/692982324 learned_addr learned my addr 172.21.15.38:0/692982324 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:02:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.613+0000 7fc07fe00700 1 --2- 172.21.15.38:0/692982324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc088105950 0x7fc08819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:02:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.613+0000 7fc086a00700 1 -- 172.21.15.38:0/692982324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc088105950 msgr2=0x7fc08819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.613+0000 7fc086a00700 1 --2- 172.21.15.38:0/692982324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc088105950 0x7fc08819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.613+0000 7fc086a00700 1 -- 172.21.15.38:0/692982324 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0700095f0 con 0x7fc0880ff950 2024-03-24T01:02:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.614+0000 7fc086a00700 1 --2- 172.21.15.38:0/692982324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0880ff950 0x7fc08819e880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d8d0 tx=0x7fc07800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:02:11.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.614+0000 7fc07f400700 1 -- 172.21.15.38:0/692982324 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc078009940 con 0x7fc0880ff950 2024-03-24T01:02:11.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.614+0000 7fc07f400700 1 -- 172.21.15.38:0/692982324 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc078010460 con 0x7fc0880ff950 2024-03-24T01:02:11.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.614+0000 7fc08d600700 1 -- 172.21.15.38:0/692982324 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc088198cf0 con 0x7fc0880ff950 2024-03-24T01:02:11.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.614+0000 7fc07f400700 1 -- 172.21.15.38:0/692982324 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc07800f5f0 con 0x7fc0880ff950 2024-03-24T01:02:11.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.614+0000 7fc08d600700 1 -- 172.21.15.38:0/692982324 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc088199240 con 0x7fc0880ff950 2024-03-24T01:02:11.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.616+0000 7fc08d600700 1 -- 172.21.15.38:0/692982324 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc088101090 con 0x7fc0880ff950 2024-03-24T01:02:11.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.617+0000 7fc07f400700 1 -- 172.21.15.38:0/692982324 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fc0780105d0 con 0x7fc0880ff950 2024-03-24T01:02:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.618+0000 7fc07f400700 1 --2- 172.21.15.38:0/692982324 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc06806dfb0 0x7fc068070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:02:11.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.618+0000 7fc07f400700 1 -- 172.21.15.38:0/692982324 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc07808dd70 con 0x7fc0880ff950 2024-03-24T01:02:11.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.619+0000 7fc07fe00700 1 --2- 172.21.15.38:0/692982324 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc06806dfb0 0x7fc068070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:02:11.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.619+0000 7fc07fe00700 1 --2- 172.21.15.38:0/692982324 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc06806dfb0 0x7fc068070470 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc070005b40 tx=0x7fc07001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:02:11.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:11.627+0000 7fc07f400700 1 -- 172.21.15.38:0/692982324 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc07805aca0 con 0x7fc0880ff950 2024-03-24T01:02:12.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.012+0000 7fc08d600700 1 -- 172.21.15.38:0/692982324 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc0881994b0 con 0x7fc0880ff950 2024-03-24T01:02:12.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.013+0000 7fc07f400700 1 -- 172.21.15.38:0/692982324 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fc07805a830 con 0x7fc0880ff950 2024-03-24T01:02:12.014 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:02:12.015 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T01:02:12.016 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 14 2024-03-24T01:02:12.016 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T01:02:12.016 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T01:02:12.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.016+0000 7fc08d600700 1 -- 172.21.15.38:0/692982324 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc06806dfb0 msgr2=0x7fc068070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.016+0000 7fc08d600700 1 --2- 172.21.15.38:0/692982324 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc06806dfb0 0x7fc068070470 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc070005b40 tx=0x7fc07001a040 comp rx=0 tx=0).stop 2024-03-24T01:02:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.016+0000 7fc08d600700 1 -- 172.21.15.38:0/692982324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0880ff950 msgr2=0x7fc08819e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:02:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.016+0000 7fc08d600700 1 --2- 172.21.15.38:0/692982324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0880ff950 0x7fc08819e880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d8d0 tx=0x7fc07800dc90 comp rx=0 tx=0).stop 2024-03-24T01:02:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.016+0000 7fc08d600700 1 -- 172.21.15.38:0/692982324 shutdown_connections 2024-03-24T01:02:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.016+0000 7fc08d600700 1 --2- 172.21.15.38:0/692982324 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc06806dfb0 0x7fc068070470 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.016+0000 7fc08d600700 1 --2- 172.21.15.38:0/692982324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0880ff950 0x7fc08819e880 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.016+0000 7fc08d600700 1 --2- 172.21.15.38:0/692982324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc088105950 0x7fc08819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:02:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.016+0000 7fc08d600700 1 -- 172.21.15.38:0/692982324 >> 172.21.15.38:0/692982324 conn(0x7fc088076640 msgr2=0x7fc0880fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:02:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.017+0000 7fc08d600700 1 -- 172.21.15.38:0/692982324 shutdown_connections 2024-03-24T01:02:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:02:12.017+0000 7fc08d600700 1 -- 172.21.15.38:0/692982324 wait complete. 2024-03-24T01:02:12.314 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:12 smithi038 ceph-mon[112108]: pgmap v287: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:12.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:02:12.315 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:12 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/692982324' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:02:12.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:12 smithi088 ceph-mon[87351]: pgmap v287: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:12.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:12 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:02:12.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:12 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/692982324' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:02:12.850 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-24T01:02:13.319 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T01:02:14.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:14 smithi038 ceph-mon[112108]: pgmap v288: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:14.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:14 smithi088 ceph-mon[87351]: pgmap v288: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:15.410 INFO:teuthology.orchestra.run.smithi038.stdout:wait for servicemap items w/ changing names to refresh 2024-03-24T01:02:16.137 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'sleep 60' 2024-03-24T01:02:16.359 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:16 smithi038 ceph-mon[112108]: pgmap v289: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:16.585 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T01:02:16.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:16 smithi088 ceph-mon[87351]: pgmap v289: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:18.460 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:18 smithi088 ceph-mon[87351]: pgmap v290: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:02:18.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:18 smithi038 ceph-mon[112108]: pgmap v290: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:02:20.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:20 smithi038 ceph-mon[112108]: pgmap v291: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:02:20.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:20 smithi088 ceph-mon[87351]: pgmap v291: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:02:22.484 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:22 smithi038 ceph-mon[112108]: pgmap v292: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:22.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:22 smithi088 ceph-mon[87351]: pgmap v292: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:24.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:24 smithi038 ceph-mon[112108]: pgmap v293: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:24.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:24 smithi088 ceph-mon[87351]: pgmap v293: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:26.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:26 smithi038 ceph-mon[112108]: pgmap v294: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:26.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:26 smithi088 ceph-mon[87351]: pgmap v294: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:27.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:02:27.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:02:28.611 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:28 smithi038 ceph-mon[112108]: pgmap v295: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:02:28.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:28 smithi088 ceph-mon[87351]: pgmap v295: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:02:30.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:30 smithi038 ceph-mon[112108]: pgmap v296: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:02:30.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:30 smithi088 ceph-mon[87351]: pgmap v296: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:02:32.582 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:32 smithi038 ceph-mon[112108]: pgmap v297: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:32.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:32 smithi088 ceph-mon[87351]: pgmap v297: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:34.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:34 smithi038 ceph-mon[112108]: pgmap v298: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:34.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:34 smithi088 ceph-mon[87351]: pgmap v298: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:36.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:36 smithi038 ceph-mon[112108]: pgmap v299: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:36.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:36 smithi088 ceph-mon[87351]: pgmap v299: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:38.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:38 smithi038 ceph-mon[112108]: pgmap v300: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:02:38.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:38 smithi088 ceph-mon[87351]: pgmap v300: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:02:40.570 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:40 smithi038 ceph-mon[112108]: pgmap v301: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:02:40.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:40 smithi088 ceph-mon[87351]: pgmap v301: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:02:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:42 smithi038 ceph-mon[112108]: pgmap v302: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:02:42.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:42 smithi088 ceph-mon[87351]: pgmap v302: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:42.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:02:43.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:43 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:02:43.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:43 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:02:44.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:44 smithi038 ceph-mon[112108]: pgmap v303: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:44.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:44 smithi088 ceph-mon[87351]: pgmap v303: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:45.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:45 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:02:45.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:45 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:02:45.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:45 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:02:45.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:45 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:02:45.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:45 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:02:45.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:45 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:02:46.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:46 smithi038 ceph-mon[112108]: pgmap v304: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:46 smithi088 ceph-mon[87351]: pgmap v304: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:48.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:48 smithi038 ceph-mon[112108]: pgmap v305: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:02:48.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:48 smithi088 ceph-mon[87351]: pgmap v305: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:02:50.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:50 smithi038 ceph-mon[112108]: pgmap v306: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:02:50.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:50 smithi088 ceph-mon[87351]: pgmap v306: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:02:52.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:52 smithi038 ceph-mon[112108]: pgmap v307: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:52.682 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:52 smithi088 ceph-mon[87351]: pgmap v307: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:02:54.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:54 smithi038 ceph-mon[112108]: pgmap v308: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:02:54.625 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:54 smithi088 ceph-mon[87351]: pgmap v308: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:02:56.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:56 smithi038 ceph-mon[112108]: pgmap v309: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:02:56.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:56 smithi088 ceph-mon[87351]: pgmap v309: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:02:57.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:02:57.880 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:02:58.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:02:58 smithi088 ceph-mon[87351]: pgmap v310: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-24T01:02:58.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:02:58 smithi038 ceph-mon[112108]: pgmap v310: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-24T01:03:00.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:00 smithi088 ceph-mon[87351]: pgmap v311: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-24T01:03:00.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:00 smithi038 ceph-mon[112108]: pgmap v311: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-24T01:03:02.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:02 smithi038 ceph-mon[112108]: pgmap v312: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:03:02.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:02 smithi088 ceph-mon[87351]: pgmap v312: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:03:04.666 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:04 smithi038 ceph-mon[112108]: pgmap v313: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:03:04.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:04 smithi088 ceph-mon[87351]: pgmap v313: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-24T01:03:06.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:06 smithi088 ceph-mon[87351]: pgmap v314: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:06.723 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:06 smithi038 ceph-mon[112108]: pgmap v314: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:08.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:08 smithi088 ceph-mon[87351]: pgmap v315: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:03:08.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:08 smithi038 ceph-mon[112108]: pgmap v315: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:03:10.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:10 smithi088 ceph-mon[87351]: pgmap v316: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:03:10.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:10 smithi038 ceph-mon[112108]: pgmap v316: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:03:12.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:12 smithi038 ceph-mon[112108]: pgmap v317: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:12.610 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:12 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:03:12.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:12 smithi088 ceph-mon[87351]: pgmap v317: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:12.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:12 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:03:14.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:14 smithi088 ceph-mon[87351]: pgmap v318: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:14.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:14 smithi038 ceph-mon[112108]: pgmap v318: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:16.724 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:16 smithi038 ceph-mon[112108]: pgmap v319: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:16.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:16 smithi088 ceph-mon[87351]: pgmap v319: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:18.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:18 smithi038 ceph-mon[112108]: pgmap v320: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:03:18.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:18 smithi088 ceph-mon[87351]: pgmap v320: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:03:19.391 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph orch ps' 2024-03-24T01:03:19.854 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T01:03:20.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:20 smithi038 ceph-mon[112108]: pgmap v321: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:03:20.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:20 smithi088 ceph-mon[87351]: pgmap v321: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:03:21.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.965+0000 7ff51b400700 1 -- 172.21.15.38:0/2658487982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff514106aa0 msgr2=0x7ff514106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:21.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.965+0000 7ff51b400700 1 --2- 172.21.15.38:0/2658487982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff514106aa0 0x7ff514106e80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff508009910 tx=0x7ff508009c20 comp rx=0 tx=0).stop 2024-03-24T01:03:21.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.965+0000 7ff51b400700 1 -- 172.21.15.38:0/2658487982 shutdown_connections 2024-03-24T01:03:21.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.965+0000 7ff51b400700 1 --2- 172.21.15.38:0/2658487982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff514100aa0 0x7ff514100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:21.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.965+0000 7ff51b400700 1 --2- 172.21.15.38:0/2658487982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff514106aa0 0x7ff514106e80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:21.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.965+0000 7ff51b400700 1 -- 172.21.15.38:0/2658487982 >> 172.21.15.38:0/2658487982 conn(0x7ff5140fc5d0 msgr2=0x7ff5140fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:21.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.965+0000 7ff51b400700 1 -- 172.21.15.38:0/2658487982 shutdown_connections 2024-03-24T01:03:21.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.965+0000 7ff51b400700 1 -- 172.21.15.38:0/2658487982 wait complete. 2024-03-24T01:03:21.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.967+0000 7ff51b400700 1 Processor -- start 2024-03-24T01:03:21.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.967+0000 7ff51b400700 1 -- start start 2024-03-24T01:03:21.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.967+0000 7ff51b400700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff514100aa0 0x7ff5140730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:21.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.967+0000 7ff51b400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff514106aa0 0x7ff5140735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:21.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.967+0000 7ff51b400700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514074d50 con 0x7ff514106aa0 2024-03-24T01:03:21.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.967+0000 7ff51b400700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514074ec0 con 0x7ff514100aa0 2024-03-24T01:03:21.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.968+0000 7ff513e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff514106aa0 0x7ff5140735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:21.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.968+0000 7ff513e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff514106aa0 0x7ff5140735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43152/0 (socket says 172.21.15.38:43152) 2024-03-24T01:03:21.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.968+0000 7ff513e00700 1 -- 172.21.15.38:0/3714984512 learned_addr learned my addr 172.21.15.38:0/3714984512 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:21.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.968+0000 7ff518a00700 1 --2- 172.21.15.38:0/3714984512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff514100aa0 0x7ff5140730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:21.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.968+0000 7ff513e00700 1 -- 172.21.15.38:0/3714984512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff514100aa0 msgr2=0x7ff5140730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:21.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.968+0000 7ff513e00700 1 --2- 172.21.15.38:0/3714984512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff514100aa0 0x7ff5140730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:21.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.968+0000 7ff513e00700 1 -- 172.21.15.38:0/3714984512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080095f0 con 0x7ff514106aa0 2024-03-24T01:03:21.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.969+0000 7ff513e00700 1 --2- 172.21.15.38:0/3714984512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff514106aa0 0x7ff5140735f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff50000b4b0 tx=0x7ff50000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:21.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.969+0000 7ff511600700 1 -- 172.21.15.38:0/3714984512 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff500011800 con 0x7ff514106aa0 2024-03-24T01:03:21.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.969+0000 7ff511600700 1 -- 172.21.15.38:0/3714984512 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff500011e20 con 0x7ff514106aa0 2024-03-24T01:03:21.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.969+0000 7ff511600700 1 -- 172.21.15.38:0/3714984512 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff50000f2e0 con 0x7ff514106aa0 2024-03-24T01:03:21.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.969+0000 7ff51b400700 1 -- 172.21.15.38:0/3714984512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff514073b30 con 0x7ff514106aa0 2024-03-24T01:03:21.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.969+0000 7ff51b400700 1 -- 172.21.15.38:0/3714984512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5141a7450 con 0x7ff514106aa0 2024-03-24T01:03:21.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.971+0000 7ff51b400700 1 -- 172.21.15.38:0/3714984512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5141090a0 con 0x7ff514106aa0 2024-03-24T01:03:21.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.972+0000 7ff511600700 1 -- 172.21.15.38:0/3714984512 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7ff500010380 con 0x7ff514106aa0 2024-03-24T01:03:21.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.973+0000 7ff511600700 1 --2- 172.21.15.38:0/3714984512 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff4fc06dfb0 0x7ff4fc070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:21.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.973+0000 7ff511600700 1 -- 172.21.15.38:0/3714984512 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff50008d8d0 con 0x7ff514106aa0 2024-03-24T01:03:21.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.973+0000 7ff518a00700 1 --2- 172.21.15.38:0/3714984512 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff4fc06dfb0 0x7ff4fc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:21.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.974+0000 7ff518a00700 1 --2- 172.21.15.38:0/3714984512 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff4fc06dfb0 0x7ff4fc070470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff508006010 tx=0x7ff50801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:21.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:21.983+0000 7ff511600700 1 -- 172.21.15.38:0/3714984512 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff500056cd0 con 0x7ff514106aa0 2024-03-24T01:03:22.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.273+0000 7ff51b400700 1 -- 172.21.15.38:0/3714984512 --> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff514074540 con 0x7ff4fc06dfb0 2024-03-24T01:03:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.305+0000 7ff511600700 1 -- 172.21.15.38:0/3714984512 <== mgr.34100 v2:172.21.15.38:6800/1632725319 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff514074540 con 0x7ff4fc06dfb0 2024-03-24T01:03:22.307 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T01:03:22.307 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (11m) 108s ago 25m 14.5M - 0.25.0 c8568f914cd2 760e7678a52b 2024-03-24T01:03:22.307 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi038 smithi038 running (2m) 108s ago 26m 7818k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 eac5e34d263f 2024-03-24T01:03:22.308 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-exporter.smithi088 smithi088 running (115s) 111s ago 24m 7096k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 10a178ef9957 2024-03-24T01:03:22.308 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (7m) 108s ago 25m 6656k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 25be5c9b7dc4 2024-03-24T01:03:22.308 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi088 smithi088 running (7m) 111s ago 24m 6660k - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b94815838d8 2024-03-24T01:03:22.308 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (11m) 108s ago 25m 74.5M - 9.4.12 a0ea5e3c46d9 5e2632a7ff06 2024-03-24T01:03:22.308 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.xnpqhp smithi038 running (3m) 108s ago 18m 15.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b78588cbee24 2024-03-24T01:03:22.309 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.yhkmsb smithi038 running (3m) 108s ago 18m 82.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e597065368f0 2024-03-24T01:03:22.309 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.cbfvvp smithi088 running (2m) 111s ago 18m 82.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 49be5a5d2c69 2024-03-24T01:03:22.309 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi088.yhksbr smithi088 running (2m) 111s ago 18m 13.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b40c5fbe832e 2024-03-24T01:03:22.309 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.huhioz smithi038 *:8443,9283,8765 running (14m) 108s ago 27m 511M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 947593dc7742 2024-03-24T01:03:22.309 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi088.njctea smithi088 *:8443,9283,8765 running (13m) 111s ago 24m 452M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0b8d0191960 2024-03-24T01:03:22.309 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (9m) 108s ago 27m 57.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ba098af28413 2024-03-24T01:03:22.309 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi088 smithi088 running (8m) 111s ago 24m 49.4M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3bace4fab2b0 2024-03-24T01:03:22.309 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (12m) 108s ago 25m 8066k - 1.5.0 0da6a335fe13 dc4122a2afc2 2024-03-24T01:03:22.309 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi088 smithi088 *:9100 running (12m) 111s ago 24m 7700k - 1.5.0 0da6a335fe13 5d43b288cf97 2024-03-24T01:03:22.310 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (7m) 108s ago 23m 106M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a74a6e0da73c 2024-03-24T01:03:22.310 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (6m) 108s ago 22m 119M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cb72ac446ba7 2024-03-24T01:03:22.310 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (6m) 108s ago 21m 100M 1218M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c73f3e8276b6 2024-03-24T01:03:22.310 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi088 running (5m) 111s ago 21m 124M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b2afaff761d8 2024-03-24T01:03:22.310 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi088 running (4m) 111s ago 20m 108M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9a6f202043cd 2024-03-24T01:03:22.310 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi088 running (4m) 111s ago 20m 109M 2241M 19.0.0-2145-g3be9a5c9 c664afe56bb0 866388fc07f6 2024-03-24T01:03:22.310 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (12m) 108s ago 25m 75.2M - 2.43.0 a07b618ecd1d 201d23de57cd 2024-03-24T01:03:22.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.308+0000 7ff51b400700 1 -- 172.21.15.38:0/3714984512 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff4fc06dfb0 msgr2=0x7ff4fc070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:22.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.309+0000 7ff51b400700 1 --2- 172.21.15.38:0/3714984512 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff4fc06dfb0 0x7ff4fc070470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff508006010 tx=0x7ff50801a040 comp rx=0 tx=0).stop 2024-03-24T01:03:22.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.309+0000 7ff51b400700 1 -- 172.21.15.38:0/3714984512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff514106aa0 msgr2=0x7ff5140735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:22.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.309+0000 7ff51b400700 1 --2- 172.21.15.38:0/3714984512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff514106aa0 0x7ff5140735f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff50000b4b0 tx=0x7ff50000b7c0 comp rx=0 tx=0).stop 2024-03-24T01:03:22.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.309+0000 7ff51b400700 1 -- 172.21.15.38:0/3714984512 shutdown_connections 2024-03-24T01:03:22.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.309+0000 7ff51b400700 1 --2- 172.21.15.38:0/3714984512 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff4fc06dfb0 0x7ff4fc070470 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:22.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.309+0000 7ff51b400700 1 --2- 172.21.15.38:0/3714984512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff514100aa0 0x7ff5140730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:22.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.309+0000 7ff51b400700 1 --2- 172.21.15.38:0/3714984512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff514106aa0 0x7ff5140735f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:22.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.309+0000 7ff51b400700 1 -- 172.21.15.38:0/3714984512 >> 172.21.15.38:0/3714984512 conn(0x7ff5140fc5d0 msgr2=0x7ff5140fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:22.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.309+0000 7ff51b400700 1 -- 172.21.15.38:0/3714984512 shutdown_connections 2024-03-24T01:03:22.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:22.309+0000 7ff51b400700 1 -- 172.21.15.38:0/3714984512 wait complete. 2024-03-24T01:03:22.723 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:22 smithi038 ceph-mon[112108]: pgmap v322: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:22.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:22 smithi088 ceph-mon[87351]: pgmap v322: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:23.165 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph versions' 2024-03-24T01:03:23.610 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T01:03:23.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:23 smithi038 ceph-mon[112108]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:03:23.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:23 smithi088 ceph-mon[87351]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T01:03:24.802 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:24 smithi038 ceph-mon[112108]: pgmap v323: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:24.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:24 smithi088 ceph-mon[87351]: pgmap v323: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:25.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.666+0000 7f16fcc00700 1 -- 172.21.15.38:0/3279338011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16f81008e0 msgr2=0x7f16f8100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:25.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.666+0000 7f16fcc00700 1 --2- 172.21.15.38:0/3279338011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16f81008e0 0x7f16f8100d60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009910 tx=0x7f16e4009c20 comp rx=0 tx=0).stop 2024-03-24T01:03:25.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.666+0000 7f16fcc00700 1 -- 172.21.15.38:0/3279338011 shutdown_connections 2024-03-24T01:03:25.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.666+0000 7f16fcc00700 1 --2- 172.21.15.38:0/3279338011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16f81008e0 0x7f16f8100d60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:25.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.666+0000 7f16fcc00700 1 --2- 172.21.15.38:0/3279338011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16f81068e0 0x7f16f8106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:25.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.666+0000 7f16fcc00700 1 -- 172.21.15.38:0/3279338011 >> 172.21.15.38:0/3279338011 conn(0x7f16f80fc3f0 msgr2=0x7f16f80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:25.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.667+0000 7f16fcc00700 1 -- 172.21.15.38:0/3279338011 shutdown_connections 2024-03-24T01:03:25.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.667+0000 7f16fcc00700 1 -- 172.21.15.38:0/3279338011 wait complete. 2024-03-24T01:03:25.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.668+0000 7f16fcc00700 1 Processor -- start 2024-03-24T01:03:25.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.668+0000 7f16fcc00700 1 -- start start 2024-03-24T01:03:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.668+0000 7f16fcc00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16f81008e0 0x7f16f80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.668+0000 7f16fcc00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16f81068e0 0x7f16f80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.668+0000 7f16fcc00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f8073b30 con 0x7f16f81008e0 2024-03-24T01:03:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.668+0000 7f16fcc00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f8073ca0 con 0x7f16f81068e0 2024-03-24T01:03:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.669+0000 7f16f7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16f81008e0 0x7f16f80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.669+0000 7f16f7400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16f81008e0 0x7f16f80730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35260/0 (socket says 172.21.15.38:35260) 2024-03-24T01:03:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.669+0000 7f16f7400700 1 -- 172.21.15.38:0/115952489 learned_addr learned my addr 172.21.15.38:0/115952489 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.669+0000 7f16f7400700 1 -- 172.21.15.38:0/115952489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16f81068e0 msgr2=0x7f16f80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.669+0000 7f16f6a00700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16f81068e0 0x7f16f80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.669+0000 7f16f7400700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16f81068e0 0x7f16f80735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.670+0000 7f16f7400700 1 -- 172.21.15.38:0/115952489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e40095f0 con 0x7f16f81008e0 2024-03-24T01:03:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.670+0000 7f16f6a00700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16f81068e0 0x7f16f80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T01:03:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.670+0000 7f16f7400700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16f81008e0 0x7f16f80730b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e920 tx=0x7f16e800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.670+0000 7f16efe00700 1 -- 172.21.15.38:0/115952489 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16e8009790 con 0x7f16f81008e0 2024-03-24T01:03:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.670+0000 7f16efe00700 1 -- 172.21.15.38:0/115952489 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16e800cba0 con 0x7f16f81008e0 2024-03-24T01:03:25.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.670+0000 7f16fcc00700 1 -- 172.21.15.38:0/115952489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f81a2a00 con 0x7f16f81008e0 2024-03-24T01:03:25.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.670+0000 7f16efe00700 1 -- 172.21.15.38:0/115952489 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16e8018650 con 0x7f16f81008e0 2024-03-24T01:03:25.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.671+0000 7f16fcc00700 1 -- 172.21.15.38:0/115952489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f81a2db0 con 0x7f16f81008e0 2024-03-24T01:03:25.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.672+0000 7f16fcc00700 1 -- 172.21.15.38:0/115952489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d80052f0 con 0x7f16f81008e0 2024-03-24T01:03:25.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.673+0000 7f16efe00700 1 -- 172.21.15.38:0/115952489 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f16e800cd10 con 0x7f16f81008e0 2024-03-24T01:03:25.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.675+0000 7f16efe00700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16dc06dfa0 0x7f16dc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:25.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.675+0000 7f16efe00700 1 -- 172.21.15.38:0/115952489 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f16e8014070 con 0x7f16f81008e0 2024-03-24T01:03:25.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.675+0000 7f16f6a00700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16dc06dfa0 0x7f16dc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:25.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.676+0000 7f16f6a00700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16dc06dfa0 0x7f16dc070460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f16e4000c00 tx=0x7f16e4009f70 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:25.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:25.679+0000 7f16efe00700 1 -- 172.21.15.38:0/115952489 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f16e8053740 con 0x7f16f81008e0 2024-03-24T01:03:26.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.123+0000 7f16fcc00700 1 -- 172.21.15.38:0/115952489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f16d80061d0 con 0x7f16f81008e0 2024-03-24T01:03:26.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.124+0000 7f16efe00700 1 -- 172.21.15.38:0/115952489 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f16e8057550 con 0x7f16f81008e0 2024-03-24T01:03:26.125 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 2 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 6 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 4 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-03-24T01:03:26.126 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-03-24T01:03:26.127 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)": 14 2024-03-24T01:03:26.127 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-03-24T01:03:26.127 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-03-24T01:03:26.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.127+0000 7f16fcc00700 1 -- 172.21.15.38:0/115952489 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16dc06dfa0 msgr2=0x7f16dc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:26.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.127+0000 7f16fcc00700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16dc06dfa0 0x7f16dc070460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f16e4000c00 tx=0x7f16e4009f70 comp rx=0 tx=0).stop 2024-03-24T01:03:26.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.127+0000 7f16fcc00700 1 -- 172.21.15.38:0/115952489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16f81008e0 msgr2=0x7f16f80730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:26.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.127+0000 7f16fcc00700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16f81008e0 0x7f16f80730b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e920 tx=0x7f16e800ece0 comp rx=0 tx=0).stop 2024-03-24T01:03:26.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.127+0000 7f16fcc00700 1 -- 172.21.15.38:0/115952489 shutdown_connections 2024-03-24T01:03:26.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.127+0000 7f16fcc00700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16dc06dfa0 0x7f16dc070460 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:26.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.127+0000 7f16fcc00700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16f81008e0 0x7f16f80730b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:26.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.127+0000 7f16fcc00700 1 --2- 172.21.15.38:0/115952489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16f81068e0 0x7f16f80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:26.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.127+0000 7f16fcc00700 1 -- 172.21.15.38:0/115952489 >> 172.21.15.38:0/115952489 conn(0x7f16f80fc3f0 msgr2=0x7f16f80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:26.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.128+0000 7f16fcc00700 1 -- 172.21.15.38:0/115952489 shutdown_connections 2024-03-24T01:03:26.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:26.128+0000 7f16fcc00700 1 -- 172.21.15.38:0/115952489 wait complete. 2024-03-24T01:03:26.725 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:26 smithi038 ceph-mon[112108]: pgmap v324: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:26.726 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:26 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/115952489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:03:26.902 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-24T01:03:26.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:26 smithi088 ceph-mon[87351]: pgmap v324: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:26.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:26 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/115952489' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:03:27.326 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T01:03:27.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:27 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:03:27.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:27 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:03:28.666 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:28 smithi038 ceph-mon[112108]: pgmap v325: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:03:28.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:28 smithi088 ceph-mon[87351]: pgmap v325: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:03:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.557+0000 7f2c6f800700 1 -- 172.21.15.38:0/1366766597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c68100150 msgr2=0x7f2c6810ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.557+0000 7f2c6f800700 1 --2- 172.21.15.38:0/1366766597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c68100150 0x7f2c6810ce10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009910 tx=0x7f2c54009c20 comp rx=0 tx=0).stop 2024-03-24T01:03:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.557+0000 7f2c6f800700 1 -- 172.21.15.38:0/1366766597 shutdown_connections 2024-03-24T01:03:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.557+0000 7f2c6f800700 1 --2- 172.21.15.38:0/1366766597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c68100150 0x7f2c6810ce10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:29.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.557+0000 7f2c6f800700 1 --2- 172.21.15.38:0/1366766597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c680ff7a0 0x7f2c680ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:29.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.557+0000 7f2c6f800700 1 -- 172.21.15.38:0/1366766597 >> 172.21.15.38:0/1366766597 conn(0x7f2c68076640 msgr2=0x7f2c680fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:29.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.557+0000 7f2c6f800700 1 -- 172.21.15.38:0/1366766597 shutdown_connections 2024-03-24T01:03:29.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.557+0000 7f2c6f800700 1 -- 172.21.15.38:0/1366766597 wait complete. 2024-03-24T01:03:29.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.558+0000 7f2c6f800700 1 Processor -- start 2024-03-24T01:03:29.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.558+0000 7f2c6f800700 1 -- start start 2024-03-24T01:03:29.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.559+0000 7f2c6f800700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c680ff7a0 0x7f2c6819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:29.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.559+0000 7f2c6f800700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c68100150 0x7f2c6819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:29.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.559+0000 7f2c6f800700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6819f400 con 0x7f2c68100150 2024-03-24T01:03:29.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.559+0000 7f2c6f800700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c681989e0 con 0x7f2c680ff7a0 2024-03-24T01:03:29.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.559+0000 7f2c67e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c68100150 0x7f2c6819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:29.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.559+0000 7f2c67e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c68100150 0x7f2c6819ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35268/0 (socket says 172.21.15.38:35268) 2024-03-24T01:03:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.559+0000 7f2c67e00700 1 -- 172.21.15.38:0/816867330 learned_addr learned my addr 172.21.15.38:0/816867330 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.560+0000 7f2c6ce00700 1 --2- 172.21.15.38:0/816867330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c680ff7a0 0x7f2c6819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.560+0000 7f2c67e00700 1 -- 172.21.15.38:0/816867330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c680ff7a0 msgr2=0x7f2c6819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.560+0000 7f2c67e00700 1 --2- 172.21.15.38:0/816867330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c680ff7a0 0x7f2c6819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.560+0000 7f2c67e00700 1 -- 172.21.15.38:0/816867330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c540095f0 con 0x7f2c68100150 2024-03-24T01:03:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.560+0000 7f2c67e00700 1 --2- 172.21.15.38:0/816867330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c68100150 0x7f2c6819ede0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400c010 tx=0x7f2c54005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.561+0000 7f2c65600700 1 -- 172.21.15.38:0/816867330 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c5401d070 con 0x7f2c68100150 2024-03-24T01:03:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.561+0000 7f2c65600700 1 -- 172.21.15.38:0/816867330 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c540052a0 con 0x7f2c68100150 2024-03-24T01:03:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.561+0000 7f2c6f800700 1 -- 172.21.15.38:0/816867330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c68198c60 con 0x7f2c68100150 2024-03-24T01:03:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.561+0000 7f2c65600700 1 -- 172.21.15.38:0/816867330 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c54022470 con 0x7f2c68100150 2024-03-24T01:03:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.561+0000 7f2c6f800700 1 -- 172.21.15.38:0/816867330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c68199150 con 0x7f2c68100150 2024-03-24T01:03:29.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.562+0000 7f2c6f800700 1 -- 172.21.15.38:0/816867330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c68068d60 con 0x7f2c68100150 2024-03-24T01:03:29.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.563+0000 7f2c65600700 1 -- 172.21.15.38:0/816867330 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f2c54003710 con 0x7f2c68100150 2024-03-24T01:03:29.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.565+0000 7f2c65600700 1 --2- 172.21.15.38:0/816867330 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f2c5006dfb0 0x7f2c50070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:29.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.565+0000 7f2c65600700 1 -- 172.21.15.38:0/816867330 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2c5408f2c0 con 0x7f2c68100150 2024-03-24T01:03:29.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.565+0000 7f2c6ce00700 1 --2- 172.21.15.38:0/816867330 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f2c5006dfb0 0x7f2c50070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:29.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.566+0000 7f2c6ce00700 1 --2- 172.21.15.38:0/816867330 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f2c5006dfb0 0x7f2c50070470 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009bf0 tx=0x7f2c5c009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:29.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.570+0000 7f2c65600700 1 -- 172.21.15.38:0/816867330 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2c5405c220 con 0x7f2c68100150 2024-03-24T01:03:29.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.953+0000 7f2c6f800700 1 -- 172.21.15.38:0/816867330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2c6804ea80 con 0x7f2c68100150 2024-03-24T01:03:29.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.954+0000 7f2c65600700 1 -- 172.21.15.38:0/816867330 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2c5405bdb0 con 0x7f2c68100150 2024-03-24T01:03:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.958+0000 7f2c6f800700 1 -- 172.21.15.38:0/816867330 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f2c5006dfb0 msgr2=0x7f2c50070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.958+0000 7f2c6f800700 1 --2- 172.21.15.38:0/816867330 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f2c5006dfb0 0x7f2c50070470 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009bf0 tx=0x7f2c5c009260 comp rx=0 tx=0).stop 2024-03-24T01:03:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.958+0000 7f2c6f800700 1 -- 172.21.15.38:0/816867330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c68100150 msgr2=0x7f2c6819ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.958+0000 7f2c6f800700 1 --2- 172.21.15.38:0/816867330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c68100150 0x7f2c6819ede0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400c010 tx=0x7f2c54005770 comp rx=0 tx=0).stop 2024-03-24T01:03:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.958+0000 7f2c6f800700 1 -- 172.21.15.38:0/816867330 shutdown_connections 2024-03-24T01:03:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.958+0000 7f2c6f800700 1 --2- 172.21.15.38:0/816867330 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f2c5006dfb0 0x7f2c50070470 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.958+0000 7f2c6f800700 1 --2- 172.21.15.38:0/816867330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c680ff7a0 0x7f2c6819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:29.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.958+0000 7f2c6f800700 1 --2- 172.21.15.38:0/816867330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c68100150 0x7f2c6819ede0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:29.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.958+0000 7f2c6f800700 1 -- 172.21.15.38:0/816867330 >> 172.21.15.38:0/816867330 conn(0x7f2c68076640 msgr2=0x7f2c68109580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:29.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.958+0000 7f2c6f800700 1 -- 172.21.15.38:0/816867330 shutdown_connections 2024-03-24T01:03:29.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:29.958+0000 7f2c6f800700 1 -- 172.21.15.38:0/816867330 wait complete. 2024-03-24T01:03:29.980 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-03-24T01:03:30.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:30 smithi038 ceph-mon[112108]: pgmap v326: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:03:30.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:30 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/816867330' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:03:30.707 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -e sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-24T01:03:30.810 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:30 smithi088 ceph-mon[87351]: pgmap v326: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:03:30.810 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:30 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/816867330' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:03:31.206 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T01:03:32.855 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:32 smithi038 ceph-mon[112108]: pgmap v327: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:32.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:32 smithi088 ceph-mon[87351]: pgmap v327: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:33.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.347+0000 7fcde9a00700 1 -- 172.21.15.38:0/3604286153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcde4106a70 msgr2=0x7fcde4106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:33.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.347+0000 7fcde9a00700 1 --2- 172.21.15.38:0/3604286153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcde4106a70 0x7fcde4106e50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009910 tx=0x7fcdd4009c20 comp rx=0 tx=0).stop 2024-03-24T01:03:33.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.347+0000 7fcde9a00700 1 -- 172.21.15.38:0/3604286153 shutdown_connections 2024-03-24T01:03:33.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.347+0000 7fcde9a00700 1 --2- 172.21.15.38:0/3604286153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde4100a70 0x7fcde4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:33.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.347+0000 7fcde9a00700 1 --2- 172.21.15.38:0/3604286153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcde4106a70 0x7fcde4106e50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:33.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.347+0000 7fcde9a00700 1 -- 172.21.15.38:0/3604286153 >> 172.21.15.38:0/3604286153 conn(0x7fcde40fc580 msgr2=0x7fcde40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:33.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.347+0000 7fcde9a00700 1 -- 172.21.15.38:0/3604286153 shutdown_connections 2024-03-24T01:03:33.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.348+0000 7fcde9a00700 1 -- 172.21.15.38:0/3604286153 wait complete. 2024-03-24T01:03:33.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.349+0000 7fcde9a00700 1 Processor -- start 2024-03-24T01:03:33.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.349+0000 7fcde9a00700 1 -- start start 2024-03-24T01:03:33.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.350+0000 7fcde9a00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde4100a70 0x7fcde419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:33.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.350+0000 7fcde9a00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcde4106a70 0x7fcde419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:33.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.350+0000 7fcde9a00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde41989a0 con 0x7fcde4106a70 2024-03-24T01:03:33.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.350+0000 7fcde9a00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4198b10 con 0x7fcde4100a70 2024-03-24T01:03:33.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.350+0000 7fcde2000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcde4106a70 0x7fcde419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:33.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.350+0000 7fcde2000700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcde4106a70 0x7fcde419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48390/0 (socket says 172.21.15.38:48390) 2024-03-24T01:03:33.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.350+0000 7fcde2000700 1 -- 172.21.15.38:0/1943898764 learned_addr learned my addr 172.21.15.38:0/1943898764 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:33.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.350+0000 7fcde2a00700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde4100a70 0x7fcde419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:33.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.351+0000 7fcde2000700 1 -- 172.21.15.38:0/1943898764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde4100a70 msgr2=0x7fcde419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:33.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.351+0000 7fcde2000700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde4100a70 0x7fcde419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:33.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.351+0000 7fcde2000700 1 -- 172.21.15.38:0/1943898764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd40095f0 con 0x7fcde4106a70 2024-03-24T01:03:33.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.351+0000 7fcde2a00700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde4100a70 0x7fcde419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T01:03:33.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.351+0000 7fcde2000700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcde4106a70 0x7fcde419ee30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000e920 tx=0x7fcdd000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:33.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.351+0000 7fcddb400700 1 -- 172.21.15.38:0/1943898764 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd0009790 con 0x7fcde4106a70 2024-03-24T01:03:33.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.352+0000 7fcddb400700 1 -- 172.21.15.38:0/1943898764 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdd000cba0 con 0x7fcde4106a70 2024-03-24T01:03:33.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.352+0000 7fcddb400700 1 -- 172.21.15.38:0/1943898764 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd0018650 con 0x7fcde4106a70 2024-03-24T01:03:33.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.352+0000 7fcde9a00700 1 -- 172.21.15.38:0/1943898764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde4198df0 con 0x7fcde4106a70 2024-03-24T01:03:33.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.352+0000 7fcde9a00700 1 -- 172.21.15.38:0/1943898764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde4199340 con 0x7fcde4106a70 2024-03-24T01:03:33.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.353+0000 7fcde9a00700 1 -- 172.21.15.38:0/1943898764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde4104d20 con 0x7fcde4106a70 2024-03-24T01:03:33.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.354+0000 7fcddb400700 1 -- 172.21.15.38:0/1943898764 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fcdd000cd10 con 0x7fcde4106a70 2024-03-24T01:03:33.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.356+0000 7fcddb400700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcdc806e000 0x7fcdc80704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:33.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.356+0000 7fcddb400700 1 -- 172.21.15.38:0/1943898764 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcdd0014070 con 0x7fcde4106a70 2024-03-24T01:03:33.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.356+0000 7fcde2a00700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcdc806e000 0x7fcdc80704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:33.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.357+0000 7fcde2a00700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcdc806e000 0x7fcdc80704c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400b760 tx=0x7fcdd4005ab0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:33.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.363+0000 7fcddb400700 1 -- 172.21.15.38:0/1943898764 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcdd0053740 con 0x7fcde4106a70 2024-03-24T01:03:33.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.726+0000 7fcde9a00700 1 -- 172.21.15.38:0/1943898764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcde4199e80 con 0x7fcde4106a70 2024-03-24T01:03:33.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.727+0000 7fcddb400700 1 -- 172.21.15.38:0/1943898764 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fcdd005b060 con 0x7fcde4106a70 2024-03-24T01:03:33.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.730+0000 7fcde9a00700 1 -- 172.21.15.38:0/1943898764 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcdc806e000 msgr2=0x7fcdc80704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:33.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.730+0000 7fcde9a00700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcdc806e000 0x7fcdc80704c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400b760 tx=0x7fcdd4005ab0 comp rx=0 tx=0).stop 2024-03-24T01:03:33.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.730+0000 7fcde9a00700 1 -- 172.21.15.38:0/1943898764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcde4106a70 msgr2=0x7fcde419ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:33.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.730+0000 7fcde9a00700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcde4106a70 0x7fcde419ee30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000e920 tx=0x7fcdd000ece0 comp rx=0 tx=0).stop 2024-03-24T01:03:33.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.730+0000 7fcde9a00700 1 -- 172.21.15.38:0/1943898764 shutdown_connections 2024-03-24T01:03:33.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.730+0000 7fcde9a00700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fcdc806e000 0x7fcdc80704c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:33.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.730+0000 7fcde9a00700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcde4100a70 0x7fcde419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:33.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.730+0000 7fcde9a00700 1 --2- 172.21.15.38:0/1943898764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcde4106a70 0x7fcde419ee30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:33.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.730+0000 7fcde9a00700 1 -- 172.21.15.38:0/1943898764 >> 172.21.15.38:0/1943898764 conn(0x7fcde40fc580 msgr2=0x7fcde4073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:33.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.730+0000 7fcde9a00700 1 -- 172.21.15.38:0/1943898764 shutdown_connections 2024-03-24T01:03:33.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:33.730+0000 7fcde9a00700 1 -- 172.21.15.38:0/1943898764 wait complete. 2024-03-24T01:03:33.756 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)" 2024-03-24T01:03:34.569 DEBUG:teuthology.parallel:result is None 2024-03-24T01:03:34.569 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T01:03:34.580 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T01:03:34.581 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-24T01:03:34.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:34 smithi038 ceph-mon[112108]: pgmap v328: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:34.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:34 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1943898764' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:03:34.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:34 smithi088 ceph-mon[87351]: pgmap v328: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:34.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:34 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1943898764' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T01:03:35.017 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/mon.smithi038/config 2024-03-24T01:03:36.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:36 smithi038 ceph-mon[112108]: pgmap v329: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:36.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:36 smithi088 ceph-mon[87351]: pgmap v329: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.046+0000 7f5da8e00700 1 -- 172.21.15.38:0/4059891126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da4100a70 msgr2=0x7f5da4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.046+0000 7f5da8e00700 1 --2- 172.21.15.38:0/4059891126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da4100a70 0x7f5da4100ef0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009910 tx=0x7f5d90009c20 comp rx=0 tx=0).stop 2024-03-24T01:03:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.047+0000 7f5da8e00700 1 -- 172.21.15.38:0/4059891126 shutdown_connections 2024-03-24T01:03:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.047+0000 7f5da8e00700 1 --2- 172.21.15.38:0/4059891126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da4100a70 0x7f5da4100ef0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.047+0000 7f5da8e00700 1 --2- 172.21.15.38:0/4059891126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5da4106a70 0x7f5da4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.047+0000 7f5da8e00700 1 -- 172.21.15.38:0/4059891126 >> 172.21.15.38:0/4059891126 conn(0x7f5da40fc580 msgr2=0x7f5da40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.047+0000 7f5da8e00700 1 -- 172.21.15.38:0/4059891126 shutdown_connections 2024-03-24T01:03:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.047+0000 7f5da8e00700 1 -- 172.21.15.38:0/4059891126 wait complete. 2024-03-24T01:03:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.048+0000 7f5da8e00700 1 Processor -- start 2024-03-24T01:03:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.048+0000 7f5da8e00700 1 -- start start 2024-03-24T01:03:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.049+0000 7f5da8e00700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5da4100a70 0x7f5da419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.049+0000 7f5da8e00700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da4106a70 0x7f5da419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.049+0000 7f5da8e00700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da41989c0 con 0x7f5da4106a70 2024-03-24T01:03:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.049+0000 7f5da8e00700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4198b30 con 0x7f5da4100a70 2024-03-24T01:03:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.049+0000 7f5da1400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da4106a70 0x7f5da419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.049+0000 7f5da1400700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da4106a70 0x7f5da419ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48408/0 (socket says 172.21.15.38:48408) 2024-03-24T01:03:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.049+0000 7f5da1400700 1 -- 172.21.15.38:0/2300320652 learned_addr learned my addr 172.21.15.38:0/2300320652 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.050+0000 7f5da1e00700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5da4100a70 0x7f5da419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.050+0000 7f5da1400700 1 -- 172.21.15.38:0/2300320652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5da4100a70 msgr2=0x7f5da419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.050+0000 7f5da1400700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5da4100a70 0x7f5da419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.050+0000 7f5da1400700 1 -- 172.21.15.38:0/2300320652 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d900095f0 con 0x7f5da4106a70 2024-03-24T01:03:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.050+0000 7f5da1e00700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5da4100a70 0x7f5da419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T01:03:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.050+0000 7f5da1400700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da4106a70 0x7f5da419ee50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5d90005f50 tx=0x7f5d9000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.051+0000 7f5d9aa00700 1 -- 172.21.15.38:0/2300320652 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d9001d070 con 0x7f5da4106a70 2024-03-24T01:03:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.051+0000 7f5d9aa00700 1 -- 172.21.15.38:0/2300320652 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d900052a0 con 0x7f5da4106a70 2024-03-24T01:03:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.051+0000 7f5da8e00700 1 -- 172.21.15.38:0/2300320652 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da4198db0 con 0x7f5da4106a70 2024-03-24T01:03:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.051+0000 7f5d9aa00700 1 -- 172.21.15.38:0/2300320652 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d90017400 con 0x7f5da4106a70 2024-03-24T01:03:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.051+0000 7f5da8e00700 1 -- 172.21.15.38:0/2300320652 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da41992a0 con 0x7f5da4106a70 2024-03-24T01:03:37.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.052+0000 7f5da8e00700 1 -- 172.21.15.38:0/2300320652 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da4104d20 con 0x7f5da4106a70 2024-03-24T01:03:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.053+0000 7f5d9aa00700 1 -- 172.21.15.38:0/2300320652 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f5d9000f7a0 con 0x7f5da4106a70 2024-03-24T01:03:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.054+0000 7f5d9aa00700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5d8806dfb0 0x7f5d88070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.055+0000 7f5d9aa00700 1 -- 172.21.15.38:0/2300320652 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5d9008f1d0 con 0x7f5da4106a70 2024-03-24T01:03:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.055+0000 7f5da1e00700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5d8806dfb0 0x7f5d88070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.056+0000 7f5da1e00700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5d8806dfb0 0x7f5d88070470 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009cb0 tx=0x7f5d94009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:37.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.063+0000 7f5d9aa00700 1 -- 172.21.15.38:0/2300320652 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d900585d0 con 0x7f5da4106a70 2024-03-24T01:03:37.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.372+0000 7f5da8e00700 1 -- 172.21.15.38:0/2300320652 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5da4199ea0 con 0x7f5da4106a70 2024-03-24T01:03:37.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.373+0000 7f5d9aa00700 1 -- 172.21.15.38:0/2300320652 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 40 v40) v1 ==== 76+0+1939 (secure 0 0 0) 0x7f5d9005bc90 con 0x7f5da4106a70 2024-03-24T01:03:37.374 INFO:teuthology.orchestra.run.smithi038.stdout:e40 2024-03-24T01:03:37.374 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T01:03:37.374 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T01:03:37.374 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-03-24T01:03:37.375 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:37.375 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-03-24T01:03:37.375 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-03-24T01:03:37.375 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 40 2024-03-24T01:03:37.376 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T01:03:37.376 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-03-24T00:44:51.138029+0000 2024-03-24T01:03:37.376 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-03-24T01:01:41.762046+0000 2024-03-24T01:03:37.376 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-03-24T01:03:37.376 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-03-24T01:03:37.377 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-03-24T01:03:37.377 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-03-24T01:03:37.377 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-03-24T01:03:37.377 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-03-24T01:03:37.377 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-03-24T01:03:37.378 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-03-24T01:03:37.378 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 83 2024-03-24T01:03:37.378 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-24T01:03:37.378 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-03-24T01:03:37.379 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-03-24T01:03:37.379 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=34446,1=34478} 2024-03-24T01:03:37.379 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-03-24T01:03:37.379 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-03-24T01:03:37.379 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-03-24T01:03:37.380 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-03-24T01:03:37.380 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-03-24T01:03:37.380 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-03-24T01:03:37.380 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-03-24T01:03:37.380 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-03-24T01:03:37.381 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-03-24T01:03:37.381 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 34446 members: 34446,34478 2024-03-24T01:03:37.381 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.yhkmsb{0:34446} state up:active seq 29 join_fscid=1 addr [v2:172.21.15.38:6826/1176900242,v1:172.21.15.38:6827/1176900242] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:03:37.381 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.cbfvvp{0:44345} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6824/2293661188,v1:172.21.15.88:6825/2293661188] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:03:37.381 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.xnpqhp{1:34478} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.38:6828/2579165695,v1:172.21.15.38:6829/2579165695] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:03:37.382 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi088.yhksbr{1:44373} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/1135457323,v1:172.21.15.88:6827/1135457323] compat {c=[1],r=[1],i=[fff]}] 2024-03-24T01:03:37.382 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:37.382 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:37.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.376+0000 7f5da8e00700 1 -- 172.21.15.38:0/2300320652 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5d8806dfb0 msgr2=0x7f5d88070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:37.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.376+0000 7f5da8e00700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5d8806dfb0 0x7f5d88070470 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009cb0 tx=0x7f5d94009260 comp rx=0 tx=0).stop 2024-03-24T01:03:37.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.376+0000 7f5da8e00700 1 -- 172.21.15.38:0/2300320652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da4106a70 msgr2=0x7f5da419ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:37.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.376+0000 7f5da8e00700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da4106a70 0x7f5da419ee50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5d90005f50 tx=0x7f5d9000b650 comp rx=0 tx=0).stop 2024-03-24T01:03:37.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.376+0000 7f5da8e00700 1 -- 172.21.15.38:0/2300320652 shutdown_connections 2024-03-24T01:03:37.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.376+0000 7f5da8e00700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f5d8806dfb0 0x7f5d88070470 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:37.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.376+0000 7f5da8e00700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5da4100a70 0x7f5da419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:37.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.376+0000 7f5da8e00700 1 --2- 172.21.15.38:0/2300320652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5da4106a70 0x7f5da419ee50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:37.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.376+0000 7f5da8e00700 1 -- 172.21.15.38:0/2300320652 >> 172.21.15.38:0/2300320652 conn(0x7f5da40fc580 msgr2=0x7f5da40731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:37.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.376+0000 7f5da8e00700 1 -- 172.21.15.38:0/2300320652 shutdown_connections 2024-03-24T01:03:37.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:37.377+0000 7f5da8e00700 1 -- 172.21.15.38:0/2300320652 wait complete. 2024-03-24T01:03:37.386 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 40 2024-03-24T01:03:37.791 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:37 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2300320652' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T01:03:37.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:37 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2300320652' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T01:03:38.222 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-24T01:03:38.236 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T01:03:38.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.447+0000 7f713be00640 1 -- 172.21.15.38:0/2215043408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f713c104d70 msgr2=0x7f713c067260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:38.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.447+0000 7f713be00640 1 --2- 172.21.15.38:0/2215043408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f713c104d70 0x7f713c067260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f712c009880 tx=0x7f712c02bce0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.447+0000 7f713be00640 1 -- 172.21.15.38:0/2215043408 shutdown_connections 2024-03-24T01:03:38.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.447+0000 7f713be00640 1 --2- 172.21.15.38:0/2215043408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f713c0677a0 0x7f713c067c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.447+0000 7f713be00640 1 --2- 172.21.15.38:0/2215043408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f713c104d70 0x7f713c067260 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.447+0000 7f713be00640 1 -- 172.21.15.38:0/2215043408 >> 172.21.15.38:0/2215043408 conn(0x7f713c0fab80 msgr2=0x7f713c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:38.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.447+0000 7f713be00640 1 -- 172.21.15.38:0/2215043408 shutdown_connections 2024-03-24T01:03:38.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.447+0000 7f713be00640 1 -- 172.21.15.38:0/2215043408 wait complete. 2024-03-24T01:03:38.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.448+0000 7f713be00640 1 Processor -- start 2024-03-24T01:03:38.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.449+0000 7f713be00640 1 -- start start 2024-03-24T01:03:38.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.449+0000 7f713be00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f713c0677a0 0x7f713c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:38.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.449+0000 7f713be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f713c104d70 0x7f713c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:38.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.449+0000 7f713be00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c19cec0 con 0x7f713c0677a0 2024-03-24T01:03:38.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.449+0000 7f713be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c1a0bd0 con 0x7f713c104d70 2024-03-24T01:03:38.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.449+0000 7f713aa00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f713c0677a0 0x7f713c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:38.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.449+0000 7f713aa00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f713c0677a0 0x7f713c19c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48428/0 (socket says 172.21.15.38:48428) 2024-03-24T01:03:38.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.449+0000 7f713aa00640 1 -- 172.21.15.38:0/953896148 learned_addr learned my addr 172.21.15.38:0/953896148 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:38.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.450+0000 7f7132000640 1 --2- 172.21.15.38:0/953896148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f713c104d70 0x7f713c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:38.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.450+0000 7f713aa00640 1 -- 172.21.15.38:0/953896148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f713c104d70 msgr2=0x7f713c19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:38.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.450+0000 7f713aa00640 1 --2- 172.21.15.38:0/953896148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f713c104d70 0x7f713c19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.450+0000 7f713aa00640 1 -- 172.21.15.38:0/953896148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c009560 con 0x7f713c0677a0 2024-03-24T01:03:38.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.450+0000 7f713aa00640 1 --2- 172.21.15.38:0/953896148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f713c0677a0 0x7f713c19c2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f712c02c220 tx=0x7f712c02cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:38.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.451+0000 7f7133e00640 1 -- 172.21.15.38:0/953896148 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f712c0044a0 con 0x7f713c0677a0 2024-03-24T01:03:38.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.451+0000 7f7133e00640 1 -- 172.21.15.38:0/953896148 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f712c003ab0 con 0x7f713c0677a0 2024-03-24T01:03:38.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.451+0000 7f7133e00640 1 -- 172.21.15.38:0/953896148 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f712c005590 con 0x7f713c0677a0 2024-03-24T01:03:38.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.451+0000 7f713be00640 1 -- 172.21.15.38:0/953896148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c1a0e50 con 0x7f713c0677a0 2024-03-24T01:03:38.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.451+0000 7f713be00640 1 -- 172.21.15.38:0/953896148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c1a13d0 con 0x7f713c0677a0 2024-03-24T01:03:38.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.452+0000 7f713be00640 1 -- 172.21.15.38:0/953896148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713c0ff2b0 con 0x7f713c0677a0 2024-03-24T01:03:38.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.453+0000 7f7133e00640 1 -- 172.21.15.38:0/953896148 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f712c004dd0 con 0x7f713c0677a0 2024-03-24T01:03:38.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.455+0000 7f7133e00640 1 --2- 172.21.15.38:0/953896148 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f712006df20 0x7f71200703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:38.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.455+0000 7f7133e00640 1 -- 172.21.15.38:0/953896148 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f712c0b01c0 con 0x7f713c0677a0 2024-03-24T01:03:38.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.455+0000 7f7132000640 1 --2- 172.21.15.38:0/953896148 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f712006df20 0x7f71200703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:38.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.456+0000 7f7132000640 1 --2- 172.21.15.38:0/953896148 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f712006df20 0x7f71200703e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7124005fd0 tx=0x7f7124005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.464+0000 7f7133e00640 1 -- 172.21.15.38:0/953896148 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f712c079540 con 0x7f713c0677a0 2024-03-24T01:03:38.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.670+0000 7f713be00640 1 -- 172.21.15.38:0/953896148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f713c19d5f0 con 0x7f713c0677a0 2024-03-24T01:03:38.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.673+0000 7f7133e00640 1 -- 172.21.15.38:0/953896148 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16736 (secure 0 0 0) 0x7f712c07cc00 con 0x7f713c0677a0 2024-03-24T01:03:38.673 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:38.674 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":87,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","created":"2024-03-24T00:35:36.968970+0000","modified":"2024-03-24T01:00:46.397340+0000","last_up_change":"2024-03-24T00:59:18.537965+0000","last_in_change":"2024-03-24T00:42:50.726360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T00:41:31.289149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-24T00:44:50.057892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-24T00:44:50.717526+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e5df40af-b5d0-4f89-94b2-59ee2e710bc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6803","nonce":1079856560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6805","nonce":1079856560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6809","nonce":1079856560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6807","nonce":1079856560}]},"public_addr":"172.21.15.38:6803/1079856560","cluster_addr":"172.21.15.38:6805/1079856560","heartbeat_back_addr":"172.21.15.38:6809/1079856560","heartbeat_front_addr":"172.21.15.38:6807/1079856560","state":["exists","up"]},{"osd":1,"uuid":"ac3c0960-dd6c-44f9-b4b0-234c10f838fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6811","nonce":1120765274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6813","nonce":1120765274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6817","nonce":1120765274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6815","nonce":1120765274}]},"public_addr":"172.21.15.38:6811/1120765274","cluster_addr":"172.21.15.38:6813/1120765274","heartbeat_back_addr":"172.21.15.38:6817/1120765274","heartbeat_front_addr":"172.21.15.38:6815/1120765274","state":["exists","up"]},{"osd":2,"uuid":"921d9dd3-2471-4b13-b6bd-01e5b6c78648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6819","nonce":874077504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6821","nonce":874077504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6825","nonce":874077504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6823","nonce":874077504}]},"public_addr":"172.21.15.38:6819/874077504","cluster_addr":"172.21.15.38:6821/874077504","heartbeat_back_addr":"172.21.15.38:6825/874077504","heartbeat_front_addr":"172.21.15.38:6823/874077504","state":["exists","up"]},{"osd":3,"uuid":"7e93813a-0036-4cea-8396-41b8b759c257","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6801","nonce":1863418220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6803","nonce":1863418220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6807","nonce":1863418220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6805","nonce":1863418220}]},"public_addr":"172.21.15.88:6801/1863418220","cluster_addr":"172.21.15.88:6803/1863418220","heartbeat_back_addr":"172.21.15.88:6807/1863418220","heartbeat_front_addr":"172.21.15.88:6805/1863418220","state":["exists","up"]},{"osd":4,"uuid":"b73f29f7-134e-4725-b341-e893b4bbff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6809","nonce":2757972748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6811","nonce":2757972748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6815","nonce":2757972748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6813","nonce":2757972748}]},"public_addr":"172.21.15.88:6809/2757972748","cluster_addr":"172.21.15.88:6811/2757972748","heartbeat_back_addr":"172.21.15.88:6815/2757972748","heartbeat_front_addr":"172.21.15.88:6813/2757972748","state":["exists","up"]},{"osd":5,"uuid":"8ea2cfb3-be7a-4f97-9993-8f9001b8b678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6817","nonce":3428886242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6819","nonce":3428886242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6823","nonce":3428886242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6821","nonce":3428886242}]},"public_addr":"172.21.15.88:6817/3428886242","cluster_addr":"172.21.15.88:6819/3428886242","heartbeat_back_addr":"172.21.15.88:6823/3428886242","heartbeat_front_addr":"172.21.15.88:6821/3428886242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:11.140794+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:49.877898+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:41:28.612279+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:02.338217+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:36.760939+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:43:13.811948+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6827/1446532250":"2024-03-25T00:59:25.362289+0000","172.21.15.38:6826/2539263555":"2024-03-25T00:59:25.362259+0000","172.21.15.38:0/3860887425":"2024-03-25T00:36:13.958115+0000","172.21.15.88:0/610291104":"2024-03-25T00:49:19.114620+0000","172.21.15.38:6827/2539263555":"2024-03-25T00:59:25.362259+0000","172.21.15.38:6800/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.38:6800/839555621":"2024-03-25T00:53:56.535603+0000","172.21.15.88:6828/832884416":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/4049972561":"2024-03-25T00:38:28.565032+0000","172.21.15.88:0/3960582080":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/2868668356":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/4189386253":"2024-03-25T00:47:53.298855+0000","172.21.15.88:0/3753305655":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/913707898":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6800/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/3845097007":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/3448795319":"2024-03-25T00:38:28.565032+0000","172.21.15.38:6801/3107810736":"2024-03-25T00:47:53.298855+0000","172.21.15.88:6824/1531706761":"2024-03-25T00:45:13.640830+0000","172.21.15.38:0/1530344157":"2024-03-25T00:53:56.535603+0000","172.21.15.88:0/3790826752":"2024-03-25T00:49:19.114620+0000","172.21.15.38:6801/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6800/3107810736":"2024-03-25T00:47:53.298855+0000","172.21.15.38:0/1949387173":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6801/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/748221873":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6801/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.88:6829/832884416":"2024-03-25T00:49:19.114620+0000","172.21.15.88:6826/1446532250":"2024-03-25T00:59:25.362289+0000","172.21.15.88:0/1572172803":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/1962732004":"2024-03-25T00:47:53.298855+0000","172.21.15.38:0/3536812262":"2024-03-25T00:47:53.298855+0000","172.21.15.38:6800/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/773042674":"2024-03-25T00:53:56.535603+0000","172.21.15.38:6801/839555621":"2024-03-25T00:53:56.535603+0000","172.21.15.88:6825/1531706761":"2024-03-25T00:45:13.640830+0000","172.21.15.38:0/633787949":"2024-03-25T00:47:53.298855+0000","172.21.15.88:0/448394986":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/884740303":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/2405459521":"2024-03-25T00:53:56.535603+0000","172.21.15.38:0/587822322":"2024-03-25T00:53:56.535603+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T01:03:38.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.676+0000 7f713be00640 1 -- 172.21.15.38:0/953896148 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f712006df20 msgr2=0x7f71200703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:38.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.676+0000 7f713be00640 1 --2- 172.21.15.38:0/953896148 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f712006df20 0x7f71200703e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7124005fd0 tx=0x7f7124005850 comp rx=0 tx=0).stop 2024-03-24T01:03:38.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.676+0000 7f713be00640 1 -- 172.21.15.38:0/953896148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f713c0677a0 msgr2=0x7f713c19c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:38.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.676+0000 7f713be00640 1 --2- 172.21.15.38:0/953896148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f713c0677a0 0x7f713c19c2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f712c02c220 tx=0x7f712c02cea0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.676+0000 7f713be00640 1 -- 172.21.15.38:0/953896148 shutdown_connections 2024-03-24T01:03:38.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.676+0000 7f713be00640 1 --2- 172.21.15.38:0/953896148 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f712006df20 0x7f71200703e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.676+0000 7f713be00640 1 --2- 172.21.15.38:0/953896148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f713c104d70 0x7f713c19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.676+0000 7f713be00640 1 --2- 172.21.15.38:0/953896148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f713c0677a0 0x7f713c19c2f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.676+0000 7f713be00640 1 -- 172.21.15.38:0/953896148 >> 172.21.15.38:0/953896148 conn(0x7f713c0fab80 msgr2=0x7f713c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:38.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.676+0000 7f713be00640 1 -- 172.21.15.38:0/953896148 shutdown_connections 2024-03-24T01:03:38.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.676+0000 7f713be00640 1 -- 172.21.15.38:0/953896148 wait complete. 2024-03-24T01:03:38.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T00:41:31.289149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-24T00:44:50.057892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-24T00:44:50.717526+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T01:03:38.697 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T01:03:38.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:38 smithi038 ceph-mon[112108]: pgmap v330: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:03:38.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:38 smithi088 ceph-mon[87351]: pgmap v330: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:03:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.931+0000 7f559f200640 1 -- 172.21.15.38:0/4206004453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55a00fffc0 msgr2=0x7f55a01003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.931+0000 7f559f200640 1 --2- 172.21.15.38:0/4206004453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55a00fffc0 0x7f55a01003a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5588009880 tx=0x7f558802bce0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.931+0000 7f559f200640 1 -- 172.21.15.38:0/4206004453 shutdown_connections 2024-03-24T01:03:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.931+0000 7f559f200640 1 --2- 172.21.15.38:0/4206004453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55a0100970 0x7f55a01049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.931+0000 7f559f200640 1 --2- 172.21.15.38:0/4206004453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55a00fffc0 0x7f55a01003a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.931+0000 7f559f200640 1 -- 172.21.15.38:0/4206004453 >> 172.21.15.38:0/4206004453 conn(0x7f55a00fb840 msgr2=0x7f55a00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.931+0000 7f559f200640 1 -- 172.21.15.38:0/4206004453 shutdown_connections 2024-03-24T01:03:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.931+0000 7f559f200640 1 -- 172.21.15.38:0/4206004453 wait complete. 2024-03-24T01:03:38.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.932+0000 7f559f200640 1 Processor -- start 2024-03-24T01:03:38.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.933+0000 7f559f200640 1 -- start start 2024-03-24T01:03:38.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.933+0000 7f559f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55a00fffc0 0x7f55a0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:38.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.933+0000 7f559f200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55a01985c0 0x7f55a019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:38.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.933+0000 7f559f200640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a0198bc0 con 0x7f55a01985c0 2024-03-24T01:03:38.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.933+0000 7f559f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a0198d30 con 0x7f55a00fffc0 2024-03-24T01:03:38.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.934+0000 7f559d400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55a01985c0 0x7f55a019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:38.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.934+0000 7f559d400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55a01985c0 0x7f55a019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48448/0 (socket says 172.21.15.38:48448) 2024-03-24T01:03:38.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.934+0000 7f559d400640 1 -- 172.21.15.38:0/1279886356 learned_addr learned my addr 172.21.15.38:0/1279886356 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:38.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.934+0000 7f559de00640 1 --2- 172.21.15.38:0/1279886356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55a00fffc0 0x7f55a0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:38.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.934+0000 7f559d400640 1 -- 172.21.15.38:0/1279886356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55a00fffc0 msgr2=0x7f55a0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:38.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.934+0000 7f559d400640 1 --2- 172.21.15.38:0/1279886356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55a00fffc0 0x7f55a0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:38.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.934+0000 7f559d400640 1 -- 172.21.15.38:0/1279886356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5588009560 con 0x7f55a01985c0 2024-03-24T01:03:38.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.934+0000 7f559d400640 1 --2- 172.21.15.38:0/1279886356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55a01985c0 0x7f55a019c960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f559000d6f0 tx=0x7f559000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:38.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.935+0000 7f5596a00640 1 -- 172.21.15.38:0/1279886356 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55900051f0 con 0x7f55a01985c0 2024-03-24T01:03:38.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.935+0000 7f5596a00640 1 -- 172.21.15.38:0/1279886356 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5590009ce0 con 0x7f55a01985c0 2024-03-24T01:03:38.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.935+0000 7f5596a00640 1 -- 172.21.15.38:0/1279886356 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5590004e80 con 0x7f55a01985c0 2024-03-24T01:03:38.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.935+0000 7f559f200640 1 -- 172.21.15.38:0/1279886356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a019cf60 con 0x7f55a01985c0 2024-03-24T01:03:38.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.935+0000 7f559f200640 1 -- 172.21.15.38:0/1279886356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a019d4c0 con 0x7f55a01985c0 2024-03-24T01:03:38.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.936+0000 7f559f200640 1 -- 172.21.15.38:0/1279886356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a019d790 con 0x7f55a01985c0 2024-03-24T01:03:38.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.938+0000 7f5596a00640 1 -- 172.21.15.38:0/1279886356 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f55900046c0 con 0x7f55a01985c0 2024-03-24T01:03:38.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.939+0000 7f5596a00640 1 --2- 172.21.15.38:0/1279886356 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f558406dec0 0x7f5584070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:38.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.939+0000 7f5596a00640 1 -- 172.21.15.38:0/1279886356 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f559008d8d0 con 0x7f55a01985c0 2024-03-24T01:03:38.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.940+0000 7f559de00640 1 --2- 172.21.15.38:0/1279886356 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f558406dec0 0x7f5584070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:38.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.940+0000 7f559de00640 1 --2- 172.21.15.38:0/1279886356 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f558406dec0 0x7f5584070380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5588000c00 tx=0x7f558802cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:38.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:38.944+0000 7f5596a00640 1 -- 172.21.15.38:0/1279886356 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5590056c50 con 0x7f55a01985c0 2024-03-24T01:03:39.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.184+0000 7f559f200640 1 -- 172.21.15.38:0/1279886356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55a01993e0 con 0x7f55a01985c0 2024-03-24T01:03:39.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.185+0000 7f5596a00640 1 -- 172.21.15.38:0/1279886356 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f559005a310 con 0x7f55a01985c0 2024-03-24T01:03:39.185 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-03-24T01:03:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.188+0000 7f559f200640 1 -- 172.21.15.38:0/1279886356 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f558406dec0 msgr2=0x7f5584070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.188+0000 7f559f200640 1 --2- 172.21.15.38:0/1279886356 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f558406dec0 0x7f5584070380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5588000c00 tx=0x7f558802cf90 comp rx=0 tx=0).stop 2024-03-24T01:03:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.188+0000 7f559f200640 1 -- 172.21.15.38:0/1279886356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55a01985c0 msgr2=0x7f55a019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.188+0000 7f559f200640 1 --2- 172.21.15.38:0/1279886356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55a01985c0 0x7f55a019c960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f559000d6f0 tx=0x7f559000dbb0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.188+0000 7f559f200640 1 -- 172.21.15.38:0/1279886356 shutdown_connections 2024-03-24T01:03:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.188+0000 7f559f200640 1 --2- 172.21.15.38:0/1279886356 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f558406dec0 0x7f5584070380 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.188+0000 7f559f200640 1 --2- 172.21.15.38:0/1279886356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55a01985c0 0x7f55a019c960 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.188+0000 7f559f200640 1 --2- 172.21.15.38:0/1279886356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55a00fffc0 0x7f55a0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.188+0000 7f559f200640 1 -- 172.21.15.38:0/1279886356 >> 172.21.15.38:0/1279886356 conn(0x7f55a00fb840 msgr2=0x7f55a00fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.188+0000 7f559f200640 1 -- 172.21.15.38:0/1279886356 shutdown_connections 2024-03-24T01:03:39.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.188+0000 7f559f200640 1 -- 172.21.15.38:0/1279886356 wait complete. 2024-03-24T01:03:39.210 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-24T01:03:39.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.435+0000 7f2351200640 1 -- 172.21.15.38:0/886865829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f234c1001a0 msgr2=0x7f234c100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:39.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.435+0000 7f2351200640 1 --2- 172.21.15.38:0/886865829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f234c1001a0 0x7f234c100580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f233c009880 tx=0x7f233c02bce0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.435+0000 7f2351200640 1 -- 172.21.15.38:0/886865829 shutdown_connections 2024-03-24T01:03:39.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.435+0000 7f2351200640 1 --2- 172.21.15.38:0/886865829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f234c100b50 0x7f234c104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.435+0000 7f2351200640 1 --2- 172.21.15.38:0/886865829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f234c1001a0 0x7f234c100580 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.435+0000 7f2351200640 1 -- 172.21.15.38:0/886865829 >> 172.21.15.38:0/886865829 conn(0x7f234c0fba20 msgr2=0x7f234c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:39.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.435+0000 7f2351200640 1 -- 172.21.15.38:0/886865829 shutdown_connections 2024-03-24T01:03:39.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.435+0000 7f2351200640 1 -- 172.21.15.38:0/886865829 wait complete. 2024-03-24T01:03:39.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.436+0000 7f2351200640 1 Processor -- start 2024-03-24T01:03:39.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.437+0000 7f2351200640 1 -- start start 2024-03-24T01:03:39.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.437+0000 7f2351200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f234c1001a0 0x7f234c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:39.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.437+0000 7f2351200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f234c198720 0x7f234c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:39.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.437+0000 7f2351200640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c198d20 con 0x7f234c198720 2024-03-24T01:03:39.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.437+0000 7f2351200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c198e90 con 0x7f234c1001a0 2024-03-24T01:03:39.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.437+0000 7f2349400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f234c198720 0x7f234c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:39.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.437+0000 7f2349400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f234c198720 0x7f234c19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48456/0 (socket says 172.21.15.38:48456) 2024-03-24T01:03:39.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.437+0000 7f2349400640 1 -- 172.21.15.38:0/4148247596 learned_addr learned my addr 172.21.15.38:0/4148247596 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:39.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.438+0000 7f2349e00640 1 --2- 172.21.15.38:0/4148247596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f234c1001a0 0x7f234c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:39.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.438+0000 7f2349400640 1 -- 172.21.15.38:0/4148247596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f234c1001a0 msgr2=0x7f234c1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:39.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.438+0000 7f2349400640 1 --2- 172.21.15.38:0/4148247596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f234c1001a0 0x7f234c1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.438+0000 7f2349400640 1 -- 172.21.15.38:0/4148247596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c009560 con 0x7f234c198720 2024-03-24T01:03:39.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.438+0000 7f2349400640 1 --2- 172.21.15.38:0/4148247596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f234c198720 0x7f234c19cac0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f233800e9a0 tx=0x7f233800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:39.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.439+0000 7f2342a00640 1 -- 172.21.15.38:0/4148247596 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f233800cc90 con 0x7f234c198720 2024-03-24T01:03:39.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.439+0000 7f2342a00640 1 -- 172.21.15.38:0/4148247596 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f233800cdf0 con 0x7f234c198720 2024-03-24T01:03:39.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.439+0000 7f2342a00640 1 -- 172.21.15.38:0/4148247596 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2338010430 con 0x7f234c198720 2024-03-24T01:03:39.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.439+0000 7f2351200640 1 -- 172.21.15.38:0/4148247596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234c19d0c0 con 0x7f234c198720 2024-03-24T01:03:39.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.439+0000 7f2351200640 1 -- 172.21.15.38:0/4148247596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234c19d620 con 0x7f234c198720 2024-03-24T01:03:39.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.440+0000 7f2351200640 1 -- 172.21.15.38:0/4148247596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234c101670 con 0x7f234c198720 2024-03-24T01:03:39.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.441+0000 7f2342a00640 1 -- 172.21.15.38:0/4148247596 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f2338005020 con 0x7f234c198720 2024-03-24T01:03:39.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.442+0000 7f2342a00640 1 --2- 172.21.15.38:0/4148247596 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f233006df10 0x7f23300703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:39.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.442+0000 7f2342a00640 1 -- 172.21.15.38:0/4148247596 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2338014070 con 0x7f234c198720 2024-03-24T01:03:39.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.442+0000 7f2349e00640 1 --2- 172.21.15.38:0/4148247596 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f233006df10 0x7f23300703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:39.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.443+0000 7f2349e00640 1 --2- 172.21.15.38:0/4148247596 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f233006df10 0x7f23300703d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f233c000c00 tx=0x7f233c02c820 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:39.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.451+0000 7f2342a00640 1 -- 172.21.15.38:0/4148247596 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2338057720 con 0x7f234c198720 2024-03-24T01:03:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.656+0000 7f2351200640 1 -- 172.21.15.38:0/4148247596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f234c100b50 con 0x7f234c198720 2024-03-24T01:03:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.657+0000 7f2342a00640 1 -- 172.21.15.38:0/4148247596 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f233805ade0 con 0x7f234c198720 2024-03-24T01:03:39.657 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-03-24T01:03:39.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.660+0000 7f2351200640 1 -- 172.21.15.38:0/4148247596 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f233006df10 msgr2=0x7f23300703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:39.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.660+0000 7f2351200640 1 --2- 172.21.15.38:0/4148247596 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f233006df10 0x7f23300703d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f233c000c00 tx=0x7f233c02c820 comp rx=0 tx=0).stop 2024-03-24T01:03:39.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.660+0000 7f2351200640 1 -- 172.21.15.38:0/4148247596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f234c198720 msgr2=0x7f234c19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:39.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.660+0000 7f2351200640 1 --2- 172.21.15.38:0/4148247596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f234c198720 0x7f234c19cac0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f233800e9a0 tx=0x7f233800ee60 comp rx=0 tx=0).stop 2024-03-24T01:03:39.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.660+0000 7f2351200640 1 -- 172.21.15.38:0/4148247596 shutdown_connections 2024-03-24T01:03:39.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.660+0000 7f2351200640 1 --2- 172.21.15.38:0/4148247596 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f233006df10 0x7f23300703d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.660+0000 7f2351200640 1 --2- 172.21.15.38:0/4148247596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f234c198720 0x7f234c19cac0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.660+0000 7f2351200640 1 --2- 172.21.15.38:0/4148247596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f234c1001a0 0x7f234c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.660+0000 7f2351200640 1 -- 172.21.15.38:0/4148247596 >> 172.21.15.38:0/4148247596 conn(0x7f234c0fba20 msgr2=0x7f234c073850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:39.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.660+0000 7f2351200640 1 -- 172.21.15.38:0/4148247596 shutdown_connections 2024-03-24T01:03:39.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.660+0000 7f2351200640 1 -- 172.21.15.38:0/4148247596 wait complete. 2024-03-24T01:03:39.688 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-24T01:03:39.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:39 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/953896148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T01:03:39.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:39 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1279886356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T01:03:39.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:39 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/953896148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T01:03:39.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:39 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1279886356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T01:03:39.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.931+0000 7fd99f200640 1 -- 172.21.15.38:0/292063274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9980fecf0 msgr2=0x7fd9980ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:39.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.931+0000 7fd99f200640 1 --2- 172.21.15.38:0/292063274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9980fecf0 0x7fd9980ff0d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009880 tx=0x7fd98c02bd70 comp rx=0 tx=0).stop 2024-03-24T01:03:39.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.932+0000 7fd99f200640 1 -- 172.21.15.38:0/292063274 shutdown_connections 2024-03-24T01:03:39.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.932+0000 7fd99f200640 1 --2- 172.21.15.38:0/292063274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9980ff610 0x7fd998103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.932+0000 7fd99f200640 1 --2- 172.21.15.38:0/292063274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9980fecf0 0x7fd9980ff0d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.932+0000 7fd99f200640 1 -- 172.21.15.38:0/292063274 >> 172.21.15.38:0/292063274 conn(0x7fd9980758e0 msgr2=0x7fd9980fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:39.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.932+0000 7fd99f200640 1 -- 172.21.15.38:0/292063274 shutdown_connections 2024-03-24T01:03:39.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.932+0000 7fd99f200640 1 -- 172.21.15.38:0/292063274 wait complete. 2024-03-24T01:03:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.933+0000 7fd99f200640 1 Processor -- start 2024-03-24T01:03:39.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.933+0000 7fd99f200640 1 -- start start 2024-03-24T01:03:39.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.934+0000 7fd99f200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9980fecf0 0x7fd99819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:39.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.934+0000 7fd99f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9980ff610 0x7fd99819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:39.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.934+0000 7fd99f200640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99819cf70 con 0x7fd9980fecf0 2024-03-24T01:03:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.934+0000 7fd99f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9981a0c10 con 0x7fd9980ff610 2024-03-24T01:03:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.934+0000 7fd99de00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9980fecf0 0x7fd99819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.934+0000 7fd99de00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9980fecf0 0x7fd99819c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48460/0 (socket says 172.21.15.38:48460) 2024-03-24T01:03:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.934+0000 7fd99de00640 1 -- 172.21.15.38:0/3117412620 learned_addr learned my addr 172.21.15.38:0/3117412620 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.935+0000 7fd99d400640 1 --2- 172.21.15.38:0/3117412620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9980ff610 0x7fd99819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.935+0000 7fd99de00640 1 -- 172.21.15.38:0/3117412620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9980ff610 msgr2=0x7fd99819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.935+0000 7fd99de00640 1 --2- 172.21.15.38:0/3117412620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9980ff610 0x7fd99819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.935+0000 7fd99de00640 1 -- 172.21.15.38:0/3117412620 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c009560 con 0x7fd9980fecf0 2024-03-24T01:03:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.935+0000 7fd99de00640 1 --2- 172.21.15.38:0/3117412620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9980fecf0 0x7fd99819c3a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd98c02c2b0 tx=0x7fd98c02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.935+0000 7fd992a00640 1 -- 172.21.15.38:0/3117412620 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd98c004520 con 0x7fd9980fecf0 2024-03-24T01:03:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.935+0000 7fd992a00640 1 -- 172.21.15.38:0/3117412620 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd98c003ab0 con 0x7fd9980fecf0 2024-03-24T01:03:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.935+0000 7fd99f200640 1 -- 172.21.15.38:0/3117412620 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9981a0e90 con 0x7fd9980fecf0 2024-03-24T01:03:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.935+0000 7fd992a00640 1 -- 172.21.15.38:0/3117412620 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd98c005500 con 0x7fd9980fecf0 2024-03-24T01:03:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.936+0000 7fd99f200640 1 -- 172.21.15.38:0/3117412620 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9981a13f0 con 0x7fd9980fecf0 2024-03-24T01:03:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.937+0000 7fd99f200640 1 -- 172.21.15.38:0/3117412620 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd998067b20 con 0x7fd9980fecf0 2024-03-24T01:03:39.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.938+0000 7fd992a00640 1 -- 172.21.15.38:0/3117412620 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd98c003c20 con 0x7fd9980fecf0 2024-03-24T01:03:39.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.940+0000 7fd992a00640 1 --2- 172.21.15.38:0/3117412620 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd98006df10 0x7fd9800703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:39.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.940+0000 7fd992a00640 1 -- 172.21.15.38:0/3117412620 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd98c0b0930 con 0x7fd9980fecf0 2024-03-24T01:03:39.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.940+0000 7fd99d400640 1 --2- 172.21.15.38:0/3117412620 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd98006df10 0x7fd9800703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:39.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.941+0000 7fd99d400640 1 --2- 172.21.15.38:0/3117412620 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd98006df10 0x7fd9800703d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd9880058c0 tx=0x7fd988005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:39.947+0000 7fd992a00640 1 -- 172.21.15.38:0/3117412620 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd98c079cb0 con 0x7fd9980fecf0 2024-03-24T01:03:40.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.149+0000 7fd99f200640 1 -- 172.21.15.38:0/3117412620 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fd99804e880 con 0x7fd9980fecf0 2024-03-24T01:03:40.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.150+0000 7fd992a00640 1 -- 172.21.15.38:0/3117412620 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd98c07d370 con 0x7fd9980fecf0 2024-03-24T01:03:40.149 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-03-24T01:03:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.153+0000 7fd99f200640 1 -- 172.21.15.38:0/3117412620 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd98006df10 msgr2=0x7fd9800703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.153+0000 7fd99f200640 1 --2- 172.21.15.38:0/3117412620 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd98006df10 0x7fd9800703d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd9880058c0 tx=0x7fd988005850 comp rx=0 tx=0).stop 2024-03-24T01:03:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.153+0000 7fd99f200640 1 -- 172.21.15.38:0/3117412620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9980fecf0 msgr2=0x7fd99819c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.153+0000 7fd99f200640 1 --2- 172.21.15.38:0/3117412620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9980fecf0 0x7fd99819c3a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd98c02c2b0 tx=0x7fd98c02c850 comp rx=0 tx=0).stop 2024-03-24T01:03:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.153+0000 7fd99f200640 1 -- 172.21.15.38:0/3117412620 shutdown_connections 2024-03-24T01:03:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.153+0000 7fd99f200640 1 --2- 172.21.15.38:0/3117412620 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fd98006df10 0x7fd9800703d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.153+0000 7fd99f200640 1 --2- 172.21.15.38:0/3117412620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9980ff610 0x7fd99819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.153+0000 7fd99f200640 1 --2- 172.21.15.38:0/3117412620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9980fecf0 0x7fd99819c3a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.153+0000 7fd99f200640 1 -- 172.21.15.38:0/3117412620 >> 172.21.15.38:0/3117412620 conn(0x7fd9980758e0 msgr2=0x7fd99810d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.153+0000 7fd99f200640 1 -- 172.21.15.38:0/3117412620 shutdown_connections 2024-03-24T01:03:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.153+0000 7fd99f200640 1 -- 172.21.15.38:0/3117412620 wait complete. 2024-03-24T01:03:40.172 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T01:03:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.381+0000 7fc799e00640 1 -- 172.21.15.38:0/1942782753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7940fffc0 msgr2=0x7fc7941003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.381+0000 7fc799e00640 1 --2- 172.21.15.38:0/1942782753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7940fffc0 0x7fc7941003a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc788009880 tx=0x7fc78802bce0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.381+0000 7fc799e00640 1 -- 172.21.15.38:0/1942782753 shutdown_connections 2024-03-24T01:03:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.382+0000 7fc799e00640 1 --2- 172.21.15.38:0/1942782753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc794100970 0x7fc7941049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.382+0000 7fc799e00640 1 --2- 172.21.15.38:0/1942782753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7940fffc0 0x7fc7941003a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.382+0000 7fc799e00640 1 -- 172.21.15.38:0/1942782753 >> 172.21.15.38:0/1942782753 conn(0x7fc7940fb840 msgr2=0x7fc7940fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.382+0000 7fc799e00640 1 -- 172.21.15.38:0/1942782753 shutdown_connections 2024-03-24T01:03:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.382+0000 7fc799e00640 1 -- 172.21.15.38:0/1942782753 wait complete. 2024-03-24T01:03:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.383+0000 7fc799e00640 1 Processor -- start 2024-03-24T01:03:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.383+0000 7fc799e00640 1 -- start start 2024-03-24T01:03:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.384+0000 7fc799e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7940fffc0 0x7fc7940725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.384+0000 7fc799e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc794072b00 0x7fc794072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.384+0000 7fc799e00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794074490 con 0x7fc794072b00 2024-03-24T01:03:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.384+0000 7fc799e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7941a1f90 con 0x7fc7940fffc0 2024-03-24T01:03:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.384+0000 7fc793e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc794072b00 0x7fc794072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.384+0000 7fc793e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc794072b00 0x7fc794072f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48478/0 (socket says 172.21.15.38:48478) 2024-03-24T01:03:40.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.384+0000 7fc793e00640 1 -- 172.21.15.38:0/745990953 learned_addr learned my addr 172.21.15.38:0/745990953 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:40.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.385+0000 7fc798a00640 1 --2- 172.21.15.38:0/745990953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7940fffc0 0x7fc7940725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:40.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.385+0000 7fc793e00640 1 -- 172.21.15.38:0/745990953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7940fffc0 msgr2=0x7fc7940725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:40.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.385+0000 7fc793e00640 1 --2- 172.21.15.38:0/745990953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7940fffc0 0x7fc7940725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.385+0000 7fc793e00640 1 -- 172.21.15.38:0/745990953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc788009560 con 0x7fc794072b00 2024-03-24T01:03:40.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.385+0000 7fc793e00640 1 --2- 172.21.15.38:0/745990953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc794072b00 0x7fc794072f60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc78000c8f0 tx=0x7fc78000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.385+0000 7fc791600640 1 -- 172.21.15.38:0/745990953 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc780004410 con 0x7fc794072b00 2024-03-24T01:03:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.385+0000 7fc799e00640 1 -- 172.21.15.38:0/745990953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7941a2130 con 0x7fc794072b00 2024-03-24T01:03:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.386+0000 7fc791600640 1 -- 172.21.15.38:0/745990953 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc780007c60 con 0x7fc794072b00 2024-03-24T01:03:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.386+0000 7fc791600640 1 -- 172.21.15.38:0/745990953 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc78000f450 con 0x7fc794072b00 2024-03-24T01:03:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.386+0000 7fc799e00640 1 -- 172.21.15.38:0/745990953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7941a2660 con 0x7fc794072b00 2024-03-24T01:03:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.387+0000 7fc799e00640 1 -- 172.21.15.38:0/745990953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc794101490 con 0x7fc794072b00 2024-03-24T01:03:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.388+0000 7fc791600640 1 -- 172.21.15.38:0/745990953 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fc7800077a0 con 0x7fc794072b00 2024-03-24T01:03:40.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.389+0000 7fc791600640 1 --2- 172.21.15.38:0/745990953 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc77c06df20 0x7fc77c0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:40.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.390+0000 7fc791600640 1 -- 172.21.15.38:0/745990953 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc78008dc20 con 0x7fc794072b00 2024-03-24T01:03:40.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.390+0000 7fc798a00640 1 --2- 172.21.15.38:0/745990953 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc77c06df20 0x7fc77c0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:40.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.391+0000 7fc798a00640 1 --2- 172.21.15.38:0/745990953 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc77c06df20 0x7fc77c0703e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc788006010 tx=0x7fc788000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.395+0000 7fc791600640 1 -- 172.21.15.38:0/745990953 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc7800570c0 con 0x7fc794072b00 2024-03-24T01:03:40.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.656+0000 7fc799e00640 1 -- 172.21.15.38:0/745990953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc79404e880 con 0x7fc794072b00 2024-03-24T01:03:40.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.658+0000 7fc791600640 1 -- 172.21.15.38:0/745990953 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5076 (secure 0 0 0) 0x7fc78005a780 con 0x7fc794072b00 2024-03-24T01:03:40.658 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:40.658 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T01:01:41.762046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34446,"mds_1":34478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34446":{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":0,"incarnation":33,"state":"up:active","state_seq":29,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34478":{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":1,"incarnation":33,"state":"up:active","state_seq":22,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44345":{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44373":{"gid":44373,"name":"cephfs.smithi088.yhksbr","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6827/1135457323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1135457323},{"type":"v1","addr":"172.21.15.88:6827","nonce":1135457323}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34446,"qdb_cluster":[34446,34478]},"id":1}]} 2024-03-24T01:03:40.658 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 40 2024-03-24T01:03:40.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.660+0000 7fc799e00640 1 -- 172.21.15.38:0/745990953 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc77c06df20 msgr2=0x7fc77c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:40.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.660+0000 7fc799e00640 1 --2- 172.21.15.38:0/745990953 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc77c06df20 0x7fc77c0703e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc788006010 tx=0x7fc788000bc0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.660+0000 7fc799e00640 1 -- 172.21.15.38:0/745990953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc794072b00 msgr2=0x7fc794072f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:40.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.660+0000 7fc799e00640 1 --2- 172.21.15.38:0/745990953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc794072b00 0x7fc794072f60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc78000c8f0 tx=0x7fc78000cdb0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.661+0000 7fc799e00640 1 -- 172.21.15.38:0/745990953 shutdown_connections 2024-03-24T01:03:40.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.661+0000 7fc799e00640 1 --2- 172.21.15.38:0/745990953 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc77c06df20 0x7fc77c0703e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.661+0000 7fc799e00640 1 --2- 172.21.15.38:0/745990953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc794072b00 0x7fc794072f60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.661+0000 7fc799e00640 1 --2- 172.21.15.38:0/745990953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7940fffc0 0x7fc7940725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.661+0000 7fc799e00640 1 -- 172.21.15.38:0/745990953 >> 172.21.15.38:0/745990953 conn(0x7fc7940fb840 msgr2=0x7fc7940fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:40.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.661+0000 7fc799e00640 1 -- 172.21.15.38:0/745990953 shutdown_connections 2024-03-24T01:03:40.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.661+0000 7fc799e00640 1 -- 172.21.15.38:0/745990953 wait complete. 2024-03-24T01:03:40.686 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 20, 'max_mds': 2, 'flags': 50} 2024-03-24T01:03:40.687 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-24T01:03:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:40 smithi038 ceph-mon[112108]: pgmap v331: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:03:40.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:40 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/4148247596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T01:03:40.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:40 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3117412620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T01:03:40.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.899+0000 7fe70aa00640 1 -- 172.21.15.38:0/1780453815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe704100970 msgr2=0x7fe7041049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:40.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.899+0000 7fe70aa00640 1 --2- 172.21.15.38:0/1780453815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe704100970 0x7fe7041049b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009880 tx=0x7fe6f002bcf0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.899+0000 7fe70aa00640 1 -- 172.21.15.38:0/1780453815 shutdown_connections 2024-03-24T01:03:40.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.899+0000 7fe70aa00640 1 --2- 172.21.15.38:0/1780453815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe704100970 0x7fe7041049b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.899+0000 7fe70aa00640 1 --2- 172.21.15.38:0/1780453815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7040fffc0 0x7fe7041003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.899+0000 7fe70aa00640 1 -- 172.21.15.38:0/1780453815 >> 172.21.15.38:0/1780453815 conn(0x7fe7040fb840 msgr2=0x7fe7040fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:40.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.899+0000 7fe70aa00640 1 -- 172.21.15.38:0/1780453815 shutdown_connections 2024-03-24T01:03:40.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.899+0000 7fe70aa00640 1 -- 172.21.15.38:0/1780453815 wait complete. 2024-03-24T01:03:40.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.900+0000 7fe70aa00640 1 Processor -- start 2024-03-24T01:03:40.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.900+0000 7fe70aa00640 1 -- start start 2024-03-24T01:03:40.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.901+0000 7fe70aa00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7040fffc0 0x7fe7040725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:40.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.901+0000 7fe70aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe704072b00 0x7fe704072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:40.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.901+0000 7fe70aa00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe704074490 con 0x7fe7040fffc0 2024-03-24T01:03:40.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.901+0000 7fe70aa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7041a1fe0 con 0x7fe704072b00 2024-03-24T01:03:40.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.901+0000 7fe709600640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7040fffc0 0x7fe7040725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:40.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.901+0000 7fe709600640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7040fffc0 0x7fe7040725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48500/0 (socket says 172.21.15.38:48500) 2024-03-24T01:03:40.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.901+0000 7fe709600640 1 -- 172.21.15.38:0/2630221959 learned_addr learned my addr 172.21.15.38:0/2630221959 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:40.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.902+0000 7fe708c00640 1 --2- 172.21.15.38:0/2630221959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe704072b00 0x7fe704072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:40.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.902+0000 7fe709600640 1 -- 172.21.15.38:0/2630221959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe704072b00 msgr2=0x7fe704072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:40.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.902+0000 7fe709600640 1 --2- 172.21.15.38:0/2630221959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe704072b00 0x7fe704072f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:40.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.902+0000 7fe709600640 1 -- 172.21.15.38:0/2630221959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f0009560 con 0x7fe7040fffc0 2024-03-24T01:03:40.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.902+0000 7fe709600640 1 --2- 172.21.15.38:0/2630221959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7040fffc0 0x7fe7040725c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800c8a0 tx=0x7fe6f800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:40.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.902+0000 7fe702000640 1 -- 172.21.15.38:0/2630221959 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6f8007b80 con 0x7fe7040fffc0 2024-03-24T01:03:40.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.902+0000 7fe702000640 1 -- 172.21.15.38:0/2630221959 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6f8007ce0 con 0x7fe7040fffc0 2024-03-24T01:03:40.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.902+0000 7fe702000640 1 -- 172.21.15.38:0/2630221959 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6f8018810 con 0x7fe7040fffc0 2024-03-24T01:03:40.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.902+0000 7fe70aa00640 1 -- 172.21.15.38:0/2630221959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7041a2180 con 0x7fe7040fffc0 2024-03-24T01:03:40.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.902+0000 7fe70aa00640 1 -- 172.21.15.38:0/2630221959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7041a26b0 con 0x7fe7040fffc0 2024-03-24T01:03:40.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.904+0000 7fe70aa00640 1 -- 172.21.15.38:0/2630221959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe704101490 con 0x7fe7040fffc0 2024-03-24T01:03:40.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.906+0000 7fe702000640 1 -- 172.21.15.38:0/2630221959 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fe6f8018a80 con 0x7fe7040fffc0 2024-03-24T01:03:40.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.907+0000 7fe702000640 1 --2- 172.21.15.38:0/2630221959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe6ec06df20 0x7fe6ec0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:40.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.907+0000 7fe702000640 1 -- 172.21.15.38:0/2630221959 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe6f808dff0 con 0x7fe7040fffc0 2024-03-24T01:03:40.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.907+0000 7fe708c00640 1 --2- 172.21.15.38:0/2630221959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe6ec06df20 0x7fe6ec0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:40.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.908+0000 7fe708c00640 1 --2- 172.21.15.38:0/2630221959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe6ec06df20 0x7fe6ec0703e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0036040 tx=0x7fe6f0000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:40.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:40.912+0000 7fe702000640 1 -- 172.21.15.38:0/2630221959 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe6f8057480 con 0x7fe7040fffc0 2024-03-24T01:03:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:40 smithi088 ceph-mon[87351]: pgmap v331: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:03:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:40 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/4148247596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T01:03:40.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:40 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3117412620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T01:03:41.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.168+0000 7fe70aa00640 1 -- 172.21.15.38:0/2630221959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fe704108320 con 0x7fe7040fffc0 2024-03-24T01:03:41.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.170+0000 7fe702000640 1 -- 172.21.15.38:0/2630221959 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v40) v1 ==== 107+0+3331 (secure 0 0 0) 0x7fe6f805ab40 con 0x7fe7040fffc0 2024-03-24T01:03:41.170 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:41.170 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:59:25.039886+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14462,"mds_1":24271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14462":{"gid":14462,"name":"cephfs.smithi038.yhkmsb","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.38:6827/2539263555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2539263555},{"type":"v1","addr":"172.21.15.38:6827","nonce":2539263555}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24271":{"gid":24271,"name":"cephfs.smithi088.yhksbr","rank":1,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6827/1446532250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1446532250},{"type":"v1","addr":"172.21.15.88:6827","nonce":1446532250}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14462,"qdb_cluster":[14462,24271]},"id":1}]} 2024-03-24T01:03:41.170 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 21 2024-03-24T01:03:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.173+0000 7fe70aa00640 1 -- 172.21.15.38:0/2630221959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe6ec06df20 msgr2=0x7fe6ec0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.173+0000 7fe70aa00640 1 --2- 172.21.15.38:0/2630221959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe6ec06df20 0x7fe6ec0703e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0036040 tx=0x7fe6f0000bc0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.173+0000 7fe70aa00640 1 -- 172.21.15.38:0/2630221959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7040fffc0 msgr2=0x7fe7040725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.173+0000 7fe70aa00640 1 --2- 172.21.15.38:0/2630221959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7040fffc0 0x7fe7040725c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800c8a0 tx=0x7fe6f800cd60 comp rx=0 tx=0).stop 2024-03-24T01:03:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.173+0000 7fe70aa00640 1 -- 172.21.15.38:0/2630221959 shutdown_connections 2024-03-24T01:03:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.173+0000 7fe70aa00640 1 --2- 172.21.15.38:0/2630221959 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe6ec06df20 0x7fe6ec0703e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.173+0000 7fe70aa00640 1 --2- 172.21.15.38:0/2630221959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe704072b00 0x7fe704072f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.173+0000 7fe70aa00640 1 --2- 172.21.15.38:0/2630221959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7040fffc0 0x7fe7040725c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.173+0000 7fe70aa00640 1 -- 172.21.15.38:0/2630221959 >> 172.21.15.38:0/2630221959 conn(0x7fe7040fb840 msgr2=0x7fe7040fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.173+0000 7fe70aa00640 1 -- 172.21.15.38:0/2630221959 shutdown_connections 2024-03-24T01:03:41.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.173+0000 7fe70aa00640 1 -- 172.21.15.38:0/2630221959 wait complete. 2024-03-24T01:03:41.192 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-03-24T01:03:41.192 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-24T01:03:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.445+0000 7feff6a00640 1 -- 172.21.15.38:0/1359229397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff00ffff0 msgr2=0x7feff01003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.445+0000 7feff6a00640 1 --2- 172.21.15.38:0/1359229397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff00ffff0 0x7feff01003d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009880 tx=0x7fefe402bcf0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.446+0000 7feff6a00640 1 -- 172.21.15.38:0/1359229397 shutdown_connections 2024-03-24T01:03:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.446+0000 7feff6a00640 1 --2- 172.21.15.38:0/1359229397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff01009a0 0x7feff01049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.446+0000 7feff6a00640 1 --2- 172.21.15.38:0/1359229397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff00ffff0 0x7feff01003d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.446+0000 7feff6a00640 1 -- 172.21.15.38:0/1359229397 >> 172.21.15.38:0/1359229397 conn(0x7feff00fb8b0 msgr2=0x7feff00fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.446+0000 7feff6a00640 1 -- 172.21.15.38:0/1359229397 shutdown_connections 2024-03-24T01:03:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.446+0000 7feff6a00640 1 -- 172.21.15.38:0/1359229397 wait complete. 2024-03-24T01:03:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.447+0000 7feff6a00640 1 Processor -- start 2024-03-24T01:03:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.447+0000 7feff6a00640 1 -- start start 2024-03-24T01:03:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.447+0000 7feff6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00ffff0 0x7feff01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.447+0000 7feff6a00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff0198610 0x7feff019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.447+0000 7feff6a00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff0198b80 con 0x7feff0198610 2024-03-24T01:03:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.447+0000 7feff6a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff0198cf0 con 0x7feff00ffff0 2024-03-24T01:03:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.447+0000 7feff4c00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff0198610 0x7feff019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.448+0000 7feff4c00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff0198610 0x7feff019c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48520/0 (socket says 172.21.15.38:48520) 2024-03-24T01:03:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.448+0000 7feff4c00640 1 -- 172.21.15.38:0/3548706394 learned_addr learned my addr 172.21.15.38:0/3548706394 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.448+0000 7feff5600640 1 --2- 172.21.15.38:0/3548706394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00ffff0 0x7feff01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.448+0000 7feff4c00640 1 -- 172.21.15.38:0/3548706394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00ffff0 msgr2=0x7feff01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.448+0000 7feff4c00640 1 --2- 172.21.15.38:0/3548706394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00ffff0 0x7feff01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.448+0000 7feff4c00640 1 -- 172.21.15.38:0/3548706394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe4009560 con 0x7feff0198610 2024-03-24T01:03:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.448+0000 7feff4c00640 1 --2- 172.21.15.38:0/3548706394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff0198610 0x7feff019c9b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00c8f0 tx=0x7fefdc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.449+0000 7fefee000640 1 -- 172.21.15.38:0/3548706394 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefdc004410 con 0x7feff0198610 2024-03-24T01:03:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.449+0000 7fefee000640 1 -- 172.21.15.38:0/3548706394 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefdc007c60 con 0x7feff0198610 2024-03-24T01:03:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.449+0000 7fefee000640 1 -- 172.21.15.38:0/3548706394 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefdc00f450 con 0x7feff0198610 2024-03-24T01:03:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.449+0000 7feff6a00640 1 -- 172.21.15.38:0/3548706394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff019cf50 con 0x7feff0198610 2024-03-24T01:03:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.449+0000 7feff6a00640 1 -- 172.21.15.38:0/3548706394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff019d460 con 0x7feff0198610 2024-03-24T01:03:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.450+0000 7feff6a00640 1 -- 172.21.15.38:0/3548706394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff01014c0 con 0x7feff0198610 2024-03-24T01:03:41.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.453+0000 7fefee000640 1 -- 172.21.15.38:0/3548706394 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fefdc0077a0 con 0x7feff0198610 2024-03-24T01:03:41.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.454+0000 7fefee000640 1 --2- 172.21.15.38:0/3548706394 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fefd806df10 0x7fefd80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:41.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.454+0000 7fefee000640 1 -- 172.21.15.38:0/3548706394 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fefdc08de70 con 0x7feff0198610 2024-03-24T01:03:41.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.454+0000 7feff5600640 1 --2- 172.21.15.38:0/3548706394 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fefd806df10 0x7fefd80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:41.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.455+0000 7feff5600640 1 --2- 172.21.15.38:0/3548706394 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fefd806df10 0x7fefd80703d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fefe4000c00 tx=0x7fefe402ca70 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:41.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.463+0000 7fefee000640 1 -- 172.21.15.38:0/3548706394 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fefdc0571f0 con 0x7feff0198610 2024-03-24T01:03:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:41 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/745990953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T01:03:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:41 smithi038 ceph-mon[112108]: pgmap v332: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:41.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:41 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2630221959' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-24T01:03:41.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.883+0000 7feff6a00640 1 -- 172.21.15.38:0/3548706394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7feff01993d0 con 0x7feff0198610 2024-03-24T01:03:41.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.884+0000 7fefee000640 1 -- 172.21.15.38:0/3548706394 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v40) v1 ==== 107+0+3295 (secure 0 0 0) 0x7fefdc014cf0 con 0x7feff0198610 2024-03-24T01:03:41.884 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:41.884 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi038.xnpqhp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/2851856830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2851856830},{"type":"v1","addr":"172.21.15.38:6829","nonce":2851856830}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44293,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2738193611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2738193611},{"type":"v1","addr":"172.21.15.88:6825","nonce":2738193611}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:59:25.362766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:41.884 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 22 2024-03-24T01:03:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.887+0000 7feff6a00640 1 -- 172.21.15.38:0/3548706394 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fefd806df10 msgr2=0x7fefd80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.887+0000 7feff6a00640 1 --2- 172.21.15.38:0/3548706394 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fefd806df10 0x7fefd80703d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fefe4000c00 tx=0x7fefe402ca70 comp rx=0 tx=0).stop 2024-03-24T01:03:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.887+0000 7feff6a00640 1 -- 172.21.15.38:0/3548706394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff0198610 msgr2=0x7feff019c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.887+0000 7feff6a00640 1 --2- 172.21.15.38:0/3548706394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff0198610 0x7feff019c9b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00c8f0 tx=0x7fefdc00cdb0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.888+0000 7feff6a00640 1 -- 172.21.15.38:0/3548706394 shutdown_connections 2024-03-24T01:03:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.888+0000 7feff6a00640 1 --2- 172.21.15.38:0/3548706394 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fefd806df10 0x7fefd80703d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.888+0000 7feff6a00640 1 --2- 172.21.15.38:0/3548706394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feff0198610 0x7feff019c9b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.888+0000 7feff6a00640 1 --2- 172.21.15.38:0/3548706394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feff00ffff0 0x7feff01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.888+0000 7feff6a00640 1 -- 172.21.15.38:0/3548706394 >> 172.21.15.38:0/3548706394 conn(0x7feff00fb8b0 msgr2=0x7feff0073960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.888+0000 7feff6a00640 1 -- 172.21.15.38:0/3548706394 shutdown_connections 2024-03-24T01:03:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:41.888+0000 7feff6a00640 1 -- 172.21.15.38:0/3548706394 wait complete. 2024-03-24T01:03:41.908 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-24T01:03:41.908 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-03-24T01:03:41.908 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-24T01:03:41.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:41 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/745990953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T01:03:41.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:41 smithi088 ceph-mon[87351]: pgmap v332: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:41.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:41 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2630221959' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-24T01:03:42.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.117+0000 7f77df000640 1 -- 172.21.15.38:0/2767162664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77d80feed0 msgr2=0x7f77d80ff2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:42.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.117+0000 7f77df000640 1 --2- 172.21.15.38:0/2767162664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77d80feed0 0x7f77d80ff2b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009c20 tx=0x7f77c802c230 comp rx=0 tx=0).stop 2024-03-24T01:03:42.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.117+0000 7f77df000640 1 -- 172.21.15.38:0/2767162664 shutdown_connections 2024-03-24T01:03:42.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.117+0000 7f77df000640 1 --2- 172.21.15.38:0/2767162664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d80ff7f0 0x7f77d8103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.117+0000 7f77df000640 1 --2- 172.21.15.38:0/2767162664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77d80feed0 0x7f77d80ff2b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.117+0000 7f77df000640 1 -- 172.21.15.38:0/2767162664 >> 172.21.15.38:0/2767162664 conn(0x7f77d80fa8e0 msgr2=0x7f77d80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:42.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.117+0000 7f77df000640 1 -- 172.21.15.38:0/2767162664 shutdown_connections 2024-03-24T01:03:42.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.117+0000 7f77df000640 1 -- 172.21.15.38:0/2767162664 wait complete. 2024-03-24T01:03:42.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.119+0000 7f77df000640 1 Processor -- start 2024-03-24T01:03:42.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.119+0000 7f77df000640 1 -- start start 2024-03-24T01:03:42.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.119+0000 7f77df000640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77d80feed0 0x7f77d819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:42.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.119+0000 7f77df000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d80ff7f0 0x7f77d819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:42.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.119+0000 7f77df000640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d819d290 con 0x7f77d80feed0 2024-03-24T01:03:42.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.119+0000 7f77df000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81a0fa0 con 0x7f77d80ff7f0 2024-03-24T01:03:42.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.120+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77d80feed0 0x7f77d819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:42.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.120+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77d80feed0 0x7f77d819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48538/0 (socket says 172.21.15.38:48538) 2024-03-24T01:03:42.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.120+0000 7f77d7e00640 1 -- 172.21.15.38:0/3666271589 learned_addr learned my addr 172.21.15.38:0/3666271589 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:42.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.120+0000 7f77d7e00640 1 -- 172.21.15.38:0/3666271589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d80ff7f0 msgr2=0x7f77d819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:42.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.120+0000 7f77d7e00640 1 --2- 172.21.15.38:0/3666271589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d80ff7f0 0x7f77d819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.120+0000 7f77d7e00640 1 -- 172.21.15.38:0/3666271589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c8009900 con 0x7f77d80feed0 2024-03-24T01:03:42.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.121+0000 7f77d7e00640 1 --2- 172.21.15.38:0/3666271589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77d80feed0 0x7f77d819c6c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f77c8037040 tx=0x7f77c8034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:42.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.121+0000 7f77d4c00640 1 -- 172.21.15.38:0/3666271589 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77c80326b0 con 0x7f77d80feed0 2024-03-24T01:03:42.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.121+0000 7f77d4c00640 1 -- 172.21.15.38:0/3666271589 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77c8032e60 con 0x7f77d80feed0 2024-03-24T01:03:42.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.121+0000 7f77df000640 1 -- 172.21.15.38:0/3666271589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d81a1220 con 0x7f77d80feed0 2024-03-24T01:03:42.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.121+0000 7f77d4c00640 1 -- 172.21.15.38:0/3666271589 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77c8004890 con 0x7f77d80feed0 2024-03-24T01:03:42.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.121+0000 7f77df000640 1 -- 172.21.15.38:0/3666271589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d81a1720 con 0x7f77d80feed0 2024-03-24T01:03:42.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.123+0000 7f77df000640 1 -- 172.21.15.38:0/3666271589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d8068d00 con 0x7f77d80feed0 2024-03-24T01:03:42.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.123+0000 7f77d4c00640 1 -- 172.21.15.38:0/3666271589 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f77c803b040 con 0x7f77d80feed0 2024-03-24T01:03:42.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.125+0000 7f77d4c00640 1 --2- 172.21.15.38:0/3666271589 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f77bc06df10 0x7f77bc0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:42.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.125+0000 7f77d4c00640 1 -- 172.21.15.38:0/3666271589 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f77c80b0a70 con 0x7f77d80feed0 2024-03-24T01:03:42.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.125+0000 7f77d7400640 1 --2- 172.21.15.38:0/3666271589 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f77bc06df10 0x7f77bc0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:42.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.126+0000 7f77d7400640 1 --2- 172.21.15.38:0/3666271589 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f77bc06df10 0x7f77bc0703d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f77d819dc10 tx=0x7f77c4006d30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.130+0000 7f77d4c00640 1 -- 172.21.15.38:0/3666271589 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f77c8079e70 con 0x7f77d80feed0 2024-03-24T01:03:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.379+0000 7f77df000640 1 -- 172.21.15.38:0/3666271589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f77d804e030 con 0x7f77d80feed0 2024-03-24T01:03:42.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.381+0000 7f77d4c00640 1 -- 172.21.15.38:0/3666271589 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v40) v1 ==== 107+0+4850 (secure 0 0 0) 0x7f77c807d530 con 0x7f77d80feed0 2024-03-24T01:03:42.381 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:42.381 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi038.xnpqhp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/2851856830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2851856830},{"type":"v1","addr":"172.21.15.38:6829","nonce":2851856830}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":34440,"name":"cephfs.smithi038.yhkmsb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/3718992353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3718992353},{"type":"v1","addr":"172.21.15.38:6827","nonce":3718992353}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44293,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2738193611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2738193611},{"type":"v1","addr":"172.21.15.88:6825","nonce":2738193611}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44303,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/983744224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":983744224},{"type":"v1","addr":"172.21.15.88:6827","nonce":983744224}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:59:25.362766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:42.382 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 23 2024-03-24T01:03:42.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.384+0000 7f77df000640 1 -- 172.21.15.38:0/3666271589 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f77bc06df10 msgr2=0x7f77bc0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:42.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.384+0000 7f77df000640 1 --2- 172.21.15.38:0/3666271589 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f77bc06df10 0x7f77bc0703d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f77d819dc10 tx=0x7f77c4006d30 comp rx=0 tx=0).stop 2024-03-24T01:03:42.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.384+0000 7f77df000640 1 -- 172.21.15.38:0/3666271589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77d80feed0 msgr2=0x7f77d819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:42.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.384+0000 7f77df000640 1 --2- 172.21.15.38:0/3666271589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77d80feed0 0x7f77d819c6c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f77c8037040 tx=0x7f77c8034670 comp rx=0 tx=0).stop 2024-03-24T01:03:42.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.384+0000 7f77df000640 1 -- 172.21.15.38:0/3666271589 shutdown_connections 2024-03-24T01:03:42.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.384+0000 7f77df000640 1 --2- 172.21.15.38:0/3666271589 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f77bc06df10 0x7f77bc0703d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.384+0000 7f77df000640 1 --2- 172.21.15.38:0/3666271589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d80ff7f0 0x7f77d819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.384+0000 7f77df000640 1 --2- 172.21.15.38:0/3666271589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77d80feed0 0x7f77d819c6c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.384+0000 7f77df000640 1 -- 172.21.15.38:0/3666271589 >> 172.21.15.38:0/3666271589 conn(0x7f77d80fa8e0 msgr2=0x7f77d810da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:42.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.384+0000 7f77df000640 1 -- 172.21.15.38:0/3666271589 shutdown_connections 2024-03-24T01:03:42.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.384+0000 7f77df000640 1 -- 172.21.15.38:0/3666271589 wait complete. 2024-03-24T01:03:42.412 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-24T01:03:42.412 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-03-24T01:03:42.412 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-24T01:03:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:42 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:03:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:42 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3548706394' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-24T01:03:42.609 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:42 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3666271589' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-24T01:03:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.636+0000 7f16ff200640 1 -- 172.21.15.38:0/4233747421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17000ff4c0 msgr2=0x7f170010c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.636+0000 7f16ff200640 1 --2- 172.21.15.38:0/4233747421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17000ff4c0 0x7f170010c0f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009880 tx=0x7f16e802bd30 comp rx=0 tx=0).stop 2024-03-24T01:03:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.636+0000 7f16ff200640 1 -- 172.21.15.38:0/4233747421 shutdown_connections 2024-03-24T01:03:42.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.636+0000 7f16ff200640 1 --2- 172.21.15.38:0/4233747421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17000ff4c0 0x7f170010c0f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.636+0000 7f16ff200640 1 --2- 172.21.15.38:0/4233747421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17000feb10 0x7f17000feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.636+0000 7f16ff200640 1 -- 172.21.15.38:0/4233747421 >> 172.21.15.38:0/4233747421 conn(0x7f17000758e0 msgr2=0x7f17000fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.636+0000 7f16ff200640 1 -- 172.21.15.38:0/4233747421 shutdown_connections 2024-03-24T01:03:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.636+0000 7f16ff200640 1 -- 172.21.15.38:0/4233747421 wait complete. 2024-03-24T01:03:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.637+0000 7f16ff200640 1 Processor -- start 2024-03-24T01:03:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.637+0000 7f16ff200640 1 -- start start 2024-03-24T01:03:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.638+0000 7f16ff200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17000feb10 0x7f1700197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.638+0000 7f16ff200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17000ff4c0 0x7f1700198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.638+0000 7f16ff200640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1700198ad0 con 0x7f17000ff4c0 2024-03-24T01:03:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.638+0000 7f16ff200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170019c7e0 con 0x7f17000feb10 2024-03-24T01:03:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.638+0000 7f16f5400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17000ff4c0 0x7f1700198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.638+0000 7f16f5400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17000ff4c0 0x7f1700198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48564/0 (socket says 172.21.15.38:48564) 2024-03-24T01:03:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.638+0000 7f16f5400640 1 -- 172.21.15.38:0/246262047 learned_addr learned my addr 172.21.15.38:0/246262047 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.639+0000 7f16fde00640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17000feb10 0x7f1700197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.639+0000 7f16f5400640 1 -- 172.21.15.38:0/246262047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17000feb10 msgr2=0x7f1700197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.639+0000 7f16f5400640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17000feb10 0x7f1700197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.639+0000 7f16f5400640 1 -- 172.21.15.38:0/246262047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e8009560 con 0x7f17000ff4c0 2024-03-24T01:03:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.639+0000 7f16fde00640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17000feb10 0x7f1700197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T01:03:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.639+0000 7f16f5400640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17000ff4c0 0x7f1700198440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f16e8006010 tx=0x7f16e802c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.640+0000 7f16f7400640 1 -- 172.21.15.38:0/246262047 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16e80035f0 con 0x7f17000ff4c0 2024-03-24T01:03:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.640+0000 7f16f7400640 1 -- 172.21.15.38:0/246262047 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16e8003c10 con 0x7f17000ff4c0 2024-03-24T01:03:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.640+0000 7f16ff200640 1 -- 172.21.15.38:0/246262047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170019cac0 con 0x7f17000ff4c0 2024-03-24T01:03:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.640+0000 7f16f7400640 1 -- 172.21.15.38:0/246262047 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16e8005670 con 0x7f17000ff4c0 2024-03-24T01:03:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.640+0000 7f16ff200640 1 -- 172.21.15.38:0/246262047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170019d020 con 0x7f17000ff4c0 2024-03-24T01:03:42.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.641+0000 7f16ff200640 1 -- 172.21.15.38:0/246262047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1700067b20 con 0x7f17000ff4c0 2024-03-24T01:03:42.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.642+0000 7f16f7400640 1 -- 172.21.15.38:0/246262047 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f16e8004ed0 con 0x7f17000ff4c0 2024-03-24T01:03:42.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.643+0000 7f16f7400640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16e406dec0 0x7f16e4070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:42.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.643+0000 7f16f7400640 1 -- 172.21.15.38:0/246262047 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f16e80b0ab0 con 0x7f17000ff4c0 2024-03-24T01:03:42.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.644+0000 7f16fde00640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16e406dec0 0x7f16e4070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:42.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.644+0000 7f16fde00640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16e406dec0 0x7f16e4070380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16f0005c40 tx=0x7f16f000a430 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:42.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.653+0000 7f16f7400640 1 -- 172.21.15.38:0/246262047 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f16e8079f80 con 0x7f17000ff4c0 2024-03-24T01:03:42.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:42 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:03:42.833 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:42 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3548706394' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-24T01:03:42.833 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:42 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3666271589' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-24T01:03:42.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.907+0000 7f16ff200640 1 -- 172.21.15.38:0/246262047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f170004e880 con 0x7f17000ff4c0 2024-03-24T01:03:42.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.908+0000 7f16f7400640 1 -- 172.21.15.38:0/246262047 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v40) v1 ==== 107+0+4071 (secure 0 0 0) 0x7f16e8003d80 con 0x7f17000ff4c0 2024-03-24T01:03:42.908 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:42.908 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi038.xnpqhp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/2851856830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2851856830},{"type":"v1","addr":"172.21.15.38:6829","nonce":2851856830}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44293,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2738193611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2738193611},{"type":"v1","addr":"172.21.15.88:6825","nonce":2738193611}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44303,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/983744224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":983744224},{"type":"v1","addr":"172.21.15.88:6827","nonce":983744224}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:59:25.362766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:42.908 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 24 2024-03-24T01:03:42.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.910+0000 7f16ff200640 1 -- 172.21.15.38:0/246262047 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16e406dec0 msgr2=0x7f16e4070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:42.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.910+0000 7f16ff200640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16e406dec0 0x7f16e4070380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16f0005c40 tx=0x7f16f000a430 comp rx=0 tx=0).stop 2024-03-24T01:03:42.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.910+0000 7f16ff200640 1 -- 172.21.15.38:0/246262047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17000ff4c0 msgr2=0x7f1700198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:42.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.910+0000 7f16ff200640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17000ff4c0 0x7f1700198440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f16e8006010 tx=0x7f16e802c840 comp rx=0 tx=0).stop 2024-03-24T01:03:42.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.910+0000 7f16ff200640 1 -- 172.21.15.38:0/246262047 shutdown_connections 2024-03-24T01:03:42.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.910+0000 7f16ff200640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f16e406dec0 0x7f16e4070380 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.910+0000 7f16ff200640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17000ff4c0 0x7f1700198440 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.910+0000 7f16ff200640 1 --2- 172.21.15.38:0/246262047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17000feb10 0x7f1700197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:42.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.910+0000 7f16ff200640 1 -- 172.21.15.38:0/246262047 >> 172.21.15.38:0/246262047 conn(0x7f17000758e0 msgr2=0x7f17000fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:42.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.911+0000 7f16ff200640 1 -- 172.21.15.38:0/246262047 shutdown_connections 2024-03-24T01:03:42.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:42.911+0000 7f16ff200640 1 -- 172.21.15.38:0/246262047 wait complete. 2024-03-24T01:03:42.931 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-24T01:03:42.931 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-03-24T01:03:42.931 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-24T01:03:43.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.158+0000 7fbfae800640 1 -- 172.21.15.38:0/3551005226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfa8100980 msgr2=0x7fbfa81049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:43.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.158+0000 7fbfae800640 1 --2- 172.21.15.38:0/3551005226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfa8100980 0x7fbfa81049c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009880 tx=0x7fbf9402bcf0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.158+0000 7fbfae800640 1 -- 172.21.15.38:0/3551005226 shutdown_connections 2024-03-24T01:03:43.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.158+0000 7fbfae800640 1 --2- 172.21.15.38:0/3551005226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfa8100980 0x7fbfa81049c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.158+0000 7fbfae800640 1 --2- 172.21.15.38:0/3551005226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa80fffd0 0x7fbfa81003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.158+0000 7fbfae800640 1 -- 172.21.15.38:0/3551005226 >> 172.21.15.38:0/3551005226 conn(0x7fbfa80fb850 msgr2=0x7fbfa80fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.158+0000 7fbfae800640 1 -- 172.21.15.38:0/3551005226 shutdown_connections 2024-03-24T01:03:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.158+0000 7fbfae800640 1 -- 172.21.15.38:0/3551005226 wait complete. 2024-03-24T01:03:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.159+0000 7fbfae800640 1 Processor -- start 2024-03-24T01:03:43.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.160+0000 7fbfae800640 1 -- start start 2024-03-24T01:03:43.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.160+0000 7fbfae800640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfa80fffd0 0x7fbfa8195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:43.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.160+0000 7fbfae800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa8196350 0x7fbfa819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:43.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.160+0000 7fbfae800640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa8196950 con 0x7fbfa80fffd0 2024-03-24T01:03:43.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.160+0000 7fbfae800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa8196ac0 con 0x7fbfa8196350 2024-03-24T01:03:43.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.160+0000 7fbfad400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfa80fffd0 0x7fbfa8195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:43.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.160+0000 7fbfad400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfa80fffd0 0x7fbfa8195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34104/0 (socket says 172.21.15.38:34104) 2024-03-24T01:03:43.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.160+0000 7fbfad400640 1 -- 172.21.15.38:0/1328733023 learned_addr learned my addr 172.21.15.38:0/1328733023 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:43.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.161+0000 7fbfaca00640 1 --2- 172.21.15.38:0/1328733023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa8196350 0x7fbfa819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:43.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.161+0000 7fbfad400640 1 -- 172.21.15.38:0/1328733023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa8196350 msgr2=0x7fbfa819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:43.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.161+0000 7fbfad400640 1 --2- 172.21.15.38:0/1328733023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa8196350 0x7fbfa819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.161+0000 7fbfad400640 1 -- 172.21.15.38:0/1328733023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf94009560 con 0x7fbfa80fffd0 2024-03-24T01:03:43.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.161+0000 7fbfad400640 1 --2- 172.21.15.38:0/1328733023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfa80fffd0 0x7fbfa8195e10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00c840 tx=0x7fbf9c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:43.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.162+0000 7fbfa6000640 1 -- 172.21.15.38:0/1328733023 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf9c0041f0 con 0x7fbfa80fffd0 2024-03-24T01:03:43.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.162+0000 7fbfae800640 1 -- 172.21.15.38:0/1328733023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa819ac80 con 0x7fbfa80fffd0 2024-03-24T01:03:43.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.162+0000 7fbfa6000640 1 -- 172.21.15.38:0/1328733023 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf9c007e70 con 0x7fbfa80fffd0 2024-03-24T01:03:43.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.162+0000 7fbfa6000640 1 -- 172.21.15.38:0/1328733023 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf9c0049d0 con 0x7fbfa80fffd0 2024-03-24T01:03:43.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.162+0000 7fbfae800640 1 -- 172.21.15.38:0/1328733023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa819b1e0 con 0x7fbfa80fffd0 2024-03-24T01:03:43.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.163+0000 7fbfae800640 1 -- 172.21.15.38:0/1328733023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa819b4c0 con 0x7fbfa80fffd0 2024-03-24T01:03:43.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.165+0000 7fbfa6000640 1 -- 172.21.15.38:0/1328733023 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fbf9c0079b0 con 0x7fbfa80fffd0 2024-03-24T01:03:43.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.166+0000 7fbfa6000640 1 --2- 172.21.15.38:0/1328733023 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf9006df20 0x7fbf900703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:43.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.166+0000 7fbfa6000640 1 -- 172.21.15.38:0/1328733023 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbf9c08dd10 con 0x7fbfa80fffd0 2024-03-24T01:03:43.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.167+0000 7fbfaca00640 1 --2- 172.21.15.38:0/1328733023 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf9006df20 0x7fbf900703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:43.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.167+0000 7fbfaca00640 1 --2- 172.21.15.38:0/1328733023 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf9006df20 0x7fbf900703e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf94006010 tx=0x7fbf94000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:43.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.171+0000 7fbfa6000640 1 -- 172.21.15.38:0/1328733023 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbf9c057090 con 0x7fbfa80fffd0 2024-03-24T01:03:43.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.429+0000 7fbfae800640 1 -- 172.21.15.38:0/1328733023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fbfa81014a0 con 0x7fbfa80fffd0 2024-03-24T01:03:43.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.430+0000 7fbfa6000640 1 -- 172.21.15.38:0/1328733023 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v40) v1 ==== 107+0+4884 (secure 0 0 0) 0x7fbf9c05fe60 con 0x7fbfa80fffd0 2024-03-24T01:03:43.430 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:43.431 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi038.xnpqhp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/2851856830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2851856830},{"type":"v1","addr":"172.21.15.38:6829","nonce":2851856830}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":44293,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2738193611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2738193611},{"type":"v1","addr":"172.21.15.88:6825","nonce":2738193611}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44303,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/983744224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":983744224},{"type":"v1","addr":"172.21.15.88:6827","nonce":983744224}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:59:25.362766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:43.431 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-03-24T01:03:43.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.433+0000 7fbfae800640 1 -- 172.21.15.38:0/1328733023 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf9006df20 msgr2=0x7fbf900703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:43.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.433+0000 7fbfae800640 1 --2- 172.21.15.38:0/1328733023 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf9006df20 0x7fbf900703e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf94006010 tx=0x7fbf94000bc0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.433+0000 7fbfae800640 1 -- 172.21.15.38:0/1328733023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfa80fffd0 msgr2=0x7fbfa8195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:43.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.433+0000 7fbfae800640 1 --2- 172.21.15.38:0/1328733023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfa80fffd0 0x7fbfa8195e10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00c840 tx=0x7fbf9c00cd00 comp rx=0 tx=0).stop 2024-03-24T01:03:43.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.433+0000 7fbfae800640 1 -- 172.21.15.38:0/1328733023 shutdown_connections 2024-03-24T01:03:43.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.433+0000 7fbfae800640 1 --2- 172.21.15.38:0/1328733023 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbf9006df20 0x7fbf900703e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.433+0000 7fbfae800640 1 --2- 172.21.15.38:0/1328733023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa8196350 0x7fbfa819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.433+0000 7fbfae800640 1 --2- 172.21.15.38:0/1328733023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfa80fffd0 0x7fbfa8195e10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.433+0000 7fbfae800640 1 -- 172.21.15.38:0/1328733023 >> 172.21.15.38:0/1328733023 conn(0x7fbfa80fb850 msgr2=0x7fbfa80fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:43.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.433+0000 7fbfae800640 1 -- 172.21.15.38:0/1328733023 shutdown_connections 2024-03-24T01:03:43.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.433+0000 7fbfae800640 1 -- 172.21.15.38:0/1328733023 wait complete. 2024-03-24T01:03:43.456 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-24T01:03:43.456 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-03-24T01:03:43.456 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-24T01:03:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.675+0000 7fb782000640 1 -- 172.21.15.38:0/365743140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb77c073010 msgr2=0x7fb77c10c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.675+0000 7fb782000640 1 --2- 172.21.15.38:0/365743140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb77c073010 0x7fb77c10c230 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb764009880 tx=0x7fb76402bd70 comp rx=0 tx=0).stop 2024-03-24T01:03:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.675+0000 7fb782000640 1 -- 172.21.15.38:0/365743140 shutdown_connections 2024-03-24T01:03:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.675+0000 7fb782000640 1 --2- 172.21.15.38:0/365743140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb77c073010 0x7fb77c10c230 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.675+0000 7fb782000640 1 --2- 172.21.15.38:0/365743140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb77c0726f0 0x7fb77c072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.675+0000 7fb782000640 1 -- 172.21.15.38:0/365743140 >> 172.21.15.38:0/365743140 conn(0x7fb77c0fba20 msgr2=0x7fb77c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.675+0000 7fb782000640 1 -- 172.21.15.38:0/365743140 shutdown_connections 2024-03-24T01:03:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.675+0000 7fb782000640 1 -- 172.21.15.38:0/365743140 wait complete. 2024-03-24T01:03:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.676+0000 7fb782000640 1 Processor -- start 2024-03-24T01:03:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.676+0000 7fb782000640 1 -- start start 2024-03-24T01:03:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.676+0000 7fb782000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb77c0726f0 0x7fb77c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.677+0000 7fb782000640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb77c073010 0x7fb77c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.677+0000 7fb782000640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c198de0 con 0x7fb77c073010 2024-03-24T01:03:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.677+0000 7fb782000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c19caf0 con 0x7fb77c0726f0 2024-03-24T01:03:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.677+0000 7fb773e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb77c073010 0x7fb77c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.677+0000 7fb773e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb77c073010 0x7fb77c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34114/0 (socket says 172.21.15.38:34114) 2024-03-24T01:03:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.677+0000 7fb773e00640 1 -- 172.21.15.38:0/1731394339 learned_addr learned my addr 172.21.15.38:0/1731394339 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:43.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.677+0000 7fb77b400640 1 --2- 172.21.15.38:0/1731394339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb77c0726f0 0x7fb77c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.677+0000 7fb773e00640 1 -- 172.21.15.38:0/1731394339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb77c0726f0 msgr2=0x7fb77c198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.677+0000 7fb773e00640 1 --2- 172.21.15.38:0/1731394339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb77c0726f0 0x7fb77c198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.678+0000 7fb773e00640 1 -- 172.21.15.38:0/1731394339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb764009560 con 0x7fb77c073010 2024-03-24T01:03:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.678+0000 7fb773e00640 1 --2- 172.21.15.38:0/1731394339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb77c073010 0x7fb77c198750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb76402c2b0 tx=0x7fb76402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.678+0000 7fb778c00640 1 -- 172.21.15.38:0/1731394339 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb764004520 con 0x7fb77c073010 2024-03-24T01:03:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.678+0000 7fb778c00640 1 -- 172.21.15.38:0/1731394339 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb764003ab0 con 0x7fb77c073010 2024-03-24T01:03:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.678+0000 7fb782000640 1 -- 172.21.15.38:0/1731394339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77c19cd70 con 0x7fb77c073010 2024-03-24T01:03:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.678+0000 7fb778c00640 1 -- 172.21.15.38:0/1731394339 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb764005500 con 0x7fb77c073010 2024-03-24T01:03:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.679+0000 7fb782000640 1 -- 172.21.15.38:0/1731394339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77c19d2d0 con 0x7fb77c073010 2024-03-24T01:03:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.680+0000 7fb782000640 1 -- 172.21.15.38:0/1731394339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77c073da0 con 0x7fb77c073010 2024-03-24T01:03:43.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.681+0000 7fb778c00640 1 -- 172.21.15.38:0/1731394339 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fb764003c20 con 0x7fb77c073010 2024-03-24T01:03:43.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.682+0000 7fb778c00640 1 --2- 172.21.15.38:0/1731394339 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb76006df10 0x7fb7600703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:43.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.682+0000 7fb778c00640 1 -- 172.21.15.38:0/1731394339 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb7640b08c0 con 0x7fb77c073010 2024-03-24T01:03:43.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.682+0000 7fb77b400640 1 --2- 172.21.15.38:0/1731394339 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb76006df10 0x7fb7600703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:43.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.683+0000 7fb77b400640 1 --2- 172.21.15.38:0/1731394339 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb76006df10 0x7fb7600703d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb76c005fd0 tx=0x7fb76c005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:43.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.687+0000 7fb778c00640 1 -- 172.21.15.38:0/1731394339 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb764079cc0 con 0x7fb77c073010 2024-03-24T01:03:43.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:43 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/246262047' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-24T01:03:43.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:43 smithi038 ceph-mon[112108]: pgmap v333: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:43.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:43 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1328733023' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-24T01:03:43.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:43 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/246262047' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-24T01:03:43.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:43 smithi088 ceph-mon[87351]: pgmap v333: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:43.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:43 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1328733023' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-24T01:03:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.987+0000 7fb782000640 1 -- 172.21.15.38:0/1731394339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fb77c106a70 con 0x7fb77c073010 2024-03-24T01:03:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.988+0000 7fb778c00640 1 -- 172.21.15.38:0/1731394339 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v40) v1 ==== 107+0+4105 (secure 0 0 0) 0x7fb76407d380 con 0x7fb77c073010 2024-03-24T01:03:43.989 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:43.989 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":44293,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2738193611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2738193611},{"type":"v1","addr":"172.21.15.88:6825","nonce":2738193611}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44303,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/983744224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":983744224},{"type":"v1","addr":"172.21.15.88:6827","nonce":983744224}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:59:25.362766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-03-24T01:03:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.991+0000 7fb782000640 1 -- 172.21.15.38:0/1731394339 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb76006df10 msgr2=0x7fb7600703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.991+0000 7fb782000640 1 --2- 172.21.15.38:0/1731394339 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb76006df10 0x7fb7600703d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb76c005fd0 tx=0x7fb76c005850 comp rx=0 tx=0).stop 2024-03-24T01:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.991+0000 7fb782000640 1 -- 172.21.15.38:0/1731394339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb77c073010 msgr2=0x7fb77c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.991+0000 7fb782000640 1 --2- 172.21.15.38:0/1731394339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb77c073010 0x7fb77c198750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb76402c2b0 tx=0x7fb76402c850 comp rx=0 tx=0).stop 2024-03-24T01:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.991+0000 7fb782000640 1 -- 172.21.15.38:0/1731394339 shutdown_connections 2024-03-24T01:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.991+0000 7fb782000640 1 --2- 172.21.15.38:0/1731394339 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb76006df10 0x7fb7600703d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.991+0000 7fb782000640 1 --2- 172.21.15.38:0/1731394339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb77c073010 0x7fb77c198750 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.991+0000 7fb782000640 1 --2- 172.21.15.38:0/1731394339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb77c0726f0 0x7fb77c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.991+0000 7fb782000640 1 -- 172.21.15.38:0/1731394339 >> 172.21.15.38:0/1731394339 conn(0x7fb77c0fba20 msgr2=0x7fb77c104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.991+0000 7fb782000640 1 -- 172.21.15.38:0/1731394339 shutdown_connections 2024-03-24T01:03:43.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:43.991+0000 7fb782000640 1 -- 172.21.15.38:0/1731394339 wait complete. 2024-03-24T01:03:44.016 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-24T01:03:44.017 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-03-24T01:03:44.017 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-24T01:03:44.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.214+0000 7f94f7200640 1 -- 172.21.15.38:0/2112779152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f80726f0 msgr2=0x7f94f8072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.214+0000 7f94f7200640 1 --2- 172.21.15.38:0/2112779152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f80726f0 0x7f94f8072ad0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009880 tx=0x7f94e802bd70 comp rx=0 tx=0).stop 2024-03-24T01:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.214+0000 7f94f7200640 1 -- 172.21.15.38:0/2112779152 shutdown_connections 2024-03-24T01:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.214+0000 7f94f7200640 1 --2- 172.21.15.38:0/2112779152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f8073010 0x7f94f810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.214+0000 7f94f7200640 1 --2- 172.21.15.38:0/2112779152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f80726f0 0x7f94f8072ad0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.214+0000 7f94f7200640 1 -- 172.21.15.38:0/2112779152 >> 172.21.15.38:0/2112779152 conn(0x7f94f80fb850 msgr2=0x7f94f80fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.214+0000 7f94f7200640 1 -- 172.21.15.38:0/2112779152 shutdown_connections 2024-03-24T01:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.214+0000 7f94f7200640 1 -- 172.21.15.38:0/2112779152 wait complete. 2024-03-24T01:03:44.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.215+0000 7f94f7200640 1 Processor -- start 2024-03-24T01:03:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.215+0000 7f94f7200640 1 -- start start 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.215+0000 7f94f7200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f80726f0 0x7f94f8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.215+0000 7f94f7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f8073010 0x7f94f81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.215+0000 7f94f7200640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f8198b70 con 0x7f94f80726f0 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.215+0000 7f94f7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f819c880 con 0x7f94f8073010 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.216+0000 7f94f5e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f80726f0 0x7f94f8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.216+0000 7f94f5e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f80726f0 0x7f94f8197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34134/0 (socket says 172.21.15.38:34134) 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.216+0000 7f94f5e00640 1 -- 172.21.15.38:0/730739622 learned_addr learned my addr 172.21.15.38:0/730739622 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.216+0000 7f94efe00640 1 --2- 172.21.15.38:0/730739622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f8073010 0x7f94f81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.216+0000 7f94f5e00640 1 -- 172.21.15.38:0/730739622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f8073010 msgr2=0x7f94f81984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.216+0000 7f94f5e00640 1 --2- 172.21.15.38:0/730739622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f8073010 0x7f94f81984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.216+0000 7f94f5e00640 1 -- 172.21.15.38:0/730739622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e8009560 con 0x7f94f80726f0 2024-03-24T01:03:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.217+0000 7f94f5e00640 1 --2- 172.21.15.38:0/730739622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f80726f0 0x7f94f8197fa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f94e8006010 tx=0x7f94e802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:44.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.217+0000 7f94eea00640 1 -- 172.21.15.38:0/730739622 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e8004520 con 0x7f94f80726f0 2024-03-24T01:03:44.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.217+0000 7f94eea00640 1 -- 172.21.15.38:0/730739622 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94e8003ab0 con 0x7f94f80726f0 2024-03-24T01:03:44.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.217+0000 7f94f7200640 1 -- 172.21.15.38:0/730739622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f819cb00 con 0x7f94f80726f0 2024-03-24T01:03:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.217+0000 7f94eea00640 1 -- 172.21.15.38:0/730739622 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e8005520 con 0x7f94f80726f0 2024-03-24T01:03:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.217+0000 7f94f7200640 1 -- 172.21.15.38:0/730739622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f819d060 con 0x7f94f80726f0 2024-03-24T01:03:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.219+0000 7f94f7200640 1 -- 172.21.15.38:0/730739622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f8073da0 con 0x7f94f80726f0 2024-03-24T01:03:44.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.221+0000 7f94eea00640 1 -- 172.21.15.38:0/730739622 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f94e8003c20 con 0x7f94f80726f0 2024-03-24T01:03:44.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.222+0000 7f94eea00640 1 --2- 172.21.15.38:0/730739622 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f94dc06df20 0x7f94dc0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:44.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.222+0000 7f94eea00640 1 -- 172.21.15.38:0/730739622 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f94e80b0990 con 0x7f94f80726f0 2024-03-24T01:03:44.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.222+0000 7f94efe00640 1 --2- 172.21.15.38:0/730739622 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f94dc06df20 0x7f94dc0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:44.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.223+0000 7f94efe00640 1 --2- 172.21.15.38:0/730739622 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f94dc06df20 0x7f94dc0703e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f94e0005fd0 tx=0x7f94e0005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.228+0000 7f94eea00640 1 -- 172.21.15.38:0/730739622 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f94e8079d10 con 0x7f94f80726f0 2024-03-24T01:03:44.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.474+0000 7f94f7200640 1 -- 172.21.15.38:0/730739622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f94f8106800 con 0x7f94f80726f0 2024-03-24T01:03:44.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.475+0000 7f94eea00640 1 -- 172.21.15.38:0/730739622 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v40) v1 ==== 107+0+4918 (secure 0 0 0) 0x7f94e807d3d0 con 0x7f94f80726f0 2024-03-24T01:03:44.476 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:44.476 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44293,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2738193611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2738193611},{"type":"v1","addr":"172.21.15.88:6825","nonce":2738193611}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44303,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/983744224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":983744224},{"type":"v1","addr":"172.21.15.88:6827","nonce":983744224}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:59:25.362766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:44.477 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-03-24T01:03:44.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.479+0000 7f94f7200640 1 -- 172.21.15.38:0/730739622 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f94dc06df20 msgr2=0x7f94dc0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:44.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.479+0000 7f94f7200640 1 --2- 172.21.15.38:0/730739622 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f94dc06df20 0x7f94dc0703e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f94e0005fd0 tx=0x7f94e0005850 comp rx=0 tx=0).stop 2024-03-24T01:03:44.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.479+0000 7f94f7200640 1 -- 172.21.15.38:0/730739622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f80726f0 msgr2=0x7f94f8197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:44.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.479+0000 7f94f7200640 1 --2- 172.21.15.38:0/730739622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f80726f0 0x7f94f8197fa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f94e8006010 tx=0x7f94e802c850 comp rx=0 tx=0).stop 2024-03-24T01:03:44.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.479+0000 7f94f7200640 1 -- 172.21.15.38:0/730739622 shutdown_connections 2024-03-24T01:03:44.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.479+0000 7f94f7200640 1 --2- 172.21.15.38:0/730739622 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f94dc06df20 0x7f94dc0703e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:44.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.479+0000 7f94f7200640 1 --2- 172.21.15.38:0/730739622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f8073010 0x7f94f81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:44.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.479+0000 7f94f7200640 1 --2- 172.21.15.38:0/730739622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94f80726f0 0x7f94f8197fa0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:44.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.479+0000 7f94f7200640 1 -- 172.21.15.38:0/730739622 >> 172.21.15.38:0/730739622 conn(0x7f94f80fb850 msgr2=0x7f94f8104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:44.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.479+0000 7f94f7200640 1 -- 172.21.15.38:0/730739622 shutdown_connections 2024-03-24T01:03:44.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.479+0000 7f94f7200640 1 -- 172.21.15.38:0/730739622 wait complete. 2024-03-24T01:03:44.497 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-24T01:03:44.497 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-03-24T01:03:44.498 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-24T01:03:44.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.756+0000 7f7b9aa00640 1 -- 172.21.15.38:0/579072894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b941010e0 msgr2=0x7f7b941014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:44.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.756+0000 7f7b9aa00640 1 --2- 172.21.15.38:0/579072894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b941010e0 0x7f7b941014c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009880 tx=0x7f7b8402bd70 comp rx=0 tx=0).stop 2024-03-24T01:03:44.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.756+0000 7f7b9aa00640 1 -- 172.21.15.38:0/579072894 shutdown_connections 2024-03-24T01:03:44.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.756+0000 7f7b9aa00640 1 --2- 172.21.15.38:0/579072894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b94101a00 0x7f7b94109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:44.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.756+0000 7f7b9aa00640 1 --2- 172.21.15.38:0/579072894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b941010e0 0x7f7b941014c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:44.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.756+0000 7f7b9aa00640 1 -- 172.21.15.38:0/579072894 >> 172.21.15.38:0/579072894 conn(0x7f7b940fa8e0 msgr2=0x7f7b940fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:44.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.757+0000 7f7b9aa00640 1 -- 172.21.15.38:0/579072894 shutdown_connections 2024-03-24T01:03:44.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.757+0000 7f7b9aa00640 1 -- 172.21.15.38:0/579072894 wait complete. 2024-03-24T01:03:44.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.758+0000 7f7b9aa00640 1 Processor -- start 2024-03-24T01:03:44.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.758+0000 7f7b9aa00640 1 -- start start 2024-03-24T01:03:44.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.758+0000 7f7b9aa00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b941010e0 0x7f7b941981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:44.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.758+0000 7f7b9aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b94101a00 0x7f7b94198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:44.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.758+0000 7f7b9aa00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b94198db0 con 0x7f7b941010e0 2024-03-24T01:03:44.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.758+0000 7f7b9aa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9419cac0 con 0x7f7b94101a00 2024-03-24T01:03:44.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.758+0000 7f7b93e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b941010e0 0x7f7b941981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:44.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.758+0000 7f7b93e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b941010e0 0x7f7b941981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34164/0 (socket says 172.21.15.38:34164) 2024-03-24T01:03:44.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.758+0000 7f7b93e00640 1 -- 172.21.15.38:0/1814263821 learned_addr learned my addr 172.21.15.38:0/1814263821 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:44.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.759+0000 7f7b93400640 1 --2- 172.21.15.38:0/1814263821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b94101a00 0x7f7b94198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:44.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.759+0000 7f7b93e00640 1 -- 172.21.15.38:0/1814263821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b94101a00 msgr2=0x7f7b94198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:44.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.759+0000 7f7b93e00640 1 --2- 172.21.15.38:0/1814263821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b94101a00 0x7f7b94198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:44.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.759+0000 7f7b93e00640 1 -- 172.21.15.38:0/1814263821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b84009560 con 0x7f7b941010e0 2024-03-24T01:03:44.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.759+0000 7f7b93e00640 1 --2- 172.21.15.38:0/1814263821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b941010e0 0x7f7b941981e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7b84006010 tx=0x7f7b8402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:44.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.760+0000 7f7b90c00640 1 -- 172.21.15.38:0/1814263821 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b84004520 con 0x7f7b941010e0 2024-03-24T01:03:44.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.760+0000 7f7b90c00640 1 -- 172.21.15.38:0/1814263821 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b84003ab0 con 0x7f7b941010e0 2024-03-24T01:03:44.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.760+0000 7f7b90c00640 1 -- 172.21.15.38:0/1814263821 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b84005520 con 0x7f7b941010e0 2024-03-24T01:03:44.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.760+0000 7f7b9aa00640 1 -- 172.21.15.38:0/1814263821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9419cd40 con 0x7f7b941010e0 2024-03-24T01:03:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.760+0000 7f7b9aa00640 1 -- 172.21.15.38:0/1814263821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9419d2a0 con 0x7f7b941010e0 2024-03-24T01:03:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.761+0000 7f7b9aa00640 1 -- 172.21.15.38:0/1814263821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b94068d00 con 0x7f7b941010e0 2024-03-24T01:03:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.762+0000 7f7b90c00640 1 -- 172.21.15.38:0/1814263821 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f7b84003c20 con 0x7f7b941010e0 2024-03-24T01:03:44.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.763+0000 7f7b90c00640 1 --2- 172.21.15.38:0/1814263821 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7b7806df10 0x7f7b780703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:44.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.764+0000 7f7b90c00640 1 -- 172.21.15.38:0/1814263821 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7b840b0920 con 0x7f7b941010e0 2024-03-24T01:03:44.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.764+0000 7f7b93400640 1 --2- 172.21.15.38:0/1814263821 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7b7806df10 0x7f7b780703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:44.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.765+0000 7f7b93400640 1 --2- 172.21.15.38:0/1814263821 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7b7806df10 0x7f7b780703d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b94199730 tx=0x7f7b80005eb0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:44.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:44.772+0000 7f7b90c00640 1 -- 172.21.15.38:0/1814263821 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7b84079ca0 con 0x7f7b941010e0 2024-03-24T01:03:44.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:44 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1731394339' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-24T01:03:44.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:44 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/730739622' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-24T01:03:44.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:44 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1731394339' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-24T01:03:44.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:44 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/730739622' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-24T01:03:45.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.194+0000 7f7b9aa00640 1 -- 172.21.15.38:0/1814263821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f7b9404e030 con 0x7f7b941010e0 2024-03-24T01:03:45.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.194+0000 7f7b90c00640 1 -- 172.21.15.38:0/1814263821 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v40) v1 ==== 107+0+4139 (secure 0 0 0) 0x7f7b840aee60 con 0x7f7b941010e0 2024-03-24T01:03:45.194 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:45.195 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44303,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/983744224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":983744224},{"type":"v1","addr":"172.21.15.88:6827","nonce":983744224}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:59:25.362766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:45.195 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-03-24T01:03:45.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.197+0000 7f7b9aa00640 1 -- 172.21.15.38:0/1814263821 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7b7806df10 msgr2=0x7f7b780703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:45.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.197+0000 7f7b9aa00640 1 --2- 172.21.15.38:0/1814263821 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7b7806df10 0x7f7b780703d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b94199730 tx=0x7f7b80005eb0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.197+0000 7f7b9aa00640 1 -- 172.21.15.38:0/1814263821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b941010e0 msgr2=0x7f7b941981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:45.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.197+0000 7f7b9aa00640 1 --2- 172.21.15.38:0/1814263821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b941010e0 0x7f7b941981e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7b84006010 tx=0x7f7b8402c850 comp rx=0 tx=0).stop 2024-03-24T01:03:45.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.197+0000 7f7b9aa00640 1 -- 172.21.15.38:0/1814263821 shutdown_connections 2024-03-24T01:03:45.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.197+0000 7f7b9aa00640 1 --2- 172.21.15.38:0/1814263821 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f7b7806df10 0x7f7b780703d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.197+0000 7f7b9aa00640 1 --2- 172.21.15.38:0/1814263821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b94101a00 0x7f7b94198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.197+0000 7f7b9aa00640 1 --2- 172.21.15.38:0/1814263821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b941010e0 0x7f7b941981e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.197+0000 7f7b9aa00640 1 -- 172.21.15.38:0/1814263821 >> 172.21.15.38:0/1814263821 conn(0x7f7b940fa8e0 msgr2=0x7f7b94100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:45.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.197+0000 7f7b9aa00640 1 -- 172.21.15.38:0/1814263821 shutdown_connections 2024-03-24T01:03:45.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.197+0000 7f7b9aa00640 1 -- 172.21.15.38:0/1814263821 wait complete. 2024-03-24T01:03:45.216 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-24T01:03:45.216 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-03-24T01:03:45.217 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-24T01:03:45.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.417+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3301455860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c0101a80 msgr2=0x7fb8c0109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:45.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.417+0000 7fb8c4c00640 1 --2- 172.21.15.38:0/3301455860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c0101a80 0x7fb8c0109f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009880 tx=0x7fb8ac02bd80 comp rx=0 tx=0).stop 2024-03-24T01:03:45.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.418+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3301455860 shutdown_connections 2024-03-24T01:03:45.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.418+0000 7fb8c4c00640 1 --2- 172.21.15.38:0/3301455860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c0101a80 0x7fb8c0109f60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.418+0000 7fb8c4c00640 1 --2- 172.21.15.38:0/3301455860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8c0101160 0x7fb8c0101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.418+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3301455860 >> 172.21.15.38:0/3301455860 conn(0x7fb8c00fa8e0 msgr2=0x7fb8c00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:45.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.418+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3301455860 shutdown_connections 2024-03-24T01:03:45.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.418+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3301455860 wait complete. 2024-03-24T01:03:45.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.419+0000 7fb8c4c00640 1 Processor -- start 2024-03-24T01:03:45.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.419+0000 7fb8c4c00640 1 -- start start 2024-03-24T01:03:45.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.420+0000 7fb8c4c00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c0101160 0x7fb8c01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:45.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.420+0000 7fb8c4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8c0101a80 0x7fb8c01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:45.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.420+0000 7fb8c4c00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c0198d80 con 0x7fb8c0101160 2024-03-24T01:03:45.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.420+0000 7fb8c4c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c019ca90 con 0x7fb8c0101a80 2024-03-24T01:03:45.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.420+0000 7fb8bde00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c0101160 0x7fb8c01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:45.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.420+0000 7fb8bde00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c0101160 0x7fb8c01981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34176/0 (socket says 172.21.15.38:34176) 2024-03-24T01:03:45.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.420+0000 7fb8bde00640 1 -- 172.21.15.38:0/3949970537 learned_addr learned my addr 172.21.15.38:0/3949970537 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:45.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.420+0000 7fb8bd400640 1 --2- 172.21.15.38:0/3949970537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8c0101a80 0x7fb8c01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:45.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.421+0000 7fb8bde00640 1 -- 172.21.15.38:0/3949970537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8c0101a80 msgr2=0x7fb8c01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:45.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.421+0000 7fb8bde00640 1 --2- 172.21.15.38:0/3949970537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8c0101a80 0x7fb8c01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.421+0000 7fb8bde00640 1 -- 172.21.15.38:0/3949970537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac009560 con 0x7fb8c0101160 2024-03-24T01:03:45.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.421+0000 7fb8bde00640 1 --2- 172.21.15.38:0/3949970537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c0101160 0x7fb8c01981b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000d8d0 tx=0x7fb8b000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:45.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.421+0000 7fb8b6a00640 1 -- 172.21.15.38:0/3949970537 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b000bea0 con 0x7fb8c0101160 2024-03-24T01:03:45.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.421+0000 7fb8b6a00640 1 -- 172.21.15.38:0/3949970537 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8b0009c00 con 0x7fb8c0101160 2024-03-24T01:03:45.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.422+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3949970537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c019cd70 con 0x7fb8c0101160 2024-03-24T01:03:45.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.422+0000 7fb8b6a00640 1 -- 172.21.15.38:0/3949970537 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b000f710 con 0x7fb8c0101160 2024-03-24T01:03:45.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.422+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3949970537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c019d2a0 con 0x7fb8c0101160 2024-03-24T01:03:45.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.423+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3949970537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c0068c80 con 0x7fb8c0101160 2024-03-24T01:03:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.425+0000 7fb8b6a00640 1 -- 172.21.15.38:0/3949970537 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fb8b000f990 con 0x7fb8c0101160 2024-03-24T01:03:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.426+0000 7fb8b6a00640 1 --2- 172.21.15.38:0/3949970537 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8a406df10 0x7fb8a40703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.426+0000 7fb8b6a00640 1 -- 172.21.15.38:0/3949970537 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb8b008db00 con 0x7fb8c0101160 2024-03-24T01:03:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.426+0000 7fb8bd400640 1 --2- 172.21.15.38:0/3949970537 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8a406df10 0x7fb8a40703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.427+0000 7fb8bd400640 1 --2- 172.21.15.38:0/3949970537 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8a406df10 0x7fb8a40703d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac006010 tx=0x7fb8ac005d50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:45.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.435+0000 7fb8b6a00640 1 -- 172.21.15.38:0/3949970537 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb8b0056f30 con 0x7fb8c0101160 2024-03-24T01:03:45.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.693+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3949970537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fb8c004e030 con 0x7fb8c0101160 2024-03-24T01:03:45.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.694+0000 7fb8b6a00640 1 -- 172.21.15.38:0/3949970537 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v40) v1 ==== 107+0+4952 (secure 0 0 0) 0x7fb8b005a5f0 con 0x7fb8c0101160 2024-03-24T01:03:45.694 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:45.694 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44303,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/983744224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":983744224},{"type":"v1","addr":"172.21.15.88:6827","nonce":983744224}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:59:25.362766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:45.695 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 29 2024-03-24T01:03:45.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.696+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3949970537 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8a406df10 msgr2=0x7fb8a40703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:45.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.696+0000 7fb8c4c00640 1 --2- 172.21.15.38:0/3949970537 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8a406df10 0x7fb8a40703d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac006010 tx=0x7fb8ac005d50 comp rx=0 tx=0).stop 2024-03-24T01:03:45.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.697+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3949970537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c0101160 msgr2=0x7fb8c01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:45.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.697+0000 7fb8c4c00640 1 --2- 172.21.15.38:0/3949970537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c0101160 0x7fb8c01981b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000d8d0 tx=0x7fb8b000dd90 comp rx=0 tx=0).stop 2024-03-24T01:03:45.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.697+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3949970537 shutdown_connections 2024-03-24T01:03:45.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.697+0000 7fb8c4c00640 1 --2- 172.21.15.38:0/3949970537 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fb8a406df10 0x7fb8a40703d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.697+0000 7fb8c4c00640 1 --2- 172.21.15.38:0/3949970537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8c0101a80 0x7fb8c01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.697+0000 7fb8c4c00640 1 --2- 172.21.15.38:0/3949970537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8c0101160 0x7fb8c01981b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.697+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3949970537 >> 172.21.15.38:0/3949970537 conn(0x7fb8c00fa8e0 msgr2=0x7fb8c0100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:45.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.697+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3949970537 shutdown_connections 2024-03-24T01:03:45.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.697+0000 7fb8c4c00640 1 -- 172.21.15.38:0/3949970537 wait complete. 2024-03-24T01:03:45.720 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-24T01:03:45.720 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-03-24T01:03:45.720 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-24T01:03:45.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:45 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:03:45.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:45 smithi038 ceph-mon[112108]: pgmap v334: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:45.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:45 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1814263821' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-24T01:03:45.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.924+0000 7f21b6e00640 1 -- 172.21.15.38:0/1470977192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21b0072670 msgr2=0x7f21b0072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:45.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.924+0000 7f21b6e00640 1 --2- 172.21.15.38:0/1470977192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21b0072670 0x7f21b0072a50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009880 tx=0x7f21a402bd80 comp rx=0 tx=0).stop 2024-03-24T01:03:45.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.924+0000 7f21b6e00640 1 -- 172.21.15.38:0/1470977192 shutdown_connections 2024-03-24T01:03:45.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.924+0000 7f21b6e00640 1 --2- 172.21.15.38:0/1470977192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0072f90 0x7f21b010c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.924+0000 7f21b6e00640 1 --2- 172.21.15.38:0/1470977192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21b0072670 0x7f21b0072a50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.924+0000 7f21b6e00640 1 -- 172.21.15.38:0/1470977192 >> 172.21.15.38:0/1470977192 conn(0x7f21b00fb8b0 msgr2=0x7f21b00fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:45.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.924+0000 7f21b6e00640 1 -- 172.21.15.38:0/1470977192 shutdown_connections 2024-03-24T01:03:45.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.924+0000 7f21b6e00640 1 -- 172.21.15.38:0/1470977192 wait complete. 2024-03-24T01:03:45.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.925+0000 7f21b6e00640 1 Processor -- start 2024-03-24T01:03:45.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.925+0000 7f21b6e00640 1 -- start start 2024-03-24T01:03:45.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.925+0000 7f21b6e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0072670 0x7f21b0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:45.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.925+0000 7f21b6e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21b0072f90 0x7f21b0198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:45.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.925+0000 7f21b6e00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b0198b80 con 0x7f21b0072f90 2024-03-24T01:03:45.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.925+0000 7f21b6e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b0198cf0 con 0x7f21b0072670 2024-03-24T01:03:45.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.926+0000 7f21b5000640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21b0072f90 0x7f21b0198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:45.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.926+0000 7f21b5000640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21b0072f90 0x7f21b0198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34196/0 (socket says 172.21.15.38:34196) 2024-03-24T01:03:45.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.926+0000 7f21b5000640 1 -- 172.21.15.38:0/3894392877 learned_addr learned my addr 172.21.15.38:0/3894392877 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:45.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.926+0000 7f21b5a00640 1 --2- 172.21.15.38:0/3894392877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0072670 0x7f21b0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:45.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.926+0000 7f21b5000640 1 -- 172.21.15.38:0/3894392877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0072670 msgr2=0x7f21b0198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:45.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.926+0000 7f21b5000640 1 --2- 172.21.15.38:0/3894392877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0072670 0x7f21b0198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:45.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.926+0000 7f21b5000640 1 -- 172.21.15.38:0/3894392877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a4009560 con 0x7f21b0072f90 2024-03-24T01:03:45.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.927+0000 7f21b5000640 1 --2- 172.21.15.38:0/3894392877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21b0072f90 0x7f21b0198580 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f219c00c8a0 tx=0x7f219c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:45.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.927+0000 7f21ae000640 1 -- 172.21.15.38:0/3894392877 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f219c007b80 con 0x7f21b0072f90 2024-03-24T01:03:45.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.927+0000 7f21ae000640 1 -- 172.21.15.38:0/3894392877 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f219c007ce0 con 0x7f21b0072f90 2024-03-24T01:03:45.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.927+0000 7f21b6e00640 1 -- 172.21.15.38:0/3894392877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b019cac0 con 0x7f21b0072f90 2024-03-24T01:03:45.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.927+0000 7f21b6e00640 1 -- 172.21.15.38:0/3894392877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b019d020 con 0x7f21b0072f90 2024-03-24T01:03:45.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.927+0000 7f21ae000640 1 -- 172.21.15.38:0/3894392877 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f219c018810 con 0x7f21b0072f90 2024-03-24T01:03:45.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.929+0000 7f21b6e00640 1 -- 172.21.15.38:0/3894392877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b0073d20 con 0x7f21b0072f90 2024-03-24T01:03:45.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:45 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T01:03:45.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:45 smithi088 ceph-mon[87351]: pgmap v334: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:45.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:45 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1814263821' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-24T01:03:45.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.930+0000 7f21ae000640 1 -- 172.21.15.38:0/3894392877 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f219c018a80 con 0x7f21b0072f90 2024-03-24T01:03:45.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.932+0000 7f21ae000640 1 --2- 172.21.15.38:0/3894392877 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f219806df10 0x7f21980703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:45.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.932+0000 7f21ae000640 1 -- 172.21.15.38:0/3894392877 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f219c08df20 con 0x7f21b0072f90 2024-03-24T01:03:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.932+0000 7f21b5a00640 1 --2- 172.21.15.38:0/3894392877 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f219806df10 0x7f21980703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:45.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.933+0000 7f21b5a00640 1 --2- 172.21.15.38:0/3894392877 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f219806df10 0x7f21980703d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f21a4006010 tx=0x7f21a4005dd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:45.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:45.936+0000 7f21ae000640 1 -- 172.21.15.38:0/3894392877 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f219c0c2740 con 0x7f21b0072f90 2024-03-24T01:03:46.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.208+0000 7f21b6e00640 1 -- 172.21.15.38:0/3894392877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f21b0106a10 con 0x7f21b0072f90 2024-03-24T01:03:46.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.209+0000 7f21ae000640 1 -- 172.21.15.38:0/3894392877 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v40) v1 ==== 107+0+4176 (secure 0 0 0) 0x7f219c0573c0 con 0x7f21b0072f90 2024-03-24T01:03:46.211 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:46.212 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:59:25.362766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 30 2024-03-24T01:03:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.212+0000 7f21b6e00640 1 -- 172.21.15.38:0/3894392877 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f219806df10 msgr2=0x7f21980703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.212+0000 7f21b6e00640 1 --2- 172.21.15.38:0/3894392877 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f219806df10 0x7f21980703d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f21a4006010 tx=0x7f21a4005dd0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.212+0000 7f21b6e00640 1 -- 172.21.15.38:0/3894392877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21b0072f90 msgr2=0x7f21b0198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.212+0000 7f21b6e00640 1 --2- 172.21.15.38:0/3894392877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21b0072f90 0x7f21b0198580 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f219c00c8a0 tx=0x7f219c00cd60 comp rx=0 tx=0).stop 2024-03-24T01:03:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.212+0000 7f21b6e00640 1 -- 172.21.15.38:0/3894392877 shutdown_connections 2024-03-24T01:03:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.212+0000 7f21b6e00640 1 --2- 172.21.15.38:0/3894392877 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f219806df10 0x7f21980703d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.212+0000 7f21b6e00640 1 --2- 172.21.15.38:0/3894392877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21b0072f90 0x7f21b0198580 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.212+0000 7f21b6e00640 1 --2- 172.21.15.38:0/3894392877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0072670 0x7f21b0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.212+0000 7f21b6e00640 1 -- 172.21.15.38:0/3894392877 >> 172.21.15.38:0/3894392877 conn(0x7f21b00fb8b0 msgr2=0x7f21b01045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:46.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.212+0000 7f21b6e00640 1 -- 172.21.15.38:0/3894392877 shutdown_connections 2024-03-24T01:03:46.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.212+0000 7f21b6e00640 1 -- 172.21.15.38:0/3894392877 wait complete. 2024-03-24T01:03:46.230 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-24T01:03:46.230 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-03-24T01:03:46.231 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-24T01:03:46.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.426+0000 7f99eb200640 1 -- 172.21.15.38:0/3364021605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4101a30 msgr2=0x7f99e4109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:46.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.426+0000 7f99eb200640 1 --2- 172.21.15.38:0/3364021605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4101a30 0x7f99e4109f10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009880 tx=0x7f99d002bd90 comp rx=0 tx=0).stop 2024-03-24T01:03:46.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.426+0000 7f99eb200640 1 -- 172.21.15.38:0/3364021605 shutdown_connections 2024-03-24T01:03:46.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.426+0000 7f99eb200640 1 --2- 172.21.15.38:0/3364021605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4101a30 0x7f99e4109f10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.426+0000 7f99eb200640 1 --2- 172.21.15.38:0/3364021605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99e4101110 0x7f99e41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.426+0000 7f99eb200640 1 -- 172.21.15.38:0/3364021605 >> 172.21.15.38:0/3364021605 conn(0x7f99e40fa930 msgr2=0x7f99e40fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:46.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.426+0000 7f99eb200640 1 -- 172.21.15.38:0/3364021605 shutdown_connections 2024-03-24T01:03:46.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.426+0000 7f99eb200640 1 -- 172.21.15.38:0/3364021605 wait complete. 2024-03-24T01:03:46.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.427+0000 7f99eb200640 1 Processor -- start 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.427+0000 7f99eb200640 1 -- start start 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.428+0000 7f99eb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99e4101110 0x7f99e41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.428+0000 7f99eb200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4101a30 0x7f99e41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.428+0000 7f99eb200640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e4198d70 con 0x7f99e4101a30 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.428+0000 7f99eb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e419ca80 con 0x7f99e4101110 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.428+0000 7f99e3400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4101a30 0x7f99e41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.428+0000 7f99e3400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4101a30 0x7f99e41986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34222/0 (socket says 172.21.15.38:34222) 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.428+0000 7f99e3400640 1 -- 172.21.15.38:0/3542948457 learned_addr learned my addr 172.21.15.38:0/3542948457 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.429+0000 7f99e3e00640 1 --2- 172.21.15.38:0/3542948457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99e4101110 0x7f99e41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.429+0000 7f99e3400640 1 -- 172.21.15.38:0/3542948457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99e4101110 msgr2=0x7f99e41981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.429+0000 7f99e3400640 1 --2- 172.21.15.38:0/3542948457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99e4101110 0x7f99e41981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.429+0000 7f99e3400640 1 -- 172.21.15.38:0/3542948457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d0009560 con 0x7f99e4101a30 2024-03-24T01:03:46.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.429+0000 7f99e3400640 1 --2- 172.21.15.38:0/3542948457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4101a30 0x7f99e41986e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99d0006010 tx=0x7f99d0004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.430+0000 7f99e0c00640 1 -- 172.21.15.38:0/3542948457 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99d0004420 con 0x7f99e4101a30 2024-03-24T01:03:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.430+0000 7f99e0c00640 1 -- 172.21.15.38:0/3542948457 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99d0003ab0 con 0x7f99e4101a30 2024-03-24T01:03:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.430+0000 7f99eb200640 1 -- 172.21.15.38:0/3542948457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e419cd00 con 0x7f99e4101a30 2024-03-24T01:03:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.430+0000 7f99e0c00640 1 -- 172.21.15.38:0/3542948457 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99d0005520 con 0x7f99e4101a30 2024-03-24T01:03:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.430+0000 7f99eb200640 1 -- 172.21.15.38:0/3542948457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e419d260 con 0x7f99e4101a30 2024-03-24T01:03:46.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.431+0000 7f99eb200640 1 -- 172.21.15.38:0/3542948457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e4068690 con 0x7f99e4101a30 2024-03-24T01:03:46.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.433+0000 7f99e0c00640 1 -- 172.21.15.38:0/3542948457 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f99d0003c20 con 0x7f99e4101a30 2024-03-24T01:03:46.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.434+0000 7f99e0c00640 1 --2- 172.21.15.38:0/3542948457 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f99c806df10 0x7f99c80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:46.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.434+0000 7f99e0c00640 1 -- 172.21.15.38:0/3542948457 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f99d00b0820 con 0x7f99e4101a30 2024-03-24T01:03:46.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.434+0000 7f99e3e00640 1 --2- 172.21.15.38:0/3542948457 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f99c806df10 0x7f99c80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:46.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.435+0000 7f99e3e00640 1 --2- 172.21.15.38:0/3542948457 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f99c806df10 0x7f99c80703d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f99d4005fd0 tx=0x7f99d4005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:46.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.441+0000 7f99e0c00640 1 -- 172.21.15.38:0/3542948457 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f99d0079c20 con 0x7f99e4101a30 2024-03-24T01:03:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:46 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3949970537' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-24T01:03:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:46 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3894392877' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-24T01:03:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:46 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:03:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:46 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:03:46.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:46 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:03:46.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.696+0000 7f99eb200640 1 -- 172.21.15.38:0/3542948457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f99e404e030 con 0x7f99e4101a30 2024-03-24T01:03:46.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.697+0000 7f99e0c00640 1 -- 172.21.15.38:0/3542948457 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v40) v1 ==== 107+0+4989 (secure 0 0 0) 0x7f99d007d2e0 con 0x7f99e4101a30 2024-03-24T01:03:46.697 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:46.697 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44373,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/1135457323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1135457323},{"type":"v1","addr":"172.21.15.88:6827","nonce":1135457323}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T00:59:25.362766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:46.697 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 31 2024-03-24T01:03:46.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.700+0000 7f99eb200640 1 -- 172.21.15.38:0/3542948457 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f99c806df10 msgr2=0x7f99c80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:46.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.700+0000 7f99eb200640 1 --2- 172.21.15.38:0/3542948457 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f99c806df10 0x7f99c80703d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f99d4005fd0 tx=0x7f99d4005850 comp rx=0 tx=0).stop 2024-03-24T01:03:46.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.700+0000 7f99eb200640 1 -- 172.21.15.38:0/3542948457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4101a30 msgr2=0x7f99e41986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:46.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.700+0000 7f99eb200640 1 --2- 172.21.15.38:0/3542948457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4101a30 0x7f99e41986e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99d0006010 tx=0x7f99d0004a10 comp rx=0 tx=0).stop 2024-03-24T01:03:46.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.700+0000 7f99eb200640 1 -- 172.21.15.38:0/3542948457 shutdown_connections 2024-03-24T01:03:46.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.700+0000 7f99eb200640 1 --2- 172.21.15.38:0/3542948457 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f99c806df10 0x7f99c80703d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.700+0000 7f99eb200640 1 --2- 172.21.15.38:0/3542948457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4101a30 0x7f99e41986e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.700+0000 7f99eb200640 1 --2- 172.21.15.38:0/3542948457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99e4101110 0x7f99e41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.700+0000 7f99eb200640 1 -- 172.21.15.38:0/3542948457 >> 172.21.15.38:0/3542948457 conn(0x7f99e40fa930 msgr2=0x7f99e4100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:46.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.701+0000 7f99eb200640 1 -- 172.21.15.38:0/3542948457 shutdown_connections 2024-03-24T01:03:46.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.701+0000 7f99eb200640 1 -- 172.21.15.38:0/3542948457 wait complete. 2024-03-24T01:03:46.719 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-24T01:03:46.720 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-03-24T01:03:46.720 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-24T01:03:46.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:46 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3949970537' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-24T01:03:46.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:46 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3894392877' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-24T01:03:46.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:46 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T01:03:46.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:46 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T01:03:46.860 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:46 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' 2024-03-24T01:03:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.930+0000 7f10a9a00640 1 -- 172.21.15.38:0/1849896906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a4102b70 msgr2=0x7f10a4106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.930+0000 7f10a9a00640 1 --2- 172.21.15.38:0/1849896906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a4102b70 0x7f10a4106bb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f108c009880 tx=0x7f108c02bce0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.930+0000 7f10a9a00640 1 -- 172.21.15.38:0/1849896906 shutdown_connections 2024-03-24T01:03:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.930+0000 7f10a9a00640 1 --2- 172.21.15.38:0/1849896906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a4102b70 0x7f10a4106bb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.930+0000 7f10a9a00640 1 --2- 172.21.15.38:0/1849896906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10a41021c0 0x7f10a41025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.930+0000 7f10a9a00640 1 -- 172.21.15.38:0/1849896906 >> 172.21.15.38:0/1849896906 conn(0x7f10a40fda80 msgr2=0x7f10a40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.931+0000 7f10a9a00640 1 -- 172.21.15.38:0/1849896906 shutdown_connections 2024-03-24T01:03:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.931+0000 7f10a9a00640 1 -- 172.21.15.38:0/1849896906 wait complete. 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.932+0000 7f10a9a00640 1 Processor -- start 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.932+0000 7f10a9a00640 1 -- start start 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.932+0000 7f10a9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10a41021c0 0x7f10a4198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.932+0000 7f10a9a00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a4198580 0x7f10a419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.932+0000 7f10a9a00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a4198b80 con 0x7f10a4198580 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.932+0000 7f10a9a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a4198cf0 con 0x7f10a41021c0 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.932+0000 7f109be00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a4198580 0x7f10a419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.933+0000 7f109be00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a4198580 0x7f10a419c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34248/0 (socket says 172.21.15.38:34248) 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.933+0000 7f109be00640 1 -- 172.21.15.38:0/278468234 learned_addr learned my addr 172.21.15.38:0/278468234 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.933+0000 7f10a3e00640 1 --2- 172.21.15.38:0/278468234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10a41021c0 0x7f10a4198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.933+0000 7f109be00640 1 -- 172.21.15.38:0/278468234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10a41021c0 msgr2=0x7f10a4198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.933+0000 7f109be00640 1 --2- 172.21.15.38:0/278468234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10a41021c0 0x7f10a4198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.933+0000 7f109be00640 1 -- 172.21.15.38:0/278468234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c009560 con 0x7f10a4198580 2024-03-24T01:03:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.933+0000 7f109be00640 1 --2- 172.21.15.38:0/278468234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a4198580 0x7f10a419c920 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f108c036040 tx=0x7f108c02c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.934+0000 7f10a1600640 1 -- 172.21.15.38:0/278468234 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108c0035f0 con 0x7f10a4198580 2024-03-24T01:03:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.934+0000 7f10a9a00640 1 -- 172.21.15.38:0/278468234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a419cec0 con 0x7f10a4198580 2024-03-24T01:03:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.934+0000 7f10a1600640 1 -- 172.21.15.38:0/278468234 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f108c003c10 con 0x7f10a4198580 2024-03-24T01:03:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.934+0000 7f10a1600640 1 -- 172.21.15.38:0/278468234 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108c0055f0 con 0x7f10a4198580 2024-03-24T01:03:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.934+0000 7f10a9a00640 1 -- 172.21.15.38:0/278468234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a419d420 con 0x7f10a4198580 2024-03-24T01:03:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.935+0000 7f10a9a00640 1 -- 172.21.15.38:0/278468234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a4103690 con 0x7f10a4198580 2024-03-24T01:03:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.936+0000 7f10a1600640 1 -- 172.21.15.38:0/278468234 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f108c004e60 con 0x7f10a4198580 2024-03-24T01:03:46.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.937+0000 7f10a1600640 1 --2- 172.21.15.38:0/278468234 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f108806df20 0x7f10880703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.938+0000 7f10a1600640 1 -- 172.21.15.38:0/278468234 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f108c0b00f0 con 0x7f10a4198580 2024-03-24T01:03:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.938+0000 7f10a3e00640 1 --2- 172.21.15.38:0/278468234 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f108806df20 0x7f10880703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:46.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.939+0000 7f10a3e00640 1 --2- 172.21.15.38:0/278468234 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f108806df20 0x7f10880703e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1094005fd0 tx=0x7f1094005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:46.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:46.942+0000 7f10a1600640 1 -- 172.21.15.38:0/278468234 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f108c0794f0 con 0x7f10a4198580 2024-03-24T01:03:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.260+0000 7f10a9a00640 1 -- 172.21.15.38:0/278468234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f10a419d700 con 0x7f10a4198580 2024-03-24T01:03:47.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.261+0000 7f10a1600640 1 -- 172.21.15.38:0/278468234 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v40) v1 ==== 107+0+4988 (secure 0 0 0) 0x7f108c07cbb0 con 0x7f10a4198580 2024-03-24T01:03:47.261 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:47.261 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44373,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/1135457323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1135457323},{"type":"v1","addr":"172.21.15.88:6827","nonce":1135457323}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T01:01:05.149559+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:47.261 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 32 2024-03-24T01:03:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.264+0000 7f10a9a00640 1 -- 172.21.15.38:0/278468234 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f108806df20 msgr2=0x7f10880703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.264+0000 7f10a9a00640 1 --2- 172.21.15.38:0/278468234 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f108806df20 0x7f10880703e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1094005fd0 tx=0x7f1094005850 comp rx=0 tx=0).stop 2024-03-24T01:03:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.264+0000 7f10a9a00640 1 -- 172.21.15.38:0/278468234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a4198580 msgr2=0x7f10a419c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.264+0000 7f10a9a00640 1 --2- 172.21.15.38:0/278468234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a4198580 0x7f10a419c920 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f108c036040 tx=0x7f108c02c7c0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.264+0000 7f10a9a00640 1 -- 172.21.15.38:0/278468234 shutdown_connections 2024-03-24T01:03:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.264+0000 7f10a9a00640 1 --2- 172.21.15.38:0/278468234 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f108806df20 0x7f10880703e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.264+0000 7f10a9a00640 1 --2- 172.21.15.38:0/278468234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10a4198580 0x7f10a419c920 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.264+0000 7f10a9a00640 1 --2- 172.21.15.38:0/278468234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10a41021c0 0x7f10a4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.265+0000 7f10a9a00640 1 -- 172.21.15.38:0/278468234 >> 172.21.15.38:0/278468234 conn(0x7f10a40fda80 msgr2=0x7f10a4105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.265+0000 7f10a9a00640 1 -- 172.21.15.38:0/278468234 shutdown_connections 2024-03-24T01:03:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.265+0000 7f10a9a00640 1 -- 172.21.15.38:0/278468234 wait complete. 2024-03-24T01:03:47.286 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-03-24T01:03:47.287 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-24T01:03:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.524+0000 7fec3fe00640 1 -- 172.21.15.38:0/1174364022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec40102b90 msgr2=0x7fec40106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.524+0000 7fec3fe00640 1 --2- 172.21.15.38:0/1174364022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec40102b90 0x7fec40106bd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009880 tx=0x7fec2c02bcf0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.524+0000 7fec3fe00640 1 -- 172.21.15.38:0/1174364022 shutdown_connections 2024-03-24T01:03:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.524+0000 7fec3fe00640 1 --2- 172.21.15.38:0/1174364022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec40102b90 0x7fec40106bd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.524+0000 7fec3fe00640 1 --2- 172.21.15.38:0/1174364022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec401021e0 0x7fec401025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.524+0000 7fec3fe00640 1 -- 172.21.15.38:0/1174364022 >> 172.21.15.38:0/1174364022 conn(0x7fec400fda40 msgr2=0x7fec400ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.524+0000 7fec3fe00640 1 -- 172.21.15.38:0/1174364022 shutdown_connections 2024-03-24T01:03:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.524+0000 7fec3fe00640 1 -- 172.21.15.38:0/1174364022 wait complete. 2024-03-24T01:03:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.525+0000 7fec3fe00640 1 Processor -- start 2024-03-24T01:03:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.525+0000 7fec3fe00640 1 -- start start 2024-03-24T01:03:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.526+0000 7fec3fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec401021e0 0x7fec40198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.526+0000 7fec3fe00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec40198560 0x7fec4019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.526+0000 7fec3fe00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec40198ad0 con 0x7fec40198560 2024-03-24T01:03:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.526+0000 7fec3fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec40198c40 con 0x7fec401021e0 2024-03-24T01:03:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.526+0000 7fec3e000640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec40198560 0x7fec4019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.526+0000 7fec3e000640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec40198560 0x7fec4019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34266/0 (socket says 172.21.15.38:34266) 2024-03-24T01:03:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.526+0000 7fec3e000640 1 -- 172.21.15.38:0/1324999807 learned_addr learned my addr 172.21.15.38:0/1324999807 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.526+0000 7fec3ea00640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec401021e0 0x7fec40198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.526+0000 7fec3e000640 1 -- 172.21.15.38:0/1324999807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec401021e0 msgr2=0x7fec40198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.527+0000 7fec3e000640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec401021e0 0x7fec40198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.527+0000 7fec3e000640 1 -- 172.21.15.38:0/1324999807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c009560 con 0x7fec40198560 2024-03-24T01:03:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.527+0000 7fec3ea00640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec401021e0 0x7fec40198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T01:03:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.527+0000 7fec3e000640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec40198560 0x7fec4019c900 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fec2c02ceb0 tx=0x7fec2c02c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.527+0000 7fec44a00640 1 -- 172.21.15.38:0/1324999807 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2c0035f0 con 0x7fec40198560 2024-03-24T01:03:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.527+0000 7fec3fe00640 1 -- 172.21.15.38:0/1324999807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4019cea0 con 0x7fec40198560 2024-03-24T01:03:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.527+0000 7fec44a00640 1 -- 172.21.15.38:0/1324999807 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec2c003c10 con 0x7fec40198560 2024-03-24T01:03:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.527+0000 7fec44a00640 1 -- 172.21.15.38:0/1324999807 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2c005690 con 0x7fec40198560 2024-03-24T01:03:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.528+0000 7fec3fe00640 1 -- 172.21.15.38:0/1324999807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4019d3b0 con 0x7fec40198560 2024-03-24T01:03:47.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.529+0000 7fec3fe00640 1 -- 172.21.15.38:0/1324999807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec401036b0 con 0x7fec40198560 2024-03-24T01:03:47.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.531+0000 7fec44a00640 1 -- 172.21.15.38:0/1324999807 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fec2c004ed0 con 0x7fec40198560 2024-03-24T01:03:47.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.532+0000 7fec44a00640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fec2406dec0 0x7fec24070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:47.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.532+0000 7fec44a00640 1 -- 172.21.15.38:0/1324999807 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fec2c0b01f0 con 0x7fec40198560 2024-03-24T01:03:47.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.532+0000 7fec3ea00640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fec2406dec0 0x7fec24070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:47.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.533+0000 7fec3ea00640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fec2406dec0 0x7fec24070380 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fec3000baa0 tx=0x7fec30005d90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:47.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.537+0000 7fec44a00640 1 -- 172.21.15.38:0/1324999807 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec2c079610 con 0x7fec40198560 2024-03-24T01:03:47.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.810+0000 7fec3fe00640 1 -- 172.21.15.38:0/1324999807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fec40199320 con 0x7fec40198560 2024-03-24T01:03:47.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.811+0000 7fec44a00640 1 -- 172.21.15.38:0/1324999807 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v40) v1 ==== 107+0+5045 (secure 0 0 0) 0x7fec2c07ccd0 con 0x7fec40198560 2024-03-24T01:03:47.811 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:47.811 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44373,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/1135457323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1135457323},{"type":"v1","addr":"172.21.15.88:6827","nonce":1135457323}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T01:01:06.148625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34446,"mds_1":34478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34446":{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34478":{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":1,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:47.812 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 33 2024-03-24T01:03:47.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.815+0000 7fec3fe00640 1 -- 172.21.15.38:0/1324999807 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fec2406dec0 msgr2=0x7fec24070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:47.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.815+0000 7fec3fe00640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fec2406dec0 0x7fec24070380 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fec3000baa0 tx=0x7fec30005d90 comp rx=0 tx=0).stop 2024-03-24T01:03:47.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.815+0000 7fec3fe00640 1 -- 172.21.15.38:0/1324999807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec40198560 msgr2=0x7fec4019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:47.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.815+0000 7fec3fe00640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec40198560 0x7fec4019c900 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fec2c02ceb0 tx=0x7fec2c02c7d0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.815+0000 7fec3fe00640 1 -- 172.21.15.38:0/1324999807 shutdown_connections 2024-03-24T01:03:47.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.815+0000 7fec3fe00640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fec2406dec0 0x7fec24070380 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.815+0000 7fec3fe00640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec40198560 0x7fec4019c900 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.815+0000 7fec3fe00640 1 --2- 172.21.15.38:0/1324999807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec401021e0 0x7fec40198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:47.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.815+0000 7fec3fe00640 1 -- 172.21.15.38:0/1324999807 >> 172.21.15.38:0/1324999807 conn(0x7fec400fda40 msgr2=0x7fec400fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:47.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.815+0000 7fec3fe00640 1 -- 172.21.15.38:0/1324999807 shutdown_connections 2024-03-24T01:03:47.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:47.815+0000 7fec3fe00640 1 -- 172.21.15.38:0/1324999807 wait complete. 2024-03-24T01:03:47.831 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:47 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3542948457' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-24T01:03:47.831 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:47 smithi038 ceph-mon[112108]: pgmap v335: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:03:47.831 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:47 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/278468234' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-24T01:03:47.834 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-03-24T01:03:47.835 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-24T01:03:47.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:47 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3542948457' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-24T01:03:47.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:47 smithi088 ceph-mon[87351]: pgmap v335: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-24T01:03:47.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:47 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/278468234' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-24T01:03:48.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.039+0000 7f86cb200640 1 -- 172.21.15.38:0/855145194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc101820 msgr2=0x7f86cc109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.039+0000 7f86cb200640 1 --2- 172.21.15.38:0/855145194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc101820 0x7f86cc109d00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009880 tx=0x7f86b802bd90 comp rx=0 tx=0).stop 2024-03-24T01:03:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.039+0000 7f86cb200640 1 -- 172.21.15.38:0/855145194 shutdown_connections 2024-03-24T01:03:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.039+0000 7f86cb200640 1 --2- 172.21.15.38:0/855145194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc101820 0x7f86cc109d00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.039+0000 7f86cb200640 1 --2- 172.21.15.38:0/855145194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86cc100f00 0x7f86cc1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.039+0000 7f86cb200640 1 -- 172.21.15.38:0/855145194 >> 172.21.15.38:0/855145194 conn(0x7f86cc0758e0 msgr2=0x7f86cc0fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:48.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.040+0000 7f86cb200640 1 -- 172.21.15.38:0/855145194 shutdown_connections 2024-03-24T01:03:48.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.040+0000 7f86cb200640 1 -- 172.21.15.38:0/855145194 wait complete. 2024-03-24T01:03:48.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.040+0000 7f86cb200640 1 Processor -- start 2024-03-24T01:03:48.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.041+0000 7f86cb200640 1 -- start start 2024-03-24T01:03:48.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.041+0000 7f86cb200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc100f00 0x7f86cc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:48.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.041+0000 7f86cb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86cc101820 0x7f86cc198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:48.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.041+0000 7f86cb200640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc198bb0 con 0x7f86cc100f00 2024-03-24T01:03:48.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.041+0000 7f86cb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc19c8c0 con 0x7f86cc101820 2024-03-24T01:03:48.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.041+0000 7f86c9e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc100f00 0x7f86cc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:48.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.041+0000 7f86c9e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc100f00 0x7f86cc197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34292/0 (socket says 172.21.15.38:34292) 2024-03-24T01:03:48.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.041+0000 7f86c9e00640 1 -- 172.21.15.38:0/3369591313 learned_addr learned my addr 172.21.15.38:0/3369591313 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:48.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.042+0000 7f86c9e00640 1 -- 172.21.15.38:0/3369591313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86cc101820 msgr2=0x7f86cc198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:48.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.042+0000 7f86c9e00640 1 --2- 172.21.15.38:0/3369591313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86cc101820 0x7f86cc198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.042+0000 7f86c9e00640 1 -- 172.21.15.38:0/3369591313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b8009560 con 0x7f86cc100f00 2024-03-24T01:03:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.042+0000 7f86c9e00640 1 --2- 172.21.15.38:0/3369591313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc100f00 0x7f86cc197fe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00d6f0 tx=0x7f86bc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.042+0000 7f86c2a00640 1 -- 172.21.15.38:0/3369591313 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86bc009a30 con 0x7f86cc100f00 2024-03-24T01:03:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.042+0000 7f86cb200640 1 -- 172.21.15.38:0/3369591313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc19cba0 con 0x7f86cc100f00 2024-03-24T01:03:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.042+0000 7f86c2a00640 1 -- 172.21.15.38:0/3369591313 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86bc0054e0 con 0x7f86cc100f00 2024-03-24T01:03:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.042+0000 7f86c2a00640 1 -- 172.21.15.38:0/3369591313 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86bc010460 con 0x7f86cc100f00 2024-03-24T01:03:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.043+0000 7f86cb200640 1 -- 172.21.15.38:0/3369591313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc19d100 con 0x7f86cc100f00 2024-03-24T01:03:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.044+0000 7f86cb200640 1 -- 172.21.15.38:0/3369591313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86cc067b20 con 0x7f86cc100f00 2024-03-24T01:03:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.045+0000 7f86c2a00640 1 -- 172.21.15.38:0/3369591313 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f86bc004ec0 con 0x7f86cc100f00 2024-03-24T01:03:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.045+0000 7f86c2a00640 1 --2- 172.21.15.38:0/3369591313 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f86b006df10 0x7f86b00703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.046+0000 7f86c2a00640 1 -- 172.21.15.38:0/3369591313 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f86bc08da50 con 0x7f86cc100f00 2024-03-24T01:03:48.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.046+0000 7f86c9400640 1 --2- 172.21.15.38:0/3369591313 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f86b006df10 0x7f86b00703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:48.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.047+0000 7f86c9400640 1 --2- 172.21.15.38:0/3369591313 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f86b006df10 0x7f86b00703d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86b8036040 tx=0x7f86b8005a20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:48.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.051+0000 7f86c2a00640 1 -- 172.21.15.38:0/3369591313 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f86bc056dd0 con 0x7f86cc100f00 2024-03-24T01:03:48.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.303+0000 7f86cb200640 1 -- 172.21.15.38:0/3369591313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f86cc04e880 con 0x7f86cc100f00 2024-03-24T01:03:48.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.304+0000 7f86c2a00640 1 -- 172.21.15.38:0/3369591313 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v40) v1 ==== 107+0+5046 (secure 0 0 0) 0x7f86bc05a490 con 0x7f86cc100f00 2024-03-24T01:03:48.305 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:48.305 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44373,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/1135457323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1135457323},{"type":"v1","addr":"172.21.15.88:6827","nonce":1135457323}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T01:01:06.171575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34446,"mds_1":34478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34446":{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34478":{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":1,"incarnation":33,"state":"up:resolve","state_seq":17,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:48.305 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 34 2024-03-24T01:03:48.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.308+0000 7f86cb200640 1 -- 172.21.15.38:0/3369591313 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f86b006df10 msgr2=0x7f86b00703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:48.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.308+0000 7f86cb200640 1 --2- 172.21.15.38:0/3369591313 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f86b006df10 0x7f86b00703d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86b8036040 tx=0x7f86b8005a20 comp rx=0 tx=0).stop 2024-03-24T01:03:48.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.308+0000 7f86cb200640 1 -- 172.21.15.38:0/3369591313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc100f00 msgr2=0x7f86cc197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:48.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.308+0000 7f86cb200640 1 --2- 172.21.15.38:0/3369591313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc100f00 0x7f86cc197fe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00d6f0 tx=0x7f86bc00dbb0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.308+0000 7f86cb200640 1 -- 172.21.15.38:0/3369591313 shutdown_connections 2024-03-24T01:03:48.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.308+0000 7f86cb200640 1 --2- 172.21.15.38:0/3369591313 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f86b006df10 0x7f86b00703d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.308+0000 7f86cb200640 1 --2- 172.21.15.38:0/3369591313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86cc101820 0x7f86cc198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.308+0000 7f86cb200640 1 --2- 172.21.15.38:0/3369591313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc100f00 0x7f86cc197fe0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.308+0000 7f86cb200640 1 -- 172.21.15.38:0/3369591313 >> 172.21.15.38:0/3369591313 conn(0x7f86cc0758e0 msgr2=0x7f86cc1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:48.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.308+0000 7f86cb200640 1 -- 172.21.15.38:0/3369591313 shutdown_connections 2024-03-24T01:03:48.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.308+0000 7f86cb200640 1 -- 172.21.15.38:0/3369591313 wait complete. 2024-03-24T01:03:48.329 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-24T01:03:48.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.526+0000 7f6036a00640 1 -- 172.21.15.38:0/2920080610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6030101820 msgr2=0x7f6030109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:48.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.526+0000 7f6036a00640 1 --2- 172.21.15.38:0/2920080610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6030101820 0x7f6030109d00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f601c009880 tx=0x7f601c02bd70 comp rx=0 tx=0).stop 2024-03-24T01:03:48.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.526+0000 7f6036a00640 1 -- 172.21.15.38:0/2920080610 shutdown_connections 2024-03-24T01:03:48.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.526+0000 7f6036a00640 1 --2- 172.21.15.38:0/2920080610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6030101820 0x7f6030109d00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.526+0000 7f6036a00640 1 --2- 172.21.15.38:0/2920080610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6030100f00 0x7f60301012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.526+0000 7f6036a00640 1 -- 172.21.15.38:0/2920080610 >> 172.21.15.38:0/2920080610 conn(0x7f60300758e0 msgr2=0x7f60300fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:48.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.526+0000 7f6036a00640 1 -- 172.21.15.38:0/2920080610 shutdown_connections 2024-03-24T01:03:48.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.526+0000 7f6036a00640 1 -- 172.21.15.38:0/2920080610 wait complete. 2024-03-24T01:03:48.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.528+0000 7f6036a00640 1 Processor -- start 2024-03-24T01:03:48.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.528+0000 7f6036a00640 1 -- start start 2024-03-24T01:03:48.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.529+0000 7f6036a00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6030100f00 0x7f60300feb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:48.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.529+0000 7f6036a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6030101820 0x7f60300ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:48.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.529+0000 7f6036a00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300ff5d0 con 0x7f6030100f00 2024-03-24T01:03:48.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.529+0000 7f6036a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300ff740 con 0x7f6030101820 2024-03-24T01:03:48.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.529+0000 7f6035600640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6030100f00 0x7f60300feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:48.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.529+0000 7f6035600640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6030100f00 0x7f60300feb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34314/0 (socket says 172.21.15.38:34314) 2024-03-24T01:03:48.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.529+0000 7f6035600640 1 -- 172.21.15.38:0/1761880052 learned_addr learned my addr 172.21.15.38:0/1761880052 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:48.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.529+0000 7f6034c00640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6030101820 0x7f60300ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:48.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.529+0000 7f6035600640 1 -- 172.21.15.38:0/1761880052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6030101820 msgr2=0x7f60300ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:48.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.529+0000 7f6035600640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6030101820 0x7f60300ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.529+0000 7f6035600640 1 -- 172.21.15.38:0/1761880052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c009560 con 0x7f6030100f00 2024-03-24T01:03:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.530+0000 7f6034c00640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6030101820 0x7f60300ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T01:03:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.530+0000 7f6035600640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6030100f00 0x7f60300feb50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f602400e970 tx=0x7f602400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.530+0000 7f602e000640 1 -- 172.21.15.38:0/1761880052 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f602400cc90 con 0x7f6030100f00 2024-03-24T01:03:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.530+0000 7f602e000640 1 -- 172.21.15.38:0/1761880052 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f602400cdf0 con 0x7f6030100f00 2024-03-24T01:03:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.530+0000 7f6036a00640 1 -- 172.21.15.38:0/1761880052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60301a1f80 con 0x7f6030100f00 2024-03-24T01:03:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.530+0000 7f6036a00640 1 -- 172.21.15.38:0/1761880052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60301a22f0 con 0x7f6030100f00 2024-03-24T01:03:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.531+0000 7f602e000640 1 -- 172.21.15.38:0/1761880052 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6024010430 con 0x7f6030100f00 2024-03-24T01:03:48.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.532+0000 7f6036a00640 1 -- 172.21.15.38:0/1761880052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6030067b20 con 0x7f6030100f00 2024-03-24T01:03:48.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.534+0000 7f602e000640 1 -- 172.21.15.38:0/1761880052 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f60240046c0 con 0x7f6030100f00 2024-03-24T01:03:48.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.535+0000 7f602e000640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f601806df10 0x7f60180703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:48.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.535+0000 7f602e000640 1 -- 172.21.15.38:0/1761880052 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6024014070 con 0x7f6030100f00 2024-03-24T01:03:48.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.535+0000 7f6034c00640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f601806df10 0x7f60180703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:48.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.536+0000 7f6034c00640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f601806df10 0x7f60180703d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f601c005ec0 tx=0x7f601c02caf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:48.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.540+0000 7f602e000640 1 -- 172.21.15.38:0/1761880052 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f60240577d0 con 0x7f6030100f00 2024-03-24T01:03:48.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:48 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1324999807' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-24T01:03:48.863 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:48 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3369591313' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-24T01:03:48.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.925+0000 7f6036a00640 1 -- 172.21.15.38:0/1761880052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f60300ff900 con 0x7f6030100f00 2024-03-24T01:03:48.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.926+0000 7f602e000640 1 -- 172.21.15.38:0/1761880052 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v40) v1 ==== 107+0+5048 (secure 0 0 0) 0x7f602405ae90 con 0x7f6030100f00 2024-03-24T01:03:48.927 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:48.927 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44373,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/1135457323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1135457323},{"type":"v1","addr":"172.21.15.88:6827","nonce":1135457323}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T01:01:14.830239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34446,"mds_1":34478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34446":{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":0,"incarnation":33,"state":"up:resolve","state_seq":26,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34478":{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":1,"incarnation":33,"state":"up:resolve","state_seq":17,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:48.927 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 35 2024-03-24T01:03:48.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.930+0000 7f6036a00640 1 -- 172.21.15.38:0/1761880052 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f601806df10 msgr2=0x7f60180703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:48.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.930+0000 7f6036a00640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f601806df10 0x7f60180703d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f601c005ec0 tx=0x7f601c02caf0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.930+0000 7f6036a00640 1 -- 172.21.15.38:0/1761880052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6030100f00 msgr2=0x7f60300feb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:48.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.930+0000 7f6036a00640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6030100f00 0x7f60300feb50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f602400e970 tx=0x7f602400ed70 comp rx=0 tx=0).stop 2024-03-24T01:03:48.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.930+0000 7f6036a00640 1 -- 172.21.15.38:0/1761880052 shutdown_connections 2024-03-24T01:03:48.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.930+0000 7f6036a00640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f601806df10 0x7f60180703d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.930+0000 7f6036a00640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6030101820 0x7f60300ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.930+0000 7f6036a00640 1 --2- 172.21.15.38:0/1761880052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6030100f00 0x7f60300feb50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:48.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.930+0000 7f6036a00640 1 -- 172.21.15.38:0/1761880052 >> 172.21.15.38:0/1761880052 conn(0x7f60300758e0 msgr2=0x7f60301021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.930+0000 7f6036a00640 1 -- 172.21.15.38:0/1761880052 shutdown_connections 2024-03-24T01:03:48.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:48.930+0000 7f6036a00640 1 -- 172.21.15.38:0/1761880052 wait complete. 2024-03-24T01:03:48.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:48 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1324999807' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-24T01:03:48.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:48 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3369591313' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-24T01:03:48.955 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-03-24T01:03:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.166+0000 7f18cfe00640 1 -- 172.21.15.38:0/3698881176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18c8101ab0 msgr2=0x7f18c8109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.166+0000 7f18cfe00640 1 --2- 172.21.15.38:0/3698881176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18c8101ab0 0x7f18c8109f90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009880 tx=0x7f18b402bd90 comp rx=0 tx=0).stop 2024-03-24T01:03:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.166+0000 7f18cfe00640 1 -- 172.21.15.38:0/3698881176 shutdown_connections 2024-03-24T01:03:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.166+0000 7f18cfe00640 1 --2- 172.21.15.38:0/3698881176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18c8101ab0 0x7f18c8109f90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.166+0000 7f18cfe00640 1 --2- 172.21.15.38:0/3698881176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18c8101190 0x7f18c8101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.166+0000 7f18cfe00640 1 -- 172.21.15.38:0/3698881176 >> 172.21.15.38:0/3698881176 conn(0x7f18c80fa930 msgr2=0x7f18c80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.166+0000 7f18cfe00640 1 -- 172.21.15.38:0/3698881176 shutdown_connections 2024-03-24T01:03:49.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.166+0000 7f18cfe00640 1 -- 172.21.15.38:0/3698881176 wait complete. 2024-03-24T01:03:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.167+0000 7f18cfe00640 1 Processor -- start 2024-03-24T01:03:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.167+0000 7f18cfe00640 1 -- start start 2024-03-24T01:03:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.168+0000 7f18cfe00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18c8101190 0x7f18c81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.168+0000 7f18cfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18c8101ab0 0x7f18c8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.168+0000 7f18cfe00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8198da0 con 0x7f18c8101190 2024-03-24T01:03:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.168+0000 7f18cfe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c819cab0 con 0x7f18c8101ab0 2024-03-24T01:03:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.168+0000 7f18cd400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18c8101190 0x7f18c81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.168+0000 7f18cd400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18c8101190 0x7f18c81981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34340/0 (socket says 172.21.15.38:34340) 2024-03-24T01:03:49.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.168+0000 7f18cd400640 1 -- 172.21.15.38:0/1471798415 learned_addr learned my addr 172.21.15.38:0/1471798415 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:49.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.169+0000 7f18cd400640 1 -- 172.21.15.38:0/1471798415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18c8101ab0 msgr2=0x7f18c8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:49.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.169+0000 7f18cd400640 1 --2- 172.21.15.38:0/1471798415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18c8101ab0 0x7f18c8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:49.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.169+0000 7f18cd400640 1 -- 172.21.15.38:0/1471798415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b4009560 con 0x7f18c8101190 2024-03-24T01:03:49.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.169+0000 7f18cd400640 1 --2- 172.21.15.38:0/1471798415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18c8101190 0x7f18c81981d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00d6f0 tx=0x7f18bc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:49.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.169+0000 7f18c6000640 1 -- 172.21.15.38:0/1471798415 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18bc009a30 con 0x7f18c8101190 2024-03-24T01:03:49.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.170+0000 7f18c6000640 1 -- 172.21.15.38:0/1471798415 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18bc0054e0 con 0x7f18c8101190 2024-03-24T01:03:49.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.170+0000 7f18c6000640 1 -- 172.21.15.38:0/1471798415 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18bc010460 con 0x7f18c8101190 2024-03-24T01:03:49.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.170+0000 7f18cfe00640 1 -- 172.21.15.38:0/1471798415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c819cd90 con 0x7f18c8101190 2024-03-24T01:03:49.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.170+0000 7f18cfe00640 1 -- 172.21.15.38:0/1471798415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c819d2f0 con 0x7f18c8101190 2024-03-24T01:03:49.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.171+0000 7f18cfe00640 1 -- 172.21.15.38:0/1471798415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c8068610 con 0x7f18c8101190 2024-03-24T01:03:49.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.172+0000 7f18c6000640 1 -- 172.21.15.38:0/1471798415 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f18bc009b90 con 0x7f18c8101190 2024-03-24T01:03:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.174+0000 7f18c6000640 1 --2- 172.21.15.38:0/1471798415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18b006df10 0x7f18b00703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.174+0000 7f18c6000640 1 -- 172.21.15.38:0/1471798415 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f18bc08d8f0 con 0x7f18c8101190 2024-03-24T01:03:49.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.174+0000 7f18cca00640 1 --2- 172.21.15.38:0/1471798415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18b006df10 0x7f18b00703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:49.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.175+0000 7f18cca00640 1 --2- 172.21.15.38:0/1471798415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18b006df10 0x7f18b00703d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f18b4036040 tx=0x7f18b4000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:49.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.180+0000 7f18c6000640 1 -- 172.21.15.38:0/1471798415 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f18bc056cf0 con 0x7f18c8101190 2024-03-24T01:03:49.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.519+0000 7f18cfe00640 1 -- 172.21.15.38:0/1471798415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f18c804e030 con 0x7f18c8101190 2024-03-24T01:03:49.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.520+0000 7f18c6000640 1 -- 172.21.15.38:0/1471798415 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v40) v1 ==== 107+0+5052 (secure 0 0 0) 0x7f18bc05a3b0 con 0x7f18c8101190 2024-03-24T01:03:49.520 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:49.520 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44373,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/1135457323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1135457323},{"type":"v1","addr":"172.21.15.88:6827","nonce":1135457323}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T01:01:15.314740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34446,"mds_1":34478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34446":{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34478":{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":1,"incarnation":33,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:49.520 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 36 2024-03-24T01:03:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.522+0000 7f18cfe00640 1 -- 172.21.15.38:0/1471798415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18b006df10 msgr2=0x7f18b00703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.522+0000 7f18cfe00640 1 --2- 172.21.15.38:0/1471798415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18b006df10 0x7f18b00703d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f18b4036040 tx=0x7f18b4000bc0 comp rx=0 tx=0).stop 2024-03-24T01:03:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.522+0000 7f18cfe00640 1 -- 172.21.15.38:0/1471798415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18c8101190 msgr2=0x7f18c81981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.522+0000 7f18cfe00640 1 --2- 172.21.15.38:0/1471798415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18c8101190 0x7f18c81981d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00d6f0 tx=0x7f18bc00dbb0 comp rx=0 tx=0).stop 2024-03-24T01:03:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.522+0000 7f18cfe00640 1 -- 172.21.15.38:0/1471798415 shutdown_connections 2024-03-24T01:03:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.522+0000 7f18cfe00640 1 --2- 172.21.15.38:0/1471798415 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18b006df10 0x7f18b00703d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.522+0000 7f18cfe00640 1 --2- 172.21.15.38:0/1471798415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18c8101ab0 0x7f18c8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.522+0000 7f18cfe00640 1 --2- 172.21.15.38:0/1471798415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18c8101190 0x7f18c81981d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.522+0000 7f18cfe00640 1 -- 172.21.15.38:0/1471798415 >> 172.21.15.38:0/1471798415 conn(0x7f18c80fa930 msgr2=0x7f18c8109af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.523+0000 7f18cfe00640 1 -- 172.21.15.38:0/1471798415 shutdown_connections 2024-03-24T01:03:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.523+0000 7f18cfe00640 1 -- 172.21.15.38:0/1471798415 wait complete. 2024-03-24T01:03:49.541 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-03-24T01:03:49.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.760+0000 7fe6b6c00640 1 -- 172.21.15.38:0/2891208325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b0100f00 msgr2=0x7fe6b01012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.760+0000 7fe6b6c00640 1 --2- 172.21.15.38:0/2891208325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b0100f00 0x7fe6b01012e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009c20 tx=0x7fe6a402c230 comp rx=0 tx=0).stop 2024-03-24T01:03:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.760+0000 7fe6b6c00640 1 -- 172.21.15.38:0/2891208325 shutdown_connections 2024-03-24T01:03:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.760+0000 7fe6b6c00640 1 --2- 172.21.15.38:0/2891208325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b0101820 0x7fe6b0109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.760+0000 7fe6b6c00640 1 --2- 172.21.15.38:0/2891208325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b0100f00 0x7fe6b01012e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.760+0000 7fe6b6c00640 1 -- 172.21.15.38:0/2891208325 >> 172.21.15.38:0/2891208325 conn(0x7fe6b00758e0 msgr2=0x7fe6b00fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.760+0000 7fe6b6c00640 1 -- 172.21.15.38:0/2891208325 shutdown_connections 2024-03-24T01:03:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.760+0000 7fe6b6c00640 1 -- 172.21.15.38:0/2891208325 wait complete. 2024-03-24T01:03:49.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.761+0000 7fe6b6c00640 1 Processor -- start 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.761+0000 7fe6b6c00640 1 -- start start 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.762+0000 7fe6b6c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b0100f00 0x7fe6b00feb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.762+0000 7fe6b6c00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b0101820 0x7fe6b00ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.762+0000 7fe6b6c00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00ff5d0 con 0x7fe6b0101820 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.762+0000 7fe6b6c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00ff740 con 0x7fe6b0100f00 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.762+0000 7fe6b4e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b0101820 0x7fe6b00ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.762+0000 7fe6b4e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b0101820 0x7fe6b00ff090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34356/0 (socket says 172.21.15.38:34356) 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.762+0000 7fe6b4e00640 1 -- 172.21.15.38:0/882520025 learned_addr learned my addr 172.21.15.38:0/882520025 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.763+0000 7fe6b5800640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b0100f00 0x7fe6b00feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.763+0000 7fe6b4e00640 1 -- 172.21.15.38:0/882520025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b0100f00 msgr2=0x7fe6b00feb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.763+0000 7fe6b4e00640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b0100f00 0x7fe6b00feb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:49.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.763+0000 7fe6b4e00640 1 -- 172.21.15.38:0/882520025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a4009900 con 0x7fe6b0101820 2024-03-24T01:03:49.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.763+0000 7fe6b5800640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b0100f00 0x7fe6b00feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T01:03:49.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.763+0000 7fe6b4e00640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b0101820 0x7fe6b00ff090 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00c8f0 tx=0x7fe69c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:49.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.763+0000 7fe6ae000640 1 -- 172.21.15.38:0/882520025 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe69c007b80 con 0x7fe6b0101820 2024-03-24T01:03:49.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.763+0000 7fe6ae000640 1 -- 172.21.15.38:0/882520025 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe69c00ae50 con 0x7fe6b0101820 2024-03-24T01:03:49.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.763+0000 7fe6ae000640 1 -- 172.21.15.38:0/882520025 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe69c004e50 con 0x7fe6b0101820 2024-03-24T01:03:49.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.763+0000 7fe6b6c00640 1 -- 172.21.15.38:0/882520025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b01a1fc0 con 0x7fe6b0101820 2024-03-24T01:03:49.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.764+0000 7fe6b6c00640 1 -- 172.21.15.38:0/882520025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b01a2330 con 0x7fe6b0101820 2024-03-24T01:03:49.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.765+0000 7fe6b6c00640 1 -- 172.21.15.38:0/882520025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b0067b20 con 0x7fe6b0101820 2024-03-24T01:03:49.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.767+0000 7fe6ae000640 1 -- 172.21.15.38:0/882520025 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fe69c005020 con 0x7fe6b0101820 2024-03-24T01:03:49.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.769+0000 7fe6ae000640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe69806df20 0x7fe6980703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:49.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.769+0000 7fe6ae000640 1 -- 172.21.15.38:0/882520025 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe69c08db70 con 0x7fe6b0101820 2024-03-24T01:03:49.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.769+0000 7fe6b5800640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe69806df20 0x7fe6980703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:49.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.770+0000 7fe6b5800640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe69806df20 0x7fe6980703e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4037040 tx=0x7fe6a40093f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:49.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:49.774+0000 7fe6ae000640 1 -- 172.21.15.38:0/882520025 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe69c056f70 con 0x7fe6b0101820 2024-03-24T01:03:49.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:49 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1761880052' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-24T01:03:49.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:49 smithi038 ceph-mon[112108]: pgmap v336: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:03:49.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:49 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1471798415' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-24T01:03:49.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:49 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1761880052' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-24T01:03:49.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:49 smithi088 ceph-mon[87351]: pgmap v336: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-24T01:03:49.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:49 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1471798415' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-24T01:03:50.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.054+0000 7fe6b6c00640 1 -- 172.21.15.38:0/882520025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7fe6b00ff900 con 0x7fe6b0101820 2024-03-24T01:03:50.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.055+0000 7fe6ae000640 1 -- 172.21.15.38:0/882520025 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v40) v1 ==== 107+0+5046 (secure 0 0 0) 0x7fe6b00ff900 con 0x7fe6b0101820 2024-03-24T01:03:50.055 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:50.055 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44373,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/1135457323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1135457323},{"type":"v1","addr":"172.21.15.88:6827","nonce":1135457323}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T01:01:16.237858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34446,"mds_1":34478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34446":{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34478":{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":1,"incarnation":33,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T01:03:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 37 2024-03-24T01:03:50.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.059+0000 7fe6b6c00640 1 -- 172.21.15.38:0/882520025 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe69806df20 msgr2=0x7fe6980703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:50.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.059+0000 7fe6b6c00640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe69806df20 0x7fe6980703e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4037040 tx=0x7fe6a40093f0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.059+0000 7fe6b6c00640 1 -- 172.21.15.38:0/882520025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b0101820 msgr2=0x7fe6b00ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:50.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.059+0000 7fe6b6c00640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b0101820 0x7fe6b00ff090 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00c8f0 tx=0x7fe69c00cdb0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.059+0000 7fe6b6c00640 1 -- 172.21.15.38:0/882520025 shutdown_connections 2024-03-24T01:03:50.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.059+0000 7fe6b6c00640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fe69806df20 0x7fe6980703e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.059+0000 7fe6b6c00640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6b0101820 0x7fe6b00ff090 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.059+0000 7fe6b6c00640 1 --2- 172.21.15.38:0/882520025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b0100f00 0x7fe6b00feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.059+0000 7fe6b6c00640 1 -- 172.21.15.38:0/882520025 >> 172.21.15.38:0/882520025 conn(0x7fe6b00758e0 msgr2=0x7fe6b01021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:50.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.059+0000 7fe6b6c00640 1 -- 172.21.15.38:0/882520025 shutdown_connections 2024-03-24T01:03:50.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.059+0000 7fe6b6c00640 1 -- 172.21.15.38:0/882520025 wait complete. 2024-03-24T01:03:50.082 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-03-24T01:03:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.290+0000 7fbedfe00640 1 -- 172.21.15.38:0/3310962350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee00feb40 msgr2=0x7fbee00fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.290+0000 7fbedfe00640 1 --2- 172.21.15.38:0/3310962350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee00feb40 0x7fbee00fef20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009880 tx=0x7fbed002bd30 comp rx=0 tx=0).stop 2024-03-24T01:03:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.290+0000 7fbedfe00640 1 -- 172.21.15.38:0/3310962350 shutdown_connections 2024-03-24T01:03:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.290+0000 7fbedfe00640 1 --2- 172.21.15.38:0/3310962350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee00ff4f0 0x7fbee010c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.290+0000 7fbedfe00640 1 --2- 172.21.15.38:0/3310962350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee00feb40 0x7fbee00fef20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.290+0000 7fbedfe00640 1 -- 172.21.15.38:0/3310962350 >> 172.21.15.38:0/3310962350 conn(0x7fbee00fa770 msgr2=0x7fbee00fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.290+0000 7fbedfe00640 1 -- 172.21.15.38:0/3310962350 shutdown_connections 2024-03-24T01:03:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.290+0000 7fbedfe00640 1 -- 172.21.15.38:0/3310962350 wait complete. 2024-03-24T01:03:50.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.291+0000 7fbedfe00640 1 Processor -- start 2024-03-24T01:03:50.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.291+0000 7fbedfe00640 1 -- start start 2024-03-24T01:03:50.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.292+0000 7fbedfe00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee00feb40 0x7fbee0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:50.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.292+0000 7fbedfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee00ff4f0 0x7fbee0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:50.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.292+0000 7fbedfe00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0198b20 con 0x7fbee00feb40 2024-03-24T01:03:50.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.292+0000 7fbedfe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee019c830 con 0x7fbee00ff4f0 2024-03-24T01:03:50.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.292+0000 7fbedea00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee00feb40 0x7fbee0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:50.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.292+0000 7fbedea00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee00feb40 0x7fbee0197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34374/0 (socket says 172.21.15.38:34374) 2024-03-24T01:03:50.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.292+0000 7fbedea00640 1 -- 172.21.15.38:0/2009434445 learned_addr learned my addr 172.21.15.38:0/2009434445 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:50.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.293+0000 7fbede000640 1 --2- 172.21.15.38:0/2009434445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee00ff4f0 0x7fbee0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:50.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.293+0000 7fbedea00640 1 -- 172.21.15.38:0/2009434445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee00ff4f0 msgr2=0x7fbee0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:50.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.293+0000 7fbedea00640 1 --2- 172.21.15.38:0/2009434445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee00ff4f0 0x7fbee0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.293+0000 7fbedea00640 1 -- 172.21.15.38:0/2009434445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed0009560 con 0x7fbee00feb40 2024-03-24T01:03:50.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.293+0000 7fbedea00640 1 --2- 172.21.15.38:0/2009434445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee00feb40 0x7fbee0197f50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbed002c270 tx=0x7fbed002c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:50.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.293+0000 7fbed7400640 1 -- 172.21.15.38:0/2009434445 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbed00035f0 con 0x7fbee00feb40 2024-03-24T01:03:50.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.294+0000 7fbedfe00640 1 -- 172.21.15.38:0/2009434445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee019cab0 con 0x7fbee00feb40 2024-03-24T01:03:50.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.294+0000 7fbed7400640 1 -- 172.21.15.38:0/2009434445 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbed0003c10 con 0x7fbee00feb40 2024-03-24T01:03:50.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.294+0000 7fbed7400640 1 -- 172.21.15.38:0/2009434445 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbed0005540 con 0x7fbee00feb40 2024-03-24T01:03:50.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.294+0000 7fbedfe00640 1 -- 172.21.15.38:0/2009434445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee019d010 con 0x7fbee00feb40 2024-03-24T01:03:50.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.295+0000 7fbedfe00640 1 -- 172.21.15.38:0/2009434445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee0068a10 con 0x7fbee00feb40 2024-03-24T01:03:50.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.296+0000 7fbed7400640 1 -- 172.21.15.38:0/2009434445 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fbed0003750 con 0x7fbee00feb40 2024-03-24T01:03:50.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.297+0000 7fbed7400640 1 --2- 172.21.15.38:0/2009434445 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbec406df10 0x7fbec40703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:50.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.297+0000 7fbed7400640 1 -- 172.21.15.38:0/2009434445 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbed00b0300 con 0x7fbee00feb40 2024-03-24T01:03:50.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.297+0000 7fbede000640 1 --2- 172.21.15.38:0/2009434445 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbec406df10 0x7fbec40703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:50.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.298+0000 7fbede000640 1 --2- 172.21.15.38:0/2009434445 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbec406df10 0x7fbec40703d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbecc005fd0 tx=0x7fbecc005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:50.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.302+0000 7fbed7400640 1 -- 172.21.15.38:0/2009434445 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbed0079680 con 0x7fbee00feb40 2024-03-24T01:03:50.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.576+0000 7fbedfe00640 1 -- 172.21.15.38:0/2009434445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7fbee004e880 con 0x7fbee00feb40 2024-03-24T01:03:50.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.577+0000 7fbed7400640 1 -- 172.21.15.38:0/2009434445 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v40) v1 ==== 107+0+5061 (secure 0 0 0) 0x7fbed007cd40 con 0x7fbee00feb40 2024-03-24T01:03:50.577 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:50.577 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44373,"name":"cephfs.smithi088.yhksbr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/1135457323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1135457323},{"type":"v1","addr":"172.21.15.88:6827","nonce":1135457323}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T01:01:18.237311+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34446,"mds_1":34478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34446":{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":0,"incarnation":33,"state":"up:active","state_seq":29,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34478":{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":1,"incarnation":33,"state":"up:active","state_seq":22,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34446,"qdb_cluster":[34446,34478]},"id":1}]} 2024-03-24T01:03:50.577 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 38 2024-03-24T01:03:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.580+0000 7fbedfe00640 1 -- 172.21.15.38:0/2009434445 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbec406df10 msgr2=0x7fbec40703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.580+0000 7fbedfe00640 1 --2- 172.21.15.38:0/2009434445 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbec406df10 0x7fbec40703d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbecc005fd0 tx=0x7fbecc005850 comp rx=0 tx=0).stop 2024-03-24T01:03:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.580+0000 7fbedfe00640 1 -- 172.21.15.38:0/2009434445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee00feb40 msgr2=0x7fbee0197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.580+0000 7fbedfe00640 1 --2- 172.21.15.38:0/2009434445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee00feb40 0x7fbee0197f50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbed002c270 tx=0x7fbed002c7b0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.580+0000 7fbedfe00640 1 -- 172.21.15.38:0/2009434445 shutdown_connections 2024-03-24T01:03:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.580+0000 7fbedfe00640 1 --2- 172.21.15.38:0/2009434445 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fbec406df10 0x7fbec40703d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.580+0000 7fbedfe00640 1 --2- 172.21.15.38:0/2009434445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee00ff4f0 0x7fbee0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.580+0000 7fbedfe00640 1 --2- 172.21.15.38:0/2009434445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee00feb40 0x7fbee0197f50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.580+0000 7fbedfe00640 1 -- 172.21.15.38:0/2009434445 >> 172.21.15.38:0/2009434445 conn(0x7fbee00fa770 msgr2=0x7fbee0108960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:50.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.580+0000 7fbedfe00640 1 -- 172.21.15.38:0/2009434445 shutdown_connections 2024-03-24T01:03:50.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.580+0000 7fbedfe00640 1 -- 172.21.15.38:0/2009434445 wait complete. 2024-03-24T01:03:50.599 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-03-24T01:03:50.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.820+0000 7f6c52400640 1 -- 172.21.15.38:0/576634813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4c102390 msgr2=0x7f6c4c102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:50.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.820+0000 7f6c52400640 1 --2- 172.21.15.38:0/576634813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4c102390 0x7f6c4c102770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009c20 tx=0x7f6c3c02c1b0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.820+0000 7f6c52400640 1 -- 172.21.15.38:0/576634813 shutdown_connections 2024-03-24T01:03:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.820+0000 7f6c52400640 1 --2- 172.21.15.38:0/576634813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102d40 0x7f6c4c106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.820+0000 7f6c52400640 1 --2- 172.21.15.38:0/576634813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4c102390 0x7f6c4c102770 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.820+0000 7f6c52400640 1 -- 172.21.15.38:0/576634813 >> 172.21.15.38:0/576634813 conn(0x7f6c4c0fdbf0 msgr2=0x7f6c4c100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.820+0000 7f6c52400640 1 -- 172.21.15.38:0/576634813 shutdown_connections 2024-03-24T01:03:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.820+0000 7f6c52400640 1 -- 172.21.15.38:0/576634813 wait complete. 2024-03-24T01:03:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.822+0000 7f6c52400640 1 Processor -- start 2024-03-24T01:03:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.822+0000 7f6c52400640 1 -- start start 2024-03-24T01:03:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.822+0000 7f6c52400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102390 0x7f6c4c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.822+0000 7f6c52400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4c198710 0x7f6c4c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.822+0000 7f6c52400640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c198d10 con 0x7f6c4c198710 2024-03-24T01:03:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.822+0000 7f6c52400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c198e80 con 0x7f6c4c102390 2024-03-24T01:03:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.823+0000 7f6c4aa00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4c198710 0x7f6c4c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.823+0000 7f6c4aa00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4c198710 0x7f6c4c19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34380/0 (socket says 172.21.15.38:34380) 2024-03-24T01:03:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.823+0000 7f6c4aa00640 1 -- 172.21.15.38:0/3957530077 learned_addr learned my addr 172.21.15.38:0/3957530077 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.823+0000 7f6c4b400640 1 --2- 172.21.15.38:0/3957530077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102390 0x7f6c4c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.823+0000 7f6c4aa00640 1 -- 172.21.15.38:0/3957530077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102390 msgr2=0x7f6c4c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.823+0000 7f6c4aa00640 1 --2- 172.21.15.38:0/3957530077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102390 0x7f6c4c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:50.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.823+0000 7f6c4aa00640 1 -- 172.21.15.38:0/3957530077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c009900 con 0x7f6c4c198710 2024-03-24T01:03:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.824+0000 7f6c4aa00640 1 --2- 172.21.15.38:0/3957530077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4c198710 0x7f6c4c19cab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800c920 tx=0x7f6c3800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.824+0000 7f6c43e00640 1 -- 172.21.15.38:0/3957530077 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c38007b80 con 0x7f6c4c198710 2024-03-24T01:03:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.824+0000 7f6c43e00640 1 -- 172.21.15.38:0/3957530077 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c380054e0 con 0x7f6c4c198710 2024-03-24T01:03:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.824+0000 7f6c43e00640 1 -- 172.21.15.38:0/3957530077 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c3800f450 con 0x7f6c4c198710 2024-03-24T01:03:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.824+0000 7f6c52400640 1 -- 172.21.15.38:0/3957530077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c19d0b0 con 0x7f6c4c198710 2024-03-24T01:03:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.824+0000 7f6c52400640 1 -- 172.21.15.38:0/3957530077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c19d610 con 0x7f6c4c198710 2024-03-24T01:03:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.825+0000 7f6c52400640 1 -- 172.21.15.38:0/3957530077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c103860 con 0x7f6c4c198710 2024-03-24T01:03:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.827+0000 7f6c43e00640 1 -- 172.21.15.38:0/3957530077 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f6c380046c0 con 0x7f6c4c198710 2024-03-24T01:03:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.828+0000 7f6c43e00640 1 --2- 172.21.15.38:0/3957530077 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f6c3006df10 0x7f6c300703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:50.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.829+0000 7f6c43e00640 1 -- 172.21.15.38:0/3957530077 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6c3808dc10 con 0x7f6c4c198710 2024-03-24T01:03:50.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.829+0000 7f6c4b400640 1 --2- 172.21.15.38:0/3957530077 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f6c3006df10 0x7f6c300703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:50.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.830+0000 7f6c4b400640 1 --2- 172.21.15.38:0/3957530077 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f6c3006df10 0x7f6c300703d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c005fa0 tx=0x7f6c3c005f30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:50.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:50.834+0000 7f6c43e00640 1 -- 172.21.15.38:0/3957530077 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6c38057010 con 0x7f6c4c198710 2024-03-24T01:03:50.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:50 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/882520025' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-24T01:03:50.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:50 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2009434445' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-24T01:03:50.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:50 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/882520025' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-24T01:03:50.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:50 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2009434445' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-24T01:03:51.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.078+0000 7f6c52400640 1 -- 172.21.15.38:0/3957530077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7f6c4c10a690 con 0x7f6c4c198710 2024-03-24T01:03:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.080+0000 7f6c43e00640 1 -- 172.21.15.38:0/3957530077 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v40) v1 ==== 107+0+5076 (secure 0 0 0) 0x7f6c3801d090 con 0x7f6c4c198710 2024-03-24T01:03:51.080 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:51.080 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T00:44:51.138029+0000","modified":"2024-03-24T01:01:18.259747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34446,"mds_1":34478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34446":{"gid":34446,"name":"cephfs.smithi038.yhkmsb","rank":0,"incarnation":33,"state":"up:active","state_seq":29,"addr":"172.21.15.38:6827/1176900242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":1176900242},{"type":"v1","addr":"172.21.15.38:6827","nonce":1176900242}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34478":{"gid":34478,"name":"cephfs.smithi038.xnpqhp","rank":1,"incarnation":33,"state":"up:active","state_seq":22,"addr":"172.21.15.38:6829/2579165695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2579165695},{"type":"v1","addr":"172.21.15.38:6829","nonce":2579165695}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44345":{"gid":44345,"name":"cephfs.smithi088.cbfvvp","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6825/2293661188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2293661188},{"type":"v1","addr":"172.21.15.88:6825","nonce":2293661188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44373":{"gid":44373,"name":"cephfs.smithi088.yhksbr","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6827/1135457323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1135457323},{"type":"v1","addr":"172.21.15.88:6827","nonce":1135457323}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34446,"qdb_cluster":[34478,34446]},"id":1}]} 2024-03-24T01:03:51.081 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 39 2024-03-24T01:03:51.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.083+0000 7f6c52400640 1 -- 172.21.15.38:0/3957530077 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f6c3006df10 msgr2=0x7f6c300703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:51.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.083+0000 7f6c52400640 1 --2- 172.21.15.38:0/3957530077 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f6c3006df10 0x7f6c300703d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c005fa0 tx=0x7f6c3c005f30 comp rx=0 tx=0).stop 2024-03-24T01:03:51.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.083+0000 7f6c52400640 1 -- 172.21.15.38:0/3957530077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4c198710 msgr2=0x7f6c4c19cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:51.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.083+0000 7f6c52400640 1 --2- 172.21.15.38:0/3957530077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4c198710 0x7f6c4c19cab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800c920 tx=0x7f6c3800cde0 comp rx=0 tx=0).stop 2024-03-24T01:03:51.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.083+0000 7f6c52400640 1 -- 172.21.15.38:0/3957530077 shutdown_connections 2024-03-24T01:03:51.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.083+0000 7f6c52400640 1 --2- 172.21.15.38:0/3957530077 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f6c3006df10 0x7f6c300703d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:51.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.083+0000 7f6c52400640 1 --2- 172.21.15.38:0/3957530077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c4c198710 0x7f6c4c19cab0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:51.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.083+0000 7f6c52400640 1 --2- 172.21.15.38:0/3957530077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102390 0x7f6c4c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:51.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.083+0000 7f6c52400640 1 -- 172.21.15.38:0/3957530077 >> 172.21.15.38:0/3957530077 conn(0x7f6c4c0fdbf0 msgr2=0x7f6c4c0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:51.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.083+0000 7f6c52400640 1 -- 172.21.15.38:0/3957530077 shutdown_connections 2024-03-24T01:03:51.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.083+0000 7f6c52400640 1 -- 172.21.15.38:0/3957530077 wait complete. 2024-03-24T01:03:51.107 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-24T01:03:51.118 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-24T01:03:51.119 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T01:03:51.119 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T01:03:51.172 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-03-24T01:03:51.172 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T01:03:51.261 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T01:03:51.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.528+0000 7f734f000640 1 -- 172.21.15.38:0/2784300539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7348100990 msgr2=0x7f73481049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:51.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.528+0000 7f734f000640 1 --2- 172.21.15.38:0/2784300539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7348100990 0x7f73481049d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7334009880 tx=0x7f733402bcf0 comp rx=0 tx=0).stop 2024-03-24T01:03:51.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.528+0000 7f734f000640 1 -- 172.21.15.38:0/2784300539 shutdown_connections 2024-03-24T01:03:51.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.528+0000 7f734f000640 1 --2- 172.21.15.38:0/2784300539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7348100990 0x7f73481049d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:51.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.528+0000 7f734f000640 1 --2- 172.21.15.38:0/2784300539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73480fffe0 0x7f73481003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:51.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.528+0000 7f734f000640 1 -- 172.21.15.38:0/2784300539 >> 172.21.15.38:0/2784300539 conn(0x7f73480fb840 msgr2=0x7f73480fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.528+0000 7f734f000640 1 -- 172.21.15.38:0/2784300539 shutdown_connections 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.529+0000 7f734f000640 1 -- 172.21.15.38:0/2784300539 wait complete. 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.530+0000 7f734f000640 1 Processor -- start 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.530+0000 7f734f000640 1 -- start start 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.530+0000 7f734f000640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73480fffe0 0x7f73481980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.530+0000 7f734f000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7348198610 0x7f734819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.530+0000 7f734f000640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7348198c10 con 0x7f73480fffe0 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.530+0000 7f734f000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7348198d80 con 0x7f7348198610 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.530+0000 7f734dc00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73480fffe0 0x7f73481980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.530+0000 7f734dc00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73480fffe0 0x7f73481980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34400/0 (socket says 172.21.15.38:34400) 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.530+0000 7f734dc00640 1 -- 172.21.15.38:0/3387381910 learned_addr learned my addr 172.21.15.38:0/3387381910 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.531+0000 7f734d200640 1 --2- 172.21.15.38:0/3387381910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7348198610 0x7f734819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.531+0000 7f734dc00640 1 -- 172.21.15.38:0/3387381910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7348198610 msgr2=0x7f734819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:51.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.531+0000 7f734dc00640 1 --2- 172.21.15.38:0/3387381910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7348198610 0x7f734819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:51.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.531+0000 7f734dc00640 1 -- 172.21.15.38:0/3387381910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7334009560 con 0x7f73480fffe0 2024-03-24T01:03:51.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.531+0000 7f734dc00640 1 --2- 172.21.15.38:0/3387381910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73480fffe0 0x7f73481980d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f733c00c840 tx=0x7f733c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:51.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.531+0000 7f7346000640 1 -- 172.21.15.38:0/3387381910 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f733c0041f0 con 0x7f73480fffe0 2024-03-24T01:03:51.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.531+0000 7f7346000640 1 -- 172.21.15.38:0/3387381910 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f733c007e70 con 0x7f73480fffe0 2024-03-24T01:03:51.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.532+0000 7f7346000640 1 -- 172.21.15.38:0/3387381910 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f733c0049d0 con 0x7f73480fffe0 2024-03-24T01:03:51.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.532+0000 7f734f000640 1 -- 172.21.15.38:0/3387381910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734819cf60 con 0x7f73480fffe0 2024-03-24T01:03:51.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.532+0000 7f734f000640 1 -- 172.21.15.38:0/3387381910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734819d4c0 con 0x7f73480fffe0 2024-03-24T01:03:51.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.533+0000 7f734f000640 1 -- 172.21.15.38:0/3387381910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73481014b0 con 0x7f73480fffe0 2024-03-24T01:03:51.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.534+0000 7f7346000640 1 -- 172.21.15.38:0/3387381910 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f733c0079b0 con 0x7f73480fffe0 2024-03-24T01:03:51.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.535+0000 7f7346000640 1 --2- 172.21.15.38:0/3387381910 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f733006df20 0x7f73300703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:51.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.535+0000 7f7346000640 1 -- 172.21.15.38:0/3387381910 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f733c08db70 con 0x7f73480fffe0 2024-03-24T01:03:51.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.535+0000 7f734d200640 1 --2- 172.21.15.38:0/3387381910 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f733006df20 0x7f73300703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:51.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.536+0000 7f734d200640 1 --2- 172.21.15.38:0/3387381910 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f733006df20 0x7f73300703e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7334006010 tx=0x7f7334005f30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:51.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.540+0000 7f7346000640 1 -- 172.21.15.38:0/3387381910 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f733c057010 con 0x7f73480fffe0 2024-03-24T01:03:51.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.762+0000 7f734f000640 1 -- 172.21.15.38:0/3387381910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7348073c70 con 0x7f73480fffe0 2024-03-24T01:03:51.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.764+0000 7f7346000640 1 -- 172.21.15.38:0/3387381910 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16736 (secure 0 0 0) 0x7f733c01d090 con 0x7f73480fffe0 2024-03-24T01:03:51.764 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:51.764 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":87,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","created":"2024-03-24T00:35:36.968970+0000","modified":"2024-03-24T01:00:46.397340+0000","last_up_change":"2024-03-24T00:59:18.537965+0000","last_in_change":"2024-03-24T00:42:50.726360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T00:41:31.289149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-24T00:44:50.057892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-24T00:44:50.717526+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e5df40af-b5d0-4f89-94b2-59ee2e710bc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6803","nonce":1079856560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6805","nonce":1079856560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6809","nonce":1079856560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6807","nonce":1079856560}]},"public_addr":"172.21.15.38:6803/1079856560","cluster_addr":"172.21.15.38:6805/1079856560","heartbeat_back_addr":"172.21.15.38:6809/1079856560","heartbeat_front_addr":"172.21.15.38:6807/1079856560","state":["exists","up"]},{"osd":1,"uuid":"ac3c0960-dd6c-44f9-b4b0-234c10f838fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6811","nonce":1120765274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6813","nonce":1120765274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6817","nonce":1120765274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6815","nonce":1120765274}]},"public_addr":"172.21.15.38:6811/1120765274","cluster_addr":"172.21.15.38:6813/1120765274","heartbeat_back_addr":"172.21.15.38:6817/1120765274","heartbeat_front_addr":"172.21.15.38:6815/1120765274","state":["exists","up"]},{"osd":2,"uuid":"921d9dd3-2471-4b13-b6bd-01e5b6c78648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6819","nonce":874077504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6821","nonce":874077504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6825","nonce":874077504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6823","nonce":874077504}]},"public_addr":"172.21.15.38:6819/874077504","cluster_addr":"172.21.15.38:6821/874077504","heartbeat_back_addr":"172.21.15.38:6825/874077504","heartbeat_front_addr":"172.21.15.38:6823/874077504","state":["exists","up"]},{"osd":3,"uuid":"7e93813a-0036-4cea-8396-41b8b759c257","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6801","nonce":1863418220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6803","nonce":1863418220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6807","nonce":1863418220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6805","nonce":1863418220}]},"public_addr":"172.21.15.88:6801/1863418220","cluster_addr":"172.21.15.88:6803/1863418220","heartbeat_back_addr":"172.21.15.88:6807/1863418220","heartbeat_front_addr":"172.21.15.88:6805/1863418220","state":["exists","up"]},{"osd":4,"uuid":"b73f29f7-134e-4725-b341-e893b4bbff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6809","nonce":2757972748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6811","nonce":2757972748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6815","nonce":2757972748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6813","nonce":2757972748}]},"public_addr":"172.21.15.88:6809/2757972748","cluster_addr":"172.21.15.88:6811/2757972748","heartbeat_back_addr":"172.21.15.88:6815/2757972748","heartbeat_front_addr":"172.21.15.88:6813/2757972748","state":["exists","up"]},{"osd":5,"uuid":"8ea2cfb3-be7a-4f97-9993-8f9001b8b678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6817","nonce":3428886242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6819","nonce":3428886242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6823","nonce":3428886242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6821","nonce":3428886242}]},"public_addr":"172.21.15.88:6817/3428886242","cluster_addr":"172.21.15.88:6819/3428886242","heartbeat_back_addr":"172.21.15.88:6823/3428886242","heartbeat_front_addr":"172.21.15.88:6821/3428886242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:11.140794+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:49.877898+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:41:28.612279+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:02.338217+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:36.760939+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:43:13.811948+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6827/1446532250":"2024-03-25T00:59:25.362289+0000","172.21.15.38:6826/2539263555":"2024-03-25T00:59:25.362259+0000","172.21.15.38:0/3860887425":"2024-03-25T00:36:13.958115+0000","172.21.15.88:0/610291104":"2024-03-25T00:49:19.114620+0000","172.21.15.38:6827/2539263555":"2024-03-25T00:59:25.362259+0000","172.21.15.38:6800/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.38:6800/839555621":"2024-03-25T00:53:56.535603+0000","172.21.15.88:6828/832884416":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/4049972561":"2024-03-25T00:38:28.565032+0000","172.21.15.88:0/3960582080":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/2868668356":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/4189386253":"2024-03-25T00:47:53.298855+0000","172.21.15.88:0/3753305655":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/913707898":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6800/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/3845097007":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/3448795319":"2024-03-25T00:38:28.565032+0000","172.21.15.38:6801/3107810736":"2024-03-25T00:47:53.298855+0000","172.21.15.88:6824/1531706761":"2024-03-25T00:45:13.640830+0000","172.21.15.38:0/1530344157":"2024-03-25T00:53:56.535603+0000","172.21.15.88:0/3790826752":"2024-03-25T00:49:19.114620+0000","172.21.15.38:6801/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6800/3107810736":"2024-03-25T00:47:53.298855+0000","172.21.15.38:0/1949387173":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6801/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/748221873":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6801/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.88:6829/832884416":"2024-03-25T00:49:19.114620+0000","172.21.15.88:6826/1446532250":"2024-03-25T00:59:25.362289+0000","172.21.15.88:0/1572172803":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/1962732004":"2024-03-25T00:47:53.298855+0000","172.21.15.38:0/3536812262":"2024-03-25T00:47:53.298855+0000","172.21.15.38:6800/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/773042674":"2024-03-25T00:53:56.535603+0000","172.21.15.38:6801/839555621":"2024-03-25T00:53:56.535603+0000","172.21.15.88:6825/1531706761":"2024-03-25T00:45:13.640830+0000","172.21.15.38:0/633787949":"2024-03-25T00:47:53.298855+0000","172.21.15.88:0/448394986":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/884740303":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/2405459521":"2024-03-25T00:53:56.535603+0000","172.21.15.38:0/587822322":"2024-03-25T00:53:56.535603+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T01:03:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.767+0000 7f734f000640 1 -- 172.21.15.38:0/3387381910 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f733006df20 msgr2=0x7f73300703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.768+0000 7f734f000640 1 --2- 172.21.15.38:0/3387381910 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f733006df20 0x7f73300703e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7334006010 tx=0x7f7334005f30 comp rx=0 tx=0).stop 2024-03-24T01:03:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.768+0000 7f734f000640 1 -- 172.21.15.38:0/3387381910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73480fffe0 msgr2=0x7f73481980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.768+0000 7f734f000640 1 --2- 172.21.15.38:0/3387381910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73480fffe0 0x7f73481980d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f733c00c840 tx=0x7f733c00cd00 comp rx=0 tx=0).stop 2024-03-24T01:03:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.768+0000 7f734f000640 1 -- 172.21.15.38:0/3387381910 shutdown_connections 2024-03-24T01:03:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.768+0000 7f734f000640 1 --2- 172.21.15.38:0/3387381910 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f733006df20 0x7f73300703e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.768+0000 7f734f000640 1 --2- 172.21.15.38:0/3387381910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7348198610 0x7f734819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.768+0000 7f734f000640 1 --2- 172.21.15.38:0/3387381910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73480fffe0 0x7f73481980d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.768+0000 7f734f000640 1 -- 172.21.15.38:0/3387381910 >> 172.21.15.38:0/3387381910 conn(0x7f73480fb840 msgr2=0x7f73480fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.768+0000 7f734f000640 1 -- 172.21.15.38:0/3387381910 shutdown_connections 2024-03-24T01:03:51.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:51.768+0000 7f734f000640 1 -- 172.21.15.38:0/3387381910 wait complete. 2024-03-24T01:03:51.783 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:51 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3957530077' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-03-24T01:03:51.783 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:51 smithi038 ceph-mon[112108]: pgmap v337: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:51.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T00:41:31.289149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-24T00:44:50.057892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-24T00:44:50.717526+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T01:03:51.786 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T01:03:51.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:51 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3957530077' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-03-24T01:03:51.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:51 smithi088 ceph-mon[87351]: pgmap v337: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:52.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.020+0000 7f9694c00640 1 -- 172.21.15.38:0/3859276010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690100f00 msgr2=0x7f96901012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:52.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.020+0000 7f9694c00640 1 --2- 172.21.15.38:0/3859276010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690100f00 0x7f96901012e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9680009880 tx=0x7f968002bd70 comp rx=0 tx=0).stop 2024-03-24T01:03:52.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.021+0000 7f9694c00640 1 -- 172.21.15.38:0/3859276010 shutdown_connections 2024-03-24T01:03:52.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.021+0000 7f9694c00640 1 --2- 172.21.15.38:0/3859276010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9690101820 0x7f9690109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.021+0000 7f9694c00640 1 --2- 172.21.15.38:0/3859276010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690100f00 0x7f96901012e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.021+0000 7f9694c00640 1 -- 172.21.15.38:0/3859276010 >> 172.21.15.38:0/3859276010 conn(0x7f96900758e0 msgr2=0x7f96900fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:52.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.021+0000 7f9694c00640 1 -- 172.21.15.38:0/3859276010 shutdown_connections 2024-03-24T01:03:52.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.021+0000 7f9694c00640 1 -- 172.21.15.38:0/3859276010 wait complete. 2024-03-24T01:03:52.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.022+0000 7f9694c00640 1 Processor -- start 2024-03-24T01:03:52.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.022+0000 7f9694c00640 1 -- start start 2024-03-24T01:03:52.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.022+0000 7f9694c00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690100f00 0x7f96900feb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:52.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.022+0000 7f9694c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9690101820 0x7f96900ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:52.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.022+0000 7f9694c00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900ff5d0 con 0x7f9690100f00 2024-03-24T01:03:52.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.022+0000 7f9694c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900ff710 con 0x7f9690101820 2024-03-24T01:03:52.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.023+0000 7f968f400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690100f00 0x7f96900feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:52.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.023+0000 7f968f400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690100f00 0x7f96900feb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34408/0 (socket says 172.21.15.38:34408) 2024-03-24T01:03:52.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.023+0000 7f968f400640 1 -- 172.21.15.38:0/2046962058 learned_addr learned my addr 172.21.15.38:0/2046962058 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:52.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.023+0000 7f968ea00640 1 --2- 172.21.15.38:0/2046962058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9690101820 0x7f96900ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:52.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.023+0000 7f968f400640 1 -- 172.21.15.38:0/2046962058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9690101820 msgr2=0x7f96900ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:52.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.023+0000 7f968f400640 1 --2- 172.21.15.38:0/2046962058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9690101820 0x7f96900ff090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.023+0000 7f968f400640 1 -- 172.21.15.38:0/2046962058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9680009560 con 0x7f9690100f00 2024-03-24T01:03:52.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.023+0000 7f968f400640 1 --2- 172.21.15.38:0/2046962058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690100f00 0x7f96900feb50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f968002c2b0 tx=0x7f968002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:52.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.024+0000 7f9687e00640 1 -- 172.21.15.38:0/2046962058 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9680004520 con 0x7f9690100f00 2024-03-24T01:03:52.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.024+0000 7f9687e00640 1 -- 172.21.15.38:0/2046962058 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9680003ab0 con 0x7f9690100f00 2024-03-24T01:03:52.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.024+0000 7f9694c00640 1 -- 172.21.15.38:0/2046962058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96900ff990 con 0x7f9690100f00 2024-03-24T01:03:52.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.024+0000 7f9687e00640 1 -- 172.21.15.38:0/2046962058 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9680005500 con 0x7f9690100f00 2024-03-24T01:03:52.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.024+0000 7f9694c00640 1 -- 172.21.15.38:0/2046962058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96901a22c0 con 0x7f9690100f00 2024-03-24T01:03:52.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.025+0000 7f9694c00640 1 -- 172.21.15.38:0/2046962058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9690067b20 con 0x7f9690100f00 2024-03-24T01:03:52.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.027+0000 7f9687e00640 1 -- 172.21.15.38:0/2046962058 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f9680003c20 con 0x7f9690100f00 2024-03-24T01:03:52.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.028+0000 7f9687e00640 1 --2- 172.21.15.38:0/2046962058 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f967406df10 0x7f96740703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:52.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.028+0000 7f9687e00640 1 -- 172.21.15.38:0/2046962058 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f96800b0840 con 0x7f9690100f00 2024-03-24T01:03:52.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.029+0000 7f968ea00640 1 --2- 172.21.15.38:0/2046962058 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f967406df10 0x7f96740703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:52.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.029+0000 7f968ea00640 1 --2- 172.21.15.38:0/2046962058 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f967406df10 0x7f96740703d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f967c005fd0 tx=0x7f967c009510 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:52.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.036+0000 7f9687e00640 1 -- 172.21.15.38:0/2046962058 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9680079c80 con 0x7f9690100f00 2024-03-24T01:03:52.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.352+0000 7f9694c00640 1 -- 172.21.15.38:0/2046962058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96901044a0 con 0x7f9690100f00 2024-03-24T01:03:52.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.353+0000 7f9687e00640 1 -- 172.21.15.38:0/2046962058 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f968007d340 con 0x7f9690100f00 2024-03-24T01:03:52.352 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-03-24T01:03:52.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.356+0000 7f9694c00640 1 -- 172.21.15.38:0/2046962058 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f967406df10 msgr2=0x7f96740703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:52.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.356+0000 7f9694c00640 1 --2- 172.21.15.38:0/2046962058 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f967406df10 0x7f96740703d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f967c005fd0 tx=0x7f967c009510 comp rx=0 tx=0).stop 2024-03-24T01:03:52.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.356+0000 7f9694c00640 1 -- 172.21.15.38:0/2046962058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690100f00 msgr2=0x7f96900feb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:52.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.356+0000 7f9694c00640 1 --2- 172.21.15.38:0/2046962058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690100f00 0x7f96900feb50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f968002c2b0 tx=0x7f968002c850 comp rx=0 tx=0).stop 2024-03-24T01:03:52.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.356+0000 7f9694c00640 1 -- 172.21.15.38:0/2046962058 shutdown_connections 2024-03-24T01:03:52.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.356+0000 7f9694c00640 1 --2- 172.21.15.38:0/2046962058 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f967406df10 0x7f96740703d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.356+0000 7f9694c00640 1 --2- 172.21.15.38:0/2046962058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9690101820 0x7f96900ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.356+0000 7f9694c00640 1 --2- 172.21.15.38:0/2046962058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9690100f00 0x7f96900feb50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.356+0000 7f9694c00640 1 -- 172.21.15.38:0/2046962058 >> 172.21.15.38:0/2046962058 conn(0x7f96900758e0 msgr2=0x7f9690102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:52.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.356+0000 7f9694c00640 1 -- 172.21.15.38:0/2046962058 shutdown_connections 2024-03-24T01:03:52.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.357+0000 7f9694c00640 1 -- 172.21.15.38:0/2046962058 wait complete. 2024-03-24T01:03:52.376 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-24T01:03:52.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.589+0000 7f8606000640 1 -- 172.21.15.38:0/4022317412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8600100980 msgr2=0x7f86001049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:52.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.589+0000 7f8606000640 1 --2- 172.21.15.38:0/4022317412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8600100980 0x7f86001049c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009880 tx=0x7f85ec02bcf0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.589+0000 7f8606000640 1 -- 172.21.15.38:0/4022317412 shutdown_connections 2024-03-24T01:03:52.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.589+0000 7f8606000640 1 --2- 172.21.15.38:0/4022317412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8600100980 0x7f86001049c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.589+0000 7f8606000640 1 --2- 172.21.15.38:0/4022317412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86000fffd0 0x7f86001003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.589+0000 7f8606000640 1 -- 172.21.15.38:0/4022317412 >> 172.21.15.38:0/4022317412 conn(0x7f86000fb850 msgr2=0x7f86000fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:52.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.589+0000 7f8606000640 1 -- 172.21.15.38:0/4022317412 shutdown_connections 2024-03-24T01:03:52.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.589+0000 7f8606000640 1 -- 172.21.15.38:0/4022317412 wait complete. 2024-03-24T01:03:52.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.590+0000 7f8606000640 1 Processor -- start 2024-03-24T01:03:52.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.591+0000 7f8606000640 1 -- start start 2024-03-24T01:03:52.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.591+0000 7f8606000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86000fffd0 0x7f8600198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:52.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.591+0000 7f8606000640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8600198560 0x7f860019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:52.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.591+0000 7f8606000640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8600198b60 con 0x7f8600198560 2024-03-24T01:03:52.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.591+0000 7f8606000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8600198cd0 con 0x7f86000fffd0 2024-03-24T01:03:52.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.591+0000 7f85ffe00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8600198560 0x7f860019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:52.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.591+0000 7f85ffe00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8600198560 0x7f860019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34414/0 (socket says 172.21.15.38:34414) 2024-03-24T01:03:52.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.591+0000 7f85ffe00640 1 -- 172.21.15.38:0/1276888452 learned_addr learned my addr 172.21.15.38:0/1276888452 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:52.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.592+0000 7f8604c00640 1 --2- 172.21.15.38:0/1276888452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86000fffd0 0x7f8600198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:52.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.592+0000 7f85ffe00640 1 -- 172.21.15.38:0/1276888452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86000fffd0 msgr2=0x7f8600198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:52.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.592+0000 7f85ffe00640 1 --2- 172.21.15.38:0/1276888452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86000fffd0 0x7f8600198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.592+0000 7f85ffe00640 1 -- 172.21.15.38:0/1276888452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec009560 con 0x7f8600198560 2024-03-24T01:03:52.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.592+0000 7f85ffe00640 1 --2- 172.21.15.38:0/1276888452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8600198560 0x7f860019c900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f85ec02ca00 tx=0x7f85ec02c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:52.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.593+0000 7f85fd600640 1 -- 172.21.15.38:0/1276888452 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85ec0035f0 con 0x7f8600198560 2024-03-24T01:03:52.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.593+0000 7f8606000640 1 -- 172.21.15.38:0/1276888452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860019cf00 con 0x7f8600198560 2024-03-24T01:03:52.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.593+0000 7f85fd600640 1 -- 172.21.15.38:0/1276888452 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85ec003c10 con 0x7f8600198560 2024-03-24T01:03:52.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.593+0000 7f85fd600640 1 -- 172.21.15.38:0/1276888452 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85ec0055f0 con 0x7f8600198560 2024-03-24T01:03:52.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.593+0000 7f8606000640 1 -- 172.21.15.38:0/1276888452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860019d460 con 0x7f8600198560 2024-03-24T01:03:52.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.594+0000 7f8606000640 1 -- 172.21.15.38:0/1276888452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86001014a0 con 0x7f8600198560 2024-03-24T01:03:52.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.595+0000 7f85fd600640 1 -- 172.21.15.38:0/1276888452 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f85ec004e60 con 0x7f8600198560 2024-03-24T01:03:52.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.596+0000 7f85fd600640 1 --2- 172.21.15.38:0/1276888452 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f85e806df20 0x7f85e80703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:52.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.596+0000 7f85fd600640 1 -- 172.21.15.38:0/1276888452 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f85ec0b0070 con 0x7f8600198560 2024-03-24T01:03:52.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.596+0000 7f8604c00640 1 --2- 172.21.15.38:0/1276888452 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f85e806df20 0x7f85e80703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:52.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.597+0000 7f8604c00640 1 --2- 172.21.15.38:0/1276888452 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f85e806df20 0x7f85e80703e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f85f4005fd0 tx=0x7f85f4005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:52.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.601+0000 7f85fd600640 1 -- 172.21.15.38:0/1276888452 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f85ec079470 con 0x7f8600198560 2024-03-24T01:03:52.818 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:52 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/3387381910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T01:03:52.818 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:52 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2046962058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T01:03:52.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.875+0000 7f8606000640 1 -- 172.21.15.38:0/1276888452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f86001040b0 con 0x7f8600198560 2024-03-24T01:03:52.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.876+0000 7f85fd600640 1 -- 172.21.15.38:0/1276888452 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f85ec041020 con 0x7f8600198560 2024-03-24T01:03:52.877 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-03-24T01:03:52.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.878+0000 7f8606000640 1 -- 172.21.15.38:0/1276888452 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f85e806df20 msgr2=0x7f85e80703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:52.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.878+0000 7f8606000640 1 --2- 172.21.15.38:0/1276888452 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f85e806df20 0x7f85e80703e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f85f4005fd0 tx=0x7f85f4005850 comp rx=0 tx=0).stop 2024-03-24T01:03:52.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.878+0000 7f8606000640 1 -- 172.21.15.38:0/1276888452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8600198560 msgr2=0x7f860019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:52.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.879+0000 7f8606000640 1 --2- 172.21.15.38:0/1276888452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8600198560 0x7f860019c900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f85ec02ca00 tx=0x7f85ec02c7d0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.879+0000 7f8606000640 1 -- 172.21.15.38:0/1276888452 shutdown_connections 2024-03-24T01:03:52.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.879+0000 7f8606000640 1 --2- 172.21.15.38:0/1276888452 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f85e806df20 0x7f85e80703e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.879+0000 7f8606000640 1 --2- 172.21.15.38:0/1276888452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8600198560 0x7f860019c900 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.879+0000 7f8606000640 1 --2- 172.21.15.38:0/1276888452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86000fffd0 0x7f8600198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:52.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.879+0000 7f8606000640 1 -- 172.21.15.38:0/1276888452 >> 172.21.15.38:0/1276888452 conn(0x7f86000fb850 msgr2=0x7f86000fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:52.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.879+0000 7f8606000640 1 -- 172.21.15.38:0/1276888452 shutdown_connections 2024-03-24T01:03:52.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:52.879+0000 7f8606000640 1 -- 172.21.15.38:0/1276888452 wait complete. 2024-03-24T01:03:52.906 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-24T01:03:52.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:52 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/3387381910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T01:03:52.931 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:52 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2046962058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T01:03:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.111+0000 7eff15c00640 1 -- 172.21.15.38:0/3633712253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff100ffff0 msgr2=0x7eff101003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.111+0000 7eff15c00640 1 --2- 172.21.15.38:0/3633712253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff100ffff0 0x7eff101003d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7eff00009880 tx=0x7eff0002bcf0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.112+0000 7eff15c00640 1 -- 172.21.15.38:0/3633712253 shutdown_connections 2024-03-24T01:03:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.112+0000 7eff15c00640 1 --2- 172.21.15.38:0/3633712253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff101009a0 0x7eff101049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.112+0000 7eff15c00640 1 --2- 172.21.15.38:0/3633712253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff100ffff0 0x7eff101003d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.112+0000 7eff15c00640 1 -- 172.21.15.38:0/3633712253 >> 172.21.15.38:0/3633712253 conn(0x7eff100fb8b0 msgr2=0x7eff100fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.112+0000 7eff15c00640 1 -- 172.21.15.38:0/3633712253 shutdown_connections 2024-03-24T01:03:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.112+0000 7eff15c00640 1 -- 172.21.15.38:0/3633712253 wait complete. 2024-03-24T01:03:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.113+0000 7eff15c00640 1 Processor -- start 2024-03-24T01:03:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.114+0000 7eff15c00640 1 -- start start 2024-03-24T01:03:53.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.114+0000 7eff15c00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff100ffff0 0x7eff100725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:53.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.114+0000 7eff15c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff10072b00 0x7eff10072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:53.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.114+0000 7eff15c00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff100744d0 con 0x7eff100ffff0 2024-03-24T01:03:53.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.114+0000 7eff15c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff101a2000 con 0x7eff10072b00 2024-03-24T01:03:53.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.114+0000 7eff0fe00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff100ffff0 0x7eff100725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:53.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.114+0000 7eff0fe00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff100ffff0 0x7eff100725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48044/0 (socket says 172.21.15.38:48044) 2024-03-24T01:03:53.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.114+0000 7eff0fe00640 1 -- 172.21.15.38:0/768853397 learned_addr learned my addr 172.21.15.38:0/768853397 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.115+0000 7eff07e00640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff10072b00 0x7eff10072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.115+0000 7eff0fe00640 1 -- 172.21.15.38:0/768853397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff10072b00 msgr2=0x7eff10072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.115+0000 7eff0fe00640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff10072b00 0x7eff10072f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.115+0000 7eff0fe00640 1 -- 172.21.15.38:0/768853397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff00009560 con 0x7eff100ffff0 2024-03-24T01:03:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.115+0000 7eff07e00640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff10072b00 0x7eff10072f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T01:03:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.115+0000 7eff0fe00640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff100ffff0 0x7eff100725c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7eff00009430 tx=0x7eff0002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.116+0000 7eff0d600640 1 -- 172.21.15.38:0/768853397 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff00004520 con 0x7eff100ffff0 2024-03-24T01:03:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.116+0000 7eff0d600640 1 -- 172.21.15.38:0/768853397 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff00003ab0 con 0x7eff100ffff0 2024-03-24T01:03:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.116+0000 7eff15c00640 1 -- 172.21.15.38:0/768853397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff101a2140 con 0x7eff100ffff0 2024-03-24T01:03:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.116+0000 7eff0d600640 1 -- 172.21.15.38:0/768853397 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff000055e0 con 0x7eff100ffff0 2024-03-24T01:03:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.116+0000 7eff15c00640 1 -- 172.21.15.38:0/768853397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff101a2670 con 0x7eff100ffff0 2024-03-24T01:03:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.117+0000 7eff15c00640 1 -- 172.21.15.38:0/768853397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff101014c0 con 0x7eff100ffff0 2024-03-24T01:03:53.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.118+0000 7eff0d600640 1 -- 172.21.15.38:0/768853397 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7eff00004dd0 con 0x7eff100ffff0 2024-03-24T01:03:53.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.120+0000 7eff0d600640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efef406dec0 0x7efef4070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:53.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.120+0000 7eff0d600640 1 -- 172.21.15.38:0/768853397 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7eff000b0190 con 0x7eff100ffff0 2024-03-24T01:03:53.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.120+0000 7eff07e00640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efef406dec0 0x7efef4070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:53.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.121+0000 7eff07e00640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efef406dec0 0x7efef4070380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efef80058c0 tx=0x7efef8005d30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:53.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.127+0000 7eff0d600640 1 -- 172.21.15.38:0/768853397 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7eff000795b0 con 0x7eff100ffff0 2024-03-24T01:03:53.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.341+0000 7eff15c00640 1 -- 172.21.15.38:0/768853397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7eff1004e880 con 0x7eff100ffff0 2024-03-24T01:03:53.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.342+0000 7eff0d600640 1 -- 172.21.15.38:0/768853397 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7eff0007cc70 con 0x7eff100ffff0 2024-03-24T01:03:53.341 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-03-24T01:03:53.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.344+0000 7eff15c00640 1 -- 172.21.15.38:0/768853397 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efef406dec0 msgr2=0x7efef4070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:53.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.344+0000 7eff15c00640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efef406dec0 0x7efef4070380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efef80058c0 tx=0x7efef8005d30 comp rx=0 tx=0).stop 2024-03-24T01:03:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.344+0000 7eff15c00640 1 -- 172.21.15.38:0/768853397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff100ffff0 msgr2=0x7eff100725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.344+0000 7eff15c00640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff100ffff0 0x7eff100725c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7eff00009430 tx=0x7eff0002c860 comp rx=0 tx=0).stop 2024-03-24T01:03:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.344+0000 7eff15c00640 1 -- 172.21.15.38:0/768853397 shutdown_connections 2024-03-24T01:03:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.344+0000 7eff15c00640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7efef406dec0 0x7efef4070380 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.344+0000 7eff15c00640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff10072b00 0x7eff10072f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.344+0000 7eff15c00640 1 --2- 172.21.15.38:0/768853397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff100ffff0 0x7eff100725c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.344+0000 7eff15c00640 1 -- 172.21.15.38:0/768853397 >> 172.21.15.38:0/768853397 conn(0x7eff100fb8b0 msgr2=0x7eff100fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.345+0000 7eff15c00640 1 -- 172.21.15.38:0/768853397 shutdown_connections 2024-03-24T01:03:53.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.345+0000 7eff15c00640 1 -- 172.21.15.38:0/768853397 wait complete. 2024-03-24T01:03:53.368 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-24T01:03:53.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.567+0000 7f0afa000640 1 -- 172.21.15.38:0/3921936616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af40726f0 msgr2=0x7f0af4072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.567+0000 7f0afa000640 1 --2- 172.21.15.38:0/3921936616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af40726f0 0x7f0af4072ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009880 tx=0x7f0ae402bd90 comp rx=0 tx=0).stop 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.567+0000 7f0afa000640 1 -- 172.21.15.38:0/3921936616 shutdown_connections 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.567+0000 7f0afa000640 1 --2- 172.21.15.38:0/3921936616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af4073010 0x7f0af410c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.567+0000 7f0afa000640 1 --2- 172.21.15.38:0/3921936616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af40726f0 0x7f0af4072ad0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.567+0000 7f0afa000640 1 -- 172.21.15.38:0/3921936616 >> 172.21.15.38:0/3921936616 conn(0x7f0af40fba00 msgr2=0x7f0af40fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.568+0000 7f0afa000640 1 -- 172.21.15.38:0/3921936616 shutdown_connections 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.568+0000 7f0afa000640 1 -- 172.21.15.38:0/3921936616 wait complete. 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.569+0000 7f0afa000640 1 Processor -- start 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.569+0000 7f0afa000640 1 -- start start 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.569+0000 7f0afa000640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af40726f0 0x7f0af41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.569+0000 7f0afa000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af4073010 0x7f0af41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.569+0000 7f0afa000640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4198d80 con 0x7f0af40726f0 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.569+0000 7f0afa000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af419ca90 con 0x7f0af4073010 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.569+0000 7f0af3400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af40726f0 0x7f0af41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.569+0000 7f0af3400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af40726f0 0x7f0af41981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48054/0 (socket says 172.21.15.38:48054) 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.569+0000 7f0af3400640 1 -- 172.21.15.38:0/1825291666 learned_addr learned my addr 172.21.15.38:0/1825291666 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.570+0000 7f0af2a00640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af4073010 0x7f0af41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.570+0000 7f0af3400640 1 -- 172.21.15.38:0/1825291666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af4073010 msgr2=0x7f0af41986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.570+0000 7f0af3400640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af4073010 0x7f0af41986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.570+0000 7f0af3400640 1 -- 172.21.15.38:0/1825291666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae4009560 con 0x7f0af40726f0 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.570+0000 7f0af2a00640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af4073010 0x7f0af41986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.570+0000 7f0af3400640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af40726f0 0x7f0af41981b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4006010 tx=0x7f0ae402df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.570+0000 7f0aebe00640 1 -- 172.21.15.38:0/1825291666 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ae40044a0 con 0x7f0af40726f0 2024-03-24T01:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.571+0000 7f0aebe00640 1 -- 172.21.15.38:0/1825291666 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ae4003ab0 con 0x7f0af40726f0 2024-03-24T01:03:53.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.571+0000 7f0afa000640 1 -- 172.21.15.38:0/1825291666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af419cd10 con 0x7f0af40726f0 2024-03-24T01:03:53.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.571+0000 7f0aebe00640 1 -- 172.21.15.38:0/1825291666 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ae4005590 con 0x7f0af40726f0 2024-03-24T01:03:53.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.571+0000 7f0afa000640 1 -- 172.21.15.38:0/1825291666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af419d270 con 0x7f0af40726f0 2024-03-24T01:03:53.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.572+0000 7f0afa000640 1 -- 172.21.15.38:0/1825291666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af4073da0 con 0x7f0af40726f0 2024-03-24T01:03:53.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.573+0000 7f0aebe00640 1 -- 172.21.15.38:0/1825291666 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f0ae4004dd0 con 0x7f0af40726f0 2024-03-24T01:03:53.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.574+0000 7f0aebe00640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ad806df20 0x7f0ad80703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:53.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.574+0000 7f0aebe00640 1 -- 172.21.15.38:0/1825291666 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0ae40b0a50 con 0x7f0af40726f0 2024-03-24T01:03:53.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.575+0000 7f0af2a00640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ad806df20 0x7f0ad80703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:53.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.575+0000 7f0af2a00640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ad806df20 0x7f0ad80703e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0af4199700 tx=0x7f0ae0005e50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:53.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.580+0000 7f0aebe00640 1 -- 172.21.15.38:0/1825291666 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ae4079dd0 con 0x7f0af40726f0 2024-03-24T01:03:53.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.818+0000 7f0afa000640 1 -- 172.21.15.38:0/1825291666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0af404e030 con 0x7f0af40726f0 2024-03-24T01:03:53.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.819+0000 7f0aebe00640 1 -- 172.21.15.38:0/1825291666 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v87) v1 ==== 81+0+2420 (secure 0 0 0) 0x7f0ae407d490 con 0x7f0af40726f0 2024-03-24T01:03:53.819 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6827/1446532250 2024-03-25T00:59:25.362289+0000 2024-03-24T01:03:53.819 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6826/2539263555 2024-03-25T00:59:25.362259+0000 2024-03-24T01:03:53.819 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3860887425 2024-03-25T00:36:13.958115+0000 2024-03-24T01:03:53.819 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/610291104 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:53.819 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6827/2539263555 2024-03-25T00:59:25.362259+0000 2024-03-24T01:03:53.819 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/1124244493 2024-03-25T00:38:28.565032+0000 2024-03-24T01:03:53.819 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/839555621 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6828/832884416 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/4049972561 2024-03-25T00:38:28.565032+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/3960582080 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2868668356 2024-03-25T00:36:55.932282+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/4189386253 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/3753305655 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/913707898 2024-03-25T00:36:13.958115+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/3284714719 2024-03-25T00:36:13.958115+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3845097007 2024-03-25T00:36:55.932282+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3448795319 2024-03-25T00:38:28.565032+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/3107810736 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6824/1531706761 2024-03-25T00:45:13.640830+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1530344157 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:53.820 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/3790826752 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/3111371494 2024-03-25T00:36:55.932282+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/3107810736 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1949387173 2024-03-25T00:36:13.958115+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/3284714719 2024-03-25T00:36:13.958115+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/748221873 2024-03-25T00:36:55.932282+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/1124244493 2024-03-25T00:38:28.565032+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6829/832884416 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6826/1446532250 2024-03-25T00:59:25.362289+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/1572172803 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1962732004 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3536812262 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/3111371494 2024-03-25T00:36:55.932282+0000 2024-03-24T01:03:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/773042674 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:53.822 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/839555621 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:53.822 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6825/1531706761 2024-03-25T00:45:13.640830+0000 2024-03-24T01:03:53.822 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/633787949 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:53.822 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/448394986 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:53.822 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/884740303 2024-03-25T00:38:28.565032+0000 2024-03-24T01:03:53.822 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2405459521 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:53.822 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/587822322 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:listed 41 entries 2024-03-24T01:03:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.822+0000 7f0afa000640 1 -- 172.21.15.38:0/1825291666 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ad806df20 msgr2=0x7f0ad80703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.822+0000 7f0afa000640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ad806df20 0x7f0ad80703e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0af4199700 tx=0x7f0ae0005e50 comp rx=0 tx=0).stop 2024-03-24T01:03:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.822+0000 7f0afa000640 1 -- 172.21.15.38:0/1825291666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af40726f0 msgr2=0x7f0af41981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.822+0000 7f0afa000640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af40726f0 0x7f0af41981b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4006010 tx=0x7f0ae402df60 comp rx=0 tx=0).stop 2024-03-24T01:03:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.822+0000 7f0afa000640 1 -- 172.21.15.38:0/1825291666 shutdown_connections 2024-03-24T01:03:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.822+0000 7f0afa000640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f0ad806df20 0x7f0ad80703e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.822+0000 7f0afa000640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0af4073010 0x7f0af41986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.822+0000 7f0afa000640 1 --2- 172.21.15.38:0/1825291666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0af40726f0 0x7f0af41981b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:53.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.822+0000 7f0afa000640 1 -- 172.21.15.38:0/1825291666 >> 172.21.15.38:0/1825291666 conn(0x7f0af40fba00 msgr2=0x7f0af4104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:53.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.822+0000 7f0afa000640 1 -- 172.21.15.38:0/1825291666 shutdown_connections 2024-03-24T01:03:53.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:53.822+0000 7f0afa000640 1 -- 172.21.15.38:0/1825291666 wait complete. 2024-03-24T01:03:53.842 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:53 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1276888452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T01:03:53.843 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:53 smithi038 ceph-mon[112108]: pgmap v338: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:53.843 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:53 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/768853397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T01:03:53.845 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-24T01:03:53.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T01:03:53.845 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-24T01:03:53.918 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T01:03:53.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T01:03:53.918 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-24T01:03:53.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:53 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1276888452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T01:03:53.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:53 smithi088 ceph-mon[87351]: pgmap v338: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-24T01:03:53.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:53 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/768853397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T01:03:53.982 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T01:03:53.982 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T01:03:54.040 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-03-24T01:03:54.041 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T01:03:54.092 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T01:03:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.285+0000 7f7211c00640 1 -- 172.21.15.38:0/3759610299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f720c075110 msgr2=0x7f720c110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.285+0000 7f7211c00640 1 --2- 172.21.15.38:0/3759610299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f720c075110 0x7f720c110710 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009880 tx=0x7f71f802bce0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.285+0000 7f7211c00640 1 -- 172.21.15.38:0/3759610299 shutdown_connections 2024-03-24T01:03:54.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.285+0000 7f7211c00640 1 --2- 172.21.15.38:0/3759610299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f720c075110 0x7f720c110710 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.285+0000 7f7211c00640 1 --2- 172.21.15.38:0/3759610299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f720c0747f0 0x7f720c074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.285+0000 7f7211c00640 1 -- 172.21.15.38:0/3759610299 >> 172.21.15.38:0/3759610299 conn(0x7f720c0fda80 msgr2=0x7f720c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:54.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.285+0000 7f7211c00640 1 -- 172.21.15.38:0/3759610299 shutdown_connections 2024-03-24T01:03:54.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.285+0000 7f7211c00640 1 -- 172.21.15.38:0/3759610299 wait complete. 2024-03-24T01:03:54.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.286+0000 7f7211c00640 1 Processor -- start 2024-03-24T01:03:54.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.286+0000 7f7211c00640 1 -- start start 2024-03-24T01:03:54.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.287+0000 7f7211c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f720c0747f0 0x7f720c19c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:54.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.287+0000 7f7211c00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f720c075110 0x7f720c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:54.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.287+0000 7f7211c00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c19cf50 con 0x7f720c075110 2024-03-24T01:03:54.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.287+0000 7f7211c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c1a0c60 con 0x7f720c0747f0 2024-03-24T01:03:54.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.287+0000 7f720b400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f720c075110 0x7f720c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.287+0000 7f720b400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f720c075110 0x7f720c19c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48066/0 (socket says 172.21.15.38:48066) 2024-03-24T01:03:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.287+0000 7f720b400640 1 -- 172.21.15.38:0/2178456703 learned_addr learned my addr 172.21.15.38:0/2178456703 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.287+0000 7f720b400640 1 -- 172.21.15.38:0/2178456703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f720c0747f0 msgr2=0x7f720c19c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.287+0000 7f720b400640 1 --2- 172.21.15.38:0/2178456703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f720c0747f0 0x7f720c19c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.287+0000 7f720b400640 1 -- 172.21.15.38:0/2178456703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f8009560 con 0x7f720c075110 2024-03-24T01:03:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.288+0000 7f720b400640 1 --2- 172.21.15.38:0/2178456703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f720c075110 0x7f720c19c8c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f71f8005ab0 tx=0x7f71f8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.288+0000 7f7208c00640 1 -- 172.21.15.38:0/2178456703 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71f80035f0 con 0x7f720c075110 2024-03-24T01:03:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.288+0000 7f7208c00640 1 -- 172.21.15.38:0/2178456703 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71f8003c10 con 0x7f720c075110 2024-03-24T01:03:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.288+0000 7f7208c00640 1 -- 172.21.15.38:0/2178456703 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71f8034440 con 0x7f720c075110 2024-03-24T01:03:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.288+0000 7f7211c00640 1 -- 172.21.15.38:0/2178456703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c1a0ee0 con 0x7f720c075110 2024-03-24T01:03:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.288+0000 7f7211c00640 1 -- 172.21.15.38:0/2178456703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c1a13e0 con 0x7f720c075110 2024-03-24T01:03:54.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.289+0000 7f7211c00640 1 -- 172.21.15.38:0/2178456703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c075f30 con 0x7f720c075110 2024-03-24T01:03:54.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.291+0000 7f7208c00640 1 -- 172.21.15.38:0/2178456703 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f71f8004dd0 con 0x7f720c075110 2024-03-24T01:03:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.292+0000 7f7208c00640 1 --2- 172.21.15.38:0/2178456703 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f71f006df30 0x7f71f00703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.292+0000 7f7208c00640 1 -- 172.21.15.38:0/2178456703 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f71f80b01a0 con 0x7f720c075110 2024-03-24T01:03:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.292+0000 7f720be00640 1 --2- 172.21.15.38:0/2178456703 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f71f006df30 0x7f71f00703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.293+0000 7f720be00640 1 --2- 172.21.15.38:0/2178456703 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f71f006df30 0x7f71f00703f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00a8f0 tx=0x7f71fc005bf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:54.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.296+0000 7f7208c00640 1 -- 172.21.15.38:0/2178456703 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f71f8079640 con 0x7f720c075110 2024-03-24T01:03:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.502+0000 7f7211c00640 1 -- 172.21.15.38:0/2178456703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f720c04e880 con 0x7f720c075110 2024-03-24T01:03:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.504+0000 7f7208c00640 1 -- 172.21.15.38:0/2178456703 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16736 (secure 0 0 0) 0x7f71f807cd00 con 0x7f720c075110 2024-03-24T01:03:54.504 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-03-24T01:03:54.504 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":87,"fsid":"4796a0d6-e976-11ee-95cd-87774f69a715","created":"2024-03-24T00:35:36.968970+0000","modified":"2024-03-24T01:00:46.397340+0000","last_up_change":"2024-03-24T00:59:18.537965+0000","last_in_change":"2024-03-24T00:42:50.726360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T00:41:31.289149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-24T00:44:50.057892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-24T00:44:50.717526+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e5df40af-b5d0-4f89-94b2-59ee2e710bc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6803","nonce":1079856560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6805","nonce":1079856560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6809","nonce":1079856560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1079856560},{"type":"v1","addr":"172.21.15.38:6807","nonce":1079856560}]},"public_addr":"172.21.15.38:6803/1079856560","cluster_addr":"172.21.15.38:6805/1079856560","heartbeat_back_addr":"172.21.15.38:6809/1079856560","heartbeat_front_addr":"172.21.15.38:6807/1079856560","state":["exists","up"]},{"osd":1,"uuid":"ac3c0960-dd6c-44f9-b4b0-234c10f838fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6811","nonce":1120765274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6813","nonce":1120765274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6817","nonce":1120765274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1120765274},{"type":"v1","addr":"172.21.15.38:6815","nonce":1120765274}]},"public_addr":"172.21.15.38:6811/1120765274","cluster_addr":"172.21.15.38:6813/1120765274","heartbeat_back_addr":"172.21.15.38:6817/1120765274","heartbeat_front_addr":"172.21.15.38:6815/1120765274","state":["exists","up"]},{"osd":2,"uuid":"921d9dd3-2471-4b13-b6bd-01e5b6c78648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6819","nonce":874077504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6821","nonce":874077504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6825","nonce":874077504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":874077504},{"type":"v1","addr":"172.21.15.38:6823","nonce":874077504}]},"public_addr":"172.21.15.38:6819/874077504","cluster_addr":"172.21.15.38:6821/874077504","heartbeat_back_addr":"172.21.15.38:6825/874077504","heartbeat_front_addr":"172.21.15.38:6823/874077504","state":["exists","up"]},{"osd":3,"uuid":"7e93813a-0036-4cea-8396-41b8b759c257","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6801","nonce":1863418220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6803","nonce":1863418220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6807","nonce":1863418220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1863418220},{"type":"v1","addr":"172.21.15.88:6805","nonce":1863418220}]},"public_addr":"172.21.15.88:6801/1863418220","cluster_addr":"172.21.15.88:6803/1863418220","heartbeat_back_addr":"172.21.15.88:6807/1863418220","heartbeat_front_addr":"172.21.15.88:6805/1863418220","state":["exists","up"]},{"osd":4,"uuid":"b73f29f7-134e-4725-b341-e893b4bbff8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6809","nonce":2757972748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6811","nonce":2757972748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6815","nonce":2757972748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2757972748},{"type":"v1","addr":"172.21.15.88:6813","nonce":2757972748}]},"public_addr":"172.21.15.88:6809/2757972748","cluster_addr":"172.21.15.88:6811/2757972748","heartbeat_back_addr":"172.21.15.88:6815/2757972748","heartbeat_front_addr":"172.21.15.88:6813/2757972748","state":["exists","up"]},{"osd":5,"uuid":"8ea2cfb3-be7a-4f97-9993-8f9001b8b678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6817","nonce":3428886242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6819","nonce":3428886242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6823","nonce":3428886242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3428886242},{"type":"v1","addr":"172.21.15.88:6821","nonce":3428886242}]},"public_addr":"172.21.15.88:6817/3428886242","cluster_addr":"172.21.15.88:6819/3428886242","heartbeat_back_addr":"172.21.15.88:6823/3428886242","heartbeat_front_addr":"172.21.15.88:6821/3428886242","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:11.140794+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:40:49.877898+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:41:28.612279+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:02.338217+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:42:36.760939+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T00:43:13.811948+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6827/1446532250":"2024-03-25T00:59:25.362289+0000","172.21.15.38:6826/2539263555":"2024-03-25T00:59:25.362259+0000","172.21.15.38:0/3860887425":"2024-03-25T00:36:13.958115+0000","172.21.15.88:0/610291104":"2024-03-25T00:49:19.114620+0000","172.21.15.38:6827/2539263555":"2024-03-25T00:59:25.362259+0000","172.21.15.38:6800/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.38:6800/839555621":"2024-03-25T00:53:56.535603+0000","172.21.15.88:6828/832884416":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/4049972561":"2024-03-25T00:38:28.565032+0000","172.21.15.88:0/3960582080":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/2868668356":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/4189386253":"2024-03-25T00:47:53.298855+0000","172.21.15.88:0/3753305655":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/913707898":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6800/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/3845097007":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/3448795319":"2024-03-25T00:38:28.565032+0000","172.21.15.38:6801/3107810736":"2024-03-25T00:47:53.298855+0000","172.21.15.88:6824/1531706761":"2024-03-25T00:45:13.640830+0000","172.21.15.38:0/1530344157":"2024-03-25T00:53:56.535603+0000","172.21.15.88:0/3790826752":"2024-03-25T00:49:19.114620+0000","172.21.15.38:6801/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6800/3107810736":"2024-03-25T00:47:53.298855+0000","172.21.15.38:0/1949387173":"2024-03-25T00:36:13.958115+0000","172.21.15.38:6801/3284714719":"2024-03-25T00:36:13.958115+0000","172.21.15.38:0/748221873":"2024-03-25T00:36:55.932282+0000","172.21.15.38:6801/1124244493":"2024-03-25T00:38:28.565032+0000","172.21.15.88:6829/832884416":"2024-03-25T00:49:19.114620+0000","172.21.15.88:6826/1446532250":"2024-03-25T00:59:25.362289+0000","172.21.15.88:0/1572172803":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/1962732004":"2024-03-25T00:47:53.298855+0000","172.21.15.38:0/3536812262":"2024-03-25T00:47:53.298855+0000","172.21.15.38:6800/3111371494":"2024-03-25T00:36:55.932282+0000","172.21.15.38:0/773042674":"2024-03-25T00:53:56.535603+0000","172.21.15.38:6801/839555621":"2024-03-25T00:53:56.535603+0000","172.21.15.88:6825/1531706761":"2024-03-25T00:45:13.640830+0000","172.21.15.38:0/633787949":"2024-03-25T00:47:53.298855+0000","172.21.15.88:0/448394986":"2024-03-25T00:49:19.114620+0000","172.21.15.38:0/884740303":"2024-03-25T00:38:28.565032+0000","172.21.15.38:0/2405459521":"2024-03-25T00:53:56.535603+0000","172.21.15.38:0/587822322":"2024-03-25T00:53:56.535603+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T01:03:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.507+0000 7f7211c00640 1 -- 172.21.15.38:0/2178456703 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f71f006df30 msgr2=0x7f71f00703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.507+0000 7f7211c00640 1 --2- 172.21.15.38:0/2178456703 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f71f006df30 0x7f71f00703f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00a8f0 tx=0x7f71fc005bf0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.507+0000 7f7211c00640 1 -- 172.21.15.38:0/2178456703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f720c075110 msgr2=0x7f720c19c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.507+0000 7f7211c00640 1 --2- 172.21.15.38:0/2178456703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f720c075110 0x7f720c19c8c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f71f8005ab0 tx=0x7f71f8004040 comp rx=0 tx=0).stop 2024-03-24T01:03:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.507+0000 7f7211c00640 1 -- 172.21.15.38:0/2178456703 shutdown_connections 2024-03-24T01:03:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.507+0000 7f7211c00640 1 --2- 172.21.15.38:0/2178456703 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f71f006df30 0x7f71f00703f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.507+0000 7f7211c00640 1 --2- 172.21.15.38:0/2178456703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f720c075110 0x7f720c19c8c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.507+0000 7f7211c00640 1 --2- 172.21.15.38:0/2178456703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f720c0747f0 0x7f720c19c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.507+0000 7f7211c00640 1 -- 172.21.15.38:0/2178456703 >> 172.21.15.38:0/2178456703 conn(0x7f720c0fda80 msgr2=0x7f720c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.507+0000 7f7211c00640 1 -- 172.21.15.38:0/2178456703 shutdown_connections 2024-03-24T01:03:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.507+0000 7f7211c00640 1 -- 172.21.15.38:0/2178456703 wait complete. 2024-03-24T01:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T00:41:31.289149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-24T00:44:50.057892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-24T00:44:50.717526+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T01:03:54.527 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T01:03:54.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.738+0000 7f18d7200640 1 -- 172.21.15.38:0/982944893 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18d81021e0 msgr2=0x7f18d81025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:54.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.738+0000 7f18d7200640 1 --2- 172.21.15.38:0/982944893 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18d81021e0 0x7f18d81025c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009880 tx=0x7f18c802bce0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.738+0000 7f18d7200640 1 -- 172.21.15.38:0/982944893 shutdown_connections 2024-03-24T01:03:54.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.738+0000 7f18d7200640 1 --2- 172.21.15.38:0/982944893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18d8102b90 0x7f18d8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.738+0000 7f18d7200640 1 --2- 172.21.15.38:0/982944893 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18d81021e0 0x7f18d81025c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.738+0000 7f18d7200640 1 -- 172.21.15.38:0/982944893 >> 172.21.15.38:0/982944893 conn(0x7f18d80fda40 msgr2=0x7f18d80ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:54.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.738+0000 7f18d7200640 1 -- 172.21.15.38:0/982944893 shutdown_connections 2024-03-24T01:03:54.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.738+0000 7f18d7200640 1 -- 172.21.15.38:0/982944893 wait complete. 2024-03-24T01:03:54.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.739+0000 7f18d7200640 1 Processor -- start 2024-03-24T01:03:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.739+0000 7f18d7200640 1 -- start start 2024-03-24T01:03:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.740+0000 7f18d7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18d81021e0 0x7f18d8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.740+0000 7f18d7200640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18d81985b0 0x7f18d819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.740+0000 7f18d7200640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d8198bb0 con 0x7f18d81985b0 2024-03-24T01:03:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.740+0000 7f18d7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d8198d20 con 0x7f18d81021e0 2024-03-24T01:03:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.740+0000 7f18d5400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18d81985b0 0x7f18d819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:54.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.740+0000 7f18d5400640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18d81985b0 0x7f18d819c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48086/0 (socket says 172.21.15.38:48086) 2024-03-24T01:03:54.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.740+0000 7f18d5400640 1 -- 172.21.15.38:0/137475076 learned_addr learned my addr 172.21.15.38:0/137475076 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:54.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.740+0000 7f18d5e00640 1 --2- 172.21.15.38:0/137475076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18d81021e0 0x7f18d8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:54.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.741+0000 7f18d5400640 1 -- 172.21.15.38:0/137475076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18d81021e0 msgr2=0x7f18d8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.741+0000 7f18d5400640 1 --2- 172.21.15.38:0/137475076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18d81021e0 0x7f18d8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.741+0000 7f18d5400640 1 -- 172.21.15.38:0/137475076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8009560 con 0x7f18d81985b0 2024-03-24T01:03:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.741+0000 7f18d5400640 1 --2- 172.21.15.38:0/137475076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18d81985b0 0x7f18d819c950 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f18c400d6f0 tx=0x7f18c400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.741+0000 7f18cea00640 1 -- 172.21.15.38:0/137475076 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c40051f0 con 0x7f18d81985b0 2024-03-24T01:03:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.741+0000 7f18cea00640 1 -- 172.21.15.38:0/137475076 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18c4009ce0 con 0x7f18d81985b0 2024-03-24T01:03:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.741+0000 7f18d7200640 1 -- 172.21.15.38:0/137475076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d819cf50 con 0x7f18d81985b0 2024-03-24T01:03:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.741+0000 7f18cea00640 1 -- 172.21.15.38:0/137475076 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c4004e80 con 0x7f18d81985b0 2024-03-24T01:03:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.741+0000 7f18d7200640 1 -- 172.21.15.38:0/137475076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d819d4b0 con 0x7f18d81985b0 2024-03-24T01:03:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.743+0000 7f18d7200640 1 -- 172.21.15.38:0/137475076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d819d780 con 0x7f18d81985b0 2024-03-24T01:03:54.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.744+0000 7f18cea00640 1 -- 172.21.15.38:0/137475076 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f18c40046c0 con 0x7f18d81985b0 2024-03-24T01:03:54.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.745+0000 7f18cea00640 1 --2- 172.21.15.38:0/137475076 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18bc06df20 0x7f18bc0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:54.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.745+0000 7f18cea00640 1 -- 172.21.15.38:0/137475076 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f18c408d7c0 con 0x7f18d81985b0 2024-03-24T01:03:54.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.746+0000 7f18d5e00640 1 --2- 172.21.15.38:0/137475076 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18bc06df20 0x7f18bc0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:54.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.746+0000 7f18d5e00640 1 --2- 172.21.15.38:0/137475076 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18bc06df20 0x7f18bc0703e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f18c8000c00 tx=0x7f18c802cf90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:54.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:54.750+0000 7f18cea00640 1 -- 172.21.15.38:0/137475076 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f18c4056bc0 con 0x7f18d81985b0 2024-03-24T01:03:54.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:54 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/1825291666' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T01:03:54.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:54 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2178456703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T01:03:54.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:54 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/1825291666' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T01:03:54.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:54 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2178456703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T01:03:55.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.033+0000 7f18d7200640 1 -- 172.21.15.38:0/137475076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18d810a4e0 con 0x7f18d81985b0 2024-03-24T01:03:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.034+0000 7f18cea00640 1 -- 172.21.15.38:0/137475076 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18c405a280 con 0x7f18d81985b0 2024-03-24T01:03:55.034 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-03-24T01:03:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.037+0000 7f18d7200640 1 -- 172.21.15.38:0/137475076 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18bc06df20 msgr2=0x7f18bc0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.037+0000 7f18d7200640 1 --2- 172.21.15.38:0/137475076 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18bc06df20 0x7f18bc0703e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f18c8000c00 tx=0x7f18c802cf90 comp rx=0 tx=0).stop 2024-03-24T01:03:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.037+0000 7f18d7200640 1 -- 172.21.15.38:0/137475076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18d81985b0 msgr2=0x7f18d819c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.037+0000 7f18d7200640 1 --2- 172.21.15.38:0/137475076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18d81985b0 0x7f18d819c950 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f18c400d6f0 tx=0x7f18c400dbb0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.037+0000 7f18d7200640 1 -- 172.21.15.38:0/137475076 shutdown_connections 2024-03-24T01:03:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.037+0000 7f18d7200640 1 --2- 172.21.15.38:0/137475076 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f18bc06df20 0x7f18bc0703e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.037+0000 7f18d7200640 1 --2- 172.21.15.38:0/137475076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18d81985b0 0x7f18d819c950 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.037+0000 7f18d7200640 1 --2- 172.21.15.38:0/137475076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18d81021e0 0x7f18d8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.037+0000 7f18d7200640 1 -- 172.21.15.38:0/137475076 >> 172.21.15.38:0/137475076 conn(0x7f18d80fda40 msgr2=0x7f18d80fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.037+0000 7f18d7200640 1 -- 172.21.15.38:0/137475076 shutdown_connections 2024-03-24T01:03:55.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.037+0000 7f18d7200640 1 -- 172.21.15.38:0/137475076 wait complete. 2024-03-24T01:03:55.055 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-24T01:03:55.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.275+0000 7ff6c7e00640 1 -- 172.21.15.38:0/229702658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6c8101820 msgr2=0x7ff6c8109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:55.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.275+0000 7ff6c7e00640 1 --2- 172.21.15.38:0/229702658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6c8101820 0x7ff6c8109d00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009880 tx=0x7ff6b402bd80 comp rx=0 tx=0).stop 2024-03-24T01:03:55.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.275+0000 7ff6c7e00640 1 -- 172.21.15.38:0/229702658 shutdown_connections 2024-03-24T01:03:55.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.275+0000 7ff6c7e00640 1 --2- 172.21.15.38:0/229702658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6c8101820 0x7ff6c8109d00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.275+0000 7ff6c7e00640 1 --2- 172.21.15.38:0/229702658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6c8100f00 0x7ff6c81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.275+0000 7ff6c7e00640 1 -- 172.21.15.38:0/229702658 >> 172.21.15.38:0/229702658 conn(0x7ff6c80758e0 msgr2=0x7ff6c80fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:55.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.275+0000 7ff6c7e00640 1 -- 172.21.15.38:0/229702658 shutdown_connections 2024-03-24T01:03:55.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.275+0000 7ff6c7e00640 1 -- 172.21.15.38:0/229702658 wait complete. 2024-03-24T01:03:55.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.276+0000 7ff6c7e00640 1 Processor -- start 2024-03-24T01:03:55.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.277+0000 7ff6c7e00640 1 -- start start 2024-03-24T01:03:55.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.277+0000 7ff6c7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6c8100f00 0x7ff6c8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:55.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.277+0000 7ff6c7e00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6c8101820 0x7ff6c81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:55.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.277+0000 7ff6c7e00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c8198b80 con 0x7ff6c8101820 2024-03-24T01:03:55.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.277+0000 7ff6c7e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c819c890 con 0x7ff6c8100f00 2024-03-24T01:03:55.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.277+0000 7ff6c6000640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6c8101820 0x7ff6c81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:55.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.278+0000 7ff6c6000640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6c8101820 0x7ff6c81984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48110/0 (socket says 172.21.15.38:48110) 2024-03-24T01:03:55.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.278+0000 7ff6c6000640 1 -- 172.21.15.38:0/850263699 learned_addr learned my addr 172.21.15.38:0/850263699 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:55.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.278+0000 7ff6c6a00640 1 --2- 172.21.15.38:0/850263699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6c8100f00 0x7ff6c8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:55.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.278+0000 7ff6c6000640 1 -- 172.21.15.38:0/850263699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6c8100f00 msgr2=0x7ff6c8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:55.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.278+0000 7ff6c6000640 1 --2- 172.21.15.38:0/850263699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6c8100f00 0x7ff6c8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.278+0000 7ff6c6000640 1 -- 172.21.15.38:0/850263699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b4009560 con 0x7ff6c8101820 2024-03-24T01:03:55.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.278+0000 7ff6c6000640 1 --2- 172.21.15.38:0/850263699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6c8101820 0x7ff6c81984f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4006010 tx=0x7ff6b4004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:55.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.279+0000 7ff6bf400640 1 -- 172.21.15.38:0/850263699 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6b4004420 con 0x7ff6c8101820 2024-03-24T01:03:55.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.279+0000 7ff6bf400640 1 -- 172.21.15.38:0/850263699 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6b4003ab0 con 0x7ff6c8101820 2024-03-24T01:03:55.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.279+0000 7ff6c7e00640 1 -- 172.21.15.38:0/850263699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c819cb10 con 0x7ff6c8101820 2024-03-24T01:03:55.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.279+0000 7ff6bf400640 1 -- 172.21.15.38:0/850263699 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6b4005520 con 0x7ff6c8101820 2024-03-24T01:03:55.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.279+0000 7ff6c7e00640 1 -- 172.21.15.38:0/850263699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c819d070 con 0x7ff6c8101820 2024-03-24T01:03:55.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.281+0000 7ff6c7e00640 1 -- 172.21.15.38:0/850263699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c8067b20 con 0x7ff6c8101820 2024-03-24T01:03:55.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.281+0000 7ff6bf400640 1 -- 172.21.15.38:0/850263699 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7ff6b4003c20 con 0x7ff6c8101820 2024-03-24T01:03:55.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.282+0000 7ff6bf400640 1 --2- 172.21.15.38:0/850263699 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff6ac06df20 0x7ff6ac0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:55.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.282+0000 7ff6bf400640 1 -- 172.21.15.38:0/850263699 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff6b40b0850 con 0x7ff6c8101820 2024-03-24T01:03:55.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.282+0000 7ff6c6a00640 1 --2- 172.21.15.38:0/850263699 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff6ac06df20 0x7ff6ac0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:55.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.283+0000 7ff6c6a00640 1 --2- 172.21.15.38:0/850263699 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff6ac06df20 0x7ff6ac0703e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8005fd0 tx=0x7ff6b8005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.291+0000 7ff6bf400640 1 -- 172.21.15.38:0/850263699 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff6b4079cf0 con 0x7ff6c8101820 2024-03-24T01:03:55.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.544+0000 7ff6c7e00640 1 -- 172.21.15.38:0/850263699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7ff6c804e880 con 0x7ff6c8101820 2024-03-24T01:03:55.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.545+0000 7ff6bf400640 1 -- 172.21.15.38:0/850263699 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff6b407d3b0 con 0x7ff6c8101820 2024-03-24T01:03:55.546 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-03-24T01:03:55.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.549+0000 7ff6c7e00640 1 -- 172.21.15.38:0/850263699 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff6ac06df20 msgr2=0x7ff6ac0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:55.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.549+0000 7ff6c7e00640 1 --2- 172.21.15.38:0/850263699 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff6ac06df20 0x7ff6ac0703e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8005fd0 tx=0x7ff6b8005850 comp rx=0 tx=0).stop 2024-03-24T01:03:55.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.549+0000 7ff6c7e00640 1 -- 172.21.15.38:0/850263699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6c8101820 msgr2=0x7ff6c81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:55.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.549+0000 7ff6c7e00640 1 --2- 172.21.15.38:0/850263699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6c8101820 0x7ff6c81984f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4006010 tx=0x7ff6b4004a10 comp rx=0 tx=0).stop 2024-03-24T01:03:55.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.549+0000 7ff6c7e00640 1 -- 172.21.15.38:0/850263699 shutdown_connections 2024-03-24T01:03:55.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.550+0000 7ff6c7e00640 1 --2- 172.21.15.38:0/850263699 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7ff6ac06df20 0x7ff6ac0703e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.550+0000 7ff6c7e00640 1 --2- 172.21.15.38:0/850263699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6c8101820 0x7ff6c81984f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.550+0000 7ff6c7e00640 1 --2- 172.21.15.38:0/850263699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6c8100f00 0x7ff6c8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.550+0000 7ff6c7e00640 1 -- 172.21.15.38:0/850263699 >> 172.21.15.38:0/850263699 conn(0x7ff6c80758e0 msgr2=0x7ff6c81002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:55.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.550+0000 7ff6c7e00640 1 -- 172.21.15.38:0/850263699 shutdown_connections 2024-03-24T01:03:55.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.550+0000 7ff6c7e00640 1 -- 172.21.15.38:0/850263699 wait complete. 2024-03-24T01:03:55.572 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-24T01:03:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.787+0000 7f6128a00640 1 -- 172.21.15.38:0/155374118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61241021c0 msgr2=0x7f61241025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.787+0000 7f6128a00640 1 --2- 172.21.15.38:0/155374118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61241021c0 0x7f61241025a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6114009880 tx=0x7f611402bce0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.787+0000 7f6128a00640 1 -- 172.21.15.38:0/155374118 shutdown_connections 2024-03-24T01:03:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.787+0000 7f6128a00640 1 --2- 172.21.15.38:0/155374118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6124102b70 0x7f6124106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.787+0000 7f6128a00640 1 --2- 172.21.15.38:0/155374118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61241021c0 0x7f61241025a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.787+0000 7f6128a00640 1 -- 172.21.15.38:0/155374118 >> 172.21.15.38:0/155374118 conn(0x7f61240fda40 msgr2=0x7f61240ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.787+0000 7f6128a00640 1 -- 172.21.15.38:0/155374118 shutdown_connections 2024-03-24T01:03:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.787+0000 7f6128a00640 1 -- 172.21.15.38:0/155374118 wait complete. 2024-03-24T01:03:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.788+0000 7f6128a00640 1 Processor -- start 2024-03-24T01:03:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.789+0000 7f6128a00640 1 -- start start 2024-03-24T01:03:55.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.789+0000 7f6128a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61241021c0 0x7f6124198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:55.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.789+0000 7f6128a00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6124198570 0x7f612419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:55.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.789+0000 7f6128a00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6124198b70 con 0x7f6124198570 2024-03-24T01:03:55.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.789+0000 7f6128a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6124198ce0 con 0x7f61241021c0 2024-03-24T01:03:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.790+0000 7f611be00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6124198570 0x7f612419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.790+0000 7f611be00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6124198570 0x7f612419c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48118/0 (socket says 172.21.15.38:48118) 2024-03-24T01:03:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.790+0000 7f611be00640 1 -- 172.21.15.38:0/329576626 learned_addr learned my addr 172.21.15.38:0/329576626 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.790+0000 7f6123400640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61241021c0 0x7f6124198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.790+0000 7f611be00640 1 -- 172.21.15.38:0/329576626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61241021c0 msgr2=0x7f6124198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.790+0000 7f611be00640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61241021c0 0x7f6124198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.790+0000 7f611be00640 1 -- 172.21.15.38:0/329576626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6114009560 con 0x7f6124198570 2024-03-24T01:03:55.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.790+0000 7f6123400640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61241021c0 0x7f6124198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T01:03:55.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.790+0000 7f611be00640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6124198570 0x7f612419c910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f610c00c920 tx=0x7f610c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:55.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.791+0000 7f6120c00640 1 -- 172.21.15.38:0/329576626 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f610c007b80 con 0x7f6124198570 2024-03-24T01:03:55.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.791+0000 7f6128a00640 1 -- 172.21.15.38:0/329576626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612419cf10 con 0x7f6124198570 2024-03-24T01:03:55.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.791+0000 7f6120c00640 1 -- 172.21.15.38:0/329576626 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f610c0054e0 con 0x7f6124198570 2024-03-24T01:03:55.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.791+0000 7f6120c00640 1 -- 172.21.15.38:0/329576626 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f610c00f450 con 0x7f6124198570 2024-03-24T01:03:55.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.791+0000 7f6128a00640 1 -- 172.21.15.38:0/329576626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612419d470 con 0x7f6124198570 2024-03-24T01:03:55.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.793+0000 7f6128a00640 1 -- 172.21.15.38:0/329576626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612419d750 con 0x7f6124198570 2024-03-24T01:03:55.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.794+0000 7f6120c00640 1 -- 172.21.15.38:0/329576626 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f610c0046c0 con 0x7f6124198570 2024-03-24T01:03:55.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.795+0000 7f6120c00640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f610806df10 0x7f61080703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:55.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.795+0000 7f6120c00640 1 -- 172.21.15.38:0/329576626 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f610c08dbf0 con 0x7f6124198570 2024-03-24T01:03:55.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.796+0000 7f6123400640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f610806df10 0x7f61080703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:55.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.796+0000 7f6123400640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f610806df10 0x7f61080703d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6114000c00 tx=0x7f611402cb60 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:55.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:55.800+0000 7f6120c00640 1 -- 172.21.15.38:0/329576626 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f610c057010 con 0x7f6124198570 2024-03-24T01:03:55.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:55 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/137475076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T01:03:55.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:55 smithi038 ceph-mon[112108]: pgmap v339: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 170 B/s wr, 4 op/s 2024-03-24T01:03:55.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:55 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/850263699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T01:03:55.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:55 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/137475076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T01:03:55.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:55 smithi088 ceph-mon[87351]: pgmap v339: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 170 B/s wr, 4 op/s 2024-03-24T01:03:55.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:55 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/850263699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T01:03:56.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.000+0000 7f6128a00640 1 -- 172.21.15.38:0/329576626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f6124199330 con 0x7f6124198570 2024-03-24T01:03:56.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.001+0000 7f6120c00640 1 -- 172.21.15.38:0/329576626 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f610c05a6d0 con 0x7f6124198570 2024-03-24T01:03:56.002 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-03-24T01:03:56.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.004+0000 7f6128a00640 1 -- 172.21.15.38:0/329576626 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f610806df10 msgr2=0x7f61080703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:56.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.004+0000 7f6128a00640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f610806df10 0x7f61080703d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6114000c00 tx=0x7f611402cb60 comp rx=0 tx=0).stop 2024-03-24T01:03:56.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.004+0000 7f6128a00640 1 -- 172.21.15.38:0/329576626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6124198570 msgr2=0x7f612419c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:56.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.004+0000 7f6128a00640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6124198570 0x7f612419c910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f610c00c920 tx=0x7f610c00cde0 comp rx=0 tx=0).stop 2024-03-24T01:03:56.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.004+0000 7f6128a00640 1 -- 172.21.15.38:0/329576626 shutdown_connections 2024-03-24T01:03:56.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.004+0000 7f6128a00640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7f610806df10 0x7f61080703d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:56.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.004+0000 7f6128a00640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6124198570 0x7f612419c910 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:56.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.004+0000 7f6128a00640 1 --2- 172.21.15.38:0/329576626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61241021c0 0x7f6124198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:56.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.004+0000 7f6128a00640 1 -- 172.21.15.38:0/329576626 >> 172.21.15.38:0/329576626 conn(0x7f61240fda40 msgr2=0x7f61240fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:56.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.004+0000 7f6128a00640 1 -- 172.21.15.38:0/329576626 shutdown_connections 2024-03-24T01:03:56.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.005+0000 7f6128a00640 1 -- 172.21.15.38:0/329576626 wait complete. 2024-03-24T01:03:56.025 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-24T01:03:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.228+0000 7fc6f6600640 1 -- 172.21.15.38:0/4281196155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6f0072660 msgr2=0x7fc6f0072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.228+0000 7fc6f6600640 1 --2- 172.21.15.38:0/4281196155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6f0072660 0x7fc6f0072a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009880 tx=0x7fc6e402bd90 comp rx=0 tx=0).stop 2024-03-24T01:03:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.229+0000 7fc6f6600640 1 -- 172.21.15.38:0/4281196155 shutdown_connections 2024-03-24T01:03:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.229+0000 7fc6f6600640 1 --2- 172.21.15.38:0/4281196155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f0072f80 0x7fc6f01107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.229+0000 7fc6f6600640 1 --2- 172.21.15.38:0/4281196155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6f0072660 0x7fc6f0072a40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.229+0000 7fc6f6600640 1 -- 172.21.15.38:0/4281196155 >> 172.21.15.38:0/4281196155 conn(0x7fc6f00fb820 msgr2=0x7fc6f00fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.229+0000 7fc6f6600640 1 -- 172.21.15.38:0/4281196155 shutdown_connections 2024-03-24T01:03:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.229+0000 7fc6f6600640 1 -- 172.21.15.38:0/4281196155 wait complete. 2024-03-24T01:03:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.230+0000 7fc6f6600640 1 Processor -- start 2024-03-24T01:03:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.230+0000 7fc6f6600640 1 -- start start 2024-03-24T01:03:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.230+0000 7fc6f6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f0072660 0x7fc6f019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.230+0000 7fc6f6600640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6f0072f80 0x7fc6f019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.230+0000 7fc6f6600640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f019cf30 con 0x7fc6f0072f80 2024-03-24T01:03:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.230+0000 7fc6f6600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f01a0bf0 con 0x7fc6f0072660 2024-03-24T01:03:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.230+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6f0072f80 0x7fc6f019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.231+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6f0072f80 0x7fc6f019c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48134/0 (socket says 172.21.15.38:48134) 2024-03-24T01:03:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.231+0000 7fc6efe00640 1 -- 172.21.15.38:0/2268797421 learned_addr learned my addr 172.21.15.38:0/2268797421 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-03-24T01:03:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.231+0000 7fc6f5200640 1 --2- 172.21.15.38:0/2268797421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f0072660 0x7fc6f019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.231+0000 7fc6efe00640 1 -- 172.21.15.38:0/2268797421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f0072660 msgr2=0x7fc6f019c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.231+0000 7fc6efe00640 1 --2- 172.21.15.38:0/2268797421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f0072660 0x7fc6f019c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.231+0000 7fc6efe00640 1 -- 172.21.15.38:0/2268797421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e4009560 con 0x7fc6f0072f80 2024-03-24T01:03:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.231+0000 7fc6efe00640 1 --2- 172.21.15.38:0/2268797421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6f0072f80 0x7fc6f019c8a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00d6f0 tx=0x7fc6dc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:56.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.232+0000 7fc6ed600640 1 -- 172.21.15.38:0/2268797421 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6dc009a30 con 0x7fc6f0072f80 2024-03-24T01:03:56.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.232+0000 7fc6ed600640 1 -- 172.21.15.38:0/2268797421 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6dc0054e0 con 0x7fc6f0072f80 2024-03-24T01:03:56.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.232+0000 7fc6ed600640 1 -- 172.21.15.38:0/2268797421 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6dc010460 con 0x7fc6f0072f80 2024-03-24T01:03:56.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.232+0000 7fc6f6600640 1 -- 172.21.15.38:0/2268797421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f01a0ed0 con 0x7fc6f0072f80 2024-03-24T01:03:56.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.232+0000 7fc6f6600640 1 -- 172.21.15.38:0/2268797421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f01a13e0 con 0x7fc6f0072f80 2024-03-24T01:03:56.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.233+0000 7fc6f6600640 1 -- 172.21.15.38:0/2268797421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f0073da0 con 0x7fc6f0072f80 2024-03-24T01:03:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.235+0000 7fc6ed600640 1 -- 172.21.15.38:0/2268797421 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 39) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fc6dc009b90 con 0x7fc6f0072f80 2024-03-24T01:03:56.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.236+0000 7fc6ed600640 1 --2- 172.21.15.38:0/2268797421 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc6d806df30 0x7fc6d80703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T01:03:56.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.236+0000 7fc6ed600640 1 -- 172.21.15.38:0/2268797421 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc6dc08d9f0 con 0x7fc6f0072f80 2024-03-24T01:03:56.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.236+0000 7fc6f5200640 1 --2- 172.21.15.38:0/2268797421 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc6d806df30 0x7fc6d80703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T01:03:56.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.237+0000 7fc6f5200640 1 --2- 172.21.15.38:0/2268797421 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc6d806df30 0x7fc6d80703f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4000c00 tx=0x7fc6e4005a20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T01:03:56.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.242+0000 7fc6ed600640 1 -- 172.21.15.38:0/2268797421 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc6dc056d70 con 0x7fc6f0072f80 2024-03-24T01:03:56.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.460+0000 7fc6f6600640 1 -- 172.21.15.38:0/2268797421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc6f004e880 con 0x7fc6f0072f80 2024-03-24T01:03:56.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.461+0000 7fc6ed600640 1 -- 172.21.15.38:0/2268797421 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v87) v1 ==== 81+0+2420 (secure 0 0 0) 0x7fc6dc05a430 con 0x7fc6f0072f80 2024-03-24T01:03:56.462 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6827/1446532250 2024-03-25T00:59:25.362289+0000 2024-03-24T01:03:56.462 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6826/2539263555 2024-03-25T00:59:25.362259+0000 2024-03-24T01:03:56.462 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3860887425 2024-03-25T00:36:13.958115+0000 2024-03-24T01:03:56.462 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/610291104 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:56.462 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6827/2539263555 2024-03-25T00:59:25.362259+0000 2024-03-24T01:03:56.462 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/1124244493 2024-03-25T00:38:28.565032+0000 2024-03-24T01:03:56.462 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/839555621 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:56.462 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6828/832884416 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/4049972561 2024-03-25T00:38:28.565032+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/3960582080 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2868668356 2024-03-25T00:36:55.932282+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/4189386253 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/3753305655 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/913707898 2024-03-25T00:36:13.958115+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/3284714719 2024-03-25T00:36:13.958115+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3845097007 2024-03-25T00:36:55.932282+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3448795319 2024-03-25T00:38:28.565032+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/3107810736 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6824/1531706761 2024-03-25T00:45:13.640830+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1530344157 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/3790826752 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:56.463 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/3111371494 2024-03-25T00:36:55.932282+0000 2024-03-24T01:03:56.464 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/3107810736 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:56.464 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1949387173 2024-03-25T00:36:13.958115+0000 2024-03-24T01:03:56.464 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/3284714719 2024-03-25T00:36:13.958115+0000 2024-03-24T01:03:56.464 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/748221873 2024-03-25T00:36:55.932282+0000 2024-03-24T01:03:56.464 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/1124244493 2024-03-25T00:38:28.565032+0000 2024-03-24T01:03:56.464 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6829/832884416 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:56.464 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6826/1446532250 2024-03-25T00:59:25.362289+0000 2024-03-24T01:03:56.464 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/1572172803 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:56.464 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1962732004 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:56.464 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3536812262 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:56.464 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/3111371494 2024-03-25T00:36:55.932282+0000 2024-03-24T01:03:56.465 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/773042674 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:56.465 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/839555621 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:56.465 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:6825/1531706761 2024-03-25T00:45:13.640830+0000 2024-03-24T01:03:56.465 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/633787949 2024-03-25T00:47:53.298855+0000 2024-03-24T01:03:56.465 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.88:0/448394986 2024-03-25T00:49:19.114620+0000 2024-03-24T01:03:56.465 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/884740303 2024-03-25T00:38:28.565032+0000 2024-03-24T01:03:56.465 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2405459521 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:56.465 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/587822322 2024-03-25T00:53:56.535603+0000 2024-03-24T01:03:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:listed 41 entries 2024-03-24T01:03:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.463+0000 7fc6f6600640 1 -- 172.21.15.38:0/2268797421 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc6d806df30 msgr2=0x7fc6d80703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.463+0000 7fc6f6600640 1 --2- 172.21.15.38:0/2268797421 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc6d806df30 0x7fc6d80703f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4000c00 tx=0x7fc6e4005a20 comp rx=0 tx=0).stop 2024-03-24T01:03:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.463+0000 7fc6f6600640 1 -- 172.21.15.38:0/2268797421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6f0072f80 msgr2=0x7fc6f019c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T01:03:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.463+0000 7fc6f6600640 1 --2- 172.21.15.38:0/2268797421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6f0072f80 0x7fc6f019c8a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00d6f0 tx=0x7fc6dc00dbb0 comp rx=0 tx=0).stop 2024-03-24T01:03:56.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.464+0000 7fc6f6600640 1 -- 172.21.15.38:0/2268797421 shutdown_connections 2024-03-24T01:03:56.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.464+0000 7fc6f6600640 1 --2- 172.21.15.38:0/2268797421 >> [v2:172.21.15.38:6800/1632725319,v1:172.21.15.38:6801/1632725319] conn(0x7fc6d806df30 0x7fc6d80703f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:56.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.464+0000 7fc6f6600640 1 --2- 172.21.15.38:0/2268797421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6f0072f80 0x7fc6f019c8a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:56.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.464+0000 7fc6f6600640 1 --2- 172.21.15.38:0/2268797421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f0072660 0x7fc6f019c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T01:03:56.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.464+0000 7fc6f6600640 1 -- 172.21.15.38:0/2268797421 >> 172.21.15.38:0/2268797421 conn(0x7fc6f00fb820 msgr2=0x7fc6f00ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T01:03:56.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.464+0000 7fc6f6600640 1 -- 172.21.15.38:0/2268797421 shutdown_connections 2024-03-24T01:03:56.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-03-24T01:03:56.464+0000 7fc6f6600640 1 -- 172.21.15.38:0/2268797421 wait complete. 2024-03-24T01:03:56.483 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-24T01:03:56.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T01:03:56.483 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-24T01:03:56.592 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-03-24T01:03:56.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T01:03:56.592 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-24T01:03:56.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T01:03:56.682 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-03-24T01:03:56.740 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-24T01:03:56.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T01:03:56.740 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-24T01:03:56.808 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:56 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/329576626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T01:03:56.808 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:56 smithi038 ceph-mon[112108]: from='client.? 172.21.15.38:0/2268797421' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T01:03:56.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T01:03:56.821 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-03-24T01:03:56.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:56 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/329576626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T01:03:56.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:56 smithi088 ceph-mon[87351]: from='client.? 172.21.15.38:0/2268797421' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T01:03:56.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T01:03:56.940 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-03-24T01:03:56.995 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-24T01:03:56.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T01:03:56.995 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-24T01:03:57.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T01:03:57.093 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-03-24T01:03:57.210 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-24T01:03:57.222 INFO:tasks.cephadm:Teardown begin 2024-03-24T01:03:57.222 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T01:03:57.303 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T01:03:57.399 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-24T01:03:57.400 DEBUG:teuthology.orchestra.run.smithi038:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-24T01:03:57.446 DEBUG:teuthology.orchestra.run.smithi088:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-24T01:03:57.506 INFO:tasks.cephadm:Stopping all daemons... 2024-03-24T01:03:57.506 INFO:tasks.cephadm.mon.smithi038:Stopping mon.smithi038... 2024-03-24T01:03:57.506 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi038 2024-03-24T01:03:57.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:57 smithi038 systemd[1]: Stopping Ceph mon.smithi038 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T01:03:57.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:57 smithi038 ceph-mon[112108]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:03:57.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:57 smithi038 ceph-mon[112108]: pgmap v340: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-24T01:03:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:57 smithi088 ceph-mon[87351]: from='mgr.34100 172.21.15.38:0/2282073132' entity='mgr.smithi038.huhioz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T01:03:57.930 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:57 smithi088 ceph-mon[87351]: pgmap v340: 65 pgs: 65 active+clean; 195 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-24T01:03:58.188 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:58 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[112104]: 2024-03-24T01:03:58.048+0000 7f325b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi038 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T01:03:58.188 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:58 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038[112104]: 2024-03-24T01:03:58.048+0000 7f325b800700 -1 mon.smithi038@0(leader) e3 *** Got Signal Terminated *** 2024-03-24T01:03:58.188 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:58 smithi038 podman[157568]: 2024-03-24 01:03:58.182817959 +0000 UTC m=+0.278169951 container died ba098af28413976f02407c0b6e9dab98a3c86c9ce6492c392fb9daff638f289b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-24T01:03:58.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:58 smithi038 podman[157568]: 2024-03-24 01:03:58.684107112 +0000 UTC m=+0.779459125 container cleanup ba098af28413976f02407c0b6e9dab98a3c86c9ce6492c392fb9daff638f289b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-03-24T01:03:58.859 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:58 smithi038 bash[157568]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038 2024-03-24T01:03:59.204 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Mar 24 01:03:58 smithi038 podman[157593]: 2024-03-24 01:03:58.877371513 +0000 UTC m=+0.675449874 container remove ba098af28413976f02407c0b6e9dab98a3c86c9ce6492c392fb9daff638f289b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi038, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-24T01:03:59.236 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi038.service' 2024-03-24T01:03:59.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T01:03:59.352 INFO:tasks.cephadm.mon.smithi038:Stopped mon.smithi038 2024-03-24T01:03:59.353 INFO:tasks.cephadm.mon.smithi088:Stopping mon.smithi088... 2024-03-24T01:03:59.353 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi088 2024-03-24T01:03:59.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:59 smithi088 systemd[1]: Stopping Ceph mon.smithi088 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T01:04:00.087 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:59 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088[87347]: 2024-03-24T01:03:59.972+0000 7fd5fe600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi088 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T01:04:00.615 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:03:59 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088[87347]: 2024-03-24T01:03:59.972+0000 7fd5fe600700 -1 mon.smithi088@1(peon) e3 *** Got Signal Terminated *** 2024-03-24T01:04:00.615 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:04:00 smithi088 podman[123611]: 2024-03-24 01:04:00.107731947 +0000 UTC m=+0.365342733 container died 3bace4fab2b0fb908fbcbd88a66e3edb7110cf4c2bc1a4d9e19c2056491af2e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-24T01:04:00.697 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:04:00 smithi088 podman[123611]: 2024-03-24 01:04:00.658583937 +0000 UTC m=+0.916194713 container cleanup 3bace4fab2b0fb908fbcbd88a66e3edb7110cf4c2bc1a4d9e19c2056491af2e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T01:04:00.697 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:04:00 smithi088 bash[123611]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088 2024-03-24T01:04:01.426 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Mar 24 01:04:00 smithi088 podman[123633]: 2024-03-24 01:04:00.935278684 +0000 UTC m=+0.809953831 container remove 3bace4fab2b0fb908fbcbd88a66e3edb7110cf4c2bc1a4d9e19c2056491af2e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-mon-smithi088, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T01:04:01.461 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@mon.smithi088.service' 2024-03-24T01:04:01.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T01:04:01.606 INFO:tasks.cephadm.mon.smithi088:Stopped mon.smithi088 2024-03-24T01:04:01.606 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-24T01:04:01.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.0 2024-03-24T01:04:02.109 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:01 smithi038 systemd[1]: Stopping Ceph osd.0 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T01:04:02.609 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:02 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0[123966]: 2024-03-24T01:04:02.287+0000 7fd34e600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T01:04:02.610 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:02 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0[123966]: 2024-03-24T01:04:02.287+0000 7fd34e600700 -1 osd.0 87 *** Got signal Terminated *** 2024-03-24T01:04:02.610 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:02 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0[123966]: 2024-03-24T01:04:02.287+0000 7fd34e600700 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T01:04:07.995 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:07 smithi038 podman[157698]: 2024-03-24 01:04:07.678863582 +0000 UTC m=+5.637353292 container died a74a6e0da73c4dc1861a47f131dcad5ae1dd7e15b87bfadb0b1d7e844828cd14 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-24T01:04:08.359 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:08 smithi038 podman[157698]: 2024-03-24 01:04:08.121789325 +0000 UTC m=+6.080279035 container cleanup a74a6e0da73c4dc1861a47f131dcad5ae1dd7e15b87bfadb0b1d7e844828cd14 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T01:04:08.359 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:08 smithi038 bash[157698]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0 2024-03-24T01:04:08.859 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:08 smithi038 podman[157717]: 2024-03-24 01:04:08.374952849 +0000 UTC m=+0.686782253 container remove a74a6e0da73c4dc1861a47f131dcad5ae1dd7e15b87bfadb0b1d7e844828cd14 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-24T01:04:09.384 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:09 smithi038 podman[157800]: 2024-03-24 01:04:09.036790717 +0000 UTC m=+0.036198216 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T01:04:09.384 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:09 smithi038 podman[157800]: 2024-03-24 01:04:09.166046131 +0000 UTC m=+0.165453634 container create 39de07264bb7fb5d3da7adf90453fbc2035e19641b3462eefacdbd698b21e614 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-24T01:04:09.860 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:09 smithi038 podman[157800]: 2024-03-24 01:04:09.478537777 +0000 UTC m=+0.477945313 container init 39de07264bb7fb5d3da7adf90453fbc2035e19641b3462eefacdbd698b21e614 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-24T01:04:09.860 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:09 smithi038 podman[157800]: 2024-03-24 01:04:09.492270879 +0000 UTC m=+0.491678436 container start 39de07264bb7fb5d3da7adf90453fbc2035e19641b3462eefacdbd698b21e614 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-24T01:04:09.860 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:09 smithi038 podman[157800]: 2024-03-24 01:04:09.568761552 +0000 UTC m=+0.568169097 container attach 39de07264bb7fb5d3da7adf90453fbc2035e19641b3462eefacdbd698b21e614 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T01:04:10.360 INFO:journalctl@ceph.osd.0.smithi038.stdout:Mar 24 01:04:09 smithi038 podman[157800]: 2024-03-24 01:04:09.98011175 +0000 UTC m=+0.979519265 container died 39de07264bb7fb5d3da7adf90453fbc2035e19641b3462eefacdbd698b21e614 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-24T01:04:10.725 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.0.service' 2024-03-24T01:04:10.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T01:04:10.840 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-24T01:04:10.840 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-24T01:04:10.840 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.1 2024-03-24T01:04:11.359 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:10 smithi038 systemd[1]: Stopping Ceph osd.1 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T01:04:11.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:11 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1[129361]: 2024-03-24T01:04:11.472+0000 7f065ec00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T01:04:11.859 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:11 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1[129361]: 2024-03-24T01:04:11.472+0000 7f065ec00700 -1 osd.1 87 *** Got signal Terminated *** 2024-03-24T01:04:11.860 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:11 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1[129361]: 2024-03-24T01:04:11.472+0000 7f065ec00700 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T01:04:17.359 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:17 smithi038 podman[157903]: 2024-03-24 01:04:17.087036559 +0000 UTC m=+5.766308148 container died cb72ac446ba713b767f4e0e8673d9dfcf93beb01a5572fd1d5b1854422d89958 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-24T01:04:17.768 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:17 smithi038 podman[157903]: 2024-03-24 01:04:17.566996049 +0000 UTC m=+6.246267638 container cleanup cb72ac446ba713b767f4e0e8673d9dfcf93beb01a5572fd1d5b1854422d89958 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-24T01:04:17.768 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:17 smithi038 bash[157903]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1 2024-03-24T01:04:18.109 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:17 smithi038 podman[157921]: 2024-03-24 01:04:17.768403164 +0000 UTC m=+0.661119936 container remove cb72ac446ba713b767f4e0e8673d9dfcf93beb01a5572fd1d5b1854422d89958 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-24T01:04:18.697 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:18 smithi038 podman[158009]: 2024-03-24 01:04:18.444647705 +0000 UTC m=+0.040986435 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T01:04:18.697 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:18 smithi038 podman[158009]: 2024-03-24 01:04:18.544245326 +0000 UTC m=+0.140584059 container create 368e6356b155b80061beef23ec36208f134172739f14474cc8b88b797f4483a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-24T01:04:19.109 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:18 smithi038 podman[158009]: 2024-03-24 01:04:18.797517067 +0000 UTC m=+0.393855806 container init 368e6356b155b80061beef23ec36208f134172739f14474cc8b88b797f4483a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-24T01:04:19.109 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:18 smithi038 podman[158009]: 2024-03-24 01:04:18.807763861 +0000 UTC m=+0.404102594 container start 368e6356b155b80061beef23ec36208f134172739f14474cc8b88b797f4483a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T01:04:19.109 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:18 smithi038 podman[158009]: 2024-03-24 01:04:18.880509687 +0000 UTC m=+0.476848487 container attach 368e6356b155b80061beef23ec36208f134172739f14474cc8b88b797f4483a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-24T01:04:19.577 INFO:journalctl@ceph.osd.1.smithi038.stdout:Mar 24 01:04:19 smithi038 podman[158009]: 2024-03-24 01:04:19.2662781 +0000 UTC m=+0.862616872 container died 368e6356b155b80061beef23ec36208f134172739f14474cc8b88b797f4483a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-24T01:04:19.959 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.1.service' 2024-03-24T01:04:20.076 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T01:04:20.076 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-24T01:04:20.076 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-24T01:04:20.077 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.2 2024-03-24T01:04:20.609 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:20 smithi038 systemd[1]: Stopping Ceph osd.2 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T01:04:21.109 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:20 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2[134752]: 2024-03-24T01:04:20.640+0000 7fba38a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T01:04:21.109 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:20 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2[134752]: 2024-03-24T01:04:20.640+0000 7fba38a00700 -1 osd.2 87 *** Got signal Terminated *** 2024-03-24T01:04:21.110 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:20 smithi038 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2[134752]: 2024-03-24T01:04:20.640+0000 7fba38a00700 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T01:04:26.444 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:26 smithi038 podman[158114]: 2024-03-24 01:04:26.117740875 +0000 UTC m=+5.607148512 container died c73f3e8276b63995c4ddb70752dcd636dc841609ba1adc15392f26e8a6a8831b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T01:04:26.721 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:26 smithi038 podman[158114]: 2024-03-24 01:04:26.653781422 +0000 UTC m=+6.143189104 container cleanup c73f3e8276b63995c4ddb70752dcd636dc841609ba1adc15392f26e8a6a8831b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T01:04:26.721 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:26 smithi038 bash[158114]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2 2024-03-24T01:04:27.110 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:26 smithi038 podman[158146]: 2024-03-24 01:04:26.947166127 +0000 UTC m=+0.811294630 container remove c73f3e8276b63995c4ddb70752dcd636dc841609ba1adc15392f26e8a6a8831b (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T01:04:28.081 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:27 smithi038 podman[158229]: 2024-03-24 01:04:27.678371321 +0000 UTC m=+0.044472646 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T01:04:28.081 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:27 smithi038 podman[158229]: 2024-03-24 01:04:27.789656385 +0000 UTC m=+0.155757722 container create ff8b488dc0a51ed1c6c5e6cd30dce19a675efe112ef6e03a32fc127e65362a5d (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-24T01:04:28.081 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:28 smithi038 podman[158229]: 2024-03-24 01:04:28.080068377 +0000 UTC m=+0.446169771 container init ff8b488dc0a51ed1c6c5e6cd30dce19a675efe112ef6e03a32fc127e65362a5d (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-03-24T01:04:28.359 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:28 smithi038 podman[158229]: 2024-03-24 01:04:28.095331484 +0000 UTC m=+0.461432819 container start ff8b488dc0a51ed1c6c5e6cd30dce19a675efe112ef6e03a32fc127e65362a5d (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-24T01:04:28.359 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:28 smithi038 podman[158229]: 2024-03-24 01:04:28.158628923 +0000 UTC m=+0.524730282 container attach ff8b488dc0a51ed1c6c5e6cd30dce19a675efe112ef6e03a32fc127e65362a5d (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-24T01:04:28.859 INFO:journalctl@ceph.osd.2.smithi038.stdout:Mar 24 01:04:28 smithi038 podman[158229]: 2024-03-24 01:04:28.619103444 +0000 UTC m=+0.985204754 container died ff8b488dc0a51ed1c6c5e6cd30dce19a675efe112ef6e03a32fc127e65362a5d (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T01:04:29.480 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.2.service' 2024-03-24T01:04:29.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T01:04:29.609 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-24T01:04:29.609 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-24T01:04:29.609 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.3 2024-03-24T01:04:29.681 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:29 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T01:04:29.205+0000 7ff9b7e00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:05.860496+0000 front 2024-03-24T01:04:05.859948+0000 (oldest deadline 2024-03-24T01:04:28.759955+0000) 2024-03-24T01:04:30.181 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:29 smithi088 systemd[1]: Stopping Ceph osd.3 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T01:04:30.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:29 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:29.763+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:30.568 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:30 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T01:04:30.182+0000 7ff9bc600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T01:04:30.568 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:30 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T01:04:30.182+0000 7ff9bc600700 -1 osd.3 87 *** Got signal Terminated *** 2024-03-24T01:04:30.568 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:30 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T01:04:30.182+0000 7ff9bc600700 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T01:04:30.569 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:30 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T01:04:30.232+0000 7ff9b7e00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:05.860496+0000 front 2024-03-24T01:04:05.859948+0000 (oldest deadline 2024-03-24T01:04:28.759955+0000) 2024-03-24T01:04:30.930 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:30 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:30.794+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:30.930 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:30 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:30.567+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:31.546 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:31 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T01:04:31.184+0000 7ff9b7e00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:05.860496+0000 front 2024-03-24T01:04:05.859948+0000 (oldest deadline 2024-03-24T01:04:28.759955+0000) 2024-03-24T01:04:31.836 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:31 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:31.545+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:32.162 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:31 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:31.835+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:32.430 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:32 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T01:04:32.162+0000 7ff9b7e00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:05.860496+0000 front 2024-03-24T01:04:05.859948+0000 (oldest deadline 2024-03-24T01:04:28.759955+0000) 2024-03-24T01:04:32.930 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:32 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:32.565+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:32.930 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:32 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:32.796+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:33.557 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:33 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T01:04:33.202+0000 7ff9b7e00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:05.860496+0000 front 2024-03-24T01:04:05.859948+0000 (oldest deadline 2024-03-24T01:04:28.759955+0000) 2024-03-24T01:04:33.818 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:33 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:33.555+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:34.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:33 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:33.818+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:34.507 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:34 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3[96918]: 2024-03-24T01:04:34.216+0000 7ff9b7e00700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:05.860496+0000 front 2024-03-24T01:04:05.859948+0000 (oldest deadline 2024-03-24T01:04:28.759955+0000) 2024-03-24T01:04:34.831 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:34 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:34.507+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:35.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:34 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:34.831+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:35.872 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:35 smithi088 podman[123753]: 2024-03-24 01:04:35.64211987 +0000 UTC m=+5.594432342 container died b2afaff761d884fdffa4d28a130599ae9b373a2a32709ade53235c11a58648c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-24T01:04:35.873 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:35 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:35.464+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:36.176 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:35 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:35.871+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:36.430 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:36 smithi088 podman[123753]: 2024-03-24 01:04:36.176015132 +0000 UTC m=+6.128327648 container cleanup b2afaff761d884fdffa4d28a130599ae9b373a2a32709ade53235c11a58648c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T01:04:36.430 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:36 smithi088 bash[123753]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3 2024-03-24T01:04:36.887 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:36 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:36.446+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:36.887 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:36 smithi088 podman[123773]: 2024-03-24 01:04:36.436049134 +0000 UTC m=+0.780831332 container remove b2afaff761d884fdffa4d28a130599ae9b373a2a32709ade53235c11a58648c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-24T01:04:37.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:36 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:36.885+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:37.570 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:37 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:37.445+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:37.570 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:37 smithi088 podman[123864]: 2024-03-24 01:04:37.135659558 +0000 UTC m=+0.034069293 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T01:04:37.570 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:37 smithi088 podman[123864]: 2024-03-24 01:04:37.270307613 +0000 UTC m=+0.168717352 container create f564e2ad67059a9d1f688bb4fbb5d6fcbcd5f2b1b5a0212cdff2c0fe5db36958 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319) 2024-03-24T01:04:37.570 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:37 smithi088 podman[123864]: 2024-03-24 01:04:37.569834676 +0000 UTC m=+0.468244422 container init f564e2ad67059a9d1f688bb4fbb5d6fcbcd5f2b1b5a0212cdff2c0fe5db36958 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-24T01:04:37.853 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:37 smithi088 podman[123864]: 2024-03-24 01:04:37.582197207 +0000 UTC m=+0.480606916 container start f564e2ad67059a9d1f688bb4fbb5d6fcbcd5f2b1b5a0212cdff2c0fe5db36958 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T01:04:37.853 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:37 smithi088 podman[123864]: 2024-03-24 01:04:37.672586079 +0000 UTC m=+0.570995865 container attach f564e2ad67059a9d1f688bb4fbb5d6fcbcd5f2b1b5a0212cdff2c0fe5db36958 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T01:04:38.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:37 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:37.867+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:38.180 INFO:journalctl@ceph.osd.3.smithi088.stdout:Mar 24 01:04:38 smithi088 podman[123864]: 2024-03-24 01:04:38.01966043 +0000 UTC m=+0.918070202 container died f564e2ad67059a9d1f688bb4fbb5d6fcbcd5f2b1b5a0212cdff2c0fe5db36958 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-24T01:04:38.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:38 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:38.430+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:38.795 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.3.service' 2024-03-24T01:04:38.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T01:04:38.910 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-24T01:04:38.910 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-24T01:04:38.911 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.4 2024-03-24T01:04:39.033 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:38 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:38.859+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:39.033 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:38 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:38.859+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:39.403 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:39 smithi088 systemd[1]: Stopping Ceph osd.4 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T01:04:39.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:39 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:39.401+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:39.681 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:39 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:39.552+0000 7f1374e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T01:04:39.681 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:39 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:39.552+0000 7f1374e00700 -1 osd.4 87 *** Got signal Terminated *** 2024-03-24T01:04:39.681 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:39 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:39.552+0000 7f1374e00700 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T01:04:40.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:39 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:39.866+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:40.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:39 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:39.866+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:40.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:40 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:40.365+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:41.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:40 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:40.863+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:41.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:40 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:40.863+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:41.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:41 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:41.361+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:41.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:41 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:41.361+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:15.905406+0000 front 2024-03-24T01:04:15.905494+0000 (oldest deadline 2024-03-24T01:04:40.605366+0000) 2024-03-24T01:04:42.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:41 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:41.857+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:42.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:41 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:41.857+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:42.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:42 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:42.375+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:42.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:42 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:42.375+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:15.905406+0000 front 2024-03-24T01:04:15.905494+0000 (oldest deadline 2024-03-24T01:04:40.605366+0000) 2024-03-24T01:04:43.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:42 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:42.876+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:43.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:42 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:42.876+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:43.545 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:43 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:43.417+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:43.546 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:43 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:43.417+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:15.905406+0000 front 2024-03-24T01:04:15.905494+0000 (oldest deadline 2024-03-24T01:04:40.605366+0000) 2024-03-24T01:04:44.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:43 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:43.877+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:44.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:43 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:43.877+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:44.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:44 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:44.402+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:06.504001+0000 front 2024-03-24T01:04:06.504453+0000 (oldest deadline 2024-03-24T01:04:30.003793+0000) 2024-03-24T01:04:44.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:44 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4[101829]: 2024-03-24T01:04:44.402+0000 7f1370600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:15.905406+0000 front 2024-03-24T01:04:15.905494+0000 (oldest deadline 2024-03-24T01:04:40.605366+0000) 2024-03-24T01:04:45.109 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:44 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:44.845+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:45.109 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:44 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:44.845+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:45.430 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:45 smithi088 podman[123968]: 2024-03-24 01:04:45.108822672 +0000 UTC m=+5.703135636 container died 9a6f202043cd0328c1ae77b5f33271cda1e20b3e04f04a22555b0f59d9e99c3a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-24T01:04:45.855 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:45 smithi088 podman[123968]: 2024-03-24 01:04:45.612104754 +0000 UTC m=+6.206417755 container cleanup 9a6f202043cd0328c1ae77b5f33271cda1e20b3e04f04a22555b0f59d9e99c3a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T01:04:45.855 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:45 smithi088 bash[123968]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4 2024-03-24T01:04:46.180 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:45 smithi088 podman[123999]: 2024-03-24 01:04:45.854768839 +0000 UTC m=+0.732364673 container remove 9a6f202043cd0328c1ae77b5f33271cda1e20b3e04f04a22555b0f59d9e99c3a (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T01:04:46.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:45 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:45.869+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:46.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:45 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:45.869+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:46.930 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:46 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:46.838+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:46.931 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:46 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:46.838+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:46.931 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:46 smithi088 podman[124085]: 2024-03-24 01:04:46.547676122 +0000 UTC m=+0.037849336 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T01:04:46.931 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:46 smithi088 podman[124085]: 2024-03-24 01:04:46.680471094 +0000 UTC m=+0.170644305 container create a4c87893319d8da58c3e5287866e8bbce9a03acc3a874d62facc80f3f452d4f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-24T01:04:47.391 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:46 smithi088 podman[124085]: 2024-03-24 01:04:46.950727601 +0000 UTC m=+0.440900913 container init a4c87893319d8da58c3e5287866e8bbce9a03acc3a874d62facc80f3f452d4f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T01:04:47.391 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:46 smithi088 podman[124085]: 2024-03-24 01:04:46.96254812 +0000 UTC m=+0.452721328 container start a4c87893319d8da58c3e5287866e8bbce9a03acc3a874d62facc80f3f452d4f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-24T01:04:47.391 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:47 smithi088 podman[124085]: 2024-03-24 01:04:47.09962475 +0000 UTC m=+0.589798028 container attach a4c87893319d8da58c3e5287866e8bbce9a03acc3a874d62facc80f3f452d4f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-24T01:04:47.680 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:47 smithi088 podman[124085]: 2024-03-24 01:04:47.391391706 +0000 UTC m=+0.881564974 container died a4c87893319d8da58c3e5287866e8bbce9a03acc3a874d62facc80f3f452d4f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-24T01:04:48.057 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:47 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:47.849+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:48.057 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:47 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:47.849+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:48.057 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:47 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:47.849+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-03-24T01:04:22.273483+0000 front 2024-03-24T01:04:22.273790+0000 (oldest deadline 2024-03-24T01:04:47.573238+0000) 2024-03-24T01:04:48.285 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.4.service' 2024-03-24T01:04:48.367 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:48 smithi088 podman[124111]: 2024-03-24 01:04:48.244900903 +0000 UTC m=+0.837559417 container remove a4c87893319d8da58c3e5287866e8bbce9a03acc3a874d62facc80f3f452d4f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T01:04:48.367 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:48 smithi088 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-24T01:04:48.368 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:48 smithi088 systemd[1]: Stopped Ceph osd.4 for 4796a0d6-e976-11ee-95cd-87774f69a715. 2024-03-24T01:04:48.368 INFO:journalctl@ceph.osd.4.smithi088.stdout:Mar 24 01:04:48 smithi088 systemd[1]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.4.service: Consumed 14.274s CPU time. 2024-03-24T01:04:48.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T01:04:48.403 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-24T01:04:48.404 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-24T01:04:48.404 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.5 2024-03-24T01:04:48.680 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:48 smithi088 systemd[1]: Stopping Ceph osd.5 for 4796a0d6-e976-11ee-95cd-87774f69a715... 2024-03-24T01:04:49.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:48 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:48.862+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:49.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:48 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:48.862+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:49.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:48 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:48.862+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-03-24T01:04:22.273483+0000 front 2024-03-24T01:04:22.273790+0000 (oldest deadline 2024-03-24T01:04:47.573238+0000) 2024-03-24T01:04:49.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:48 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:48.962+0000 7f7083a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T01:04:49.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:48 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:48.962+0000 7f7083a00700 -1 osd.5 87 *** Got signal Terminated *** 2024-03-24T01:04:49.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:48 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:48.962+0000 7f7083a00700 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T01:04:50.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:49 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:49.869+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:50.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:49 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:49.869+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:50.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:49 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:49.869+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-03-24T01:04:22.273483+0000 front 2024-03-24T01:04:22.273790+0000 (oldest deadline 2024-03-24T01:04:47.573238+0000) 2024-03-24T01:04:51.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:50 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:50.835+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:51.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:50 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:50.835+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:51.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:50 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:50.835+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-03-24T01:04:22.273483+0000 front 2024-03-24T01:04:22.273790+0000 (oldest deadline 2024-03-24T01:04:47.573238+0000) 2024-03-24T01:04:52.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:51 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:51.859+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:52.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:51 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:51.859+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:52.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:51 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:51.859+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-03-24T01:04:22.273483+0000 front 2024-03-24T01:04:22.273790+0000 (oldest deadline 2024-03-24T01:04:47.573238+0000) 2024-03-24T01:04:53.140 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:52 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:52.854+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:53.141 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:52 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:52.854+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:53.141 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:52 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:52.854+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-03-24T01:04:22.273483+0000 front 2024-03-24T01:04:22.273790+0000 (oldest deadline 2024-03-24T01:04:47.573238+0000) 2024-03-24T01:04:54.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:53 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:53.903+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-03-24T01:04:04.269289+0000 front 2024-03-24T01:04:04.269491+0000 (oldest deadline 2024-03-24T01:04:28.969125+0000) 2024-03-24T01:04:54.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:53 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:53.903+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-03-24T01:04:14.671978+0000 front 2024-03-24T01:04:14.672274+0000 (oldest deadline 2024-03-24T01:04:38.771875+0000) 2024-03-24T01:04:54.181 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:53 smithi088 ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5[106998]: 2024-03-24T01:04:53.903+0000 7f707a200700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-03-24T01:04:22.273483+0000 front 2024-03-24T01:04:22.273790+0000 (oldest deadline 2024-03-24T01:04:47.573238+0000) 2024-03-24T01:04:54.883 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:54 smithi088 podman[124190]: 2024-03-24 01:04:54.537157034 +0000 UTC m=+5.702646150 container died 866388fc07f6c498c8065a24c3c89c5144475d811ca6b80c6cf79415d9279cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True) 2024-03-24T01:04:55.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:55 smithi088 podman[124190]: 2024-03-24 01:04:55.005500085 +0000 UTC m=+6.170989233 container cleanup 866388fc07f6c498c8065a24c3c89c5144475d811ca6b80c6cf79415d9279cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5) 2024-03-24T01:04:55.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:55 smithi088 bash[124190]: ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5 2024-03-24T01:04:55.680 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:55 smithi088 podman[124211]: 2024-03-24 01:04:55.273469372 +0000 UTC m=+0.726228937 container remove 866388fc07f6c498c8065a24c3c89c5144475d811ca6b80c6cf79415d9279cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-24T01:04:56.430 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:56 smithi088 podman[124299]: 2024-03-24 01:04:56.021707549 +0000 UTC m=+0.034703746 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-24T01:04:56.431 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:56 smithi088 podman[124299]: 2024-03-24 01:04:56.157593863 +0000 UTC m=+0.170590055 container create 3e5e8729ba5a5ac707ab8beac030a0817ca003c1ea632e022d632361947e6323 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-24T01:04:56.870 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:56 smithi088 podman[124299]: 2024-03-24 01:04:56.436076542 +0000 UTC m=+0.449072808 container init 3e5e8729ba5a5ac707ab8beac030a0817ca003c1ea632e022d632361947e6323 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True) 2024-03-24T01:04:56.871 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:56 smithi088 podman[124299]: 2024-03-24 01:04:56.447074702 +0000 UTC m=+0.460070908 container start 3e5e8729ba5a5ac707ab8beac030a0817ca003c1ea632e022d632361947e6323 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True) 2024-03-24T01:04:56.871 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:56 smithi088 podman[124299]: 2024-03-24 01:04:56.576768649 +0000 UTC m=+0.589764846 container attach 3e5e8729ba5a5ac707ab8beac030a0817ca003c1ea632e022d632361947e6323 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T01:04:57.180 INFO:journalctl@ceph.osd.5.smithi088.stdout:Mar 24 01:04:56 smithi088 podman[124299]: 2024-03-24 01:04:56.871004673 +0000 UTC m=+0.884000901 container died 3e5e8729ba5a5ac707ab8beac030a0817ca003c1ea632e022d632361947e6323 (image=quay.ceph.io/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-4796a0d6-e976-11ee-95cd-87774f69a715-osd-5-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-24T01:04:57.634 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4796a0d6-e976-11ee-95cd-87774f69a715@osd.5.service' 2024-03-24T01:04:57.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T01:04:57.754 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-24T01:04:57.754 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-24T01:04:58.120 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T01:05:33.170 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-24T01:05:33.457 INFO:teuthology.orchestra.run.smithi088.stdout:Deleting cluster with fsid: 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T01:05:54.712 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T01:05:54.820 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T01:05:54.924 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-24T01:05:54.926 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441/remote/smithi038/crash 2024-03-24T01:05:54.926 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/crash -- . 2024-03-24T01:05:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:tar: /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-24T01:05:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:tar: Error is not recoverable: exiting now 2024-03-24T01:05:55.035 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441/remote/smithi088/crash 2024-03-24T01:05:55.036 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/crash -- . 2024-03-24T01:05:55.129 INFO:teuthology.orchestra.run.smithi088.stderr:tar: /var/lib/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-24T01:05:55.129 INFO:teuthology.orchestra.run.smithi088.stderr:tar: Error is not recoverable: exiting now 2024-03-24T01:05:55.132 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-24T01:05:55.133 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-24T01:05:55.237 INFO:teuthology.orchestra.run.smithi038.stdout:2024-03-24T00:56:03.306065+0000 mon.smithi038 (mon.0) 251 : cluster [WRN] Health check failed: Degraded data redundancy: 52/273 objects degraded (19.048%), 19 pgs degraded (PG_DEGRADED) 2024-03-24T01:05:55.238 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-24T01:05:55.239 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[SEC\]' /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-24T01:05:55.333 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]' /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-24T01:05:55.423 INFO:teuthology.orchestra.run.smithi038.stdout:2024-03-24T01:00:00.000198+0000 mon.smithi038 (mon.0) 618 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-24T01:05:55.424 INFO:tasks.cephadm:Compressing logs... 2024-03-24T01:05:55.424 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-24T01:05:55.429 DEBUG:teuthology.orchestra.run.smithi088:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-24T01:05:55.494 INFO:teuthology.orchestra.run.smithi038.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-24T01:05:55.494 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-24T01:05:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi038.xnpqhp.log 2024-03-24T01:05:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-24T01:05:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi038.xnpqhp.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi038.log 2024-03-24T01:05:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi038.yhkmsb.log 2024-03-24T01:05:55.501 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi038.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mgr.smithi038.huhioz.log 2024-03-24T01:05:55.502 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi038.yhkmsb.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-24T01:05:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mgr.smithi038.huhioz.log: 94.1% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi038.log.gz 2024-03-24T01:05:55.505 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.1.log: 91.4% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-24T01:05:55.506 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mon.smithi038.log 2024-03-24T01:05:55.507 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-24T01:05:55.510 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mon.smithi038.log: /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-24T01:05:55.510 INFO:teuthology.orchestra.run.smithi088.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2024-03-24T01:05:55.510 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-24T01:05:55.511 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mon.smithi088.log 2024-03-24T01:05:55.512 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi088.cbfvvp.log 2024-03-24T01:05:55.514 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mon.smithi088.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi088.log 2024-03-24T01:05:55.515 INFO:teuthology.orchestra.run.smithi088.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-24T01:05:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi088.cbfvvp.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-24T01:05:55.516 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.log 2024-03-24T01:05:55.516 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-24T01:05:55.516 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mgr.smithi088.njctea.log 2024-03-24T01:05:55.517 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-24T01:05:55.517 INFO:teuthology.orchestra.run.smithi038.stderr: 85.6% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-24T01:05:55.518 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi088.log: /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-24T01:05:55.518 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-24T01:05:55.520 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mgr.smithi088.njctea.log: 94.1% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi088.log.gz 2024-03-24T01:05:55.521 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-24T01:05:55.522 INFO:teuthology.orchestra.run.smithi088.stderr: 91.6% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-24T01:05:55.522 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-24T01:05:55.525 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.5.log: /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-24T01:05:55.529 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.log 2024-03-24T01:05:55.530 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.log: /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.2.log: /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-volume.log: 88.3% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-24T01:05:55.531 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-24T01:05:55.534 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.log: 85.7% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-24T01:05:55.537 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi088.yhksbr.log 2024-03-24T01:05:55.543 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi088.yhksbr.log: 88.3% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-24T01:05:55.569 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mgr.smithi088.njctea.log.gz 2024-03-24T01:05:55.576 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-24T01:05:55.585 INFO:teuthology.orchestra.run.smithi088.stderr: 90.9% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-24T01:05:55.625 INFO:teuthology.orchestra.run.smithi088.stderr: 93.5% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi088.yhksbr.log.gz 2024-03-24T01:05:55.891 INFO:teuthology.orchestra.run.smithi088.stderr: 92.4% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mon.smithi088.log.gz 2024-03-24T01:05:55.963 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mgr.smithi038.huhioz.log.gz 2024-03-24T01:05:56.929 INFO:teuthology.orchestra.run.smithi038.stderr: 90.4% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mon.smithi038.log.gz 2024-03-24T01:06:00.441 INFO:teuthology.orchestra.run.smithi088.stderr: 93.5% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-24T01:06:01.162 INFO:teuthology.orchestra.run.smithi088.stderr: 94.8% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi088.cbfvvp.log.gz 2024-03-24T01:06:02.115 INFO:teuthology.orchestra.run.smithi038.stderr: 93.5% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-24T01:06:02.437 INFO:teuthology.orchestra.run.smithi038.stderr: 93.6% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-24T01:06:02.498 INFO:teuthology.orchestra.run.smithi088.stderr: 93.6% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-24T01:06:02.804 INFO:teuthology.orchestra.run.smithi088.stderr: 93.6% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-24T01:06:02.808 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-03-24T01:06:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:real 0m7.337s 2024-03-24T01:06:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:user 0m23.379s 2024-03-24T01:06:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:sys 0m2.035s 2024-03-24T01:06:03.090 INFO:teuthology.orchestra.run.smithi038.stderr: 93.7% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-24T01:06:04.559 INFO:teuthology.orchestra.run.smithi038.stderr: 94.9% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi038.xnpqhp.log.gz 2024-03-24T01:07:39.544 INFO:teuthology.orchestra.run.smithi038.stderr: 92.5% -- replaced with /var/log/ceph/4796a0d6-e976-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi038.yhkmsb.log.gz 2024-03-24T01:07:39.549 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-03-24T01:07:39.549 INFO:teuthology.orchestra.run.smithi038.stderr:real 1m44.088s 2024-03-24T01:07:39.549 INFO:teuthology.orchestra.run.smithi038.stderr:user 2m7.099s 2024-03-24T01:07:39.549 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m9.082s 2024-03-24T01:07:39.550 INFO:tasks.cephadm:Archiving logs... 2024-03-24T01:07:39.551 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441/remote/smithi038/log 2024-03-24T01:07:39.552 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-24T01:07:58.252 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441/remote/smithi088/log 2024-03-24T01:07:58.254 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-24T01:08:01.801 INFO:tasks.cephadm:Removing cluster... 2024-03-24T01:08:01.801 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 --force 2024-03-24T01:08:02.097 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T01:08:03.246 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4796a0d6-e976-11ee-95cd-87774f69a715 --force 2024-03-24T01:08:03.525 INFO:teuthology.orchestra.run.smithi088.stdout:Deleting cluster with fsid: 4796a0d6-e976-11ee-95cd-87774f69a715 2024-03-24T01:08:04.201 INFO:tasks.cephadm:Removing cephadm ... 2024-03-24T01:08:04.201 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-24T01:08:04.262 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-24T01:08:04.319 INFO:tasks.cephadm:Teardown complete 2024-03-24T01:08:04.319 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-24T01:08:04.331 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-24T01:08:04.331 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-24T01:08:04.339 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-24T01:08:04.490 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-24T01:08:04.501 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-24T01:08:04.502 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T01:08:04.510 DEBUG:teuthology.orchestra.run.smithi088:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T01:08:04.561 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-03-24T01:08:04.564 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-03-24T01:08:04.571 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T01:08:04.571 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-03-24T01:08:04.572 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 +186us[ +186us] +/- 83ms 2024-03-24T01:08:04.572 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 -1037us[-1015us] +/- 65ms 2024-03-24T01:08:04.572 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +1142us[+1164us] +/- 84ms 2024-03-24T01:08:04.572 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T01:08:04.573 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T01:08:04.573 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-03-24T01:08:04.573 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 25 +192us[ +201us] +/- 83ms 2024-03-24T01:08:04.574 INFO:teuthology.orchestra.run.smithi088.stdout:^* hv02.front.sepia.ceph.com 3 8 377 24 -1024us[-1016us] +/- 65ms 2024-03-24T01:08:04.574 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 159 +1169us[+1174us] +/- 82ms 2024-03-24T01:08:04.574 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T01:08:04.575 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-24T01:08:04.586 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-24T01:08:04.587 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-24T01:08:04.598 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-24T01:08:04.861 DEBUG:teuthology.orchestra.run.smithi088:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-24T01:08:05.094 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T01:08:05.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T01:08:05.200 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T01:08:05.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T01:08:05.305 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-24T01:08:05.317 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-24T01:08:05.329 INFO:teuthology.task.internal:Duration was 2535.785660 seconds 2024-03-24T01:08:05.329 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-24T01:08:05.340 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-24T01:08:05.340 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-24T01:08:05.348 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-24T01:08:05.515 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T01:08:05.520 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T01:08:05.957 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-24T01:08:05.957 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-03-24T01:08:05.958 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-24T01:08:06.028 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi088.front.sepia.ceph.com 2024-03-24T01:08:06.028 DEBUG:teuthology.orchestra.run.smithi088:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-24T01:08:06.119 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-24T01:08:06.120 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-24T01:08:06.126 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-24T01:08:06.218 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-24T01:08:06.218 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-24T01:08:06.224 DEBUG:teuthology.orchestra.run.smithi088:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-24T01:08:07.042 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-24T01:08:07.067 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-24T01:08:07.067 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-24T01:08:07.167 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-24T01:08:07.267 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-24T01:08:07.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-24T01:08:07.328 DEBUG:teuthology.orchestra.run.smithi088:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-24T01:08:07.435 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-03-24T01:08:07.438 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = core 2024-03-24T01:08:07.493 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-24T01:08:07.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T01:08:07.543 DEBUG:teuthology.orchestra.run.smithi088:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-24T01:08:07.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T01:08:07.598 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-24T01:08:07.611 INFO:teuthology.task.internal:Transferring archived files... 2024-03-24T01:08:07.612 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441/remote/smithi038 2024-03-24T01:08:07.612 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-24T01:08:07.743 DEBUG:teuthology.misc:Transferring archived files from smithi088:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441/remote/smithi088 2024-03-24T01:08:07.743 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-24T01:08:07.854 INFO:teuthology.task.internal:Removing archive directory... 2024-03-24T01:08:07.854 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-24T01:08:07.862 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-24T01:08:07.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-24T01:08:07.964 INFO:teuthology.task.internal:Not uploading archives. 2024-03-24T01:08:07.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-24T01:08:07.976 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-24T01:08:07.977 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-24T01:08:07.986 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-24T01:08:08.037 INFO:teuthology.orchestra.run.smithi038.stdout: 393611 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 24 01:08 /home/ubuntu/cephtest 2024-03-24T01:08:08.041 INFO:teuthology.orchestra.run.smithi088.stdout: 393936 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 24 01:08 /home/ubuntu/cephtest 2024-03-24T01:08:08.045 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-24T01:08:08.057 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-24T01:08:08.083 INFO:teuthology.nuke:Checking targets against current locks 2024-03-24T01:08:08.110 DEBUG:teuthology.nuke:shortname: smithi038 2024-03-24T01:08:08.110 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T01:08:08.129 DEBUG:teuthology.nuke:shortname: smithi088 2024-03-24T01:08:08.130 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T01:08:08.133 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 00:16:24.042579', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKv0iFJvmiR8q1udVuYrwMqOSoSQVX/zb3R8/t+aBZmvy1eOCZvaK7DJSjp9kChJbNz3DqATqEdH22swFM9tUsU='} 2024-03-24T01:08:08.155 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 00:16:24.044596', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsoD5ZKKVF75fb76X9tSs6uWMDeeNyemcJAdz71ILPjqQ3dkkblKXfdVUpJ2zT9jMCEkIqi8n9Xi4e9soCwnpo='} 2024-03-24T01:08:08.183 INFO:teuthology.orchestra.console.smithi038:Power off 2024-03-24T01:08:08.183 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-24T01:08:08.212 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off 2024-03-24T01:08:08.212 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T01:08:08.228 INFO:teuthology.orchestra.console.smithi088:Power off 2024-03-24T01:08:08.228 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-24T01:08:08.237 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-03-24T01:08:08.251 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off 2024-03-24T01:08:08.251 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T01:08:08.277 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-03-24T01:08:12.238 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T01:08:12.278 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T01:08:12.363 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-03-24T01:08:12.388 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-03-24T01:08:16.369 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T01:08:16.389 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T01:08:16.480 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off 2024-03-24T01:08:16.499 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-03-24T01:08:16.581 INFO:teuthology.orchestra.console.smithi038:Power off completed 2024-03-24T01:08:20.503 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T01:08:20.612 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-03-24T01:08:20.713 INFO:teuthology.orchestra.console.smithi088:Power off completed 2024-03-24T01:08:20.818 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2535.785659790039 failure_reason: '"2024-03-24T01:00:00.000198+0000 mon.smithi038 (mon.0) 618 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline" in cluster log' flavor: default host.a-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 host.b-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 owner: mchangir success: false 2024-03-24T01:08:20.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-24T01:08:20.902 INFO:teuthology.run:FAIL